Oozie workflows can be defined. The parameters come from a configuration file called as property file. We can run multiple jobs using same workflow by using multiple .property files (one property for each job).

Suppose we want to change the jobtracker url or change the script name or value of a param.

We can specify a config file (.property) and pass it while running the workflow.

Property File

Variables like ${nameNode} can be passed within the workflow definition. The value of this variable will be replaced at the run time with the value defined in the ‘.properties’ file.

Following is an example of a property file we will use in our workflow example.

File name — job1.properties

# proprties

nameNode = hdfs://rootname

jobTracker = xyz.com:8088

script_name_external = hdfs_path_of_script/external.hive

script_name_orc=hdfs_path_of_script/orc.hive

script_name_copy=hdfs_path_of_script/Copydata.hive

database = database_name

Now to use this property file we will have to update the workflow and pass the parameters in a workflow as shown in the following program.

<!-- This is a comment -->

<workflow-app xmlns = "uri:oozie:workflow:0.4" name = "simple-Workflow">

<start to = "Create_External_Table" />

<action name = "Create_External_Table">

<hive xmlns = "uri:oozie:hive-action:0.4">

<job-tracker>${jobTracker}</job-tracker>

<name-node>${nameNode}</name-node>

<script>${script_name_external}</script>

</hive>

<ok to = "Create_orc_Table" />

<error to = "kill_job" />

</action>

<action name = "Create_orc_Table">

<hive xmlns = "uri:oozie:hive-action:0.4">

<job-tracker>${jobTracker}</job-tracker>

<name-node>${nameNode}</name-node>

<script>${script_name_orc}</script>

</hive>

<ok to = "Insert_into_Table" />

<error to = "kill_job" />

</action>

<action name = "Insert_into_Table">

<hive xmlns = "uri:oozie:hive-action:0.4">

<job-tracker>${jobTracker}</job-tracker>

<name-node>${nameNode}</name-node>

<script>${script_name_copy}</script>

<param>${database}</param>

</hive>

<ok to = "end" />

<error to = "kill_job" />

</action>

<kill name = "kill_job">

<message>Job failed</message>

</kill>

<end name = "end" />

</workflow-app>

Now to use the property file in this workflow we will have to pass the –config while running the workflow.

oozie job –oozie http://host_name:8080/oozie

–config edgenode_path/job1.properties -D oozie.wf.application.path

hdfs://Namenodepath/pathof_workflow_xml/workflow.xml –run

Note − The property file should be on the edge node (not in HDFS), whereas the workflow and hive scripts will be in HDFS.

At run time, all the parameters in ${} will be replaced by its corresponding value in the .properties file.

Also, a single property file can have more parameters than required in a single workflow and no error will be thrown. This makes it possible to run more than one workflow by using the same properties file. But if the property file does not have a parameter required by a workflow then an error will occur.

So, this brings us to the end of blog. This Tecklearn ‘How to Configure Oozie Workflow using Property File’ helps you with commonly asked questions If you are looking out for a job in Apache oozie and Big Data Hadoop Testing.

If you wish to learn Oozie and build a career in Big Data Hadoop domain, then check out our interactive, Big Data Hadoop Testing Training, that comes with 24*7 support to guide you throughout your learning period. Please find the link for course details:

https://www.tecklearn.com/course/big-data-hadoop-testing/

Big Data Hadoop Testing Training

About the Course

Big Data analysis is emerging as a key advantage in business intelligence for many organizations. Hadoop testing training will provide you with the right skills to detect, analyse and rectify errors in Hadoop framework. You will be trained in the Hadoop software, architecture, MapReduce, HDFS and various components like Sqoop, Flume and Oozie. With this Hadoop testing training you will also be fully equipped with experience in various test case scenarios, proof of concepts implementation and real-world scenarios. It is a comprehensive Hadoop Big Data training course designed by industry experts considering current industry job requirements to help you learn Big Data Hadoop Testing.

Why Should you take Big Data Hadoop Training?

The Average Salary for BigData Hadoop Tester ranges from approximately $34.65 per hour for Senior Tester to $124,599 per year for Senior Software Engineer. – Glassdoor.com
Hadoop Market is expected to reach $99.31B by 2022 growing at a CAGR of 42.1% from 2015 – Forbes.
Amazon, Cloudera, Data Stax, DELL, EMC2, IBM, Microsoft & other MNCs worldwide use Hadoop.

What you will Learn in this Course?

Introduction to Hadoop

The Case for Apache Hadoop
Why Hadoop?
Core Hadoop Components
Fundamental Concepts

HDFS

HDFS Features
Writing and Reading Files
NameNode Memory Considerations
Overview of HDFS Security
Using the Namenode Web UI
Using the Hadoop File Shell

Getting Data into HDFS

Ingesting Data from External Sources with Flume
Ingesting Data from Relational Databases with Sqoop
REST Interfaces
Best Practices for Importing Data

Hadoop Testing

Hadoop Application Testing
Roles and Responsibilities of Hadoop Testing Professional
Framework MRUnit for Testing of MapReduce Programs
Unit Testing
Test Execution
Test Plan Strategy and Writing Test Cases for Testing Hadoop Application

Big Data Testing

BigData Testing
Unit Testing
Integration Testing
Functional Testing
Non-Functional Testing
Golden Data Set

System Testing

Building and Set up
Testing SetUp
Solary Server
Non-Functional Testing
Longevity Testing
Volumetric Testing

Security Testing

Security Testing
Non-Functional Testing
Hadoop Cluster
Security-Authorization RBA
IBM Project

Automation Testing

Query Surge Tool

Oozie

Why Oozie
Installation Engine
Oozie Workflow Engine
Oozie security
Oozie Job Process
Oozie terminology
Oozie bundle

Got a question for us? Please mention it in the comments section and we will get back to you.

1,036