Skip to content

Demonstrates how to develop an Oozie workflow application and aim's to show-case Oozie's features.

Notifications You must be signed in to change notification settings

stlava/oozie-examples

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Oozie workflow examples

Demonstrates how to develop an Oozie workflow application and aim's to show-case some of Oozie's features.

Build

Maven is used to build the application bundle and it is assumed Maven is installed and on your path.

To build the application simply run:

mvn package

The Maven assembly plugin is used to generate a .tar.gz file which contains all of the workflow and configuration files in the required layout:

oozie-examples-[VERSION]-bundle.tar.gz

/workflow.xml
/config-default.xml
|
/conf/ 	(job config)
/lib/ 	(*.jar;*.so)

Deploy

rm -rf examples-oozie
tar -xvzf oozie-examples-0.0.1-SNAPSHOT-bundle.tar.gz

hadoop fs -rmr /workflows/oozie-examples
hadoop fs -put oozie-examples /workflows/oozie-examples

Run

export OOZIE_URL=http://ubuntu:11000/oozie
oozie job -config oozie-examples/job.properties -run

Run parallel mapreduce jobs in sub-workflow

oozie job -config oozie-examples/job.properties -D jump.to=parallel -run

Coordinator

The Oozie Coordinator system allows you to define and execute recurrent and interdependent workflow jobs (data application pipelines). A data application pipeline is a chain of coordinator/workflow jobs that can run at regular intervals, different intervals, or be triggered by some external event (data availability). For example, the output of the last 4 runs of a workflow that runs every 15 minutes become the input of another workflow that runs every 60 minutes.

The coordinator job bundled with this example simply runs the workflow at 5 minute intervals between the given start and end dates. To deploy the coordinator job run the following command:

oozie job -config oozie-examples/coordinator/coord.properties -D start=$(date -u +"%FT%H:%MZ") -D end=$(date -u -d "+ 1 hour" +"%FT%H:%MZ") -D mode=single -run

To stop the coordinator job run:

oozie job -kill [coord job id]

About

Demonstrates how to develop an Oozie workflow application and aim's to show-case Oozie's features.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published