Apache NiFi - Creating Flows


Advertisements

Apache NiFi offers a large number of components to help developers to create data flows for any type of protocols or data sources. To create a flow, a developer drags the components from menu bar to canvas and connects them by clicking and dragging the mouse from one component to other.

Generally, a NiFi has a listener component at the starting of the flow like getfile, which gets the data from source system. On the other end of there is a transmitter component like putfile and there are components in between, which process the data.

For example, let create a flow, which takes an empty file from one directory and add some text in that file and put it in another directory.

Creating Flow
  • To begin with, drag the processor icon to the NiFi canvas and select GetFile processor from the list.

  • Create an input directory like c:\inputdir.

  • Right-click on the processor and select configure and in properties tab add Input Directory (c:\inputdir) and click apply and go back to canvas.

  • Drag the processor icon to the canvas and select the ReplaceText processor from the list.

  • Right-click on the processor and select configure. In the properties tab, add some text like “Hello tutorialspoint.com” in the textbox of Replacement Value and click apply.

  • Go to settings tab, check the failure checkbox at right hand side, and then go back to the canvas.

  • Connect GetFIle processor to ReplaceText on success relationship.

  • Drag the processor icon to the canvas and select the PutFile processor from the list.

  • Create an output directory like c:\outputdir.

  • Right-click on the processor and select configure. In the properties tab, add Directory (c:\outputdir) and click apply and go back to canvas.

  • Go to settings tab and check the failure and success checkbox at right hand side and then go back to the canvas.

  • Connect the ReplaceText processor to PutFile on success relationship.

  • Now start the flow and add an empty file in input directory and you will see that, it will move to output directory and the text will be added to the file.

By following the above steps, developers can choose any processor and other NiFi component to create suitable flow for their organisation or client.

Useful Video Courses


Video

Apache Spark Online Training

46 Lectures 3.5 hours

Arnab Chakraborty

Video

Apache Spark with Scala - Hands On with Big Data

23 Lectures 1.5 hours

Mukund Kumar Mishra

Video

Learn Apache Cordova using Visual Studio 2015 & Command line

16 Lectures 1 hours

Nilay Mehta

Video

Delta Lake with Apache Spark using Scala

52 Lectures 1.5 hours

Bigdata Engineer

Video

Apache Zeppelin - Big Data Visualization Tool

14 Lectures 1 hours

Bigdata Engineer

Video

Olympic Games Analytics Project in Apache Spark for Beginner

23 Lectures 1 hours

Bigdata Engineer

Advertisements