NiFi offers a large number of API, which helps developers to make changes and get information of NiFi from any other tool or custom developed applications. In this blog, we will use postman app in google chrome to explain some examples.

To add postman to your Google Chrome, go to the below mentioned URL and click add to chrome button. You will now see a new app added to your Google Chrome.

The current version of NiFi rest API is 1.8.0 and the documentation is present in the below mentioned URL.

https://nifi.apache.org/docs/nifi-docs/rest-api/index.html

Following are the most used NiFi rest API Modules −

http://<nifi url>:<nifi port>/nifi-api/<api-path>
In case HTTPS is enabled https://<nifi url>:<nifi port>/nifi-api/<api-path>

S.No.	API module Name	api-path	Description
1	Access	/access	To authenticate user and get access token from NiFi.
2	Controller	/controller	To manage the cluster and create reporting task.
3	Controller Services	/controller-services	It is used to manage controller services and update controller service references.
4	Reporting Tasks	/reporting-tasks	To manage reporting tasks.
5	Flow	/flow	To get the data flow metadata and component status and query history
6	Process Groups	/process-groups	To upload and instantiate a template and create components.
7	Processors	/processors	To create and schedule a processor and set its properties.
8	Connections	/connections	To create a connection, set queue priority and update connection destination
9	FlowFile Queues	/flowfile-queues	To view queue contents, download flowfile content, and empty queue.
10	Remote Process Groups	/remote-process-groups	To create a remote group and enable transmission.
11	Provenance	/provenance	To query provenance, and search event lineage.

Let us now consider an example and run on postman to get the details about the running NiFi instance.

Request

GET http://localhost:8080/nifi-api/flow/about

Response

{

“about”: {

“title”: “NiFi”,

“version”: “1.7.1”,

“uri”: “http://localhost:8080/nifi-api/”,

“contentViewerUrl”: “../nifi-content-viewer/”,

“timezone”: “SGT”,

“buildTag”: “nifi-1.7.1-RC1”,

“buildTimestamp”: “07/12/2018 12:54:43 SGT”

}

Apache NiFi – Data Provenance

Apache NiFi logs and store every information about the events occur on the ingested data in the flow. Data provenance repository stores this information and provides UI to search this event information. Data provenance can be accessed for full NiFi level and processor level also.

9.1

The following table lists down the different fields in the NiFi Data Provenance event list have following fields −

S.No.	Field Name	Description
1	Date/Time	Date and time of event.
2	Type	Type of Event like ‘CREATE’.
3	FlowFileUuid	UUID of the flowfile on which the event is performed.
4	Size	Size of the flowfile.
5	Component Name	Name of the component which performed the event.
6	Component Type	Type of the component.
7	Show lineage	Last column has the show lineage icon, which is used to see the flowfile lineage as shown in the below image.

9.2

To get more information about the event, a user can click on the information icon present in the first column of the NiFi Data Provenance UI.

There are some properties in nifi.properties file, which are used to manage NiFi Data Provenance repository.

S.No.	Property Name	Default Value	Description
1	nifi.provenance.repository.directory.default	./provenance_repository	To specify the default path of NiFi data provenance .
2	nifi.provenance.repository.max.storage.time	24 hours	To specify the maximum retention time of NiFi data provenance.
3	nifi.provenance.repository.max.storage.size	1 GB	To specify the maximum storage of NiFi data provenance.
4	nifi.provenance.repository.rollover.time	30 secs	To specify the rollover time of NiFi data provenance.
5	nifi.provenance.repository.rollover.size	100 MB	To specify the rollover size of NiFi data provenance.
6	nifi.provenance.repository.indexed.fields	EventType, FlowFileUUID, Filename, ProcessorID, Relationship	To specify the fields used to search and index NiFi data provenance.

So, this brings us to the end of Deep Dive into Apache NiFi User Interface blog. This Tecklearn ‘Understanding Apache NiFi APIs with request and response example’ helps you with commonly asked questions if you are looking out for a job in Apache NiFi and Big Data Domain.

If you wish to learn Apache NiFi and build a career in Apache NiFi or Big Data domain, then check out our interactive, Apace NiFi Training, that comes with 24*7 support to guide you throughout your learning period. Please find the link for course details:

https://www.tecklearn.com/course/apache-nifi-training/

Apache NiFi Training

About the Course

Tecklearn Apache NiFi Training makes you an expert in Cluster integration and the challenges associated, Usefulness of Automation, Apache NiFi configuration challenges and etc. Apache NiFi that helps you master various aspects of automating dataflow, managing flow of information between systems, streaming analytics, the concepts of data lake and constructs, various methods of data ingestion and real-world Apache NiFi projects .Transforming the databases is becoming a challenge for many organizations and thus they often look for those who have certification in Apache NiFi to help them in automating the flow of data between the systems.

Why Should you take Apache NiFi Training?

The Average Salary for Apache NiFi Developers is $96,578 per year. – paysa.com
Micron, Macquarie Telecom Group , Dovestech, Payoff, Flexilogix , Hashmap Inc. & many other MNC’s worldwide use Ansible across industries.
Apache NiFi is an open source software for automating and managing the flow of data between systems. It is a powerful and reliable system to process and distribute data. It provides a web-based User Interface for creating, monitoring, & controlling data flows.

What you will Learn in this Course?

Overview of Apache NiFi and its capabilities

Understanding the Apache NiFi
Apache NiFi most interesting features and capabilities

High Level Overview of Key Apache NiFi Features

Key features categories: Flow management, Ease of use, Security, Extensible architecture and Flexible scaling model

Advantages of Apache NiFi over other traditional ETL tools

Features of NiFi which make it different form traditional ETL tool and gives NiFi an edge over them

Apache NiFi as a Data Ingestion Tool

Introduction to Apache NiFi for data ingestion
Apache NiFi Processor : Data ingestion tools available for transferring , importing , loading and processing of data

Data Lake Concepts and Constructs (Big Data & Hadoop Environment)

Concept of data lake and its attributes
Support for colocation of data in various formats and overcoming the problem of data silos

Apache NiFi capabilities in Big Data and Hadoop Environment

Introduction to NiFi processors which sync with data lake and Hadoop ecosystem
An overview of the various components of the Hadoop ecosystem and data lake

Installation Requirements and Cluster Integration

Apache NiFi installation requirements and cluster integration
Successfully running Apache NiFi and addition of processor to NiFi
Working with attributes and Process of scaling up and down
Hands On

Apache NiFi Core Concepts

Apache NiFi fundamental concepts
Overview of FlowFile, Flow Controller ,FlowFile Processor, and their attributes
Functions in dataflow

Architecture of Apache NiFi

Architecture of Apache NiFi
Various components including FlowFile Repository, Content Repository, Provenance Repository and web-based user interface
Hands On

Performance Expectation and Characteristics of NiFi

How to utilize maximization of resources is particularly strong with respect to CPU and disk
Understand the best practices and configuration tips

Queuing and Buffering Data

Buffering of Data in Apache NiFi
Concept of queuing, recovery and latency
Working with controller services and directed graphs
Data transformation and routing
Processor connection, addition and configuration
Hands On

Database Connection with Apache NiFi

Apache NiFi Connection with database
Data Splitting, Transforming and Aggregation
Monitoring of NiFi and process of data egress
Reporting and Data lineage
Expression language and Administration of Apache NiFi
Hands On

Apache NiFi Configuration Best Practices

Apache NiFi configuration Best Practices
ZooKeeper access, properties, custom properties and encryption
Guidelines for developers
Security of Data in Hadoop and NiFi Kerberos interface
Hands On

Apache NiFi Project

Apache NiFi Installation
Configuration and Deployment of toolbar
Building a dataflow using NiFi
Creating, importing and exporting various templates to construct a dataflow
Deploying Real-time ingestion and Batch ingestion in NiFi
Hands On

Got a question for us? Please mention it in the comments section and we will get back to you.

726