NiPyApi

Nifi-Python-Api: A rich Apache NiFi Python Client SDK

Features

Three layers of Python support for working with Apache NiFi:

• High-level Demos and example scripts
• Mid-level Client SDK for typical complex tasks
• Low-level Client SDKs for the full API implementation of NiFi and selected sub-projects

Functionality Highlights:

• Detailed documentation of the full SDK at all levels
• CRUD wrappers for common task areas like Processor Groups, Processors, Templates, Registry Clients, Registry Buckets, Registry Flows, etc.
• Convenience functions for inventory tasks, such as recursively retrieving the entire canvas, or a flat list of all Process Groups
• Support for scheduling and purging flows, controller services, and connections
• Support for fetching and updating Variable Registries
• Support for import/export of Versioned Flows from NiFi-Registry
• Docker Compose configurations for testing and deployment
• A scripted deployment of an interactive environment, and a secured configuration, for testing and demonstration purposes

Please see the issue register for more information on current development.

Quick Start

There are several scripts to produce demo environments in nipyapi.demo.*

The mid-level functionality is in nipyapi.canvas / nipyapi.security / nipyapi.templates / nipyapi.versioning

You can access the entire API using the low-level SDKs in nipyapi.nifi / nipyapi.registry

The easiest way to install NiPyApi is with pip:

# in bash
pip install nipyapi

# or with docker support for demos
pip install nipyapi[demo]

You can set the config for your endpoints in the central config file:

# in python
import nipyapi
nipyapi.config.nifi_config.host = 'http://localhost:8080/nifi-api'
nipyapi.config.registry_config.host = 'http://localhost:18080/nifi-registry-api'

Then import a module and execute tasks:

nipyapi.canvas.get_root_pg_id()
>'4d5dcf9a-015e-1000-097e-e505ed0f7fd2'

You can use the Docker demos to create a secured interactive console showing many features:

from nipyapi.demo.secured_console import *
from nipyapi.demo.console import *

You can also explore the scripts to get ideas for how NiPyAPi can be used to automate your environment.

Please check out the Contribution Guide if you are interested in contributing to the feature set.

Background and Documentation

For more information on Apache NiFi, please visit https://nifi.apache.org

For Documentation on this package please visit https://nipyapi.readthedocs.io.

NiFi Version Support

Currently we are testing against NiFi versions 1.9.2 - 1.27.0, and NiFi-Registry versions 0.3.0 - 1.27.0.

We have also tested against the latest 2.0.0-M4 release candidates using the 1.x SDK and have found that basic functionality works as expected.

Apache NiFi offers no compatibility guarantees between major versions, and while many functions may work the same, you should test carefully for your specific use case.

In future we will create a specific branch of NiPyAPI for NiFi 2.x SDKs, and maintain separate NiFi 1.x and NiFi 2.x clients.

If you find a version compatibility problem please raise an issue

Python Support

Python 2.7 or 3.7-12 supported, though other versions may work.

We will shortly stop supporting Python2.

OSX M1 chips have various issues with Requests and Certificates.

Tested on AL2023, developed on OSX 14.2 - Windows testing not attempted.

Outside of the standard Python modules, we make use of lxml, DeepDiff, ruamel.yaml and xmltodict in processing, and Docker for demo/tests.