NiFi

nifi.md

█████ ██████   █████ ███████████ ███████████     █████████   ██████████   ██████████

░░███ ░░██████ ░░███ ░█░░░███░░░█░░███░░░░░███ ███░░░░░███ ░░███░░░░███ ░░███░░░░░█ ░███ ░███░███ ░███ ░ ░███ ░ ░███ ░███ ░███ ░███ ░███ ░░███ ░███ █ ░ ░███ ░███░░███░███ ░███ ░██████████ ░███████████ ░███ ░███ ░██████
░███ ░███ ░░██████ ░███ ░███░░░░░███ ░███░░░░░███ ░███ ░███ ░███░░█
░███ ░███ ░░█████ ░███ ░███ ░███ ░███ ░███ ░███ ███ ░███ ░ █ █████ █████ ░░█████ █████ █████ █████ █████ █████ ██████████ ██████████ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░░░░░░ ░░░░░░░░░░

Host : INT-NIFI-1 OS : Ubuntu Server 22.04 IP : 192.168.0.22

Purpose

Serves a Apache NiFi instance for integration tasks regarding DWH

Notes requires Java 8 or newer, 11 used as NiFi 2 will not support Java 8 the single-user-credential can be found in bitwarden the Xms and Xmx should be configured to the VM allocation

Colors SQL #ebeb2a
JSON #f78bdc
Text #b5acac
Scripts #97e391
Log #151d85
Messaging #d19017
Attributes #b8cfbd
HTTP #a4def5
Files #387f85

Configuration

Create service user

sudo useradd -r -s /bin/false nifi

install Java 11

sudo apt install openjdk-11-jdk

download NiFi

wget https://dlcdn.apache.org/nifi/1.21.0/nifi-1.21.0-bin.zip unzip nifi-1.21.0-bin.zip sudo mv nifi-1.21.0 /opt/

change owner

sudo chown -R nifi:nifi /opt/nifi-1.21.0

configure NiFi

sudo vim /opt/nifi-1.21.0/conf/nifi.properties nifi.web.https.host=192.168.0.22 nifi.web.https.port=8443 nifi.web.proxy.host=41.204.217.80:60443

sudo vim /opt/nifi-1.21.0/conf/bootstrap.properties run.as=nifi java.arg.2=-Xms java.arg.4=-Xmx

install service and configure single user

sudo /opt/nifi-1.21.0/bin/nifi.sh install sudo /opt/nifi-1.21.0/bin/nifi.sh set-single-user-credentials nifi password_here

add jdbc libraries

wget https://cdn.mysql.com//Downloads/Connector-J/mysql-connector-j-8.0.33.tar.gz wget https://jdbc.postgresql.org/download/postgresql-42.6.0.jar sudo mv *.jar /opt/nifi-1.21.0/lib/

start the service

sudo systemctl start nifi

NiFi 2 - SNI ERROR 400 due to Jetty

Create a new PCKS12 keystore

keytool -genkeypair
-alias nifi-cert
-keyalg RSA
-keysize 2048
-validity 365
-keystore keystore.p12
-storetype PKCS12
-dname "CN=192.168.10.70, OU=zakk-ws, O=zakk-ws, L=Centurion, S=Gauteng C=ZA"
-ext "SAN=IP:192.168.10.70,IP:10.147.17.111"

Export Cert

keytool -exportcert -alias nifi-cert -file nifi-cert.crt -keystore keystore.p12 -storetype PKCS12

Create truststore

keytool -importcert -alias nifi-cert -file nifi-cert.crt -keystore truststore.p12 -storetype PKCS12