In Apache NiFi 0.7.0, there are new processors to Get and Put data to an MQTT broker, which is popular in IoT because of it's small footprint and speed. MQTT is supported by Eclipse and IBM.
I created an example on the HDP 2.4.2 Sandbox. I downloaded and installed the latest Apache NiFi 0.70 there as well as an example MQTT Broker (Mosquitto) http://mosquitto.org/.
To Install Mosquitto on HDP 2.4 Sandbox (Centos 6.x)
sudo wget http://download.opensuse.org/repositories/home:/oojah:/mqtt/CentOS_CentOS-6/home:oojah:mqtt.reposudo cp *.repo /etc/yum.repos.d/ sudo yum -y update sudo yum -y install mosquitto
To Verify the Settings and Prepare Logs
[root@sandbox opt]# cat /etc/mosquitto/mosquitto.conf # Place your local configuration in /etc/mosquitto/conf.d/ pid_file /var/run/mosquitto.pid persistence true persistence_location /var/lib/mosquitto/ #log_dest file /var/log/mosquitto/mosquitto.log include_dir /etc/mosquitto/conf.d [root@sandbox opt]# vi /etc/mosquitto/mosquitto.conf [root@sandbox opt]# mkdir -p /var/log/mosquitto [root@sandbox opt]# chmod 777 /var/log/mosquitto/ [root@sandbox opt]# touch /var/log/mosquitto/mosquitto.log [root@sandbox opt]# chmod 777 /var/log/mosquitto/
Run MQTT Broker Server
The default port for MQTT and Mosquitto is 1883. Make sure that port is not blocked by Firewalls, Virus software and if one the sandbox it is exposed.
Running Mosquitto on Sandbox
[root@sandbox demo]# hdfs dfs -ls /mqtt root hdfs 2783 2016-07-20 14:56 /mqtt/37115929161818 root hdfs 2805 2016-07-20 14:56 /mqtt/37115930927495