VagrantでApache Nifi 1.6.0とApache Cassandraがインストールされた仮想マシン(Debian Stretch/9.4)を構築する。

Apache Nifiは様々なデータを処理・分配するためのソフトウェアです。

〇Apache Nifiの画面

〇構築方法
1.以下のVagrantfileを使用して、Apache Nifi 1.6.0とApache Cassandraがインストールされた仮想マシン(Debian Stretch/9.4)を構築します。

Vagrantfile

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "bento/debian-9.4"
  config.vm.hostname = "db94nifi160cassandra"
  config.vm.provider :virtualbox do |vbox|
     vbox.name = "db94nifi160cassandra"
     vbox.cpus = 2
     vbox.memory = 2048
     vbox.customize ["modifyvm", :id, "--nicpromisc2","allow-all"]
  end
config.vm.network "private_network", ip: "192.168.55.105", :netmask => "255.255.255.0"
config.vm.network "public_network", ip:"192.168.1.105", :netmask => "255.255.255.0"
  config.vm.provision "shell", inline: <<-SHELL
apt-get -y install task-japanese
sed -i -e 's/# ja_JP.UTF-8 UTF-8/ja_JP.UTF-8 UTF-8/' /etc/locale.gen
locale-gen
update-locale LANG=ja_JP.UTF-8
localectl set-locale LANG=ja_JP.UTF-8
localectl set-keymap jp106
apt-get update
#DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade

apt-get -y install curl
echo "deb http://www.apache.org/dist/cassandra/debian 311x main" >> /etc/apt/sources.list.d/cassandra.sources.list
curl https://www.apache.org/dist/cassandra/KEYS | apt-key add -
apt-get update
apt-get -y install cassandra

# configure authentication.
sed -i -e 's/authenticator: AllowAllAuthenticator/authenticator: PasswordAuthenticator/' /etc/cassandra/cassandra.yaml
systemctl enable cassandra.service
systemctl start cassandra.service

# CQL for test
cat << EOF > /home/vagrant/sample.cql
create keyspace mykeyspace with replication = {'class':'SimpleStrategy', 'replication_factor':1};
use mykeyspace;
create table mytable (
name text PRIMARY KEY,
value text
);
insert into mytable (name, value) values ('test1', 'cassandra');
select * from mytable;
EOF

# wait until it listens on 9042 port.
while netstat -lnt | awk '$4 ~ /:9042$/ {exit 1}'; do sleep 10; done
sleep 10

cqlsh -u cassandra -p cassandra -f /home/vagrant/sample.cql


# maximum file handles & maximum forked processes
echo '*  hard  nofile  50000' >> /etc/security/limits.conf
echo '*  soft  nofile  50000' >> /etc/security/limits.conf
echo '*  hard  nproc  10000' >> /etc/security/limits.conf
echo '*  soft  nproc  10000' >> /etc/security/limits.conf

echo '*  soft  nproc  10000' >> /etc/security/limits.d/90-nproc

# download and install Apache Nifi
wget http://ftp.riken.jp/net/apache/nifi/1.6.0/nifi-1.6.0-bin.tar.gz
tar xvfz nifi-1.6.0-bin.tar.gz
mv nifi-1.6.0 /opt

cat << EOF > /etc/systemd/system/nifi.service
[Unit]
Description=Apache Nifi
After=syslog.target network.target

[Service]
Type=forking
ExecStart=/opt/nifi-1.6.0/bin/nifi.sh start
ExecStop=/opt/nifi-1.6.0/bin/nifi.sh stop
KillMode=none

[Install]
WantedBy=multi-user.target
EOF
systemctl enable nifi.service
systemctl start nifi.service

echo 'access url -> http://192.168.1.105:8080/nifi/'

SHELL
end

2.Query Cassandra Processorで、以下のようにパラメータを設定してローカルのCassandraにアクセスします。
Cassandra Constact Points : localhost:9042
Keyspace : mykeyspace
Username : cassandra
Password : cassandra
CQL select query : select name, value from mytable

○関連情報
・Apache NiFiに関する他の記事はこちらを参照してください。

・Apache Cassandraに関する他の記事はこちらを参照してください。