VagrantでJupyter LabとApache Igniteをインストールした仮想マシン(Ubuntu18.04)を構築する

Jupyter Labでインタラクティブなコンピューティング環境を提供する事ができます。

○Jupyter Labの画面

構築方法

以下のVagrantfileで、Jupyter LabとApache Igniteをインストールした仮想マシン(Ubuntu18.04)を構築する事ができます。
Jupyter LabからApache IgniteにODBCプロトコルで接続します。

Vagrantfile

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "bento/ubuntu-18.04"
  config.vm.hostname = "ub1804jupyterlabignite"
  config.vm.provider :virtualbox do |vbox|
     vbox.name = "ub1804jupyterlabignite"
     vbox.cpus = 2
     vbox.memory = 2048
     vbox.customize ["modifyvm", :id, "--nicpromisc2","allow-all"]
  end
config.vm.network "private_network", ip: "192.168.55.101", :netmask => "255.255.255.0"
config.vm.network "public_network", ip:"192.168.1.101", :netmask => "255.255.255.0"
  config.vm.provision "shell", inline: <<-SHELL
# update packages
apt-get update
#DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade
locale-gen ja_JP.UTF-8
localectl set-locale LANG=ja_JP.UTF-8


# install java
apt-get -y install openjdk-8-jdk

apt-get -y install unzip
wget http://ftp.meisei-u.ac.jp/mirror/apache/dist//ignite/2.4.0/apache-ignite-fabric-2.4.0-bin.zip
unzip apache-ignite-fabric-2.4.0-bin.zip
mv apache-ignite-fabric-2.4.0-bin /opt/ignite

cat << EOF > /etc/systemd/system/ignite.service
[Unit]
Description=Apache Ignite

[Service]
ExecStart=/opt/ignite/bin/ignite.sh
WorkingDirectory=/opt/ignite
KillMode=process
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF
systemctl enable ignite.service
systemctl start ignite.service

# wait until it starts to listen on 11211 port.
while netstat -lnt | awk '$4 ~ /:11211$/ {exit 1}'; do sleep 10; done
sleep 10

# execute sample statements.
wget http://wiki.metawerx.net/attach/SJSQL/sjsql.java
wget http://wiki.metawerx.net/attach/SJSQL/sjsql.class

cat << EOF >> /home/vagrant/test.sql
create table test1 (test_id integer primary key, test_desc varchar(100));
insert into test1 (test_id, test_desc) values (1, 'hello world!');
insert into test1 (test_id, test_desc) values (2, 'テストだよ');
select * from test1;
EOF
export LC_ALL=ja_JP.UTF-8 && java -cp .:/opt/ignite/libs/ignite-core-2.4.0.jar sjsql org.apache.ignite.IgniteJdbcThinDriver jdbc:ignite:thin://127.0.0.1/ test test /home/vagrant/test.sql


# install anaconda & jupyterlab
wget https://repo.continuum.io/archive/Anaconda3-5.1.0-Linux-x86_64.sh
chmod +x Anaconda3-5.1.0-Linux-x86_64.sh
./Anaconda3-5.1.0-Linux-x86_64.sh -b -p /opt/anaconda
source /opt/anaconda/bin/activate
pip install --upgrade pip
pip install jupyterlab

# install ignite driver
pip install ignite


useradd py
mkdir -p /home/py
chown -R py:py /home/py
sudo -u py bash -c "mkdir /home/py/.jupyter"
sudo -u py bash -c "cat << EOF > /home/py/.jupyter/jupyter_notebook_config.py
conf = get_config()
conf.NotebookApp.ip = '*'
conf.NotebookApp.open_browser = False
conf.NotebookApp.port = 8080
conf.NotebookApp.token = 'jupyter'
EOF"

cat << EOF > /etc/systemd/system/jupyter.service
[Unit]
Description=Jupyter notebook
[Service]
Type=simple
EnvironmentFile=/opt/anaconda/bin/activate
ExecStart=/opt/anaconda/bin/jupyter lab
User=py
Group=py
WorkingDirectory=/home/py
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl enable jupyter
sudo systemctl start jupyter

# build odbc driver and install
apt-get -y install build-essential libtool m4 automake unixodbc unixodbc-dev
cd /opt/ignite/platforms/cpp/
libtoolize && aclocal && autoheader && automake --add-missing && autoreconf
./configure --enable-odbc --disable-node --disable-core
make
make install
odbcinst -i -d -f /opt/ignite/platforms/cpp/odbc/install/ignite-odbc-install.ini
echo '/usr/local/lib' >> /etc/ld.so.conf.d/ignite.conf
ldconfig

echo 'access -> http://192.168.55.101:8080/?token=jupyter'

SHELL
end

〇動作確認用コード

import pyodbc
conn = pyodbc.connect('DRIVER={Apache Ignite};Address=127.0.0.1:10800;CACHE=MyCache',autocommit=True)
conn.setdecoding(pyodbc.SQL_CHAR, 'utf-8')
conn.setdecoding(pyodbc.SQL_WCHAR, 'utf-8')
cur = conn.cursor()
cur.execute("select * from test1")
rows = cur.fetchall()
for row in rows:
  print("%d %s" % (row[0], row[1]))

関連情報

・Apache Igniteに関する他の記事はこちらを参照してください。