2018年6月18日月曜日

Vagrantでnextcloud13.04とPostgreSQL10がインストールされた仮想マシン(Debian Stretch/9.4)を構築する

nextcloudで、Dropboxのようにファイルを管理・共有する事ができます。

〇nextcloudの画面


〇構築方法
1.以下のVagrantfileを使用して、nextcloud13.04とPostgreSQL10がインストールされた仮想マシン(Debian Stretch/9.4)を構築できます。

Vagrantfile
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "bento/debian-9.4"
  config.vm.hostname = "db94nextcloud1304pg10"
  config.vm.provider :virtualbox do |vbox|
     vbox.name = "db94nextcloud1304pg10"
     vbox.cpus = 2
     vbox.memory = 2048
     vbox.customize ["modifyvm", :id, "--nicpromisc2","allow-all"]
  end
  config.vm.network "private_network", ip: "192.168.55.105", :netmask => "255.255.255.0"
  config.vm.network "public_network", ip:"192.168.1.105", :netmask => "255.255.255.0"
  config.vm.provision "shell", inline: <<-SHELL
apt-get -y install task-japanese
sed -i -e 's/# ja_JP.UTF-8 UTF-8/ja_JP.UTF-8 UTF-8/' /etc/locale.gen
locale-gen
update-locale LANG=ja_JP.UTF-8
localectl set-locale LANG=ja_JP.UTF-8
localectl set-keymap jp106
apt-get update
#DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade


# install postgresql
echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list
apt-get -y install wget ca-certificates
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
apt-get update
apt-get upgrade
apt-get -y install postgresql-10

echo "listen_addresses='*'" >> /etc/postgresql/10/main/postgresql.conf

#sed -i 's/host.*all.*all.*127.0.0.1/#host    all             all             127.0.0.1/g' /etc/postgresql/10/main/pg_hba.conf

echo "host    all         all         127.0.0.1/32          password" >> /etc/postgresql/10/main/pg_hba.conf
echo "host    all         all         192.168.1.0/24          password" >> /etc/postgresql/10/main/pg_hba.conf
echo "host    all         all         192.168.55.0/24          password" >> /etc/postgresql/10/main/pg_hba.conf

su - postgres << EOF
createdb -T template0 --locale=ja_JP.UTF-8 --encoding=UTF8 nextcloud
psql -c "
alter user postgres with password 'postgres';
create user nextcloud with password 'nextcloud';
grant all privileges on database nextcloud to nextcloud;
"
EOF
echo "postgres:postgres" | chpasswd
systemctl restart postgresql.service



apt-get -y install apache2 libapache2-mod-php7.0
apt-get -y install php7.0-gd php7.0-json php7.0-pgsql php7.0-curl php7.0-mbstring
apt-get -y install php7.0-intl php7.0-mcrypt php-imagick php7.0-xml php7.0-zip
apt-get -y install unzip

# install nextcloud
wget https://download.nextcloud.com/server/releases/nextcloud-13.0.4.zip
unzip nextcloud-13.0.4.zip
mv nextcloud /opt/nextcloud
chown -R www-data:www-data /opt/nextcloud
ln -s  /opt/nextcloud /var/www/html
service apache2 restart
echo 'access http://192.168.1.105/nextcloud/'
echo 'user: nextcloud  password:nextcloud  db:nextcloud, host: localhost:5432'
SHELL
end

2.ブラウザからhttp://192.168.1.105/nextcloud/にアクセスして、初期設定を行います。管理者の情報の他、以下の情報を指定します。
DB user: nextcloud, DB password: nextcloud, DB: nextcloud, DB host: localhost:5432


VagrantでJupyter LabとCockroachDBをインストールした仮想マシン(Ubuntu18.04)を構築する

Jupyter Labでインタラクティブなコンピューティング環境を提供する事ができます。

〇Jupyter Labの画面

仮想マシン構築後、ブラウザからhttp://192.168.55.104:8888/?token=jupyterにアクセスします。

〇仮想マシンの構築
以下のVagrantfileで、Jupyter LabとCockroachDBをインストールした仮想マシン(Ubuntu18.04)を構築する事ができます。

Vagrantfile
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "bento/ubuntu-18.04"
  config.vm.hostname = "ub1804jupyterlabcockroachdb"
  config.vm.provider :virtualbox do |vbox|
     vbox.name = "ub1804jupyterlabcockroachdb"
     vbox.cpus = 2
     vbox.memory = 2048
     vbox.customize ["modifyvm", :id, "--nicpromisc2","allow-all"]
  end
config.vm.network "private_network", ip: "192.168.55.104", :netmask => "255.255.255.0"
config.vm.network "public_network", ip:"192.168.1.104", :netmask => "255.255.255.0"
  config.vm.provision "shell", inline: <<-SHELL
# update packages
apt-get update
#DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade
locale-gen ja_JP.UTF-8
localectl set-locale LANG=ja_JP.UTF-8

# install cockroachdb
wget https://binaries.cockroachdb.com/cockroach-v2.0.2.linux-amd64.tgz
tar xvfz cockroach-v2.0.2.linux-amd64.tgz
mv cockroach-v2.0.2.linux-amd64 /opt/cockroach

cat << EOF > /etc/systemd/system/cockroachdb.service
[Unit]
Description=cockroachdb
Requires=network.target
After=zookeepepr.service

[Service]
Type=forking
WorkingDirectory=/opt/cockroach
ExecStart=/opt/cockroach/cockroach start --insecure --host=192.168.55.104 --background
ExecStop=/opt/cockroach/cockroach quit

[Install]
WantedBy=multi-user.target
EOF
systemctl enable cockroachdb.service
systemctl start cockroachdb.service

# execute sample sql statements
/opt/cockroach/cockroach user set maxroach --insecure --host=192.168.55.104
/opt/cockroach/cockroach sql --insecure --host=192.168.55.104 << EOF
CREATE DATABASE test;
CREATE TABLE test.messages (message_id INT PRIMARY KEY, message varchar(100));
INSERT INTO test.messages VALUES (1, 'hello world.');
select * from test.messages;
GRANT ALL ON DATABASE test TO maxroach;
GRANT ALL ON TABLE test.messages TO maxroach;
EOF

# install anaconda & jupyterlab
wget https://repo.continuum.io/archive/Anaconda3-5.1.0-Linux-x86_64.sh
chmod +x Anaconda3-5.1.0-Linux-x86_64.sh
./Anaconda3-5.1.0-Linux-x86_64.sh -b -p /opt/anaconda
source /opt/anaconda/bin/activate
pip install --upgrade pip
pip install jupyterlab

# install psycopg2
apt-get -y install libpq-dev python-dev
pip install psycopg2-binary

useradd py
mkdir -p /home/py
chown -R py:py /home/py
sudo -u py bash -c "mkdir /home/py/.jupyter"
sudo -u py bash -c "cat << EOF > /home/py/.jupyter/jupyter_notebook_config.py
conf = get_config()
conf.NotebookApp.ip = '*'
conf.NotebookApp.open_browser = False
conf.NotebookApp.port = 8888
conf.NotebookApp.token = 'jupyter'
EOF"

cat << EOF > /etc/systemd/system/jupyter.service
[Unit]
Description=Jupyter notebook
[Service]
Type=simple
EnvironmentFile=/opt/anaconda/bin/activate
ExecStart=/opt/anaconda/bin/jupyter lab
User=py
Group=py
WorkingDirectory=/home/py
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl enable jupyter
sudo systemctl start jupyter

echo 'cockroachdb -> http://192.168.55.104:8080'
echo 'jupyter -> http://192.168.55.104:8888/?token=jupyter'
SHELL
end

〇動作検証用コード
import psycopg2

con = psycopg2.connect("dbname=test host=192.168.55.104 user=maxroach port=26257")

try:
  cur= con.cursor()
  sql = "select message_id, message from messages"
  cur.execute(sql)

  for row in cur.fetchall():
    print(row)

except psycopg2.Error as er:
    print('psycopg2.Error:', er)
cur.close
con.close

VagrantでMariaDBとphpMyAdminが一緒にインストールされた仮想マシン(Debian Stretch/9.4)を構築する

phpMyAdminでwebインターフェイスからMariaDBを操作できます。

〇phpMyAdminの画面

ブラウザからhttp://192.168.1.104/phpmyadmin/にアクセスします。ユーザ名/パスワードは、root/root, test/testです。

〇構築方法
以下のVagrantfileを使用して、MariaDBとphpMyAdminをインストールした仮想マシン(Debian Stretch/9.4)を構築する事ができます。

Vagrantfile
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "bento/debian-9.4"
  config.vm.hostname = "db94phpmyadminmariadb"
  config.vm.provider :virtualbox do |vbox|
     vbox.name = "db94phpmyadminmariadb"
     vbox.cpus = 2
     vbox.memory = 2048
     vbox.customize ["modifyvm", :id, "--nicpromisc2","allow-all"]
  end
  config.vm.network "private_network", ip: "192.168.55.104", :netmask => "255.255.255.0"
  config.vm.network "public_network", ip:"192.168.1.104", :netmask => "255.255.255.0"
  config.vm.provision "shell", inline: <<-SHELL
apt-get -y install task-japanese
sed -i -e 's/# ja_JP.UTF-8 UTF-8/ja_JP.UTF-8 UTF-8/' /etc/locale.gen
locale-gen
update-locale LANG=ja_JP.UTF-8
localectl set-locale LANG=ja_JP.UTF-8
localectl set-keymap jp106
apt-get update
#DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade

# install mariadb
echo "mariadb-server-10.0 mysql-server/root_password password root" | sudo debconf-set-selections
echo "mariadb-server-10.0 mysql-server/root_password_again password root" | sudo debconf-set-selections
apt-get -y install mariadb-server
mysql -uroot -proot -e "SET PASSWORD FOR root@localhost=PASSWORD('');"
mysql -uroot -proot -e "CREATE DATABASE test DEFAULT CHARACTER SET utf8;"
mysql -uroot -proot -e "CREATE USER test@localhost IDENTIFIED BY 'test';"
mysql -uroot -proot -e "GRANT ALL PRIVILEGES ON test.* TO 'test'@'localhost';"
mysql -uroot -proot -e "FLUSH PRIVILEGES;"


echo "phpmyadmin phpmyadmin/dbconfig-install boolean true" | debconf-set-selections
echo "phpmyadmin phpmyadmin/mysql/admin-pass password root" | debconf-set-selections
echo "phpmyadmin phpmyadmin/reconfigure-webserver multiselect apache2" | debconf-set-selections
echo "phpmyadmin phpmyadmin/mysql/app-pass password root" | debconf-set-selections

apt-get install -y phpmyadmin
mysql -uroot -proot -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' identified by 'root';"
mysql -uroot -proot -e "FLUSH PRIVILEGES;"

service apache2 restart

echo 'access http://192.168.1.104/phpmyadmin/'
echo 'user: root, password: root'
SHELL
end

2018年6月17日日曜日

DockerでPostgreSQL10.4、pgAdmin4がインストールされたコンテナを構築する

pgAdmin4はPostgreSQL用のwebインターフェイスです。

〇pgAdmin4の画面

ユーザはadmin@localhost、パスワードにはadminpassを指定します。
接続情報として、DBとしてdwh、ユーザ名postgres、パスワードdwhを指定します。

〇構築方法
PostgreSQL10.4、pgAdmin4がインストールされたコンテナの構築・実行(docker-compose.ymlがあるフォルダで実行)
docker-compose up -d

docker-compose.yml
version: "2"
services:
  db:
    image: postgres:10.4-alpine
    container_name: "mezzanine-db"
    ports:
      - "5432:5432"
    volumes:
      - "db-data:/var/lib/postgresql/data"
    environment:
      POSTGRES_DB: dwh
      POSTGRES_PASSWORD: dwh
  pgadmin4:
    image: dpage/pgadmin4
    container_name: "pgadmin4"
    environment:
      PGADMIN_DEFAULT_EMAIL: admin@localhost
      PGADMIN_DEFAULT_PASSWORD: adminpass
    ports:
      - "80:80"
    volumes:
      - "pgadmin4-data:/var/lib/pgadmin"
    depends_on:
      - db
volumes:
  db-data:
    driver: local
  pgadmin4-data:
    driver: local

VagrantでBookStackとMariaDBをインストールした仮想マシン(CentOS7.4)を構築する

BookStackはブック・ページといった単位で管理できるWikiソフトウェアです。

〇BookStackの画面

仮想マシン構築後、ブラウザからhttp://192.168.1.103/にアクセスします。デフォルトユーザ/パスワードはadmin@admin.com/passwordです。

〇構築方法
以下のVagrantfileを使用してBookStackとMariaDBをインストールした仮想マシン(CentOS7.4)を構築する事ができます。

Vagrantfile
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "bento/centos-7.4"
  config.vm.hostname = "co74bookstackmariadb"
  config.vm.provider :virtualbox do |vbox|
     vbox.name = "co74bookstackmariadb"
     vbox.cpus = 2
     vbox.memory = 2048
     vbox.customize ["modifyvm", :id, "--nicpromisc2","allow-all"]
  end
config.vm.network "private_network", ip: "192.168.55.103", :netmask => "255.255.255.0"
config.vm.network "public_network", ip:"192.168.1.103", :netmask => "255.255.255.0"
  config.vm.provision "shell", inline: <<-SHELL
localectl set-locale LANG=ja_JP.UTF-8

# install mariadb
yum -y install mariadb mariadb-server
sudo systemctl enable mariadb.service
sudo systemctl start mariadb.service

# change password and create users and databases.
mysql -uroot -e "SET PASSWORD = PASSWORD('root'); FLUSH PRIVILEGES;"
mysql -uroot -proot -e "CREATE DATABASE bookstack DEFAULT CHARACTER SET utf8mb4;"
mysql -uroot -proot -e "CREATE USER bookstack@localhost IDENTIFIED BY 'bookstack';"
mysql -uroot -proot -e "GRANT ALL PRIVILEGES ON bookstack.* TO 'bookstack'@'localhost';"
mysql -uroot -proot -e "FLUSH PRIVILEGES;"

# install apache2
yum -y install epel-release
rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
yum -y install --enablerepo=remi,remi-php72 httpd php php-pdo php-mbstring php-mysql php-gd php-xml php-zip php-json php-zip php-tidy git

# install composer
curl -s https://getcomposer.org/installer | php
mv composer.phar /usr/local/bin/composer

# install bookstack
cd /opt
git clone https://github.com/BookStackApp/BookStack.git --branch release --single-branch
cd /opt/BookStack
/usr/local/bin/composer install

cp .env.example .env
#DB_HOST=localhost
sed -i -e 's/DB_DATABASE=database_database/DB_DATABASE=bookstack/' /opt/BookStack/.env
sed -i -e 's/DB_USERNAME=database_username/DB_USERNAME=bookstack/' /opt/BookStack/.env
sed -i -e 's/DB_PASSWORD=database_user_password/DB_PASSWORD=bookstack/' /opt/BookStack/.env
php artisan key:generate --force
php artisan migrate --force


chown -R apache:apache /opt/BookStack
sed -i -e 's#DocumentRoot "/var/www/html"#DocumentRoot "/opt/BookStack/public"#' /etc/httpd/conf/httpd.conf
cat << EOF >> /etc/httpd/conf/httpd.conf
<Directory /opt/BookStack/public>
  AllowOverride All
  Require all granted
</Directory>
EOF

systemctl restart httpd

echo 'access to http://192.168.1.103/'
echo 'default account : admin@admin.com/password'

SHELL
end

VagrantでApache Nifi 1.6.0とPercona Serverがインストールされた仮想マシン(Debian Stretch/9.4)を構築する

Apache Nifiは様々なデータを処理・分配するためのソフトウェアです。

〇Apache Nifiの画面


〇構築方法
1.以下のVagrantfileを使用して、Apache Nifi 1.6.0とPercona Serverがインストールされた仮想マシン(Debian Stretch/9.4)を構築します。

Vagrantfile
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "bento/debian-9.4"
  config.vm.hostname = "db94nifi160percona"
  config.vm.provider :virtualbox do |vbox|
     vbox.name = "db94nifi160percona"
     vbox.cpus = 2
     vbox.memory = 2048
     vbox.customize ["modifyvm", :id, "--nicpromisc2","allow-all"]
  end
config.vm.network "private_network", ip: "192.168.55.104", :netmask => "255.255.255.0"
config.vm.network "public_network", ip:"192.168.1.104", :netmask => "255.255.255.0"
  config.vm.provision "shell", inline: <<-SHELL
apt-get -y install task-japanese
sed -i -e 's/# ja_JP.UTF-8 UTF-8/ja_JP.UTF-8 UTF-8/' /etc/locale.gen
locale-gen
update-locale LANG=ja_JP.UTF-8
localectl set-locale LANG=ja_JP.UTF-8
localectl set-keymap jp106
apt-get update
#DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade

# install percona server
export DEBIAN_FRONTEND=noninteractive
echo "percona-server-server-5.7 percona-server-server/root_password password root" | sudo debconf-set-selections
echo "percona-server-server-5.7 percona-server-server/root_password_again password root" | sudo debconf-set-selections
wget https://repo.percona.com/apt/percona-release_0.1-4.$(lsb_release -sc)_all.deb
dpkg -i percona-release_0.1-4.$(lsb_release -sc)_all.deb
apt-get update
apt-get -y install percona-server-server-5.7
mysql -uroot -proot -e "CREATE DATABASE test DEFAULT CHARACTER SET utf8;"
mysql -uroot -proot -e "CREATE USER test@localhost IDENTIFIED BY 'test';"
mysql -uroot -proot -e "GRANT ALL PRIVILEGES ON test.* TO 'test'@'localhost';"
mysql -uroot -proot -e "FLUSH PRIVILEGES;"


# install jdbc driver for mysql
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java_8.0.11-1debian9_all.deb
dpkg -i mysql-connector-java_8.0.11-1debian9_all.deb

# maximum file handles & maximum forked processes
echo '*  hard  nofile  50000' >> /etc/security/limits.conf
echo '*  soft  nofile  50000' >> /etc/security/limits.conf
echo '*  hard  nproc  10000' >> /etc/security/limits.conf
echo '*  soft  nproc  10000' >> /etc/security/limits.conf

echo '*  soft  nproc  10000' >> /etc/security/limits.d/90-nproc

# install java
apt-get -y install openjdk-8-jdk

# download and install Apache Nifi
wget http://ftp.riken.jp/net/apache/nifi/1.6.0/nifi-1.6.0-bin.tar.gz
tar xvfz nifi-1.6.0-bin.tar.gz
mv nifi-1.6.0 /opt

cat << EOF > /etc/systemd/system/nifi.service
[Unit]
Description=Apache Nifi
After=syslog.target network.target

[Service]
Type=forking
ExecStart=/opt/nifi-1.6.0/bin/nifi.sh start
ExecStop=/opt/nifi-1.6.0/bin/nifi.sh stop
KillMode=none

[Install]
WantedBy=multi-user.target
EOF
systemctl enable nifi.service
systemctl start nifi.service

echo 'access url -> http://192.168.1.104:8080/nifi/'
SHELL
end

2.DBCPConnectionPoolで、以下のようにパラメータを設定してローカルのPercona Serverにアクセスします。
Database Connection -> URL jdbc:mysql://localhost/test
Database Driver Class Name -> com.mysql.jdbc.Driver
Database Driver Location(s) -> /usr/share/java/mysql-connector-java-8.0.11.jar
Database Userr -> test
Password -> test


2018年6月16日土曜日

Vagrantでpipenvをインストールした仮想マシン(CentOS7.4)を構築する

pipenvで、簡単にpython開発環境をセットアップする事ができます。

以下のVagrantfileを使用して、pipenvをインストールした仮想マシンを構築する事ができます。
Vagrantfile
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "bento/centos-7.4"
  config.vm.hostname = "co74pipenv"
  config.vm.provider :virtualbox do |vbox|
     vbox.name = "co74pipenv"
     vbox.cpus = 2
     vbox.memory = 2048
     vbox.customize ["modifyvm", :id, "--nicpromisc2","allow-all"]
  end
config.vm.network "private_network", ip: "192.168.55.105", :netmask => "255.255.255.0"
config.vm.network "public_network", ip:"192.168.1.105", :netmask => "255.255.255.0"
  config.vm.provision "shell", inline: <<-SHELL
localectl set-locale LANG=ja_JP.UTF-8

yum -y install epel-release
yum -y install python36
yum -y install python-pip
pip install --upgrade pip
pip install --upgrade setuptools
pip install pipenv

mkdir env1
cd env1
pipenv --python 3.6
pipenv install
pipenv run python -V

SHELL
end