RiakCS on a Raspberry Pi 3 with Raspbian (Debian Jessie 8.0)
The Raspberry Pi 3 single board computer from the Raspberry Pi Foundation has a 1.2 GHz quad-core ARMv8 processor, 1 GByte main memory and a 10/100 Mbps Ethernet interface.
This installation tutorial explains the installation of a Raspberry Pi 3 device from scratch and the configuration of s3cmd on your computer.
This implies that you have already an installation of s3cmd on the system you want to use for the interaction with Riak CS.
$ wget https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2017-03-03/2017-03-02-raspbian-jessie-lite.zip
$ unzip 2017-03-02-raspbian-jessie-lite.zip
Check which one is the correct device! If you use an internal card reader, it is often /dev/mmcblk0.
# dd bs=1M if=2017-03-02-raspbian-jessie-lite.img of=/dev/sdb && sync
Default login of this image is pi/raspberry. To become user root, execute sudo su.
Older Raspian versions started the SSH server by default. Because of security reasons (as explained here), all Raspian versions since version 2016-11-25 (see the release notes) have the SSH server disabled by default. To get the SSH server automativally activated during boot time, create a file ssh with any content (or just an empty file) inside the boot partition (it is the first partition) of the micro SD card.
# mount /dev/sdb1 /media/
# touch /media/ssh
# umount /media/
Insert the micro SD card into the Raspberry Pi computer, connect it with the ethernet cable and the micro USB cable for power suppy and switch on the power supply. The operating system will try to fetch network configuration by using DHCP on the Ethernet interface per default. If you activated the SSH server, you can now log in via SSH.
The system used with this tutorial had these characteristics
$ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description: Raspbian GNU/Linux 8.0 (jessie)
Release: 8.0
Codename: jessie
$ uname -a
Linux raspberrypi 4.4.50-v7+ #970 SMP Mon Feb 20 19:18:29 GMT 2017 armv7l GNU/Linux
$ sudo raspi-config
This is not required for running Riak CS, but the Raspberry Pi 3 has just 1 GB of main memory which is not much at all and there is no need to waste 48 MB.
A part of the main memory (just 1 GB) is assigned to the GPU. A pure server does not need a GPU at all. The share can me specified via the raspi-config tool. The minimum value is 16 MB, which is more useful compared with the dafault value (64 MB).
After the new value is specified and after a reboot, the new value should be visible:
# vcgencmd get_mem gpu
gpu=16M
This is not required for running Riak CS, but it is always useful to configure the operating system properly
$ sudo dpkg-reconfigure tzdata
$ cat /etc/timezone
Europe/Berlin
It is also possible to specify the time zone via the raspi-config tool.
This is not required for running Riak CS, but it is always useful to have the correct time on a computer
$ sudo apt-get update && sudo apt-get install -y ntp ntpdate
Now the time sould be synchronized with several NTP servers.
$ ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*time3.hs-augsbu 129.69.1.153 2 u - 128 377 31.891 7.194 1.053
-hotel.zq1.de 235.106.237.243 3 u 113 128 377 38.442 0.272 0.896
+bandersnatch.ro 213.239.154.12 3 u 75 128 377 28.497 6.777 3.761
-ntp.janetzki.eu 46.4.77.168 3 u 90 128 377 35.747 -0.316 27.235
+fritz.box 194.25.134.196 3 u 62 128 377 0.368 4.667 0.674
Check the time and date:
# date -R
Sat, 01 Apr 2017 14:57:33 +0200
Some of them are not required for the installation of Riak CS, but just nice to have.
$ sudo apt-get install -y curl htop joe nmap git
$ sudo apt-get install -y build-essential autoconf libncurses5-dev openssl \
libssl-dev fop xsltproc unixodbc-dev libpam0g-dev openjdk-7-jdk oracle-java8-jdk maven
$ sudo apt-get install -y libxml2-utils libwxbase2.8 libwxgtk2.8-dev libqt4-opengl-dev libpam0g-dev
$ cd ~
$ wget http://s3.amazonaws.com/downloads.basho.com/erlang/otp_src_R16B02-basho10.tar.gz
$ tar xvzf otp_src_R16B02-basho10.tar.gz
$ cd OTP_R16B02_basho10/
./otp_build autoconf
$ ./configure
$ make
$ sudo make install
Test if Erlang has been installed.
$ erl -eval "erlang:display(erlang:system_info(otp_release)), erlang:display(erlang:system_info(system_architecture)), halt()." -noshell
"R16B02_basho10"
"armv7l-unknown-linux-gnueabihf"
$ cd ~/src
$ git clone https://github.com/basho/riak.git
$ cd riak
$ make locked-deps
$ cd deps/eleveldb/c_src/leveldb
$ git checkout develop
Previous HEAD position was e6a2939... add (int) cast to fix compile errors on 10 of 16 platforms ... argh.
Switched to branch 'develop'
Your branch is up-to-date with 'origin/develop'.
$ cd ..
$ sed -i "221s/71/7/g" eleveldb.cc
$ cd ../../../
To avoid this problem during compilation...
Compiling /home/pi/src/riak/deps/eleveldb/c_src/eleveldb.cc
/home/pi/src/riak/deps/eleveldb/c_src/eleveldb.cc:46:34: fatal error: leveldb_os/expiry_os.h: No such file or directory
#include "leveldb_os/expiry_os.h"
^
compilation terminated.
ERROR: compile failed while processing /home/pi/src/riak/deps/eleveldb: rebar_abort
Makefile:23: recipe for target 'compile' failed
make: *** [compile] Error 1
the file ~/riak/deps/eleveldb/c_src/eleveldb.cc needs to be modified. Replace this line...
#include "leveldb_os/expiry_os.h"
with this line:
#include "util/expiry_os.h"
Compile Riak KV.
$ cd ~/riak/
$ make rel
Insert this line into the file riak.conf.
$ echo "leveldb.limited_developer_mem = on" >> ~/riak/rel/riak/etc/riak.conf
Now, some entries inside the file ~/riak/rel/riak/etc/riak.conf need to be modified. If it is a multi node installation, these values need to be modified.
nodename = [email protected]
listener.http.internal = 127.0.0.1:8098
listener.protobuf.internal = 127.0.0.1:8087
In case of a single node deployment, it is not necessary.
Start Riak KV.
$ ~/riak/rel/riak/bin/riak start
$ nmap localhost
...
PORT STATE SERVICE
22/tcp open ssh
8087/tcp open simplifymedia
8099/tcp open unknown
$ cd ~
$ wget http://downloads.basho.com.s3-website-us-east-1.amazonaws.com/stanchion/2.1/2.1.1/stanchion-2.1.1.tar.gz
$ tar zxvf stanchion-2.1.1.tar.gz
$ cd stanchion-2.1.1
$ make rel
If it is a multi node installation, these values in the file ~/stanchion-2.1.1/rel/stanchion/etc/stanchion.conf need to be modified.
listener = 127.0.0.1:8085
riak_host = 127.0.0.1:8087
In case of a single node deployment, it is not necessary.
$ cd ~
$ wget http://downloads.basho.com.s3-website-us-east-1.amazonaws.com/riak-cs/2.1/2.1.1/riak-cs-2.1.1.tar.gz
$ tar zxvf riak-cs-2.1.1.tar.gz
$ cd riak-cs-2.1.1
$ make rel
Modify inside the file ~/riak-cs-2.1.1/rel/riak-cs/etc/riak-cs.conf this line
anonymous_user_creation = off
into this line
anonymous_user_creation = on
and check if these values are correct:
riak_host = 127.0.0.1:8087
stanchion_host = 127.0.0.1:8085
nodename = [email protected]
The file ~/riak/rel/riak/etc/advanced.config requires this content:
[
{riak_kv, [
{add_paths, ["/home/pi/riak-cs-2.1.1/ebin/"]},
{storage_backend, riak_cs_kv_multi_backend},
{multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]},
{multi_backend_default, be_default},
{multi_backend, [
{be_default, riak_kv_eleveldb_backend, [
{total_leveldb_mem_percent, 30},
{data_root, "/home/pi/riak/rel/riak/data/leveldb"}
]},
{be_blocks, riak_kv_bitcask_backend, [
{data_root, "/home/pi/riak/rel/riak/data/bitcask"}
]}
]}
]}
].
Insert a line into this file riak.conf.
$ echo "buckets.default.allow_mult = true" >> ~/riak/rel/riak/etc/riak.conf
Start Riak KV.
$ ~/riak/rel/riak/bin/riak start
$ nmap localhost
...
PORT STATE SERVICE
22/tcp open ssh
8087/tcp open simplifymedia
8099/tcp open unknown
Start Stanchion.
$ ~/stanchion-2.1.1/rel/stanchion/bin/stanchion start
$ nmap localhost
...
PORT STATE SERVICE
22/tcp open ssh
8085/tcp open unknown
8087/tcp open simplifymedia
8099/tcp open unknown
Start Riak CS
$ ~/riak-cs-2.1.1/rel/riak-cs/bin/riak-cs start
$ nmap localhost
...
PORT STATE SERVICE
22/tcp open ssh
8080/tcp open http-proxy
8085/tcp open unknown
8087/tcp open simplifymedia
8099/tcp open unknown
$ free -h
total used free shared buffers cached
Mem: 973M 930M 42M 4.3M 75M 669M
-/+ buffers/cache: 186M 787M
Swap: 99M 2.2M 97M
Create an admin user (this is only possible if anonymous_user_creation was set to value on inside the file ~/riak-cs-2.1.1/rel/riak-cs/etc/riak-cs.conf before).
$ curl -H 'Content-Type: application/json' \
-XPOST http://localhost:8080/riak-cs/user \
--data '{"email":"[email protected]", "name":"admin"}'
{
"email":"[email protected]",
"display_name":"admin",
"name":"admin",
"key_id":"4DFDODR7LO49H-P5TXL0",
"key_secret":"ZXxWESPfXz4sQ0VKAXS5wONu4sIt86GZIX1bwA==",
"id":"285c5eadfa9157278a415d1c4c0a0391cfe9956bccc5c5d5e2073ab713e98984",
"status":"enabled"
}
To access the Riak CS port from another node, modify the value of listener inside the file /home/pi/riak-cs-2.1.1/rel/riak-cs/etc/riak-cs.conf
listener = 10.0.0.253:8080
Restart Riak CS
$ ~/riak-cs-2.1.1/rel/riak-cs/bin/riak-cs restart
ok
Just these lines need to be modified:
access_key = 4DFDODR7LO49H-P5TXL0
host_base = <the_ip_of_your_raspberry>:8080
host_bucket = <the_ip_of_your_raspberry>:8080
secret_key = ZXxWESPfXz4sQ0VKAXS5wONu4sIt86GZIX1bwA==
use_https = False
Now s3cmd should work properly with the Riak CS service, if the port of Riak CS is accessible.
$ nmap 10.0.0.253
...
PORT STATE SERVICE
22/tcp open ssh
8080/tcp open http-proxy
8099/tcp open unknown
$ s3cmd mb s3://testbucket
Bucket 's3://testbucket/' created
$ s3cmd ls
2017-05-06 06:55 s3://testbucket
$ s3cmd put /etc/hostname s3://testbucket
upload: '/etc/hostname' -> 's3://testbucket/hostname' [1 of 1]
6 of 6 100% in 0s 10.72 B/s done
$ s3cmd ls s3://testbucket
2017-05-06 06:55 6 s3://testbucket/hostname
$ s3cmd get s3://testbucket/hostname /tmp/
download: 's3://testbucket/hostname' -> '/tmp/hostname' [1 of 1]
6 of 6 100% in 0s 149.99 B/s done
$ s3cmd del s3://testbucket/hostname
delete: 's3://testbucket/hostname'
$ s3cmd rb s3://testbucket
Bucket 's3://testbucket/' removed
Many hints of this tutorial have been taken from the work of Henry Cocos and from here and here.
If Riak KV has been started, it is possible to check if its running via the command riak ping.
$ ~/riak/rel/riak/bin/riak ping
pong
If Riak KV is running, it is possible to check if its able to read and write data via the command riak-admin test.
$ ~/riak/rel/riak/bin/riak-admin test
Successfully completed 1 read/write cycle to '[email protected]'