• Version: 0.x

Raspberry Pi

How to use this page

Please read the instructions on this page in the order they are presented. A section may depend on a section before it and we will refrain from repeating information.

Required

  • Raspberry Pi 3 (or newer) running Raspbian GNU/Linux 9 (stretch) with network connectivity via SSH
  • Docker, installed both locally and on the Raspberry Pi
  • Access to a package which contains a distribution of the LoRaWAN Network Server of xDP
  • Familiarity with Docker and the command line

Recommended

  • For maximum durability and performance, the Raspberry Pi should be configured to boot from a USB thumb drive. A temporary SD card may be required for installation depending upon your model. See Boot from USB for more details.

Installing xDP

Overview

Installing xDP consists of the following major activities:

  1. Install streambed-cli on your workstation.
  2. Setup your environment
  3. Setup Docker Network
  4. Create Docker images by using streambed install --arch arm.
  5. Load lora-server into Docker.
  6. Save/upload/start jsaas.
  7. Package/save/upload/start fdp.
  8. Provision an initial user identity.
  9. Provision an admin user
  10. Provision Control Center secrets
  11. Configure Network

streambed-cli is a suite of command line tools that communicate with xDP’s services, including control-center and streambed-mqtt-gateway, and can produce Docker images for xDP services.

jsaas is a service that xDP uses to evaluate custom JavaScript programs.

fdp bundles all of the JVM-based xDP services into a single package for deployment to the Raspberry Pi. This consists of the following services:

  • streambed-durable-queue-server is an application that persists and streams sensor and platform data using the MQTT protocol.
  • lora-server is xDP’s application that provides a LoRaWAN Network Server, soil moisture/temperature services (including a web app for observing sensors), sensor registration services (including a web app for provisioning) and an MQTT gateway for integration with Kinetic DCM.
  • control-center is a web application that serves as the primary interface for xDP. With it, you can visualize sensor data, create custom transformers and dashboards, and provision provision LoRaWAN sensors. The application can be interacted with either using the streambed-cli or via a web user interface.
  • streambed-mqtt-gateway enables xDP to send sensor data to and receive sensor data from Cisco Kinetic DCM. From DCM, data can be routed anywhere a customer desires, and vice-versa.
  • fdp-soilstate-ui is an optional reference web application that visualizes soil moisture/temperature observations. It also contains a transformer service that transforms LoRaWAN packets from soil moisture/temperature sensors into a domain representation.

1. streambed-cli

Note that if you’ve been through the getting started experience then you can skip this section as you’ve already installed the command line tools.

Please select the type of installation and follow the instructions.

Debian/Ubuntu
# Setup Repository
wget -qO - https://repositories.streambed.io/public.key | \
  sudo apt-key add - && \
  echo "deb [arch=amd64] https://repositories.streambed.io/apt/streambed stable main" | \
  sudo tee /etc/apt/sources.list.d/streambed.list && \
  sudo apt-get install apt-transport-https -y && \
  sudo apt-get update
# Install the streambed-cli
sudo apt-get && sudo apt-get install streambed-cli
RedHat/CentOS/Fedora
# Setup Repository
sudo tee /etc/yum.repos.d/streambed.repo << EOT
[streambed]
name=streambed
baseurl=https://repositories.streambed.io/yum
gpgcheck=0
enabled=1
EOT
# Install the streambed-cli
sudo yum install streambed-cli
macOS
# Setup Repository
brew tap streambed/repository https://github.com/streambed/homebrew-repository
# Install the streambed-cli (use `brew upgrade` if installed)
brew install streambed/repository/streambed-cli
Windows
Download the latest win-<arch>.zip file and extract it to a directory on your PATH.
You can find the files here: https://repositories.streambed.io/packages/streambed-cli/

Two commands in particular are now at your disposal: streambed and lora. We’ll use these shortly.

2. Setup Environment

The following variables must be set to continue with the rest of the guide.

Certificates

An X.509 certificate and private key must be made available to xDP and JSaaS so that they can terminate TLS. They must be in PEM format, have the name caf_cert.pem and caf_key.pem, and be signed for the following:

  • localhost
  • jsaas
  • fdp

The following commands will set the variables required by the rest of the guide. Be sure to substitute the /path-to reference with the actual path on your local system.

export CAF_CERT_PATH=/path-to/caf_cert.pem
export CAF_KEY_PATH=/path-to/caf_key.pem

Raspberry Pi Env

The rest of the guide assumes that you’ve set the following environment variables. Substitute as necessary with the IP address (or hostname) and username of your device.

export RPI_USER=pi
export RPI_HOST=raspberrypi.local
export RPI_PORT=22

SSH

This guide uses SSH extensively. You may wish to consult other resources to learn how to setup an SSH key so that you only have to authenticate once.

3. Setup Docker Network

xDP uses Docker’s docker network feature to create a network so that containers can communicate with each other.

The following command creates an fdp network that the services are configured to use:

ssh -p "$RPI_PORT" "$RPI_USER@$RPI_HOST" 'docker network create fdp'

4. Create Docker images

To download the latest application images, run the following command:

streambed install --arch arm

This will ensure that the latest images have been pulled down and tagged with the appropriate tag.

5. Load lora-server

lora-server is provided as a separate download. Once acquired, you’ll need to load it into Docker. Be sure to substitute /path-to/lora-server.tar with the actual path of the file on the system.

docker load < /path-to/lora-server.tar.gz

Tag the docker image with latest for easy usage:

docker tag lora-server-iox-sss:<IMAGE-TAG> streambed-cli/arm/lora-server-iox-sss:latest

6. Install JSaaS

JSaaS (jsaas) is used to evaluate dynamically provisioned applications. By running streambed install --arch arm earlier, it was downloaded to your machine.

Packaging JSaaS for Docker

First, change your current directory into its packaging directory:

mkdir -p ~/.streambed/cli/install/arm/jsaas && cd ~/.streambed/cli/install/arm/jsaas

Then, use docker save to save the Docker image:

docker save streambed-cli/arm/jsaas:latest -o jsaas-arm-latest.tar.gz

Upload JSaaS

Now, setup a data directory and copy the certificates into it:

mkdir data
cp "$CAF_CERT_PATH" data
cp "$CAF_KEY_PATH" data

Next, we’ll create a start script that will start (and restart) JSaaS for us:

cat <<EOF > ./start
#!/usr/bin/env bash

set -e

sudo docker stop jsaas &>/dev/null || true

sudo docker rm jsaas &>/dev/null || true

sudo chown -R 9999:9999 ~/jsaas/data

sudo docker run -d --name=jsaas --net=fdp --restart=always \
  -e JSAAS_TLS_PUBLIC_CERTIFICATE_PATH=/data/caf_cert.pem \
  -e JSAAS_TLS_PRIVATE_KEY_PATH=/data/caf_key.pem \
  -e JSAAS_BIND_ADDR=0.0.0.0:9880 \
  -p 9880:9880 \
  -v "\$HOME/jsaas/data:/data" \
  -m 64M streambed-cli/arm/jsaas:latest
EOF

chmod +x start

Then, use scp to copy the local directory to the Raspberry Pi:

scp -P "$RPI_PORT" -pr "$(pwd)" "$RPI_USER@$RPI_HOST:~"

Install JSaaS

Now that we’ve uploaded everything, we’ll need to install our image. Run the following command:

ssh -p "$RPI_PORT" "$RPI_USER@$RPI_HOST" 'docker load < jsaas/jsaas-arm-latest.tar.gz'

Start JSaaS

The image is now loaded and ready to start. To start it, run the following command:

ssh -p "$RPI_PORT" "$RPI_USER@$RPI_HOST" 'jsaas/start'

7. Install xDP

xDP is installed as a single package on the Raspberry Pi that contains a number of services.

Packaging for Docker

First, change your current directory into its packaging directory:

mkdir -p ~/.streambed/cli/install/arm/fdp && cd ~/.streambed/cli/install/arm/fdp

Next, declare some environment variables pertaining to the site’s network and credentials. Be sure to substitute in the site’s id, username, and password.

export NETID=00000013
export CLOUD_SITE_ID=demo
export CLOUD_USERNAME=myclouduser
export CLOUD_PASSWORD=mycloudpassword

Note that 00000013 is as used by The Things Network. Yours may be different.

…then package xDP:

The following command bundles all services into an xDP package. If there are any other services that need to be installed, be sure to specify them with additional --image flags. In particular, note the xDP Soilstate UI section at the end of this document.

streambed package --name fdp --base-image linux-java --arch arm \
    --image streambed-cli/arm/streambed-durable-queue-server:latest \
        --property streambed.durable-queue.remote.server.tcp.tls.bind.enabled=true \
    --image streambed-cli/arm/iox-sss-stub-server:latest \
        --property streambed.http-server.tls.bind.enabled=true \
        --property streambed.durable-queue.remote.client.host=127.0.0.1 \
        --readiness-check tcp://localhost:9878 \
    --image streambed-cli/arm/control-center-iox-sss:latest \
        --property streambed.http-server.bind.host=0.0.0.0 \
        --property streambed.http-server.bind.port=9871 \
        --property streambed.http-server.tls.bind.enabled=on \
        --property streambed.http-server.tls.bind.host=0.0.0.0 \
        --property streambed.http-server.tls.bind.port=9870 \
        --property streambed.durable-queue.remote.client.host=127.0.0.1 \
        --property control-center.jsaas.scheme=https \
        --property control-center.jsaas.host=jsaas \
        --property control-center.jsaas.port=9880 \
        --property control-center.cloud-provisioner.credentials.host="https://$CLOUD_SITE_ID.titanclass.com.au:9870" \
        --property control-center.cloud-provisioner.credentials.username="$CLOUD_USERNAME" \
        --property control-center.cloud-provisioner.credentials.password="$CLOUD_PASSWORD" \
        --property control-center.replicator.enabled=on \
        --property control-center.replicator.scheme="https" \
        --property control-center.replicator.host="xdp.titanclass.com.au" \
        --property control-center.replicator.port=443 \
        --property control-center.replicator.namespace="$CLOUD_SITE_ID" \
        --property control-center.replicator.credentials.username="$CLOUD_USERNAME" \
        --property control-center.replicator.credentials.password="$CLOUD_PASSWORD" \
		--property akka.http.server.default-host-header="$CLOUD_SITE_ID" \
        --readiness-check tcp://localhost:9876 \
    --image streambed-cli/arm/lora-server-iox-sss:latest \
        --property lora-server.downlink.bind-host=0.0.0.0 \
        --property lora-server.downlink.bind-port=1692 \
        --property streambed.durable-queue.remote.client.host=127.0.0.1 \
        --property lora-server.uplink.bind-host=0.0.0.0 \
        --property lora-server.uplink.bind-port=1690 \
        --property "lora-server.netID=$NETID" \
        --readiness-check tcp://localhost:9870

The streambed package command above ensures that the image is entirely suitable for Docker by including an operating system (Debian) and a JVM.

Then, use docker save to save the Docker image:

docker save streambed-cli/arm/fdp:latest -o fdp-arm-latest.tar.gz

Upload

Now, setup a data directory and copy the certificates into it:

mkdir data
cp "$CAF_CERT_PATH" data
cp "$CAF_KEY_PATH" data

Next, we’ll create a start script that will start (and restart) xDP for us:

cat <<EOF > ./start
#!/usr/bin/env bash

set -e

sudo docker stop fdp &>/dev/null || true

sudo docker rm fdp &>/dev/null || true

sudo chown -R 9999:9999 ~/fdp/data

sudo docker run -d --name fdp --net=fdp --restart=always \
  -e CAF_APP_APPDATA_DIR=/data \
  -e EXTERNAL_STORAGE=/data \
  -e JAVA_OPTS="-Diox-sss-stub-server.encryption-key=00000000000000000000000000000000" \
  -p 5732:5732 \
  -p 9870-9879:9870-9879 \
  -p 1690:1690/udp \
  -p 1692:1692/udp \
  -v "\$HOME/fdp/data:/data" \
  -m 512M streambed-cli/arm/fdp:latest
EOF

chmod +x start

Then, use scp to copy the local directory to the Raspberry Pi:

scp -P "$RPI_PORT" -pr "$(pwd)" "$RPI_USER@$RPI_HOST:~"

Install

Now that we’ve uploaded everything, we’ll need to install our image. Run the following command:

ssh -p "$RPI_PORT" "$RPI_USER@$RPI_HOST" 'docker load < fdp/fdp-arm-latest.tar.gz'

Start xDP

The image is now loaded and ready to start. To start it, run the following command:

ssh -p "$RPI_PORT" "$RPI_USER@$RPI_HOST" 'fdp/start'

8. Provision an initial user identity

At this stage there are no users who are able to authenticate with xDP. We must now create one. To do this we login to xDP’s container and create a user by sending a request directly to the secret store using its credentials.

We login using ssh:

ssh -t -p "$RPI_PORT" "$RPI_USER@$RPI_HOST" 'docker exec -ti --user root fdp /bin/bash'

Once logged in, we need to set some environment variables so we can gain access secret store:

export CAF_APP_ID=some-user
export CAF_SYSTEM_UUID=some-password
export CAF_SS_IP_ADDR=localhost
export CAF_SS_PORT=9876

We will need curl:

apt-get update && apt-get -y install curl

Provision a temporary user

Next, we obtain a token:

export SS_TOKEN=$(curl -v -k "$CAF_SS_IP_ADDR:$CAF_SS_PORT/SS/TOKEN/$CAF_APP_ID/$CAF_SYSTEM_UUID")

We now create a user named temp with an insecure password of password. We will shortly delete this user once a new one is setup.

curl -v -k \
  "$CAF_SS_IP_ADDR:$CAF_SS_PORT/SS/$CAF_APP_ID/Object" \
  -H 'Expect:' -H 'content-type:multipart/form-data' \
  -F ss-token="$SS_TOKEN" \
  -F object-type=Object \
  -F object-name=default/secrets.users.temp.password \
  -F ss-content=5bc0be4d1aaedb5b513449a59f85da3bd668c1e581d5a8a8129471f1b1b9521692ea588f8a7ec5a7884a7bfbe85875011d0c33d755541f0d2d91e51b08d40687ac02d8391466dac1cdb3a8ff

You must receive an HTTP/1.1 200 OK response. If you do not then please ensure that you copied the token and secret correctly. It can be easy to make a mistake.

You have now provisioned a user named temp.

Provision an application secret

Following on from provisioning a primary user, you must now associate a secret with the application. The secret is used to sign any tokens that may be issued and is done to prevent tampering.

Note that if this is an upgrade, and not an installation then you should use the same key as last time, and not generate a new one. If you do not then you will not have access to the data that you had before.

export APP_SECRET=$(cat /dev/urandom | tr -dc 'A-F0-9' | fold -w 32 | head -n 1)

Note the key produced for performing subsequent upgrades. DO NOT LOSE THIS KEY!!! (use echo $APP_SECRET to see the key)

curl -v -k \
  "$CAF_SS_IP_ADDR:$CAF_SS_PORT/SS/$CAF_APP_ID/Object" \
  -H 'Expect:' -H 'content-type:multipart/form-data' \
  -F ss-token="$SS_TOKEN" \
  -F object-type=Object \
  -F object-name=default/streambed.application-secret \
  -F ss-content="$APP_SECRET"

Exit out of the console and back to your machine.

9. Provision an admin user

This step will use the temp user with the insecure password we created earlier. We will create a new user and then remove the temp one.

Note that it will take about 5 minutes for xDP to startup on an IR800 series router and similar resource constrained devices. If you experience connectivity issues then it is likely that the application is still starting up. To track the progress of startup, login to the GOS and use virsh console fdp to view the application’s output. When ready, you will see that various ports have been bound to.

We first establish an environment for streambed command line tool communication:

export CONTROL_CENTER_HOST="$RPI_HOST"
export STREAMBED_MQTT_GATEWAY_HOST="$RPI_HOST"

Now login as the temp user with as password of password:

Note that we use the --insecure option as we are using a self-signed certificate.

streambed --insecure login

Supposing your user is cisco, the following command will prompt you for a password:

streambed --insecure user add cisco

Now that you have that user setup, remove the insecure one we setup earlier.

streambed --insecure user rm temp

10. Provision Control Center secrets

xDP’s Control Center encrypts its transformers and dashboards using secrets that need to be created.

Additionally, the LoRa server needs a key to encrypt its end device data.

First, generate three random secrets:

export END_DEVICES_SECRET="$(LC_CTYPE=C tr -dc 'A-F0-9' < /dev/urandom | fold -w 32 | head -n 1)"
export TRANSFORMERS_SECRET="$(LC_CTYPE=C tr -dc 'A-F0-9' < /dev/urandom | fold -w 32 | head -n 1)"
export DASHBOARDS_SECRET="$(LC_CTYPE=C tr -dc 'A-F0-9' < /dev/urandom | fold -w 32 | head -n 1)"
export LAYERS_SECRET="$(LC_CTYPE=C tr -dc 'A-F0-9' < /dev/urandom | fold -w 32 | head -n 1)"

Then, create the secrets:

streambed --insecure secret add secrets.lora-server.end-devices.key "$END_DEVICES_SECRET"
streambed --insecure secret add secrets.dashboards-events.key "$DASHBOARDS_SECRET"
streambed --insecure secret add secrets.layer-events.key "$LAYERS_SECRET"
streambed --insecure secret add secrets.transformer-program-events.key "$TRANSFORMERS_SECRET"

11. Configure Network

In general, it is best to assign a static IP address to xDP so that the Network Server can be resolved easily by equipment not having to rely on DHCP over unreliable networks. Edit /etc/dhcpcd.conf to assign an address. By convention, we use 192.168.1.254 as this often sits outside of the DHCP range (check though!).

We need to configure a packet forwarder to use the address of the Network Server (NS). The NS binds to two UDP ports which are 1690 and 1692 by default. Given this and a configuration of the NS to bind to 192.168.1.254, you should now able to configure your packet forwarder.

Finale

To verify the installation, visit https://raspberrypi.local:9870/ (as per the location of the PI).

Next: Maintenance