Configuration
As a first step, we’ll need to configure the Control Center. This segment of the tutorial will walk you through starting the application via the Developer Sandbox, and using the Streambed CLI to create the various resources that are required.
Start the Developer Sandbox
First up, we’ll need to start the Developer Sandbox. This starts a number of essential Streambed services, such as its durable queue, secret store, and MQTT gateway, as well as the Control Center itself.
For a fresh experience, run the
streambed cleancommand to remove any existing data before continuing.
The following command will start the sandbox using Docker Compose:
sandbox | docker-compose -p xdp-sandbox -f - up
A Quick Overview
Streambed’s architecture is centered around a durable queue. When a payload arrives from a device, it is encrypted and stored on disk in a particular topic. Payloads typically arrive from LoRaWAN-enabled devices or a configured MQTT gateway. You can find a small introduction on both of these technologies below.
A transformer is then responsible for reading from that topic, decoding the payload, and persisting an observation to a different topic.
Dashboards can be created to visualize and analyze the observations produced by transformers.
LoRaWAN
LoRaWAN is a media access control (MAC) protocol for wide area networks. It is particularly designed for low-powered devices to be able to communicate over long range wireless connections. LoRaWAN communications are encrypted using AES-128 and a number of 128-bit keys. For more information, The Things Network has compiled an extensive overview of its design.
Streambed has a LoRaWAN-compliant network server that can receive LoRaWAN payloads, encrypt them, and append them to a topic specific to the device that transmitted them.
MQTT
MQTT is a protocol that defines a publish/subscribe message transport that is widely used in the IoT. Using MQTT, clients connect to an MQTT broker and publish messages to a topic, and any clients that have previously subscribed to that topic will receive those messages. Architecturally, this provides decoupling of the publisher from the subscriber.
Streambed provides an MQTT gateway that can receive MQTT publications, encrypt them, and append them to a topic specific to the device they are associated with.
Defining Device Types, Devices, and Observation Types
The following sections use commands to provision the system. If preferred, the web interface may be used instead. Login with username “admin” and password “password”, then select Settings.
First, let’s create a device type which declares the topic where the raw payloads received for the device are stored.
The following will create a Device Type for our sensors:
lora type add air-temp-data-up-mac-payload "My Air Temperature Sensor"
Next, we’ll need to create an Observation Type. An Observation Type describes the domain model of the system. To help illustrate this, imagine a scenario where you have three sensors from different vendors. Each of these may have a different payload format, and thus their own transformer, but may share the same output format, the observation type. An observation type defines the topic where observations should be appended, the path to a secret that the observations are encrypted with, a name to help identify it, and a view field which helps the system visualize the data contained in associated observations.
For our air temperature sensor, we provide a function that can append the unit (°C) and indicate if it’s too hot.
Execute the following to create the Observation Type:
streambed observation-type add air-temp-data-up-json \
--name 'My Air Temperature Observations' \
--secret-path secrets.air-temp.key \
--view - <<EOF
function(endDevice, latestObservation) {
return {
observationsMapValue: 'temperature',
observationsMapGauge: 'temperature',
fields: {
temperature: {
value: latestObservation.data.temperature,
name: 'Temperature',
text: latestObservation.data.temperature + ' °C',
color: latestObservation.data.temperature > 30 ? 'danger' : 'success',
scaleMin: 0,
scaleMax: 45
}
}
};
}
EOF
This command created the air-temp-data-up-json observation type, which encrypts observations that are recorded using the secret stored at secrets.air-temp.key. It also defined a view function which is responsible for returning metadata given the latest observation. This function returns metadata about the fields contained in the observation, temperature in this case, and specifies which fields should be used for indicating its latest value and gauge.
Now, we’ll store a secret that is used to encrypt the transformed observations:
streambed secret add secrets.air-temp.key 3B7E151628AED2A6ABF7158809CF4F3D
By encrypting all observations, tight control can be provided over which applications can access the data. It also aids in GDPR compliance. If regulatory measures require the destruction of data, this can be performed by simply destroying the key.
Encryption
Lastly, we’ll provision a device so that we can generate synthetic payloads for it:
lora end-device add air-temp-data-up-mac-payload v1 abp \
--dev-eui B4B05FF9F3C388E9 \
--dev-addr F3C388E9 \
--app-s-key 8F74FD863E5F7854D58D7CACC15804CA \
--nwk-s-key F82D9D49EA6E88E69DC40A98E6B90D79 \
--pos "-37.15,146.19" \
--name "My Air Temperature Sensor #1"
The above command specifies the device’s DevEUI and DevAddr, which are used for assigning a NwkAddr used internally by the system, as well as LoRaWAN AppSKey and NWkSKey values that ensure that communcations are secure. These particular values are used for development, as we’ll be using MQTT to generate payloads, instead of LoRaWAN.
Defining a Transformer
When payloads are persisted to a topic by the LoRaWAN network server or MQTT gateway, they are processed by a transformer which receives these payloads and produces observations, persisting them to a separate topic. We typically refer to these raw payloads as the inlet, and the transformed observations as the outlet.
Transformers for the Control Center are written in the JavaScript programming language, but may also be programmed in Java or Scala for more advanced cases. See the Advanced Tutorial for more.
When the Control Center invokes a transformer, it provides the time associated with the payload (ISO 8601), the NwkAddr of the device, the LoRaWAN FPort value (if available), and the payload itself, encoded as a Base64 encoded string. Base64 is chosen because the payload, being binary, needs to be encoded in a format that makes it easy to consume from within the JavaScript environment. The FPort can be used to distinguish between the type of payload, if the device in question takes advantage of that LoRaWAN feature.
The following command will create a transformer that can decode our air temperature sensor’s payloads. For simplicity, our sensor reports its readings in °C using an ASCII encoding.
streambed transformer add \
--name 'My Air Temperature Transformer' \
--inlet-topic air-temp-data-up-mac-payload \
--outlet-topic air-temp-data-up-json \
--source - <<EOF
function transform(time, nwkAddr, fPort, payload) {
return {
time: time,
nwkAddr: nwkAddr,
temperature: parseFloat(atob(payload))
};
}
function test() {
return transform("2019-03-29T06:02:04.539Z", 65959, 15, "MjUuNQo=").temperature === 25.5;
}
EOF
In our transformer’s source code, we specify a transform function that performs the computation. This function will receive a base64 encoded payload, so we’ll use atob to decode it into a byte string, and then parseFloat to parse it into a numeric type.
Additionally, we also specify a test function. This can help provide confidence that the transformer is working as expected.
When editing a transformer using the web interface, the Test button can be selected to invoke the tests.
Configure MQTT
Streambed provides MQTT integration by way of its MQTT Gateway. This allows easy integration with any other systems that provide MQTT support.
In production, many of Streambed’s payloads will be transmitted from devices using LoRaWAN, but for now we’ll leverage its MQTT support instead, allowing us to easily simulate the transmission of payloads that our transformer will then process.
The following configuration will ensure that data published to the /tutorial/air-temp/app2dev/data air-temp-data-up-mac-payload MQTT topic will be persisted to the air-temp-data-up-mac-payload DurableQueue topic:
streambed mqtt add down \
tutorial-air-temp \
/tutorial/air-temp/app2dev/data \
air-temp-data-up-mac-payload \
--data-is-binary
Next Steps
The Control Center is now configured and ready to visualize sensor data. In the next segment, we’ll focus on creating a dashboard and then finally generate some example payloads that will be transformed and visualized in real-time.