The framework architecture has been inspired by the Unix/Linux philosophy with respect to decomposition of features and functions.

Instead of being made of a big monolothic software, it is broken in small parts, each one have a well defined and limited role, and runing in its own system process. In addition, services provided by the underlying Linux are exploited as much as possible to optimize the development efforts.

Communications between these building blocks are based on 3 main principles :

• a messsage bus for asynchronous communication and notification of spontaneous events
• inter-process communications
• open and documented file formats

First two items above rely on D-Bus features. In addition to make inter-process transparent, it also allows implementation language neutral invocations.

The overall components set is divided into :

• the core, which is the only mandatory part, and which contains the base mechanisms, such as the events based communication layer, the device network configuration management,...
• a collection of extensions, which implement complementary features, among which :
  • sensor events local storage
  • Web based user interface framework
  • support of protocols such as ModBus, NMEA, 1-wire, including the drivers for a first set of off-the-shelf commercial products

Most of the time, active extensions such as device network coordinators for instance are implemented as a service started during the system init sequence, and often acting as a producer or/and consumer of messages on the communication bus. For instance the ModBus network coordinator service takes care of polling the attached devices and emitting the events corresponding to the data returned by them, according to the policy defined in the network configuration. The sensor events storage service will in contrary listen for sensor events on the bus and store them using the appropriate persistence layer.

Other services can just deal with stored data. As an exemple, a periodic sensor data upload service will be scheduled to run at fixed periods (relying on the system cron service), extract events data from the local storage thanks to inter-process communication with the local storage manager service, format them according to the destination tier and send the result over Internet using the appropriate protocol.

The communications inside CSTBox framework are based on a few key concepts and paradigms :

• programing language neutral message bus
• white board
• explicit and extensible message format

The combination of these base recipes allows an open and as affordable as possible extension and customization process, since assembled parts can be developed in any suited language.

Communication path

The downstream path, from the equipments to the application, is illustrated hereafter :

One can see here that what we call the driver is responsible for translating the specific data frames coming from the equipments into technology agnostic events emitted on the communication bus.

On the data consumers side, the involved components listen to the traffic on the communication bus, eventually tuning them for selecting the messages related to changes of variables they are interpreted by. This filtering is based on the content of the message fields. This behaviour can be seen what we do when choosing a radio station on a receiver.

The reverse path, from application to equipments, is shown in next picture :

The drivers use the same message listening and filtering as before, to select those related to changes in the control variables pertaining to the kind of devices they manage.

Messages format

In order to be easily produced and analyzed, the transported messages are standard strings encoded in JSON. Although a bit more costly in terms of data size and processing, this option as been preferred to a binary format for the following reasons :

• strings don't have the problem of bytes ordering for representing numeric values, thus relieving producers and consumers to bother with this concern,
• strings are human readable, which is important for us, since self-explanatory data are one of the keystones of our approach,
• applications and usages targeted by the framework don't generate a high throughput of messages, and these messages are generally short, so we can afford the extra size and processing needed.
• JSON provides a way to serialize structured data at a very reasonable cost, unlike XML for instance, and is easy to parse and process

timestamp

the event time stamp, in milliseconds, and relative to the standard epoch on Unix systems

vartype

the type of the variable which change is notified by the event (ex: temperature)

varname

the name of the variable

data

an message dependent payload, represented as a dictionary containing the data fields key and value

value

(mandatory) the variable value

units

if relevant, units used to represent the value

A comfortable collection of components is provided out of the box, covering the core features required to deal with sensor data collection and storage, box configuration, process scheduling,…

Here they are :

timed data local storage

Manages the storage and retrieval of timed events corresponding to changes of the state variables used to monitor the observed system. This component is made of two parts :

• a daemon process, listening for variables value changes events and storing them locally,
• a library providing high-level services for querying and retrieval of data from the local storage.

administration and configuration Web based console

Provides an HTTP server, hosting a Web based application for :

• editing the device network
• browse data locally stored
• configure system level parameters
• manage active services
• consult and retrieve components execution logs

This application uses a plugin based architecture, allowing integrators to add their own applets for extending the supported features.

Here is a screen-shot of the desktop, showing some applets (both stock and application specific ones) :

• 
• 
• 

‹ ›

For running CSTBox

The core part of the framework is written in Python.

Python being included in common Linux distributions, even lighter ones, you should have nothing special to do most of the time, apart if you are building a Linux from scratch or using some more exotic distributions.

We have used Debian based distributions for most of the applications we have deployed in the field during the various projects and experiments CSTBox has been used for, installed on nano-PC type boxes, and even on card credit size computers such as the Raspberry Pi. In some specific cases, we were constrained to use very low-end targets (field controllers for instance) running LFS or alike, but could successfully install and run the Python VM, and execute CSTBox based applications without any problem.

For developing your components

Since our approach is implementation language agnostic, there is no mandatory dependency.

For instance, if you only need to work with sensor data which have been stored by our included recorder, you do not need any special library, since the storage is organized in a collection of pure tabulated text files. Although this can sound strange, it has been proven well suited. For instance, a currently running application, collecting various ambient data using sensors disseminated in a house, manages nearly 3 million records using this technique, without any problem and using a reasonable storage space.

If the component you want to create will communicate with some of the ones included in the framework distribution, you will need to be able to send and/or receive messages on D-Bus. Fortunately, D-Bus implementation is very portable (it even runs on Mac OSX and Windows) and bindings exist for a lot of programming languages. So you'll just have to pick and install the appropriate one.

Some recipes for common tasks in the life of a CSTBox developper or integrator.

Create a Python service

Learn how to create a new CSTBox service written in Python. More...

Create a Java service

Learn how to create a new CSTBox service written in Java. More...

Support a new kind of hardware

Learn how to add the support of devices working with a communication mode (e.g. a protocol) not yet supported by the CSTBox. More...

Use stored events data

Learn how to access stored event data, either by using the Python library directly or via inter-process communication from any supported programming language. More...

Use external data providers

Learn how to used a remote web service to get data from it and use them as if they were produced by a local piced of hardware. More...

Inter-operate with an external application

Learn how to interface the CSTBox world with an other application running on the same hardware More...

Push data to a remote system

Learn how to make use of data export infrastructure to send local data to a remote system More...

CSTBox can be developed using different languages. Here is a list of available documentation for some of them:

Python

Python modules and libraries composing the core framework and some publicly released extensions.

C-API

Collection of C libraries for the development of software modules for CSTBox.