Deep into Connectors

Let’s go deep into connectors to better undestand how it works and how to develop new Ciao Connectors to connect to external applications.

Connectors are classified in managed and standalone, and also in internal and external. Manged Connectors are Connectors that are started and stopped by Ciao, otherwise standalone are started/stopped manually (or automatically by a system configuration) that connect to Ciao when it's available.

Internal Connectors are those Connectors that come with Ciao, they are the main Connectors for Ciao:

Internal Connector are managed.  External Connectors are instead those Connector that can be installed separately. 

Each connector is a standalone module that communicate with Ciao Core component via TCP using JSON, and has at least one configuration file to provide Ciao Core some mandatory information:

  • what's the name of the connector?
  • is the connector enabled/disabled?
  • what features does the connector provide (read, write, writeResponse)?
Such configuration files are stored under the configuration directory of Ciao Library: /usr/lib/python2.7/ciao/conf/

How To Develop a Connector

As stated above Ciao Core has been designed to be modular, in this way anyone can develop one (or more) Connector for its own needs.
They can be developed in whatever programming language supported by Linino OS. By default Linino OS supports:

  • Python
  • Lua
  • C / C++
Linino OS supports also Node.js but it’s not installed by default. See here how to install Node.js on Linino OS 

We recommend python to approach to the development of new connectors becouse we create a module called CiaoTools that aims to semplify the development of a Connector.

Before starting

To better understand how to develop a Connector, we start from an existing Connector and go in to step by step. The Connector is SMTP connector, you can find the entire code here on GitHub. As you already know SMTP (Simple Mail Transfer Protocol)is the protocol used to send emails.

Note: the name of the Connector we are going to create is smtp, this name will be used in naming folder and configuration file.

Folders structure

Consider the main folder as the repository name in, and for semplicity we adopted this naming ciao-connector-<connector name>

ciao-connector-smtp
|___ examples
       |___ SendEmail
         |___ SendEmail.ino

|___ smtp
      |___ smtp.py
      |___ smtp.json.conf
|___ smtp.ciao.json.conf

examples folder will contains all arduino sketch example files, and smtp will contains the connector itself, with configuration file and python code. The name of the directory is the same of the connector. smtp.ciao.json.conf is the Ciao Core Configuration File which will be exaplained below.

Ciao Core Configuration File

Ciao Core Configuration File is a file used by the Ciao Core to identify the Connector. It contains the main informations that allows Ciao to interact with the Connector and also the user interfaces implemented. Following an example Ciao Core Configuration File for a generic Connector (eg: samplecon.ciao.json.conf).

{
      "name":"samplecon",
      "enabled":true,
      "type":"managed",
      "core":">=0.1.0",
      "commands": {
           "start": ["/usr/lib/python2.7/Ciao/connectors/samplecon/samplecon.py"],
            "stop":["/usr/bin/killall","-s","HUP","samplecon.py"]
      },
      "implements": {
          "read":{"direction":"in","has_params":false},
          "write":{"direction":"out","has_params":true},
          "writeresponse":{"direction":"out","has_params":true }
      }
}

name: it's the name of the connector (lower case). It has to be used during registration with Ciao Core, it's the same name that will be used on MCU side (sketch). It's a best practice to name the file according with this parameter (e.g. samplecon.ciao.json.conf)

enabled: it can be true or false, it states if the Ciao Core should start/stop the connector and accept communication from it

type: it can be "managed" or "standalone", managed means the Ciao Core will start/stop the connector, otherwise standalone means the connector is "self-handled" (e.g. a system service - or daemon - who connects to Ciao only if it's available)

core: it specify which version of Ciao is required to run this Connector. Ciao Core version uses semver.

commands: [required only if type=managed]
  • start (the command to START the connector, it can be a relative or absolute path)
  • stop (the command to STOP the connector, it can be a relative or absolute path)
implements: it's an array representing the features implemented by the connector (from the MCU point-of-view)
  • read: if specified enables the instruction "Ciao.read()" will be available on the MCU side
  • write: if specified enables the instruction "Ciao.write()" will be available on the MCU side
  • writeresponse:  if specified enables the instruction "Ciao.writeResponse()" will be available on the MCU side each one of such entry must have two keys:
  • direction: this value represents the direction of the action from the MCU point-of-view, it can be "in", "out" or "result".
    • "in": the command waits for an interaction from the "outside World" - such as a chat message or an incoming HTTP request
    • "out": the command has to send message/data - such as an outgoing chat message
    • "result": the command has to make a request and provide - once done - a response
  • has_params: this value can be set to true or false, it specifies if there will be any parameter passed through the read/write/writeResponse methods by the MCU sketch to this connector
Lets write the Ciao Core Configuration File for the SMTP Connector: smtp.ciao.json.conf

{
   "name":"smtp",
   "enabled":true,
  "type":"managed",
  "core":">=0.1.0",
  "commands": {
   "start":["/root/.ciao/smtp/smtp.py"],
   "stop":["/usr/bin/killall","-s","HUP","smtp.py"]
    },
  "implements": {
   "write":{"direction":"out","has_params":true }
    }
}

It implements only write becouse the SMTP is a protocol used only to send emails, and we will use ciao.write, to send emails out (direction) from the microcontroller. Of course the write will have parameters used to specify receiver and mail body.

Connector Configuration File

The Connector Configuration File is a json file used by the Connector to read parameters and configurations when it starts. It is named sampleconn.json.conf and it is in the connector folder. Following an example of Connector Configuration File:

{
   "name":"samplecon",
   "description":"Sample Connector",
   "authors":["Jhon Doe <jhon.doe@email.org>;"],
   "repository":"https://github.com/jhon-doe/ciao-connector-samplecon",
   "version":"0.0.1",
   "params": {
   "paramA":true,
   "paramB":"value",
   "paramC": 1
    },
   "log":{
   "level":"debug"
    }
}

name: it's always the name of the connector (lower case). It’s used in the previous file name, folder and in Ciao library to identify the Connector

description: the description of the Connector.

authors: is an array containing the name and email of the authors in the format “Name <email>”.

repository: the repository address of the Connector source code.

version: version number of the Connector

params: is an array of params in the form of key-value. As the JSON format integers, strings and boolean are supported.

log: is an optional value, here you can set the logger for the connector.

Lets write the Connector Configuration File for the SMTP Connector (smtp.json.conf) and focus your attention on which parameters could be useful for a Connector which will sends email.

{
   "name":"smtp",
   "description":"SMTP connector for Ciao",
   "authors":["Sergio Tomasello <sergio@arduino.org>;"],
   "repository":"https://github.com/arduino-org/ciao-connector-smtp",
   "version":"0.0.1",
   "params": {
   "host":"smtp.gmail.com",
   "port":465,
   "sender":"YOUR-EMAIL@gmail.com",
   "user":"YOUR-EMAIL@gmail.com",
   "password":"YOURPASSWORD",
   "ssl":true,
   "tls":true,
   "auth":true
    },
   "log":{
   "level":"info"
    }
}

Note: Before run this a Connector please pay attention to the paramenters, fill the values with your own credentials/informations.

The Connector

Now that the configuration file are ready let’s  go to Connector code. To code it we made use of the CiaoTools python module created to semplify the development and to take care of the communication with Ciao Core. Following smtp/smtp.py code:

import smtplib, ciaotools, os
from email.mime.text importMIMEText

# DEFINE CONNECTOR HANDLERS AND FUNCTIONS
def sendEmail(to, subject, body):

   #create email message
   msg =MIMEText(body)
   msg['Subject']= subject
   msg['From']= smtp_sender
   msg['To']= to
   try:
  if smtp_ssl:
   smtpserver = smtplib.SMTP_SSL(smtp_host, smtp_port)
   smtpserver.login(smtp_user, smtp_pwd)
  else:
  if smtp_tls:
   smtpserver = smtplib.SMTP(smtp_host, smtp_port)
   smtpserver.ehlo()
   smtpserver.starttls()
   smtpserver.ehlo()
   smtpserver.login(smtp_user, smtp_pwd)
  else:
  if smtp_auth:
   smtpserver = smtplib.SMTP(smtp_host, smtp_port)
   smtpserver.login(smtp_user, smtp_pwd)
  else:
   smtpserver = smtplib.SMTP(smtp_host, smtp_port)
   smtpserver.sendmail(smtp_sender ,[to], msg.as_string())
   smtpserver.close()
   exceptException, e:
   logger.error("Error seding Email[%s] to %s: %s"%( subject, to, str(e)) )
def handler(mcu_req):
   if mcu_req["type"]=="out":
   sendEmail(mcu_req["data"][0], mcu_req["data"][1], mcu_req["data"][2])

# the absolute path of the connector

working_dir = os.path.dirname(os.path.abspath(__file__))+ os.sep

# LOAD CONFIGURATION

# load configuration object with configuration file smtp.conf.json
config = ciaotools.load_config(working_dir)

# load parameters
smtp_sender = config["params"]["sender"]
smtp_user = config["params"]["user"]
smtp_pwd = config["params"]["password"]
smtp_host = config["params"]["host"]
smtp_port = config["params"]["port"]
smtp_ssl = config["params"]["ssl"]
smtp_tls = config["params"]["tls"]
smtp_auth = config["params"]["auth"]

# name of the connector
name = config["name"]

# CREATE LOGGER
log_config = config["log"]if"log"in config elseNone
logger = ciaotools.get_logger(name, logconf=log_config, logdir=working_dir)

# CALL BASE CONNECTOR

#Call a base connector object to help connection to ciao core
ciao_connector = ciaotools.BaseConnector(name, logger, config["ciao"])

#register an handler to manage data from core/mcu
ciao_connector.receive(handler)

# start the connector thread
ciao_connector.start()

Let’s go step by step to exaplain the main code of the module

Get the current working directory for the Connector and load parameters from the smtp.json.conf file

working_dir = os.path.dirname(os.path.abspath(__file__))+ os.sep
config = ciaotools.load_config(working_dir)
Load each parameters

smtp_sender = config["params"]["sender"] 
smtp_user = config["params"]["user"]
smtp_pwd = config["params"]["password"]
smtp_host = config["params"]["host"]
smtp_port = config["params"]["port"]
smtp_ssl = config["params"]["ssl"]
smtp_tls = config["params"]["tls"]
smtp_auth = config["params"]["auth"]

#name of the connector
name = config["name"]
Create the Logger object

log_config = config["log"]if"log"in config elseNone 
logger = ciaotools.get_logger(name, logconf=log_config, logdir=working_dir)
Create the Connector from a Base Connector of the Ciao Tools module

ciao_connector = ciaotools.BaseConnector(name, logger, config["ciao"]) 
Register a handler to manage data from core/mcu

ciao_connector.receive(handler) 
handler is a function that will be triggered every time that a ciao.write().read() or .writeResponse() is called in the sketch.
Finally start the Connector

ciao_connector.start() 

The handler function, triggered by the BaseConnector has an arguments called mcu_request which contains all the information about the request that comes from the microcontroller:

def handler(mcu_req):

   if mcu_req["type"]=="out":

   sendEmail(mcu_req["data"][0], mcu_req ["data"][1], mcu_req["data"][2])
The mcu_req is an object which contains these items:
type: is a string, and the possible values are: out, in and result. depends on the kind of implementations done in the microcontroller, see the smtp.ciao.json.conf
data: is an array of string, they are the arguments of the ciao.write() ( or .read() or .writeResponse() )
checksum: is a string used to identify a request, it’s useful when the Connector wants to create a response for the mcu and bind the received request.

Finally the sendEmail function allows to use the SMTP protocol to send emails. It accepts three arguments (which comes from the mcu_request). They are the receiver, the email subject and the email text body.

TODO CIAO TOOLS

To close the loop, take a look at the Arduino example in example/SendEmail/SendEmail.ino

#include<Ciao.h>
// named constant for the pin the sensor is connected to
constint sensorPin = A0;

// treshold temperature in Celcius
constfloat tresholdTemp =20.0;

// Receveiver subject and body used for send emails
String emailTo ="RECEIVER EMAIL ADDRESS";//<-- SET HERE THE EMAIL ADDRESS OF THE RECEIVER
String emailSubject ="Temp. Alert";
String emailBody ="The Room Temperature is: ";

void
setup() {
 Ciao.begin();
}

void loop() {

 // read the value on AnalogIn pin 0
 int sensorVal = analogRead(sensorPin);

 // convert the ADC reading to voltage
 float voltage =(sensorVal /1024.0)*5.0;

 // convert the voltage to temperature in degrees C
 float temperature =(voltage -.5)*100;

 //check if the temperature exceeds the treshold
 if(temperature > tresholdTemp){

//send an email via ciao smtp connector
Ciao.write("smtp", emailTo, emailSubject, emailBody + temperature );
delay(270000);
 }
 delay(30000);
}

This simple sketch shows how to read the temperature value from a temperature sensor, check a treshold and send email. The call to send the email is simple and explicit:

Ciao. write("smtp", emailTo, emailSubject, emailBody + temperature );

The ciao.write() has three arguments (without considering the connector name smtp) called emailTo, emailSubject and emailBody with the temperature value. They match the data array items of the mcu_req in the handler function, in Connector code.

Push your Connector to the Arduino Connectors repository

If you think that your Ciao Connector could also be useful to others Arduino community users, why don’t push it to the Arduino Connectors repository?
Arduino Connectors repository is a Linino OS feed, once developed a connector you only need few steps to push to the Arduino Connectors Repository:

  • Fork and clone the repo
  • Create a folder for your connector (pay attention at the naming convention)
  • Create a Makefile for your connector starting from a provided template of Makefile
  • Create a Pull Request
For more information read here.

After Connector push approving, you will able to install the connector by running this commands in your Linino OS console:

opkg update
opkg install ciao-connector-<your-connector>