Tuesday, March 26, 2019

Other Inputs

The beauty of Node-Red is it's flexibility. If you have looked on the list of nodes, you may have noticed there are several inputs besides MQTT. Some of the inputs include GPIO, TCP, UDP and Web sockets. The GPIO inputs are the pins on the RaspberryPI. Sockets need an external server.

In previous posts, I mentioned using a DS18B20 temperature sensor. This sensor works great, but has a limitation in where I put it. The garage node I connected it to is physically too close to the house, so the temperature it reads is about 15-20 degrees warmer than it really is. The house is radiating heat, and the sensor is picking up the heat.

To get around this, I could have run a longer wire and put the sensor out in the yard farther. I could have built a temperature node, and mounted it in the yard. Or I could get the input from another source. Running wires farther out in the yard becomes a challenge since people walk in the yard, burying the cable would be possible, but having the sensor close to the ground will measure the radiated ground temperature instead of the air temperature. Getting power out to a remote sensor is a problem, wires would be similar to the remote sensor, solar would be the best, but that adds complexity.

I chose to get the temperature from an external source. I am using openweathermap. There is a node red openweathermap node available. Install the node like the previous post, using the top right hamburger menu, select manage palette, install tab. Enter openweathermap in the search, and press install. A new section will be created in the left node selector section called "weather".

Follow the instructions in the openweathermap Usage section. To use the node, the openweathermap people want to control the volume of queries made against their servers, so they require users to get a key. The normal node-red usage pattern is pretty light against their servers, so it seems they tolerate this volume of use. Follow the directions on the appid page, and check your email for the appid needed.

Once you have your appid, start a new flow. The first node should be openweathermap in node (the one with only a connector on the right). Drag the node to your flow. Double click on the new node, to get to the edit dialog. Enter your appid in the top box, set your node to get the "Current Weather For" and select coordinates for location. (I couldn't get the city, country to work for my location). Give your location a name. Press done.

Enter a debug node next, and connect the two nodes. Press deploy. In the debug tab on the right, the payload will display. The current weather is pretty detailed, including temperature, cloud cover, and other items. The payload looks like:
{ weather: "Clouds",
   detail: "overcast clouds",
   icon: "04n",
   tempk: 281.56,
   tempc: 8.4,
   temp_maxc: 9.4,
   temp_minc: 7.7,
   humidity: 57,
   pressure: 1023,
   maxtemp: 282.59,
   mintemp: 280.93,
   windspeed: 4.1,
   winddirection: 200,
   location: "Eden Prairie",
   sunrise: 1553601964,
   sunset: 1553646776,
   clouds: 90,
  description: "The weather in Eden Prairie at coordinates: 44.84, -93.47 is Clouds (overcast clouds)."}

The openweathermap page has details of things like the icon. The temperature in Kelvin isn't useful to most people, but there are simple conversions to Fahrenheit. Some people can work in millibars but most people, in the US at least, think in terms of inches of mercury for the barometric pressure. 

For me, I just needed the temperature. So I created a function that can strip out just the temperature. The Javascript for this function is:

context.tempC = msg.payload.tempc;
msg.payload = context.tempC;
return msg;

The function reads the messages payload.tempc, and creates a new message that only contains that in the payload. The function can be modified to return any or all of the payload desired. If the dashboard was to show clouds and summary that can be done as well. 

I pass that temperature to a copy of the degCtoF that I wrote for the garage node already, and then I send that value out to the graph widget in the garage tab of the UI. The flow looks like:


This will show the nice graph of the last hour or so, without any database or special storage. 

Many external sites work on the token userid concept. It is related to your account on the server side. Since you aren't sending your name, login or anything special it keeps prying eyes out as that value goes across the internet. Using SSL should ensure the value isn't easily seen either. 

Next time, I'll show another external service, allowing voice control of node red. 

Come on back.

Monday, March 18, 2019

More Node-Red


Node-Red is amazingly powerful, and flexible. Maybe in a future post, I set some recipies, including recurring timers, Google Assistant integration and notifications.



Dedicated System Setup


One advantage of using Node-Red in your home is you don't have to worry about security too much. You aren't publishing your daily habits to anyone in the cloud. One disadvantage, you need to give the system all it's smarts. It doesn't mean you can't integrate with the cloud, but it doesn't happen automatically.

Using a RaspberryPI as the dedicated controller will allow the 'PI to be the WiFi hub, DHCP server, MQTT server agent, and provide all the flows without worrying about tying up another machine (home office computer), or having that system walk out the door (laptop).

In my example ESP8266 nodes, I always hard coded the IP address of the server. That works great, since the RaspberryPI is providing all the DHCP and WiFi for the various nodes in the house. If I want to move the system somewhere else, I don't have to worry about reprogramming the nodes server IP address.



The above diagram shows how the home WiFi is separate from the IoT WiFi. To connect to the rest of the internet, a wired connection can be made to the home Cable Mode. Connecting the RaspberryPI to the internet will allow updates and other features to be easier. It doesn't have to be connected to the internet, it is up to you. Since the RaspberryPi connected to the modem will NAT all internet traffic, unless routes are set up in the modem, the RaspberryPI is not visible from the outside network.

For my network, I run the IoT network on WiFi Channel 11 since my home network is on channel 1. Keeping them on separate channels should prevent IoT traffic from being picked up on the home WiFi devices theoretically making things faster. Radio is radio, so if the devices are close enough, they may interfere anyway. (For some reason the Sony PS4 seems to interfere with WiFi devices near it, so if the kids bring the PS4 into the sun room, the IoT devices perform poorly).

My WiFi config file on the RaspberryPI looks like:

pi@raspberrypi:~ $ cat /etc/hostapd/hostapd.conf
driver=nl80211
ctrl_interface=/var/run/hostapd
ctrl_interface_group=0
beacon_int=100
auth_algs=1
wpa_key_mgmt=WPA-PSK
ssid=raspi-webgui
channel=11
wpa=2
hw_mode=g
wpa_passphrase=XXXXXXXX
interface=wlan0
#wpa_pairwise=TKIP
rsn_pairwise=CCMP
country_code=US


The dns config file is the default, really no changes. 

Node-Red Dashboard

This will be a gentle introduction, just barely covering what it is capable of. This is really the big win for Node-Red. 

First there you need to install the dashboard nodes. From the Node-Red main window, hamburger menu on the far right. Left click on that to bring up the configuration menu. Select the "Manage Palette". A dialog with two tabs will appear, select the "Install" tab. Type in "dashboard" in the "Filter nodes box" There will be a list of dashboard items available. For now, we only want the default "node-red-dashboard". Select "install" and give it a couple minutes. After the update completes, scroll toward the bottom of the nodes on the left. 

There will be a section of nodes labled "dashboard" with nodes like "button", "slider", "dropdown", etc. These items can be added to a flow, and used to set settings or view values available in the flow. The updated garage flow with simple outputs for the garage state and temperature graph looks like:

The light blue nodes are dashboard nodes. The garage door ones are the current state as text (open/closed) and the temperature is a graph showing the current trend, with degrees in Fahrenheit. To set the node display, use the payload to get the value:


Create a group, as you wish. For my house these nodes are in the Garage, so I created a Garage group. I also have a Sunroom Group and a Basement group. They are all part of the House, as a hidden group. The payload is enclosed in "{{" and "}}" I'll expand on why that later. Alignment can be changed as desired, the top row above is left, center and right aligned. The next is fill, and the bottom right is label over value. The look that this allows is:


The temperature trend for the last 20 minutes or so is shown, getting colder. Both doors are closed. Pretty simple, but a quick look and you can see it. Note the URL. I chose the home network, since my RaspberryPI is plugged into the home network. Node Red is running on port 1880, the default. But after that there is a /ui, so I don't see the whole Node Red flow builder, but just the dashboard. 

How about if you want to see this on your phone? No problem, that is built in:

Enter the same url (like 192.168.0.35:1880/ui in chrome, safari or whatever browser you prefer. All the dashboard features are there. No special apps, or knowledge of programming needed. 

There is so much more that can be done in Node-Red. Start playing, and you should find ideas that I never thought of. More next time. 


Wednesday, March 13, 2019

An Input Node

In the last post I mentioned that I would build an input node. This node will be capable of monitoring 2 doors, and the temperature. I have this node out in the garage. There are magnetic reed switches near the two doors and a Dallas DS18B20 temperature sensor near the door.

I built the node with the ESP8266, using the Arduino IDE. There is a "Garage" Flow in my Node-Red environment for monitoring this node. The node code is very similar to the tone generator node, but only sends output. It has the capability to turn on the LED, but there isn't anything in the Flow that actually uses it.

Here is the code:


/**
 * Simple action node. Read sensors and send out status
 * 
 * garage/bigDoor to get status of garage door
 * garage/littleDoor to get status of garage door.
 * garage/outsideTemp to read ds18b20 temperature sensor. 
 * 
 * They are separate devices, but using Node Red can be connected.
 * 
 * 
 * TGB 9/30/18
 */

#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <PubSubClientTools.h>
#include <OneWire.h>
#include <DallasTemperature.h>

const char* ssid = "raspi-webgui";
const char* password = "XXXXXXXX";
const char* MQTT_SERVER = "10.3.141.1";

// Data wire is plugged into port D3 on the ESP8266
#define ONE_WIRE_BUS D3

// Setup a oneWire instance to communicate with any OneWire devices (not just Maxim/Dallas temperature ICs)
OneWire oneWire(ONE_WIRE_BUS);

// Pass our oneWire reference to Dallas Temperature. 
DallasTemperature sensors(&oneWire);

//DS18B20
#define ONE_WIRE_BUS D3 //Pin to which is attached a temperature sensor
#define ONE_WIRE_MAX_DEV 15 //The maximum number of devices

WiFiClient espClient;
PubSubClient client(MQTT_SERVER, 1883, espClient);
PubSubClientTools mqtt(client);

/*
ThreadController threadControl = ThreadController();
Thread thread = Thread();
*/

long lastMsg = 0;
int value = 0;
String s = "";

void setup() {
  Serial.begin(115200);

  pinMode(BUILTIN_LED, OUTPUT);     // Initialize the BUILTIN_LED pin as an output

  pinMode(D1, INPUT_PULLUP);      // big door
  pinMode(D2, INPUT_PULLUP);      // little door
  
  // Connect to WiFi
  Serial.print(s+"Connecting to WiFi: "+ssid+" ");
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println(s+" connected with IP: "+WiFi.localIP());

  // Connect to MQTT
  Serial.print(s+"Connecting to MQTT: "+MQTT_SERVER+" ... ");
  if (client.connect("GarageControl")) {
    Serial.println("connected");
    digitalWrite(BUILTIN_LED, HIGH); // turn LED off

    mqtt.subscribe("garage/light", topic1_subscriber);
  } else {
    Serial.println(s+"failed, rc="+client.state());
  }

  // Start up the library
  sensors.begin();
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("GarageControl")) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      client.publish("outTopic","garageControl Reconnect");
      mqtt.subscribe("garage/light", topic1_subscriber);
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" Resetting");
      // Wait 5 seconds before retrying
    }
  }
}

void loop() {
  if (!client.connected()) {
    reconnect();
  }
  
  client.loop();
  publisher();
}

void publisher() {
  long now = millis();
  if (now - lastMsg > 5000)  // 10 second loop.
  {
    lastMsg = now;
    ++value;
    Serial.println("Publish messages: ");


    int bd = digitalRead(D1);
    int ld = digitalRead(D2);

    String bigDoor="closed";
    String littleDoor="closed";

    if (bd) {
      bigDoor = "open";
    }
    if (ld) {
      littleDoor = "open";
    }
    Serial.print(" bigDoor: ");
    Serial.println(bigDoor);
    Serial.print(" littleDoor: ");
    Serial.println(littleDoor);
            
    mqtt.publish("garage/bigDoor", bigDoor);
    mqtt.publish("garage/littleDoor", littleDoor);

    sensors.requestTemperatures(); // Send the command to get temperatures
    int temp = sensors.getTempCByIndex(0); 
    Serial.print(" temperature: ");
    Serial.println(temp);
    String tempS = String(temp);
    mqtt.publish("garage/outsideTemperature", tempS);
  }
}

void topic1_subscriber(String topic, String message) {
  Serial.println(s+"Message arrived in function 1 ["+topic+"] "+message);
  if (message.equals("on"))
  {
    digitalWrite(BUILTIN_LED, LOW);   // Turn the LED on (Note that LOW is the voltage level
  }
  else
  {
    digitalWrite(BUILTIN_LED, HIGH);
  }
}



The Dallas DS18B20 is a simple to use device that uses the Dallas 1-wire protocol for reading data. Several 1-wire devices can be added to the single pin needed, and only ground and power are needed (actually for some devices power is optional, since the devices can be powered by the serial link). The 1-wire library is all that is needed to used these devices with almost any Arduino system. 

The code starts out initializing the 1-Wire device:


#include <OneWire.h>
#include <DallasTemperature.h>

// Data wire is plugged into port D3 on the ESP8266
#define ONE_WIRE_BUS D3

// Setup a oneWire instance to communicate with any OneWire devices (not just Maxim/Dallas temperature ICs)
OneWire oneWire(ONE_WIRE_BUS);

// Pass our oneWire reference to Dallas Temperature. 
DallasTemperature sensors(&oneWire);

We need to include the various libraries. I've assigned the serial data to come in on the D5 ESP8266 pin. Again, I am using the 8266's library definition to avoid having to map to the Arduino GPIO pin names. I find it easier to use the library names, since that is what is silk-screen printed on the device. 

The setup function is mostly identical to the tone generator. The only difference is setting the input mode for the two reed switches and starting the sensor monitoring that checks the ds1820. The "INPUT_PULLUP" setting for the D1 and D2 allow me to have the reed switch connected to the D1 or D2 input pins and the other side connected to ground. 

  pinMode(BUILTIN_LED, OUTPUT);     // Initialize the BUILTIN_LED pin as an output

  pinMode(D1, INPUT_PULLUP);      // big door
  pinMode(D2, INPUT_PULLUP);      // little door
    // Start up the library
  sensors.begin();



Once everything is started, the main loop() just checks the WiFi connection state, and then calls publish(), a new function. The publish function checks the state of the devices and publishes the results. This is the new part that is different from the tone generator node.

void publisher() {
  long now = millis();
  if (now - lastMsg > 5000)  // 10 second loop.
  {
    lastMsg = now;
    ++value;
    Serial.println("Publish messages: ");


    int bd = digitalRead(D1);
    int ld = digitalRead(D2);

    String bigDoor="closed";
    String littleDoor="closed";

    if (bd) {
      bigDoor = "open";
    }
    if (ld) {
      littleDoor = "open";
    }
    Serial.print(" bigDoor: ");
    Serial.println(bigDoor);
    Serial.print(" littleDoor: ");
    Serial.println(littleDoor);
            
    mqtt.publish("garage/bigDoor", bigDoor);
    mqtt.publish("garage/littleDoor", littleDoor);

    sensors.requestTemperatures(); // Send the command to get temperatures
    int temp = sensors.getTempCByIndex(0); 
    Serial.print(" temperature: ");
    Serial.println(temp);
    String tempS = String(temp);
    mqtt.publish("garage/outsideTemperature", tempS);
  }
}

There is a global variable called "lastMsg" that contains the last time the sensors were checked. If the last time was more than 5000 milliseconds (5 seconds) ago, then the sensors will be checked. If less than 5 seconds ago, this function will not do anything. The two door switches are checked. If the switch is closed, the bd/ld values will be 1, otherwise those values will be 1. The MQTT payload is initialized to be "closed". If the big door switch is closed, the "bd" variable will be 1, and the bigDoor payload will be updated to "open". Once both doors have been checked, the mqtt.publish will send out the payload for the two topics, "garage/bigDoor" and "garage/littleDoor".

After the doors have had the MQTT message published, the DS1820 will be checked. The return value from the sensor.getTempCByIndex() will be the temperature in degrees C.  The temperature value will be published on the "garage/outsideTemperature" topic.


A couple thoughts about MQTT


MQTT brokers can maintain state for various topics. There are good things and bad things about maintaining state. If the state is maintained, the last payload will be kept. If a node dies, there may not be a way to determine when the last time the node reported the state. For temperature, a maintained state may not good, since it may say the value is comfortably warm, when the real temperature is very cold or hot.

Node-Red is generally stateless. That is, unless someone publishes an MQTT message, node-red will not check for state of an item. If node-red is restarted, the node states will not be known, until the next time they publish their state.

I generally have each node publish state every few seconds. Even if the state hasn't changed, I assume that node-red doesn't know. It does mean there will be messages going on the network, that may be unnecessary. MQTT messages are small, and even 802.11n WiFi is fast enough for thousands of messages a second.


Garage Flow


The garage flow shows some new concepts. There will be some programming, but should be easy to understand.




Starting on the left top, there are 2 MQTT input nodes. Both of these will use the "localhost" MQTT server. Each will be subscribed to unique MQTT topics. The littleDoor node will be subscribed to the "garage/littleDoor" topic defined above, and the bigDoor node will be subscribed to the bigDoor MQTT topic. Both MQTT nodes are connected to debug nodes, allowing me to see if the message is being received when something comes in. A third MQTT input node is subscribed to the "garage/outsideTemperature"

The temperature node is connected to another debug node, as well as the cDegToF node. The cDegToF node converts an MQTT payload containing a temperature in Celsius and converts that to a payload in degrees Fahrenheit. The output of cDegToF node is connected to a debug node, and a MQTT output node. The MQTT output node publishes the Fahrenheit temperature on the topic "outside/farenheight". Other nodes can subscribe to "garage/outsideTemperature" or "outside/farenheight" depending on their needs.

The cDegToF node is a function node (note the F on the left). This means it is a small program. Nodes can be programmed in JavaScript. The code looks like:



The msg.payload that comes in is converted using the F=9/5C+32 calculation. To edit a function, simply double click on the node after dragging it to the work area. The above dialog will appear.

The mode advanced function node in this flow is the "Door Open Alarm" node. As you can see from the cDegToF node above, functions are easy with one message coming in. For this node, there are two messages that can cause the alarm to sound. Either big door or little door messages will cause the alarm to come on. If one door is open, and the other closed, the function may not know if anything is open, if the node only looks at all msg.payloads.

I need to introduce context. State of nodes in Node-Red have several contexts. The node has a context for the instant it is running, there is a flow based context, that all nodes within a flow can know about, and there is the global node-red context that all flows share.

Code for the Door Open Alarm:

// if either door is open, after 11pm and before 6am 
// sound an alarm.

context.hour = context.global.hour || 12;
context.bigDoor = context.bigDoor || 'closed';
context.littleDoor = context.littleDoor || 'closed';

if (msg.topic === 'garage/bigDoor') {
  context.bigDoor = msg.payload;
} else if (msg.topic === 'garage/littleDoor') {
  context.littleDoor = msg.payload;
} else if (msg.topic === 'time/ISO-8601') {
    context.hour = msg.payload.hour;
}

// check the status, to know if alarm should be on
if (((context.bigDoor === 'open') || (context.littleDoor === 'open')) &&
    (context.hour > 22) || (context.hour < 6)) {
    // msg.payload = "on " + context.therm + " " + context.temp;
    msg.payload = 'on';
} else {
    msg.payload = 'off';
}
return msg;


For the "Door Open Alarm" node, I maintain state in a the node's context. The context is initialized to the current state or if unknown, to the 'closed' state. The time is global context that I have in another node (I'll talk about that in another post).

If a message comes in on a topic that the node knows about (garage big/little door), then the context for that door is updated to the current state. If either door is open during the alarm time, a new message is published containing the payload "on". This node doesn't know the recipient of the payload, only the payload.

That last node in this flow is the MQTT output node that will send the alarm payload to the tone generator. It will publish the payload passed here on the "home/alarm" topic.

This may be lots to digest. Hopefully it will give some ideas of the capabilities of a RaspberryPi and some 8266's.

More to come.





Friday, March 8, 2019

Starting Node-Red

This will be the start of the Node-Red posts. There will be several, and as far as the MQTT/IoT posts go, it will be on going. Node-Red is such a capable and flexible tool it will be very complex to talk about all of it, and provide examples.

The Node-Red page talks about installing the tool. There are ways to install it on the various systems, and the installation page covers all of them. I shouldn't need to expand on that too much more. For any of the extensions, I will cover any of the nuances needed for a specific example, as I cover that. I am open to answering questions, if the instructions aren't clear enough.

Node-Red introduces the concept of "flows" to the IoT landscape. A flow will involve one or more inputs to cause one or more things to happen or output. Flows are not typically written, but rather drawn.



When Node-Red is opened the first time, there are no flows, only nodes. On the left side of the screen are all the available nodes. On the right are tabs allowing the developer to know details about the node selected, and to debug all the flows. To build a flow, simply drag a node from the list to the flow. Then the nodes can be connected by drawing lines between them. To set the properties of a node (name, action, etc), double click on it and a dialog will pop up.

Simple First Flow

To build a simple flow, start with a source node. The first node "inject" when dragged changes names to "timestamp" (this is unusual, but don't worry about it). The first output node "debug" also changes to "msg.payload" when dragged. Drag one of each to the flow, and draw a line from the output of the "timestamp" node to the input of the "msg.payload" node. The screen will look like:



The first thing to notice is the blue dots by both nodes. This indicates that the nodes haven't been "deployed" or saved. Until the nodes are deployed, they will not change actions. The Deploy button is also bright red ready to be pressed. To have this flow do something noticable, double click on the "timestamp node". The edit dialog will appear. For the "Payload" enter "HelloWorld" (or what ever you want to see output). Then press "Done":


The edit dialog will go away. Press the "Deploy" button at the top, and select the Bug icon just below the "Deploy" button. Pressing the "Deploy" button caused a pop up saying the flow has been deployed, the name changed on the "msg.payload" node and the blue dots on the nodes clear as well. On the left side of the "HelloWorld" node there is a nub, that is really a button that can be pressed. 

Press this button, and the "HelloWorld" text will display in the right side debug tab, similar to:



Changing the payload in the original input node will cause the msg.payload node to output whatever is in the payload to output in the debug tab. The debug node can be used to test the payload sent from any node in any flow. 

Adding External Nodes

There are lots of outputs that are available, including mqtt, qweb socket, tcp, udp, etc. There are special ones for email and twitter. To build on the last couple of posts, we will add the MQTT tone node. We can build extra things in the current flow. Add the MQTT output node to the current "Flow 1" that we have been working with, or start a new flow by pressing the "+" tab at the top of the tab pane window. 

Double click on the MQTT node, to bring up the edit dialog. Select "enter new mqtt-broker" and press the pencil button. 


I'll assume Mosquito is running on the same machine as Node-Red. If not enter the IP address or host name of the MQTT broker in the server field. Leave the port as 1883, unless your mosquito server is running on another port. If Mosquito is running on the same machine as Node-Red, then enter "localhost" or 127.0.0.1 for the server name. Leave the other fields and tabs blank for now. Then press "Update". The dialog will switch back to the "Edit mqtt out node".

For this test, we will just test the light of the node. For the topic, enter "alarm/light" and change the name to "light". Then press done. The edit dialog will go away.

Drag two "inject" nodes on the screen. Change the string to sent to "on" for one, and "off" for the other. Drag the output from each to the input to the MQTT input. The screen will look something like:



Press deploy, and the the blue dots will go away and the light node should have a "Connected" square under it. If you can see the Alarm output node, pressing the "on" button will turn the light on, pressing the "off" button will turn the light off. The debug tab will not show what is happening, because all output of the inject nodes goes to the MQTT "light" node. Adding a debug above the MQTT light node will allow you to draw from the "on" and "off" nodes and then the output will be visible in the debug tab.


To input something from MQTT will be similar. Since there is already an MQTT broker defined, select the one created from before. Mqtt being pub/sub, will allow Node Red to be the publisher and the subscriber. It is possible to hear the published message in the same flow. Set the subject as "alarm/light and create a debug node:



It ends up as a loop back, but you can see the MQTT in is subscribed to the topic. Any MQTT node that outputs can be subscribed in node-red that easily.

I've made you dangerous, now go try some more. Maybe have a flow email you a message or something. Next time, we will build an input physical node and start to use it in node-red.


Monday, March 4, 2019

Talking MQTT

This post will be about using MQTT from the command line. Don't be afraid, next post will be instructions on how to do the same thing through the GUI. This post will allow someone to do some troubleshooting if the GUI isn't working, as well as some familiarity with some of the concepts of MQTT.

Before getting started, ensure Mosquitto is installed and running on an agent. For now, it doesn't need to be on the final server, it can be on any supported platform. There is no need even for another node. If running on a Linux system including Raspberry PI be sure to install both the server/agent as well as the clients:

sudo apt-get install mosquitto
sudo apt-get install mosquitto-clients

Once mosquitto is running, then it will be necessary to have two windows connected to the server, where commands can be entered. In one window, enter the command "mosquitto_sub" to subscribe to a topic using the local agent:

pi@raspberrypi:~ $ mosquitto_sub -h localhost -t test

In the other window, enter "mosquitto_pub" to publish a message to that topic:

pi@raspberrypi:~ $ mosquitto_pub -h localhost -t test -m "hello world"

The "-t" option to both "mosquitto_sub" and "mosquitto_pub" indicates the topic the message will be published to. The "-h" option to both is the agent that will handle the messages for the topic, and can be the host name, or the ip address. The mosquitto_pub has a "-m" option for setting the payload of the message that will be published. The mosquitto_sub will just output the payload to stdout. The two windows look something like:


Multiple windows can be opened to be subscribers. When the pub message is sent, they all will receive identical copies of the payload:


If there are nodes, payloads can be sent to them as well. The server/agent host (-h option) must be the same server/agent that the node is attached to. An example, the alarm sounder from the last post can be turned on using the mostuitto_pub application. The topics it is subscribed to are "alarm/light" for the LED, and "home/alarm" for the tone generator.

 pi@raspberrypi:~ $ mosquitto_pub -h localhost -t alarm/light -m "on"

or 

pi@raspberrypi:~ $ mosquitto_pub -h localhost -t home/alarm -m "off"

Using a simple topic/payload combination many items can be controlled. The next post will be the beginning of using Node-Red, a GUI for these commands. 


Stay tuned...

Friday, March 1, 2019

ESP8266 and MQTT

The ESP8266 is a wonderful kludge. It found exposure as a module (ESP-01) that could be added to a micro controller giving it WiFi connectivity. People eventually figured out it was a processor that was more powerful than the typical micro controller it was being added to. The ESP8266 is a 32bit RISC processor running at 80MHz. Eventually there were various development kits put together for it, including the Arduino IDE.

Looking at the Wikipedia page, you can see the device is available in multiple form factors, from small boards to full blown DevKits. The bare boards are designed to be added to various other projects. The Devkits are standalone systems that will typically have a USB port provided for programming, powering and dataIO.

For most of the work I have been doing in my home automation systems, I have been using the NodeMCU or WeMos clones. These are available from many vendors, including Sparkfun, Banggood and others. I have programmed them using the Arduino IDE. I have built little boxes for them using 3D printed boxes from designs that are already on Thingiverse.



Because these modules are so inexpensive, I am able to limit the complexity of the individual node. I have one node that is in my bedroom that only has a NodeMcu and audio oscillator. This will alert me when something is out of the ordinary in the house. If I go to bed and the garage door was left open, or if someone breaks into the house.

The most complex node is the thermostat I have in the sunroom. This node has a touch screen display for displaying the current temperature in the room, and allows setting the thermostat. It also has a temperature and humidity sensor for measuring the conditions in the room.  There is a separate node that will turn on the heater if the temperature is too cold.

Coding for a node


Most nodes have a similar pattern. All the sensors in the node will send a payload on an MQTT topic. All the outputs in the node will listen for specific MQTT topics to take action on. Most of the nodes will only be action nodes (relays, lights, sound, etc), or sensor nodes (door open, temperature, switches, etc). Keeping the node simple will allow the system to be more reliable. Typically, nodes that do both input and output will not have input actions do output actions.

The node must know the MQTT agent to connect to. For my house, I am using a RaspberryPI that is acting as it's own WiFi hotspot. This will allow me to move the RaspberryPI and all the nodes on that network without affecting the home WiFi. The RaspberryPi doesn't need to be connected to the internet for this to work. I'll show in future posts how connecting it to the internet can be useful and safe. A spare Windows, Mac or Linux computer can be used as well, I just like the portability of the RaspberryPi.



A simple output node

/**
 * Simple action node. makes noise when receives alert
 * 
 * home/alert
 * 
 * They are separate devices, but using Node Red can be connected.
 * 
 * TGB 9/30/18
 */

#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <PubSubClientTools.h>


const char* ssid = "raspi-webgui";
const char* password = "XXXXXXXX";
const char* MQTT_SERVER = "10.3.141.1";


WiFiClient espClient;
PubSubClient client(MQTT_SERVER, 1883, espClient);
PubSubClientTools mqtt(client);

String s = "";

void setup() {
  Serial.begin(115200);

  pinMode(BUILTIN_LED, OUTPUT);     // Initialize the BUILTIN_LED pin as an output
  pinMode(D5, OUTPUT);
  
  // Connect to WiFi
  Serial.print(s+"Connecting to WiFi: "+ssid+" ");
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println(s+" connected with IP: "+WiFi.localIP());

  // Connect to MQTT
  Serial.print(s+"Connecting to MQTT: "+MQTT_SERVER+" ... ");
  if (client.connect("HomeAlert")) {
    Serial.println("connected");
    digitalWrite(BUILTIN_LED, HIGH); // turn LED off

    mqtt.subscribe("home/alarm", topic1_subscriber);
    mqtt.subscribe("alarm/light", topic2_subscriber);
  } else {
    Serial.println(s+"failed, rc="+client.state());
  }

}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("HomeAlert")) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      client.publish("outTopic","garageControl Reconnect");
      mqtt.subscribe("home/alarm", topic1_subscriber);
      mqtt.subscribe("alarm/light", topic2_subscriber);
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" Resetting");
      // Wait 5 seconds before retrying
    }
  }
}

void loop() {
  if (!client.connected()) {
    reconnect();
  }

  client.loop();
}


void topic1_subscriber(String topic, String message) {
  Serial.println(s+"Message arrived in function 1 ["+topic+"] "+message);
  if (message.equals("on"))
  {
    tone(D5, 1000, 0);
  } else {
    noTone(D5);
  }
}

void topic2_subscriber(String topic, String message) {
  Serial.println(s+"Message arrived in function 1 ["+topic+"] "+message);
  if (message.equals("on"))
  {
    digitalWrite(BUILTIN_LED, LOW);   // Turn the LED on (Note that LOW is the voltage level
  }
  else
  {
    digitalWrite(BUILTIN_LED, HIGH);
  }
}



For Arduino programmers, this should look familiar. There are some definitions at the top, including the SSID, password for the WiFi network. The IP address of the MQTT server is also there. The global defines are next, where the wifiClient is defined, along with pubsub client definitions:


PubSubClient client(MQTT_SERVER, 1883, espClient);
PubSubClientTools mqtt(client);

The client is defined to use the MQTT_SERVER ip address, on port 1883 (normal MQTT server port). and will use the espClient WiFi client. The mqtt tools will use this client definition. I define a String s, but it is only used for debugging, and could be removed. 

Then the normal Arduino Setup function is defined. This function completes the setup. 
It defines the debugging serial port baud rate, and sets the pinModes to "output" for the built in LED and the in that the tone generator is connected to. Looking at the NodeMCU pinout, D5 is next to a ground so it makes a very easy interface, requiring only a 2 pin header (see picture above). The NodeMCU has constants for the labeled pins, so it isn't necessary to translate Arduino GPIOs to NodeMCU pins in your code. 

The WiFi connection is attempted until a connection is made, with a half second between tries. Lots of debug print when successful. The build in LED will be turned on when the NodeMCU is powered up, I turn the LED off once the WiFi connection is established. This helps when the node is away from a debug terminal. 

The MQTT client is used to connect next. Many examples aren't clear, but each node MUST have a unique name. If two nodes have the same name, the MQTT server will be confused and not know where to send messages to. Once the connection to the MQTT agent is established, subscriptions can be made. The topics are subscribed to along with the callback methods that should be invoked with a message on that topic is made available. This node will allow the LED to be controlled and the tone will be sounded. 

There is a reconnect method that will be nearly identical to the connect bits in the setup() function. The reconnect will connect if the client looses the connection, like with the MQTT server is rebooted. The main loop() method will check for a connection, and if not there, will call reconnect(). After the connection is restored, then the client.loop() function is called to see if any messages have arrived on subscribed topics. 

If a message arrives on any of the subscribed topics, one of the two callback methods is called. The topic1 messages will turn the sound on or off. The topic 2 messages will turn the LED on. The message that the callbacks check is the payload of the message. This node has simple payloads, only "on" or "off". Other nodes could have more sophisticated payloads. Even this node could have different frequencies for the tone as the payload to make a simple alert more useful (low tone for window open, higher tone for the garage open). 




This will build in the Arduino IDE using the "LOLIN(WEMOS) D1 R2 & mini" board setting in the Arduino IDE.  

It may not be the most efficient, but to run this node, I have a simple wall wart, and USB cable connected to the microUSB port on the board. It runs no trouble from a 500ma USB wall wart. Larger nodes may need more current. 

Next post, I'll start talking about using the MQTT server and how to control and hear from the nodes.