Low Cost Wireless JMRI Signal boards with the ESP8266 and MQTT

I want to use this space to present some work I have been doing, along with Gert "Speed" Muller, on using low cost ESP8266 WiFi microcontrollers to control signaling.

In February on the Arduini Yahoo group I got a little carried away when someone suggested that the European MERG boards were the best solution for CAN based signaling. My experience with MERG 15+ years ago was not great and after seeing a couple consecutive posts where buying boards from MERG was the solution given, I challenged the well meant suggestions in a less than tactful manner. I only mention this because that flame-war was the starting point of this project.

John Plocher mentioned using MQTT with a WeMos D2 Mini R2 board to control signals over WiFi using tried and true TCP protocols. While I had heard of affordable WiFi chips becoming popular recently, I never considered using MQTT to control signals, so there was a big learning curve.

MQTT is an Internet of Things (IoT) connectivity protocol. The IoT buzzword is super hot right now and there is a lot of interest in anything that has to do with connecting things to the Internet. Any time there is a lot of excitement around some technology, there is usually some way to apply it to model railroading.

This isn't meant to be an introduction to MQTT, there are thousands of resources where you can get up to speed on that. I suggest going to Youtube and searching for MQTT Tutorial.

The first thing I had to do was determine if JMRI can be made to understand the MQTT protocol. I knew JMRI has the ability to run scripts and a number of examples are included in every version. Because JMRI is written in Java, the creators cleverly decided to include the Jython interpreter with JMRI. Jython allows Python scripts to be run natively in the Java environment and use the JMRI libraries written in Java. This is amazingly powerful and useful in a way that is probably not obvious at first glance. Some basics of Python scripting in JMRI can be found at JMRI: The Python/Jython language.

The MQTT library for Python is called Paho and is from Eclipse. I set an initial goal of getting the library to load without errors, which turned out to be harder than I expected. Here are the steps I had to take to get there.

Many of these steps involve the command line and editing files. I will add a prompt before the command and use a Courier font. $ for MacOS, > for Windows and % for Linux. If text is added or edited, I will use blue.

Starting with MacOS.

1. Download latest jython installer from http://www.jython.org/downloads.html
2. $ java -jar jython-installer-2.7.0.jar to install jython in ~/jython2.7.0
3. $ ~/jython2.7.0/bin/pip install paho-mqtt
4. Copy /Volumes/Macintosh\HD/Applications/JMRI/python.properties to ~/Library/Preferences/JMRI
5. Add the following to ~/Library/Preferences/JMRI/python.properties python.path=~/jython2.7.0/Lib/site-packages                                                            python.startup =
6. Change lines in ~/jython2.7.0/Lib/site-packages/paho/mqtt/client.py file from listensock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_IP) to listensock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) as discussed here.
7. Add import paho.mqtt.client as mqtt to the top of a generic python script
8. Run script and look for errors in console
9. After script runs, check that the python.path is correct under Help -> Context...

In Windows.

1. Download latest jython installer from http://www.jython.org/downloads.html
2. > java -jar jython-installer-2.7.0.jar to install jython in C:\jython2.7.0
3. > C:\jython2.7.0/bin/pip.exe install paho-mqtt
4. Copy C:\Program Files (X86)\JMRI\python.properties to C:\Users\username\JMRI\
5. Add the following to C:\Users\username\JMRI\python.properties (had to escape out backslashes and dash with JMRI version 4.7.2 )
6. Change lines in C:\jython2.7.0\Lib\site-packages\paho\mqtt\client.py file from listensock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_IP) to listensock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) as discussed here.
7. Add import paho.mqtt.client as mqtt to the top of a generic python script
8. Run script and look for errors in console
9. After script runs, check that the python.path is correct under Help -> Context...

In Ubuntu 16.04.

1. Download jython-installer-2.7.0.jar
2. % java -jar jython-installer-2.7.0.jar and install jython in ~/jython2.7.0
3. % ~/jython2.7.0/bin/pip install paho-mqtt
4. Copy <JMRI_FOLDER>/python.properties to ~/.jmri/
5. Add the following to ~/.jmri/python.properties python.path=~/jython2.7.0/Lib/site-packages                                                            python.startup =
6. Add import paho.mqtt.client as mqtt to the top of a generic python script
7. Run script and look for errors in console
8. After script runs, check that the python.path is correct under Help -> Context...

So, the key to these instructions is that you have to install Jython. then you have to install Paho so Jython can find it. The reason for editing the python.properties is described here, at first it wasn't right, but I posted a note to the JMRI Yahoo group and it was updated. Thank you to the editor. Basically, you have to install the full version of Jython that can have libraries added to it. The standalone version that is shipped with JMRI can't be extended like this.

I found a bug pop up in the paho library, luckily some Google foo found a solution as noted above. We'll get more into the Python code for handlers, etc., later but any of the examples in the Python Examples directory can be used to test that you have installed everything correctly.

Next time we'll talk about the ESP8266.