MQTT and Graphite With mqtt2graphite

For the past few years I have been operating a small number of Arduino Ethernets with DHT22 sensors, push data into Graphite. They have worked reasonably well, but had one major issue. Graphite required that the data flow include the Unix Epoch time and the Arduino time counter over flows every ~50 days.

The graphite data needs to look like this:

topic value epochtime


house.temp.crawlspace 17.600000 1590181123

So with one Arduino, I would just occasionally unplug the device and then on startup it would sync with the local NTP server and use millis() to figure out how long it had been since that initial sync. Sure, I had ideas of ways to work around this, like setting up the watchdog to reset the Arduino automatically, but that still has problems. My goal was to eleminate the need for the Arduino to know what time it was at all. Initially I was going to write a small proxy to handle adding the time into the data stream. This would give me a small project to practice my C skills and learn about network sockets. As this was percolating in the back of my mind I learned about MQTT at a talk by Jan-Piet Mens gave at BSDCan 2019.

As I started researching more things I could do with MQTT, I also learned about this fantastic HomeBridge Plugin called MQTT-Thing. This convinced me it was time to setup an MQTT broker and start connecting various things to it.

For the past few months I have been using JP’s mqtt2graphite to take the data from MQTT and shovel it into Graphite. Earlier this month I performed and upgrade and I noticed that data stopped flowing. After trying a frustrating day beating my head trying to figure out why data was not coming out of mqtt2graphite, I noticed with tcpdump that it was not even putting any data on the wire. Which eventually lead me to add debugging around the sendto() call. This lead to me rediscovering a big flaw in the Paho MQTT library for Python, and the worst part is, it is a flaw I have run into before! Low and behold it is a small world, I find a issue raised by JP against the Paho MQTT Python git repo about exceptions not being raised in callbacks #365.

So I used try/except to see what was going on and modified the code to exit when an error with the socket happens. Somewhere along the Python 3.x train sendto() was changed to require bytes and will not accept a string. So I have made the required changes and would send a pull request over to JP, but I noticed he has archived the repo for mqtt2graphite, but I hope others will find my changes to be useful. The repo is here:

So thanks JP for the work you have put into mqtt2graphite and for teaching me about MQTT in the first place!