Mit MQTT einen Roboter steuern - [Teil 1]

Most blog posts are about simpler controls or regulations that a microcontroller handles. But the IoT devices, short for Internet of Things, can do much more. The basis for this is provided by protocols that ensure communication to the individual devices, such as the MQTT used here. This first post of the blog series about MQTT teaches the basics. At the end of the blog series, a small robot is controlled using MQTT, similar to a remote control.

Hardware

You only need a few components for this blog post, see Table 1.

Quantitiy Component
1 Raspberry Pi
1 Matching power supply

Table 1: Required hardware

With the Pi, keep in mind that in addition to the hardware mentioned above, you'll also need a microSD card. For this you should install Raspberry Pi OS (formerly Raspbian) as an image on the card.

Hint at this point

This tutorial covers MQTT and how to set up your own MQTT broker and clients. Therefore, only for this part you need a Raspberry Pi to follow and copy all the steps shown here. In the further parts of the blog series more clients will be integrated, like the microcontroller board compatible with Arduino Uno R3 and NodeMCU's.

What is MQTT?

MQTT is the abbreviation for Message Queuing Telemetry Transport and is also known under the older name WebSphere MQTT, WSMQTT for short. It is an open network protocol for machine-to-machine communication, e.g. to transmit sensor data and/or to control actuators. The background of this protocol is that a transmission of information can be guaranteed even in so-called unstable networks. The term "unstable networks" originates from the time when networks were still very susceptible to interference.

MQTT, if installed by default, is accessible via network port 1883. The second port 8883 is the secured network port, but must be encrypted using the TLS protocol. In order to use MQTT, you need an MQTT server, the so-called broker, as well as the corresponding end devices that send or receive the data, also called clients. Usually the broker is found on a Linux-based operating system. However, Windows operating systems are also possible. Meanwhile, MQTT can be found in automation technology and IoT projects, since it is relatively easy to set up an MQTT network with few resources.

The protocol version 3.1.1 used here, as well as the previous version 3.1, were published in 2010 under a free license, which was specified by the OASIS standardization committee. This protocol version is therefore already over 10 years old at the time the blog was published. The latest protocol version 5.0 was published in 2019 and at the same time specified by the OASIS standardization committee.

How does MQTT work

MQTT uses the event-driven publish/subscribe architecture. There is no end-to-end connection, as for example with a website, but a server, the so-called broker, to which senders and receivers, the so-called clients, connect. In order to better understand these connections, Figure 1 illustrates this in a simplified form.

Figure 1: Illustration of the MQTT architecture

A client, in this case the sensor, the mobile device or the IoT device, can always send and/or receive data; the broker merely serves as an information buffer that forwards the data to other clients. In order for the client to receive a corresponding message from the broker, the client must subscribe to the so-called topics when the connection is established or during operation at the broker.

You can think of a Topic as a kind of URL (Uniform Resource Locator) that reflects the desired content. A topic for a temperature sensor in the living room could publish the measured values on Home/Ground floor/Living room/Temperature. To be clear right away, it is not an address to the sensor, but allows access to the last transmitted sensor data. When the reading is updated, the broker checks which client has subscribed to receive this information and forwards it.

Data exchange with the broker

As mentioned at the beginning, MQTT was intended for unstable connections, which is why there are three different service qualities, Quality of Service or QoS. With level 1 or QoS = 0, the message is sent exactly once. An acknowledgement, as is the case with the other QoS, is completely ignored. Only the publishing, the so-called PUBLISH, is important.

With QoS = 1, one also speaks of "delivered at least once", which means that the sender waits for an acknowledgement from the receiver. This acknowledgement is called PUBACK and is obligatory for the remote station, otherwise the sender transmits until an acknowledgement is received. This leads in reverse to the fact that a message is transmitted several times to the receiver.

QoS = 2 is the combination of QoS = 0 and QoS = 1, whereby it also represents the slowest transmission. Here the message is sent exactly only once. To achieve this, a two-stage confirmation of receipt is necessary.

First, the sender sends (PUBLISH) its message to the recipient, which accepts the message and sends a confirmation message (PUBREC), with the content from the sender attached as a copy. If this confirmation message (PUBREC) is received by the sender, the latter saves the information and also replies with a confirmation message (PUBREL).

Finally, when the recipient has received the PUBREL, this sends the last confirmation message (PUBCOMP). At the end of such a transfer of data, it is ensured that the message has actually arrived. To keep it more understandable, shows
Figure 2 once again schematically how the message flow looks with all three QoS variants.

Figure 2: MQTT QoS overview

Subscribe to data from the broker

Now the question arises as to how a client can now query sensor data? There are again several variants that can lead to the goal. In the first variant, as already shown above for the simple sensor data transfer, you can specify the topic completely. To do this, simply enter the topic Home/Ground floor/Living room/Temperature. However, you will then only receive exactly this one value.

When using variant two, the + operator comes into play. With Home/+/+/Temperature you get the temperature data for all rooms on each floor. Useful I this if you want to predefine such a structure for all sensors in their house in such a way and display e.g. bundled.

Variant three goes one step further with the # operator. By Home / Ground Floor / # get all the information available for each room on the ground floor. Depending on the quantity of information, a lot can come together at this point.

The fourth and last case is probably the most data-intensive topic subscription, if you send a lot of data. With /# you subscribe to the root directory and every change is sent to you. I do not recommend this variant at this point! You should think in advance about your structure or the measurement data you want to include and define it beforehand.

Finally, the question probably arises, what happens if a transmitter suddenly stops transmitting? For this purpose, there is the so-called retained message in MQTT, which contains the last will and testament. In order for this to be used, the sender, in our case a sensor, must provide this retained message when connecting to the broker. This stores what is to be sent if the sender can no longer be reached. An individual content can then be written by the broker for each topic. Since MQTT already provides monitoring by default, you do not need to worry about this point. Since it goes too deeply into the matter, I can give at this point however gladly the keyword LWT message, with which you will find enough reading material in the Internet.

The Raspberry Pi becomes a broker

As mentioned in the “What is MQTT?” Section, this protocol is very popular in the IoT scene. Many shy away from using their personal data or data to control their home on the Internet, for example mqtt-dashboard.com/ save. Therefore, it makes sense to set up a small, local broker with the Raspberry Pi. At first it does not matter which Raspberry Pi version you are using and whether it is already taking on any tasks.

First of all, open the terminal of your Raspberry Pi, see Figure 3 and then issue the commands Code 1 a.

Figure 3: Open Terminal in Pi

sudo apt upgrade

sudo apt dist-upgrade

Code 1: update Raspbian

With this you update the Raspberry Pi and all installed packages. Then issue the command Code 2 a. This will load the popular mosquitto broker and the client application onto your Raspberry Pi. Confirm the installation in the terminal at the appropriate point.

sudo apt install mosquitto mosquitto clients

Code 2: Install Mosquitto server and client

Use the command so that the broker also starts immediately after a restart Code 3.

sudo systemctl enable mosquitto.service

Code 3: Start the mosquitto service when the Pi is restarted

Since the daemon may not yet run on the Pi, start it with the command Code 4.

sudo mosquitto -d

Code 4: start mosquitto

From this point you have set up a local, unencrypted MQTT broker in your home network. If you have not set up port forwarding to the Raspberry Pi via your router, this cannot be reached on the Internet. But it should be enough for private use. So that the broker remains accessible from the Pi after a restart, you should assign the Raspberry Pi a fixed IP address via the router.

Send and receive data to the broker

Now it's time to put theory into practice and send messages to the broker and also receive messages from the broker. For this the terminal commands "mosquitto_sub" for the subscribe function and "mosquitto_pub" for the publish message function are needed.

The terminal command "mosquitto_sub" will start, and the most important commands are explained below. If you want to go beyond the commands shown here, I recommend the internally supplied Linux manual at this point, which you can call up in the terminal with the command “man mosquitto_sub”, “man mosquitto_pub” or “man mosquitto”. The Linux command man stands for manual, in German manual or manual.

To see in detail what the mosquitto subscriber is processing in the background, and also to receive all messages, use the command Code 5.

mosquitto_sub -d -q 2 -t / #

Code 5: Let Mosquitto subscribers listen for root nodes

The one shown here Code 5 uses the options Table 2.

option Explanation
-d Activates debug mode and displays all useful information.
-q Specifies which QoS the subscriber should communicate with the broker. QoS 2 is used in Code 5.
-t Specifies which topic the subscriber is to subscribe to. In Code 5, the subscriber subscribes to all topics.

Table 2: Options of the mosquitto subscriber

Got the command Code 5 is entered, the output will first indicate the successful connection to the broker, see Figure 4 Mark 1.

Figure 4: Terminal output from moquitto_sub

In marker 1 of Figure 4 you can see that a connection is being established and the remote station, the broker, has accepted the connection. Immediately afterwards, the subscriber transmits which topic is to be subscribed to, here "/ #", and which QoS was requested, here "QoS: 2". Finally, the subscribe request is made using SUBACK, the Subscription Acknowledged, accepted. The connection to the broker is established and the desired topic is subscribed to. 

Mark 2 of Figure 4 will appear in your terminal after a while. As described in "How does MQTT work", this is the query as to whether the subscriber is still online. The broker asks whether the subscriber is still online, the so-called Ping Request. Normally, the client sends the corresponding response, the so-called Ping Response, to the Broke.

The next step, now that we have a subscriber, is to use a publisher to send data to the broker. The first message should of course be "Hello world". The publisher sends this small sentence to the topic / test / testpub.

To do this, open a second terminal window and issue the command accordingly Code 6 a . 

mosquitto_pub -d -q 2 -t / test / testpub -m "Hello world"

Code 6: mosquitto_pub "Hello world"

The options used for Code 6 are in Table 3 listed.

option Explanation
-d Activates debug mode and displays all useful information.
-q Specifies which QoS the publisher should communicate with the broker. QoS 2 is used in Code 6.
-t Specifies the topic to which the data should be sent, here / test / testpub
-m Send the following message, here "Hello world"

Table 3: Options of the mosquitto publisher

Let's take a look at what data is being transmitted, see Figure 5 Mark 1, sent to the broker. As before, QoS 2 was selected in the options, which is why you can also see the corresponding communication flow.

Figure 5: Terminal output from moquitto_pub and mosquitto_sub

Immediately afterwards the broker sends the new message to the subscriber, see Figure 5 Mark 2, because he has subscribed to all topics. On the upper edge of marker 2 of Figure 5 you can see that new data has been received in the / test / testpub topic. The QoS 2 handshake is displayed directly behind it, with the message "Hello world" received.

You have now learned the basics of MQTT and the technical background to MQTT. Feel free to play with the commands once and send data to the broker and, if necessary, open other terminals that subscribe to other topics.

In the next part of the blog series you will connect an Arduino microcontroller or ESP32 to the MQTT and send data, and the remote station will react accordingly.

This and other projects can be found on GitHub at https://github.com/M3taKn1ght/Blog-Repo

Projekte für anfängerRaspberry pi

7 comments

Ruediger

Ruediger

@Jörn: Ein paar Informationen gibt es hier: https://www.facebook.com/flyingthewinglets

Jörn Weise

Jörn Weise

Hallo Herr Kress,
sofern Sie fertige Bibliotheken benutzen, müssen Sie sich um das Quittieren kann Sorgen machen. Bauen Sie aber eine eigene Bibliothel oder Funktion die MQTT übernehmen soll, müssen Sie natürlich daran denken. Ich empfehle hier ganz klar die fertigen Bibliotheken, da spart man sich die Zeit und den Streß.

Hallo Lothar, Egon,
Danke für das Feedback, freut mich als Autor und als Blogger, wenn Beiträge gut ankommen.

Hi jm1704,
thanks for the feedback. Funny part is, that I will show people MQTT.fx in part two of this blog serie. I’m also using this tool for my normal buissnes to test and check signals.

Hallo Ruediger,
das hört sich nach einem sehr spannenden Projekt an. Das interessiert mich wahnsinnig, gibt es dazu ggf. eine Website, wo man diesen Status sehen kann? Zu der Thematik mit den Controllern muss ich gestehen, hatte ich bisher noch keine Aussetzer. Egal welchen ich verwendet habe, hatte ich eine zuverlässige Verbindung zum Wifi. Sicher das an irgendeiner Stelle nicht der verbaute Watchdog vom ESP32 oder ESP8266 zuschlägt? Ich hatte das bei ein paar Tests mal, da hatte ich eine Endlosschleife und habe erst später bemerkt, dass hier der Watchdog immer meinen MicroController neu gestartet hatte.

Gruß / Greetings
Weise

jm1704

jm1704

Good MQTT article with MOSQUITTO
For peoples who want to control or simulate MQTT transaction and check the critical or not and when problem was.
I am also using 2 tools MQTT.fx and MQTT Explorer

Ruediger

Ruediger

Ich betreibe privat einen B737 Flugsimulator und habe vor 1 Jahr damit begonnen Teile der Steuerung auf WiFi und MQTT umzustellen. Das funktioniert soweit recht gut, verwende es aber derzeit noch nicht für kritische Signale. Meine Erfahrung ist, das die WiFi Verbindung immer mal wieder zusammenbricht. Diese wird von dem betroffenen Client zwar wieder aufgebaut, dauert aber 3-5 sek. Während dieser Zeit kann viel passieren. Deshalb mein Hinweis mit MQTT bei kritischen Verbindungen vorsichtig zu sein. Woran es liegt, dass die Wifi Verbindung hin und wieder zusammenbricht, kann ich so noch nicht sagen. Nur soweit, dass ich mit dem ESP8266 ESP-01S noch keine Probleme hatte. Wohl aber mit dem ESP WROOM 32. Eigentlich hätte ich es eher anders herum erwartet. Ingesamt betreibe ich derzeit 4 ESP8266 ESP-01S Clienets und 3 ESP WROOM 32 Clients.

Egon

Egon

Kurz und knapp, sehr verständlich erklärt. Macht weiter so. Warte schon auf den nächsten Beitrag.

Lothar

Lothar

Vielen Dank für das tolle Tutorial zum Thema MQTT. Bin schon sehr gespannt, wie weiter geht.

E. Kress

E. Kress

Es wäre gut zu wissen, ob man als Progeammierer bei QoS 2 selbst für die Quittung-Tätigkeiten sorgen muss oder ob das im MQTT-System automatisch geschieht.

Leave a comment

All comments are moderated before being published