IoT Projects & Protocolsintermediateadiy labs
MQTT Protocol Explained: ESP32 Publish & Subscribe Tutorial with Real Examples
MQTT explained for IoT: ESP32 publish and subscribe with PubSubClient, Mosquitto broker setup, topics and QoS, with real examples.
By ADIY11 min read
Technical review pending. Figures marked “verify” or “check your part” have not been confirmed against your exact hardware revision. Check the manufacturer datasheet before relying on them in a design.
1. Introduction: What is MQTT and Why It Dominates IoT#
If you are building IoT projects in India using the ESP32, ESP8266, or Arduino boards, you have likely come across the term MQTT. Created by IBM in 1999 for monitoring oil pipelines via satellite, MQTT (Message Queuing Telemetry Transport) has evolved into the de facto standard for Internet of Things (IoT) communication.
MQTT is an extremely lightweight, publish-subscribe network protocol that transports messages between devices. Unlike HTTP, which requires a heavy request/response cycle, MQTT maintains a continuous, low-bandwidth connection. This makes it perfect for constrained environments like microcontrollers and unreliable mobile networks.
In this comprehensive tutorial, we will explain the MQTT protocol from the ground up, set up a local broker, and write practical code for both the ESP32 (C++) and the ADIY FLY Pico RP2040 (MicroPython).
2. MQTT vs HTTP vs WebSocket#
When starting with IoT, beginners often use HTTP GET/POST requests. Let's look at why MQTT is generally the superior choice for real-time sensor data and control applications.
| Feature | MQTT | HTTP | WebSocket |
|---|---|---|---|
| Architecture | Publish/Subscribe via Broker | Client/Server (Request/Response) | Client/Server (Bi-directional) |
| Header Overhead | Tiny (as small as 2 bytes) | Large (can be hundreds of bytes) | Medium (varies by frame) |
| Connection Type | Persistent, Stateful | Stateless (TCP setup overhead per request) | Persistent, Stateful |
| QoS (Quality of Service) | Yes (0, 1, 2) | No native support | No native support |
| IoT Suitability | Excellent (Built for constrained devices) | Poor for real-time telemetry | Good for web dashboards, heavy for MCUs |
3. Core MQTT Concepts#
To use MQTT effectively, you must understand its topology. The architecture is decoupled—devices never talk directly to each other.
- The Broker: The central hub/server. It receives all messages, filters them, and distributes them to interested clients. It acts like a post office.
- The Client: Any device (ESP32, Raspberry Pi, smartphone, cloud dashboard) that connects to the broker. Clients can be Publishers, Subscribers, or both.
- Topic: A UTF-8 string that acts as a routing key for the broker. It represents the "channel" that messages are published to.
- Publish: Sending a data payload to the broker under a specific Topic.
- Subscribe: Telling the broker you want to receive all messages sent to a specific Topic.
- Retained Messages: A flag indicating the broker should save the last message on a topic and send it immediately to any new subscriber.
- LWT (Last Will and Testament): A message specified by a client upon connection. If the client disconnects ungracefully (e.g., power loss), the broker publishes this message to notify others.
4. Topic Naming Best Practices#
Topics use a hierarchical structure separated by forward slashes (/), much like a file system path. Designing a good topic tree is crucial for scalable IoT systems.
Good Practice Structure: project/location/device/sensor
Examples:
home/livingroom/esp32_1/temperaturefactory/mumbai/machine_A/vibrationfarm/greenhouse/relay/pump1/command
MQTT also supports Wildcards for subscribers (never for publishers):
- Single Level (
+):home/+/temperaturematcheshome/bedroom/temperatureandhome/kitchen/temperature. - Multi Level (
#):home/bedroom/#matches everything under bedroom (temperature, humidity, lights). Must be at the end of the topic.
5. Setting Up Mosquitto Broker (Local Network)#
Eclipse Mosquitto is a lightweight, open-source MQTT broker written in C. Running it locally on your PC or Raspberry Pi is the best way to develop without relying on cloud internet access.
For Windows:#
- Download the installer from mosquitto.org.
- Run the installer and install it as a Windows Service.
- Open Command Prompt as Administrator and navigate to the install directory (usually
C:\Program Files\mosquitto). - Run the broker:
mosquitto -v(verbose mode to see connections).
For Linux / Raspberry Pi:#
# Update package list
sudo apt update
# Install mosquitto and client tools
sudo apt install mosquitto mosquitto-clients
# Enable it to start on boot
sudo systemctl enable mosquitto
# Start the service
sudo systemctl start mosquitto6. Using Free Public Brokers for Testing#
If you don't want to host your own broker initially, you can use public cloud brokers. Note: Never send sensitive or personal data over public unencrypted brokers! Anyone can subscribe to your topics.
- Eclipse:
mqtt.eclipseprojects.io(Port 1883) - Mosquitto:
test.mosquitto.org(Port 1883) - HiveMQ:
broker.hivemq.com(Port 1883)
7. Installing PubSubClient in Arduino IDE#
For ESP32/ESP8266 boards programmed via the Arduino IDE, the PubSubClient by Nick O'Leary is the standard library.
- Open Arduino IDE.
- Go to Sketch > Include Library > Manage Libraries.
- Search for PubSubClient.
- Install the one authored by Nick O'Leary.
8. Code 1: ESP32 MQTT Publisher (DHT22 Sensor)#
Let's write a program for an ESP32 to connect to Wi-Fi, connect to our MQTT broker, and publish temperature data from a DHT22 sensor every 5 seconds.
// ESP32 MQTT Publisher Example
#include <WiFi.h>
#include <PubSubClient.h>
#include <DHT.h>
// Network Credentials
const char* ssid = "YOUR_WIFI_SSID";
const char* password = "YOUR_WIFI_PASSWORD";
// MQTT Broker details
const char* mqtt_server = "192.168.1.100"; // Replace with your Broker IP
const int mqtt_port = 1883;
// DHT Sensor setup
#define DHTPIN 4
#define DHTTYPE DHT22
DHT dht(DHTPIN, DHTTYPE);
WiFiClient espClient;
PubSubClient client(espClient);
unsigned long lastMsg = 0;
void setup_wifi() {
delay(10);
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("\nWiFi connected. IP: ");
Serial.println(WiFi.localIP());
}
void reconnect() {
while (!client.connected()) {
Serial.print("Attempting MQTT connection...");
// Client ID must be unique
String clientId = "ESP32Client-" + String(random(0xffff), HEX);
if (client.connect(clientId.c_str())) {
Serial.println("connected");
} else {
Serial.print("failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 5 seconds");
delay(5000);
}
}
}
void setup() {
Serial.begin(115200);
setup_wifi();
client.setServer(mqtt_server, mqtt_port);
dht.begin();
}
void loop() {
if (!client.connected()) {
reconnect();
}
client.loop(); // Maintain connection and check for incoming messages
unsigned long now = millis();
if (now - lastMsg > 5000) { // Publish every 5 seconds
lastMsg = now;
float t = dht.readTemperature();
if (!isnan(t)) {
char tempString[8];
dtostrf(t, 1, 2, tempString);
Serial.print("Publishing temperature: ");
Serial.println(tempString);
// Publish to topic 'adiy/office/temp'
client.publish("adiy/office/temp", tempString);
}
}
}9. Code 2: ESP32 MQTT Subscriber (Relay Control)#
Now, let's create a subscriber. This ESP32 will listen to the topic adiy/office/light. If it receives a "ON", it turns on a relay (or built-in LED). If "OFF", it turns it off.
// ESP32 MQTT Subscriber Example
#include <WiFi.h>
#include <PubSubClient.h>
const char* ssid = "YOUR_WIFI_SSID";
const char* password = "YOUR_WIFI_PASSWORD";
const char* mqtt_server = "192.168.1.100";
const int relayPin = 2; // Onboard LED or Relay PIN
WiFiClient espClient;
PubSubClient client(espClient);
// The callback function handles incoming messages
void callback(char* topic, byte* payload, unsigned int length) {
Serial.print("Message arrived on topic: ");
Serial.print(topic);
Serial.print(". Message: ");
String messageTemp;
for (int i = 0; i < length; i++) {
messageTemp += (char)payload[i];
}
Serial.println(messageTemp);
if (String(topic) == "adiy/office/light") {
if (messageTemp == "ON") {
Serial.println("Turning Light ON");
digitalWrite(relayPin, HIGH);
}
else if (messageTemp == "OFF") {
Serial.println("Turning Light OFF");
digitalWrite(relayPin, LOW);
}
}
}
// ... setup_wifi() and reconnect() are same as previous example ...
// Make sure to add this inside reconnect():
// client.subscribe("adiy/office/light");
void setup() {
Serial.begin(115200);
pinMode(relayPin, OUTPUT);
setup_wifi();
client.setServer(mqtt_server, 1883);
client.setCallback(callback); // Register callback!
}
void loop() {
if (!client.connected()) {
reconnect();
}
client.loop();
}10. Code 3: Building an MQTT Dashboard with Node-RED#
Node-RED is a brilliant visual programming tool. You can install it on your PC or Raspberry Pi to act as a dashboard.
- Install Node-RED and start it (
node-redin terminal). - Go to
http://localhost:1880. - Install the
node-red-dashboardpalette via the Menu > Manage Palette. - Drag an mqtt in node. Set the server to your broker IP (
localhostif on same machine) and topic toadiy/office/temp. - Drag a dashboard gauge node. Connect it to the mqtt in node.
- Drag a dashboard switch node. Configure it to output "ON" for true and "OFF" for false.
- Drag an mqtt out node. Connect the switch to it, set topic to
adiy/office/light. - Click Deploy and visit
http://localhost:1880/uito see your dashboard!
11. Code 4: ADIY FLY Pico RP2040 with MQTT (MicroPython)#
If you prefer Python, our ADIY FLY Pico RP2040 (which features onboard Wi-Fi via ESP8285 or similar network setups depending on the variant) is an excellent choice. Here is how you use umqtt.simple.
# MicroPython on Raspberry Pi Pico W / ADIY FLY Pico
import network
import time
from umqtt.simple import MQTTClient
from machine import Pin
ssid = "YOUR_WIFI_SSID"
password = "YOUR_WIFI_PASSWORD"
mqtt_server = "192.168.1.100"
# Connect to Wi-Fi
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect(ssid, password)
while not wlan.isconnected():
time.sleep(1)
print("Connected to Wi-Fi")
# MQTT Callback
def sub_cb(topic, msg):
print(f"Received {msg} on {topic}")
# Setup MQTT
client = MQTTClient("PicoClient", mqtt_server)
client.set_callback(sub_cb)
client.connect()
client.subscribe(b"adiy/office/temp")
print("Connected to MQTT Broker, waiting for messages...")
try:
while True:
client.wait_msg() # Blocks until a message is received
finally:
client.disconnect()12. QoS Levels Explained with Practical Examples#
MQTT guarantees message delivery through Quality of Service (QoS) levels.
- QoS 0 (At most once): Fire and forget. No acknowledgment is expected. Used for high-frequency sensor data where losing one reading (e.g., temperature) doesn't matter.
- QoS 1 (At least once): Guaranteed delivery, but duplicates can occur. The broker acknowledges the message. Used for non-critical alerts (e.g., motion detected).
- QoS 2 (Exactly once): Safest but slowest. Uses a 4-step handshake. Used for critical commands like financial transactions or unlocking a smart door.
Note: PubSubClient library for Arduino currently only fully supports QoS 0 and QoS 1.
13. MQTT Security Basics#
Security is paramount in IoT. A bare MQTT connection sends payload in plaintext.
- Authentication: Always configure your broker to require a username and password. In PubSubClient, you pass them during connection:
client.connect(clientId, mqttUser, mqttPassword). - Encryption (TLS/SSL): Standard MQTT uses port 1883. Secure MQTT (MQTTS) uses port 8883. To use TLS on an ESP32, you must use
WiFiClientSecureinstead of standardWiFiClientand provide the broker's Root CA certificate. - Cloud platforms like HiveMQ Cloud or AWS IoT enforce TLS by default.
14. Troubleshooting Common Issues#
15. Frequently Asked Questions (FAQ)#
Why is MQTT better than HTTP for IoT? ▼
MQTT is designed for constrained devices and low-bandwidth, high-latency networks. It uses a publish/subscribe model with a tiny packet header (2 bytes), minimizing overhead compared to HTTP's request/response model and large headers. It also natively supports QoS (Quality of Service) levels.
Can I use MQTT without internet? ▼
Yes! You can set up a local MQTT broker like Mosquitto on a Raspberry Pi or a local PC on your local Wi-Fi network. Your ESP32 devices can communicate with each other over the local network without needing internet access.
What is the difference between QoS 0, 1, and 2? ▼
QoS 0 is "fire and forget" (message delivered at most once). QoS 1 guarantees delivery at least once (might result in duplicates). QoS 2 guarantees delivery exactly once (safest but highest overhead).
How do I secure my MQTT connection? ▼
At a minimum, you should use username and password authentication on your broker. For production environments, you must use TLS/SSL encryption (MQTTS on port 8883) via WiFiClientSecure to encrypt the data packets over the network, preventing interception.
Share
