MQTT with QoS levels in Raspberry Pi - Deep Dive

Overview - MQTT with QoS levels

What is it?

MQTT is a simple messaging protocol used to send data between devices, especially in small computers like Raspberry Pi. QoS stands for Quality of Service, which controls how messages are delivered to make sure they arrive correctly. There are three levels of QoS that decide how reliable the message delivery is. This helps devices communicate smoothly even if the network is not perfect.

Why it matters

Without QoS levels, messages could get lost or duplicated without any control, causing confusion or errors in connected devices. QoS ensures that important messages reach their destination reliably, which is critical for things like home automation or sensor data. It makes communication trustworthy, so devices can act on accurate information.

Where it fits

Before learning MQTT with QoS, you should understand basic networking and how MQTT works as a publish-subscribe protocol. After this, you can explore MQTT security, advanced MQTT features, or how to build IoT applications using MQTT on Raspberry Pi.

Mental Model

Core Idea

MQTT QoS levels are like delivery options that control how carefully messages are sent and confirmed between devices.

Think of it like...

Imagine sending letters by mail: QoS 0 is like dropping a postcard in the mailbox without tracking, QoS 1 is like sending a letter with delivery confirmation, and QoS 2 is like sending a registered letter that requires a signature to confirm receipt.

┌─────────────┐       ┌─────────────┐
│ Publisher   │──────▶│ MQTT Broker │
└─────────────┘       └─────────────┘
       │                    │
       │ QoS 0: Fire & Forget
       │ QoS 1: At least once
       │ QoS 2: Exactly once
       ▼                    ▼
┌─────────────┐       ┌─────────────┐
│ Subscriber  │◀──────│ MQTT Broker │
└─────────────┘       └─────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding MQTT Basics

Concept: Learn what MQTT is and how it works as a messaging protocol.

MQTT is a lightweight protocol where devices called publishers send messages to a broker, which then forwards them to subscribers. It uses topics to organize messages, like channels. This setup is great for small devices like Raspberry Pi because it uses little power and bandwidth.

Result

You know how devices send and receive messages using MQTT topics.

Understanding MQTT's publish-subscribe model is key to grasping how QoS controls message delivery.

2

Foundation

Introducing QoS Concept

Concept: Learn what Quality of Service means in MQTT and why it matters.

QoS in MQTT defines how the broker and devices handle message delivery. It helps decide if messages can be lost, duplicated, or must be delivered exactly once. This is important because networks can be unreliable, and devices need to trust the data they get.

Result

You understand that QoS controls message reliability in MQTT.

Knowing QoS exists prepares you to choose the right delivery level for your application.

3

Intermediate

Exploring QoS Level 0: Fire and Forget

🤔Before reading on: Do you think QoS 0 guarantees message delivery or not? Commit to your answer.

Concept: QoS 0 means the message is sent once without any confirmation or retry.

When a publisher sends a message with QoS 0, it just sends it and does not wait for any acknowledgment. The broker tries to deliver it, but if the message is lost, it won't be resent. This is the fastest but least reliable method.

Result

Messages may be lost but are sent with minimal delay and overhead.

Understanding QoS 0 helps you see the trade-off between speed and reliability.

4

Intermediate

Understanding QoS Level 1: At Least Once Delivery

🤔Before reading on: Does QoS 1 allow duplicate messages or not? Commit to your answer.

Concept: QoS 1 ensures the message is delivered at least once by requiring acknowledgment and retry.

With QoS 1, the publisher sends a message and waits for a PUBACK from the broker. If no acknowledgment arrives, it resends the message. This guarantees delivery but can cause duplicates if the acknowledgment is delayed or lost.

Result

Messages are reliably delivered but may appear more than once.

Knowing QoS 1's behavior helps you handle possible duplicate messages in your application.

5

Intermediate

Mastering QoS Level 2: Exactly Once Delivery

🤔Before reading on: Do you think QoS 2 is slower than QoS 1? Commit to your answer.

Concept: QoS 2 uses a four-step handshake to ensure messages are delivered exactly once without duplicates.

This level involves a complex exchange: PUBLISH, PUBREC, PUBREL, and PUBCOMP messages between publisher and broker. This process confirms the message is received and processed only once, avoiding duplicates but adding delay.

Result

Messages arrive exactly once, with no loss or duplication, but with more overhead.

Understanding QoS 2 reveals how MQTT balances reliability and performance for critical data.

6

Advanced

Implementing QoS in Raspberry Pi MQTT Clients

🤔Before reading on: Do you think setting QoS affects only the publisher or both publisher and subscriber? Commit to your answer.

Concept: Learn how to set QoS levels in MQTT clients on Raspberry Pi and how both publisher and subscriber QoS interact.

In MQTT libraries for Raspberry Pi, you specify QoS when publishing and subscribing. The effective QoS is the lower of the two. For example, if publisher sends with QoS 2 but subscriber subscribes with QoS 1, the message is delivered at QoS 1. This ensures compatibility and resource efficiency.

Result

You can control message delivery reliability in your Raspberry Pi projects.

Knowing how publisher and subscriber QoS combine prevents unexpected message delivery behavior.

7

Expert

QoS Impact on Network and System Resources

🤔Before reading on: Does higher QoS always mean better performance? Commit to your answer.

Concept: Explore how QoS levels affect network traffic, latency, and resource use on devices like Raspberry Pi.

Higher QoS levels require more message exchanges and storage for acknowledgments, increasing network load and CPU usage. On constrained devices, this can slow down other tasks or drain battery faster. Choosing the right QoS balances reliability with system performance.

Result

You understand trade-offs between message reliability and system efficiency.

Recognizing QoS resource costs helps design efficient, reliable IoT systems.

Under the Hood

MQTT QoS works by adding acknowledgment messages and message IDs to track delivery status. QoS 0 sends messages without tracking. QoS 1 uses a PUBACK to confirm receipt, resending if missing. QoS 2 uses a four-step handshake (PUBLISH, PUBREC, PUBREL, PUBCOMP) to ensure exactly one delivery. The broker and clients keep state to manage these exchanges, storing messages temporarily until confirmed.

Why designed this way?

MQTT was designed for unreliable networks and low-power devices, so QoS levels provide flexible reliability options. The simple QoS 0 suits fast, non-critical data. QoS 1 and 2 add complexity for reliability, but only when needed. This design balances performance and trustworthiness, avoiding overhead when unnecessary.

┌───────────────┐
│ Publisher     │
│ (Sends Msg)   │
└──────┬────────┘
       │ PUBLISH
       ▼
┌───────────────┐
│ MQTT Broker   │
│ (Tracks Msg)  │
└──────┬────────┘
       │ PUBACK (QoS1)
       │ PUBREC (QoS2)
       ▼
┌───────────────┐
│ Subscriber   │
│ (Receives Msg)│
└───────────────┘

QoS 0: Single PUBLISH, no ACK
QoS 1: PUBLISH → PUBACK
QoS 2: PUBLISH → PUBREC → PUBREL → PUBCOMP

Myth Busters - 4 Common Misconceptions

Quick: Does QoS 1 guarantee no duplicate messages? Commit yes or no.

Common Belief: QoS 1 guarantees messages are delivered only once, no duplicates.

Reality: QoS 1 guarantees delivery at least once, but duplicates can occur if acknowledgments are lost.

Why it matters: Assuming no duplicates can cause errors if your application processes the same message multiple times.

Quick: Is QoS 2 always the best choice for all messages? Commit yes or no.

Common Belief: Using QoS 2 is always better because it ensures perfect delivery.

Reality: QoS 2 adds overhead and latency, so it is best reserved for critical messages, not all traffic.

Why it matters: Using QoS 2 unnecessarily can slow down your system and waste resources.

Quick: Does the subscriber's QoS setting override the publisher's? Commit yes or no.

Common Belief: The subscriber's QoS setting controls how messages are delivered regardless of the publisher's QoS.

Reality: The effective QoS is the lower of publisher and subscriber QoS settings.

Why it matters: Misunderstanding this can lead to unexpected message delivery quality.

Quick: Does QoS 0 mean the message is never lost? Commit yes or no.

Common Belief: QoS 0 messages are always delivered because they are sent immediately.

Reality: QoS 0 messages can be lost if the network fails or the broker is down.

Why it matters: Relying on QoS 0 for important data risks losing critical information.

Expert Zone

1

QoS 2's four-step handshake can cause message delays that affect real-time systems if overused.

2

Some MQTT brokers optimize QoS handling internally, affecting performance and message ordering subtly.

3

Retained messages interact with QoS levels, influencing how new subscribers receive past messages.

When NOT to use

Avoid QoS 2 for high-frequency sensor data where latency matters; use QoS 0 or 1 instead. For unreliable networks, consider MQTT over TCP with TLS or alternative protocols like CoAP for constrained devices.

Production Patterns

In production, QoS 1 is commonly used for most messages to balance reliability and performance. QoS 2 is reserved for critical commands like device firmware updates. QoS 0 is used for telemetry data where occasional loss is acceptable.

Connections

TCP/IP Protocol

MQTT QoS levels build on TCP's reliable transport but add application-level guarantees.

Understanding TCP helps grasp why MQTT still needs QoS to handle message delivery at the application layer.

Postal Mail Delivery

MQTT QoS levels mirror postal service options from untracked postcards to registered mail.

This connection clarifies how different delivery guarantees affect speed, cost, and reliability.

Distributed Systems Consensus

QoS 2's handshake resembles consensus protocols ensuring exactly-once operations.

Knowing consensus algorithms helps understand the complexity and necessity of QoS 2's multi-step confirmation.

Common Pitfalls

#1 Using QoS 2 for all messages without considering system load.

Wrong approach:

client.publish('sensor/data', payload, qos=2)

Correct approach:

client.publish('sensor/data', payload, qos=1)

Root cause:

Believing maximum reliability is always best without weighing performance costs.

#2 Ignoring subscriber QoS when setting publisher QoS.

Wrong approach:

publisher sends with qos=2 but subscriber subscribes with qos=0 expecting QoS 2 delivery.

Correct approach:

Ensure subscriber subscribes with qos=2 or qos=1 to match publisher's QoS.

Root cause:

Misunderstanding that effective QoS is the minimum of publisher and subscriber settings.

#3 Assuming QoS 0 messages are never lost.

Wrong approach:

Using qos=0 for critical alerts expecting guaranteed delivery.

Correct approach:

Use qos=1 or qos=2 for critical alerts to ensure delivery.

Root cause:

Not recognizing that QoS 0 offers no delivery guarantee.

Key Takeaways

MQTT QoS levels control how reliably messages are delivered between devices, balancing speed and trust.

QoS 0 sends messages once without confirmation, risking loss but offering speed.

QoS 1 guarantees delivery at least once but can cause duplicates, requiring careful handling.

QoS 2 ensures exactly one delivery through a multi-step handshake, adding overhead.

Choosing the right QoS level depends on your application's need for reliability versus performance.