Inventing MQTT: building a protocol from the ground up

By Arlen Nipper, Co-creator of MQTT and CTO of Cirrus Link Solutions

Just joining us? Read part 1 and part 2 of the MQTT story.

In that small office at Phillips 66, Andy Stanford-Clark and I began the process of designing what would eventually become MQTT. We weren’t looking to create a standard. We were just trying to solve a real-world problem under some very tight constraints.

The constraints that shaped MQTT

Steve Konig explained that the VSAT bandwidth costs were so high, every byte of overhead was significant — thousands of dollars per year. And their SCADA system, using poll/response, was saturating the entire space segment just to function.

We needed to start fresh. MQSeries was too bloated. We had to design something from scratch.

Here were the original MQTT design constraints:

  1. Simple – It had to run on 8-bit microcontrollers with just 64KB of RAM.
  2. Efficient – Every byte counted. We had to reduce bandwidth as much as possible.
  3. Bidirectional – Operators needed to monitor and control remote equipment.
  4. Stateful – The system had to support a Report by Exception (RBE) model.

These became the pillars of the protocol.

Leveraging TCP/IP

One of the smartest decisions we made early on was to lean on TCP/IP instead of reinventing the wheel.

TCP/IP already handled:

  • Error correction
  • In-order delivery
  • Reliable data transport

There was some debate about using UDP (User Datagram Protocol), but in the end, we realized we’d end up writing a poor imitation of TCP. So we embraced what was already proven.

Topics and payloads

We built MQTT around the concept of a Topic and Payload:

  • Topic – A slash-delimited UTF-8 string (e.g. Pipeline1/Booster1/PLC1) that provided routing and context
  • Payload – A binary blob, up to 256MB, that could contain anything

Instead of locking payloads into XML or JSON, we kept it flexible. That made MQTT incredibly adaptable—especially for industrial systems needing compact, efficient binary data structures.

In hindsight, this was one of MQTT’s biggest advantages. You could shape the protocol around your application, not the other way around.

The architecture takes shape

With these building blocks, we defined two core components:

  • MQTT clients – Devices in the field and host systems that publish or subscribe to topics
  • MQTT broker – The central server that receives, filters, and routes messages between clients

From there, we started defining the verbs, the actual commands the broker would recognize, to make the publish/subscribe system work.

We were building a lightweight message bus for industrial systems, decades before that term became popular.

In the final installment: how MQTT challenged the status quo, why Report by Exception was so disruptive, and how a side project to solve one company’s problem became a global standard.