NAV
cURL Arduino Python Circuitpython Ruby

Adafruit IO MQTT API

MQTT, or message queue telemetry transport, is a protocol for device communication that Adafruit IO supports. Using a MQTT library or client you can publish and subscribe to a feed to send and receive feed data.

If you aren't familiar with MQTT check out this introduction from the HiveMQ blog. All of the subsequent posts in the MQTT essentials series are great and worth reading too.

Client Libraries

To use the MQTT API that Adafruit IO exposes you'll need a MQTT client library. For Python, Ruby, and Arduino you can use Adafruit's IO libraries as they include support for MQTT. For other languages or platforms look for a MQTT library that ideally supports the MQTT 3.1.1 protocol.

MQTT Connection Details

We strongly recommend connecting using SSL (Port 8883) if your client allows it. Port 443 is for MQTT-over-Websockets clients which generally run in browsers, like Eclipse Paho, HiveMQ Websockets, or MQTTJS.
Host io.adafruit.com
Secure (SSL) Port 8883
Insecure Port 1883
**MQTT over Websocket 443
Username Your Adafruit IO Username
Password Your Adafruit IO Key

Need to manually set a Client ID for your MQTT client? Use a unique value such as a random GUID. Whenever possible, leave the client ID field blank.

We do not currently limit the number of clients that can connect to Adafruit IO, but we do limit the rate at which connections may be attempted to 20 per minute. Exceeding the connection attempt rate limit will result in a temporary ban.

MQTT QoS

One feature of MQTT is the ability to specify a QoS, or quality of service, level when publishing feed data. This allows an application to confirm that its data has been sucessfully published.

If you aren't familiar with MQTT QoS levels be sure to read this great blog post explaining their meaning.

For publishing feed values the Adafruit IO MQTT API supports QoS level 0 (at most once) and 1 (at least once) only.

QoS level 2 (exactly once) is not currently supported.

MQTT API Rate Limiting

Adafruit IO's MQTT server imposes a rate limit to prevent excessive load on the service. If a user performs too many publish actions in a short period of time then some of the requests will be rejected and an error message will be published on your /throttle topic. The current rate limit is at most 30 requests per minute for free accounts, 60 per minute with an IO+ account, and expandable via Adafruit IO+ Boost applied to your account.

We also limit a few other actions on Adafruit IO's MQTT broker. If you send, within a minute, more than: 100 MQTT SUBSCRIBE requests, 10 failed MQTT SUBSCRIBE requests, or 10 failed MQTT PUBLISH requests; or you send enough messages after passing the rate limit; or you attempt to log in more than 20 times within a minute, your account will be temporarily banned from the MQTT broker.

Most of the systems connecting to Adafruit IO are simple, automated devices and MQTT client scripts. A runaway ESP8266 sending MQTT SUBSCRIBE packets inside the main loop() function of an Arduino sketch can send about 500 packets per minute. It wouldn't take many poorly written clients connecting to our MQTT broker to crash the service for everyone if we didn't add rate limits to every action.

If you exceed a rate limit, a notice will be sent to the (username)/throttle topic. If your account is temporarily banned, a notice will be sent to the {username}/errors topic. While developing your project, you should always keep subscriptions to the error topics open so you can see when you're getting close to the Adafruit IO rate limits and when you've been blocked.

This limit applies to all connections so if you have multiple devices or clients publishing data be sure to delay their updates enough that the total rate is below 2 requests/second.

Rate Limit Bans

Temporary MQTT bans start at 30 seconds and scale as a multiple of the number of bans within the last hour up to one hour maximum. For example, 10 rejected MQTT SUBSCRIBE requests will trigger a temporary ban of 30 seconds, that's 1 ban. If it's a device that automatically wakes up, connects to Adafruit IO, and sends a stream of rejected SUBSCRIBE packets, it will be banned again. The second ban results in 60 seconds offline, the third is 90 seconds, and so on.

You can visit your Adafruit IO monitor page any time

Preventing MQTT Bans

A misbehaving MQTT device--for example, a device configured for a different MQTT broker like Home Assistant--that attempts to publish to invalid Adafruit IO MQTT topics can lock every device and browser session out of your Adafruit IO MQTT account for an hour at a time.

The easiest way to prevent a temporary or permanent MQTT block being put against your account is to delay how frequently your MQTT client sends messages.

The second thing to check is that you are not using a different Internet of Things software toolkit to talk to Adafruit IO. The Adafruit IO MQTT broker

MQTT Data Format

There are a few ways to send data to our MQTT API if you're writing your own client library.

The simplest way to send values to an IO Feed topic is to just send the value. For example, a temperature sensor is going to produce numeric values like 22.587. If you're sending to mosfet/feeds/photocell-one you can use the raw number or a string. That means either 22.587 or 22.587 will be accepted as a numeric value. Adafruit IO does its best to treat data as numeric values so that we can show you your data as a chart on an Adafruit IO dashboard and through our Charting API.

Sending data with location

To tag your data with a location value, you'll either need to wrap it in a JSON object first or send it to the special /csv formatted MQTT topic.

Sending JSON

JSON can be sent to either the base topic or the /json topic - for example mosfet/feeds/photocell-one or mosfet/feeds/photocell-one/json. The proper format for location tagged JSON data is:

Example JSON topic object:

{
  "value": 22.587,
  "lat": 38.1123,
  "lon": -91.2325,
  "ele": 112
}

Specifically, JSON objects must include a value key, and may include lat, lon, and ele keys.

Sending CSV

Alternatively, you can send location tagged data to /csv topics. In this example, that would be the topic mosfet/feeds/photocell-one/csv instead of mosfet/feeds/photocell-one. Both store data in the same feed. The format IO expects for location tagged CSV data is VALUE, LATITUDE, LONGITUDE, ELEVATION.

With the example data shown before, that means you could publish the string "22.587,38.1123,-91.2325,112" to mosfet/feeds/photocell-one/csv. to store the value "22.587" in the location latitude: 38.1123, longitude: -91.2325, elevation: 112.

An example is displayed below, which uses a simple Ruby MQTT library and the data shown, all these examples publish the same data to the same feed.

# first you'll need https://github.com/njh/ruby-mqtt
require 'mqtt'

username = 'test_username'
key      = 'not-a-real-key'
url      = "mqtts://#{ username }:#{ key }@io.adafruit.com"

mqtt_client = MQTT::Client.connect(url, 8883)

# simplest thing that could possibly work
mqtt_client.publish('test_username/feeds/example', 22.587)

# sending numbers as strings is fine, IO stores all data internally
# as strings anyways
mqtt_client.publish('test_username/feeds/example', '22.587')

# CSV formatted, no location
mqtt_client.publish('test_username/feeds/example/csv', '22.587')

# CSV formatted, with location
mqtt_client.publish('test_username/feeds/example/csv',
                  '22.587,38.1123,-91.2325,112')

# JSON formatted, no location
mqtt_client.publish('test_username/feeds/example', '{"value":22.587}')
mqtt_client.publish('test_username/feeds/example/json', '{"value":22.587}')

# JSON formatted, with location
mqtt_client.publish('test_username/feeds/example',
'{"value":22.587,"lat":38.1123,"lon":-91.2325,"ele":112}')
mqtt_client.publish('test_username/feeds/example/json',
'{"value":22.587,"lat":38.1123,"lon":-91.2325,"ele":112}')

Sending JSON Data through Adafruit IO

Because Adafruit IO supports additional features beyond a basic MQTT brokering service, such as location tagging for data points, the service supports data in the JSON format described above. Namely, the example JSON response on the sidebar.

JSON Response Format Example

  {
    "value": 22.587,
    "lat": 38.1123,
    "lon": -91.2325,
    "ele": 112
  }

This lets us store the individual value, 22.587, and data about the value: its latitude, longitude, and elevation. Metadata!

But what happens when the value you want to send is itself JSON? Good news! There are a few solutions available to you in that situation.

IO-Formatted JSON

The simplest way to send JSON data to Adafruit IO is to wrap it in the format described above. For example, if instead of 22.587, I wanted to send something like, {"sensor-1":22.587,"sensor-2":13.182}, the "wrapped" version would look like this:

Example of IO-Formatted JSON

{
  "value": {"sensor-1":22.587,"sensor-2":13.182},
  "lat": 38.1123,
  "lon": -91.2325,
  "ele": 112
}

It's worth noting that because Adafruit IO parses the entire JSON object that you send it, any valid JSON will be parsed and when it is stored in our system and forwarded to any subscribers, it will be regenerated. The significance of that is that if you publish JSON data with whitespace, it will be stored and republished without whitespace, because our generator produces the most compact JSON format possible.

Double-Encoded JSON Strings

The second way you can send JSON data as a value is to "double encode" it before sending, in which case IO will treat it as a raw string. If you're using something like javascript's JSON.stringify function or Ruby's JSON.generate, double encoding means passing the result of JSON.stringify through JSON.stringify a second time. In this node.js console example, you can see the difference:

Here's an example of sending double-encoded strings through Adafruit IO:

> JSON.stringify({"sensor-1":22.587,"sensor-2":13.182})
'{"sensor-1":22.587,"sensor-2":13.182}'
> JSON.stringify(JSON.stringify({"sensor-1":22.587,"sensor-2":13.182}))
'"{\"sensor-1\":22.587,\"sensor-2\":13.182}"'

The double encoded JSON string can be sent directly through Adafruit IO without interference from our processing system, because the processing system will not interpret it as JSON. In your receiving code, because the value passed through includes surrounding double quotes, you have to call your parse function twice to restore the JSON object.

Here's an example of interpreting double-encoded JSON Strings sent through Adafruit IO:

> var input = '"{\\\"sensor-1\\\":22.587,\\\"sensor-2\\\":13.182}"'
> JSON.parse(JSON.parse(input))
{ 'sensor-1': 22.587, 'sensor-2': 13.182 }

Non-IO Formatted JSON

The third way you can send raw JSON data is to just send it. If Adafruit IO doesn't find a "value" key in the JSON object you send, it will treat the whole blob as plain text and store and forward the data. That means with our example JSON object, sending the string {"sensor-1":22.587,"sensor-2":13.182} will result in {"sensor-1":22.587,"sensor-2":13.182} being stored in IO and sent to MQTT subscribers.

Retained values

MQTT is a tremendously useful protocol for building small connected devices and is relatively simple to understand and implement (if implementing networking protocols is your thing). Unfortunately, a few features of the Adafruit IO platform make it difficult for us to support the entire MQTT 3.1+ protocol specification in our broker. Specifically, one particular feature, the publish retain flag.

MQTT Retain

In the MQTT protocol, setting the retain flag on a published message asks the MQTT broker (server) to store that message. Then any new clients which connect and subscribe to that topic will immediately receive the retained message. Retain makes writing basic MQTT-only Internet of Things clients easier, without it, a client that connects and subscribes to a feed topic has to wait until a new value is published on the feed to know what state it should be in. In the case of slowly updated feeds, there could be hours between updates which means a device that disconnects and reconnects (for example, due to power cycling or sleeping) might lose the current value for a long time between updates.

Adafruit IO's Limitations

Among other factors, our scale, Adafruit IO's mix of MQTT & HTTP APIs, the speed at which we’re taking in new data, and the fact that we’re already storing almost every message that is sent mean that a “simple” feature like retain becomes difficult to support without making MQTT service performance worse for everyone.

Since we don’t actually store data in the broker but at a lower level and can’t support PUBLISH retain directly, we’re proposing a different solution for the retaining problem: the /get topic modifier.

Using the */get topic

For any given Adafruit IO MQTT feed or group, subscribe to the appropriate topic using the feed or group key, then add /get to the topic you subscribed to and publish anything to that new topic (our Arduino library uses the null character: \0). IO will immediately publish, just for that client, the most recent value received on the feed.

For example, let's imagine we have a device that subscribes to a counter feed: uname/f/counter. If we want to get the latest value, the /get topic we should publish to is uname/f/counter/get. After connecting to IO, subscribing to uname/f/counter, and publishing to uname/f/counter/get, we will immediately receive a message on our uname/f/counter subscription with the last value that was published to counter.

If you’re using the Adafruit IO Arduino library, you can add /get support to your project using the following code snippet:

// ... from the adafruitio_01_subscribe example sketch
AdafruitIO_Feed *counter = io.feed("counter");

void setup() {
  // 1. start IO connection
  io.connect();

  // 2. prepare MQTT subscription with handler function
  counter->onMessage(handleMessage);

  // 3. wait for successful connection
  while(io.mqttStatus() < AIO_CONNECTED) {
    delay(500);
  }

  // 4. send /get message, requesting last value, triggering
  //    the handleMessage handler function
  counter->get(); // ask Adafruit IO to resend the last value
}

You can also perform a /get using the Adafruit IO CircuitPython library in one line of code:

python io.get('temperature')

MQTT Topics

Feed Topic Format

Adafruit IO's MQTT API exposes feed data using special topics. You can publish a new value for a feed to its topic, or you can subscribe to a feed's topic to be notified when the feed has a new value.

Group Topic Format

Small Topic Format

If you're using an embedded system and need to conserve space, use the /f/ or /g/ topic formats:

Using a Wildcard

You can also subscribe to the parent 'feeds' path to be notified when any owned feed changes using MQTT's # wildcard character. For example, the user could subscribe to either:

Once subscribed to the path above any change to a feed owned by Your_Adafruit_IO_Username will be sent to the MQTT client. The topic will specify the feed that was updated, and the payload will have the new value.

Be aware the MQTT server sends feed updates on all possible paths for a specific feed. For example, subscribing to IO-Username/f/# and publishing to IO-Username/f/photocell-one would produce messages from: IO-Username/f/photocell-one, IO-Username/f/photocell-one/json, and IO-Username/f/photocell-one/csv; each referring to the same updated value. To reduce noise, make sure to grab the specific topic of the feed / format you're interested in and change your subscription to that.

If you'd like to avoid the formatted feeds (/json and /csv topics) but still see all the feeds you're publishing to, you can use MQTT's + wildcard in place of #. In this case, subscribing to IO-Username/f/+ would produce output on IO-Username/f/photocell-one, but not IO-Username/f/photocell-one/json.

Group Topics

You aren't limited to just Feed based MQTT topics, Adafruit IO supports grouped feeds as well. Similar to our HTTP group data creation API, you can publish to multiple feeds or subscribe to multiple feeds through a single MQTT topic.

The topics formats for publishing or subscribing are:

Publish to Feeds in a Group

If you use the /json or /csv endings on your group MQTT topic, your data should be formatted in JSON or CSV, respectively. By default, IO expects published values to be in JSON format.

The JSON and CSV format IO expects are displayed below:

JSON Expected Response from Adafruit IO

{
  "feeds": {
    "key-1": "value 1",
    "key-2": "value 2",
    "key-3": "value 3"
  },
  "location": {
    "lat": 0.0,
    "lon": 0.0,
    "ele": 0.0
  }
}

CSV Format Expected Response from Adafruit IO

key-1,value 1
key-2,value 2
key-3,value 3
location,0.0,0.0,0.0

Group Guidelines

Subscribing to groups

If you use the /json or /csv endings when subscribing to your group MQTT topic, your data will be formatted in JSON or CSV, respectively. By default, IO publishes values in JSON format.

The formats your subscription will receive are the same as the formats IO expects to receive. Expected responses for JSON and CSV are below.

Expected Group JSON Response

{
  "feeds": {
    "key-1": "value 1",
    "key-2": "value 2",
    "key-3": "value 3"
  },
  "location": {
    "lat": 1.0,
    "lon": 2.0,
    "ele": 3.0
  }
}

Expected Group CSV Response

key-1,value 1
key-2,value 2
key-3,value 3
location,1.0,2.0,3.0

It's important to note that you will only receive updated values for the feeds that received new values. That means if you're subscribed to {username}/groups/example and publish to {username}/feeds/key-1, the subscription will receive:

{
  "feeds": {
    "key-1": "value 1"
  }
}

It's also worth noting that JSON subscription formats will always receive string type values, regardless of whether a string or number was published.

Time Topics

Adafruit IO provides some built-in MQTT topics for getting the current server time. The current available topics are:

Most of the Adafruit IO client libraries provide helper functions for easy use of these topics:

time/seconds

This topic publishes the current time in Unix epoch seconds.

The current Unix epoch time at the time of writing is: 1562164754.

time/millis

This topic publishes the same thing as time/seconds every second, but with slightly higher precision and more digits.

The current Unix epoch time in milliseconds at the time of writing is: 1562165081839

time/ISO-8601

This topic publishes the current time in ISO 8601 UTC format. That's year-month-dayThour:minute:second.millisecondsZ where "-", ":", "T", and "Z" are character literals / always included in the time string.

The current time in ISO 8601 UTC format at the time of writing is: 2019-07-03T14:47:16.038Z

time/hours

This topic publishes the current hour of the day in 24 hour format (the values 0 - 23) in the UTC time zone. It only publishes once every hour.

Randomizer Topics

Create random data streams that can generate colors, words, numbers, or pick a value from a custom data set and send it directly to your devices.

Prior to subcribing to the randomizer topic, you need to generate the topics to get the id.

Subscribe to the Generated Topic

MQTT random word subscriptions will publish data once per minute to every client that is subscribed to the same topic.

Weather Topics (IO+)

With access to weather data powered by Dark Sky, you can get hyper-local forecasts sent directly to your devices. Weather data is updated at most once every 20 minutes.

When you have an IO Plus account, you can track the weather through our HTTP or MQTT APIs for up to 5 locations at a time.

Prior to subscribing to the weather topic, you'll need to generate the topics to get the id.

Subscribe to the Generated Topic

Where :id is the ID of the weather record you want data for and :type is the kind of forecast data you want. Valid types are:

MQTT weather subscriptions will publish updated data once per minute to every client that is subscribed to a valid topic.

Error Topics

Adafruit IO provides two error reporting MQTT topics you can subscribe to:

{username}/errors

This topic publishes error messages related to publishing and subscribing to Adafruit IO feeds. Because MQTT is an asynchronous and compact messaging format, we cannot respond to invalid PUBLISH or SUBSCRIBE packets with a comprehensive error message, so we use this topic to update subscribers when an error occurs.

You may only subscribe to your personal errors feed.

{username}/throttle

This topic publishes throttle warning and error messages to all subscribed clients. We do our best to provide an accurate accounting in the error message of the minimum amount of time you should wait before attempting to publish again.

Be warned: exceeding the rate limit too often within a short time span will result in a temporary ban from the Adafruit IO MQTT service.

You may only subscribe to your personal throttle feed.

Adafruit IO Monitor

Both error feeds can be seen live on your Adafruit IO Monitor page.

Troubleshooting

What can you do when something is going wrong with your project? Adafruit IO provides a few ways for you to find out what's happening behind the scenes with your MQTT connected devices.

Understanding Error Messages

The two MQTT error topics produce alerts, warnings, and error messages when there's a problem with your MQTT data. MQTT error messages are not stored permanently, so if you want to see them you'll need to keep the monitor page open while developing your project.

We try to be descriptive enough that you can figure out what went wrong, but there are a few categories of error messages you may run into. We list an example message for each category of error, yours will look different but most are in the form:

CLIENT_ID IP_ADDRESS MQTT_ACTION MQTT_TOPIC ...message

Exceeded Rate Limit

On {username}/throttle:

When MQTT packets are arriving too quickly (SUBSCRIBE only, warning):

io-badstuff-23111 172.18.0.1 SUBSCRIBE time/seconds only 5 attempts remaining in the next 60 seconds, please slow your request rate

When publishing data too quickly (warning):

test_username data rate limit reached, 21 seconds until throttle released

On {username}/errors:

When MQTT packets are arriving too quickly (SUBSCRIBE only, error):

io-badstuff-23111 172.18.0.1 SUBSCRIBE time/seconds made too many requests, wait before reconnecting

After a certain number of throttle warning messages have been sent:

enforcement limit reached, your account is banned for 30 seconds

Invalid MQTT topic

There are several variations of invalid MQTT topic errors. All error messages will be published on the {username}/errors topic.

When making a PUBLISH or SUBSCRIBE request for an MQTT topic not recognized by Adafruit IO:

io-badstuff-12558 172.18.0.1 SUBSCRIBE LWT/$SYS/obe/pixel/adafruit.server/sessions-005 rejected, not a valid resource topic

Sometimes invalid MQTT topics or topics that don't fit the {username}/{type}/{key}/{format} pattern can look like a different error to IO. The most common is seeing an error message like this when PUBLISHING:

io-mqtt-149edf5e 172.18.0.1 PUBLISH ,,21,9,,,33,, OK -72.223332,12.100,0.17,28.1,1,, rejected, only shared feeds may be published to

Or:

io-mqtt-149edf5e 172.18.0.1 PUBLISH rejected, only shared feeds may be published to

In each case, IO is showing the topic your code specified in between "PUBLISH" and "rejected." In the first example, the given topic was ,,21,9,,,33,, OK -72.223332,12.100,0.17,28.1,1,, and in the second example, the given topic was an empty string, . Both examples probably mean there was an error in the sketch that caused the MQTT topic to become corrupted in memory before publishing data.

A good idea if you're getting any error messages when publishing or subscribing from a new sketch is to print out the MQTT topic you're using to the serial console (Serial.println in Arduino, print() in CircuitPython) in your sketch before you publish or subscribe just to make sure the data you're sending is the data you think you're sending.

The meaning of the error message, "rejected, only shared feeds may be published to", is that IO saw a username other than the one you authenticated with in the {username} part of an MQTT topic and something other than "feeds" or "f" in the {type} part of an MQTT topic. Because the topic part of the MQTT packet IO received was, ,,21,9,,,33,, OK -72.223332,12.100,0.17,28.1,1,, IO couldn't find a username or a type in the string, so it had to reject the PUBLISH message.

MQTT action flood

On {username}/errors:

When subscribing or publishing to resources faster than IO can verify that you have permission to access them:

io-badstuff-94864 172.18.0.1 SUBSCRIBE time/seconds request flood, disconnecting

Resource not available

On {username}/errors:

When you don't have permission to access a shared resource:

io-badstuff-29349 172.18.0.1 SUBSCRIBE other_username/feeds/something.fake-11 rejected, resource not available or not authorized

Action attempted while banned

On {username}/errors:

When attempting to publish or subscribe after your account has been temporarily banned:

io-mqtt-0d63acfb 172.18.0.1 PUBLISH test_username/feeds/example.counter-3 rejected, user is temporarily blocked

Writing Safe MQTT Code

Always. Use. Delay. Make sure there is time in between messages to the Adafruit IO MQTT broker. If you're saving data in an Arduino or CircuitPython sketch inside the program's main loop, make sure you're delaying, sleeping, or using some other technique to put space in between messages.

Use current libraries. Adafruit provides complete API client libraries for Arduino, Python, Ruby, and Javascript. When in doubt, use our code to build your next project.

Use the least data possible. It is always better to send data from a sensor infrequently and slowly increase the frequency as needed rather than sending as quickly as possible from the beginning. An old device or project that you forgot about can easily take down your whole account when you go to add something new. Also, when dealing with sensors like temperature, outdoor light level, or wind speed, reality doesn't change very fast so there's no need for your data to change fast either.

Never reuse Internet of Things projects from other systems without making sure it conforms to Adafruit IO's requirements. Code that works with Home Assistant, Node Red, or any other open source MQTT broker is likely to use MQTT topic conventions that will not work with Adafruit IO. Plugging Adafruit IO's broker credentials into an example project from another system is a quick way to get your account locked by a runaway process.

Ask questions on the forums or Discord! There's a community of people who may have already run into the same problem you're having. Search the forums, ask a question, stop by and see what people are working on. Whatever approach you take, you being involved in the Adafruit community can make the difference for someone else. You can find the forums here: https://forums.adafruit.com/viewforum.php?f=56 and sign up for Adafruit's Discord chat server here: https://discord.gg/adafruit

cURL Arduino Python Circuitpython Ruby