New Feature: Feed Webhooks

What’s a webhook? According to Wikipedia:

A webhook in web development is a method of augmenting or altering the behaviour of a web page, or web application, with custom callbacks. These callbacks may be maintained, modified, and managed by third-party users and developers who may not necessarily be affiliated with the originating website or application.

For Adafruit IO, webhooks are unique URLs that you can use to send data to your feeds from anywhere on the internet, without having to share you IO API secret keys. They’re kind of like one-off API endpoints that you can manage separately from your security credentials and feed privacy.

There have long been requests for “shared write access” to feeds, which can be done using the feed sharing feature already built into IO, but that requires everyone sending data to have an Adafruit IO account and use their credentials to send data. That, unfortunately, limits you from, for example, creating a simple web form to publish data directly to a feed.

There are a few use cases we’ve come up with so far, but we hope to be surprised with what the community can come up with. Here’s two things you can try.

One-off IFTTT Applets

First, other web services that can publish to Webhook URLs like IFTTT’s maker webhooks can be used to directly send data to a feed without entering Adafruit credentials or signing into your Adafruit IO account through IFTTT.

We already support IFTTT directly, but we can’t support every service that exists on the web. We can work with any service that knows how to send webhook-style requests, though :D

Public write-only connections to IO feeds

Second, webhook URLs can be used directly in simple HTML forms, to create a deploy-anywhere application that sends data directly to your feed. For example:

<form target="_blank" action="https://io.adafruit.com/api/v2/webhooks/feed/dQTc61soU5S4tMv8jSuSjY9uSqVz" method="post">
    <label>
        Make a choice:
        <select name="value">
            <option value="YES" selected>Yes</option>
            <option value="NO">No</option>
        </select>
    </label>
    <button type="submit">Save my vote!</button>
</form>

And the actual form in action:

After clicking “Save my vote!” you should see a standard Adafruit IO data record on a new tab, which means your vote was recorded. If you see a “request failed - Throttle error” message instead, that’s because this particular webhook URL’s data rate limit of 3 events per minute has already been reached.

By setting a very low data rate for the webhook, I can share the URL publicly without risking my account, unlike sharing my Adafruit IO Keys. Even better, if the webhook ends up flooded with data, I can just delete it from my account to shut off the stream. And because webhook URLs are write-only, I don’t need to make the feed public.

Publicly controllable Christmas light displays? Crowd-sourced weather monitoring system? GitHub -> Discord -> Adafruit IO cross-posting webhook circle of fun? Let us know what you come up with. Join us in the forum or on Discord in the adafruit-io channel with questions, comments, or suggestions!

Adafruit IO knows the weather!

Or, more accurately, Dark Sky knows the weather and now Adafruit IO knows how to pass it on.

But wait, there’s more!

Adafruit IO ❤️’s Services

We’re excited to introduce you to Adafruit IO’s new “External Service Integration” feature. Also known as “Service Integrations”, also known as “Services”. You can find a link to the new Services page in the navigation sidebar.

The first thing you may notice if you’ve used If This Then That (IFTTT) with Adafruit IO is that we moved that service over to the new Services page.

We’ve also started building out a Zapier integration–currently in private beta–so if you prefer Zapier to IFTTT or you’d just like to keep your options open, you’ll find that on the services homepage also.

But the star of the show is the new Dark Sky-powered weather service.

Weather Data in Adafruit IO

We have wanted to add weather data to Adafruit IO for a long time, and even made an attempt more than two years ago. The biggest issues we faced are that: 1) free weather data services are inconsistent or have very low API limits and 2) even paid weather services often don’t include interesting weather data.

With Dark Sky, we were able to overcome the second problem, but the first still puts some limits on the service we’re able to offer. This means that at launch the weather service is also our first IO Plus exclusive feature.

When you create a new weather location, you have direct access to Dark Sky’s Developer API data through Adafruit IO with some minor limits. Specifically, we’re passing sending a subset of all available forecast data (~4KB of JSON) instead of all available forecast data (~30KB of JSON), and limiting MQTT subscriptions to just one type of data at a time. For example: just the current weather, 5 minute forecast, or 1 day forecast rather than all current and forecasted data with every update.

The Weather API is available via HTTP GET requests or MQTT subscriptions from any authenticated device. IO Plus subscribers can find full documentation for the weather service on the weather service home page.

This is just the first iteration of the Weather API and we’re very interested in hearing your feedback. If you’re an Adafruit IO Plus subscriber, give it a try and let us know what you think.

Random Data Service

Also available now, and to all Adafruit IO accounts, is our brand new Random Data Service! We’ve picked a few types of data (randomly, ha!) to generate and built the system in a way that allows separate Adafruit IO accounts to receive the same data based on the seed value used.

Once every minute, MQTT subscribers get a brand new, hand-crafted (kind of), guaranteed random value. You could build a pair of one time password devices, a color-cycling sculpture, a nonsense Tweeting robot, a dice roller / coin flipper / choose your own adventure pilot…? We don’t know! We just built the thing and hoping that you’ll find an interesting use for it.

We wrote a Ruby library to go along with it for generating word and color values and we’re 100% open to pull requests (celebrate Hacktoberfest with us!) or feature requests if you have an idea for new kinds of data that can be randomly generated.

We’re Just Getting Started

We’re going to be spending a lot of our time in the coming year connecting new services directly to Adafruit IO and improving our existing Triggers system. Keep checking back and please feel encouraged to get in touch with us on the Adafruit IO forum with any questions, concerns, or bug reports.

We’re excited to tell you about two new blocks we’ve added to Adafruit IO dashboards this week to help you build richer interfaces. Icons and Indicators, two features that have been requested a lot in the last few years.

Icon

Icon blocks take a feed value and display the appropriate icon, when the feed value is a string selected from the collection shown here.

click to download video

You can use Icon blocks as live changeable visual displays by sending different values to the feed, as in the example above, or as permanent labels to improve the display of your data. Drop a text block next to an icon block on your dashboard and drop some icons into the text block to see how they look.

Indicator

Indicator blocks let you pick an ON color and an OFF color and then describe which state the Indicator block should be in based on conditions you set. The block compares the selected feed’s current value to the given conditions and chooses a color accordingly.

You can use one condition or as many as you like. Conditions, by default, use Javascript’s parseFloat function to attempt to convert the feed’s current value and the condition values to a number for comparison. If either value, the feed or the string, can’t be converted to a number (isNaN(f_value) === true), then the condition is compared string-to-string.

For example, I could create an indicator for a temperature sensor that stays in the ON state as long as the value is between 70 and 78.

In Javascript, those conditions would be evaluated like this:

function compare(feed_value) {
  return feed_value >= 70 && feed_value <= 78;
}

Or I could create an indicator that turns on whenever my feed matches a particular value.

In this case, the code equivalent would be:

function compare(feed_value) {
  return feed_value === "ALPHA" ||
      feed_value === "BETA" ||
      feed_value === "GAMMA" ||
      feed_value === "BONANZA";
}

We don’t do any checking on your conditions to make sure they’re logically consistent. The conditions editor will happily let you enter impossible conditions (for example < 2 and > 2 or = 1 and = 0), so make sure to do your own testing before creating the block.

You may notice in the block editor screenshot above that we’ve given you access the “Test Value” that’s used in the block preview. That should help in testing your conditional logic to make sure the indicator turns on and off at the times you are expecting. Set your conditions and play with the value to see when they trigger. No data is sent to IO when you change the test value, so it’s safe to play with settings on any block.

BONUS! Drag and drop for image blocks

It’s now possible to drag and drop images onto any Image dashboard block to automatically publish the properly formatted Base64 image data string to your feed.

There are some important things to keep in mind when using this feature. Normal feeds are limited to 1KB of data, or about 1024 bytes, for publishing. Turning off feed history from the feed settings page allows publishing up to 100KB, or 102400 bytes, of data. Image conversion from binary to Base64 happens inside the browser, with no image pre-compression, and more importantly, conversion from binary to Base64 expands the size of the image data.

You’ll have to do your own testing to figure out what an appropriate image size and format (png, gif, or bmp) for you are. For example, the .png image used for testing below has an on disk size of 68089 bytes, but a Base64 value of 90788 bytes, an expansion factor of about 150%, which is really close to the limit.

test image

test publishing in block editor

One funny artifact of the drag and drop feature is that IO doesn’t actually care if you’re using the block to publish image data. Any file that can be converted into Base64 (which is any file) can be dropped onto an image block and sent to MQTT subscribers.

Let us know what you’re making!

Join us in the Adafruit IO forum or on Discord in the adafruit-io channel with questions, comments, or suggestions. We love seeing what you all come up with when we add new features to Adafruit IO!

New Feature Launch: Feed Sharing!

You’ve always been able to make your feeds completely public or completely private. Private feeds are hidden from the outside world, no other user can see them and no one browsing Adafruit IO can stumble across them. Public feeds are available as a read-only data source to anyone in the world. Until now, though, there hasn’t been a way to receive data from other Adafruit IO users in your feeds and no way to share your data with just one person.

Feed sharing introduces the ability to share a particular feed with a particular user as a read-only or read-write feed. This means you can keep a feed “private” as far as the whole world is concerned, but allow another Adafruit IO account to send data to it and read data from it.

There are a few new features we had to add to Adafruit IO to make sharing directly with other users possible. Here’s a quick guide to sharing and how to find your way around the new parts of Adafruit IO.

Sharing a feed

Step 1 is to invite another account to share your feed. You can do that from any existing feed page using the new “Sharing” form.

When you invite an account, the person you invite will receive a brief notification email and see the invitation on their shares page. If you share to an email address and they haven’t signed up yet, the person you shared with will have the chance to create a new Adafruit IO account.

If you share with an existing Adafruit IO user via the email address on their Adafruit account or their Adafruit IO username, they’ll be able to review and approve your invitation immediately. After you’ve shared, the user you’re sharing with will get an email:

And a link to the sharing invitation review page:

The sharing page

This is the hub for all your sharing information. Right now you’ll see listings for feeds you are sharing with other Adafruit IO users, and feeds other Adafruit IO users are sharing with you. If you’re blocking any users from inviting you to share with you, you’ll see those accounts listed on this page too.

As we add sharing features to Adafruit IO, this will be where they show up first.

Sharing feed writes

Since you’re able to share feeds as read/write data streams with other registered Adafruit IO users, data rate limiting has to come into the picture. For this new feature, all data rate usage comes from the publishing user. That means when you share your feed with another user and they publish data to it, your data rate usage will remain unchanged while theirs will be affected.

Related MQTT changes

In order to support in-browser MQTT subscriptions to other users’ shared, non-public feeds, we’ve made changes to the output of our */json MQTT topics. Because these topics aren’t used in any of our libraries and aren’t highlighted in our guides, this change will have very little impact on existing MQTT subscribers. Less than 0.4% of MQTT subscriptions will be affected.

Before, the output of a subscription to test_username/feeds/feed-a/json looked like this:

{
  "username": "test_username",
  "owner": {
    "id": 1,
    "username": "test_username"
  },
  "id": 4,
  "name": "Feed B",
  "description": null,
  "history": true,
  "unit_type": null,
  "unit_symbol": null,
  "last_value": "-23.908",
  "visibility": "private",
  "license": null,
  "created_at": "2018-05-21T20:52:17Z",
  "updated_at": "2018-07-26T18:38:00Z",
  "status_notify": false,
  "status_timeout": 60,
  "enabled": true,
  "key": "feed-b",
  "groups": [
    {
      "id": 1,
      "key": "default",
      "name": "Default",
      "user_id": 1
    }
  ]
}

That’s the same JSON record you would get if you used the HTTP API to retrieve /api/v2/test_username/feeds/feed-b, which is helpful if you want information about the feed, but kind of unhelpful if you actually want the new feed data record that prompted the MQTT message.

After the launch of feed sharing, the same MQTT subscription output looks like this:

{
  "last_value": "-23.908",
  "updated_at": "2018-07-26 18:38:00 UTC",
  "key": "feed-b",
  "data": {
    "created_at": "2018-07-26T18:38:00.560Z",
    "value": "-23.908",
    "location": [
      39.4194156246497,
      -76.69504058954743,
      null
    ],
    "id": "0DY1QDB7K3VN8YDCYCAD538Z5Q"
  }
}

Because of the always-on nature of the MQTT API and the difficulty of changing topics all at once on client devices, we include the legacy last_value field in the new JSON output. If you have a client using */json MQTT topic subscriptions, you should update from json.last_value to json.data.value to access your feed’s data. After a few months of this new MQTT JSON output format, we will deprecate the last_value field and remove it from the MQTT API.

Social web services and your safety and security

Because Adafruit IO is an open web service, we don’t review every single feed sharing invitation before it’s sent. In order to prevent repetitive unwelcome sharing invitations from anyone, we will always give you the ability to: ignore individual invites, block a user from being able to invite you to share, or unsubscribe from all sharing invitation emails.

When you ignore an invite, it will be cleared from your sharing page and the user who invited you won’t be able to invite you to that feed again, but they will be able to invite you to share any other feed they own.

When you block a user, they will no longer be able to share any feeds with you at all. Users will never be explicitly informed that you have blocked them, but they may be able to infer it if they can’t share anything with you. Sharing blocks are permanent until and unless you remove them.

Choosing to unsubscribe from sharing invitation emails will keep your inbox clean, so you’ll have to visit your sharing page to see if you have any new sharing invitations.

Adding a social layer to Adafruit IO has always been part of our plan, but we’ll never make your shares public. If you keep every feed and do all your sharing through direct user-to-user feed sharing, your public Adafruit IO page will still exist but remain empty. We believe that your information should be private by default.

While we do not require moderator approval for social activity on Adafruit IO, we do monitor use of the platform. If we believe a feature is being abused for any reason, we continue to reserve the right to disable the feature or limit your use of Adafruit IO. We care about building the best Internet of Things platform for makers and part of that is making sure that it continues to be a safe space for everyone who uses it.

Guides are coming!

We’ll be adding more information to the Adafruit IO welcome guide with most of this information and we’re always checking the Adafruit IO forums, so if you have any questions about feed sharing please stop by and ask.

This is the first iteration of a pretty new feature for us and we’re excited to see how people use it! Join us in the forum or on Discord in the adafruit-io channel with questions, comments, or suggestions.

Hello Adafruit IO Community,

We have a lot to talk about this week. We’ve been focusing development on our Arduino and Python Adafruit IO client libaries in the last couple of weeks and we’re ready to share information about them.

Adafruit IO Python Library

We’ve officialy released version 2.0.0 of the Adafruit IO Python Client Library. This is a major update which not only adds a lot of under-the-hood enhancements, but also a bunch of new requested features. We’re going to support versions =>2.0.0 of this library going forwards. We’ve also switched this library over to Python 3, support for Python 2 has officially been dropped.

Notable Features

• API-V2 Support: Adafruit IO Python now uses the Adafruit IO REST API v2

• Location Support: Sending Adafruit IO Feeds locational values is now possible from this library (lat/lon/ele). Usage example for this feature, location.py, added in examples/basics. (#48)

• MQTT Security: The MQTT Client now establishes a secure connection with Adafruit IO by default. (#45)

  • You can unset this by defining a secure=false boolean in the client initialization: client = MQTTClient(username, key, secure=False)

  • Time Topic Subscriptions:** Want to get the current time, date, or year? We added support for time/seconds, time/milliseconds and time/ISO-8601 topic subscriptions. Usage example for this feature, time_topics.py added in examples/basics. (#54)

• MQTT Error Handling: We added a few verbose error checks for when the mqtt_client.py encounters an error. This means error messages will be easier to understand, and your programs will be easier to troubleshoot as a result. (#44)

• New Documentation: We gave the documentation previously on the readme some t.l.c and a facelift. Documentation for this project is available on ReadTheDocs.

• Examples: New examples have been added to /examples/basics, including some usage examples for Adafruit CircuitPython with the Raspberry Pi (more on this soon!)

Breaking Changes

• Python 2 support has been discontinued. We are only supporting Python Version >=3.6.

• Adafruit IO REST API v1 support has been discontinued. We are only supporting the API v2.

• MQTT on_message method no longer requires the retain flag.

We actively maintain, develop and improve this library with the support of the community. If you’d like to see an ehancement or feature to this libary, let us know by filing an issue on the GitHub repository for the IO Python Client.

Adafruit IO Arduino Library

We’ve also been working on the Adafruit IO Arduino Library.

Features and Notes

• ESP32 Compatability: We’ve added compatabilty for the Feather HUZZAH32 (and other ESP32-based-boards) to this library.

• New Documentation: Just like the Python library, the Arduino Library now has a dedicated documentation page on ReadTheDocs

• Time Topic Subscriptions: Don’t have any available pins to use a RTC? Just poll the Adafruit IO Server’s time feeds instead.

  • Adafruit_IO_Arduino client library updated to support three types of time helpers: time/seconds, time/milliseconds, and time/ISO-8601.
  • Example for subscribing to all three of the feed subscriptions added to examples/adafruitio_17_time_subscribe/adafruitio_17_time_subscribe.ino

Get in touch with us on the Adafruit IO forums or ping @adam.io or @brentru on the Adafruit Discord, #adafruit-io channel if you have any questions.