At Adafruit, when we push or merge changes to one of our hundreds of Arduino libraries on GitHub, we have to ask ourselves “Does this change work?”. In a lot of cases, the best answer we can give, even after extensive manual testing, is “Maybe?”. For Adafruit IO, it is important that we are able to easily answer “Yes.” to that question. A large portion of Adafruit IO users could be impacted by a bug in a library such as the Adafruit IO Arduino Library, and we want to make sure our client code is reliable.

The hard truth is that there aren’t enough hours in the day to test each change to every library on all of the supported platforms, so most of the time we have to settle with testing one or two platforms manually, and hoping for the best on the rest of the supported platforms.

What do we mean by platform? In this case, we are talking about the multiple microcontrollers that are supported in the Arduino IDE via the Arduino 1.6.x Third Party Hardware Specification. Arduino Uno, ESP8266, Feather M0 (ATSAMD21), and the Feather WICED (STM32F205) are all good examples different platforms that require separate testing.

First Attempt: Build Verification using Travis CI

We had previously attempted automated build checking using Travis CI, and that method works great for checking if sketches build on multiple platforms.

As you may have guessed, this leaves us with a huge blind spot. A passing build does not mean the code will do what you expect it to do.

Second Attempt: Manual Compatibility Testing

Our next attempt involved building a ‘compatibility matrix’ web app that allowed for easy logging of the results of manual tests on supported platforms. The app pushes test results to a GitHub repo as JSON (in case we need to access it programatically). It also automatically adds/updates a compatibility table in the README of the target library.

Here’s an example of the README output:

This method is very effective for answering “Does this change work?”, but it is very impractical to manually test the daily changes to libraries on every supported platform.

Latest Attempt: Running Tests on Hardware

Our latest approach to this problem uses Jenkins CI running on an AWS EC2 instance, and small fleet of local Raspberry Pi test nodes. Each node uploads and runs unit & integration tests on hardware attached to USB ports.

Each Raspberry Pi can support multiple platforms, and each platform can be targeted by the test. The tests themselves output results using the TAP protocol over a serial connection, and the Raspberry Pi sends the results back to Jenkins so we are aware when builds fail.

Here’s a visual overview of how it works:

Test status can be monitored in real time from anywhere, and we are able to view the raw console output from the test nodes when needed.

TAP results are also parsed and attached to each build:

We will be sharing the details of this in a tutorial soon, but if you would like to take a look at our early experiments with Jenkins, head to jenkins.adafruit.com or check out our Jenkins Pipeline Library on GitHub.

Here are the stats for the past week:

* 37.3 million inserts of logged data in the last 7 days
* 15,121 users
* 11,080 online feeds (37,600 feeds total)

We have been working hard on a new version of the Adafruit IO Arduino Library. We are releasing it in ‘beta’ form today for existing IO users to try, and we will be adding new tutorials and updating existing tutorials with usage examples very soon.

Device Independence

v2.0.0 of the library provides a simple device independent interface for interacting with Adafruit IO. This allows you to switch beween WiFi (ESP8266, M0 WINC1500, & WICED), Cellular (32u4 FONA), and Ethernet (Ethernet FeatherWing) with only a two line change in your sketch. No changes are required to the sketch to switch between any of the supported Feather WiFi boards.

The included examples focus on specific concepts without extra boilerplate related to setting up the specific WiFi, cell, or ethernet hardware being used. This will allow you to prototype your sketch on WiFi hardware, and easily move to cellular or ethernet with a very small change to your config file.

Location

The library makes it simple to publish GPS location info with your data. For example, if you wanted to publish your location with your current speed, you would send it like this:

car->save(speed, lat, lon, ele);

You could then use the map block on io.adafruit.com to display your location info. The included adafruitio_04_location example demonstrates how to send and receive location info.

Type Conversion

The library adds type conversion helpers for both publishing and receiving data. You can publish any data type to the save(value) method of your feed, and you can use helpers like toInt(), toBool(), & toFloat() to easily convert received messages to the appropriate data type. The included adafruitio_05_type_conversion example demonstrates converting to and from all available types.

Updating to v2.0.0

The README for the library has more info about required dependencies, and you should check there to make sure you have all of the requirements for the hardware you are using. Make sure you install v2.0.0 or higher of the Adafruit IO Arduino Library, and v0.16.0 of the Adafruit MQTT Library using the Arduino IDE’s Library Manager.

Here are the stats for the past week:

* 37.6 million inserts of logged data in the last 7 days
* 14,701 users
* 10,713 online feeds (36,320 feeds total)
* ~60 inserts per second via MQTT
* ~10 inserts per second via REST API

Here are the stats for the past week:

* 37.3 million inserts of logged data in the last 7 days
* 13,840 users
* 10,138 online feeds (33,776 feeds total)
* ~60 inserts per second via MQTT
* ~10 inserts per second via REST API

To help with the increased load, we have added an additional sidekiq server to help process jobs, and it has helped with ensuring that messages from users are processed as soon as they arrive. The image below shows how adding the new server has helped smooth out job processing over the last day.

This week Adafruit IO passed the 10,000 online feeds milestone!

We have some exciting new things coming soon, and we are always looking for feedback about Adafruit IO. Please post any questions, feature requests, or show off your project in the Adafruit IO forum.