OpenSSH version 7.3 introduced a very handy Include feature, which is great for people who have to manage connection info for multiple servers. This makes it easy for us to generate updated SSH configs via AWS CLI for the multiple EC2 instances that serve Adafruit IO.

Here is how you can use Include to pull in separate SSH config files from your main ~/.ssh/config. First, you will need to install OpenSSH version 7.3 or higher. If you are using Linux, you will need to install version 7.3+ via your package manager (yum,apt-get, etc), or build it from source.

On OS X, you can do this via homebrew:

$ brew install homebrew/dupes/openssh

Confirm that you are now running 7.3 or higher by running ssh -V:

$ ssh -V
OpenSSH_7.3p1, OpenSSL 1.0.2j  26 Sep 2016

Now you can create a new child config file in ~/.ssh using a text editor. For example, we can create an example child config at ~/.ssh/pi_config and add configuration info just as we would in the main SSH config file:

Host pi-1
  HostName 10.0.0.10
  User pi
  IdentityFile ~/.ssh/pi_cluster

Host pi-2
  HostName 10.0.0.11
  User pi
  IdentityFile ~/.ssh/pi_cluster

Host pi-3
  HostName 10.0.0.12
  User pi
  IdentityFile ~/.ssh/pi_cluster

From your main ~/.ssh/config, add the following line at the top:

Include ~/.ssh/pi_config

You should now be able to connect to your servers as you normally would:

$ ssh pi-1

That’s it! Check out this OpenSSH feature request if you would like more info.

Here are the stats for the past week:

* 45.3 million inserts of logged data in the last 7 days
* 16,227 users
* 11,985 online feeds (41,024 feeds total)
* ~75 inserts per second via MQTT
* ~10 inserts per second via REST API

Here’s a quick tip to help you export one of your Adafruit IO feeds as CSV, Excel, JSON, or XML. First, you will need to navigate to io.adafruit.com/feeds. Once your feed list has loaded, select the feed you wish to export:

Then, click the download button below the chart:

A modal will pop up and allow you to download your feed as CSV, Excel, JSON, or XML:

If you would like to download all a copy of all of your Adafruit IO data, you can do this by navigating to io.adafruit.com/settings, and clicking the Download All Data button. This button will currently only allow you to download your data in JSON format.

Was this tip helpful? Please visit our IO forum to ask questions or share your thoughts.

Here are the stats for the past week:

* 44.40 million inserts of logged data in the last 7 days
* 15,671 users
* 11,503 online feeds (39,364 feeds total)
* ~70 inserts per second via MQTT
* ~10 inserts per second via REST API

At Adafruit, when we push or merge changes to one of our hundreds of Arduino libraries on GitHub, we have to ask ourselves “Does this change work?”. In a lot of cases, the best answer we can give, even after extensive manual testing, is “Maybe?”. For Adafruit IO, it is important that we are able to easily answer “Yes.” to that question. A large portion of Adafruit IO users could be impacted by a bug in a library such as the Adafruit IO Arduino Library, and we want to make sure our client code is reliable.

The hard truth is that there aren’t enough hours in the day to test each change to every library on all of the supported platforms, so most of the time we have to settle with testing one or two platforms manually, and hoping for the best on the rest of the supported platforms.

What do we mean by platform? In this case, we are talking about the multiple microcontrollers that are supported in the Arduino IDE via the Arduino 1.6.x Third Party Hardware Specification. Arduino Uno, ESP8266, Feather M0 (ATSAMD21), and the Feather WICED (STM32F205) are all good examples different platforms that require separate testing.

First Attempt: Build Verification using Travis CI

We had previously attempted automated build checking using Travis CI, and that method works great for checking if sketches build on multiple platforms.

As you may have guessed, this leaves us with a huge blind spot. A passing build does not mean the code will do what you expect it to do.

Second Attempt: Manual Compatibility Testing

Our next attempt involved building a ‘compatibility matrix’ web app that allowed for easy logging of the results of manual tests on supported platforms. The app pushes test results to a GitHub repo as JSON (in case we need to access it programatically). It also automatically adds/updates a compatibility table in the README of the target library.

Here’s an example of the README output:

This method is very effective for answering “Does this change work?”, but it is very impractical to manually test the daily changes to libraries on every supported platform.

Latest Attempt: Running Tests on Hardware

Our latest approach to this problem uses Jenkins CI running on an AWS EC2 instance, and small fleet of local Raspberry Pi test nodes. Each node uploads and runs unit & integration tests on hardware attached to USB ports.

Each Raspberry Pi can support multiple platforms, and each platform can be targeted by the test. The tests themselves output results using the TAP protocol over a serial connection, and the Raspberry Pi sends the results back to Jenkins so we are aware when builds fail.

Here’s a visual overview of how it works:

Test status can be monitored in real time from anywhere, and we are able to view the raw console output from the test nodes when needed.

TAP results are also parsed and attached to each build:

We will be sharing the details of this in a tutorial soon, but if you would like to take a look at our early experiments with Jenkins, head to jenkins.adafruit.com or check out our Jenkins Pipeline Library on GitHub.