We have been working on finishing up v2 of the REST API this past week, and you can read more about that here.

We are also in the process of finishing up transitioning our frontend javascript to React from jQuery and Backbone. When combined with API v2, this change should increase performance, and also will make development easier and faster. We will post more about this change in the next few days.

Here are the stats for the past week:

* 25,951,672 inserts of logged data in the last 7 days
* 9,296 users
* 6,949 online feeds (19,084 feeds total)
* ~50 inserts per second via MQTT
* ~5 inserts per second via REST API

We are currently in the process of developing version 2.0 of the Adafruit IO REST API, which should address a large portion of user requests. The current version of the REST API (v1.0) will still be available at io.adafruit.com/api/v1/.

Here are some of the highlights (not including bug fixes):

NEW

• New permission system that will allow for sharing read and write access to your feeds, groups, & dashboards with other users.
• New AWS inspired HTTP request signing to help avoid exposing the user’s AIO key over insecure connections.
• Dashboards, blocks, & triggers will be able to be modified via the REST API.

CHANGES

• Username will be a required component of the URL.
• You will no longer be able to access feeds & groups via numeric ID. Feed key & group keys will be used as the unique identifier in API v2.
• Feeds will be able to be added to many groups, and the feed’s data will be namespaced to the group the data is pushed to. You will also be able to access all of the feed’s data by accessing the feed directly.

We had a request in the forums to reduce the length of the AIO Key to 32 characters, and we decided to use UUID v4 (with the dashes stripped) to generate keys. Version 4 UUIDs are randomly generated, so they should work well as AIO Keys:

…after generating 1 billion UUIDs every second for the next 100 years, the probability of creating just one duplicate would be about 50%.

We have been writing user’s feed data to both Cassandra 3.3 and PostgreSQL 9.5 for the past couple weeks as the first step in transitioning fully to Cassandra as the primary data store for feed data. We will still be using Postgres as our primary database for relational data, but it should be much easier to scale Cassandra horizontally as the large amount of logged data increases.

Here’s the current write performance data from Cassandra’s nodetool tablestats:

Write Count: 59051081
Write Latency: 0.012758596713919598 ms.

That’s a little over 59 million writes in the past ~17 days, with the load on a m4.large barely ever passing 0.05.

load average: 0.01, 0.01, 0.05

Today, we have also started testing read performance in production. For now users are still receiving their data from Postgres, but we are also hitting Cassandra to see if our configuration can handle production loads. We’ll post an update once we have more info about how reads are performing.

Here’s some early info after the deploy:

Read Count: 314
Read Latency: 0.6644044585987261 ms.

We moved Adafruit IO’s node.js services to a group of t2.nano EC2 instances, and I wanted to document my dsh setup. I used it reguarly a couple years ago for managing the backend workers when working on other services, but I had forgot how it was configured. It came in really handy when I needed to quickly check or kick anything on the servers.

First, you will need to install dsh using brew, apt-get, yum, etc:

$ brew install dsh

You will need to make a couple config directories for your user:

$ mkdir -p ~/.dsh/group

Create a new dsh.conf file using your favorite text editor:

$ vim ~/.dsh/dsh.conf

Paste the following config into ~/.dsh/dsh.conf, and save the file:

showmachinenames = 1
remoteshell = ssh

Now, you can create a dsh group for the set of servers you would like to send commands to. Create a new file called example in the ~/.dsh/group folder using your favorite text editor:

$ vim ~/.dsh/group/example

Enter the host names for your servers, and save the file:

example-1.adafruit.com
example-2.adafruit.com
example-3.adafruit.com
example-4.adafruit.com

Now you can send commands (like uptime) to all servers in the group using the following command:

$ dsh -g example -w 'uptime'
example-1.adafruit.com:  14:17:44 up 17:15,  0 users,  load average: 0.08, 0.03, 0.05
example-2.adafruit.com:  14:17:43 up 16:36,  0 users,  load average: 0.00, 0.01, 0.05
example-3.adafruit.com:  14:17:45 up 16:41,  0 users,  load average: 0.00, 0.01, 0.05
example-4.adafruit.com:  14:17:45 up 16:41,  0 users,  load average: 0.00, 0.01, 0.05

Using -w will send the command to each server in the order they appear in the group config, and will wait for a response before continuing. If you want to send the commands concurrently to all servers in the list, use the -c option to send the command:

$ dsh -g example -c 'uptime'
example-1.adafruit.com:  14:22:42 up 17:20,  0 users,  load average: 0.00, 0.01, 0.05
example-3.adafruit.com:  14:22:42 up 16:46,  0 users,  load average: 0.00, 0.01, 0.05
example-2.adafruit.com:  14:22:41 up 16:41,  0 users,  load average: 0.00, 0.01, 0.05
example-4.adafruit.com:  14:22:42 up 16:46,  0 users,  load average: 0.10, 0.05, 0.05

You can get an interactive shell for all servers in the group by using the -c and -i commands together. Use CTRL-D to exit the session:

$ dsh -g -c -i