We moved Adafruit IO’s node.js services to a group of t2.nano EC2 instances, and I wanted to document my dsh setup. I used it reguarly a couple years ago for managing the backend workers when working on other services, but I had forgot how it was configured. It came in really handy when I needed to quickly check or kick anything on the servers.

First, you will need to install dsh using brew, apt-get, yum, etc:

$ brew install dsh

You will need to make a couple config directories for your user:

$ mkdir -p ~/.dsh/group

Create a new dsh.conf file using your favorite text editor:

$ vim ~/.dsh/dsh.conf

Paste the following config into ~/.dsh/dsh.conf, and save the file:

showmachinenames = 1
remoteshell = ssh

Now, you can create a dsh group for the set of servers you would like to send commands to. Create a new file called example in the ~/.dsh/group folder using your favorite text editor:

$ vim ~/.dsh/group/example

Enter the host names for your servers, and save the file:

example-1.adafruit.com
example-2.adafruit.com
example-3.adafruit.com
example-4.adafruit.com

Now you can send commands (like uptime) to all servers in the group using the following command:

$ dsh -g example -w 'uptime'
example-1.adafruit.com:  14:17:44 up 17:15,  0 users,  load average: 0.08, 0.03, 0.05
example-2.adafruit.com:  14:17:43 up 16:36,  0 users,  load average: 0.00, 0.01, 0.05
example-3.adafruit.com:  14:17:45 up 16:41,  0 users,  load average: 0.00, 0.01, 0.05
example-4.adafruit.com:  14:17:45 up 16:41,  0 users,  load average: 0.00, 0.01, 0.05

Using -w will send the command to each server in the order they appear in the group config, and will wait for a response before continuing. If you want to send the commands concurrently to all servers in the list, use the -c option to send the command:

$ dsh -g example -c 'uptime'
example-1.adafruit.com:  14:22:42 up 17:20,  0 users,  load average: 0.00, 0.01, 0.05
example-3.adafruit.com:  14:22:42 up 16:46,  0 users,  load average: 0.00, 0.01, 0.05
example-2.adafruit.com:  14:22:41 up 16:41,  0 users,  load average: 0.00, 0.01, 0.05
example-4.adafruit.com:  14:22:42 up 16:46,  0 users,  load average: 0.10, 0.05, 0.05

You can get an interactive shell for all servers in the group by using the -c and -i commands together. Use CTRL-D to exit the session:

$ dsh -g -c -i

Adafruit IO has been growing steadily, and we decided to add a changelog so it’s easier for users to see what we are up to. We will be posting here reguarly whenever we add or update features, and we will also be posting stats so you have more information about the growth of IO.

We have moved IO to a handful of Amazon EC2 instances, and we are finding it much easier to deal with increased load. The individual services are now split out into separate instances, which will allow us to scale horizontally.

Here’s a screenshot of what it looks like when we are monitoring the state of the new EC2 instances:

We have added IO to the Adafruit Status Page, which will allow you to check and see if any of IO’s components are currently down. You can also follow @adafruitstatus if you would like to be notified of outages.

Here are the stats for the past week:

* 24,290,428 inserts in the last 7 days
* 8,997 users
* ~6,700 online feeds (~18,200 feeds total)
* ~50 inserts per second via MQTT
* ~5 inserts per second via REST API