Author: Carl Shotwell

IoT Course Week 11: Load Testing

Screen Shot 2015-10-15 at 3.07.21 PM

Last week, we dove into into the importance of incorporating and collecting analytics through your connected device, how that information helps provide business value, and played with some of the ways that information can be displayed using some pretty graphs.

This Week

This week, we’ll continue our focus on non-functional requirements and start load testing. With connected devices, if the device can’t call home to its shared services, it loses a lot of its value as a smart device. These services need to be highly reliable, but things get interesting when thousands or millions of devices decide to call home at the same time.

To load test, we’ll generate concurrent usage on system until a limit, bottleneck, unexpected behavior, or issue is discovered. This usage should model real-life usage as close as possible, so the analytics we put in place last week will be a valuable resource. In instances where we don’t have data to work with, we can build out user funnels and extrapolate based on anticipated usage. Bad things will happen if we ship thousands of products without any idea how our system will react under the load. This data will also be a useful baseline for capacity planning and system optimization experiments.

The Lampi system has two shared services that we need to put under load. One is the Django web server that handles login, and the other is the MQTT broker that handles sending messages to the lamp.

Load Testing with Locust

To test the web server we use Locust. Locust has become a LeanDog favorite due to its simple design, scalability, extensibility, and scriptability. We’ve used it to generate loads of 200,000 simultaneous users distributed across the US, Singapore, Ireland, and Brazil. These simulated users (locusts) walked through multi-page workflows at varying probabilities, modeling the end-to-end user interaction, complete with locusts dropping out of the user funnel at known decision points.

Locusts are controlled via a locustfile.py. The one below shows a user logging in and going to the home page:

from locust import HttpLocust, TaskSet, task

class UserBehavior(TaskSet):

def on_start(self):
self.login()

def login(self):
response = self.client.get("/accounts/login/?next=/")
csrftoken = response.cookies.get('csrftoken', '')
self.client.post("/accounts/login/?next=/", {"csrfmiddlewaretoken": csrftoken, "username": {{USERNAME}}, "password": {{PASSWORD}} })

@task(1)
def load_page(self):
self.client.get("/")

class WebsiteUser(HttpLocust):
task_set = UserBehavior
min_wait = 5000
max_wait = 9000

In order to run locust, we’ll need a machine outside of the system to simulate a number of devices. Locust is a python package, so it can run on most OSes. It uses a master/slave architecture so you can distribute the simulated users and allow for more and more load.

Once you install locust and start the process, you control the test via a web interface.

Screen Shot 2016-05-04 at 12.09.46 PM

Locust will aggregate the requests to a particular endpoint and provide statistics and errors for those requests.  

Screen Shot 2016-05-04 at 12.09.55 PM

Screen Shot 2016-05-04 at 12.10.03 PM

Load Testing with Malaria

To test MQTT we used a fork of Malaria. Malaria was designed to exercise MQTT brokers. Like locust, Malaria spawns a number of processes to publish MQTT messages. Unlike locust, it’s not easy to script; you have to fork it to do parametric testing or randomize data.

usage: malaria publish [-D DEVICE_ID] [-H HOST] [-p PORT] [-n MSG_COUNT] [-P PROCESSES]

Publish a stream of messages and capture statistics on their timing

optional arguments:
-D DEVICE_ID (Set the device id of the publisher)
-H HOST (MQTT host to connect to (default: localhost))
-p PORT, (Port for remote MQTT host (default: 1883))
-n MSG_COUNT (How many messages to send (default: 10))
-P PROCESSES (How many separate processes to spin up (default: 1))

By modulating MSG_COUNT and PROCESSES you can control the load being sent to the broker.

Running some Example loads
Small load: Using 1 process, send 10 messages, from device id [device_id]

loadtest$ ./malaria publish -H [broker_ip] -n 10 -P 1 -D [device_id]

Produces results similar to this:

Clientid: Aggregate stats (simple avg) for 1 processes
Message success rate: 100.00% (10/10 messages)
Message timing mean 344.51 ms
Message timing stddev 2.18 ms
Message timing min 340.89 ms
Message timing max 347.84 ms
Messages per second 4.99
Total time 14.04 secs

Large load: Using 8 processes, send 10,000 messages each, from device id [device_id]

loadtest$ ./malaria publish -H 192.168.0.42 -n 10000 -P 8 -D [device_id]

Monitoring The Broker

MQTT provides a set of topics that allow you to monitor the broker.

This command will show all the monitoring topics (note that the $ is escaped with a backslash):

cloud$ mosquitto_sub -v -t \$SYS/#

The sub topics ‘…\load...’ are of particular interest.

Gather data

Before we start testing, we should figure out what metrics we want to measure. Resources on the shared system (CPU, memory, bandwidth, file handles) are good candidates for detecting capacity issues. Focusing on the user experience (failure rate, response time, latency) will help you hone in on the issues that will incur support costs or retention problems. Building the infrastructure to gather, analyze and visualize those metrics can be a significant part of the load testing process – but those tools are also necessary to do useful operational support in production. For the class, students used sysstat, locust, mqtt and malaria to gather metrics. A production-like system might use AWS Cloudwatch, New Relic, Nagios, Cacti, Munin, or a combination of other excellent tools.

The point of load testing is to find the limits and then decide what to do about them. There will be a point where the cost to rectify the issue is greater than any immediate benefit, load testing will help you find that bar. During the class, limits of a 1000 simultaneous users for web and 5,000-10,000 MQTT messages per process were common.

Final project

For their final project two students from the class, Matthew Bentley and Andrew Mason, decided to take on some of the problems with mqtt-malaria and extend Locust to publish MQTT messages. Using Locust they were able to scale their load test infrastructure across many machines and put a broker under more stress. In their previous testing with malaria, they found the point where a single device could send no more messages (at a reasonable rate), but they could not scale malaria to determine at what point the broker would not process any additional connected devices’ messages. Through their efforts, they reached 100% CPU on the broker, pushing 1 million messages a minute to 4000 users. As a result of their work they also open sourced their contribution to locust.

IoT Course Week 10: Analytics

IoTBackground

Last week we got our feet wet with an introduction to Bluetooth Low-Energy on iOS. This week, we’ll dive into analytics, provide business value, and make some pretty graphs.

Why Analytics?

When building a new product, there are always a variety of options on the table with which to improve that product. At LeanDog, we practice a software development cycle that includes short sprints coupled with an open and honest feedback loop that provides us with the information we need to make informed decisions about where to focus our efforts and resources. This allows us to make sure that we are building the right thing the first time and minimize the amount of risk inherent in the process.

Until relatively recently, collecting feedback about a product in-use was a long process that required either direct observation or careful reading of written user reviews and complaints. Due to the complex and inconsistent nature of users, collecting strong quantitative data about a product experience can be difficult. In a now infamous incident from 2013, a New York Times journalist wrote a negative review of the Tesla Model S, only to have the car’s onboard analytics refute many of his claims. It is not uncommon for a customer to report one thing, but end up doing something entirely different, and your user experience process will need to account for these inconsistencies. One of the many ways we solve that problem is through the use of analytics platforms and reporting tools.

In addition to uncovering potential pitfalls, analytics are a powerful way for product owners, designers, and developers to understand how a product is actually used. For companies that make physical devices, this provides insights that are difficult to collect otherwise. Imagine receiving a coupon in the mail for a smart GE light bulb you love that’s nearing the end of it’s lifetime. The only way GE could possibly anticipate that your current bulb is about to go out (without calling you every day to ask how often you turned it on in the last 24 hours) is through analytics. With analytics, you get an avenue outside of sales to start to figure out which features and products your users actually love, which have problems or aren’t worth further development, and even identify disengaged users for retention campaigns.

Enter Keen IO
For this class, we will use a popular analytics platform called Keen.io. Keen is a general purpose tool, not locked into web, mobile, or embedded specifics. It has a large number of supported software development kits (SDK’s), including Ruby, iOS, Python, .NET, etc. It also offers a powerful free tier, which is perfect for the amount of traffic currently being driven on student’s LAMPi systems. Registering and sending a notification in Python is as simple as as this:

from keen.client import KeenClient

client = KeenClient(
project_id="xxxx",
write_key="yyyy",
)

client.add_event("sign_ups", {
"username": "lloyd",
"referred_by": "harry"
})

This will send an event containing the signup data to Keen’s database. Now back at LAMPi headquarters we can track those signups on a giant web dashboard:

var series = new Keen.Query(“count”, {
eventCollection: “sign_ups”,
timeframe: “previous_7_days”,
interval: “daily”
});

client.draw(series, document.getElementById(“signups”), {
chartType: “linechart”,
label: “Sign Ups”,
Title: “Sign Ups By Day”
});

image01

Keen also provides a number of ways to pull out the analytics data and do additional processing to get exactly the view we wanted. Like if we wanted to build a tree of who our top referrers are what their “network” looks like:

image00

What’s next?
Analytics can also provide a leading indicator to help model the number of users that will be pounding on your infrastructure. To learn more about how to address that issue, join us next week when we talk about load testing!

IoT Course Week 9: Introduction to Bluetooth Low Energy

 

Internet of Things Course

To continue our goal of providing industry experience to the students of EECS397 Connected Devices, this week we will be diving deep into Bluetooth LE on iOS.

Recap

Last week students completed setting up a UI on iOS and Android that mirrored the interactions present on the LAMPi display and the web. The goal for this week is to connect those pieces

CoreBluetooth and Bluetooth 4.0

With the release of iOS 5 and the iPhone 4S, Bluetooth LE was positioned and continues to be one of the most common methods of short range data communication.  CoreBluetooth is the framework that Apple provides to developers to interact with Bluetooth LE hardware and peripherals. This is useful, as the current Bluetooth LE spec weighs in at over 2000 pages in PDF form.

Communication with LAMPi through CoreBluetooth can be broken into a four step process:

  1. Scanning for LAMPi device (from provided array of service id’s)
  2. Connect to discovered service (lamp-service)
  3. Probe characteristics (hsv, brightness, on/off).
  4. Subscription notifies when something changes. notify on property write.

Scanning for LAMPi

Students began the class by making an update to their LAMPi’s. Each team was given a BlueGiga BLED112 to plug into their Raspberry Pis, as well as updated Python services which allow the LAMPi to act as a Generic Attribute Profile (GATT) server. What the GATT server does is broadcast a number of available services to any BLE devices nearby that care to listen. In the case of the LAMPi, there is only one service being exposed, which is aptly called the Lampi Service.

Screen Shot 2016-04-12 at 11.18.43 AM

The service being advertised from the LAMPi includes a device id, which students use to identify their unique LAMPi in a classroom containing many more. Once discovered, it is time to connect.


- (void)startScanningIfEnabled {

if(self.shouldConnect) {

[self.delegate onLoading:@"Searching for sensor..."];

NSArray *services = @[[CBUUID UUIDWithString:LAMPI_SERVICE_UUID]];

[self.bluetoothManager scanForPeripheralsWithServices:services options:nil];

}

}

Connecting to a Peripheral

Screen Shot 2016-04-12 at 11.20.51 AM

CoreBluetooth abstracts away much of the detail required in making a connection to a BLE peripheral. When a peripheral is discovered, our code will immediately attempt to connect. If that connection is successful, we search through the set of services that exist on the peripheral, looking for one that is recognized.

“`

– (void)centralManager:(CBCentralManager *)central

didDiscoverPeripheral:(CBPeripheral *)lampPeripheral

    advertisementData:(NSDictionary *)advertisementData

                 RSSI:(NSNumber *)RSSI {

       …

       [self.bluetoothManager connectPeripheral:self.lampPeripheral options:nil];

}

 

– (void)centralManager:(CBCentralManager *)central

 didConnectPeripheral:(CBPeripheral *)lampPeripheral {

   NSLog(@”Peripheral connected”);

   [self.delegate onLoading:@”Found lamp! Reading…”];

   lampPeripheral.delegate = self;

   …

       // Search for a known service

       for (CBService *service in lampPeripheral.services) {

           if([service.UUID isEqual:[CBUUID UUIDWithString:LAMPI_SERVICE_UUID]]) {

               self.lampService = service;

           }

}

“`

Services and Characteristics

At this point, we are connected to a Lamp Service, which is now providing us with a collection of characteristics. Characteristics are how communication in BLE works. To make a comparison to software, Services can be thought of as Classes while Characteristics are more like the properties on an Object. Characteristics support four different actions: read, write, notify and indicate. While read and write are arguably fairly straightforward, notify and indicate both have to do with a subscription flow that we will use heavily in the iOS application.

Screen Shot 2016-04-12 at 11.23.20 AM

Subscribing to Characteristics

Because LAMPi has both on device and cloud controls, we want to be able to track the state of the LAMPi in real time while the iOS app is running. If a user were to change the color of LAMPi by using the Raspberry Pi UI, the Bluetooth service would send a notification to iOS that the HSV characteristic had been changed.  The following block of code is an example of a discovered Characteristic being initialized. It reads the current hue and saturation of the HSV Characteristic, and then tells the app to subscribe to the Notify value (the notification) of the lamp peripheral.

“`

           [self.lampPeripheral readValueForCharacteristic:self.hsvCharacteristic];

           if(self.hsvCharacteristic.value != nil) {

               float fHue = [self parseHue:self.hsvCharacteristic.value];

               float fSat = [self parseSaturation:self.hsvCharacteristic.value];

               [self.delegate onUpdatedHue:fHue andSaturation:fSat];

           }

           

           [self.lampPeripheral setNotifyValue:YES forCharacteristic:self.hsvCharacteristic];

“`

At this point, when the LAMPi HSV Characteristic changes, CoreBluetooth will call a delegate method that is triggered from the setNotifyValue line.

“`

– (void)peripheral:(CBPeripheral *)peripheral

didUpdateValueForCharacteristic:(CBCharacteristic *)characteristic

            error:(NSError *)error ;

“`

It is in this block of code that the HSV value is updated in the app, and logic to refresh the UI is executed.

Fun Fact: Origins of Bluetooth Name

As a bonus for making it this far, did you know that the origin of the word “Bluetooth” comes from a c. 970 King of Denmark, called Harald Bluetooth? In fact, the Bluetooth logo is comprised of the Nordic runes for H(8px-Runic_letter_ior.svg) and B (12px-Runic_letter_berkanan.svg), Harald’s initials. 

IoT Course Week 8: Intro to Mobile Development

Internet of Things Course

For a change of pace, we are taking a step back from the cloud -> server model we have been working so diligently on, and instead turn our eyes towards mobile. As more and more people around the world enter the global smartphone market, the Internet of Things space is becoming increasingly reliant on smartphone interfaces to control connected devices. This is because the components native to the smartphones that many of us carry, are simple and effective media of interaction with the connected world around us.

The Mission

The goals of week 8 are to provide an introductory course on modern mobile development, in both native iOS with Objective-C and native Android in Java. This week will set up a user interface to control the LAMPi, which next week, will be extended to operate over Bluetooth.

iOS

While the students were almost equally divided on Android/iOS, most of the students utilized Mac laptops, so we made the pairs heterogeneous as each pair had to build the app for both platforms.  This is necessary due to restrictions that Apple places on its developers, where software meant to run on the iOS platform must be developed on a machine running some recent version of OSX. Students were led through the creation of a project in Xcode, and some initial configuration that Apple requires developers to follow in order to sign and run their code. Students used Xcode to create a single view project, and they got to work.

Working in Interface Builder and a UIViewController, students were guided through adding and connecting a UISlider and UILabel to an IBOutlet.

“`

#import <UIKit/UIKit.h>

@interface LampiViewController : UIViewController

@property (nonatomic, strong) IBOutlet UISlider *slider;

@property (nonatomic, strong) IBOutlet UILabel *label;

-(void)onSliderChanged:(UISlider*)sender;

@end

“`

“`

#import “LampiViewController.h”

@interface LampiViewController ()

@end

@implementation LampiViewController

– (void)viewDidLoad {

   [super viewDidLoad];

   NSLog(@”slider: %@ \n label: %@”, self.slider, self.label);

}

-(void)onSliderChanged:(UISlider*)sender {

   NSLog(@”slider changed”);

}

@end

“`

A this point, a slider appeared on the screen that could be interacted with to update the label.  However a problem existed when the screen was rotated on its side.

Screen Shot 2016-04-05 at 9.36.44 AM

As can be seen, the slider and logo fail to expand out when the screen changes. This can quickly be remedied by utilizing one of the much loved nuances of Xcode’s Interface Builder, which is defining constraints on the views.pin_image

And once applied, everything came together. As can be seen here, the iOS Simulator is capable of fluid rotation from portrait to landscape, without any upsetting of the slider position.

with_constraints

Android

Android development, unlike iOS development, is capable of running on a much larger swath of machines. Since its introduction, Google’s Android has been open sourced and designed to run on the Java Virtual Machine (JVM).

Students began by downloading Android Studio and the latest Android SDK Tools. Using the Android Studio Interface, students created a new Android Studio project, allowing the template to be generated for a blank activity. Android Studio wants to get developers started quickly. They offer some shims that help this process. Because our project will be full of custom layouts and widgets, choosing a blank activity is the ideal scenario. After all was said and done, students were left with an application containing one activity.shim_project

Students were given a custom UI Widget in order to define a consistent slider for the class to use. The slider was added to the created activity layout in the XML here:

“`

<?xml version=”1.0″ encoding=”utf-8″?>

<RelativeLayout xmlns:android=”http://schemas.android.com/apk/res/android”

   xmlns:tools=”http://schemas.android.com/tools”

   android:layout_width=”match_parent”

   android:layout_height=”match_parent”

   android:paddingTop=”@dimen/activity_vertical_margin”

   tools:context=”.Lampi”>

 

   <View

       android:id=”@+id/bar”

       android:layout_width=”match_parent”

       android:layout_height=”10dp”

       android:layout_alignParentLeft=”true”

       android:layout_alignParentTop=”true” />

 

   <com.leandog.widgets.hsv.sliders.HueSliderView

       android:id=”@+id/hue_slider”

       android:layout_width=”match_parent”

       android:layout_height=”wrap_content”

       android:layout_alignParentLeft=”true”

       android:layout_alignParentTop=”true” />

</RelativeLayout>

“`

and when rendered by the running application it looked like this:

hue_slider_example

An event listener is added to the HueSlider, which tells a changeColor to run whenever the slider is interacted with.  It also allows the slider to initialize to a certain position or color, which will be useful when starting and restarting the LAMPi appication.

“`

package com.leandog.lampi;

 

import android.os.Bundle;

import android.support.v7.app.AppCompatActivity;

 

import com.leandog.widgets.hsv.sliders.HueSliderView;

import com.leandog.widgets.hsv.sliders.OnSliderEventListener;

 

public class Lampi extends AppCompatActivity {

 

   HueSliderView hueSliderView;

   View bar;

 

   @Override

   protected void onCreate(Bundle savedInstanceState) {

       super.onCreate(savedInstanceState);

       setContentView(R.layout.activity_lampi);

       bar = findViewById(R.id.bar);

       setupHueSlider();

   }

 

   private void changeColor(int color) {

       bar.setBackgroundColor(color);

   }

 

   private void setupHueSlider() {

       hueSliderView = (HueSliderView) findViewById(R.id.hue_slider);

 

       hueSliderView.setOnSliderEventListener(new OnSliderEventListener() {

           @Override

           public void onChange(int color) {

               changeColor(color);

           }

 

           @Override

           public void onSliderInitialized() {

               changeColor(hueSliderView.getHue());

           }

       });

   }

}

“`

Where we end up

The homework for this week was to follow the patterns of user interface and code interaction just demonstrated, and build a fully operational user interface for both iOS and Android. These user interfaces will essentially mirror that which we have already built on both the LAMPi screen, and the web interface.

iOS Simulator screen capture:

ios_assignment

Android Emulator screen capture:

android_assignment

What is next!

Next week we will add the bluetooth functionality that connects our devices directly to the bluetooth hardware running on the Raspberry Pi controlled LAMPi.  For the sake of brevity and focus, from this point onward, mobile development will be done primarily on the iOS platform.

IoT Course Week 7: Platform Security Part B (#1)

Internet of Things Course

Welcome to Week 7, part B #1

Note: This is part 1 of a 3 part series about assignment 7B in the course. This particular assignment was quite information-dense, and if you are unfamiliar with some of the pieces involved, things can be a bit confusing.

In the previous week, we added some security to the system with Transport Layer Security (TLS). This encrypts the web, websockets, and MQTT communications. It also allows clients to authenticate that the servers are who they say they are, and Lampi Devices to authenticate the MQTT bridge connection to the EC2 MQTT Broker. This week we will be focusing on a few remaining security gaps in the model, including:

  • protecting the Cloud MQTT broker namespace
  • limiting which devices Django users can communicate with at the MQTT level
  • providing limitations on Lampi devices specifying how and which topics are mapped into the global MQTT topic namespace on EC2 (and a compromised device could potentially wreak havoc on the entire system)
  • requiring authentication of users connecting to the MQTT websockets interface

Where do we begin?

There are a few different ways to skin this security cat. We decided that integrating Mosquitto with Django would provide us with a way to have a single source of truth for Human User authentication, Human User authorization, and Device User authorization. Mosquitto is already configured to authenticate the LAMPIs via the TLS certificates we configured in the previous post. Also, we thought that a REST interface would be easiest to implement for the integration. While we used Django, really any system that supports REST would work with the architecture we’re outlining today.

The high-level components:
Screen Shot 2016-03-07 at 3.13.10 PM

What does this look like overall?

There are a few moving parts to this solution we decided to go with. Let’s take a look at it overall:

Screen Shot 2016-03-07 at 3.10.16 PM

You’ll notice there will be a plugin to Mosquitto that we’ll leverage to connect the Django web app with the MQTT system.

Next post will delve into the ACL support for LAMPI and the Human Users of the system. Stay tuned!

IoT Course Week 7: Platform Security (Part A)

Anytime you are making a product, platform security is something that needs to be addressed. In this post, we look at platform security and the steps you can take protect your users.

Platform Security internet of things collage IoT
By now we’ve created a page that controls LAMPI and are serving it out of Django. We have a user authentication scheme, although we’re not putting it to great use just yet. Before we get there, we have a huge problem — there is really no security to speak of anywhere in this system. Our MQTT bridging is “secured” with a username and password, but those are sent unencrypted over the network. It would be trivial to sniff this communication and discover these credentials.

We have two paths into our EC2 server we need to secure — the path from the lamp to the cloud, and the path from the user’s browser back to the cloud.

TLS Certificate

Rather than use username and password, we are going to use Transport Layer Security (TLS) to encrypt our communications. Using this channel of public key cryptography will ensure that messages being sent between the lamp and the cloud are impossible to decrypt or interfere with, without first compromising the private key of the Certificate Authority.

We will create our own local certificate authority which we will use to issue and sign certificates for both the lamp and web client. A CA functions via the following steps:

  1. Someone decides to be a CA
  2. Generate a Public and Private Key Pair
  3. Generate a CA Certificate, including the Public Key, and Sign it with the Private Key
  4. Distribute your CA Certificate to the World

Note: For a real CA, Private Key should be stored securely, probably on a non-networked computer. Compromise of the Private Key compromises all derived certificates!
If you want to read more about how TLS (and it’s predecessor SSL) work, here is an excellent Stack Overflow post.

For the purposes of this discussion, the Common Name of the TLS certificate has to be unique to the LAMPI instance. This is because the Common Name will be used as part of the authentication process later.

Add to Bridging

As was set up in Lecture 4, MQTT bridging between cloud and LAMPi is handled using the tool Mosquitto. Mosquitto supports TLS security natively, and turning it on is as simple as ensuring each device has the appropriate CA and generated keys to communicate with each other. On the LAMPi, the configuration is:

/etc/mosquitto/conf.d/bridging.conf

connection b827eb74663e_broker
 address ec2-52-20-29-213.compute-1.amazonaws.com:52122
bridge_cafile /etc/mosquitto/ca_certificates/lampi_ca.crt
 bridge_certfile /etc/mosquitto/certs/b827eb74663e_broker.crt
 bridge_keyfile /etc/mosquitto/certs/b827eb74663e_broker.key
 bridge_tls_version tlsv1.2
topic lamp/set_config in 1 "" devices/b827eb74663e/
 topic lamp/changed out 1 "" devices/b827eb74663e/
 topic lamp/connection/+/state out 2 "" devices/b827eb74663e/
 cleansession true

and on the cloud, the configuration is set to:

/etc/mosquitto/conf.d/port.conf:
 listener 52122 0.0.0.0
 cafile /etc/mosquitto/ca_certificates/lampi_ca.crt
 certfile /etc/mosquitto/certs/lampi_server.crt
 keyfile /etc/mosquitto/certs/lampi_server.key
 tls_version tlsv1.2
 require_certificate true
 use_identity_as_username true

Next, we need to allow that “user” access, so we configure Mosquitto’s passwords file and add the Common Name of the LAMPI instance. Once these are set, and Mosquitto is restarted, each Mosquitto instance is reset and the bridging will now be TLS secured!

Serving Django pages from NGINX

Next we’ll need to secure our web client. The current Django web server does not support TLS, and we are hoping to use TLS to communicate between the cloud and a user’s web browser. Since we already have nginx installed and it is a TLS capable web server, we will use that. nginx does not natively know how to serve a Django app, so we’ll use uWSGI as a WSGI adapter between nginx and Django. uWSGI provides detailed instructions on making this web server change in their docs.

lampi_server_nginx.conf

# configuration of the server
 server {
# the port your site will be served on
 listen 443 ssl;
 ssl_certificate /home/ubuntu/ssl_keys/lampi_server.crt;
 ssl_certificate_key /home/ubuntu/ssl_keys/lampi_server.key;
 ssl_protocols TLSv1.2;
# the domain name it will serve for
 server_name ec2-52-20-29-213.compute-1.amazonaws.com; # substitute your machine's IP address or FQDN
 charset utf-8;

Restarting uWSGI and nginx at this point will apply the changes. Now traffic when navigating to https://{django_web_address} will be secured via the CA that we created!

Securing Websockets

Since we’re getting some of the real-time information over websockets, and MQTT is serving that data using the websockets protocol, we want to also encrypt the transport of that data from MQTT to the Client’s web browser. Also, since this websocket data is in a web page that is already coming from a now encrypted NGINX server, we need MQTT to encrypt that websocket data with the same TLS certificate that we use for our NGINX configuration.

/config/mosquitto_cloud/conf.d/websockets.conf

listener 8081
 protocol websockets
 cafile /home/ubuntu/ssl_keys/lampi_ca.crt
 certfile /home/ubuntu/ssl_keys/lampi_server.crt
 keyfile /home/ubuntu/ssl_keys/lampi_server.key

Here is an example of what this would look like:

Platform Security Diagram for IoT Class

Wrapup

Device

  • no services listening on anything but localhost
  • MQTT bridge using TLS with unique certificate

EC2

  • Require TLS on MQTT (MQTT and Websockets)
  • Require TLS on NGINX HTTP (HTTPS)

Next Week

This week laid down the groundwork that makes up our platform’s security. Next week, we will continue this robust buildout, specifically focusing on the MQTT layer. User accounts will be limited through Django to lock down communication to specific LAMPi devices. The MQTT web sockets interface will also undergo some lockdown.

In case you missed it, here’s a link to last week’s post: IoT Week 6: Setting Up User Accounts or, if you’re hankering to snack on some Cucumber, check out How to Run Cucumber Tests with Docker.

IoT Course Week 6: Setting Up User Accounts

PastedGraphic-2

Welcome to Week 6 of the Case Western Reserve University IoT course. Over the next two weeks, we will be tackling a new kind of challenge: security. This week kicks off by setting up the web framework Django.

Last Week

Last week, students spent time setting up a static webpage on EC2 that communicates via MQTT to their hardware LAMPi’s. We kicked this week of with a demo and discussion about what everyone was able to accomplish.  See the previous week’s post here: WEEK 5 POST

Why Django?

Despite the multitude of web frameworks available, using Django was an easy choice for this project. It is a mature, modern, web framework with a highly active community. It is easy to configure to work with many different databases, and is shipped with a robust ORM at its core. It supports user account configuration out-of-the-box, as well as a powerful admin interface. The fact that Django is Python-based is just an added bonus, allowing the student’s Python experience on LAMPi to be transferred to the Web.

Linking user accounts

Step one was to get Django set up and configured to have user accounts through its default interface.  Details of how to do that can be found in the Django Documentation.  Students were provided with a sample login template which, when loaded through Django, looks just like this.

Screen Shot 2016-02-11 at 10.10.59 AM

Students changed their static hosted pages to be hosted through Django, and added the provided login template to web/lamp/templates/login.html. With a configuration of the routes, navigating unauthenticated to the root page will now redirect the user to this login screen.

urls.py 
urlpatterns = patterns('lamp.url',
    url(r'^$', login_required(views.index), name='index'),
    url(r'^logout/$', views.logout_user, name='logout_user')
)

From here, the static LAMPi control page is moved to the index template at  web/lamp/templates/index.html and the request is handled in the Django view logic through the addition of an index function in web/lamp/views.py.

def index(request):
   context={}
   user = request.user
   lamps = user.lamp_set.all()
   context['lamps'] = lamps
   return render(request, 'index.html', context)

Whats next

At this point, students have taken the first step into providing user access control to the LAMPi system. Our more security conscious readers will note, however, that virtually every point of integration within the LAMPi system at this point still remains completely insecure. Moving into Week 7, students will receive a crash course in common attack vectors and practical implementation of modern communications cryptography.

IoT Course Week 5 : Lampi Control From The Web

PastedGraphic-2

Welcome to week 5 of the Case Western Reserve University IoT course. Last week’s lesson, though backed by a massive amount of behind the scenes work, left something to be desired from a usability standpoint. By the end of this week’s lesson, the pieces will all begin to fit into place.

Demoing Previous Week’s Work

Class kicked off with a discussion around the previous week’s work. By following the MQTT protocol, students had bridged their Python controlled LAMPi’s to an Ubuntu server running on Amazon EC2. See the previous week’s post here: Week 4.

Building a Web Interface

To continue our effort of doing the simplest thing possible, students began building their web interfaces directly on NGINX. Though other popular web frameworks (e.g. Rails, Django) have web servers built in, in many ways they are rarely designed for high transaction throughput. In the future, it will also be important for students to be able to configure the web server to handle static and dynamic content, HTTP redirects, URL writing and redirects, HTTP/HTTPs, etc. Keeping things simple with NGINX ensures that the projects will begin on a scalable, configurable system.

Students were provided with a directory of sample HTML, CSS, and Javascript files to base their static websites from. The page resembled very much the display of the LAMPi’s control screen, just in a web form.  It is up to the students to build the functionality for the page.  

PastedGraphic-1

MQTT and Websockets

Last week, we were left with an EC2 instance that had been configured to listen to MQTT topics via bridging. We require a method for Mosquitto, our MQTT broker, to communicate with the controls on our static web page. TCP, the Transmission Control Protocol, via WebSockets, will provide the channel we need for this communication to take place. WebSockets provide a fast and scalable two-way communication that our web client can harness via Javascript. There is an added bonus that Mosquitto supports WebSockets out of the box, making setup as simple as specifying a TCP port to accept connections from.

Paho and Javascript

Paho was briefly covered in the Week 3 post. Following that pattern of use, we utilized the Paho Javascript client to communicate with Mosquitto. Paho supports websockets, so connecting to Mosquitto is as simple as connecting to the same TCP port that Mosquitto is active on.  Below, we have included an example of coding that demonstrates connecting paho.js to tcp.


var client = new Paho.MQTT.Client(hostAddress, Number(hostPort), clientId);
…
onConnect : function(response) {
client.subscribe("devices/+/label/changed");
},
onMessageArrived : function(message) {
console.log(message);
console.log(message.payloadString);
configurationState = JSON.parse(message.payloadString);
console.log(configurationState);
obj.updateUI();
},

 

Once this connection is made, we add the following code to sync the LAMPi state from MQTT to a javascript object that looks like this:

var lampState = {
color : {
h : .5,
s : 1,
},
brightness : 1
};

And finally, a little bit of Javascript which uses Jquery to manipulate the sliders to match this lamp state:

function setSliderValues(hue, saturation, brightness) {
$( ".hue" ).each(function(index, hueSlider) {
hueSlider.value = hue;
});
$( ".saturation" ).each(function(index, saturationSlider) {
saturationSlider.value = saturation;
});
$( ".brightness" ).each(function(index, brightnessSlider) {
brightnessSlider.value = brightness;
});

updateSliderStyles(hue, saturation, brightness);
}

Witness the Big Picture

We now have a full path messaging system, from the Lampi to the cloud, and back again! When the LAMPi interface is changed on the device or the static webpage just built, the change is mirrored on the other interface instantly.

Next Week!

Next week, we’ll tackle a glaring problem in this system: security. Security in the IoT space has been a huge point of conversation as devices continue the path of hyperconnectivity and intelligence. At this point, there is no access control or security of the LAMPi system. We will take a big step forward, and add a web framework to give us greater control over users and access.

 

IoT Course Week 4 : Introduction to the Cloud

IoTBackground

It is week four of the IoT course and students of Case Western Reserve University have successfully built their very own touch-screen controlled LAMPi’s. This week students will take a major step towards greater functionality and control.

Demoing Previous Week’s Work

Students began the week by demoing their assignment from the previous week – building a  Python service to act as an abstraction layer between the LAMPi hardware and the various user interfaces. You can check out the post for the week here: Week 3

Learning IoT the hard way – Why we are not using an IoT platform

This week’s lecture started with a brief introduction to available commercial IoT platforms. For this course, students will forego working with a commercial IoT platform, and delve directly into Amazon Web Services. By having the students build their own platform on top of AWS, they will gain a greater understanding of the various components involved, which will allow them to make informed decisions when dealing with commercial IoT platforms in a professional capacity.

To the cloud – Amazon EC2

The cloud! Yes, that cloud. Those nebulous servers in the sky. Students will be maintaining their cloud connectivity using AWS Elastic Compute Clusters, or EC2.  At the beginning of week 4, the services running on LAMPi were locally controlled through a MQTT broker called Mosquitto. The goal for this week is to bridge the MQTT broker running on the Raspberry Pi, to another MQTT broker running on EC2. Luckily for our students, with a little configuration, Mosquitto offers this functionality.

Students began by setting up free tier EC2 instances running Ubuntu. After configuring AWS security groups for their instances, they installed the Mosquitto command line utilities mosquitto_pub and mosquitto_sub.

Bridging between Mosquitto brokers is done through editing configuration files on both the host and the remote. When bridging, the user can configure what topics are transmitted in which direction — for instance, we can specify that only lamp state updates go out to the cloud, and only configuration changes are received from the cloud. Any topics not configured do not get to cross the bridge and will remain on the system they originated. The configuration to the remote follows the structure here:

connection <bridge_name>
address <IP>:<port>
topic <pattern> <direction> <qos> <local-prefix> <remote-prefix>
remote_clientid <remote_clientid>
cleansession <true or false> (defaults to false)

Following this pattern will open the bridge between the MQTT brokers. From that point, depending on the configuration of “In” and “Out” directions, messages will be passed from the appropriate publisher to its subscribers.  It is worthwhile to note that, although the broker is now up and running on EC2, there is not yet any functionality beyond simply passing messages. This will be the topic of the next lesson. Part of the reason for this is that configuration of Mosquitto can be complex and that complexity is compounded by the configuration of the whole system — security, supervisord, pigpiod, etc. By doing this in steps, we can test our system at each step of the way. For now, a few more steps are required to ensure our path from the Raspberry Pi and EC2 instance remain unbroken.

Unexpected Events – Last Will and Testament

The MQTT specification defines a mechanism called a Will. In the event that a client disconnects unexpectedly the MQTT server will publish the Will message. The students will use this mechanism to announce the connection state of the lamp. In Python, the code looks like this:

       client = mqtt.Client(client_id=CLIENT_ID, protocol=MQTT_VERSION)
       client.will_set(get_client_status_topic(CLIENT_ID),
                       “0”, qos=1, retain=True)

Service management – Supervisor

The students were introduced to a Linux service called Supervisor which is responsible for ensuring that all of the processes that we care about stay running. If the LAMPi is unplugged, or the cloud service restarts for some reason, a proper Supervisor setup will ensure that all systems are restarted and running. These concepts were applied to the LAMPi’s by creating a Supervisor D configuration file that will keep the lamp service and the UI running.

What is up next?

Next week, the students will be introduced to web interfaces and the servers that host them. They will be utilizing HTML, CSS, Javascript, Websockets, and NGINX to create a web interface to LAMPi. This will build upon the the work they accomplished this week by hosting the web server on their EC2 instance.