Raleigh Code Camp 2013 Netduino Azure Session

by Mike Linnen 30. October 2013 21:31

I am presenting two sessions in the Raleigh Code Camp 2013 Builder Faire track on November 9th.  The first session is called Building a cloud enabled home security system Part 1 of 2 (the presentation).  The second session is Building a cloud enabled home security system Part 2 of 2 (the lab).  You really need to come to both sessions as the first session explains what you will be building in the second session.  Yes I said that right, if you attend the second session you will be building a Netduino based security system that connects to Windows Azure.  Check out the project website for more details at Cloud Home Security.

I hope to see you there!!

Carolina Code Camp 2013 Netduino Azure Session

by Mike Linnen 3. May 2013 21:39

I am presenting 2 sessions in the Carolina Code Camp 2013 Builder Faire track.  The first session is called Building a Home Security System – The Introduction.  The second session is Building a Home Security System – The Lab.  You really need to come to both sessions as the first session explains what you will be building in the second session.  Yes I said that right, if you attend the second session you will be building a Netduino based security system that connects to Windows Azure.  Check out the project website for more details at Cloud Home Security.    

Demo connecting 11 Netduinos to Windows Azure Service

by Mike Linnen 14. February 2013 23:28

I put together a talk that includes a lab on building a security/home automation system using 11 netduinos communicating over MQTT with a broker located in Windows Azure.  The attendees of this talk will walk through the lab and build out various components of a security system.

Here is a video demonstrating the various components of the system.  

The source for the project can be found on github:

The Security System website is hosted on a Web Role and it contains all the documentation for the lab.

Getting Really Small Message Broker running in Azure

by Mike Linnen 1. June 2012 23:34

In my previous post I talked about changing my home automation messaging infrastructure over to MQTT.  One of my goals was to also be able to remotely control devices in my house from my phone while I am not in my home.  The good news is that this is easily done by setting up a bridge between two brokers.  My on-premise broker is configured to connect to the off-premise broker as a bridge.  This allows me to publish and subscribe to topics on the off-premise broker which in turn get relayed to the on-premise broker. Well we need to host the off-premise broker somewhere and that somewhere can be an Azure worker role.

Really Small Message Broker (RSMB)  for windows is simply a console application that can be launched in a Worker Role.  In this blog post I will be showing you how to do just that.  One thing to note here is make sure you read the License agreement of RSMB before you use this application for your purposes.

Of course to actually publish this to Azure you will need to have an Azure account but this will also run under the emulator.  If you don’t have the tools to build windows azure applications head on over to the Windows Azure Developer portal  and check out the .Net section to get the SDK bits.  Also the following instructions assume you have downloaded RSMB and installed it onto your windows machine.

Create a new Cloud Windows Azure Project

image

Once you press the Ok button you will be asked what types of roles you want in the new project.  Just select a Worker Role and add it to the solution.

image

To make things easier rename the role as I have done below.

image

After selecting the Ok button you need to set up an endpoint for the worker role that will be exposed through the load balancer for clients to connect to.  Select the worker role and view the properties of the role.  Select the Endpoints tab and add a new endpoint with the following settings:

  • Name: WorkerIn
  • Type:
  • Protocol: tcp
  • Public Port: 1883

image

Add a new folder under the RSMBWorkerRole project called rsmb

image

Copy the following RSMB files to the new folder and add them to the RSMBWorkerRole project with Copy to Output Directory set to Copy Always

  • rsmb_1.2.0\windows\broker.exe
  • rsmb_1.2.0\windows\mqttv3c.dll
  • rsmb_1.2.0\windows\mqttv3c.lib
  • rsmb_1.2.0\messages\Messages.1.2.0

image

Add a class level declaration as follows:

Process _program = new Process();

Make sure you have a using statement for system.io at the top of the class.

using System.IO;

Add code to the OnStart Method as follows:

public override bool OnStart()
{
    // Set the maximum number of concurrent connections
    ServicePointManager.DefaultConnectionLimit = 12;

    string rsbroot = Path.Combine(Environment.GetEnvironmentVariable("RoleRoot") + @"\\", @"approot\\rsmb");
    int port = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["WorkerIn"].IPEndpoint.Port;

    ProcessStartInfo pInfo = new ProcessStartInfo(Path.Combine(rsbroot, @"broker.exe"))
    {
        UseShellExecute = false,
        WorkingDirectory = rsbroot,
        ErrorDialog = false,
        CreateNoWindow = true,
    };
    _program.StartInfo = pInfo;
    _program.Start();

    return true;
}

You should be able to launch the project under the Azure emulator and then use an MQTT client to connect to a topic like $SYS/# and the client should connect without error and start receiving notifications for the published messages.  If you need to setup some additional broker configurations such as the broker.cfg then just add it to the project under the rsmb folder and make sure it is set to copy to the output directory always.  You might want to enhance the code in the OnStart method to redirect output of the RSMB console to the azure diagnostics to make troubleshooting issues easier.  You also need to setup the on-premise broker to connect to the remote broker as a bridge.  The instructions to set up the local broker as a bridge can be found in the README.htm where you installed RSMB.

Switching out my home automation messaging infrastructure for MQTT

by Mike Linnen 1. June 2012 21:09

I am re-vamping my home automation strategy from a home grown publish/subscribe messaging system to use MQTT instead.  I was using Azure Service Bus to connect remote devices such as my phone with devices in my home such as my lawn irrigation system.  This worked well as a messaging infrastructure for remote devices but I wanted to have a more standard messaging infrastructure that could work in my home network without connectivity to the outside world. 

A few reasons why I switched to MQTT:

  • Light weight
  • Many Clients already exist for many platforms and languages
  • Support for on-premise message broker
  • Support for off-premise message broker
  • Support for bridging brokers (on-premise to off-premise)
  • Fast
  • Used by companies like COSM (was Pachube) and Github
  • Simple topic subscription model that is also powerful
  • I don’t want to write a message broker

For the most part I am moving toward having devices in the home that are relatively dumb and having services running on a home server that add the smarts behind the devices.  This will give me the flexibility to change the behavior of the system a lot quicker without the hassle of tearing apart a device to upgrade the software on it.  This means I needed to have my services available all the time.  Placing these services in the cloud for mission critical things would mean I am left with devices in the home that cannot function while my internet connectivity is down.  This was the biggest reason I moved to an off the self pub/sub infrastructure like MQTT.

Like most messaging protocols, MQTT works on the notion of a topic and a message.  The topic is just a unique way of addressing a message.  I struggled a lot and I probably will continue to struggle on what my topic structure for my home automation should look like.  One thing I wanted to do is try to make the topics readable so that troubleshooting message problems would be easier.  Hear are a few standards I am trying to settle on:

  • When a device or service changes state and wishes to notify interested parties the topic will end with /event 
  • When a device or service wants to know the status of another device or service the topic will end with /getstatus
  • When a device or service receives a topic /getstatus the response topic it generates will end with /status
  • When a device or service needs to set the state of another device or service the topic will end with /set

Here are a few examples of topics and messages for my irrigation system:

Description Topic Message
Zone 1 turned on irrigation/zone/event z1 On
Request the current schedule from the irrigation service. The service will respond to this request by publishing various status topics irrigation/schedule/getstatus  
Set the time that the irrigation service should start watering irrigation/schedule/starttime/set 09:00 AM
Status of the schedule start time in response to the getstatus request irrigation/schedule/starttime/status 09:00 AM
Set the days of the week that the irrigation system will water irrigation/schedule/days/set MON WED FRI
Status of the scheduled days of the week in response to the getstatus request irrigation/schedule/days/status MON WED FRI
Set the zones that the irrigation system will run and how long irrigation/schedule/zones/set z1 10 z2 8 z3 10
Status of the scheduled zones in response to the getstatus request irrigation/schedule/zones/status z1 10 z2 8 z3 10
Sets the zones to run on the irrigation device and how long irrigation/zones/run z1 10 z2 8 z3 10

MQTT does have a concept of publishing messages with a retain bit. This just tells the broker to hang onto the last message for a topic and when a new subscription arrives the client will receive the last message.  I could have used this concept instead of the /getstatus standard that I have show above.  I might change over to using the retain bit but for now the /getstatus works for me. I am also making my messages a little verbose as they tend to contain multiple values that could have been broken down into more granular topics.

Overall I really like how simple MQTT is and it is very easy to get a device like the Netduino to understand MQTT messages.  I am sure I will make modifications on how I actually define my topics and message body over time as I develop more and more devices and services that do useful stuf in my home.    

Lawn Sprinkler the Data Flow Part 3

by Mike Linnen 4. September 2011 16:57

This is part 3 of a multipart blog series that shows how commands that control my sprinkler flow through the infrastructure and reach the final destination of the Netduino Plus.  This video covers what components are used in the system to connect a Windows Phone 7 to the Netduino based Lawn Sprinkler as well as how a Weather Service can send data to the Netduino.  So connecting devices to devices and services to devices all using the Azure AppFabric Service Bus.

Here is the video:

Lawn Sprinkler the Demo Part 2

by Mike Linnen 21. August 2011 18:34

As mentioned in a previous post I am building a Home Automation project that consists of replacing my Lawn Sprinkler.  This is part 2 of the blog series but if you want to look at other posts related to this then here are some links to them as well:

Here is a video that demonstrates how the Lawn Sprinkler system works.

The code for the project is located on bitbucket at https://bitbucket.org/mlinnen/lawnsprinkler.

I will be doing a presentation on this project at the Charlotte Alt.Net meeting on August 24th 2011

The power point slides for the presentation is also posted in the docs folder of the source repository https://bitbucket.org/mlinnen/lawnsprinkler/src/f5cd6cda9501/Docs/.

Lawn Sprinkler the Introduction Part 1

by Mike Linnen 2. July 2011 09:00

Overview

The new craze for Home Automation is to use technology to Go Green.  One aspect of Going Green is about managing resources in a more efficient way.  I have seen a number of other hobbyists build projects that manage the amount of electricity or gas that they use within their home.  In this project I am going to manage the amount of water I use for watering my lawn.  In part 1 of this series I am going to cover the big picture of what I am attempting to do.

Since this is a multipart post I am including the links to the other parts here as well:

Requirements 

Of course I needed a few requirements to define the scope of what I am attempting to do.

  • Support for up to 4 zones
  • Be able to manually turn on 1 or more zones (max 4) and have them run for a period of time
  • Be able to schedule 1 or more zones (max 4) to come on daily at a specific time of the day multiple times a day.
  • Be able to schedule 1 or more zones (max 4) to come on every Mon, Wed and Friday at a specific time of the day multiple times a day.
  • Be able to schedule 1 or more zones (max 4) to come on every Tuesday and Thursday at a specific time of the day multiple times a day.
  • Be able to turn off the system so that the scheduled or manual zones will immediately turn off or not turn on at their scheduled time.
  • Be able to do any of the above requirements remotely.
  • Do not turn on the sprinkler if rain is in the forecast (Go Green)
  • Do not turn on the sprinkler if the ground is already moist enough (Go Green)
  • Be able to automatically set the clock when daylight savings time changes.

At first I was going to make the sprinkler system a completely stand alone device where I could setup the schedule by using a keypad and an LCD.  This would allow me to completely control the device without remotely connecting to it.  But since I wanted to control the device remotely anyway and the cost of hardware and development efforts would be higher for a stand alone device, I decided to abandon the “Stand Alone” capabilities.  I did want the ability to turn off the sprinkler system without remotely connecting to it and I also wanted a quick way to know if the device was off or not.  A push button switch can be used to turn the sprinkler immediately off.  A couple LEDs can be used to let you know what mode the sprinkler is in.

The Sprinkler

I am using a Netduino Plus as the microcontroller that operates my sprinkler heads.  I choose this device because it uses the .Net Micro framework and it also has an onboard Ethernet controller which makes connecting it to my network a real easy task.  You could very easily use another device to control the sprinklers as long as it could handle the HTTP messages and had enough I/O to interface to the rest of the needed hardware. 

This device is responsible for the following:

  • Monitor the schedule and turn on the sprinklers if it is time to do so
    • 4 Digital Outputs
    • Onboard clock to know when to run the scheduled time
  • Watch for HTTP JSON requests that originate from the Windows Phone
    • The onboard Etherent works well for this
  • Watch for HTTP JSON requests that originate from the weather service telling the sprinkler the chance of rain
    • The onboard Etherent works well for this
  • Watch for HTTP JSON requests that originate from the time service telling the sprinkler to change it’s onboard clock
    • The onboard Etherent works well for this
  • On power up ask the time service for the correct time
    • The onboard Etherent works well for this
  • Monitor the Off pushbutton and cycle the mode of the sprinkler through the 3 states: Off/Manual/Scheduled
    • 1 Digital Input
  • Yellow LED goes on when in the Manual state
    • 1 Digital Output
  • Green LED goes on when in the Schedule state
    • 1 Digital Output
  • Monitor the ground moisture (Note: I haven’t done much research on how these sensors work so this might change)
    • 1 Analog Input
  • Persist the Manual and Scheduled programs so that a power cycle wont these values

The sprinkler modes need a little more discussion.  When in the Off mode the sprinkler heads will not turn on but the board will be powered up and listen for any HTTP requests and monitor the push button.  When cycling to the Off mode from any other mode the sprinklers will turn off if they where on.  When cycled to the Manual mode from any other mode the sprinkler will immediately run the manual schedule turning on the appropriate zones for the appropriate length of time.  If no Manual schedule exists then the sprinkler does nothing. In Scheduled mode the sprinkler waits for the programmed day and time to turn on the appropriate zones for the appropriate length of time unless the ground is already wet or rain is in the forecast.

The Remote Control

The remote control is the only way to program the sprinkler since it doesn’t have any UI for this task.  There can be many different devices that serve as the remote control but I intend to use my Samsung Focus Windows Phone 7 for this purpose. 

The application on this device just needs to send HTTP Get and Post requests.  Depending on the type of request a JSON message might be required in the body of the request (i.e. sending data to the sprinkler).  Also depending on the type of request the response may contain JSON(i.e. returning data from the sprinkler).

I chose to use HTTP and JSON as the communication mechanism between the remote control and the sprinkler so that I could remain platform independent.       

Connecting the Remote to the Sprinkler

The Netduino sprinkler sits behind my home firewall.  If I want to talk to the sprinkler with a device that is not behind the firewall then things start to get a little painful.  I would basically have the following options:

  • Don’t expose the sprinkler to the outside world (kind of limiting).
  • The sprinkler microcontroller would have to poll some server on the internet for any new messages that it should process (lots of busy work for the controller).
  • Punch a hole in my firewall so I can get through it from the internet (can you please hack me).
  • Use Windows Azure Service Bus(no brainer).

The Service Bus allows me to make outbound connections to Windows Azure cloud infrastructure and it keeps that connection open so that any external device can make remote procedure calls to the endpoint behind the firewall. I have decided to use the v 1.0 release of service bus for now, but in the future I could see this changing where I would use more of a publish/subscribe messaging infrastructure (which is in a future release of service bus) rather than a remote procedure call.

To leverage the Service Bus you must have a Host that sits behind the firewall and makes the connection to the Azure cloud platform.  For the purpose of this post I am calling this service the Home Connector.  The responsibility of this service is to connect to the Service Bus as a host so that it can accept remote procedure calls from a client.  The client in this case I call the remote connector.

The Home Connector

The Home Connector is a windows service that runs on one of my windows machines behind my firewall.  When a Remote Procedure Call comes in it is converted to an HTTP Get or Post JSON request that is sent to the Netdunio sprinkler.  The response from the Netduino is then parsed and returned back to the RPC caller.  This routing of Service Bus messages to devices behind my firewall is built with the mindset that more than one Netduino microcontroller will be servicing RPC calls from a remote device over the internet.  So this architecture is not limited to just the Sprinkler System.  I intend to add more microcontrollers in the same manor and register them with the home connector so that they too can service RPC requests. 

The Remote Connector

I could have skipped this layer between the phone and the sprinkler.  Since the phone would not be able to use the Service Bus DLL’s directly I could have used the Service Bus WebHttpRelayBinding which would allow me to submit messages to the bus over a REST style api directly from the phone.  But I wanted another layer between the Phone and the Sprinkler so that I could cache some of the requests to prevent my sprinkler from getting bombarded with messages.  I needed a lightweight web framework that would make creating HTTP Get/Post JSON messages easy.

I choose to use the NancyFX framework because it seemed to fit the bill of being quick and easy to get up and running.  That sure was the case when I pulled it down and started building out the first HTTP Get handler.  I simply created an empty web site and used nugetto install NancyFX into this existing blank site.  After that I created a module class and defined my routes and handlers for the routes and I was running with my first Get request in about 15 minutes.  The NancyFX framework also handled processing my JSON messages with very little effort on my part.  All I really needed to do is have a model that represented the JSON message and performed a bind operation on it and the model ended up fully populated.  I haven’t tried to play around with caching the responses yet but I don’t think that will be too hard.

It is important to understand that this remote connector does not have to be on an Azure web role to work.  I could easily deploy this web site to another hosting provider that might be a little cheaper to use.

Conclusion

The Netduino, Service Bus and NanacyFX web framework all seemed to be pretty easy to get me going on connecting devices in my home to my phone.  At the time of this post I haven’t finished the sprinkler system but I got an end to end example of using the Windows Phone to control my Netduino behind my firewall without punching any holes in my router.  I spent most of my time working out the JSON parsing issues across multiple devices then actually getting the infrastructure in place.

This opens up a whole new world of possibilities for me of connecting multiple home devices to my phone and other services.  Before I go to a multiple device household I will most likely move away from the RPC calls and introduce a more publish/subscribe model of passing messages around.  That way I can decouple the message producers from the message consumers.  I will probably wait for the newer Azure Service Bus bits before I tackle that problem though. 

One thing that I started to think about while doing this project is how much smarts (code) should I be placing in the Netduino device.  Right now I have a considerable amount of code that performs all the scheduling functionality in the Netduino.  So once the Netduino receives its pre-programmed schedule it basically can run without any other communications from the outside world (as long as the power doesn’t cycle).  However the scheduling functionality that is built into my sprinkler code is kind of limiting.  If I wanted to add more features to the scheduling functionality it would require me to build a lot of the logic into the Netduino sprinkler code.  This also means I need to deploy more bits to my sprinkler device.  As you can imaging this could develop into a deployment nightmare if a lot of customers are using this product.  There are ways to solve that kind of deployment issues by automating the update process but another solution is to remove the scheduling smarts from the sprinkler device itself and place that logic into a cloud service.  Basically the sprinkler device would know nothing about a schedule and it would be told when it should turn on and how long the zones should run for.  This would eliminate a lot of code that is on the device and make it easier to add new features to the service.  Of course that means the sprinkler device has to be connected to the internet at all times in order to work but that’s doable.  Well I don’t intend to move in that direction yet but I think once I finish out the original design I will explore building out a Home Automation as a Service (HAAS) model.

Keep a watch on my blog for the future posts where I will be diving deeper into each layer of the system and showing some code.  Also I will be posting the source code to the project at some point for others to see.

Unit Testing Netduino code

by Mike Linnen 20. March 2011 20:19

I really enjoy being able to write C# code and deploy/debug it on my Netduino device.  However there are many cases where I would like to do a little Test Driven Development to flush out a coding problem without deploying to the actual hardware.  This becomes a little difficult since the .Net Micro Framework doesn’t have an easily available testing framework.  There are some options that you have in order to write and execute tests:

Well I have another option that works with the full blown NUnit framework if you follow a few conventions when writing your Netduino code.  This approach does not use the emulator so you need to be able to break up your application into two different types of classes:

  • Classes that use .Net Micro or Netduino specific libraries
  • Classes that do not use .Net Micro or Netduino specific libraries.

This seems a little strange but another way to look at the organization of your classes is that if any code does any IO then it belongs in the non testable classes.  Any other code that performs logical decisions or calculations belongs in the testable classes.  This is a common approach that is done in a lot of systems and the IO classes are usually categorized as a hardware abstraction layer.  You can use interfaces to the hardware abstraction layer so that during unit testing you can fake or mock out the hardware and simulate conditions that test the decision making code.

Enough talking about approaches to unit testing lets get going on an example project that shows how this will work.  For this example I am creating a Netduino application that reads an analog sensor that measures the intensity of light and turns an LED on or off.   

Here is how the solution is set up so that I can do unit testing.

The solution consists of two projects:

  • Netduino.SampleApplication - a Netduino Application built against the .Net Micro Framework
  • Netduino.SampleApplication.UnitTests – a .Net 4.0 Class Library

The Netduino.SampleApplication.UnitTests project references the following:

image

Notice that this unit test project does not reference the assembly that it will be targeting for testing.  This is done on purpose because a .Net 4.0 Assembly cannot reference an assembly built against the .Net Micro Framework.  The project does reference the NUnit testing framework.  

Now lets talk about the class that we are going to write tests against. Since analog sensors can sometimes be a little noisy I wanted to take multiple samples of the sensor and average the results so that any noisy readings will be smoothed out.  This class is able to accept sensor readings and provides an average of the last N readings. 

Here is the AnalogSmoother class

image

This is a pretty simple class that exposes one operation called Add and one property called Average.  One thing to notice is that I have removed any using statements (Microsoft.SPOT) that would make this class .Net Micro specific or Netduino specific. 

To test this we need to use a cool Visual Studio feature called “Add as Link” where you can add an existing class to another project by linking to the original file.  If you change the original file the project that has the linked file will also see the change.  To add the linked file you simply right click on the Netduino.SampleApplication.UnitTests project and select Add –> Existing Item and navigate to the AnalogSmother.cs file and select the down arrow on the Add button.

image

So now you have a single file that is compiled in the Netduino project and the Unit Test project.  This makes it very easy to create a test fixture class in the unit test project that exercises the linked class. 

Here is the test fixture class:

image

So I was able to test this class without starting up an emulator or deploying to the Netduino.  This is great for classes that do not need to perform any IO but eventually you are going to run into a case where you need to access the specific hardware of the Netduino.  This is where the hardware abstraction layer comes into play. 

In this sample application I created the following interface:

image

Here is the class that implements the interface and does all the actual IO:

image

Here is the class that uses the IHardwareLayer interface that has some more logic that can be tested using the same approach of adding the linked file to the unit test project.

image

This class will have to be tested a little differently though because it actually expects the IHardwareLayer to return values when calling ReadLight.  We can simulate the hardware returning correct values by providing a fake implementation of the IHardwareLayer interface.  This can be done easily by creating a FakeHardwareLayer that implements the IHardwareLayer and returns the expected values.  Or you can use a mocking framework such as Moqto do the work for you.

image

The Moq mocking framework allows you to Setup specific scenarios and Verify that those scenarios are working.  The above test verifies that the LED does turn on and off for specific values of Light Readings.

Conclusion

I have been able to show you that unit testing is doable for Netduino projects if you follow a couple design patterns and you don’t have to wait for a testing framework to be available for the .Net Micro Framework.   

UPDATE: I made a couple small tweaks to the code and posted it on my NetduinoExamples repository under the UnitTestingExample subfolder.

About the author

Mike Linnen

Software Engineer specializing in Microsoft Technologies

Month List