Lightweight Messaging for a Connected Planet

Lightweight Messaging
for a Connected Planet
Andy Piper
WebSphere Messaging Community Lead, IBM

The Internet of Things

Trillions of smart
devices instrument
our world today

Interconnecting these
smart devices creates
a Central Nervous
System

Building a Smarter Planet

+ + =

The Networked Enterprise
Connectivity for Applications
Head Offices,
Transport WebSphere MQ Data centres
REST/HTTP Intelligent Cloud Computing
WS*

Mission-Critical Connectivity & Intelligence

Messages WebSphere MQ
Regional Offices
Stores, Outlets Files WebSphere MQ File Transfer Edition
Interconnected
Transform, WebSphere Message Broker
Enrich, WebSphere ESB
Mediate DataPower
WebSphere Sensor Events

Catalog WebSphere Service Registry & Repository

Intelligence WebSphere Business Events
Instrumented Remote Systems and Devices Cognos. ILOG, SPSS
InfoSphere Streams
Connectivity for Smart Devices Other InfoSphere and Tivoli products

Transport MQ Telemetry
HTTP
Multicast Actuators
Embedded Controllers Sensors
Power meters, weather data Tag printers, status lights, Load
Millions of Filtering of duplicate read events, Store-
based HVAC & lighting controls, SCADA sensors, pressure, volume, generation, HVAC and lighting,
RFID readers, Motion detectors… Valves, switches and pumps…
devices Industrial Network Gateways (SCADA)

MQTT =
MQ Telemetry
Transport

Design principles
■ Publish/subscribe messaging paradigm
(useful for the majority of sensor
applications)
■ Minimise the on-the-wire footprint.
■ Expect and cater for frequent network
disruption – built for low bandwidth, high
latency, unreliable, high cost networks
■ Expect that client applications may have
very limited processing resources available.
■ Provide traditional messaging qualities of
service where the environment allows.
■ Publish the protocol royalty-free, for ease
of adoption by device vendors and third-
party software developers.

Key facts
■ Low complexity and footprint
■ Simple publish/subscribe messaging semantics

Asynchronous (“push”) delivery of messages to applications

Simple verbs: connect, publish, (un)subscribe, disconnect

Minimised on-the-wire format

Plain byte array message payload

No application message headers

Protocol compressed into bit-wise headers and variable
length fields

Smallest possible packet size is 2 bytes
■ In-built constructs to support loss of contact between client and
server

“Last will and testament” to publish a message if the client
goes offline

Stateful “roll-forward” semantics and “durable” subscriptions

Good point.
Here's a (very) quick
comparison.

Data-centricity

MQTT is agnostic of data content and transfers
simple byte arrays, making drip-feeds of
updating information trivial.

HTTP is (basically) document-centric.

Simplicity

MQTT has few methods
(publish/subscribe/unsubscribe) and is quick to
learn.

HTTP can be complex (although it is often well-
understood) - there are a multitude of return
codes and methods.
REST is a great principle but not always the best
for simple data applications
(POST/PUT/GET/DELETE? er what?)

Light on the network

The smallest possible packet size for an MQTT
message is 2 bytes.
The protocol was optimised from the start for
unreliable, low-bandwidth, expensive, high-
latency networks.

HTTP is relatively verbose - lots of "chatter" in a
POST

Easy distribution of data

MQTT distributes 1-to-none, 1-to-1 or 1-to-n
via the publish/subscribe mechanism
→ very efficient

HTTP is point-to-point (can be
mediated/clustered but no distribution
mechanism). To distribute to multiple receivers a
large number of POSTs may be required.

Lightweight Stack (CPU/Mem)

MQTT has been trivially implemented on tiny to
larger platforms in very small libraries
[IBM ref implementation = ~80Kb for full broker]

HTTP (often with associated XML or JSON
libraries for SOAP and REST etc) can be relatively
large on top of OS network libraries
Plus... even if the client is small, consider
whether it is really necessary to run an HTTP
server on every device

Variable Quality-of-Service

MQTT supports fire-and-forget or fire-and-
confirm (aka QoS 0/1/2)

HTTP has no retry / confirmation / attempt at
once-only delivery. It is basically brittle, i.e. retry
needs to be written in at the application level.
Applications must also handle timeouts.

Sounds like there's a
place for both!
Where is MQTT in
use?

•
Simple
•
Lightweight (CPU,Mem,**Net)
•
Data-centric
•
Distributes data (pub/sub)
•
Range of QoS
→ strong developer community

“strong developer
community” huh...

Ok, to be fair, I have
no knowledge of their
physical strength, but
they are all awesome...

Home automation

http://chris.yeoh.info/?p=188

Gardening

“It all started with the seemingly
simple question – “How can I water
the garden without leaving my
laptop/phone/sofa using tech?””
- Dan Fish

http://www.ossmedicine.org/home_automation/arduino/12/watering-the-garden-oss-style-a-year-with-some-open-hardware/

TV, Android, burglar “control”

http://www.hardill.me.uk/wordpress/?p=204

Mind-controlled Taxis

b

“Kevin already had the headset
hooked up to MQTT, so it would be
trivial to use my Arduino MQTT
library to get them all talking.”
- Nick O'Leary

http://knolleary.net/2010/04/22/how-i-got-onto-prime-time-bbc-one/

Flashing Arduino-controlled ducks
“Now, you may wonder why I
would want 20 rubber ducks to
flash when my phone goes off....
There is no scientific or technical
reason in itself. I just had a Mini
Cooper’s worth of rubber ducks
sitting around, unemployed.”
- Chris Phillips

http://eightbar.co.uk/2009/03/12/the-amazing-mqtt-enabled-ducks/

This sounds
moderately
interesting (and fun)
Lemme at it!

Seriously.
We've been waiting
for 25 slides
already.
CODE, DAMMIT

The IBM way
•
http://www.alphaworks.ibm.com/tech/rsmb
•
Download rsmb-1.2.0.zip
•
Unzip
•
Run nohup ./broker >> /dev/null &
•
Play with C client utils, code, and be merry!

•
Available for Linux IA32, IA64 kernel 2.6.8+; Linux on IBM System z;
Linux for ARM XScale, kernel 2.0.0+ (Crossbow Stargate or Eurotech
Viper); Windows XP; Mac OS X Leopard; Unslung (Linksys NSLU2) –
Binary only, request other platforms from IBM

Alternatively...
•
http://mosquitto.org
•
On e.g. Ubuntu:
sudo add-apt-repository ppa:mosquitto-dev/mosquitto-
ppa && sudo apt-get update && sudo apt-get install
mosquitto
(optional: mosquitto-pub, mosquitto-sub, python-
mosquitto)
•
Runs as a daemon; IPv4/IPv6-capable
•
C++ and Python examples available

•
Packaged for Ubuntu, Fedora, RHEL, OpenSuSE, CentOS, Debian,
Mandriva; Windows - binary; OS X – binary (homebrew compile via
github package); source tarball; dev version in bitbucket

Client APIs
•
C
•
C++
•
Java (your route to Android and Kindle goodness)
•
.NET (yes, Mono too...)
•
Delphi
•
Perl

•
Python
•
PHP
•
Ruby
•
Erlang
•
Arduino
•
mbed

Show us the code!

public void sendAMessage() throws MqttException {
MqttProperties mqttProps = new MqttProperties(); Create a connection using the
mqttProps.setCleanStart( true ); connection factory, this time
MqttClient client = MqttClientFactory. INSTANCE. for a clean starting client
createMqttClient("testClient",
“tcp://localhost:1883”, mqttProps);
Register the class as a listener and
client.registerCallback(this);
connect to the broker
client.connect();
client.publish(“abc/123”, new MqttPayload((“Hello World!”).getBytes(),0),
(byte) 2, false);
client.disconnect(); Publish a message to the
} given topic and disconnect

public void publishArrived (String topicName,
MqttPayload payload,
byte qos, boolean retained, int msgId) {
System.out.println(“Got it!”);
} On receipt of a publication,
simply print out a message on
the console to say we received it

Moar code plz
#!/usr/bin/python
import pynotify
import mosquitto Setup a callback – post a
# define what happens after connection desktop notification with
message topic and payload
def on_connect(rc): when a publication arrives
print "Connected"
# On receipt of a message create a pynotification and show it
def on_message(msg):
n = pynotify.Notification (msg.topic, msg.payload)
n.show ()
# create a client connection
mqttc = mosquitto.Mosquitto("python_sub")
# define the callbacks
mqttc.on_message = on_message
mqttc.on_connect = on_connect Set a client id (“python_sub”),
connect to localhost, and
# connect subscribe on topic “test”
mqttc.connect("localhost", 1883, 60, True)
# subscribe to topic test
mqttc.subscribe("test", 2)
# keep connected to broker
while mqttc.loop() == 0:
pass

http://chemicaloliver.net/programming/first-steps-using-python-and-mqtt/

Community
•
http://mqtt.org (including wiki)
•
rsmb forum at IBM alphaWorks
•
#mqtt on freenode
•
mosquitto project on launchpad

•
http://homecamp.org.uk
•
many bloggers, developers, etc...

More random-but-cool schtuffs
•
File sync over MQTT?
http://mquin.livejournal.com/177855.html

•
Desktop notifications
http://ceit.uq.edu.au/content/mqtt-and-growl and
http://chemicaloliver.net/programming/first-steps-using-python-and-mqtt/

•
Web thermometers
http://chemicaloliver.net/internet/mqtt-and-websocket-thermometer-using-the-html5-meter-tag/

•
Digital-to-analogue readouts
http://chemicaloliver.net/arduino/mqtt-and-ammeters/

•
CEIT @ UQ research projects
http://ceit.uq.edu.au/content/messaging-protocol-applications

•
LEGO microscope control
http://eprints.soton.ac.uk/45432/

Elsewhere in the
world, exciting things
were occurring...

Integrating to the Enterprise
WebSphere MQ Telemetry
Sensors

DIRECT
MQTT CLIENT
WebSphere Mobile, smart
meters
MQ 7.0.1
MQXR Applications

BRIDGE
Rich clients MQTT
BRIDGE
requiring buffering, WebSphere MQ Telemetry
remote management Daemon for Devices DIRECT
capabilities, or MQTT CLIENT
Microbroker
advanced data handling (Lotus Expeditor,
WebSphere
Sensor Events)

Smarter Healthcare
Medical organization created a remote pace-maker monitoring
solution to provide better patient care

Client Pains
 Physicians needed better monitoring
of cardiac patients
 Improve efficiency of checkups
 Meet healthcare data capture
standards

Enables higher level of patient care and peace of mind
Improves administrative efficiency and maintenance
Helps conform to standards and ease integration of data

Smarter Energy
Utility company developing an Intelligent Utility Network offering for
optimizing load on electricity grids

Business Partner
 Needs robust middleware
technology to connect to remote
smart meters
 Needs to be able to rapidly scale
solution nationally

Able to offer daily energy savings of 15-20%
Enables utilities to reduce peaks and avoid punitive charges
Helps save electricity through better peak load management

KTHXBAI!
Andy Piper
@andypiper
http://andypiper.co.uk

Thanks!!
•
Roger Light @ralight (mosquitto awesomeness)
•
Nick O'Leary @knolleary (Arduino/MQTT awesomeness –
images from Flickr – and general uber-awesomeness)
•
Chris Yeoh @ckbyeoh (home hacking awesomeness)
•
Benjamin Hardill @hardillb (TV hacking awesomeness)
•
Chris Phillips @cminion (Rubber Duck awesomeness)
•
Oliver Smith @chemicaloliver (lots of webby awesomeness)
•
Dan Fish @ossmedicine (garden awesomeness)
•
Donna Benjamin @kattekrab (Inkscape awesomeness)

Lightweight Messaging for a Connected Planet

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (9)

More from Andy Piper

More from Andy Piper (20)

Recently uploaded

Recently uploaded (20)

Lightweight Messaging for a Connected Planet

Editor's Notes