Tag Archives: agile

4 Elements of a Successful Open Workspace

In recent years, many people have written about a wide range of experiences with Open Space Work Configurations. Some have experienced benefits to productivity, innovation, and collaboration, while others have witnessed a decrease in productivity, team morale, and focus. This discrepancy in outcomes has resulted in a facile argument heard in offices across the world: “Yeah an open workspace works for some companies, but it would never work here.”

 

Image result for the office jim and dwight“Our office is…special.”

This difference in experiences is not necessarily due to limitations in the Open Workspace Concept, but rather a misunderstanding in their application.

I have been fortunate enough to be in and around over 40 uniquely different space configurations (some less “open” than advertised) both early in my career as a member of different teams, and later as a consultant helping others create effective space.  I have witnessed amazing improvements in collaboration, productivity and morale from successfully implemented open space settings, as well as the fallout from poorly implemented ones.

The successful configurations all had four common concepts working for them:

 

The “Open” SpaceThe Main Collaboration Areas

2-hootsuite-blog-0534Flexibility is key here.  Flexibility allows for those who use a space to own its function, and ownership contributes greatly to the initial and continued success of any space. The more flexible the space, the broader range of activities it can accommodate.

Improper planning and implementation of an open space area can create a very limiting and sometimes chaotic environment.  To avoid this, an effective open space should be designed to allow teams to conduct many types of work efforts.  This helps promote a layout that is practical, dynamic, versatile, profitable, and fun.  Also, you don’t need to spend an inordinate amount of money to create an effective open space.  Simplicity, flexibility and diversity of configuration should always be a top focus. Save your money for talent, quality tools of your trade, and comfortable chairs!

 

The “Other” Space – Complementary Quiet Gathering and Break-Out Spaces

The “open” space is only part of an effective space.  An effective structure is a blending of open areas and private gathering spaces.  Ensuring sufficient “other” space enables private conversations, group break-out sessions, or occasional quiet/focus time, all of which are necessary to maximize any team’s potential.

This structure also allows for better utilization of many existing configurations. I’ve seen effective setups where offices and cubes which were primary work spaces become breakout/quiet space and former large meeting areas become the base locations of teams to gather and work.  Open space without complementary gathering/break-out space will fall short of achieving gains and may fail altogether.

 

Utilization of the Space Complementary Techniques, Tools, Ceremonies, and Cadence

get-a-way-spaceA great space (open and other) does not guarantee team success on its own.  In many cases, the open space concept and structure is completely foreign to those expected to utilize it.  Teams need to learn how to effectively leverage the newly designed space in ways that enhance productivity and innovative thinking.

Learning and leveraging complementary techniques for working and collaborating in this new environment are critical to gaining early positive momentum within the space and key to achieving sustainable success within it.

 

 

 

 

Focus on Sustainability Proper Mindset, Team Dynamic, and Organizational Support

04_pair_programming_with_junior_developers

So you have the space, the techniques, and the talent…but will it last?  Companies have started open space concepts with successful early outcomes only to watch the benefits fade over time.  If given enough time to mature, the success of an open space concept will become part of the cultural dynamic of the organization.  Organizational and cultural support for the approach during the early stages of learning and over a sustained period of time is necessary for long-term success.

Eventually, given time, a productive space teaches leaders and talent that it’s less about the way people work together in a specific part of the building and more about the way people work together period.

Is Your Office Space Hindering Your Employees?

If you work in a traditional office, you’ve probably experienced the frustration of wandering through a maze of cubicles like a lab mouse in search of cheese, trying to find the one co-worker who can answer your question. It only gets more irritating when you discover that said co-worker has abandoned their own post in search of someone else to answer their question. With a heavy sigh, you trudge back to your seat, punch out your query in an email, and watch time tick away as you wait for a reply.

20080409_cubiclesfromofficespace

The Age of the Open Workspace

Despite our progression towards an increasingly virtual world, face-to-face communication is still extremely valuable. Relying predominantly on emails for collaboration results in misunderstandings and really, REALLY slow feedback loops.

In open workspaces, teams use open seating to facilitate communication, shorten feedback time, and speed up ideation. It is often the hardest practice to implement when making internal process improvements, but the resulting transparency and increased communication make it incredibly impactful.

anigif_enhanced-25762-1428527325-22

Okay, bad example.

Assess Your Space

If you are considering transitioning to a more open environment, you’ll want to put some thought into what isn’t working now and what you anticipate a more open office space would help your team achieve. The following are some questions to help determine how your current workspace configuration affects collaboration and to get your juices flowing around how to turn your space into a productivity tool.

OfficeSpace_006Pyxurz

*sigh*

To get started, ask yourself:

  • How quickly can you currently get questions answered or receive feedback on requests from your peers? How about your management? Senior leadership?
  • How many emails back and forth does it take to resolve an easy issue? How about a complex one?
  • How often do you find yourself picking up the phone or typing out an email to speak to someone down the hall or across the room?
  • Do most of the steps on your Fitbit come from wandering around the office looking for team members?
  • How many meetings are scheduled simply so you can get all the right people in a room together to work on a single issue?
  • Do you see pictures of offices like Google or QuickenLoans and think “God, I wish I could work in a place like that?”
  • Do your teams keep stashes of vitamin D in their desk drawers because the only time they are exposed to natural light is during their drive to work?

Open Space Guidelines

Depending on your answers to the above, you may be wondering what a more collaborative, optimized environment would look like. Below are some guidelines to creating a space that breaks down the barriers to communication and teamwork. Note: These are generally considered good practices, but they are not the “rules.” The team (not the boss or HR) should be allowed to work together to create an environment that best supports their needs and goals.

9-Office-Space-quotes

No.

Workspace

  • There are no assigned seats or other personal spaces (ie. cubicles)
  • Breakout rooms are available for meetings, privacy, or periods of quiet-time, but should not to be turned into personal offices.
  • White boards, cork boards or flip charts are used to provide a dynamic workspace for collaboration.
  • Team members use open seating to facilitate pairing on tasks and projects
  • Leadership sits in the open space with the team (no ivory towers)
  • Locate the team space in a highly visible area, be proud to showcase the space to visitors
  • Team members have a designated area to store personal belongings – this prevents “claiming” of a seat or area
  • Furniture is movable without permission and the team has ability to reshape the space as needed
  • Allow picture skins or stickers on laptops (family, dogs, sports, etc)
  • Allow teams to be creative and introduce fun into the workspace

Team Norms

  • Some noise indicates collaboration and communication, silence is troubling
    • The Parent’s Ear Rule: bad sounds are complete silence or kicking & screaming
  • “Hey Team” cooperation when roadblocks are discovered – Everyone swarms to help out
  • Teams sit side-by-side and/or facing each other, not with backs to one another
  • Requires that facilities, HR, and leadership understand the goals, and are engaged with the team(s)
  • All rules deserve team discussion

A Note On Introverts and Open Spaces

One of the biggest misconceptions about open workspaces is that they do not accommodate the preferences of introverted team members. The fact is everyone, even extroverts, sometimes need to work independently and in quiet so that they can focus. For many, the idea of working in an open environment with lots of “chatter” can be a rather unpleasant concept. The good news is that properly configured open workspaces, combined with the right team norms, can actually support the needs of introverts even more than traditional offices currently do.

milton1

“Go on…”

First, break-out rooms or designated quiet areas allow team members to retreat to a private place to focus on a particular task without disruption, then return to the group for collaboration when they are ready.  Second, having no assigned seating means that people are free to move away from distracting activities or noise rather than be forced to deal with it all day. Finally, the freedom to choose where one works conveys a stronger sense of empowerment and trust from company leadership. It’s as if you are saying to your employees, “I see you as adults, not schoolchildren. This is your space and I trust that you know best how to get things done.”

How It All Fits

While the goal of creating a more productive, collaborative environment is certainly worth pursuing, it should be noted that simply taking kicking down your cubicle walls while declaring “open workspace!!” will not do the trick.  It will, however, get you some frightened looks from your employees and a rather stern talking-to from HR.

office-space-gif

Pictured: Not the answer.

The implementation of an open workspace is not a solution in itself. Rather, it is a critical step in a larger movement towards a more optimized operation. Some would even argue that the switch to an open workspace is really just a byproduct of adopting more collaborative, transparent practices – that it naturally emerges as processes become more streamlined. It is therefore critical to define what your goals are for your open workspace and to consider what operational changes will need to take place in order to ensure the transition is successful.

If you are considering optimizing your office space and processes, a good place to start is by taking a quick assessment of your current situation with this one-minute quiz. The results will give you a good idea of how adaptable your organization is today and provide steps you can take to make incremental improvements.

 

6 Tips To Designing Effective Information Radiators

SWRREC0K3A

Your team space should deliver a message

Recently I’ve been helping a new team setup their team space. Now, I’m a big fan of hanging stuff on walls. Big visible stuff. But there’s more to it than that. When deciding what you should put where you are actually crafting a message to anyone that walks into the space.

My personal metric for knowing when you’ve done this effectively is to answer the following question: Does it change someone’s mood?

In my opinion, when anyone walks into your team space, from the newest dev to the most senior executive, what’s hanging on the walls should make them feel differently within a matter of minutes. If that doesn’t happen, something’s off.

Now I’m not advocating that teams throw a bunch of crap up to simply appease people, especially those not usually in the space. In fact, the big visible stuff should be minimalistic; the least amount of information required to paint a deep, rich picture of what is going on, what value is being added, when things are happening, and anything else that is useful.

I’m not going to go into what specifically any of the stuff should be. There is tons of information out there on that such as information radiators, styles of team boards, card maps and lots more. Instead, I encourage you to think about the message you want your space to convey.

Here are a few things to consider in addition to the actuall stuff you choose to hang up:

1) If it’s hanging up, make sure it’s actually big and visible 

Sometime I wish plotters were never invented. Release burn-down charts and code coverage trends are examples of useful things to hang up. Too often though, I see people (usually PMs with whiz-bang tools) print them out. I assume they do this because they think it’s easier, but walk across to the opposite side of the team space and tell me if you can read it…all of it. It’s not enough to see the trend lines if you don’t know what the chart is trending or what the axes are. Instead, draw it out on some paper or a white board with a big fat marker. Now go walk across the room…yeah, betcha can read that!

2) If it’s hanging up, make sure it’s useful

like to think of this as pruning. Something that was once useful may have run its course. If so, tear it down. If you need it later, recreate it. However, before just tearing stuff down, make sure the whole team agrees that what ever it is is no longer useful. When in doubt leave it up, there are usually ways to make more room if you need it. It’s also important for the team to consider organizational usefulness. Not everything will be the most useful to the team itself, but sometimes to managers or other stake holders. PMs like release burn downs and cost burn-ups, CFOs like value stories etc… These should not dominate the space, but they are still useful, and it’s important for the team to understand their organizational usefulness as well 🙂

3) If you need something useful, make sure you hang it up 

The space is not a fixed thing. Obviously if we can tear stuff down we can put new stuff up. The process of software development is journey or learning and discovery. Visualizing different things along the way can help a team communicate, both with each other as well as those outside the team. If you think something might be useful, hang it up for a while, try it out. If it doesn’t add the value you thought, ask how it can be improved. If after a while it is still not providing value, see #2.

4) If it’s hanging up, make sure it’s in a good position 

Not all wall space is created equal. This could be due to many things such as lighting, vantage point, furniture arrangement etc… The best thing to do is plan a little bit before you hang something up. How often will it be updated? Is it a conversation centerpiece? Should it be visible to a passer-by? Once decided, go hang it up. If things change and it needs moved, move it…it’s only paper and tape right?

5) If it’s hanging up, take pride in making it

Remember, you are crafting a message. You want things to be visible, digestible and useful. Those traits can be hard to achieve if your big visible stuff looks messy, half-assed or cluttered. It doesn’t take that long to use a straight edge instead of free-hand drawing. Create a color scheme of post-its or stickers. Use different size index cards to mean different things. Create a legend. It doesn’t have to be a work of art, but it should be tidy (within reason) and professional looking to communicate your message effectively. If you think something has gotten too messy over time, clean it up or redo it…it usually doesn’t take very long.

6) If it’s hanging up, it’s a living document

Don’t be afraid to enhance anything that’s hanging up. Feel free to draw, stick stickers, add post-its or whatever else adds value. You can usually tell which documents (big visible charts) a team find the most useful by the amount of enhancements made to it!

Finally, don’t forget to test this stuff out when you think you are done. Stand back from your space and look at it. What does it say to you? Stand in the doorway or hallway and look in. What does it look like from there? Have people from other teams walk through. Is there anything that is unclear to them? Ask a manager, director or executive to walk by. What were they able to tell by just looking? Maybe do a few of these every now and then as your space evolves with different stuff.

Ultimately, don’t forget that original question from way up there: Does it change someone’s mood? Folks should feel better about what the team is doing, where the project is headed, and what they are getting for all the team’s effort. If this is the case, then you’ve successfully crafted both an effective team space as well as an effective message.

This post was originally published by the Matt Barcomb on odbox.co and was reposted with permission. 

Preaching To The Choir

church-choir-408412_960_720

Recently, I was giving a class and during a portion of the class I discuss some of the cultural changes usually required for organizational progressive elaboration of work. During the break that followed this particular discussion, one of the attendees came up to me and said:

“Seems you’re preaching to the choir here. We all agree with what you’re sayin’…but it wont do any good unless we get the managers in here.”

While I certainly understand this reaction, and unfortunately find that it’s not that uncommon, I still find it a bit disheartening…and a little frustrating.

This attitude of “I can’t change the things that influence me”, and “what I can control isn’t big enough to really change anything” is entirely the wrong attitude.

Changing the things that you can control, no matter how seemingly insignificant, should never be down-played. One can always take complete control of their own actions, behaviors, and reactions. Combined with a little learning and a bit of creativity this can become very powerful indeed. So, try changing the things you can; the things over which you have direct control. Share what you’ve tried, what you’ve learned, what worked well and what didn’t. Share up, share down, make it visible, let people ask about it and then share with them too. Eventually this can influence up, down, and across the network of an organization.

Sometimes it only take one person to make a significant change. Other times it spreads slowly before snowballing into a major organization shift. And many times making a “major” or “significant” change isn’t even needed. Sometimes it’s the little things that count.

So, the moral of the story is that even the choir can use the message, and even the choir is able to go out into the world and try to make it a better place through their own actions…and employees who think their organization could be better are just as much on the hook for trying to improve it as the managers and executives are.

This post was originally published by the Matt Barcomb on odbox.co and was reposted with permission. 

Does Your Team Have a Blame Well?

Here is an important role for every budding team…It’s called the “Blame Well”.images-1
Now, the way it works is, each week the team rotates the role of Blame Well to a new team member. During that week, if anything goes wrong that team member immediately assumes all blame for anything that goes amiss.

The role of Blame Well enables the team and stakeholders to immediately get past trying to figure out whose fault anything is. Instead, the team can directly move into useful discussion about resolving whatever issue came up.

So…this post is definitely tongue-in-cheek. I think if a team really needed a Blame Well there are deeper problems to address.

What I hope folks can take away from this is that focusing energy on assigning fault and blame is pretty pointless. A better approach is to simply figure out what the next best thing can be done in the current context.

This doesn’t mean teams shouldn’t inspect and adapt…but true reflection and growth happens without fault and blame.

I’d enjoy hearing stories about how people helped their own teams or organizations get out of the blame game.

This post was originally published by the same author on odbox.co. It was reposted with permission because we threatened to throw him in the blame well for a month if he said no. 😀

The Multitasking Myth

As a parent of an ADD (Attention Deficit Disorder) child, I have had the unplanned but eye-opening experience of learning how to deal with ADD.  Why eye-opening?  I have to admit, I have always been skeptical of the validity of some of today’s conditions that become accepted by the medical community at large.  The sheer rate of new conditions grows each year.  When my child was first diagnosed, I wondered if ADD was truly a legitimate issue or if it was created by pharmaceutical companies who conveniently happened to have a medication to manage the condition.  It also seemed to me like a convenient excuse for those who were lazy, unmotivated, or simply capable of handling the daily demands of the modern world.  I have since learned that I was completely wrong.struggling

After experiencing the effects of ADD has on my child and consequently putting together a plan to manage it, I have seen a 180 degree turn in my child’s ability to deal with the anxiety that accompanies ADD.  My child has gone from a very challenged student to a peak performer at school almost immediately.  Also, my child’s satisfaction level with achievements and self-confidence is at an all-time high.

There are three things we implemented which have directly contributed to the successful turn-around:

  1. A sustainable and recognizable daily routine
  2. Prioritizing what is most important, communicating it to our child, and focusing on that list one item at a time
  3. Constantly re-evaluate #2 and adjusting  accordingly

Since being introduced to ADD I have become familiar with the effects it has on performance, self-satisfaction, and self-confidence.   I have also noticed similarities between these effects and the effects of multitasking on performance, self-satisfaction, and self-confidence in the workplace.  Over the years, I have even seen a number of cases of what one could term as “artificially-manufactured ADD”.

Why make this comparison?  Because many in the business community treat multitasking in the same way I first treated ADD; the effects on productivity are really over-hyped and those who can’t multitask effectively are just lazy, unmotivated, or incapable of handling the tasks of today’s business climate. 

Multitasking is not a modern concept; in fact it is believed to have been around for a long time.  Today’s work environments drive multitasking demands on our time almost by default.  CNN describes multitasking as “a post-layoff corporate assumption that the few can be made to do the work of many”.  I’m not sure if I completely agree with this viewpoint, but some studies show that multitasking is a less efficient approach to work than focusing on similar types of tasks at the same time, or focusing on one specific deliverable at a time.   There are many suggestions as to how to address and minimize the effects of multitasking or how to operate to avoid it.  In my experience, the best way to minimize performance loss of multitasking is similar to the approach we have taken to counteract the effects of ADD with my child:

  1. Be consistent and predictable wherever possible
  2. Prioritize your work and single-thread your efforts whenever possible
  3. Constantly re-evaluate #2 for updated priorities and adjust when needed

Highly productive teams groom, prioritize, re-groom, and re-prioritize their work constantly.  They also are consistently inquiring about priority and adjusting accordingly.  Most importantly, they work to keep their efforts as single-threaded (one item at a time) as possible to maximize their productivity.   The effect on your group’s performance, as well as your group’s output quality and agility, will be greater than you think.

Want to learn more ways to create high-performing teams? Check out another post by Mike Jebber – Team Building: Diversity Uncovers What Experience Can’t.

Agile…and BEYOND!!

LeanDog Agile experts, Matt Barcomb, Mike Kvintus, and Jeff Morgan, are at Agile and Beyond today and tomorrow. Take a peek at some of the knowledge they will be dropping while they are there:

Screen Shot 2016-05-02 at 4.58.17 PM

Barcomb

Matt (@mattbarcomb) is a product development specialist with a penchant for organization design. He works with companies to turn software development into a core competency by integrating product development activities with business practices. Matt takes a pragmatic, systems approach to improvement, working with stakeholders throughout medium and large organizations. He has experience working with product management and software delivery teams as well as executive leadership teams, sales, services, and operations in a variety of industries. Matt enjoys challenging mental models, simplifying the seemingly complex, and uncovering the “why” behind the what. He shares his experiences, questions and ideas at www.odbox.co

Thursday, May 5 @ 10AM Value-focused prioritization & decision making

Does prioritizing your development portfolio seem unclear or mired in politics? Ever feel like the decisions for what gets worked on when are somewhere between arbitrary and emotional? Ever get tired of providing cost estimates for work of uncertain value? If you answered yes to any of the above questions, this session is for you! Matt Barcomb will open with introductory concepts about shifting from a cost focus to a value focus for development work. Next, providing business value for user stories will be debunked. Then, a collaborative framework for prioritization, Benefit Mapping, will be discussed. Finally, Matt will end with ways to simplify the cost evaluation of work and risk.

Friday, May 6 @ 10AM Using Flow-based Road Mapping & Options

If you’d like an alternative to typical, quarter-by-quarter, schedule oriented road mapping (and all the associated waste) then this session is for you. Matt Barcomb will introduce a Cadenced Flow approach to flow-based road mapping. He will first cover how to layout and execute a road map based on models that better fit software planning as well as how to transform your existing plans. Next, using options thinking to frame work will be explored and how to use starting and stopping triggers for options, reducing the need of blind budgeting or project practices. Finally, Matt will wrap up by touching on a few key metrics that will let you monitor and evaluate your new road map.

Screen Shot 2016-05-02 at 5.00.09 PMCheezy

Cheezy is an international speaker and keynote presenter in different Agile conferences. He has spoken 6 times at the Agile 20XX conferences as well as other ones like Agile development East and West, Mile High Agile, Agile and Beyond, Path to Agility, etc.

Friday, May 6 • 3:00pm – 4:40pm Tested!

You’ve heard that quality belongs to everybody on an Agile team. You’ve heard that testers and developers should “collaborate” in order to drive quality higher. You’ve heard that automated tests help a team continuously validate the quality. It’s time to stop thinking about it! It’s time to stop talking about it! It’s time to make it happen! Watch Ardi and Cheezy do this in front of your eyes. They will build a web application driven by acceptance and unit tests.You will see how a Product Owner, Tester and Developer will create executable User stories, develop the code to validate these stories and refactor along the way. At the end, you will get a taste of what a Continuous Delivery pipeline looks like. Prepare to collect your jaws from the floor!
Screen Shot 2016-05-02 at 4.58.39 PM

Kvintus

Friday, May 6 • 10:00am – 10:45am Worthless Story Card Estimates

How much of your time is wasted estimating story cards? We’ll explore some alternatives to estimating story cards and review real-world comparisons of tracking work using story points vs. counting story cards. Not sure when story card estimates are needed? We’ll discuss that too. All discussions will be based on real-world examples and comparisons of alternatives for several projects. We’ll also discuss #NoEstimates and how it fits in. You’ll leave with an understanding of ways to plan/track agile projects and the tradeoffs involved with alternatives to story card estimates.

IoT Course Week 11: Load Testing

Screen Shot 2015-10-15 at 3.07.21 PM

Last week, we dove into into the importance of incorporating and collecting analytics through your connected device, how that information helps provide business value, and played with some of the ways that information can be displayed using some pretty graphs.

This Week

This week, we’ll continue our focus on non-functional requirements and start load testing. With connected devices, if the device can’t call home to its shared services, it loses a lot of its value as a smart device. These services need to be highly reliable, but things get interesting when thousands or millions of devices decide to call home at the same time.

To load test, we’ll generate concurrent usage on system until a limit, bottleneck, unexpected behavior, or issue is discovered. This usage should model real-life usage as close as possible, so the analytics we put in place last week will be a valuable resource. In instances where we don’t have data to work with, we can build out user funnels and extrapolate based on anticipated usage. Bad things will happen if we ship thousands of products without any idea how our system will react under the load. This data will also be a useful baseline for capacity planning and system optimization experiments.

The Lampi system has two shared services that we need to put under load. One is the Django web server that handles login, and the other is the MQTT broker that handles sending messages to the lamp.

Load Testing with Locust

To test the web server we use Locust. Locust has become a LeanDog favorite due to its simple design, scalability, extensibility, and scriptability. We’ve used it to generate loads of 200,000 simultaneous users distributed across the US, Singapore, Ireland, and Brazil. These simulated users (locusts) walked through multi-page workflows at varying probabilities, modeling the end-to-end user interaction, complete with locusts dropping out of the user funnel at known decision points.

Locusts are controlled via a locustfile.py. The one below shows a user logging in and going to the home page:

from locust import HttpLocust, TaskSet, task

class UserBehavior(TaskSet):

def on_start(self):
self.login()

def login(self):
response = self.client.get("/accounts/login/?next=/")
csrftoken = response.cookies.get('csrftoken', '')
self.client.post("/accounts/login/?next=/", {"csrfmiddlewaretoken": csrftoken, "username": {{USERNAME}}, "password": {{PASSWORD}} })

@task(1)
def load_page(self):
self.client.get("/")

class WebsiteUser(HttpLocust):
task_set = UserBehavior
min_wait = 5000
max_wait = 9000

In order to run locust, we’ll need a machine outside of the system to simulate a number of devices. Locust is a python package, so it can run on most OSes. It uses a master/slave architecture so you can distribute the simulated users and allow for more and more load.

Once you install locust and start the process, you control the test via a web interface.

Screen Shot 2016-05-04 at 12.09.46 PM

Locust will aggregate the requests to a particular endpoint and provide statistics and errors for those requests.  

Screen Shot 2016-05-04 at 12.09.55 PM

Screen Shot 2016-05-04 at 12.10.03 PM

Load Testing with Malaria

To test MQTT we used a fork of Malaria. Malaria was designed to exercise MQTT brokers. Like locust, Malaria spawns a number of processes to publish MQTT messages. Unlike locust, it’s not easy to script; you have to fork it to do parametric testing or randomize data.

usage: malaria publish [-D DEVICE_ID] [-H HOST] [-p PORT] [-n MSG_COUNT] [-P PROCESSES]

Publish a stream of messages and capture statistics on their timing

optional arguments:
-D DEVICE_ID (Set the device id of the publisher)
-H HOST (MQTT host to connect to (default: localhost))
-p PORT, (Port for remote MQTT host (default: 1883))
-n MSG_COUNT (How many messages to send (default: 10))
-P PROCESSES (How many separate processes to spin up (default: 1))

By modulating MSG_COUNT and PROCESSES you can control the load being sent to the broker.

Running some Example loads
Small load: Using 1 process, send 10 messages, from device id [device_id]

loadtest$ ./malaria publish -H [broker_ip] -n 10 -P 1 -D [device_id]

Produces results similar to this:

Clientid: Aggregate stats (simple avg) for 1 processes
Message success rate: 100.00% (10/10 messages)
Message timing mean 344.51 ms
Message timing stddev 2.18 ms
Message timing min 340.89 ms
Message timing max 347.84 ms
Messages per second 4.99
Total time 14.04 secs

Large load: Using 8 processes, send 10,000 messages each, from device id [device_id]

loadtest$ ./malaria publish -H 192.168.0.42 -n 10000 -P 8 -D [device_id]

Monitoring The Broker

MQTT provides a set of topics that allow you to monitor the broker.

This command will show all the monitoring topics (note that the $ is escaped with a backslash):

cloud$ mosquitto_sub -v -t \$SYS/#

The sub topics ‘…\load...’ are of particular interest.

Gather data

Before we start testing, we should figure out what metrics we want to measure. Resources on the shared system (CPU, memory, bandwidth, file handles) are good candidates for detecting capacity issues. Focusing on the user experience (failure rate, response time, latency) will help you hone in on the issues that will incur support costs or retention problems. Building the infrastructure to gather, analyze and visualize those metrics can be a significant part of the load testing process – but those tools are also necessary to do useful operational support in production. For the class, students used sysstat, locust, mqtt and malaria to gather metrics. A production-like system might use AWS Cloudwatch, New Relic, Nagios, Cacti, Munin, or a combination of other excellent tools.

The point of load testing is to find the limits and then decide what to do about them. There will be a point where the cost to rectify the issue is greater than any immediate benefit, load testing will help you find that bar. During the class, limits of a 1000 simultaneous users for web and 5,000-10,000 MQTT messages per process were common.

Final project

For their final project two students from the class, Matthew Bentley and Andrew Mason, decided to take on some of the problems with mqtt-malaria and extend Locust to publish MQTT messages. Using Locust they were able to scale their load test infrastructure across many machines and put a broker under more stress. In their previous testing with malaria, they found the point where a single device could send no more messages (at a reasonable rate), but they could not scale malaria to determine at what point the broker would not process any additional connected devices’ messages. Through their efforts, they reached 100% CPU on the broker, pushing 1 million messages a minute to 4000 users. As a result of their work they also open sourced their contribution to locust.

IoT Course Week 10: Analytics

IoTBackground

Last week we got our feet wet with an introduction to Bluetooth Low-Energy on iOS. This week, we’ll dive into analytics, provide business value, and make some pretty graphs.

Why Analytics?

When building a new product, there are always a variety of options on the table with which to improve that product. At LeanDog, we practice a software development cycle that includes short sprints coupled with an open and honest feedback loop that provides us with the information we need to make informed decisions about where to focus our efforts and resources. This allows us to make sure that we are building the right thing the first time and minimize the amount of risk inherent in the process.

Until relatively recently, collecting feedback about a product in-use was a long process that required either direct observation or careful reading of written user reviews and complaints. Due to the complex and inconsistent nature of users, collecting strong quantitative data about a product experience can be difficult. In a now infamous incident from 2013, a New York Times journalist wrote a negative review of the Tesla Model S, only to have the car’s onboard analytics refute many of his claims. It is not uncommon for a customer to report one thing, but end up doing something entirely different, and your user experience process will need to account for these inconsistencies. One of the many ways we solve that problem is through the use of analytics platforms and reporting tools.

In addition to uncovering potential pitfalls, analytics are a powerful way for product owners, designers, and developers to understand how a product is actually used. For companies that make physical devices, this provides insights that are difficult to collect otherwise. Imagine receiving a coupon in the mail for a smart GE light bulb you love that’s nearing the end of it’s lifetime. The only way GE could possibly anticipate that your current bulb is about to go out (without calling you every day to ask how often you turned it on in the last 24 hours) is through analytics. With analytics, you get an avenue outside of sales to start to figure out which features and products your users actually love, which have problems or aren’t worth further development, and even identify disengaged users for retention campaigns.

Enter Keen IO
For this class, we will use a popular analytics platform called Keen.io. Keen is a general purpose tool, not locked into web, mobile, or embedded specifics. It has a large number of supported software development kits (SDK’s), including Ruby, iOS, Python, .NET, etc. It also offers a powerful free tier, which is perfect for the amount of traffic currently being driven on student’s LAMPi systems. Registering and sending a notification in Python is as simple as as this:

from keen.client import KeenClient

client = KeenClient(
project_id="xxxx",
write_key="yyyy",
)

client.add_event("sign_ups", {
"username": "lloyd",
"referred_by": "harry"
})

This will send an event containing the signup data to Keen’s database. Now back at LAMPi headquarters we can track those signups on a giant web dashboard:

var series = new Keen.Query(“count”, {
eventCollection: “sign_ups”,
timeframe: “previous_7_days”,
interval: “daily”
});

client.draw(series, document.getElementById(“signups”), {
chartType: “linechart”,
label: “Sign Ups”,
Title: “Sign Ups By Day”
});

image01

Keen also provides a number of ways to pull out the analytics data and do additional processing to get exactly the view we wanted. Like if we wanted to build a tree of who our top referrers are what their “network” looks like:

image00

What’s next?
Analytics can also provide a leading indicator to help model the number of users that will be pounding on your infrastructure. To learn more about how to address that issue, join us next week when we talk about load testing!

IoT Course Week 9: Introduction to Bluetooth Low Energy

 

Internet of Things Course

To continue our goal of providing industry experience to the students of EECS397 Connected Devices, this week we will be diving deep into Bluetooth LE on iOS.

Recap

Last week students completed setting up a UI on iOS and Android that mirrored the interactions present on the LAMPi display and the web. The goal for this week is to connect those pieces

CoreBluetooth and Bluetooth 4.0

With the release of iOS 5 and the iPhone 4S, Bluetooth LE was positioned and continues to be one of the most common methods of short range data communication.  CoreBluetooth is the framework that Apple provides to developers to interact with Bluetooth LE hardware and peripherals. This is useful, as the current Bluetooth LE spec weighs in at over 2000 pages in PDF form.

Communication with LAMPi through CoreBluetooth can be broken into a four step process:

  1. Scanning for LAMPi device (from provided array of service id’s)
  2. Connect to discovered service (lamp-service)
  3. Probe characteristics (hsv, brightness, on/off).
  4. Subscription notifies when something changes. notify on property write.

Scanning for LAMPi

Students began the class by making an update to their LAMPi’s. Each team was given a BlueGiga BLED112 to plug into their Raspberry Pis, as well as updated Python services which allow the LAMPi to act as a Generic Attribute Profile (GATT) server. What the GATT server does is broadcast a number of available services to any BLE devices nearby that care to listen. In the case of the LAMPi, there is only one service being exposed, which is aptly called the Lampi Service.

Screen Shot 2016-04-12 at 11.18.43 AM

The service being advertised from the LAMPi includes a device id, which students use to identify their unique LAMPi in a classroom containing many more. Once discovered, it is time to connect.


- (void)startScanningIfEnabled {

if(self.shouldConnect) {

[self.delegate onLoading:@"Searching for sensor..."];

NSArray *services = @[[CBUUID UUIDWithString:LAMPI_SERVICE_UUID]];

[self.bluetoothManager scanForPeripheralsWithServices:services options:nil];

}

}

Connecting to a Peripheral

Screen Shot 2016-04-12 at 11.20.51 AM

CoreBluetooth abstracts away much of the detail required in making a connection to a BLE peripheral. When a peripheral is discovered, our code will immediately attempt to connect. If that connection is successful, we search through the set of services that exist on the peripheral, looking for one that is recognized.

“`

– (void)centralManager:(CBCentralManager *)central

didDiscoverPeripheral:(CBPeripheral *)lampPeripheral

    advertisementData:(NSDictionary *)advertisementData

                 RSSI:(NSNumber *)RSSI {

       …

       [self.bluetoothManager connectPeripheral:self.lampPeripheral options:nil];

}

 

– (void)centralManager:(CBCentralManager *)central

 didConnectPeripheral:(CBPeripheral *)lampPeripheral {

   NSLog(@”Peripheral connected”);

   [self.delegate onLoading:@”Found lamp! Reading…”];

   lampPeripheral.delegate = self;

   …

       // Search for a known service

       for (CBService *service in lampPeripheral.services) {

           if([service.UUID isEqual:[CBUUID UUIDWithString:LAMPI_SERVICE_UUID]]) {

               self.lampService = service;

           }

}

“`

Services and Characteristics

At this point, we are connected to a Lamp Service, which is now providing us with a collection of characteristics. Characteristics are how communication in BLE works. To make a comparison to software, Services can be thought of as Classes while Characteristics are more like the properties on an Object. Characteristics support four different actions: read, write, notify and indicate. While read and write are arguably fairly straightforward, notify and indicate both have to do with a subscription flow that we will use heavily in the iOS application.

Screen Shot 2016-04-12 at 11.23.20 AM

Subscribing to Characteristics

Because LAMPi has both on device and cloud controls, we want to be able to track the state of the LAMPi in real time while the iOS app is running. If a user were to change the color of LAMPi by using the Raspberry Pi UI, the Bluetooth service would send a notification to iOS that the HSV characteristic had been changed.  The following block of code is an example of a discovered Characteristic being initialized. It reads the current hue and saturation of the HSV Characteristic, and then tells the app to subscribe to the Notify value (the notification) of the lamp peripheral.

“`

           [self.lampPeripheral readValueForCharacteristic:self.hsvCharacteristic];

           if(self.hsvCharacteristic.value != nil) {

               float fHue = [self parseHue:self.hsvCharacteristic.value];

               float fSat = [self parseSaturation:self.hsvCharacteristic.value];

               [self.delegate onUpdatedHue:fHue andSaturation:fSat];

           }

           

           [self.lampPeripheral setNotifyValue:YES forCharacteristic:self.hsvCharacteristic];

“`

At this point, when the LAMPi HSV Characteristic changes, CoreBluetooth will call a delegate method that is triggered from the setNotifyValue line.

“`

– (void)peripheral:(CBPeripheral *)peripheral

didUpdateValueForCharacteristic:(CBCharacteristic *)characteristic

            error:(NSError *)error ;

“`

It is in this block of code that the HSV value is updated in the app, and logic to refresh the UI is executed.

Fun Fact: Origins of Bluetooth Name

As a bonus for making it this far, did you know that the origin of the word “Bluetooth” comes from a c. 970 King of Denmark, called Harald Bluetooth? In fact, the Bluetooth logo is comprised of the Nordic runes for H(8px-Runic_letter_ior.svg) and B (12px-Runic_letter_berkanan.svg), Harald’s initials.