The Magic of Agile Swarming
Being an agile software development team certainly means different things to different people. There are degrees of adoption across a very wide spectrum, with apparently very few organizations that think they do it well. According to VersionOne’s State of Agile Survey (released April of 2017), 80% of their respondents say they are “at or below a still maturing level.” Unfortunately, development teams often don’t put a lot of effort into the “learn” part of the iteration. We want to hurry up and get the Scrum ceremonies over with so we can get back to writing code. After all, there’s so much work to do! But is insufficient coding time really the problem?
For many of us, fire fighting might as well be specifically listed in our job description. We go to work every day knowing that we need to be ready to slide down the pole at a moment’s notice, grab our hats, and jump on the truck. We accept it as just the way things are, and we assume there’s nothing we can do about it. But, what if the root cause of our struggles is a severe lack of efficiency? Everyone knows how important it is to do it better than that other company over there. We just can’t seem to get there--we don’t seem to have the bandwidth. Managers add more people and balloon up the size of their organizations and still have the same struggles. You can’t seem to get over the hump because your teams aren’t developing software efficiently (and you’re not alone).
Principles in Efficient Development
So what causes us to be inefficient? For most of us, the first thing that comes to mind is lack of automation (automated builds, deployments, testing). “Once we have enough automation, life will get better.” Unfortunately that’s only part of the solution. Consider the effect of rework on your project. The most efficient way to build a feature is to build it once correctly and never go back and touch it again. Bugs, refactoring, and other similar activities are essentially reopening the patient after he’s left the operating room and there’s inherent risk associated with that. We can’t eliminate rework, but we should certainly endeavor to minimize it.
“But doesn’t agile embrace rework (ex. refactoring)?” It actually does in a way, because the creators of agile understood that two key causes of rework are unforeseen circumstances and changing business requirements. It turns out that humans are terrible at predicting the future. Agile creators also understood that a huge contributor to inefficiency is what developers call “gold-plating”--packing in functionality we think someone will use even though end users never actually asked for it. It’s like pork for your software product--a complete waste of time. "Don't build a space station when all they're asking for is a Volvo." So, companies wisely started leaving out the pork and embracing refactoring instead, only adding functionality when there is a clear need. But life’s unpredictability isn’t the only driver for rework, is it?
Missed details at any stage of feature development will ultimately waste time and money. Collaborating effectively up front will, over time, save you a ton of rework (dealing with missed requirements, a shortsighted design, etc.). We all have blind spots, and we all need extra sets of eyes. Many development teams embrace this on the back end during code review, but put much less energy into collaborating early on when problems can be resolved cheaply and after minimal investment.
How many times have you implemented a feature and found significant flaws near the end that should have been caught during requirements/design discussions? It's like trying to drive from Atlanta to Montgomery and realizing several hours into the trip that you accidentally drove to Birmingham instead. How much time was spent trying to get the code just right only to open the patient up again later because significant requirements were missed. Leveraging collective intelligence absolutely would save time and money, but instead developers often work on features in isolation.
Traditional swarming means that the team works collaboratively on stories with several people working on a small feature at the same time, shortening the feedback loop and reducing the overall completion time for the feature (ie. divide and conquer). This is essentially swarming within each discipline (backend developers, UI developers, etc.). Before development begins, the UI developers work to identify independent tasks that can be performed concurrently. They discuss interface points so each person knows how their piece fits into the whole. The team members then can proceed to completion on their assigned tasks and bring everything together at the end during integration. Frequent commits and periodic code reviews help ensure that everything stays on the rails. This approach requires collaboration between the developers, which helps to produce a better end result anyway. We often prioritize time spent writing code (any code) over time spent making sure we don’t write the wrong code. When you consider the time potentially saved, the value becomes clear.
Another valuable approach to swarming is to focus the team in the early going on dependency mitigation in order to facilitate concurrent development across the disciplines. Consider the natural development flow of a UI feature. The automation testers (SDETs) are dependent on a working UI to test against, the UI developers are dependent on a working backend API, and the backend developers are dependent on configuration, database updates, and automated builds/deployments. So UI developers might not start their work until the APIs are done and SDETs might not start their work until the feature is complete. Each discipline is working in isolation, which hampers collaboration because the people you need to interact with are busy working on other things. But what if you could mitigate the dependencies earlier and allow the disciplines to all work concurrently on the same feature?
Here are some examples:
1. Deployed Functional UI w/ Stubs
In order to unblock SDETs, UI developers can give them a functioning UI that works just enough to let them write tests. Backend API integration and CSS styles can still be pending, since automated test frameworks like Selenium won't care if those things are unfinished. It can all be smoke and mirrors. While changes may occur that cause some rework, the benefit to starting the tests early outweighs that risk.
2. Deployed Backend APIs (stubbed, hard-coded data)
Providing backend APIs that the UI developers can test against allows early detection of integration issues between the front end and the API(s). Sometimes you find out that the API provided doesn't meet the needs of the front end developers. Entire calls could be missing, the signature could be wrong, or the structure of the data might have issues. If there is a disconnect, you might as well find out about it early before anything has hardened.
3. Create a HelloWorld version of new applications and services.
If a new service is needed (ex. a microservice), create the repo and build a “hello world” version of the service. This allows dev-ops resources to start on Jenkins jobs and deployment scripts before the service is actually developed.
These optimizations facilitate an early feedback loop where someone can say “I need something different” before development is complete on the component that requires the change.
It’s incredibly important that we figure out how to shorten time to market on features we’re working on. The business gets no value from having a bunch of features that are in progress, and developers desperately need features to be implemented quickly so defects can be resolved as close to the point of injection as possible. Developers also desperately need to interact with each other even though all they really want to do is write code. It’s better for everyone involved, including the end user who just wants a better product. If you don’t give it to them, they’ll go somewhere else to find it.
Swarming is an extremely valuable tool in your organization’s toolbox, if people take the time to learn how to do it. It’s not a framework or even an activity--it’s a mindset. For every user story, team members ask themselves two questions:
1. How do we organize tasks for this story to get several people contributing at once?
2. What is the minimum I need to do in order to unblock someone who is waiting on me?
What if your team quickly built features together rather than slowly building a bunch of features independently? They could actually respond to changing business requirements and meet the needs of the business when the business needs them to be met. Competitors would fear you--customers would love you. That is a recipe for a successful business.