Are Software Developers Overworked or Undecided ?

During the last 15 years, I’ve worked in many different teams. Every team has its own way of working. I’ve been at places where everyone seemed busy all the time. Did we deliver more ? No. Often quite the opposite ! The most productive teams I’ve been in used to do no more than 6 hours of solid work every day …

A developer in a mouse wheel

What’s the reason behind that ?

In Lean Management theory, lead time is the time it takes to get a feature from customer request to delivery. Here is a graph that draws typical lead time over factory load (1 – idle time percentage).

A graph of the lead time versus utilization

What does all this tell us ? 

When utilization becomes too high, people start multitasking. As multitasking increases, productivity drops, delaying projects. As a consequence, products are late to hit the market. On the other side, a low utilization means paying people to be idle. That might put any company out of business. Factory managers’ best practice is to limit Work In Progress by loading their assembly lines around 80%. Not too high to avoid over-utilization, but not too low to avoid wasting time.

Every software product is new product development (Craig Larman). As such, they are way less predictable than any assembly line. It means we need more slack to absorb unexpected variations (Donald G. Reinertsen). This tells us that we should have more than 20% of slack in our schedules and plannings. Unfortunately, overloading and multitasking is a wide spread disease in the software world.

How to get rid of task switching ?

The answer is obvious : prioritization ! Only work on the most important item at a time. It’s not because we start everything now that they’ll sell them earlier ! It’s quite the opposite in fact !

A comparision of sequential and concurrent product development

Here is another advantage to prioritization. It decreases the time between start and completion of a feature. This, in turn, decreases the likelihood of building something obsolete.

One last thing about prioritization …

A graph of values of features in decreasing order

“Black Swan Farming Using Cost of Delay” Joshua J.Arnold and Özlem Yüce

This graph tells us that a small fraction of the features bring most of the value, while a very large portion of the features bring almost no value … As puzzling as this graph may be, it’s also a wonderful opportunity. If we manage to focus one the most valuable features only, we can save a ton of work, while being more effective ! Lean startup and market research are techniques to identify these features up-front.

💡 You don’t need more time, you just need to decide ! Seth Godin

Culture issue

Becoming more effective by banning overtime, enforcing monotasking and being serious about prioritizing ? These ideas are heresy in some workplaces ! Have a look around you. Are people overworked ? Are they fire fighting all the time ? Are they drowning in multitasking ? If it’s the case, I bet there’s a wide scale lack of prioritization issue at your workplace.

People at the top of the company need to control the overall WIP. If they don’t, multitasking will catch you up, however you try to escape it. Multitasking can be obvious e.g., when we need to work on different projects at the same time. It can also sneak in as maintenance, or unrelated meetings marathons. Another symptom is when developers need to take on “submarine” refactoring initiatives. From experience, that’s often the case in sales driven companies. Where any customer request is agreed and sold, regardless of the company’s capacity.

💡 Require​ ​everyone​ ​in​ ​product​ ​and​ ​engineering​ ​to​ ​read​ ​​Principles​ ​of​ ​Product​ ​Development​ ​Flow by Donald Reinertsen. John Cutler

There’s an interesting parallel in the Stephen R. Covey’s 7 Habits of Highly Effective People. He says we should drop the urgent and not important for the important and not urgent. In software term, it means spending time on important and long term work rather than short term stuff. We should do more refactoring, hiring, training and continuous improvement. We should stoping building features we are not sure of the value.

Let’s summarize. Here are my advices to work less and be more productive. Enforce hard WIP limits at all levels. Do serious prioritization. Provide enough slack. In my next post, I’ll show how XP addresses these issues.

Mobbing Around the World for 48h … Who’s in ?

In Far From the mobbing crowd the Cucumber guys explain how they combined mob programming and remote work. Matt and Steve also explain that a mob is both resilient and fast. They explain that they sometimes have to leave the mob for 15 minutes or so. When they come back the code has changed a lot ! It’s as if the mob continues no matter who’s in.

One of them then suggests an idea. 

With enough people around the world, we could run a mob that never stops!

A drawing of a mob walking around a binary world

What’s the point ?

In The Ultimate Guide To Remote Work the Zapier team explains how to take advantage of time zones and remote work. We can get a feature done faster by having a pair of programmers from different time zones work on it ! Instead of a single day of work in 24h, you get 2 ! We could get features in half the time.

The cover of The Ultimate Guide To Remote Work

In Lean Management, the time to get a feature is called the cycle time. A short cycle time has a lot of benefits. You get feedback faster. The needs of the users have less risk of changing. You’ll get less interruptions, which increases your focus and effectiveness.

Zapier found that time zones and remote work can divide the cycle time by 2. This is already a major improvement. Let’s replace this pair of programmers by a mob ! The cucumber guys explain that 4 hours of mobbing is about the most someone can do in a single day. With a mob of 5 people at any time, that’s around :

24/4 * 5 = 30

30 developers working on the same feature, full time !

What would be the effect of this on cycle time ? I don’t expect it to cut the cycle time by 30, but from my experience with pairing, I’d still expect a drastic reduction. Imagine a complex feature that you estimated to 30 days to finish. Could it be done in less than a week ?

Working on a single story at a time, we could forget merges and complex CIs ! More generally, it would lead to an important Work In Progress (WIP) reduction. Very related to cycle time, reducing the WIP also brings a lot of improvements. One of the most important is an increased throughput. Here is a great article detailing the benefits of reducing the WIP.

💡 Mob programming reduces WIP and cycle time

And what about the team structure ? Could a team of 30 be effective in this setting ? 7 is the magic count for a team size. If we could grow past this number, we could imagine completely new org charts :

  • More decentralized
  • More resilient 
  • Less synchronization and coordination overhead

A screen capture of 'Far from the mobbing crowd'

Let’s try it !

As any new idea, it’s also very likely to fail miserably … The communication overhead could turn out overwhelming. The mob might not be resilient to frequent changes after all. It could work but not bring any reduction to cycle time. There is a ton of reasons for this to fail.

The best way to find out is to try !

💡 Stupid ideas that work become great ideas. [XP moto]

We just need to find enough people spread around the world to give it a try. Could we do 48 hours straight of mobbing and go twice around the globe ?

We would need people distributed across all time zones. Pairing and incremental design would be de-facto practices. It would work better with people used to XP.

We’d need at least a mob of 3 doing 5h of mobbing per day for 2 days. That would be :

24/5*3 = 15

We could go with 15 people but 30 would be ideal. Once we have enough volunteers, we’ll have to agree on a few other things :

  • A date for the experiment
  • A subject we can all understand
  • Technologies and tools to use
  • A time-schedule

There’s nothing impossible here !

Call for volunteers

If you want to participate, leave a comment below or contact me through twitter. I’ll keep you updated about the status of the experiment.

One last thing … as you’ve guessed this is an experiment, not paid work. Anything we would produce will be open sourced on Github.

What Rails Teaches Us About Building Platforms and Frameworks

More than ever, the cheapest way to build a framework is to refactor it out of a specific app.

Rails is a web development server side framework built in Ruby. It’s been and remains very popular. It set the tone for all the MVC web frameworks that followed. People have ported it to many other languages. Rails is now in it’s 5th version, what is less known is how it was built in the first place.

Drawing of a train being detached from its rails

The story of Rails

David Heinemeier Hansson, the creator of Rails, is a cult in the Ruby community. You can find the history of Rails at many places. If you have a bit of time, read this great article from Wired. If you don’t have a lot of time, here is a summary.

Jason Fried and DHH were working at 37signals. They were working on project management app for small businesses called Basecamp. After releasing the first version of Basecamp, DHH extracted and open sourced Rails out of it.

37signals later then re-used Rails to build other apps like Campfire, Highrise and Backpack.

We all know the end of the story, Rails adoption exploded. Many successful companies like Github, Airbnb, Twitter & Shopify have used it. It evolved a lot, through the darwinism of open source. Today, Rails might not be the latest and coolest web framework, but it is still very productive and popular.

The Rails logo

Economic sense

To summarize, here is how they did it :

  1. Built a specific app, and monetized it
  2. Extracted an open source framework from it
  3. Built other apps on the framework

If you are not a programmers, you’re likely to assume that the cost of reusing software is negligible. If you are a programmer, I’ll ask you to assume that it is for a while.  Under this hypothesis, all they did makes a lot of business sense :

  • Building a specific app first is the fastest path to paying customers. Building a framework first would need more work.
  • Once you have paying customers, you’ve got money to fuel further work.
  • In particular, you have money to finance the extraction of an open source framework.
  • Putting the framework open source increases its reusability through bug reports and contributions.
  • Building the next app is even easier thanks to a mature framework and the revenues from the original app

💡 Once you have paying customers, you’ve got money to fuel further work.

How can it work ?

Cover of Rework book

Ok, enough for common sense. Let’s see what happens when you take into account the real cost of changing software ? All software developers know that changing software is far from cheap. Very often, it’s more expensive to adapt than to rebuild

So how did the Rails guys manage it ? Rails has 3 specificities that explain that :

  • Rails has automated testing built in and out
  • Basecamp guys are agile with a lower ‘a’. They don’t follow Scrum or any method. Read their books and you’ll understand how agile they are. They follow principles like KISS, YAGNI, Lean startup discovery …
  • Finally, it was open sourced !

Agile principles and technical practices is what allowed them to take the common-sense path.

💡 Thanks to agile practices, the cheapest way to build a framework is to extract it from a specific app.

Open question

If it is possible to do the thing that makes the most economic sense, why isn’t it the default way ? Why are so many of us still losing a ton of money writing large frameworks and platforms up-front ? How could we get large companies to adopt the more nimble ‘basecamp’ way ?

If you have remarks, answers or different point of views, I’d love to read your comment !

How to Run Your First Improvement Kata

The improvement kata can solve problems that typical retrospectives fail to address. Although there is a halo of mystery around it, it’s actually not that difficult to get started ! Here is a guide.

During the last few weeks, I’ve been blogging about the improvement kata. You can read the full story of the first time we applied it in our team to gain 25% of productivity. If you are more interested by what it taught us, check this other post.

Illustration of the improvement kata cookbook

A 5 bullets summary

Here is how I would explain what the improvement kata is :

  • It’s a continuous improvement technique. It relies on the scientific method to reach a target state.
  • It involves running experiments to know if your ideas are valid.
  • It can be long to run through, but it works on tricky situation where retrospectives don’t.
  • It’s 100% scientific. It uses data analysis and deduction. Not gut feeling and community best practices.
  • It can be part of the backlog, as any other item. It does not have to be a special event like retrospectives usually are.

The main steps of the improvement kata

💡 The improvement kata is 100% scientific, it uses data and deduction, not gut feeling and best practices

Let’s give it a try ! 

Is that enough for you to give it a try ? If so, great, read on ! If you need a bit more convincing, check the full story of how we gained 25% of productivity with it.

Here is how to get started :

  1. The first thing is to read about it. If you have the time, Toyota Kata is a good read. If you read French, the “Petit guide de management lean à l’usage des équipes agiles” is a very pleasant and easy read. Finally if you want to cut it as short as possible, read the Toyota Kata website.
  2. Pick a topic to try it on. Best candidates are clear and important problems. They might have emerged out of a retrospective for example. The scope should be small enough not to get lost.
  3. Once you’ve identified a topic, someone or a pair should take ownership of the kata. It’s very unlikely that you’ll be able to do the full kata in one afternoon. Understanding happens when the brain is at rest, and experiments take time. The owners need to dedicate some time to follow up on the kata.
  4. Repeatedly ask yourself the coaching kata questions. This will help you and your pair to stay on track.

    1. What’s the target condition ? (Describe what are you trying to achieve)
    2. What’s the actual condition ? (Describe your current situation)
    3. What obstacles do you think are preventing you from reaching the target condition ? Which one are you addressing now ? (Describe the first problem you are about to try to fix)
    4. What is your next step ? What do you expect ? (Describe the experiment you are going to run to test a solution)
    5. When can we go and see what we have learned from taking that step ? (Run the experiment and learn from the results. Decide on a process change or repeat from an earlier step)
  5. Going through the kata in pair is a great way to spread the practice within the team. At some point you might be able to run many improvement katas in parallel ! Just make sure not to walk on each other’s toes …

Expect the first time to be a bit rocky, and to feel lost from time to time …

Start today !

The main steps of the improvement kata

Many practices and techniques seem daunting at first. Remember the first time you wrote a test before the code. The first time you tried to program using only immutable data structures. Or the time you wrote your first “hello world” program !

💡 We can learn anything on our own by just doing it

The improvement kata is no different. Give it a go, and you’ll learn a powerful technique.

Whether you have already used the kata, you plan to use it or you have questions about it, I’d like to hear from you ! Leave a comment.

Lessons Learned From Running Our First Improvement Kata

During the past few weeks, I blogged the story of our first improvement kata.

Doing this first improvement kata taught us many lessons. We re-discovered best practices of the software industry. We also learned more general things about the improvement kata itself.

Drawing of books

Rediscovered best practices

As we went through the kata, we ‘proved’ many known best practices. We did not have to believe them, we had data explaining that they worked. For example :

We also pushed the #NoBug policy further than it is usually done. We defined a clear definition for bugs that anyone could use. Doing so, we removed the product owner (or on-site customer) from the picture. Very often, the PO is the only one who can sort stories from bugs out. We created what Donald Reinertsen calls a distributed rule in the flow book. It increased empowerment, removes bottlenecks, while ensuring alignment.

The 'Flow' book cover

The improvement kata

The first general lesson that we learned is that the improvement kata works !

At the beginning, we were very uneasy not to have perfect data. Remember how we had to resort to velocity as a proxy measure for productivity. In the end, that was not a severe problem. It still allowed us to understand what was going on.

We also learned that retrospective is not the only road to continuous improvement. In fact, the improvement kata and retrospectives are very different :

  • The time frame is different. A retrospective lasts for 1 or 2 hours and yield immediate results. An improvement kata is a background task that could take months in theory !
  • But the improvement kata also digs a lot deeper in the topic and brings true understanding. In our case, it fixed a problem that retrospectives where failing to address.
  • Ownership is also different. Retrospectives are a team activity. The improvement kata needs one or a few owners. It would be very difficult to align everyone on the same path of thoughts if we did it as a group activity.
  • Being a team activity, retrospectives have built-in alignment. The conclusions of the improvement kata must explained and agreed for a team to act on them. A good practice is to have regular (short) team meetings to share the current state of an improvement kata.
  • As the improvement kata is a more individual activity, it is more remote friendly. Team members can run the kata on their side, sharing everything through a wiki, or a blog for example.

Keep in mind that this was our first try at the kata. Some of our difficulties might disappear with a bit of practice !

What’s next ?

Hybrid continuous improvement

I clarified the differences between the improvement kata and retrospectives. That’s not the end of it. I’m sure a mixed format could be very interesting ! Start with a retrospective to collect the team’s problems, and vote on the more important. Add a corresponding improvement kata task to the backlog. Someone would then handle this improvement task, sharing with the team along the way.

This might be a great opportunity to reduce meeting time with shorter retrospectives.

💡 Reduce meeting time with mixed retrospective & improvement kata

Data science for software

Going through this improvement kata made me realize something else. It is very difficult to get quality data to investigate. We had to resort to what was available all the way.

What’s striking is that we use software tool for all our work ! These tools have logs and could record usage data somewhere. Imagine all we could learn by mining all this data ! Our IDEs, CI servers, quality monitors, test tools, version control and project management tools should store everything we do at the same place !

With all this data, could we improve our estimations ? Could we find creative ways to be more productive ? Could we estimate the speed up that fixing some technical debt would bring ?

💡 What are we waiting to apply data science to the development process ?

As the saying goes, “The cobbler’s children go barefoot”. We are building such systems for others, but not for ourselves.

Hopefully, new tools like CodeScene are emerging to fill this gap. You can learn more about CodeScene on their website, or from the founder’s book. It analyses the version control history to find hot spots and other things in your code.

The 'Code as a Crime Scene' book cover

While we keep dreaming of such tool, I’ll continue to blog. Next week, I will write a short guide of how to run your first improvement kata.

How We Used the Improvement Kata to Gain 25% of Productivity - Part 5

This is the fifth (and last) post of a series of 5 about the improvement kata. If you haven’t read the beginning of the story, I recommend you start from part 1.

In the previous post, we decided to adjust our definition of a bug to limit the time lost on nice-to-have bug fixes.

It would take a while to know if adjusting the definition of a bug would help us or not. At the same time, we knew it would not help us to reduce the number of bugs we escaped to other teams.

A 'SUCCESS' banner in the wind

Idea 3 : More exploratory testing

We decided to push on this matter as well. This means that we would be running two PDCAs (Plan-Do-Check-Act) at the same time. This is not the improvement kata procedure by the book. That could have been an error from our side, as first time users of the kata. Or maybe it’s a pragmatic habit to adapt the kata to real life … I guess we’ll know better as we apply the kata more often. The danger is that the different experiments can conflict with each other. When measuring the results, it becomes difficult to link observations with experiments. Anyway, back to our own situation, as you’ll see, it ended up well for us.

The first thing was to know a bit more about our bugs. Checking the recently closed bugs yielded suspicions about a recent features. Analyzing further proved our gut feeling.

Curve of how bugs were fixed on the last 2 months

Curve of how bugs were fixed on last 2 months

Curve of the origin of bugs on the last 2 months

Curve of the origin of bugs on the last 2 months

Ignoring the Christmas drop at the middle of the curve, we concluded 2 things from these graphs :

  • We were leaking bugs to the product
  • Bugs mostly came from newly added features

Despite all our automated tests and regular  exploratory testing, we were leaking bugs.

We decided to do more exploratory testing for a while ! We were already doing exploratory testing at the end of every story. We added an extra 1 hour team session of exploratory testing every sprint.

Do, Check & Act

We used these new conventions for a few weeks. We did more exploratory testing, and would be more strict about what a bug was. We stuck to our prioritization : first improvements, then bugs and only after, stories.

After a few weeks of that, we were able to update our bug trend and do a linear regression on it again. Here were the results :

Curve of the origin of bugs on the last 2 months

Hurray ! As you can see, we were to be done with bugs around April 2017, which was 3 months away at that time.

💡 Quality is free, but only for those willing to pay for it ! [Tom DeMarco in Peopleware]

Cover of the 'Peopleware' book by Tom DeMarco & Timothy Lister

We confidently adopted these practices as part of our working agreements. This brought our first improvement kata to its end.

💡 The improvement kata not only brings improvements, it also teaches you why they work.

3 months later

As you know, April 2017 is long gone. I can now give you a more up to date report of the actual effects on the team’s daily work. 

First, the backlog does not contain bugs anymore. We payed the bug debt back. Second, we still discover some bugs from time to time, but a lot less than we used to. To summarize, there is now a pair of developers (25%) of the team that can work on user stories instead of fixing bugs.

As we are still fixing bugs as they appear, the 25% productivity gain claim might be an overstatement, but 20% is not. At the same time, less bugs are now escaping. This means that the whole organization is saving on interruptions and rework. 25% might not be such a bold claim after all !

This is it !

This was post 5 in a series of 5 about the improvement kata. I’m not completely done writing about this improvement kata though. In the coming weeks, I’ll post about the lessons learned and how to start your own improvement kata.

How We Used the Improvement Kata to Gain 25% of Productivity - Part 4

This is the fourth post on a series of 5 about the improvement kata. If you haven’t read the beginning of the story, I recommend you start from part 1.

We ended the previous post with a target condition :

We’d like to be done with bugs within 3 months.

Currently, 1 pair (one quarter of the team) is constantly busy fixing bugs. If we manage to find a way to spend less time on bugs, we can expect a productivity increase of about 25%.

Anti bug insecticide can

Next step in the improvement kata is to run Plan-Do-Check-Act (PDCA) experiments. Before you run experiments, you need ideas !

Idea 1 : Stop exploratory testing

We’d like to spend less time fixing bugs. At the same time, we know that we started to spend more time on bug fixing when we started exploratory testing.

We thought that one easy way to do less bug fixing was to stop exploratory testing altogether ! We listed the pros and the cons of exploratory testing.

The obvious cons

  • It discovers bugs. Fixing these bugs costs time. We’d rather spend this time delivering new features

But the pros are many

  • It helps us discover technical debt and improve the quality of our code, which makes us faster in the long run. Clean code has less bugs. When we discover many bugs related to a particular region of the code, it means we should refactor it.
  • It speeds up integration with other teams’ work in many ways :
    • It saves other teams from debugging their time away trying to use broken things
    • Fixing bugs is sometimes just a matter of providing clear ‘not supported yet’ errors. Doing this saves a tremendous amount of time to other teams.
    • Avoids blocking other teams as they wait for your bug fixes
    • Reduces interruptions from bug reports and fixes bouncing between teams.
    • By reducing unplanned rework, it makes any commitment you do more likely

In the light of all this, stoping exploratory testing did not seem like a great idea. We had to find something else.

💡Saving time by not fixing bugs might not be a great idea

Exploration map with a red cross

Idea 2 : Review what a bug is

We needed to find a middle ground between where we were and stoping exploratory testing. We wanted :

  • not to let bugs escape
  • raise clean errors on things that are not yet supported
  • prevent scope creep from bug reports

A few months ago, when we decided to prioritize bugs before features, we had defined a definition for bugs. Here is what it looked like. Something is a bug if any one of these is true :

  1. It used to work
  2. It corrupts data
  3. It silently returns wrong results
  4. It returns an internal error (like an internal exception stack trace)

We examined this definition, discussed, and decided to dump the first clause. Our logic was that it was very difficult to know if something had already been working. The best thing we could refer to was our extensive suite of tests. If we discover that something is not working, it is very likely that it has never been. If it had been working, it means there was a hole in our test suite.

💡A hole in a test suite is a bug to fix

We decided to test this new bug definition for a while.

Not there yet !

We’re getting closer to the end, but you’re not done with PDCA yet.

This was post 4 in a series of 5 about the improvement kata. In the next and last post of this story you’ll learn how we ended the PDCAs for great results.

How We Used the Improvement Kata to Gain 25% of Productivity - Part 3

This is the third post on a series of 5 about the improvement kata. If you haven’t read the beginning of the story, I recommend you start from part 1.

In the previous post, I explained how we started to understand what was going on. We were now questioning our way of handling bugs.

Are we spending too much time on bugs ?

Bugs drawn on top of code

More understanding

Types of tasks

To answer this question, we decided to plot the different types of tasks we had completed per sprint.

Bar chart with the types of tasks over sprints

Think again of the velocity curve we started with. We see an almost exact correlation between story count (green bars above) and story points (blue curve below).

💡#NoEstimates works

Velocity graph

We can also see that after sprint 56, we were spending more time on bugs and improvements. Improvements are supposed to improve productivity, so we decided to focus on bugs first. Here is what we get if we compute the average number of bugs per sprint :

Periods Sprints Bugs Average bugs fixed per sprint
2015, Before sprint 56 15 21 1.4
After sprint 56 34 210 6.1

Starting sprint 56, we were fixing 4 times as many bugs as we used to do before !

What is going on with bugs ?

At this point, we felt we’d made a great step forward in our understanding. We almost thought we were done with it …

After a bit of thinking though, it was clear that we were not ! We still needed to understand why we were in this situation.

We started by listing more questions :

  • Could it be that we just got a lot better at testing ? Since sprint 56, we had been doing regular exploratory testing. Exploratory testing sessions were very effective at finding bugs.
  • Were we paying back a bug debt ? The created versus resolved trend seemed to show so. But it could also be that we weren’t testing as well as we used to !

Created vs Resolved Bugs graph

  • If we were paying back a bug debt, how close were we to the end of the payback ?
  • Were we creating too many flaws in the software ?
  • Are we fixing too many bugs ? If so, what should we do to fix less ?
  • Are the bugs coming from other teams using our component or from our own testing ?
  • Are bugs on new or old code ?

A lot of questions, all difficult to answer. We decided to first see if we were paying back a bug debt. If this was the case, most other questions would become more or less irrelevant. With a bit of thinking, we came up with a measure to get the answer.

Are we paying back a bug debt ?

We first started to do exploratory testing at sprint 56. To do this, we would run a 1 hour session, where the pair finding the more bugs would win fruits. (Later on, we streamlined exploratory testing as part of the workflow for every story) At that time, we used to find more than 10 bugs in 1 hour.

💡Gamification transforms nice developers into berserk testers !

Explo Test Sesssion 61 62 63 64 66 16.01
Bugs found 16 6 16 10 11 11

We would do another such a session. If we found significantly less than 10 bugs, let’s say less than 6, it would mean that :

  • we improved the quality of our software
  • our streamlining of exploratory testing works
  • if we continue to search and fix bugs as we do currently, we’ll reach a point where we won’t find any more bugs

Otherwise, none of these stand, and we’ll have to continue our investigations.

So we did a 1 hour, fruit-powered, exploratory testing session. And we found only 3 bugs ! Indeed, we were paying back a bug debt. The question became

When should payback be over ?

A linear regression on the created vs resolved bug trend showed that we still had 15 more months to go !

Bug trend graph

Target condition

At that point, the target condition became obvious :

We’d like to be done with bugs within 3 months.

Currently, around 1 pair (25% of the team) was busy fixing bugs. If we’d manage to bring this down, we’d have a 25% productivity boost.

This was post 3 in a series of 5 about the improvement kata. Next post will be about PDCA.

How We Used the Improvement Kata to Gain 25% of Productivity - Part 2

In my previous post, I described the productivity issue our team was facing. How retrospectives did not work, and how I started looking at the improvement kata.

We had gone through the first phase of the improvement kata : set the end goal.

Generating enough profit for the business while sticking to a sustainable pace.

Time to start the second phase : Understand.

Drawing of a question mark transforming into a light bulb


Where we really slower ? Or was it an illusion ?

When trying to understand, you have to start with the data you have. You continue digging until you get a deeper understanding of the situation.


We started with available data : story points and velocity. For sure this is a pretty bad measure of productivity. (Note : we should never use velocity for performance appraisal) In our case though, it felt like a good starting proxy measure.

Here is our velocity curve over 2 years.

Velocity graph

It’s obvious that something changed. There are 2 parts to this graph. The velocity dropped between sprint 54 and 16.01. That’s a clue that our gut feeling was not completely wrong. Our productivity did change.

Man days

Our first hypothesis was that team members turnover was the cause. As with any team, some people came, and some people left. Let’s superpose the man days and velocity curves.

Velocity vs Manpower graph

That could only explain part of the problem !

We tried to fine tune the man days curve. We took people’s involvement in tasks outside of programming into account. We used coefficients depending on the developers’ experience. That did not provide a better explanation.

We had to find another explanation.

Velocity computation

As I said earlier, velocity is not a measure of productivity. Any change in the way we were computing velocity would impact this curve.

We had kept photos and Trello boards of our retrospectives meetings. We searched them through for anything that could impact velocity. Here is what we found :

  • At sprint 55, we decided to ditch the focus-factor
  • At sprint 61, we started to do regular exploratory-testing. Exploratory testing discovers more bugs on top of user reported bugs. This made us spend more time on fixing bugs.
  • At sprint 62, as we opted for a No Bug policy we decided not to count story points for bugs

💡Keep Photos and Trello boards of Retrospectives as a log of your working agreements changes

The timings almost perfectly matched what we had observed in the first place. The question that came to our minds was :

Are we spending too much time on bugs ?

Halfway through understanding

This is how we started to dig into our situation. It’s a good time to give you a bit of feedback about how we felt at that point.

It was the first time we tried the improvement kata. More than that, we did not find any tutorial or guides about how to run it. The only instructions we had were theoretical descriptions or super concrete examples. We had to bridge the gap and come up with our own way.

To summarize, we felt a bit lost, we had gathered data from here and there, and we did not know what to look at next. On top of that, the quality of the data we were collecting was not great. We were wondering if we would get anything out of these investigations.

The cover of the book 'The First 20 Hours'

It felt a bit like when I did the 20 hours experiment to learn anything. We did exactly what had worked with the learning experiment : we pushed through !

💡If you feel lost when doing something for the first time. Push through !

In next week’s post, I’ll continue to detail the ‘understand’ phase. The series also gained an extra post, and will now be 5 posts long.

More to read next week.

How We Used the Improvement Kata to Gain 25% of Productivity - Part 1

If you are serious about continuous improvement, you should learn the improvement kata.

Retrospectives are great to pick all the low hanging improvements. Once you’ve caught up with the industry’s best practices, retrospectives risk drying up. Untapped improvement opportunities likely still exist in your specific context. The improvement kata can find those.

Low and high hangling fruits on a tree

Here is how we applied the improvement kata to gain 25% of productivity in my previous team.

The Situation

Thanks to repeated retrospectives, the team had been improving for 2 years. Retrospectives seemed like a silver bullet. We would discuss the current problems, grasp an underlying cause and pick a best practice. Most of the time, that fixed the problem.

Sometimes it did not work though. Even if the issue came back in a later retrospective, it would not survive a second scrutiny. In the previous two years, the team had transformed the way it worked. It adopted TDD, incremental refactoring, pair programming, remote work, automated performance testing and many others.

Lately though, things did not work so well. The team was struggling with productivity issues. The team was not slowing down, but the scope and complexity of the product had grown. Features were not getting out of the door as fast as they used to. We had the habit of prioritizing improvements and bug fixes over features. That used to improve the flow enough to get more and more feature done. It did not seem to work anymore.

We tried to tackle the issue in retrospectives. We would change the way we prioritized features … To be later bogged down by bugs, technical debt or bad tools. We would discuss that in retrospective, and decide to change the priorities again … The loop went on and on a few times.

We were getting nowhere.

The improvement kata 

That’s why I started to look for other ways to do continuous improvement. I stumbled upon a book called Small Lean Management Guide for Agile Teams. The book is in french, but I wrote an english review. I fell in love with the way the authors dug into the hard data of how they worked to understand and fix their problems.

To learn more about this technique, I read Toyota Kata. It details two management tools used at Toyota : the improvement and coaching katas. Some say these are Toyota’s special weapon. The thing that grew them from a small shop to the largest car manufacturer in the world.

They are katas because they are routines. They must be re-execute many times. The improvement kata should improve the flow of work. The coaching kata helps someone (or yourself) to learn the improvement kata. Every time we go through the kata, we also understand it better.

Here is how the improvement kata goes :

  1. Describe your end goal
  2. Understand where you stand about this goal by measuring facts and data
  3. Based on your end goal and the current situation, define where you’d like to be in 3 months or less
  4. Use Plan-Do-Check-Act to experiment your way to this new situation
    1. Plan an experiment
    2. Do this experiment
    3. Check the results of this experiment
    4. Act on these results. 
      • Either drop the experiment and plan a new one (go back to ‘Plan’).
      • Or spread the change at a larger scale.

The Improvement Kata Pattern

Image from Mike Rother on Toyota Kata Website

The coaching kata is a way to coach someone into applying the improvement kata. The fun thing is that you can coach yourself ! The idea is to ask questions to the coachee to remind him of where he stands in his improvement kata.

The Coaching Kata Questions

You’ll find tons of details and material about these katas on the Toyota Kata website.

Our end goal

That’s how I started to apply the improvement kata in my team. I already had an idea of our end goal : to be more productive. To be more precise :

Generating enough profit for the business while sticking to a sustainable pace.

Retrospectives could not get us there. Would the improvement kata would succeed ?

This is the first part of a series of 4 posts relating our first use of the improvement kata. In the next post, I’ll explain what we did to understand the current situation.