Software Is Like Writing and Revising a Giant Book

With time, I discovered a way of explaining the subtleties of my developer job to my uninformed relatives and friends.

Sharing what your developer job is about with others can be very frustrating. Some people think you are “Just playing around with computers” as if it was not serious work. Others think that it is an extremely Cartesian and solitary activity with no place for communication or creativity. Trying to explain the diversity and richness of a developer’s work is often a disappointing experience.

The book metaphor

I explain to people that writing software is not unlike writing a huge book. A book long many millions of lines. I explain that the challenge is to maintain the whole story of the book coherent. I stress how difficult this is given that no one can take the time to read the full book, that the authors come and go, and that the book is under constant heavy revision.

Details of our work

The metaphor is quite valid and even holds about more subtle aspects of our work. Here is a summary of the last discussion I had about the book metaphor :

  • (Him) Do you have some ‘gatekeepers’ for different sections of the story to make sure that these parts remain coherent ?
  • (Me) That’s what we would call strong code ownership, this was the norm 20 years ago, but that the industry is moving to more collective code ownership, in order to decrease the bus factor.
  • (Him) What is the bus factor ?
  • (Me) The number of person who need to be hit by a bus to block your development progress
  • (Him) I see, that’s kind of funny. But then, does that mean that any developer can change any part of the code ? Isn’t that dangerous ?
  • (Me) Sure that would be difficult, that’s not really how we do it. We make a lot of efforts to split the big story in many smaller independent ones as much as possible. This way, teams specialize on different ‘chapters’ and work collectively within it.
  • (Him) I see
  • (Me) And if you need to change something in another ‘chapter’ you should ask the guys who know it better to help you.
  • (Him) That makes sense. And what happens if two developers want to modify the same part of the story at the same time
  • (Me) That can happen. People might even want to modify the story in conflicting ways ! We’ve got tools, processes and best practices to minimize this. If nothing is done, we’ll get what we call a ‘merge conflict’.
  • (Him) Ho … I wouldn’t have guessed, but it seems collaboration is really important in your work, isn’t it ?
  • (Me) Sure it is !

We are software writers

All this reminds me of DHH’s talk about writing software :

How I Fixed the Unknown Language Pygments Error in Octopress

Last time I tried to insert a code snippet in my Octopress blog, I was hurt by the following error :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
vagrant@ae4a04cebb73:/vagrant$ bundle exec rake generate
## Generating Site with Jekyll
unchanged sass/screen.scss
Configuration from /vagrant/_config.yml
Building site: source -> public
/vagrant/plugins/pygments_code.rb:27:in `rescue in pygments': Pygments can't parse unknown language: ruby. (RuntimeError)
  from /vagrant/plugins/pygments_code.rb:24:in `pygments'
  from /vagrant/plugins/pygments_code.rb:14:in `highlight'
  from /vagrant/plugins/backtick_code_block.rb:37:in `block in render_code_block'
  from /vagrant/plugins/backtick_code_block.rb:13:in `gsub'
  from /vagrant/plugins/backtick_code_block.rb:13:in `render_code_block'
  from /vagrant/plugins/octopress_filters.rb:12:in `pre_filter'
  from /vagrant/plugins/octopress_filters.rb:28:in `pre_render'
  from /vagrant/plugins/post_filters.rb:112:in `block in pre_render'
  from /vagrant/plugins/post_filters.rb:111:in `each'
  from /vagrant/plugins/post_filters.rb:111:in `pre_render'
  from /vagrant/plugins/post_filters.rb:166:in `do_layout'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/post.rb:195:in `render'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/site.rb:200:in `block in render'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/site.rb:199:in `each'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/site.rb:199:in `render'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/site.rb:41:in `process'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/bin/jekyll:264:in `<top (required)>'
  from /vagrant/vendor/bundle/bin/jekyll:22:in `load'
  from /vagrant/vendor/bundle/bin/jekyll:22:in `<main>'

I had been switching to Docker in Vagrant to host my Octopress installation, but that did not seem related …

Plan A : Upgrade Octopress

It turned out it was related to my migration to Vagrant and Docker.xs This kind of nasty error often comes from obsolete environment. Internet confirmed confirmed it :

  • Ubuntu now uses Python3 instead of Python2
  • Octopress2 uses an old version of Pygments
  • That version of Pygments requires Python2

I thought that I did not want to play this game against open source libs, so I set out to update. Octopress 3 has been released. It’s not yet documented yet, and there is no official upgrade guide, but some guys have done it :

I tried it … but I stopped before the end. It’s kind of working, but fundamental features like category archives required too much Jekylling for me at the time. And themes are still an issue … I mean, stock theme uses a clean and simple gem based update workflow, but most other themes rely on a tricky git clone-merge update workflow.

Plan B : Find a hosted blog engine for hackers

I found quite a few options :

  • Medium Looks cool, but there is no simple migration tool, and I’ve been burned by postero.us shutting down, I want to keep control on my content
  • Posthaven Having loved postero.us, this one seems cool, It’s just I’m not sure the guys are keeping the motivation to increase their feature list with their current business model
  • Ghost Looked great, but too expensive for me (19$ per month)
  • Codrspace This is a special blogging platform for coders, but it seems it has not been updated for a long time
  • LogDown This also is a special blogging platform for hackers. It seemed great and had everything I was looking for … I was ready to spend the 50$ per year for it as it could save me a lot of hassle. Unfortunately, I could not find a way to forward the categories of a post as Twitter hashtags as I’ve been doing with Octopress and Feedburner

In the end, I was rather disappointed by the alternatives, even though I was ready to pay for them.

Plan C : Fix the error in Octopress 2

Eventually, the fix was rather easy. People had fixed it by patching mentos.py file in the pygments.rb gem to run Python 2 instead of the stock Python. That’s not great because it means that I’d have to redo the patch every time I updated my gems, or rebuilt my workspace. Here is what I added to my Vagrantfile instead :

1
2
sudo apt-get -y install python2.7
sudo ln -sf /usr/bin/python2.7 /usr/bin/python

Here is the commit on my Github repo.

Conclusion

I’m sticking with Octopress2 for the moment. Maybe I’ll re-try to migrate to Octopress3 later on, when an official migration tool or guide is available. Meanwhile, I’m still dreaming of my perfect blogging platform :

  • Open source
  • Static HTML generation
  • Self or SAAS hosted
  • Great integration with social media and the web
  • Easy to maintain
  • Plugable themes

I’m looking for my next side project at the moment, that could be a useful one ! Anyone interested ?

Real Developers Ship

If you are a developer and you are not shipping, you’re in the danger zone. I believe it’s mostly your fault, and it’s time to act.

Nothing new

It’s been told before. Famous blogs have said the same thing :

What does it really mean ?

If you are not shipping, are you really developing something ? Maybe it’s more some kind of research project, or maybe you are just there for learning ? Real Developers know what they are in for.

If you are not shipping because the quality is too bad, that’s because you’ve been developing poor quality software … Real developers write quality software. (BTW, shipping bug fixes is not shipping)

If you are not shipping because you don’t have yet enough features, that’s because your MVP is not small enough (despite the current Skateboard metaphor, this is not an MVP). So Real developers follow lean (startup) principles.

If you are not shipping because it’s much work, that’s because you did not automate delivery. Real developers use continuous delivery.

If you are not shipping because someone else said so, maybe it’s time to convince her. Real developers know how to deal with people.

If you are not shipping because you are in a shitty environment … Real developers work on their skills to be able to quit when needed.

Shipping above all else

My point is that every developer or organization containing developers should put shipping value as their absolute priority, and things should get a lot simpler.

Pair Programming, From Pain Zone to Skill Zone

We’re doing pair programming almost all the the time in our team. A few weeks ago, we went to Devoxx Paris, and 2 team mates used pair programming at a hands on session : they finished way before the others and had the time to take a 30 minute break. Pair programming delivers more … when it works.

Flavors of Pair Programming

Throughout our experience, we have identified some situations where it does or does not work :

Pain Zone Skill Zone
Wandering buddies Mates
Expert / Novice Mentor & Apprentice

The Pain Zone

Pair programming can be painful. It can feel like a waste of time or extremely frustrating. Here are the typical situations and how to fix them.

The Wandering Buddies

That’s what you’d get if you asked to newcomers straight out of school to pair program on existing source code on day 1. They’d be dabbling here and there, without going anywhere. Each one would try to get the keyboard to try his way, but in the end, no general direction would be followed.

The cure in this case (as often) is to stop and think. At least one in the pair needs to lean how to pair program. It’s not difficult, that’s just :

  • Discuss and agree on a general plan first
  • Keep a todo list
  • Track your advancement on your todo list while you don’t have the keyboard
  • Stop and discuss from time to time

By starting that, the pair will quickly move into the ‘mates’ state.

The Export / The Novice

That’s when a ‘lone wolf kind of’ expert pairs with a junior. The expert picks the keyboard, dashes through the tasks, doesn’t make sure his pair follows (which usually means it does not).

Regular pair programming means a change in job responsibilities for such expert programmers. If they were expert doers before, they should now become expert mentors or coaches. That’s quite a difference. To them, it means that their goal must be to make his pair finish the task … It can be very frustrating at first. Nevertheless, the overall skill level in teams doing that increases so fast that’s it’s very fulfilling for the team as a whole.

The Skill Zone

The is where we want to be. When it happens, pair programming feels productive, interesting and fulfilling. Contrary to what is often thought, this can happen whatever the skill levels of the 2 programmers.

The Mates

The programmers have equivalent skills on the subject at hand. They organized in a way that they are both learning from each other. By having to verbalize what and why they are doing things, by challenging each others assumptions and by sometimes doing differently than what they’d alone.

The Mentor & The Apprentice

Other times, there can be a real difference in skills for what the pair is attempting to do. In this situation, the goal of the most knowledgeable is to train his pair, who’s goal is to learn. The task becomes some kind of exercise on which to practice.

During a typical pairing session, a pair might switch many times between mentor and mates states. Each one in the pair might assume the mentor’s role for different aspect of the work. At the end of the day, everyone should feel as if they had learned and accomplished good work.

Last Word of Caution

Pair programming is exhausting. Don’t expect to be able to do 40 hours of pair programming per week. After 6 hours in a day, one usually starts to get a lot less efficient.

Both juniors and experienced programmers also need some time by their own, to experiment things by themselves or to complete a simple task without having worry about explaining everything.

These are the reasons why XP stresses having a sustainable pace and leaving space for Slack time

5 Books About Data Driven Lean Software Development

In french, we say “Le cordonnier est toujours le plus mal chaussé” which I found an English equivalent in “the shoemaker’s son always goes barefoot”. I believe this is nowhere more true than in the software industry !

“Software is eating the world” they say. Software is now able to do things that only humans used to be. So why the hell are we driving our projects as if we were a horde of amateur hitch hikers ?

What does it mean to be data driven ?

Being data driven would allow us to answer questions such as :

  • How much is the feature we delivered last week contributing to the bottom line ?
  • How much is the feature we are currently developing expected to contribute to the bottom line ?
  • What are the estimated cost and value of increasing our test coverage of 1% ?
  • What are the estimated interests and nominal amounts of our current technical debt ?
  • Which is the most valuable : improving our build system or building this new feature ?

Most projects I’ve worked in have absolutely no clue about the answers to these questions. The decision is left to experts, to the one with most influence, or simply to the developer, who can do how he thinks is best …

The books

Hopefully, some people are thinking differently, they believe it is possible to quantify all this, they even explain how !

Running Lean: Iterate from Plan A to a Plan That Works by Ash Maurya

Details a very practical guide about the lean startup process, which is a very good starting point to any kind of lean software development.

Kanban: Successful Evolutionary Change for Your Technology Business by David J. Anderson

This book explains with real world examples how to use Kanban board to control your work queues and improve your flow of work, a real basic for any lean product development.

The Principles of Product Development Flow: Second Generation Lean Product Development by Donald G. Reinertsen

This book is rather theoretical, but it links all subjects together : lean startup, risk management, Kanban, and economics. I guess it’s a must read on the subject.

How to Measure Anything: Finding the Value of Intangibles in Business by Douglas W. Hubbard

If the flow book gives a big picture view of what you want, this one explains how you can actually measure all the aspects of your product development in $ value.

Waltzing With Bears: Managing Risk on Software Projects by Tom DeMarco and Timothy Lister

Although this book is getting old, and is a bit outdated when compared to agile development practices, it provides real world examples of how scientific measurement can be applied to software product development.

An opportunity

Reading these books was a real eye opener for me. The software development world is plagued with cargo cult and supposed best practices. We follow advises, but most often without verifying if they actually work ! I believe that by applying the techniques in these books, we could create standard ways to measure the values of productivity, technical debt, quality, testing …

I see real opportunities to avoid a lot of useless argument between proponents of A and B, but also to communicate better with all stakeholders and finally, to reduce stress for all of us.

A Lego Office Experiment

Rome wasn’t built in a day, neither will be your Lego Office … This week is Devoxx in Paris, for the occasion, we decided to stream a Lego office experiment that my team did.

What are Lego Offices

I wrote about Lego office in a previous post, as a concept and a set of material that would allow teams to easily configure their workspaces as they wish. At that time, I thought of :

  • laptops
  • extra monitors
  • tables and chairs on wheels
  • movable walls
  • movable white boards
  • movable wall screens
  • a good Wifi

Our experiment

The last floor of our building in Paris has a nice (really) open space area with rolling tables that we could use. We moved all the team there for one week. As we don’t have laptops with Wifi, we simply asked for enough power and network plugs.

The place is large enough to have dedicated rest and meeting space. We managed to create a visio setup using our build monitor. We even had a remote lunch once, which we should repeat more often !

What’s missing

Obviously, this was a quick experiment, nothing like a fully architectured Lego workspace, still it helped us to discover issues and blocking points. It turns out that cables are an issue !

After getting some feedback throughout the company, it turns out that some developers would never trade their high end connected desktop workstation for a Wifi laptop. Our C++ build can be pretty CPU and network intensive … I guess that means that a real Lego office might require one of the following :

  • A technical floor with all the place required to bring any kind of cables to your machine
  • Powerful servers in a data center to do the grunt work for you
  • Single cable machines (Apple’s USB-C is the only one I can think of right now)

Last minute question

Thinking of it … could different remote offices such as your home, or a co-working space be considered as Lego blocks ?

How We Decentralized Our Company’s Training Program

Maybe your company too has a development program you can use to track and organize your training path. That’s already great ! It is often not perfect though. Sometimes, the initiative in itself can feel like an afterthought. Some other times, the process is completely left to the employee and his manager, with varying results, from great to forced upon everyone.

How did we rebuild this program in our team

As you might remember if you read my previous posts, we started a self organization initiative using a delegation board. When we tried to decentralize the end of year evaluation, we started discussing the subjects of personal development. We decided to have a dedicated workshop about this subject

Guiding principles

In our company, the development program allows a manager to assign specific learning goals to his collaborators, and anyone can also create his own goals to justify going to specific trainings. Starting from that, we agreed that the development program should be used to help anyone in the team to improve skills that are interesting for both the company and the employee. If the employee is interested in something completely unrelated to work, then he should obviously tackle that in his own time. If the company needs its employee to acquire some immediate mandatory skills, then that should be part of the daily job and subject to the end of year evaluation.

The idea is to try to find a win-win combinations, were employees are motivated to work their best at a company that they know helps them to full-fill their long term goals.

What did we came us with ?

This workshop was quite effective :

  • First of all, everyone is responsible for his own development, it’s a chance offered by the company. It cannot be forced unto people.
  • It is based on volunteer mentorship : everyone should have regular meetings with his mentor to asses his progress on his long term goal, to try to get feedback and ideas about how to move forward. We started by saying that the team leader is the default mentor, but anyone can find another mentor at any moment.
  • The process starts with the search for a long term goal. Examples are “I want to be an agile coach”, or “I want to be a performance expert”
  • With the help of the mentor if needed, the long term goal should be split in yearly objectives and tracked using OKRs
  • Everyone is free to track his progress has he wishes. Visual tracking makes the discussion with the mentor a lot easier. We have been looking at tools to track OKRs but it was surprisingly difficult to find one that suited our use. Most are enterprisy tools with manager-managee relationships. We just wanted simple personal tracking tools. For my part, I adapted a Trello board I found on the internet.

Conclusion

After doing a few months of this, I can say that this has a positive aspect on motivation, people told me that they felt more in control of their destiny. Something else I noted, is that junior developers need help and guidance. They often don’t really know what they want to do in a few years, so they really need a mentor to help them find out the what and the how. The other side of this is that a team obviously needs senior developers to act as mentor … among other things.

Trello Templates to Boost Your Remote Retrospectives

I already wrote how we started to use Trello to do our remote retrospectives. A while ago, my team mate Bilal Tayara started to collect our activities into trello templates.

New activities

I just imported new activities we have been using in the past months :

  • SMART Goals
  • 5 Whys
  • Circle of Questions
  • Communication
  • Engagement
  • ESVP
  • Fishbone
  • Focus On/Focus Off
  • Force Field Analysis
  • Gifts
  • Glad Sad Mad
  • Health Radar
  • HHH
  • Hot Air Balloon
  • How are you feeling today ?
  • Improvements Brainstorming
  • Learning Matrix
  • Note To Self
  • Plus / Delta
  • Problem –> Action
  • Punctual Paulo
  • Retrospective Planning Game
  • Risk Brainstorming
  • ROTI
  • Sail Boat
  • Satisfaction Histogram
  • Take Away
  • Team
  • Team Radar
  • Warmup Question
  • What happened since last retro ?

How to use these templates

Clone the template to your organization every time you want to use it. There are instructions in the template to explain how to use it. For more detailed instructions, have a look at the original :

One last thing : in order to improve this collection, we’d like to have your feedback on existing activities, or your own new templates for other activities.

How to Deal With the Incentive System in an Agile Team ?

For an organization, there’s something schizophrenic about valuing team work, wanting agility but sticking to individual objectives and assessments.

As highlighted in Build To Last genuine company values must permeate all aspects of the organization. Being agile also means putting teamwork first and that is quite incompatible with typical individual management by objectives and bonuses.

This is nothing new, many have already written about the subject

The problem is you usually don’t have the choice ! Whether your team wants to or not, the chances are that your company already has a mandatory individual bonus scheme in place.

Our delegation initiative

As you might remember if you read my previous articles, our team decided to move to more self organization. After identifying the end of year evaluation as something that could be further delegated, we embarked on brainstorming sessions about how to do it.

We discussed quite some options during our workshops. Things like :

  • What is the goal of the annual evaluation ?
  • Should some people have a central role in the process ?
  • Should we encourage efforts or results ?
  • Should evaluations be public or private ?
  • What’s at stack at the annual evaluation ?
  • Should individual learning plans be part of the evaluation ?

What we came up with

As a guiding principle, we agreed to value efforts rather than results. Sometimes, results are independent of efforts, and through efforts, it is always possible to get the best of any situation, even if the best is not a good result in itself.

Throughout the year

We are all sharing the same team objectives :

  • “the team finishes the prioritized epics” : we do regular epics prioritization sessions with our product owner and add the main epics to everyone’s objectives
  • “work on the same subject as the whole team” : we have an overspecialization matrix that uses correlation to make sure that everyone touches the same parts of the system as the rest of the team

Every 2 months, we are doing 360° feedback sessions, for everyone to give continuous feedback to others.

Optionally, we setup Kudo boxes to give and receive thanks from others.

At the end of the year

When the time of the official end of year evaluation comes, we do 360°, 1 to 1 rating on the following topics.

  • commitment & courage (doing not so fun work, find difficult compromises, say the truth, strive for quality work)
  • helping others (giving a training, providing coaching, help others, do some evangelization)
  • self improvement (taking feedback into account, improve technical skills, work on soft skills)
  • team player (build friendly environment, contribute to team improvement, positive attitude, use a non violent communication)

Grades are signed, and anyone can ask someone else for a discussion about his grades.

Overall, everyone gets an average grade, which we use to suggest a company grade (below, achieved, exceeded) to the HR system. It then gets homogenized with the rest of the company, but this is then out of our control.

One thing that made this possible is also that after taking a close look at the company grade system, we saw that not so much was at stake … Provided we make sure everyone has what it takes to do the job, anyone can get an achieved evaluation by doing enough efforts. Getting an exceeded is like a bonus for a particular good year.

Last word

I’m pretty sure we could make this more simple in an environment with more end to end feedback (start-up or pizza teams) but we feel that our current alternative is a lot better than having the manager deciding everything on his own.

Make Hiring Everyone’s Business

Let me tell you a typical hiring story. A bit more than 10 years ago, I was contractor at a bank on a C++ front office application. The system had initially been well engineer, but it had been stuffed together merged with another system that had a completely different architecture. Ha, and there were no automated tests. As you’d guess, we had quite our dose of bugs, and maintaining this application was not easy.

The manager wanted to hire a C++ programmer to beef up the team. Along with the division’s architect he was conducting interviews to find just that guy. A few weeks later, he found a known C++ & Windows local expert. A few weeks later, the team unanimously declared that we could not work with him because he was both pretentious and incompetent.

I’m pretty sure every developer already witnessed a similar story.

Managers do the hiring

This is the usual practice. The motivation is that managers have more experience so they should be better at it, and also that it alleviates developers from non-programming work. As we saw in the introduction story, there are issues with this approach too :

  • it’s pretty bad for the team ‘fit’ : there is no team building here. You often end up with a collection of individual working at the same place rather than with a real team.
  • during these interviews, the candidate only interacts with a particular type of profiles, who can tell how he’ll be doing with more junior team members for example ?
  • as usual, this administrative work removes some programming from the manager’s time. That’s often not a good bargain
  • it decreases the team sense of control, as it is being ‘forced’ to work with someone they did not choose. This makes it harder for them to take responsibility of what they are trying to achieve.

How we self organized hiring

As I already explained in a previous post we engaged in various self delegation initiative by doing a delegation board. During this meeting, we identified hiring as something that could be self delegated.

In order to fix that, we started a workshop for 1 hour per day with a sub group of the team. We started with a brainstorming, then we voted, and finally presented our proposition to the rest of the team, which actually validated it. The whole thing lasted for one week (5 hours).

What we decided for hiring

Here is the process we have decided to follow to handle hiring team members

  1. First, everyone in the team is responsible to speak up when he thinks we need to hire someone, during our retrospectives for example
  2. To handle new applications, we have prepared an online interview. It contains open questions and online coding exercises, on cyber-dojo, which allows us to track all the candidates iterations
  3. We do a round robin to find out who will follow the candidate through the hiring process. This means taking notes, organizing interviews, keeping the rest of the team up to date.
  4. When a candidate passes the online interview, we will receive him for an technical and general interview with 2 people from the team.
  5. All the team will then meet the candidate for a coffee break, in order to know if he will fit in the team
  6. We’ll then play the Google hiring committee until we reach a consensus
  7. When in doubt, we’ll do extra interviews, and repeat

That’s just what we came up with, if you start this road, you might well come up with a very different process.

One more thing

It’s great to have a selective hiring, but you’ll also need to improve the input of the hiring pipeline, but that’s another story