How I Fixed ‘Invalid Byte Sequence in US-ASCII’ Exception With Octopress (and Vagrant)

I don’t know if you noticed, but Octopress 2 preview is slow as hell on Virtual Box. I wanted to try using Docker instead. So a few weeks ago, I started to upgrade my Vagrant, Virtual Box and Docker stack …

Hell loose !

A naughty dog with a banner "I'm in upgrade hell"

I know all open source software needs to be kept up to date at all time, but it’s not that easy. Especialy when the versions in the Ubuntu repostiories are really old and you had to install them manually. Anyway, after a bit of tedious work, I finally got to the point where my Octopress blog was running on a brand new stack. And I got this error :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
vagrant@239c4077ae16:/vagrant$bundle exec rake generate
## Generating Site with Jekyll
unchanged sass/screen.scss
/vagrant/vendor/bundle/gems/liquid-2.3.0/lib/liquid/htmltags.rb:44: warning: duplicated key at line 47 ignored: "index0"
Configuration from /vagrant/_config.yml
Building site: source -> public
YAML Exception reading index.markdown: invalid byte sequence in US-ASCII
/vagrant/plugins/backtick_code_block.rb:13:in `gsub': invalid byte sequence in US-ASCII (ArgumentError)
  from /vagrant/plugins/backtick_code_block.rb:13:in `render_code_block'
  from /vagrant/plugins/octopress_filters.rb:12:in `pre_filter'
  from /vagrant/plugins/octopress_filters.rb:28:in `pre_render'
  from /vagrant/plugins/post_filters.rb:112:in `block in pre_render'
  from /vagrant/plugins/post_filters.rb:111:in `each'
  from /vagrant/plugins/post_filters.rb:111:in `pre_render'
  from /vagrant/plugins/post_filters.rb:166:in `do_layout'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/page.rb:100:in `render'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/site.rb:204:in `block in render'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/site.rb:203:in `each'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/site.rb:203:in `render'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/lib/jekyll/site.rb:41:in `process'
  from /vagrant/vendor/bundle/gems/jekyll-0.12.1/bin/jekyll:264:in `<top (required)>'
  from /vagrant/vendor/bundle/bin/jekyll:23:in `load'
  from /vagrant/vendor/bundle/bin/jekyll:23:in `<main>'

After some searching on the internet, I found this post by Alex Zeitler that made me figure out that locales were not set correctly by default on my vm. I just needed to set this once and for all in my Vagrantfile

1
2
3
4
5
6
7
8
9
10
config.vm.provision "shell", privileged: false, inline: <<-SHELL
  if [ -z "$LANG" ]; then
    echo "Setting LANG to us UTF8"
    echo 'export LANG=en_US.UTF-8' >> ~/.bashrc
  fi
  if [ -z "$LC_ALL" ]; then
    echo "Setting LC_ALL to us UTF8"
    echo 'export LC_ALL=en_US.UTF-8' >> ~/.bashrc
  fi
SHELL

After reprovisioning my box, generate and preview were working as expected !

:–( Now I’ve got a pygement-ruby issue to deal with, I guess I’ll be back on the subject soon …

How to Keep Programming When Assigned a Management Job

Remaining a competent developer is a career long effort, if you stop programming, you’ll loose it ! As time goes, we are regularly offered pushed into management positions, sometimes by cluelessness, other times by cheer necessity ! Be it temporary or long term, here are some techniques to remain a relevant developer.

How Joe became a manager

1
2
3
4
5
6
7
def work
  if manager?
    go_to_meetings
  else
    program
  end
end

Imagine you’re Joe, an expert developer in a small software company. Everything is going fine, he’s working on interesting subjects with 4 other, rather junior, team mates. Management is OK, at least good enough not to cause too many troubles.

Suddenly, the team manager leaves for a better position in another company. Unprepared for this, the small organization has difficulties to find a suitable replacement, and, you’ll guess it, Joe is asked to take on management responsibilities until someone gets hired. How is Joe going to continue to do a good job at both programming and management ?

His first reaction

Two weeks into the job, Joe takes a step back and summarizes :

  • he feels depressed when he does not program for a long time
  • he has the feeling he’s been jumping from A to B to C to Z to A, and so on for two weeks without actually getting a lot done
  • he feels everyone is waiting for him to do things before they move
  • his mailbox is starting to make him nervous
  • it’s difficult to program anything if you are interrupted by meetings throughout the day

Joe figures out that he needs to reduce his management work if he wants to get back to programming. After setting up an efficient TODO list. He has already read some personnal effectiveness books, so he knows how one can reduce his amount of work :

  • say no : he’ll have to keep this in mind all the time, especially when accepting meetings or when asked for some new work
  • automate : some tasks can be automated, others can be partly automated by a good process. He’ll have to regularly stop and think to improve how he works
  • delegate : he’d like to push more work to the team, but it will take some time. He decides to read things on self-organized team to know how to do

Joe’s measure of programming time

In order to get some continuous time for programming, Joe books ‘programming days’ in his calendar. This is also a way to say no to other meetings during this period. He starts with all Mondays, he hopes he’ll be able to book others days as he manages to do more programming. His ideal would be to book all week for programming (while still having the management job get done !)

This works well, and gives Joe a motivation boost since he has the opportunity to program again :–). There are still a few glitches though :

  • Joe often doesn’t program ‘on his own’ on Mondays because he is spending is time with junior programmers who are really appreciating his help
  • Joe once made the mistake of committing to a critical programming task that he did not manage to finish in 1 day and had to hand it over to another developer, losing more time

Joe is realizing that he is more of a programming coach than a direct developer now, and that he should not commit on critical tasks but rather help others to do so. Pair programming is becoming the norm for him.

Communication time

While doing his full programming days, Joe realized that emails can wait. If people want an immediate answer, let them use the phone ! He now reads and answers his emails in the morning when he arrives at work, at noon just after lunch and on evenings, just before leaving. That’s good enough for a day !

Joe also had to interrupt his programming day once for an urgent meeting … he now reserves 1 hour at the end of his programming days to handle such urgent meetings without interrupting his programming time.

Self organized teams

Meanwhile, Joe read some books about self organized teams, such as Management 3.0 Workout by Jurgen Appelo.

Self organization is some kind of aggressive delegation. Joe sees management as described in these books as a way to increase purpose and motivation throughout the team, while getting him more time for programming.

This kind of management perfectly suits agile teams. His team is already using some agile practices, such as automated testing, some kind of continuous integration, and quite a few others, to varying degrees. Joe decides to embark his team on a road to self organization, and to start by applying all the standard Scrum and eXtreme Programming practices. In parallel, he introduces the rolling Scrum Master role, where sprint after sprint, a different team member is responsible for :

  • organizing the recurring team meetings
  • representing the team in outside meetings
  • grooming the backlog before the demo and planning

This alone frees Joe 1 or 2 days of programming every week. He now helps his team mates to master all the technical agile practices they are not used to.

More self organization

A few months later, the team is doing well, Joe has some time for programming, but he still has some pure management job to do. From what he read from self organized teams, even these subjects can be delegated ! He decides to start workshops with the team :

  • How can we handle vacations in a self organized way ?
  • How can we handle the hiring in a self organized way ?
  • How can we handle performance feedback and pay raises in a self organized way ?

The journey is long and rough : one team member left as they were going seriously into agile. Nevertheless the team is already more motivated than it ever was, and delivers more value than it ever did.

The end result for Joe

Joe is now programming nearly as much as his team mates ! Most of his programming time is coaching time though. He does not sit and hack his way into a feature as he used to do. He’s missing that a bit, he’s also missing learning new technologies.

That’s why he started working on side projects at home, using the latest hype JavaScript framework. He’s reading generalist programming books also, such as the 7 XXX in 7 weeks series which allow him to learn without loosing time on technical quirks he’s not interested in at the moment …

Good luck Joe !

How I Fixed ‘Warning: Authentication Failure. Retrying’ With Vagrant

Lately, I went into an upgrade cycle : Ubuntu, which forced me to upgrade Docker, and then Vagrant … you know the story. Unfortunately, my vagrant config did not want to start anymore with the following error :

1
2
3
4
5
6
7
8
Warning: Authentication failure. Retrying
Warning: Authentication failure. Retrying
Warning: Authentication failure. Retrying
Warning: Authentication failure. Retrying
Warning: Authentication failure. Retrying
Warning: Authentication failure. Retrying
Warning: Authentication failure. Retrying
...

I tried to destroy and re-create my box with the same result : it eventually timedout, but the provisions were not executed.

After searching the internet and a lot of experimentation, I managed to make it work by commenting out the following line in my Vagrantfile

1
# config.ssh.private_key_path = "~/.ssh/id_rsa"

I think I added this a long time ago to be able to push to github from my Vagrantbox, but after trying it, it work with the difference of my having to validate the remote ssh key, that’s not of a big deal.

Most Common Ways to Speed Up an Algorithm

Algorithms are hard, and making them fast is even harder … But there are shortcuts that work quite often !

The Challenge

Imagine you just arrived to your new job, and you are asked to make a part of the system faster. After a bit of investigation, you discover that most of the time is spent in some weird in-house algorithm that seems to take forever. How can you optimize this without deep knowledge neither in algorithm science nor in the code itself ?

Here are 4 tricks to reduce the complexity of algorithms (I’m using fairly basic examples for the sake of understandably. Most of these exact examples could be done better using standard libraries, but I hope it will be easy to adapt to other situations) :

Replace a nested loop by first building a hash and then looping

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# before
orders.each do |order|
  client = list_of_clients.find {|client| client.id = order.client_id }
  handle_order(order, client)
end

# after
clients_by_id = {}
list_of_clients.each do |client|
  clients_by_id[client.id] = client
end
orders.each do |order|
  handle_order(order, clients_by_id[order.client_id])
end

This reduces the complexity from O(2) to O(1). This is tremendous. On large lists O(2) algorithms are terrible.

Remove unnecessary accumulations

The most classic example is the use of a string buffer :

1
2
3
4
5
6
7
8
9
10
11
12
13
// Before
report = ""
line_items.each do |line_item|
  report += line_item.to_s + "\n"
end

// After
report = []
line_items.each do |line_item|
  report << line_item.to_s
  report << "\n"
end
report.join

Again, this reduces the complexity from O(2) to O(1). Every language has variants of Java’s StringBuilders. This does not only apply for strings, it works any time you are repeatedly accumulating results inside a loop but where you could do it only once at the end.

Cache intermediate or previous results

This is called memoization. Some algorithms (especially recursive algorithms) repeatedly compute the same thing again and again. Spotting this pattern is an opportunity to move an algorithm out of exponential complexity. For example, Dijsktra’s algorithm for finding the shortest path in a graph uses this technique to go from O(en) to O(n2) complexity. If you suspect this could be helpful, your best friend is logging to trace actual parameters and results.

A word of caution : using memoization with mutable inputs or outputs will harm your mental health.

Zip merge

There are 2 ways to merge sorted lists into a unique sorted list : the fast, and the slow …

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# The slow
(list_1 + list_2).sort

# The fast
i_1 = 0
i_2 = 0
result = []

while i_1 < list_1.size and i_2 < list_2.size
  if list_1[i_1] <= list_2[i_2]
    result << list_1[i_1]
    i_1 += 1
  elsif list_2[i_2] <= list_1[i_1]
    result << list_2[i_2]
    i_2 += 1
  end
end

while i_1 < list_1.size
  result << list_1[i_1]
  i_1 += 1
end

while i_2 < list_2.size
  result << list_2[i_2]
  i_2 += 1
end

Obviously, the slow version is a lot easier to read than the fast one. And the fast one could benefit from a bit of refactoring also … Nevertheless, the slow version is at best in O(n.ln(n)) whereas the fast on is in O(n). On large data, that can make a big difference.

Is that all ?

Obviously not, there can be a lot of other things going on slowly in algorithms, but from my experience, a software engineer can have a good career without knowing more about algorithms theory than that.

In the end, you manage to optimize this in-house algorithm, you become the company’s hero, you need your job and get a pay raise !

End word

The fact is, in 15 years of writing software, I did not write a lot of algorithmic code. I can categories my working with algorithms in 3 :

  1. Write a simple algorithm for a non performance critical feature
  2. Optimize an existing somewhat algorithmic part of code
  3. Write a complex algorithm for a performance critical part of the system

Case 1. is not really an issue since however the code will be written, it will run fast enough. If you’re in case 3, there’s no shortcut, you’ll have to dig deep into algorithms and optimization, this happens rather rarely though. This leaves us with case 2, which I just wrote about.

Interestingly, my current job is deep into case 3 ! We’re building a risk engine for corporate markets and are borrowing a lot of techniques from database science … which is, you can guess, rather algorithmic !

How We Started Exploratory Testing

Manual testing is important. Here is how we got to love exploratory testing.

Initial situation

At work we are building a risk computation engine for the financial markets. It uses a DSL to describe the exact computations to estimate the risk on the data it knows. This in itself is already complex enough to justify the heavy investment in automated testing we did.

With 90% of automated test coverage, Cucumber scenarios to verify quality, everything should just work … shouldn’t it ?

First try at exploratory testing

In The art of agile development, James Shore details the practice of exploratory testing as a great way of both :

  • improving the quality of the product by finding bugs
  • improving the process

That’s why we gave it a try. Here is the recipe for an exploratory session :

  1. Book 1 hour for the full team to do exploratory testing
  2. Prepare a downloadable zip with all the material required to run and test your software
  3. Ask everyone to pick a particular aspect of the system to test during this session
  4. Record bugs when you find one
  5. Spend 30 minutes just after the session to filter duplicate bugs and make sure they are well described

Obviously, we also added our special sauce

  1. We were to do exploratory testing in pairs, as we do programming, to find more bugs
  2. We gamified it by granting a price to the pair that finds the most bugs. Do you remember how I brought fruits for sale in the office ? It turns out we are slowly earning money with the fruits … enough for the price to be 5 fruits each for the wining pair !

This transformed developers into ferocious testers ! I guarantee that with such incentives you’ll find bugs … as we did.

Doing it systematically

We took some time to fix all these bugs during a few sprints. And did another exploratory testing session a few months after …

As we were still finding bugs in the second session, we decided to make them part of the every sprint. As we got better at testing, a lot of bugs started to get uncovered ! So much that we had to change something.

Improving our process

We meet in retrospective and here is what we decided :

  1. Fix all bugs before working on any other story (obviously, automated tests are added in the process)
  2. Classify what exactly is a bug. For example, for us, a bug is something that used to work or silently returns bad results or corrupts the data
  3. Add exploratory testing by another pair to our definition of done

The number of bugs we find during exploratory testing sessions is starting to decrease. We hope that we’ll soon be able to do exploratory testing on the fly, as part of our daily work and to completely remove the specific sessions.

Benefits

Exploratory testing brought us a lot.

  • Obviously, the product is a lot more solid
  • We are surely saving time that would have been lost if the bugs had been uncovered by the users
  • Fixing the bugs forced us to fix some technical debt : bugs often came from areas of the code that we were not so proud or confident of

Exploratory testing is a nobrainer. All teams should do it.

So, in the end, as in the old days, we are back to regular manual testing … but only for exploration.

How to Write Good Performance Stories

If you’re having difficulties writing good performance related stories for your project, that’s no surprise ! We’ve been through the same troubles and we found a way that works a lot better.

Solution 1 : Performance is a feature

Right ? In typical agile way, a story about performance would be written like

As marketing, I want the page load to perform under 1 second, In order for the customers to stay on the site

If the performance bottleneck and the fix are obvious, that might work very well. If that’s your case, then go on, that’s the simplest way!

Unfortunately for us, we are not in that case. At work we are building a risk engine and we need to perform extremely well on a wide set of different scenarios. Imagine a story such as

As a risk manager, I want the VAR scenario to compute in less than 1 second, in order to have real time data

  • we’re pretty sure that there won’t be 1 but many bottlenecks to fix before reaching the expected performance
  • speeding up the VAR scenario might slow down other scenarios

That makes this kind of story too fuzzy. We tried them, and we had difficulties to estimate and close them.

Solution 2 : Back to the traditional way

Without an agile backlog, developers would have worked on tasks such as

Add caching to the computation engine

That’s estimable, but we’ve got no clue of the why ! When it’s done, we won’t be able to know whether it’s worth to keep it or not. In fact, it’s just not a story) !

Solution 3 : Mixing both

Here is how we we are now writing performance stories at work :

As a risk manager, I want to have caching in the computation engine, in order to the VAR scenario compute under 1 second (in order to have real time data)

It’s now estimable, we know what it’s there for. It’s obviously not enough though : we know that we will very likely need to do other performance stories after this one.

Embrace uncertainty

The fact is that in our context, getting better performances is hard :

  • it requires a lot of work
  • we don’t know how much at the beginning
  • we often try things that don’t work

This makes the whole project more uncertain, so we’re better off embracing this uncertainty in the way we write and prioritize our performance stories.

Great Developers Are Free

For many reasons. But mostly because they are key to efficiently growing a software organization.

A tale of 2 teams

Let’s have a look at 2 software teams in the world of corporate finance.

The junior in-house team

One is an in-house software development team in a large bank. The project already has a bad reputation among developers.The bank has difficulties to hire so the team is constituted of 10 rather junior developers. Unfortunately, without guidance and long term vision, the quality of the code suffers. It looks like an ad-hoc composition of various technolgies, glue code, and reinvented frameworks. The overall result is a barely good enough product that costs a small fortune in maintainance and support. The user experience is awful, which make it difficult to the the team leaders and the users to collaborate effectively. As a result of all this, the project is suffering from high turnover. Managing this project is really challenging … and kind of depressing.

The experienced software team

Now let’s imagine the same product developed at a software house. It’s been on the market for a while now, but new features are regularly added to it at a good pace. The product is solid and the users are happy using it. The structure of the team is completely different : 5 developers, mainly experienced, coming from various backgrounds. As a result, the team builds on all their different expertises to build real competitive advantages. They tend to get into healthy debates about a lot of things, such as :

  • Should we re-use or re-write ?
  • Is this gold plating or plain technical debt ?
  • Which technology should we use to build this ?

In the end, that makes the product even better. The humane side of the team is also completely different. These experienced developers have all been exposed to the big picture during their careers, and they know things like :

  • ‘Business talk’ and so they can discuss product topics with the product managers
  • The best way to go through chores at work is to do them right now
  • Enough management, testing, ops to make the team self organized

The single junior developer in the team tends to mimic this model, and has ease finding mentors, to teach him their trade and to give him career counsels.

As a result, the product and the team remain healthy, and keep providing benefits to all stakeholders.

Benefits of experienced teams

First, from a purely short term financial point of view, it’s a no brainer ! Even if you pay experienced developers twice as much as juniors, the costs will still be on par. But the difference in created value is huge !

Let’s then have a look at the longer term, organizational aspect of things. Obviously, managing such teams requires a lot less work ! Both because of their size and because experienced developers tends to manage themselves very well … Promoting self-organized & cross-functionnal teams is a great opportunity to reduce the management cost and friction, making the organization more reactive and cost efficient in the long run.

What is an experienced developer ?

Obviously, I’ve worked with people that were experienced on paper, but not in practice. I read that repeating 10 times the same year of work only amounts to 1 year of experience.

If I was asked to give a definition, I’d say that experienced developers have worked on different kind of systems (embeded, web, client, server …) using different technologies (Java, Ruby, C, Spring, Rails, Javascript …). Their experiences need not be professional, I’ve seen a physics teacher that hacked himself into a great hacker through side projects and open source. Speaking of the subject, good developers always spent a lot of time learning, reading, doing side projects and contributing to open source. Some can be found in local user groups and online programming communities.

So How do you get them ?

I can only think of 3 obvious ways :

  • train them
  • keep them
  • hire them

I’m pretty sure training them will not be an issue if you already have enough of them. So that brings us to the 2 other points.

Lot’s of articles have been written about attracting and keeping the best developers. Some companies even made it their differentiating point. Here are a few links :

These are just a few examples and are by no way the only things that motivate developers. To summarize, provide the best working conditions (careful, that not money), and communicate a lot about it.

Hiring is too important to be left to HR

Jurgen Appelo said

Management is too important to be left to managers.

I guess the same thing can be said about hiring.

Engineers are absolutely required to take the main role when hiring other engineers. Companies that seriously want to have the best developers on board are spending substantial engineering time on hiring. Here are some well known examples :

  • Google is known to conduct around 9 interviews before hiring someone
  • In its Handbook for New Employee Valve explicitly states that hiring is your most important role

Endword

As Tom DeMarco said about quality, in Peopleware :

Great developers are free, but only for those who are willing to pay for them

How to Do Screencasts on Ubuntu

It is easy to record and share great screencasts on Ubuntu (and I guess on Linux in general).

Recently, I had to create a screencast to demonstrate how to use Storexplore, a gem I wrote that transforms online stores into APIs. Here is the result

Don’t lose your time searching for anything else than SimpleScreenRecorder (I did, and actually did lose quite some time). Follow the installation instructions :

1
2
3
4
5
sudo add-apt-repository ppa:maarten-baert/simplescreenrecorder
sudo apt-get update
sudo apt-get install simplescreenrecorder
# if you want to record 32-bit OpenGL applications on a 64-bit system:
sudo apt-get install simplescreenrecorder-lib:i386

It’s dead simple to use, just try to record your first video, and you should know it all !

At the difference of other tools I tried (I’m looking at you RecordMyDesktop) I could easily encode my video to efficient formats (mp4, h.264 & mp3) that are well known on the internet, and in particular, by YouTube.

My Dream Lean Software Development Tool (Lean Software Development Part 9)

Current software project tracking tools suck !

A stamp that writes "SUCKS"

By writing this series of posts, I realized that they should help us to prioritize rather than just serving as a nice GUIs on top of a ticket database.

Let’s take the story of developer Joe as an example. Joe is working for Megacorp, where he makes a good living, but feels like a small cog in the machine. He has lost the thrill of programming … He’s still staying informed about latest trends in software world, but his job involves aging technologies. He knows that if he wants to remain in the course, he has to change gears. One day, he decides to do something now, and starts a side project in the hope of someday making a living out of it.

He does not have a large amount of money to invest in his project, so he decides to bootstrap his product using Lean Startup. He reads and understands that there is a lot of different kind of activities to do if he wants to realize his dream : experiments, marketing, sales, design, coding …

Unfortunately, he’s so tired of programming boring stuff that he contracted the developer bias : he starves to code cool things again ! He knows the rest is important, he tries to do some of it, but he just doesn’t realize how much needs to be done. So, without realising it, he wastes a lot of time coding something that people eventually don’t buy. After 1 or 2 years of hard work, a working product but no users, he eventually quits, quite depressed …

I guess you don’t want to be Joe ! I know you’ll think it won’t happen to you … believe me, it can, I learned it the hard way !

Now suppose you had a tool that estimates the expected values of all different tasks :

  • By applying Monte Carlo simulations to probabilistic value and size ranges, it could compute which features we should work on first. As a side effect, this could give us some release date estimates for a feature, depending on its position in the backlog.
  • By having explicit experiment tasks with target value or size ranges, it could compute the value of information and prioritize experiments versus other tasks.
  • Crossing informations from IDEs, VCS, Quality Trackers and CIs, it could detect technical debt items (aka. productivity opportunities), compute their values in the long run, and prioritize them with other tasks.
  • If we found a way to integrate marketing, sales and design tasks, any kind of activity could be stored in the backlog, valued and prioritized.

Here is what the backlog could look like :

All different kind of activities (developments, marketing, experiments, refactorings) would fit in the backlog. The system would estimate the value of every task for Joe, advising him what to work on first. It would make it obvious just how much market experiments are important when starting ! With such a tool, he is a lot more likely to build his new life.

The good thing with such a tool is that it could be built by feeding itself. Eating its own dog’s food has always been the nice thing about writing software tools.

I’d like to know, what do you think of the idea ? Would you use such a tool ? Is it worth me spending my nights building it ?

This was 9th and last part of my suite of article about Lean Software Development, Part 8 was How I’ll Measure the Lean Startup Value of Information in My Next Side Project.

How I’ll Measure the Lean Startup Value of Information in My Next Side Project (Lean Software Development Part 8)

There must be a way to know the real money value of Lean Startup ‘knowledge’.

Lean Startup is about gathering ‘knowledge’ through experiments instead of building things. Flow book suggests that the value of an information is its expected impact on the bottom line. So by combining both, there should be a way to compute a quantitative value for the knowledge.

The overall plan would be :

  1. Start with a rough business plan, but assume that all that’s in it has got 1 chance out of 10 of being right
  2. Design experiments to precise the likelyhood of what’s in the business plan
  3. Compute the value of each experiment as the difference of the expected bottom line after and before running it (This should be possible if relying on the business plan for other parameters)
  4. Work on the task with the highest expected value, be it an experiment, a development, a sales task, or whatever
  5. Adapt you business plan with any new information
  6. Repeat
  7. If it becomes obvious that the expected final earnings are too low, pivot

Flow explains that all the principles are pretty robust to bad inputs and errors. In short, the final estimations of earnings are still a lot better than without using any specific method. In the case of Lean Startup, it means that a false business plan is a good enough starting point.

How to assess the likelyhood of a figure ?

By default let’s assume the value of a feature is somewhere between 0 and your highest estimate. As we’ll go through experiments we’ll be able to narrow this value range and have a more accurate estimate for the real value.

To keep things simple we could assume that the probability of value is uniformaly distributed within this range. From what I read in Waltzing With Bears, here would be a more realistic probability distribution.

How to estimate the value of an experiment ?

I’m not going to go into the details here, because the logic is exactly the same as with technical experiments (which I already wrote about)

Since then though, I read Waltzing With Bears which presents a way of doing Monte Carlo simulations in order to estimate risks on your project. I guess the same thing could be used for value, which would save us from a lot of complex math.

When to stop doing experiments ?

Isn’t all this overkill ? One could argue that the point of Lean Startup should be to build just enough knowledge ! In fact, this value of information concept naturaly helps to find out when running the experiment is not worth it compared to actually doing the real thing.

BTW, a tool doing all this for your automaticaly would be great, wouldn’t it ?

What I’ll do for my next bootstrap side project

  1. Start with a target hourly earning rate.
  2. Collect a list of projects I’d be interested in.
  3. Pick a project and write a business goal for it.
  4. Assign likelyhoods and ranges on every figure in the business plan.
  5. Design experiments to improve the figures.
  6. Create tasks to actually build and sell the product.
  7. Compute values for each of these tasks.
  8. Work the most valueable first.
  9. If the figures prove that I will not make my target wage with this project, I’ll pivot, and restart from 2.
  10. Otherwise, I’ll repeat from 4.

This was part 8 of my suite of article about Lean Software Development, Part 7 was A Plan for Technical Debt, Part 9 will be My Dream Lean Software Development Tool.