Wednesday, 9 February 2011

Estimation? No a distribution of confidence ...

I have always been wary of estimation... 

Well no, that's not quite true, I wasn't until I gave an estimate to a salesman, and then I found myself having to meet a hard release date... once bitten... twice bitten... well in reality I am a slow learner so it was probably a couple of dozen times.

Most estimates are really figures plucked from the air by a developer, or worse someone who doesn't actually work on the code, or even far far worse by someone who have never even seen the code.

So here is my current thoughts on estimation.. not fully formed, and probably poorly communicated, but I still think worth sharing, if only to improve my own understanding.

Even with Agile poker planning methods, estimation is still a dubious business.

An estimation is only valuable if you are aware of the context in which the estimation was made.  Then you must remember that estimation is exactly that, an estimation - it is in fact some sort of distribution curve of the confidence of the person making the estimation.


What if you plot the 'confidence' of the person making the estimation against the possible solution time.  Typically this might look something like this.  

The estimate they will give should be around the peak of this curve - well pessimistic estimators will be a bit to the right to give them some margin, and optimistic to the left.

Now the shape of this curve is what is interesting.  It will be determined by how 'known' the problem is.  So a really well known task may have a graph like so:

And a conversely really poorly understood problem may have a distribution like:

At either end of the spectrum shown here estimation is pointless.

Either you know the problem so well that time spent estimating is actually waste or you have no idea of the problem so it becomes essentially a random guess. A random guess is more dangerous than not estimating at all.

So estimation is of most value in this middle region, where you have a reasonable understanding of the problem, but not so well that estimation is just waste.  A goldilocks zone if you like.

Good poker planning sessions often are more about the conversations and the discovery around a problem.  These discussions attempt to drag the flat distribution into a more normal one where it is sufficient to make an estimate.

One of the things I have seen here is some teams then get into an obsessive perfection of estimation loop.  Where every retrospective has a key outcome of 'we must improve our estimation'.  And I've seen teams haggle for what feels like hours as to whether it really is a 7 hour or 13 hour story.

Their key outcome is wrong, they are focusing on the wrong thing.  The real outcome should be, we should improve our understanding of the problems at hand.

I think that one way to avoid this obsession is to use T-Shirt sizing - step away from the numbers and use more coarse grain concepts like Small, Medium, Large.

Note that I think story points are better than real time measures but I still think the psychology of dealing with the numbers is the same.

T-Shirt sizes are coarse, deliberately so.  You can start with a rough equivalent to 1 day, 3 day, 5 day - but they should not be thought of in that way when doing the estimation - you just try to group them into general categories of S, M, L.

Over time you can measure what your outputs are, and then you can get average times for each of your sizes and these then can be used by those interested in trying to predict the future.

So estimation or planning sessions should really be about discovery, learning and then putting that problem into some general category.  Importantly don't be afraid to turn to the group and say - Hang on, I don't think we can even categorise this, it requires better understanding, more learning - lets take it out of this process and do some more focused work.

And even if you don't change a single thing about the way you work just think a little about the way you work.  Remind yourself that estimation is just that, estimation and just as important remind those who use your estimations.

No comments: