My wife was asked to do a 'Feasibility check' of a project idea. She was a bit lost what to put there. I'm not surprised. She was already indicated a due date and she didn't really know the solution they (she?) should deliver.

From a management point of view, the whole question is about risk management. I don't think her managers really understood that, but their instinct still told them, they need better clarity on expenses. When I heard about the project, I had the gut feeling that this project can be as fast as 1 month for a lone person (a coordinator rather than a developer) or 3–4 years of a team (3–4 developers + project coordinator). It's a huge difference. It's a multiplier of 240. No manager can accept such a budgeting risk - even though, I don't think they really checked it from a that perspective.

So what can they do? They can ask an expert to please give us a better estimate. And the expert can go away and collect some information and come back with better confidence interval. And this is exactly what they did.

We have a clear starting point, an initial estimate, a 90% CI range between 1 person-month and 240 person-months. After the project, we will also know the end state. With 100% accuracy, we can tell how much time we invested into it (regardless if it was a successful or failed project).

The question is, how does our 90% CI becomes more narrow over time. Can I invest 0.5 person-month and get the range down to a multiplier less than 10x. Is x10 acceptable? For a research or experimental project, probably it is. For a product or feature development project, I don't think so. We need to invest more time and get the confidence interval down under x2, if possible.

With other words, what are the characteristic of the improvements of our estimates? Can we front-load it? Can we start by improving the estimate?

What is the problem of my wife?

She was not only given this project to check, but - as I mentioned earlier - she was given an imaginary due date: "It would be great to deliver it ~2 month time". Probably her confidence interval is a bit narrower than mine, but she is facing a >>100x interval still, whilst her manager already went with wishful thinking. Probably, not to be irrational, he picked not the lowest value, he could imagine but doubled it. She actually got the task to communicate that this confidence interval is insane and even reducing it would cost time (⇒ money). And now we are back to square one, because she got another project (a subproject) to get estimation on how long it would take to bring the confidence interval down. For that subproject, she probably has an estimate of 1 week - 2 month (9x) which is still too large to promise a date of an update meeting where they could discuss.

Solution

There should be a decision on how much time can be invested into the analysis (feasibility check, discovery, you can name it), and after the given time, the progress should be checked and the project reevaluated. In the light of the new knowledge, a new confidence interval should be established, and a decision should be made if they want to continue investing into the project or they call the day and drop it.

It can be difficult to communicate because the whole solution should be coming from the managers, and they should create a safe environment, where direct reports feel confident to communicate their opinion.

Fortunately, I have never seen a company that didn't work like this. Every company is continuously evaluating the progress of the projects and adjusting the confidence intervals and the budgets allocated to them. Even if they don't know it. 😉

The real problem is, that these managers can be very optimistic and take the lower end of the estimates. They are again not intuitively wrong. These estimates should have a long tail. The average (mean) time is way below the median value. If they pick a single number, they really shouldn't pick the end of the 90% CI. Just by picking the double of the lower end of the CI, he followed his intuitions.

There is also a physiological factor: giving too much time for teams can make the team focusing on marginal gains. But that's another story.