Using The Fermi Problem Approach in Estimating New Product Impact
Transforming Product Impact Estimation: A Deep Dive into the Fermi Problem Approach
Over the years, one question I’ve been asked frequently is “what is the potential impact of a new product feature?” More often than not, this question aims to help product managers prioritize their roadmaps.
If you are a product manager, analyst, or conversion optimiser, you’ve likely found yourself grappling with this question during a priority scoring exercise. Traditional prioritization frameworks like RICE (Reach, Impact, Confidence, Effort) are typically employed. These frameworks often avoid quantitative assessments of impact in terms of revenue or orders, opting instead for basic qualitative scoring methods like “1 to 5” or “t-shirt size”. Although swift, these methods tend to leave much to be desired. They incorporate a degree of ambiguity, lack scientific rigor, and leave the door wide open to bias, leading to overestimations of preferred outcomes or underestimations of alternatives.
However, impact estimation isn’t exclusive to prioritisation. Should you be venturing into a new market, launching a new product or service, you will inevitably be faced with estimating the size of the opportunity at hand. The result of this critical exercise can shape the course of your company’s strategy, whether to invest, launch the product, or enter the market.
Startup founders may also find themselves calculating the total addressable market (TAM) before actual product development. This is especially the case when courting investors, as they will be keen on understanding market size and revenue forecasts. I can guarantee you, t-shirt sizes just won’t cut it in these situations.
The question then arises, how do you create these estimates, and how can you gauge their accuracy? Reality dictates that total accuracy is an elusive goal. However, you can obtain a ballpark figure that is both practical and reliable using a technique known as the Fermi Estimate.
Fermi Estimates: A Quick Primer
Inspired by the work of the celebrated Italian physicist Enrico Fermi, the Fermi estimate is a simple, rough calculation yielding a logically accurate, reasonable estimate, complete with all its unknowns. Fermi problems are a cornerstone of physics or engineering education, serving as quintessential estimation problems that teach dimensional analysis — the analysis of relationships between different quantities using base quantities and units of measurement. They’re the kind of calculations you’d whip up on the back of an envelope.
Decoding the Fermi Problem Approach in Product Releases
You might be puzzled, “How can Fermi problems be applied to product releases?” The answer is relatively straightforward, and it could considerably refine your estimation process.
The first step is decomposing the problem. Fermi was renowned for his knack for dissecting complex issues into solvable, manageable segments. For instance, to estimate the revenue a new feature could generate, break down the problem into smaller, more concrete questions:
How many current users are likely to use the new feature?
How often would they use it?
What potential increase in engagement or average revenue per user (ARPU) could be expected?
Next comes the estimation. For example, suppose you have a feature adding a premium service to your app. You predict that out of your 1 million users, approximately 10% will opt for the premium service. If you’re charging £10 monthly and anticipate these users will use the service for an average of six months, your estimated revenue would be:
1,000,000 users * 10% * £10 * 6 months = £6,000,000
Now, let’s face uncertainty. Even with educated guesses, uncertainty persists. The Fermi problem approach caters to this reality through ‘order-of-magnitude’ estimates. For instance, if you’re uncertain whether 10% or 20% of your users will adopt the feature, you could calculate the impact with both estimates, providing you a range of outcomes and shedding light on the potential upside and downside.
Lastly, refine your estimate iteratively. As fresh data rolls in, adjust your estimate accordingly. This iterative refinement is the heart of the Fermi approach — beginning with a ballpark estimate and fine-tuning it as more information surfaces.
Conclusion
The application of Fermi estimates in product releases allows you to transcend vague guesses, providing concrete, data-driven estimates. Remember, pinpoint accuracy isn’t the endgame; the goal is to generate a logical, defendable estimate to guide your product development and investment decisions.
Fermi problems challenge us to think creatively about the knowledge we have and what we can logically infer from it. In the intricate, data-saturated world of product management, this approach can cut through the noise, enabling you to zero in on key questions: What is the potential impact of this feature? How substantial could this opportunity be? By breaking down these questions, making educated guesses, and continually refining your estimates, you can provide answers that are not only more credible but also significantly more strategic.
I’m currently in the process of building a bunch of self-serve analytics tools, including a fermi-problem impact calculator, to empower non-analysts to do their own analysis. If you’d like to be part of the launch just add your details to the waiting list.