Turning Ambiguity into Action: A Framework
For more writing check out my new publication! thenextmile.substack.com
In my current role, I commonly work on new delivery products. It is inherently ambiguous because we are not constrained by existing operations or customer expectations. Not only is there no precedent for the many decisions that need to be made, we may not even know the right questions to ask. Continuously working in this environment has led to a framework to handle ambiguous projects or decisions. It focuses on generating a relevant dataset to drive constructive discussion, rather than providing just a simple recommendation.
In this post I present my framework for tackling ambiguous asks. It will need modification for different use cases, but is an example of an approach that creates the necessary structure for effective decision making.
Decisions rarely have an obviously “optimal” solution. If they did, there wouldn’t be anything to decide because there is an objectively best answer. Instead of finding an optimum, a decision is usually made when your team has found a satisfactory tradeoff between success metrics. The key to handling ambiguous problems (and the focus of this framework) is quantifying these tradeoffs- know your most important metrics, and understand how they change based on the decision you are making. To better explain this and apply my framework, I use a fictional scenario:
There is a transportation company called Flash that has specialized in ridesharing since their launch 4 years ago. They’ve developed a vast network of drivers and mastered on-demand labor, so they want to branch out to delivery of physical goods. They’ve partnered with Walgreens to pilot 4-hour delivery on pharmacy items. A director reaches out to you and asks- “What is our optimal customer coverage for delivery?”
Given the fact that Flash has never delivered physical goods, this problem is highly ambiguous and requires a framework to efficiently reach a recommendation. The steps below outline my typical methodology for approaching these types of asks
1. Translate the ask into a solvable problem
If you are being asked to provide an analysis or a recommendation, you are most likely the expert in the problem space. The requester will lack some depth of knowledge, and will often not ask the technically correct question (and they shouldn’t have to). You need to understand the spirit of the ask, and how it can be broken down into a solvable problem that will address both the technically correct question and the more ambiguous business question
“What is the optimal/best XXX?” is a common ask, however decisions come down to tradeoffs, so there is not really an “optimal” solution. In the case of Flash, there will be a tradeoff between the number of people to which we offer delivery, and the cost of delivery. As we increase the delivery radius from stores, we reach more potential customers, but must pay drivers more for increased drive times. There isn’t really an optimal coverage area- if we’re focused on revenue growth, we will bias toward covering more customers, but if we’re focused on profitability, we will prefer a smaller delivery radius.
The technically correct question is “what is the tradeoff between potential customer count and delivery cost?” If we can create a cost vs. customer curve, we can select a desirable tradeoff based on our business goals. This is a solvable problem that allows us to take data-driven approach to the ask
This step is crucial to producing a logical and structured model. It will lead to clear insights that not only provide a recommendation, but the relevant information to constructively discuss alternative options. You are the expert in your space- take ownership over the ask, take the time to understand the underlying question, and create a problem statement that will provide the best information possible
2. Identify the key tradeoff(s) that need to be considered, and the inputs that drive a change in these metrics
As mentioned previously, decisions come down to tradeoffs. There are 2 key questions that need to be answered in order to structure your analysis
- What is an output vs. an input?
- What is controllable vs. uncontrollable?
Your “tradeoff” will be the relationship between your key output metrics. In the case of Flash, these output metrics are delivery cost and potential customers covered. There are many inputs that will influence these metrics, but I will focus on two for simplification:
- Delivery radius- this is a controllable input that impacts how many customers we reach and how far we must drive
- Order Conversion (orders/person covered)- The number of orders generated within a given coverage radius will influence the density of routes, and therefore the cost per delivery (higher density reduces the time between deliveries). This is uncontrollable.
(Note: While the order conversion can technically be influenced by Flash more broadly, it is not actually influenced by our decision. I therefore consider this uncontrollable).
The controllable input is the decision point- in the case of Flash we need to decide the delivery radius. Keep in mind that the more controllable inputs you include, the more complex the analysis becomes. For example if we are also deciding between 2-hour and 4-hour delivery speeds, we will need to vary both speed and radius, doubling the amount of analysis needed. Try to simplify to the fewest inputs at any given time. This will make your analysis simpler, and forces recognition of the separate decisions that must be made (vs. conflating them)
3. Define your constraints of the business
Before you can design your analytical model, you need to define your constraints. This step is essential because it limits your analysis by setting boundaries for your controllable input; you do not want to waste your time analyzing scenarios that would never possibly align with business goals.
Remember in the Flash case, we need to complete delivery within 4 hours. The simplest range of delivery radii would be 0 minutes to ~3.5 hours (you need time to pick-up and drop-off orders), however there will be stricter constraints considering we wouldn’t launch a delivery service that covered no customers (0 minute radius). For Flash, assume that we’re required to cover at least 300k people per city, and require the ability to make 3 deliveries per route. Through some scrappy analysis, we estimate the delivery radius can be anywhere from 30 minutes to 2 hours
(Note 2: The constraints in this example are very straightforward, and you could even reasonably test 0 mins to 4 hours. Other problems will be much more complex. For example, if we are defining the delivery speed, we would need to consider things like total fulfillment time, length of delivery windows (if we have them), number of delivery options, etc.)
4. Design your model
At this point we have broken down the original ambiguous ask to measuring the impact of a constrained, controllable input. You can now design your actual analysis.
Start by creating a simple mock-up of a final result that includes the controllable input and the key output metrics. This step works backwards from your customer (the person making the ask, in this case your director), ensuring you design a model that with the necessary data to make a confident decision. For Flash it will be a table with columns: delivery radius (input), potential customers (output), and cost (output)
Next, determine the values of the controllable input to test. In the case of the delivery radius this is a simple task (probably equally spaced values from 30–120 minutes), however it can be much more complicated for a different ask. The key is to select a values that are representative of the spectrum of possibilities, at a granularity that lets you confidently fill in the blanks. Select scenarios that are near your constraints, and add a few in between that will give you a broad understanding of results. For example if we are instead determining the customer delivery options, we may consider the following delivery windows (given our speed and routing constraints)
- 1.5-hour (minimum length to average 3 deliveries on a route with a single window)
- 4-hour (maximum length to deliver within 4 hours)
- 1-hour / 4-hour (maximum length to deliver within 4 hours, represents a scenario with 2 delivery options)
- 2-hour / 3-hour (another scenario with 2 delivery option that satisfies constraints)
From these 4 scenarios we can roughly estimate the cost of 2-hour, 3-hour, and other multi-window combinations
5. Break your constraints.
It is extremely important to understand the value of breaking your constraints. Maybe there are dense pockets just outside of 120 minutes that are worth extending our max radius
6. Get feedback on your model
Present your mockup to your stakeholders. Before doing any actual analysis, validate it will provide the information they need. Get feedback on the controllable input values to see if there are other specific scenarios that need consideration
7. Develop and execute analysis methodology
You need to develop a methodology for estimating the outputs. The analysis may be based on sophisticated simulations, or something much scrappier. This will be the most time consuming step, but I won’t be diving in to analysis building here (yet) and assume you have the ability to estimate the key output metrics. The most important things to your model will be making inputs/assumptions configurable when possible, and clearly documenting your methodology and assumptions
8. Perform a sensitivity on uncontrollable inputs
When estimating the cost for Flash, there will be a series of assumptions that are made. You need to answer the question- does our tradeoff or recommendation change if our assumptions are wrong?
There will be many unknowns in any ambiguous project. You need to focus on the 1–2 least confident assumptions that have the largest influence on the outputs. Once identified, estimate the impact of these assumptions changing by picking a few scenarios that adequately represent the spectrum of possibilities.
In the case of Flash, the most significant unknown with the largest impact will be order conversion (orders / person covered). High order conversion will lower cost by increasing density of routes, so you may be more willing to increase the radius. If order conversion is low, Flash may want to reduce their average drive time to customers to offset poor route density. Because our radius decision may change based on this uncontrollable input, we should estimate the cost vs. coverage tradeoff for different order conversion scenarios
Now we have something that looks like this:
(Note 3: In this case, the uncontrollable input does not impact the total potential customers within a given radius, so we only need to add extra columns for cost. This will not always be true, and depending on your uncontrollable input, you may need to have additional columns for all outputs)
9. Communicate results and provide a clear recommendation
Your results table will set you up to clearly quantify your key tradeoff(s) and make a well-informed business decision. When presenting the results, have a clear recommendation, even if you know your data will spark debate. Select the tradeoff that you think is most attractive based on the needs of the business, and use that as an anchor point for further discussion
I won’t go in depth on general communication of business ideas and recommendations, but anything with a complex model should include:
- Problem statement
- Clear recommendation
- Most relevant data
- Methodology and model design
- Assumptions- include level of confidence, and the risk of being incorrect
The Framework, Reduced
The framework seems complicated when explained through text, but it’s actually pretty simple. Summarized, it looks like this:
- Create a clear, solvable problem statement
- Identify key output metrics that measure success, and the controllable inputs that influence those metrics
- Estimate the output metrics while varying the controllable input within business constraints
- Select a value of the controllable input that results in the desirable tradeoff of success metrics
Highly ambiguous questions need a framework to solve systematically. Without a framework, you will suffer from a wandering analysis and scope creep, and will struggle to effectively communicate ideas. The framework I present is in no way “correct” and won’t fit all use cases, but it is an example of an approach required to tackle big questions. When given an ambiguous project, spend the time to deeply understand the ask, structuring your planned output, and designing a modeling methodology. Remove the ambiguity by quantifying the most significant tradeoffs, and you will provide your team the tools necessary to make an effective decision.
