Confidence Is A Dial: Turn It With Evidence, Not Guesswork
We turn late-stage design surprises into a strategic plan by assigning explicit confidence levels, stacking evidence, and using the three-dial model of time, cost, and confidence boost. We show how to work backward from a system test to cheaper steps that drive faster, clearer decisions.
• applying the three dials of time, cost, confidence
• sequencing with the work-backwards strategy
• avoiding overtesting, undertesting, wrong testing
• turning confidence into a team communication tool
• practical next steps to build the confidence muscle
Transcript:
From Engineering to Emergency: The Shift in Mindset
In Quality During Design, we were talking about engineering aspects of product development. Pierce the Design Fog covers concept development—early product development before we’ve actually designed anything, although we have an idea, but we still need to work with our team to develop that idea into a product.
Now here in October, November, December on Substack, we’re exploring late-stage design failures.
So we’re through product development—we’ve already settled on a design, we’re verifying and validating it, and we’ve found a problem or a hiccup, or there’s a decision we need to make and we’re not that confident in it.
This is different than planning things out ahead of time when there’s a space of cool heads and calm planning. This is more of an emergency situation. Something unexpected came up and we’re trying to address it.
Last Month’s Insight: Evidence Stacking
In the last episode, I talked about evidence stacking.
In October, we identified more about the problem—describing it, and assigning a confidence level. And that gives us a starting point to do other things.
It gives us an inkling of what activities we should do based on the problem that we’re facing. It also gives us a starting point for stacking evidence.
Because if we are 40% confident in something, how do we get to 80% confidence? So we can start looking at stacking evidence to be able to get to the confidence level that we need.
Today’s Focus: Choosing the Right Sequence of Activities
Today I want to talk about how we can choose the right sequence of activities to boost our confidence efficiently—without wasting time and cost.
Let’s talk more about it after this brief introduction.
The Scenario: Late-Stage Failure in Action
Here’s the scenario. I described it a little bit in the introduction, but let’s get a little more into it.
Late-stage failure—what do we do about it?
We’ve already assessed the risk of it, meaning we’ve identified what kind of impact it has and our confidence level in it, and that’s helping us to make decisions and what to do.
And now we also recognize that we can stack different tests or analysis methods to be able to get us to a more confident decision.
Again, these are really useful tools in the heat of the moment. They help us to avoid those knee-jerk reactions, and they give us a systematic approach to be able to solve a problem that everybody’s panicking about.
You can also use this in your planning in the earlier stages of product development. But for now, we have a problem—we already have evidence, we have a low confidence, it’s a high impact to our project. What do we do first?
The Three Dials: Time, Cost, and Confidence Boost
At this point in the project—well, let me back up.
At any point in the project, we’re always considering time and cost.
In late-stage development, when we’re in late-stage development with these surprises, these surprises are introducing a time and cost debt.
I want you to think about when you’re deciding what to do first to improve your confidence level, is to think of three dials.
Think of time, cost, and your confidence boost.
Every type of test, research, expert consultation that can help you improve your decision—consider not just the time and cost, but also the confidence boost that you’re going to get from it.
This kind of goes along with what we talked about last month—assigning a confidence.
As engineers, we would do well to get better at assigning confidence in some of our decisions, and then be able to back that up with the reasons why we’re assigning that type of confidence level—whether it’s our uncertainty, whether it’s the level of impact, what is leading us to that uncertainty.
And then we can also do that when we’re choosing activities in order to boost our confidence.
Really think about how that activity is going to be adding confidence to your decision and name the confidence, name the confidence level, assign it a value.
This forces you to be strategic about what you’re doing.
A Practical Example: Literature Search vs. System Testing
For example, a literature search is low cost and low time, and it might give you a moderate boost in your confidence.
Compared to a system-level test, which is going to cost a lot, perhaps take a lot of time, and would give you an extremely big confidence boost.
Which one should you do?
Should you do both?
Should you do both at the same time?
Should you skip the literature search and only focus on the system-level test?
It depends.
We need to make a decision. And we can use the time, cost, and confidence boost to help us make a decision.
The Work Backwards Strategy
Here’s a strategy that you can use when you’re facing this kind of decision.
Use the work backwards strategy.
You can start with the most difficult, expensive endpoint, which could be your system testing, but then walk backwards from that to find cheaper preliminary steps that build toward it.
If your options are:
- Literature standards
- Consultation
- Analysis
- Simulation
- Component testing
- System-level testing
Maybe don’t just jump to system-level testing.
And maybe don’t just do them all at once, trying to fast-track your decision.
Think backwards from what you think you might need to these earlier problem-solving methodologies, like the component testing or the analysis and simulation.
Always consider those three dials: time, cost, and confidence boost.
I know that this concept isn’t foreign, and as you’re listening to it, it probably sounds pretty obvious because we’re engineers—we’re trained to be able to work through problems like this, to be able to make decisions.
But we can get stuck when we’re having these emergencies in focusing on not just the time and cost, but also getting explicit about the confidence boost that whatever analysis is going to give us is going to help us make decisions about what to do first instead of doing all the things.
Really pause and give it an assignment.
When you step back and look at it, you may realize that, you know what, a literature search, an expert consultation—this isn’t going to help us in this situation. It’s only going to give us a 5% boost in total. So we’re not going to waste our time doing that.
By assigning a confidence boost to your test plans, you’re more critically assessing and thinking through what kind of information that you need that you really need to be able to be more confident in your decision.
The Switch: From Intuition to Intentionality
This is the switch that I encourage you to make.
Take these natural thought processes that you have toward test design and solving problems and stop and start assigning a confidence level to them.
Assign a confidence level to the problem, and then assign a confidence level boost to whatever solution that you have.
This is going to not only help you plan out how to react to the situation, it can also help you work with your team to develop a test plan that will actually help you make a decision.
And if it ends up being that you really need that component test or that system-level test in order to solve this problem, having a confidence boost and using confidence levels will help you articulate the level of impact that this kind of test will have on your decision and on the project as a whole.
Combining that all with evidence stacking and acknowledging that there are time and cost constraints that we need to think about, but these other methods aren’t going to give us what we need, or they will get us pretty far, and let’s start with those.
Your Action Step: Start Flexing Your Confidence Muscles
Now from this month, you have the stacking principle, the three-dial model, and the work backwards strategy.
If you want the sequence and the case study and ready-to-use templates that save you time and prevent errors, subscribe to the Substack. This is the toolkit you need to implement this framework. Why am I pushing you to Substack and not just describing it here? Because these are not easy concepts to be able to explore in a 10-minute podcast. On Substack, I’m providing more details about it, guidelines, visuals, and examples that help you apply this idea to your work. Builder+ tier qualityduringdesign.substack.com
Other Quality during Design podcast episodes you might like:
Design Input & Specs vs. Test & Measure Capability
Choosing a Confidence Level for Test using FMEA
Leave a Reply