The Most Expensive Question You Didn’t Ask

Sometimes engineering has to do some rework. Usually, it isn’t caused by bad answers. This engineering rework is caused by questions that were never asked or the right questions answered with the wrong tool at the wrong fidelity.

Prototyping should be the antidote to that. But for a lot of engineering teams, it’s actually part of the problem. Not because prototyping is wrong, but because the way most teams practice it skips the thinking that should come first.

Here are three traps I see teams fall into, and they all share the same root cause.

Prototypes can inject a lot of benefits to new product development.

But we can apply it at the wrong time or spend too much time developing it and not really getting answers to the questions that we should be asking.

I wanted to introduce the three traps that I see most often in engineering with prototypes.

Trap 1: The ‘ta-da’ trap

The first trap I call the ta-da trap. This is when a team builds a polished prototype to impress stakeholders, but before the concept is even stable. Somebody is presented with an idea or a problem, and they hand it to engineering. As engineers, we are trained to solve problems. If somebody gives us a problem or an idea to fix a certain problem, we’re likely jumping into the solution space where we can figure out how to solve the problem. So we start designing before we have all the information that we may need. we build a polished prototype to impress stakeholders before the concept is even stable.

It looks great in the review. We can ooh and awe everybody with this great product concept, and it does generate excitement to be able to look at and hold a prototype, interact with it. But now the team is anchored to it. Fixedness kicks in. They’ve invested time, money, and now ego into a physical thing, and every subsequent conversation becomes about refining this solution rather than exploring whether it’s the right solution.

The ta-da prototype skips past the hard conceptual work and replaces it with something that feels like progress but isn’t.

I talk about this trap a lot in Pierce the Design Fog. I replace this trap with structured concept development, actually having co-working meetings around structured content ideas, so you can develop an idea before you start designing it, before you start putting physical shapes to it or figuring out how it’s going to work.

The whole point of structured concept development is to stay in the problem space long enough that when we do build something, we’re building the right thing.

Trap 2: The “I’m smarter than cardboard” trap

The second trap that I see engineers do in relation to prototypes is the “I’m smarter than cardboard” trap. This is the engineer who skips the low fidelity check because they can instead model it, simulate it, or calculate their way through it.

This one is really seductive because it feels rigorous. You ran the analysis, you check the numbers, and you’re confident. But some questions just aren’t calculation questions.

If you’re building something that’s going to need maintenance and somebody needs to interact with that device, you don’t necessarily need to model that device in CAD all day. Maybe instead you need to build a low-fidelity physical model with cardboard or foam or whatever you have in order to just test it out at the extremes.

Sometimes our question in product design isn’t “what are the dimensions”, it’s “can a human use this”? It’s a different kind of question, and it needs a different kind of answer. The trap is mistaking analytical confidence for real-world knowledge. Sometimes a simple prototype can just give you that real-world knowledge.

Trap 3: The “validation theater” trap

The third trap I see with prototyping is the “validation theater trap”. This is when we’re building an elaborate test rig or prototype to answer a highly technical question that sounds rigorous and sometimes genuinely is. This is the most nuanced trap because sometimes it’s the right call. If you’re working at the edges of known physics, like fluid dynamics for a pressure injectable catheter or an implantable heart pump, there’s no cardboard shortcut.

You need the expensive rig because the question itself is expensive. The trap here isn’t necessarily the tool, it’s the default. It’s when teams reach for validation level testing before they’ve done the conceptual work to know they’re even asking the right question.

The discipline is making sure we’ve exhausted the cheaper, lower fidelity ways of learning before we commit to the expensive one and making sure the question we’re answering with that expensive rig is actually the highest risk unknown, not just the most technically interesting one.

The Root Cause

Those are the three traps that I see the most often: the “ta-da” trap, the “I’m smarter than cardboard” trap, and the “validation theater” trap. All three of these traps had the same root cause. It’s that the team jumped from, “we have an idea”, to “let’s build something”, without spending enough time in the middle asking, what do we actually need to learn and what’s the right fidelity to learn it?

That middle space between having an idea and committing to a direction is where the most valuable engineering thinking happens, and it’s where most teams spend the least time.

If you’ve fallen into one of these traps, it’s all right. It’s part of the engineering experience. But it’s good to stand back and recognize it so you don’t fall into those traps.

Other podcast episodes you might like:

Up next: Hear from a guest who’s built an entire engineering firm around this philosophy.

Raise Your Confidence by Strategically Stacking Evidence

How to Choose Risk Tools That Actually Help Decisions