Define Kill Criteria to Avoid Zombie Projects

View the Episode Transcript

Overcoming the Sunk Cost Trap

For engineers, killing a project after investing so much time, energy, and resources into it is often the hardest thing to do. We are trained to keep innovating until we find a solution. However, this dedication can lead to the sunk cost fallacy, a cognitive bias where we continue investing in a decision based on prior, irretrievable investments, rather than factoring in future benefits or nonbenefits.

When this bias takes hold, a project can turn into a “zombie project”: one that never sees the light of day and fizzles out only after consuming significant time and resources. Rational decision-making dictates that costs already incurred should not influence our future choices.

To avoid this, we must define our kill criteria (or quit criteria) upfront. This practice is similar to running tests with acceptance criteria; you define the limits where the project fails before you start development.

Defining these criteria and getting alignment on them is crucial. By setting these boundaries, you ensure that resources are focused on the most feasible options.

For engineers, killing a project after investing so much time, energy, and resources into it is often the hardest thing to do. We are trained to keep innovating until we find a solution. However, this dedication can lead to the sunk cost fallacy, a cognitive bias where we continue investing in a decision based on prior, irretrievable investments, rather than factoring in future benefits or nonbenefits.

When this bias takes hold, a project can turn into a “zombie project”: one that never sees the light of day and fizzles out only after consuming significant time and resources. Rational decision-making dictates that costs already incurred should not influence our future choices.

To avoid this, we must define our kill criteria (or quit criteria) upfront. This practice is similar to running tests with acceptance criteria; you define the limits where the project fails before you start development.

Defining these criteria and getting alignment on them is crucial. By setting these boundaries, you ensure that resources are focused on the most feasible options.

Other Quality during Design podcast episodes you might like:

Exploring Product Development and AI Through Literature: Insights from ‘Loonshots’, ‘AI 2041’, ‘Quit’, and “How Big Things Get Done’ (QDD Book Cast)

Episode Transcript

Welcome back to Quality During Design. Today we’re looking at Tesla’s aggressive battery targets. To manage the risk of pursuing multiple battery chemistries simultaneously, a necessary element of portfolio thinking, Tesla developed a framework to identify winners and eliminate losers. As engineers, the hardest thing we do is kill a project, often falling victim to the sunk cost policy, where prior investments of our resources influence our future choices, risking the creation of a zombie project. Tesla proactively defined failure conditions up front using five constraint categories, including economic, performance, and system filters, to measure project viability. Join me as we explore why defining these explicit kill criteria before development begins is the most valuable and often overlooked exercise in product engineering.

Tesla’s Aggressive Battery Targets: A Case Study in Portfolio Thinking

I’ve been reading up on Tesla’s battery development program and looking at it from a case study point of view, starting with their 2020 Battery Day. All sorts of versions of this are on YouTube that you can go and view it yourself. In 2020, they announced some pretty aggressive Go criteria, like 56% reduction in battery cost per kilowatt hour, and a 54% increase in vehicle range. To meet these kind of targets, they started to self-develop their own battery, the 4680 cell, a custom design, custom manufacturing. They not only had these types of benchmarks that they wanted to meet, but they also had to keep producing and meeting other targets for their existing product line. So they started running multiple battery chemistries simultaneously. And we can all sort of guess why. When you’re in development, you’re not sure about the sure thing. You don’t want to put all your eggs in one basket, so to speak. You don’t want to focus all of your resources onto one idea. In case it fails, you have a backup plan. That’s the portfolio thinking of this case study, right? You run parallel options, and then throughout you systematically choose the winners.

The Importance of Defining Kill Criteria Up Front

So the company has these targets they want to achieve, and they’re doing design and innovation to be able to meet that. And those are a good criteria to have because they are benchmarks, your goal, what you’re trying to achieve. And the closer you can get to that, the better. But there’s a whole lot of other things going on too. We also need a project kill criteria. If we’re developing multiple things at once to be able to reach the same end goal, you need to be able to decide when to stop pursuing some of the other options. Not only where to focus with what’s performing the best, but what’s not going to be feasible to continue to innovate.

Tesla’s Five Constraint Categories for Innovation Management

Some of the constraint categories that I’m seeing that Tesla used were cost, this is the economic filter. For example, this is the 56% target they announced on their battery day. So if the innovation raises the pack price or fails to hit the cost curve, it’s rejected regardless of density. And speaking of density, that’s their performance filter. They wanted to significantly improve the volumetric and gravimetric density over existing cells. So if the innovation path that they’re developing only offers marginal improvements or compromises safety for density, it’s rejected. So those are two constraint categories or kill criteria for this portfolio management, economics and performance.

But I’m seeing that they had maybe three other constraint categories. One was scalability, which is related to manufacturability. Whatever they’re developing must have been compatible with rapid, high volume production. And if the process required exotic materials, complex wet processing, or it can’t be implemented on the dry battery electrode line, they abandoned it. The next to last category was a resource filter related to the supply chain. They wanted to utilize the globally abundant and environmentally stable materials. For example, they wanted to move away from the high cobalt materials. So if their development relies on the scarce, geopolitically sensitive, or environmentally damaging materials, they rejected it. Finally, the last constraint category that I saw was the system filter. They wanted to make sure that whatever they were developing integrated seamlessly into the vehicle structure, the cell to pack or the cell to vehicle. And if the design couldn’t function as a structural element, the technology was rejected as it compromises the overall vehicle efficiency and cost.

So if you think about this, this is Tesla’s battery innovation development kill criteria: economics, manufacturability, performance, resources, and the system filters. And I think they’re really well-defined and clear-cut criteria against which to measure whatever it is that they’re developing, whatever it is they’re managing.

Why Kill Criteria Are Often Overlooked—And Why They Matter

So think about your current projects that you’re working on. You definitely have Go criteria. Those are the user needs, requirements, and other performance requirements that you’re trying to meet. What is your kill criteria? Do you have any? You might have some, but you don’t know what it is. How can you find out? And how can you use that to help you make decisions?

When we’re developing kill criteria, the project kill criteria, we’re actually defining the failure conditions up front. Doesn’t that make things clearer? You see this in non-engineering and product development communities also, talking a lot about New Year’s resolutions right now. Sometimes people will give themselves a cutoff time. You know, I’m going to start this new business. I’m going to give myself I’m going to give myself two years to have this many customers. And if I don’t, I’m going to pivot and do something different.

The Emotional Challenge of Killing a Project

The hardest thing to do when you’ve invested so much time and energy, research and resources into a project is to kill it. You’re invested in it and you want to see it succeed. You want to see it through to the end. I think many of us engineers think that way. It may have something to do with how we were trained as engineers to find the problem and then keep digging or evolving or innovating until you find the solution. We might find a solution, but it may not be feasible against all the other criteria because it’s not just performance, is it? It’s also cost and manufacturability and what the customer wants, what the business needs. I sometimes refer to some of these other aspects of product design and engineering as our internal customers. Yes, we need to meet some outward external goals for our customers or whoever’s going to be buying our product, but we also have a responsibility to engineer for many other people and divisions and things.

So kill criteria is something that we want to be able to define for our projects. Project manager or some of the people that are deciding what projects are pursued in the business probably have this kill criteria. Maybe you don’t know what it is. It might be worth finding out. If there is no kill criteria and you’re just going to keep developing until you get it, your project might turn into a zombie project where it never really gets done, it never sees the light of day, and it either just fizzles out or eventually somebody pulls the plug.

The Sunk Cost Fallacy in Engineering Projects

There can be sunk cost fallacies in our engineering projects, also, where we have an emotional attachment to our design or our project. The sunk cost fallacy is a cognitive bias where we continue investing in a decision based on our prior investments. So we’ve already put so much time, energy, and effort into whatever we’re developing that it doesn’t make sense for us to not continue developing it. Whereas really we’re eliminating or not factoring in the future benefits or future non-benefits. If we continue investing in this innovation or this development, is it going to pay out in the end, really? Or is it better worth our time and resources to be working on something different? When we’re being rational about our decision making, these costs that we’ve already incurred in our project should not influence our future choices. They’re irretrievable.

During product development, we may be working under our sunk cost fallacy. It’s something we need to be aware of and recognize and pivot our own decision making if we need to when we recognize it.

The Power of Defining Quit Criteria Early

So in addition to the success thresholds of our projects, which we need, we should also define and get alignment on the kill criteria of our projects. And this is an often overlooked exercise. In her book, *Quit: The Power to Know When to Walk Away*, author Annie Duke shows us the value of knowing when to quit. She offers a lot of examples from different points of view, different industries. She even looks at the sport of boxing. She also gets into some of the psychological and cognitive biases that we have. Define your kill criteria before you start testing, and for your projects, even before you start developing.

What’s Next? Expected Value and Decision-Making Under Uncertainty

In the next episode, I will talk about the expected value calculation that helps end the debate of whether to keep going or not. Join me on Substack for the Ask Me Anything post. This month it’s called *Choose It: How to Decide Between Viable Options when you’ll never have 100% confidence*. And then later in the month, I’ll be publishing another post, including some of the information that I’ve discovered about Tesla’s battery development and what we can derive from that and how we can use it for ourselves.

Find me on Substack at *Quality During Design*. And if you’re just finding me on this podcast, please subscribe to your favorite podcast provider. This has been a production of Deeney Enterprises. Thanks for listening!