New Product Risk in Spacecraft & Startups
I recently watched a video in which Destin, an engineering PhD candidate, gives about an hour’s worth of constructive criticism to engineers and management about the Artemis Program, NASA’s plan to return people to the surface of the Moon.
It’s got some gems in it for people interested in the design of complex systems. Around the 41:45 mark, Destin takes an interest in one particular point about risk made by George Low that is super salient not just for rocket scientists, but for founders and builders across fields.1
Destin goes into George’s point that crewed spaceflight has intrinsic risks to the crew. Each crewed launch must accomplish enough new things to justify these risks, but not so many new things that the risks become intolerable. Then Destin applies this to the plan for the first crewed landing of the Artemis Program. This concept put more generally should matter to founders, and really to anybody building something new.
An essential performance indicator in any new product is the effort expended to reduce a unit of risk.What is “risk”?
I used to be an engineer, so I’m trained to think about risk primarily in terms of technical risk. The NASA Systems Engineering Handbook defines technical risk:
The risk associated with the evolution of the design and the production of the system of interest affecting the level of performance necessary to meet the stakeholder expectations and technical requirements.
From a founder’s perspective, technical risk is the likelihood that a product’s design or production will cause a failure whose effect is that the product won’t meet customer expectations. This could be a software bug, a server outage, a loose solder joint, or a million other things; it just has to be rooted in the product’s engineering.
Not all startups will have technical risks that are unique and substantial. A startup that’s applying existing technology to a new use case probably doesn’t have many new things to worry about here.
However, all startups will have risks that founders need to be aware of and mitigate. Litigation risk, regulatory risk (especially in the AI, healthcare, and defense domains), and schedule risk (shipping the product late) are not just product-killers; they are startup-killers.
Risk is quantifiable, and there are a variety of ways of expressing it as a number. I’ve most often seen technical risk expressed as a percent likelihood of success, or as mean time between failures, but there are other options. Because we can quantify risk, we can measure it, and therefore entrepreneurs and intrapreneurs can systematically approach its elimination.
A systematic approach to risk reduction is vital to startups because of their limited resources. It’s not enough to reduce the risks in a new product; the risks must be reduced efficiently, with respect to an entrepreneurial team’s effort.
“an essential performance indicator”
Reducing risks definitely should not be the only objective of a new product, especially in a startup organization. Engagement, traction, and other metrics are mission-critical for entrepreneurs and intrapreneurs.
But retiring risks should be, if not easy, at least doable for the product a founder is building because they are the right person to build this product. The fact that dealing with some of these risks should be doable means that they should be tackled now, because it might not be possible to address other risks until later in the development timeline.
Studying approaches to risk reduction doesn’t just lead to meaningful comparisons between similar startup products and the teams that build them. It is itself a useful way to compare and evaluate entrepreneurial efforts!
At the same time, it is absolutely possible to over-index on this before launching a product. In fact, that’s a large part of what I think has gone wrong with NASA’s Artemis Program. Risk mitigation is essential to crewed spaceflight, but NASA is spending so much time and effort eliminating risk that at this point, the original schedule is a work of fiction. Recognizing that there’s some risks everybody just has to learn to live with might have helped NASA avoid the schedule/financial/management failures that have become the calling card of the Artemis Program.
“any new product”
As far as I can tell, this approach to risk reduction is agnostic to the the product. The allowable risk in a system is certainly not product-agnostic, but the importance of risk mitigation is. It shouldn’t matter whether one is building a web app or a Mars lander.
Importantly, this contextualizes the discussion of risk to a piece of software or hardware; risks of a new venture itself are a different topic. It also shouldn’t matter what stage in development the product is at, from ideation through the 2.0 release.
Why not the 1.0 release? That’s when a founder thinks their product is feature complete and ready for the market. The customer gets a vote here, and it’s pretty rare to see a situation where there’s no significant constructive criticism about risks in the product. Ideally the founders respond to the market by addressing the issues they raise. But after the 2.0 release, there really shouldn’t be any risk remaining in the product beyond a black swan event.
“effort expended”
An economist might say that what I mean by “effort” is just “labor” in the Marginal Rate of Technical Substitution equation, only instead of talking about production quantities, we’re actually talking about eliminating risks. They’d be right! But that’s not a particularly helpful way to think about effort within an entrepreneurial context for two reasons.
First, that equation is interested in the relationship between person-hours and capital expenditures; the whole point is that they’re interchangeable. However, a startup typically has very little of either resource so substitution of one for the other is unlikely to solve a founder’s problems. Furthermore, because early-stage startups run so lean, the time of an engineer, let alone a founder, is invaluable. For startups to succeed, these people must remain laser-focused on their tasks. Just as critically, their tasks on product R&D need to be well-scoped. All of their time should be focused on features that create value for the customer or reduce risk in the product.
Second, that equation is studying production of an existing product, not creation of a new product. Labor and capital aren’t easily interchangeable in this function today because we haven’t figured out how to get AI to create new goods and services from scratch better than founders can. Generative AI and LLMs are developing rapidly to a point where they can do some ideation, and can certainly help flesh out ideas; perhaps in the next few years this will change. But if we were already there today, nobody except prompt engineers would be able to raise funds from VCs!
An effort-based approach to risk reduction is helpful because it can create flexibility in the R&D schedule for those unexpected hiccups that are all too frequent. In other words, tackling easier-to-address epistemic risks earlier in the timeline mitigates the schedule impacts of unforeseeable aleatory risks that may appear later in the process.
Furthermore, by tackling either the low-hanging fruit or the most significant risks first, it’s possible to create a product with substantially lower risks earlier in the development process, which should ultimately lead to faster customer adoption.
Low’s approach to risk in the Apollo Program is more generally applicable to new product development. Creating new things involves risks, and the rewards from the product must be sufficient to justify the risk. It’s not enough to work on reducing those risks; builders of new products need to do so efficiently with respect to effort.
George Low ran the Apollo Spacecraft Program Office from shortly after the Apollo 1 tragedy through the Apollo 11 lunar landing. Think of him as the PM for everything except the rocket.
Your writing is concise and well thought out. It's really fun reading your articles!