A variation on the Heilmeier Catechism
Drawing a line from emerging tech research to early-stage startups
Formed in 1958, DARPA, the Defense Advanced Research Projects Agency, is the Department of Defense’s emerging technology research and development hub. As the National Science Foundation’s SBIR and STTR grants are "America’s Seed Fund", DARPA is DoD’s in-house technology accelerator.
DARPA played (and continues to play) a major role bringing new technologies to bear on national security, from the internet to stealth aircraft. The model works so well that other government institutions, like the Department of Energy and National Institutes of Health, have created similar agencies.
All this is to provide context for George Heilmeier, who was the director of DARPA in the 1970s, when the organization was evaluating programs like space-based infrared sensors and artificial intelligence.
Among engineers, he is known for having played a leading role in Liquid Crystal Display research. For program managers, his major contribution was the development of a series of questions for evaluating new project proposals:
The Heilmeier Catechism
What are you trying to do? Articulate your objectives using absolutely no jargon.
How is it done today, and what are the limits of current practice?
What's new in your approach and why do you think it will be successful?
Who cares? If you're successful, what difference will it make?
What are the risks and the payoffs?
How much will it cost?
How long will it take?
What are the midterm and final "exams" to check for success?
Startups are not DoD Research Projects
These are great questions to evaluate a research proposal. As a result, they’re important to consider when evaluating early-stage startups — but not by themselves sufficient.
After all, R&D to develop a moat is a substantial part of the early work of a startup with potential to turn into a unicorn. Much of the remainder is figuring out how to sell the product under development, which…isn’t a problem DARPA has or is trying to solve.
So the set of questions needs a bit of tweaking.
I have started considering the following list of questions (my modifications in italics) when I meet with founders, and then reflect on those conversations.
What are you trying to do? Articulate your objectives using absolutely no jargon.
How is it done today, and what are the limits of current practice?
What's new in your approach and why do you think it will be successful?
Why do you think you will be successful?
Who cares? If you're successful, who will use it? More importantly, who will pay you for it — and how much?
What are the risks? And what does it mean for this startup to fail for the right reasons?
How much more will it cost than what you’re raising now?
What are the midterm and final "exams" to check for success? Is the final "exam" to check for success in less than 10 years?
The changes are worth elaborating on, because they speak to key differences in structure and desired outcomes between DARPA as an government R&D institution and venture capital as an asset class.
Why do you think you will be successful?
This first change, a totally new question, is about why the founding team believes they’re set up to succeed.
This is probably the most important item in my battery of questions. It is particularly critical in a startup in a way that it’s not for research, for a couple reasons:
Founders have got to have a significant, almost irrational, level of self-confidence — in a way that government research program Principal Investigators don’t. If a founder is not both obsessed with what they’re spending time on and very confident it’ll work, it’s not likely that they’ll convey confidence to potential investors or customers. This checks for that.
Many great startups that go on to become enduring companies have some sort of unique, almost unfair aspect — or "moat". Rick Zullo of Equal Ventures has written a couple awesome essays about moats that get into the idea in more detail. This question also assesses whether the founders have a moat or a path to one, and how they think about it.
Who cares? If you're successful, who will use it? More importantly, who will pay you for it — and how much?
This modification is necessary to account for the different purposes of the questions. Heilmeier wanted to understand impact specifically as it applied to his DoD customers; I’m looking to see if there’s potential for venture-scale returns on an investment for my Limited Partners.1
We want different things at the end of the day, and therefore the question about utility has to be phrased differently.
What are the risks? And what does it mean for this startup to fail for the right reasons?
The change to the risk question is a reflection of startup success rates. Notwithstanding The Producers, nobody goes into business intending to fail — but statistically speaking, most venture-backed startups will not succeed.2 I need to answer for myself what it means to fail for an acceptable reason.
How much more will it cost than what you’re raising now?
Similar to the question about who cares, the modification to the cost question is a concession to the reality of startups following the VC playbook — they burn cash fast. I’ve heard of a couple founders mention over the past few months that they aimed to raise a pre-seed or seed round, and that’s it; they wanted to organically grow the rest of the way. I learned recently that this is described by the term "venture strapped" — a portmanteau of venture backed and bootstrapped. It’s a new enough concept to me that I’ve not yet heard of founders doing this to exit.
But the underlying issue really is one of anticipated dilution. I want to understand before making an investment how the ownership I’m considering purchasing will be diluted as the startup raises more money. Dilution is OK and expected as the startup grows; a smaller piece of a bigger pie is an expected part of the process! This is, after all, an optimistic scenario.3 I just think it’s important to be cognizant of this when evaluating a startup as an early-stage investment candidate, because this will have an important effect on the economic outcomes of a potential investment.
What are the midterm and final "exams" to check for success? Is the final "exam" to check for success in less than 10 years?
The addition to the last question is also new, and is similarly a function of structural differences between VC Funds and government agencies:
DARPA gets money every year from Congress to use in accordance with the budget. Congress is never going to see that money again, and they’re ok with it.
VC funds typically have a ten year lifecycle. This means Limited Partners commit money to the fund, and expect to receive it back in about ten years when the ownership of the startups that the VC has invested in are sold. If the VC fund doesn’t return capital to its investors when it said it would, it’s got a problem. VC investors typically do not want to invest in startups where this is unlikely to be possible in an optimistic scenario.
In many cases, it doesn’t really matter how great a startup concept is if it’s more than ten years out from a successful exit in an optimistic case. In that situation, the idea should probably be funded by grants or bootstrapping, rather than a typical VC. There’s some exceptions, yes, but they’re the type of exception that defines the rule.
Limited Partners are the people and institutions who invest in a VC fund.
From a VC’s perspective, “success” is an exit in which the VC’s ownership stake sells for at least as much as the total amount raised from investors for that particular fund.
In a pessimistic outcome, the startup folds, the investment value becomes $0, and nobody makes any money.