Are Manufacturing Readiness Levels relevant?
A couple weeks ago, when I put out my piece on Technology Readiness Levels (TRLs), Steven left a really interesting comment about other types of readiness. I’d never heard of these things before, but there seemed to be interest in me engaging with the ideas.
Manufacturing Readiness Levels are a Department of Defense (DOD) concept that defines the suitability for mass production of a technology, component, or system. That NASA isn’t involved here makes sense intuitively; the DOD wants to buy things in quantities NASA doesn’t dream of. Moreover, the existence of a technology that is helpful but not ready for mass production doesn’t really move the needle all that much for DOD, while it has major effects on NASA.
From a more tactical point of view, Department of Defense is concerned about manufacturing risk, which they reasonably view as a problem in an acquisition system that many stakeholders perceive as broken. This looks like an issue along the entire acquisition pipeline, from to hardware that’s been in production for a while now, like the F-35 fighter, to systems that have yet to begin trials, such as the new Constellation-class frigates.
TRLs & MRLs in conversation
Unlike TRLs which run 1-9, MRLs go from 1-10. At this point, I find it easier to think about MRLs and what they really mean by way of comparison to TRLs, which I’m much more familiar with. Ross et al. assert the following relationship between the readiness levels:
TRL 1, basic principle observed, has no equivalent MRL.
TRL 2, technology concept/application formulated, is associated with MRL 1, basic manufacturing implications identified. In the same sense that TRL 1 is just a vision for technology, MRL 1 is just a broadly defined vision for how to produce it at scale.
TRL 3, analytical critical function or characteristic proof of concept, is associated with MRL 2, manufacturing concepts identified. This essentially means that the designer should have a vision for who’s going to make the product to what specifications, and what constraints they’ll have to deal with.
And so on and so forth…
The gap between TRL and MRL grows to a peak at TRL 8, product qualified through test and demonstration, which coincides with the end of MRL 5, capability to produce a prototype in a production-relevant environment.
The back half of the MRLs only are achieved at TRL 9, when the product has operational heritage.
Grouping MRLs
The creators of the plot below, Petrovic and Hossain, also consider TRLs and MRLs in conversation in their paper about fuel cell development. However, they take an additional step, engaging with the concept and making it more digestible by placing the MRLs into different buckets.
They use five, but I think only three are necessary. Having half as many baskets as items to put in them doesn’t actually make it easier to compare and contrast the items, and understand their shared characteristics.
The first category, Material Solutions Analysis, is named well, but in my opinion it should only contain MRL 1 through MRL 3.
The second group, what Petrovic and Hossain refer to as Technology Development, contains MRL 5 and MRL 6; I contend that it should contain MRL 4 through MRL 7. Having participated in this process, manufacturing in a production representative environment is still more similar to manufacturing in a laboratory than it is to a pilot production line. I found that this type of scaling was still often (though not always) negatively impacted by new technology problems more than new manufacturing process problems; it’s only when the new technology problems are a negligible portion of anomalies that I see capability development as complete. Since the subject here is manufacturing readiness, not technology readiness, I found calling this section “Technology Development” unnecessarily confusing. Furthermore, all four MRLs at this stage share as their exit gate a sort of new capability, which thematically ties them together. So not only would I add MRL 4 and MRL 7, I also would rename it “Capability Development”.
It’s clear to me what Petrovic and Hossain are getting at in their third group, Engineering and Manufacturing Development; it makes sense that they view MRL 7 and MRL 8 as two halves of a larger whole. However, I find the idea that both of the remaining MRLs are in their own buckets a little bit silly. I would put MRL 8 through MRL 10 into one bucket together, because they’re all characterized by production line capability. The different MRLs refer to different maturity levels of the line, but at this point, all the impediments to mass production have been addressed, and the question has become one of industrial engineering — how do we design and create a facility to make as many widgets as possible? This question gets to the heart of the issue; this part of manufacturing is really about growing the production line than anything else.
As a result, my way of organizing MRLs looks a little bit different; not better or worse, just easier for me to understand.
Who needs MRLs?
MRLs appear helpful in thinking about things from a customer’s perspective, but other than that, I’m not sure what their utility is for founders or investors.
DOD pretty clearly needs a metric like this because they’re buying many different types of products, with different levels of manufacturing risk, in very large quantities — and whether or not the products are delivered on time substantially affects their ability to be successful as an institution.
Other institutions that aren’t vertically integrated and are producing large quantities of many differentiated products also seem like they’d benefit from this way of thinking about manufacturing risk, but there are very few institutions that accept upon themselves such a broad scope.
As a result, I’m not sure that this way of thinking is actually helpful for the vast majority of startups, because:
Most technology startups build software, which has no manufacturing risk
Most hardware startups’ customers are either consumers, who don’t care about manufacturing risk, or institutions which are far more specialized than DOD and therefore better equipped to manage domain-specific risks
Fundamentally, I see MRLs as a Band-Aid for DOD to address the way they accomplish procurement. It sounds like it works to reduce some risks, but it doesn’t address the underlying institutional issues.
That’s certainly not ideal for entrepreneurs, who would benefit from significant acquisition reform. However, it is tolerable to many founders given the size and type of contracts DOD is prepared to hand out to companies that can deliver.1
A startup that wants to sell to DOD, or a similar type of customer, might want to internalize this way of thinking. An investor in these types of startups should also study it. I don’t think it’s worth most other peoples’ time.
At the end of the day, I think of MRLs as a concept almost like an exhibit in a museum. It’s interesting to visit and consider from time to time, and maybe it should inform your way of thinking about some things. It’s also not likely to affect most people, either because they don’t visit the exhibit, or because it doesn’t resonate with them.
Bessemer Venture Partners put out a defense tech roadmap this week which discusses these types of contracts, and their outlook on the industry.