Congress and defense officials for decades have lamented that the Pentagon cannot field anything fast. While the digital revolution launches new products for consumers practically every week, sustained Secretary of Defense-level attention is required for the US military to create a product quickly and at scale, such as mine-resistant vehicles used in Iraq and Afghanistan. Today, if troops need new technology, they often end up buying it themselves.
The Pentagon’s failure to innovate is not for lack of trying. Hundreds of experiments are underway across the force to assess everything from unmanned systems to new ways of connecting disparate units. They are supported by dozens of software factories building cloud-based delivery mechanisms that update system programs for new applications.
However, these efforts risk becoming merely “innovation theater.” Like theater, the new gadgets and tactics look good on stage and stimulate creative thinking. But, like theater, these sometimes-revolutionary combinations of hardware, software and people are fleeting. The use cases are highly scripted and the systems involved are often one-off products the government may not even own, which cannot be scaled and proliferated throughout the force.
One of the roadblocks keeping an experimental success from becoming a fielded system is testing and evaluation, or T&E. Just because a new piece of gear makes troops more effective in an experiment does not mean it has the durability to last more than a couple of days, the interoperability to talk with enough other systems to be useful, or the versatility to be useful in other situations. These are the questions program managers must answer when taking gear from lab to field.
Traditionally, T&E is done at the end of a defense program’s development. After years of analyzing requirements, assessing alternatives, maturing technology, and assembling systems, a new product is evaluated developmentally by the vendor and operationally by the government. Since it precedes deployment of the system, T&E results do not reflect how it will be used in practice and are only a snapshot of how the system performed in limited settings.
Around 2008, software engineers adopted a new approach to creating products that addresses the limitations of traditional T&E — the development-operations, or DevOps, cycle. DevOps speeds fielding while maintaining quality through near-continuous automated instrumentation and testing. And unlike Pentagon T&E, DevOps continues after a new product delivers, assessing its effectiveness as the product’s use cases and environment evolve. Potential needs or opportunities revealed in the field are evaluated for security and utility and implemented in the next hardware or software update.
The DevOps model could be extended from information technology to defense systems using T&E. Rather than being the hurdle between development and operations, T&E would become the glue that integrates them and provides acquisition managers and executives the confidence that a new program or employment concept is worth scaling or adapting. This approach would allow experiments and exercises to break out of the theater and become repeatable use cases.
Modernizing T&E to be the engine of defense DevOps will require substantial process and technology changes. Overall, the Department of Defense T&E enterprise needs to flatten and extend its level of effort across a program’s life cycle, as called for in the Director of Operational T&E’s most recent strategy. Instead of devoting all its resources to the testing milestone when a program delivers, T&E will need to focus more attention on assessing experimentation and prototyping to help identify good ideas or foreclose bad ones. After a system goes into a demonstration or operations, T&E would continue to assess its utility.
Supporting both developmental and operational evaluations will require the establishment of digital infrastructure that can represent and evaluate a potential new system, which can be validated as the system undergoes developmental and operational testing. The resulting digital models could be employed operationally to guide commanders in the field as they evaluate potential tactics or force compositions and integrate the new system into joint mission packages. While this approach may not be feasible for every program, it could be employed for high-priority efforts such as the Air Force and Navy Next Generation Air Dominance family of systems.
The main roadblock to creating a Pentagon version of DevOps is money. Today, operational and developmental T&E activities are largely funded by the programs under test. Acquisition managers and executives have no incentive to spend on new testing infrastructure that will change their programs’ T&E plans. And T&E organizations cannot spare funding for new tools or people without detracting from ongoing testing.
Congress can help. By devoting some of the proposed increase in T&E funding toward digital tools and systems alongside the physical ranges that have been the backbone of T&E, lawmakers could move from complaining about sluggish acquisition to fixing it using the DevOps cycle that gave us everything from distributed finance to the iPhone. In the process, Congress could move Pentagon innovation out of the theater and into the field.