Curbing Construction Overruns

Despite technological and organizational advances, 48 percent of projects completed for the Department of Defense (DOD) in the last decade have experienced some form of overrun, accounting for $501 billion in overruns. These issues occur at the expense of overtasked contracting and construction personnel, altered budgets, and, ultimately, impact the ability to award future projects.

Deferring projects can result in missed requirements, lower morale, reduced effectiveness, and delays in mission-essential readiness. Furthermore, cost and schedule overruns can lead to a need to use fixed operations and maintenance funds. And with more than 585,000 facilities to maintain and an existing $116 billion backlog of projects, the causes of these overruns must be identified and mitigated through every available means.

For at least the past four decades, researchers have performed hundreds of analyses on overrun using various methods, though DOD has not found itself at the center of much of this research. The development of smarter, more effectively executed contracts is a current priority of the National Defense Strategy. Through a partnership between the Air Force Civil Engineer Center and the Air Force Institute of Technology, a research project was funded to review and analyze the contract data for all military projects designated as maintenance, alteration, repair, or construction.

In summary, the research focused on determining which contract attributes significantly affected project performance.

ANALYZING THE DATA

OVERCOMING OVERRUN

Improvements to existing attributes and new, additional attributes tracked in the Federal Procurement Database System could be beneficial in increasing the accuracy of the models and providing greater understanding of the causes of overrun.
• Preventing zero values for awarded cost and duration.
• Objective guidance for product service code entry.
• Specific reasons for modifications.
• Government estimate to compare to award price.
• Contractor evaluation (CPARS).
• Type of work in man-hours/cost involved.
• Controllable/uncontrollable modification reasons.
• Value-added/not added indication for modifications.
• Information from engineering databases like TRIRIGA, BUILDER, and TRACES, including pay apps and project progress, building and component conditions, and project
metrics.
• Real-time metrics of projects, including actual working days versus available working days; percentage of equipment downtime; percentage of labor downtime; time to rectify defects; number of accidents; problems discovered in construction documentation; and a log of requests for information and responses.
• Live-time top factors such as “Non-Value Added, Controllable Cost Overrun,” and “Value Added, Controllable Cost Overrun.”
• Real-time access to average cost/schedule overrun of current projects, past projects, and specific project types.

Using the Federal Procurement Database System-Next Generation (now beta.SAM.gov), 10 years of contract data was obtained and transformed into a repository housing 79,894 maintenance, alteration, repair, and construction projects. The data contained location, duration, cost, and modification information. Initial analysis revealed differences in performance based on attributes such as contracting agents, funding agents, and award month. These results would serve as the foundation for more in-depth analysis.

Predictive Benefits. Further investigation using logistic regression produced models that accounted for the complex interactions between contract attributes to help predict the likelihood of overruns and grasp a holistic view of the attribute’s roles in overrun occurrence. The dependent variable (overrun) was converted from a percentage to a simple “yes” or “no” for all projects, and eight models were created to determine the significance of each attribute.

The resulting models could predict whether a project experienced overrun, along with an understanding of how each significant attribute changes a project’s probability of overrun. Accuracies ranging between 66 percent and 75 percent were achieved. Additionally, all models exceeded the “no information rate”—a key performance indicator for logistic regression modeling. The “no information rate” is, essentially, an educated guess given no other information beyond the distributions of the attributes contained within the data. If we know that 50 percent of all DOD projects experienced overrun, then we have a 50 percent chance of guessing that a given project experiences overrun.

The drawback is that the accuracy lay in predicting the likelihood of a project that would not experience overrun. However, the information is still of use because it identifies projects that represent less risk and likely require no additional vetting or mitigation methods to prevent cost or schedule overruns.

Variability Factors. The contract attributes that greatly increased the probability of overrun were the project duration at award, award month, and award type. For the length of duration at award, the probability of overrun increases as the project’s length increases. Definitive contracts increased the likelihood of overrun compared to other award types, including delivery orders.

Projects awarded in September were found to have a higher probability of overrun. A closer look revealed that nearly 50 percent of all projects awarded at the end of the fiscal year experienced overrun while, on average, the other months experienced only 39 percent. Yet because 38 percent of DOD projects were issued in September, that factors into the scope of the problem. Further analysis and additional data not currently available in the Federal Procurement Database System would be required to better understand each attribute’s significance.

BETTER INFORMING DECISIONS

While the goal of the investigation has been to aid planners and programmers in analyzing the risk of overruns using contract attributes currently available in the procurement database, its impact stretches beyond post-hoc analysis. This research could serve as the starting point for data-informed decisions regarding planning within DOD construction. It also can be used to assess construction project execution efficacy at the base level to fine-tune local procurement methods and as a means of performance reporting and accountability. However, these analyses and decisions rely on the veracity and relevancy of their source. Improving existing attributes, adding supplemental information, and maintaining an up-to-date repository of projects is vital to ensuring success.

The research concluded by providing a list of changes that could be implemented to increase DOD’s capability to curb overruns through more effective risk management in the procurement process. It was noted that, throughout this research effort, several of the contract attributes recorded in the Federal Procurement Database System were input inconsistently. Moreover, a review of previous overrun studies revealed additional attributes that could be used to increase the modeling accuracy and create a better understanding of the causes of overrun when they do occur.

The vast majority of these attributes already exist in some form or fashion within project documentation or even within other databases used by DOD. Researchers, planners, and programmers would benefit from a centralized system that maintains this information, if for no other reason than to provide a project-specific source of lessons learned. By arming personnel with this knowledge, it is hoped that future construction projects will be delivered with fewer overruns, enabling more projects to be funded and the backlog of required work to be reduced.

Lt. Adam Teston, M.SAME, USAF, is Master’s Student, Lt. Tyler Stout, M.SAME, USAF, is Master’s Student, and Lt. Col. Steven Schuldt, Ph.D., P.E., M.SAME, USAF, is Assistant Professor and Engineering Management Program Director, Air Force Institute of Technology. They can be reached at adam.teston@afit.edu; tyler.stout@afit.edu; and steven.schuldt@afit.edu.

[This article first published in the May-June 2021 issue of The Military Engineer.]