Which is an advantage that regression techniques have over other cost estimation?

A cost estimate is an evaluation and analysis of future costs generally derived by relating historical cost, performance, schedule and technical data of similar items or services. The four major analytical methods or cost estimation techniques used to develop cost estimates for acquisition programs are Analogy, Parametric (Statistical), Engineering (Bottoms Up), and Actual Costs.

Generally, the cost estimating technique used for an acquisition program progresses from the analogy to actual cost method as that program becomes more mature and more information is known. The analogy method is most appropriate early in the program life cycle when the system is not yet fully defined. This assumes there are analogous systems available for comparative evaluation. As systems begin to be more defined (such as when the program enters the Engineering and Manufacturing Development Phase (EMD)), estimators are able to apply the parametric method. Estimating by engineering tends to begin in the latter stages of EMD and LRIP when the design is fixed and more detailed technical and cost data are available. Once the system is being produced or constructed (i.e., LRIP and Full Rate Production), the actual cost method can be more readily applied (See Figure 1).

Few estimates employ the same estimating technique for every cost element. The techniques used to develop the estimates for various cost elements should take into account the stage of the acquisition cycle that the program is in when the estimate is made (e.g., EMD). OSD prefers that heavy reliance be placed on the parametric method (although the analogy and engineering methods could be acceptable), for Milestone B and C reviews; while extrapolation from actual costs should be used to the maximum extent possible in preparing estimates for the Full Rate Production Review and any subsequent actions. A comparison of several estimates using different cost estimating methods is encouraged.

Of the four cost estimation methods presented, the use of actual costs is the most supportable, but difficult to accomplish early in the acquisition program. The analogy method is most often used early in the program, when little is known about the specific system to be developed. The parametric technique is useful throughout the program, provided there is a database of sufficient size, quality, and homogeneity to develop valid cost estimating relationships. The engineering estimate is used later in program development and production, when the scope of work is well defined and an exhaustive Work Breakdown Structure can be developed. Finally, estimating by actual costs produces the lowest risk estimate due to the fact that the system cost is derived from a trend from the current contract to estimate.

Expert opinion, although not addressed in this article, can be used to support any or all the four estimating methods. One or more experts can provide the basis for the cost estimate by bringing a wealth of experience and knowledge. Experts can identify analogous systems and recommend “most intuitive” cost estimating relationships. Expert judgment can prove invaluable for estimating parameter impacts along with impacts to labor and material costs. For example, experts have been used to provide estimates of software lines of code (SLOC), weight, dimensions, system complexity, specifications and performance impacts. Many times, experts have already collected data on labor hours to build, operate or maintain a system. At the very least, an expert can provide his or her opinion on cost drivers, functional form of a regression and engineering general guidelines.

No matter what the estimation technique, the program manager must ensure the cost estimate completely defines the program and is technically sound and reasonable. The cost estimate must be defensible with well-reasoned analysis. A program manager who is totally familiar with the program's cost estimate, including the rationale for the method(s) used to develop that estimate, generally has a greater chance of maintaining control of the cost of that program.

The parametric, or statistical, method uses regression analysis of a database of two or more similar systems to develop cost estimating relationships (CERs) which estimate cost based on one or more system performance or design characteristics (e.g., speed, range, weight, thrust). The parametric method is most commonly performed in the initial phases of product description, such as after Milestone B when the program is in the Engineering and Manufacturing Development (EMD) phase. Although during this phase an acquisition program is unable to provide detailed information (like drawings and standards), the program can specify top-level system requirements and design characteristics. In other words, estimating by parametrics is a method to show how parameters influence cost.

Parametric estimating is used widely in government and industry because it can yield a multitude of quantifiable measures of merit and quality (i.e., probability of success, levels of risk, etc.). Additionally, CERs developed using the parametric method can easily be used to evaluate the cost effects of changes in design, performance, and program characteristics. Note that the parametric method, which makes statistical inferences about the relationship between cost and one or more system parameters is very different from drawing analogies to multiple systems.

A critical consideration in parametric cost estimating is the similarity of the systems in the underlying database, both to each other and to the system which is being estimated. A good parametric database must be timely and accurate, containing the latest available data reflecting technologies similar to that of the system of interest (design, manufacturing/assembly, material). Of course, a general rule when collecting data for statistical analysis is the more data, the better. Finally, as with estimating by analogy, parametric data must be normalized to represent a given economic year and remove any quantity effects.

For example, attempting to estimate the cost of a “today” computer (electronic memory chips) using a database of older computers (magnetic core memory) would yield an estimate much higher than the actual cost of the current system because the memory chips are much cheaper to produce and install than the old core memory. In addition, changes in manufacturing technology or processes have occurred, such as the use of automatic insertion equipment instead of hand insertion of components onto printed circuit boards (PCBs). This has led to major reductions in the labor content associated with the assembly of PCBs.

Additionally, the database must be homogenous. A data element entry for one system must be consistent with the same data element entry for every other system included in the database. For example, in a rocket motor database where there is an element called the "motor weight", each weight entry should be based on the same assumptions for each system. Assume that each motor is defined to include the rocket grain, motor case, and nozzle. If some systems report a motor weight that does not include one or more of these components (or includes additional components), then the database is not homogenous and CERs developed from the database are questionable. Too often a database is built over time, with inputs from various sources, without any one individual responsible for insuring the homogeneity of the data.

The validity of a CER is usually judged by its regression statistics, which measure the accuracy of the fit of the CER to the sample data points used in developing the CER. The most commonly used regression statistic is the coefficient of determination (R2), although there are serval other regression statistics such as Standard Error (SE) and Coeficient of Variation (CV).

Analysts need to ensure that the value of the new system’s parameters fall inside the range of the parameter values for the existing systems. If not, it may not be a good estimate regardless of how good the regression statistics are. For example, a CER developed from data on aircraft that travel at less than the speed of sound may not predict costs well for a system which is to travel at supersonic speeds.

Estimating by the parametric method is appropriate relatively early in the program life cycle when a detailed design specification is not available, but a database of like systems and a performance specification are available. The parametric method is also useful as a check against an estimate made using another method.

Estimating by the parametric method has many advantages over other estimating methods. Because the CER is based upon more than a single data point, estimating by parametrics is less risky than estimating by analogy. A major benefit of applying statistical methods is that one can also measure error from a derived CER and readily perform cost sensitivity analysis based on parameters within each derived CER. The biggest downside of estimating by parametrics is that such a technique is constrained by the amount and quality of the data. Many times an analyst unknowingly incorporates flawed data into the database, in effect producing inaccurate CERs. For this reason and a variety of other reasons, the resulting statistics can be misleading. By providing a much more detailed view of what is being estimated, estimating by engineering circumvents the necessity for statistical analysis.

What is the advantage of using regression analysis to determine the cost equation?

How regression analysis is used in cost estimation?

Which of the following cost estimation methods all apply regression analysis?

What are the advantages and disadvantages of cost estimation?