Marines, The Few, the Proud - link to the Official Marine Corps website.
Configuration Management Plan

Configuration Audit Activity

 

WHAT:

The following activity model, extracted from MIL-HDBK-61A, showing the various constraints, mechanisms/facilitators, inputs and outputs of the activities associated with the Configuration Verification and Audit.

The dictionary definition of the word "audit as "a final accounting" gives some insight into the value of conducting configuration audits.  Configuration management is used to define and control the configuration baselines for the Configuration Items (CIs) and the system.  In general, a performance specification is used to define the essential performance requirements and constraints that the CI must meet.  When a performance specification is baselined by the Government, those requirements are contractual, so it is prudent for the Government to ascertain that the contractor has provided the expected performance capabilities. For complex systems and CIs, a performance audit is necessary to make this determination. Also since development of an item involves the generation of product documentation, it is prudent to ascertain that the documentation is an accurate representation of the design being delivered. To the extent that the Government is buying the CIs to approved detail specifications, the Government would perform this kind of design audit. However, the design activity should perform both performance and design audits on all CIs in the deliverable product. The operation and life cycle support of the CI is based on this documentation.  To fail to assure its accuracy can result in acceptance of items that will not perform as specified, or to greatly complicate future logistics support of the CI.

Configuration audits provide the framework, and the detailed requirements, for verifying that the contractor's development effort has successfully achieved all of the requirements specified in the configuration baselines.  If there are any problems, it is the auditing activity's responsibility to ensure that all action items are identified, addressed and closed out before the design activity can be deemed to have successfully fulfilled the requirements.

Configuration audits consist of the Functional Configuration Audit (FCA) and the Physical Configuration Audit (PCA).

 

WHEN:

Configuration audits are performed before establishing a product baseline for the item.

Additional PCAs may be performed during production for selected changes to the item's configuration documentation or when contractors are changed.

The FCA is conducted on pre-production prototypes that have undergone testing for compliance with the system specification.  The PCA is performed on the first production article.

 

WHO:

The PM is responsible for the planning and conduct of Configuration Audits.

The contractor, in accordance with the terms of the contract, tasked with the development or production of the item shall:

  • Support the conduct of the FCA/PCA.
  • Participate in the resolution of discrepancies identified during the conduct of the FCA/PCA.

A Relationship Map shows the activities associated with reviews and audits.  Their duties and responsibilities should be clearly presented in the CM Plan and in audit plans.

 

WHERE:

Configuration Audits are normally conducted at the contractor facility where configuration documentation and hardware is more readily available.

 

HOW:

Terms and Definitions

General References

Lessons Learned (no audit lessons available yet)

Specific references

There are three phases to the audit process, and each is very important.

  • The pre-audit part of the process sets the schedule, agenda, facilities and the rules of conduct and identifies the participants for the audit.
  • The actual audit itself is the second phase, and
  • The third is the post-audit phase in which diligent follow-up of the audit action items must take place.

For complex products such as major weapon systems, the configuration audit process is a series of sequential/parallel audits of various CIs and the system to Government-controlled System and CI performance specifications conducted over a period of time to verify all relevant elements in the weapon system product structure. Audit of a CI may include incremental audits of lower-level items to assess the degree of achievement of requirements defined in specifications/documentation not controlled by the government.

The audit process will normally involve audits conducted by prime contractors on subcontracted items at subcontractor facilities with or without Government participation (at Government option) and audits of prime contractor developed items conducted by the Government at the contractor’s facility. Each CI may be subjected to separate functional and physical audits, or both audits may be conducted at the same time.

Successful completion of verification and audit activities results in a verified System/CI(s) and a documentation set that may be confidently considered a Product Baseline. It also results in a validated process to maintain the continuing consistency of product to documentation.

As shown in the CM Audit Activity Model below, inputs to the configuration verification and audit activity are:

  • Configuration, status, and schedule information from status accounting,
  • Approved configuration documentation (which is a product of the configuration identification process),
  • The results of testing and verification,
  • The physical hardware CI or CSCI and its representation
  • Manufacturing/build instructions and engineering tools, including the software engineering environment, used to develop, produce, test and verify the product

The configuration verification and audit process includes:

  • Configuration verification of the initial configuration of a CI, and the incorporation of approved engineering changes, to assure that the CI meets its required performance and documented configuration requirements

  • Configuration audit of configuration verification records and physical product to validate that a development program has achieved its performance requirements and configuration documentation or the system/CI being audited is consistent with the product meeting the requirements.

The common objective is to establish a high level of confidence in the configuration documentation used as the basis for configuration control and support of the product throughout its life cycle. Configuration verification should be an imbedded function of the contractor’s process for creating and modifying the CI or CSCI. Validation of this process by the Government may be employed in lieu of physical inspection where appropriate.

EIA-649 Principles applicable to Configuration Verification and Audit:

  • Principle 39.  Verification that a product's requirement attributes have been met and the product design meeting those attributes has been accurately documented is required to baseline the product configuration.
  • Principle 40.  Verification that a design achieves its goals is accomplished by a systematic comparison of requirements with the results of tests, analyses or inspections.
  • Principle 41.  Documentation of a product's definition must be complete and accurate enough to permit reproduction of the product without further design effort.
  • Principle 42.  Where necessary, verification is accomplished by a configuration audit.
  • Principle 43.  Periodic reviews verify continued achievement of requirements, identify and document changes in performance, and ensure consistency with documentation.

Configuration Verification and Audit Concepts and Principles

There is a functional and a physical attribute to both configuration verification and configuration audit. Configuration verification is an on-going process. The more confidence the Government has in a contractor's configuration verification process, the easier the configuration audit process becomes.  The reward for effective release, baselining and configuration/change verification is delivery of a known configuration that is consistent with its documentation and meets its performance requirements. These are precisely the attributes needed to satisfy the ISO-9000 series requirements for design verification and design validation as well as the ISO 10007 requirement for configuration audit.

Configuration Verification

Configuration verification is a process that is common to configuration management, systems engineering, design engineering, manufacturing, and quality assurance. It is the means by which a contractor verifies his design solution. The functional aspect of configuration verification encompasses all of the test and demonstrations performed to meet the quality assurance sections of the applicable performance specifications. The tests include verification/qualification tests performed on a selected unit or units of the CI, and repetitive acceptance testing performed on each deliverable CI, or on a sampling from each lot of CIs, as applicable. The physical aspect of configuration verification establishes that the as-built configuration is in conformance with the as-designed configuration. The contractor accomplishes this verification by physical inspection, process control, or a combination of both.

Once the initial configuration has been verified, approved changes to the configuration must also be verified. Figure 2 (below) illustrates the elements in the process of implementing an approved change.

Change verification may involve a detailed audit, a series of tests, a validation of operation, maintenance, installation, or modification instructions, or a simple inspection. The choice of the appropriate method depends upon the nature of the CI, the complexity of the change, and upon the support commodities that the change impacts.  If the change is being introduced into a production line, and all future units will have the change incorporated via the production process, it is normally sufficient to ensure that:

  • Manufacturing instructions contain the change and are released for use (as with a work order), and
  • The first articles produced are inspected for compliance.

However, if support elements are impacted, or the change requires incremental retrofit to many units, complete implementation and verification of the change can be a lengthy process.  Under these circumstances, implementation planning must define the extent to which the change to each unit and support commodity is to be verified; and the records to be maintained.  When materials, parts, or retrofit kits are ordered in incremental stages (e.g., per year, per month), the incremental ordering and supply actions should also be verified.

Retrofit changes to organically supported items are verified and reported to the Government's status accounting system by the activity given installation and checkout responsibility for the retrofit. Changes retrofit by the contractor for contractor supported items are verified by the contractor.

Configuration Audit

There are three phases to the audit process, and each is very important. The pre-audit part of the process sets the schedule, agenda, facilities and the rules of conduct and identifies the participants for the audit. The actual audit itself is the second phase, and the third is the post-audit phase in which diligent follow-up of the audit action items must take place. For complex products such as major weapon systems, the configuration audit process is a series of sequential/parallel audits of various CIs and the system to Government-controlled System and CI performance specifications conducted over a period of time to verify all relevant elements in the weapon system product structure. Audit of a CI may include incremental audits of lower-level items to assess the degree of achievement of requirements defined in specifications/documentation not controlled by the government.

The process will normally involve audits conducted by prime contractors on subcontracted items at subcontractor facilities with or without Government participation (at Government option) and audits of prime contractor developed items conducted by the Government at the contractor’s facility. Each item may be subjected to separate functional and physical audits, or both audits may be conducted at the same time.

Functional Configuration Audit

The Functional Configuration Audit (FCA) is used to verify that the actual performance of the CI meets the requirements stated in its performance specification and to certify that the CI has met those requirements.  For systems, the FCA is used to verify that the actual performance of the system meets the requirements stated in the system performance specification.  In some cases, especially for very large, complex CIs and systems, the audits may be accomplished in increments.  Each increment may address a specific functional area of the system/CI and will document any discrepancies that are found in the performance capabilities of that increment.  After all of the increments have been completed, a final (summary) FCA may be held to address the status of all of the action items that have been identified by the incremental meetings and to document the status of the FCA for the system or CI in the minutes and certifications.  In this way, the audit is effectively accomplished with a minimum of complications.

Although an FCA is only required once for each CI or system, a number of FCA-like activities may be accomplished at other times during the life cycle of the CI or system.

  • Many Class I ECPs incorporate a new design into the baselined design.  The performance of each new design element must be verified to ensure that it will not degrade performance of the CI or system below the performance specified by it's Government-controlled performance specification.  The degree and type of verification will be included as part of the ECP; it may vary from a simple analysis of the similarity to the old design to a lengthy program of testing similar to the original verification testing accomplished during the System Development and Demonstration (SD&D) phase.  However, it is important to understand that a complete retest and FCA are not required for each ECP; only the verifications specified in the ECP are required.
  • If the Government is controlling the detailed design, a production contract may require a "first article" inspection to be accomplished.  This would include more comprehensive "testing" than the normal production acceptance tests, and the test data resulting from the "first article" would be subject to a review process not unlike an FCA.
  • An ECP or a new contract may call for the development of a new CI(s) and incorporation of the new CI into the system via a modification program.  The expected performance of the new CI would commonly be defined in a performance specification, and the results of the verification testing of the CI would be checked at an FCA for the new CI. In addition, some retesting of the existing system elements with the new CI incorporated would normally be required, and those results would also be subject to a review similar to an FCA.

Physical Configuration Audit

The Physical Configuration Audit (PCA) is used to examine the actual configuration of the CI that is representative of the product configuration in order to verify that the related design documentation matches the design of the deliverable CI.  It is also used to validate many of the supporting processes that the contractor uses in the production of the CI.  The PCA is also used to verify that any elements of the CI that were redesigned after the completion of the FCA also meet the requirements of the CI's performance specification.  In cases where the Government does not plan to control the detail design, it is still essential that the contractor conduct an internal PCA to define the starting point for controlling the production design and to establish a product baseline. Additional PCAs may be accomplished later during CI production if circumstances such as the following apply:

  • The original production line is "shut down" for several years and then production is restarted
  • The production contract for manufacture of a CI with a fairly complex, or difficult-to-manufacture, design is awarded to a new contractor or vendor.

This re-auditing in these circumstances is advisable regardless of whether the contractor or the government controls the detail production design.

Application of Audits during Life Cycle

It is extremely unlikely that FCAs or PCAs will be accomplished during the Concept Refinement and Technology Development phases of the life cycle.  Audits are intended to address the acceptability of a final, production-ready design and that is hardly the case for any design developed this early in the life cycle. 

It is during the System Development and Demonstration (SD&D) phase that the final, production, operationally ready design is developed.  Thus, this phase is normally the focus for the auditing activity. Either the Government or the contractor will conduct a PCA for each hardware CI that has completed the FCA process to "lock down" the detail design by establishing a product baseline. Hardware CIs built during this phase are sometimes "pre-production prototypes" and are not necessarily representative of the production hardware. Therefore, it is very common for the PCAs to be delayed until early in the Production and Deployment phase of the program.

Requirements to accomplish FCAs for systems and CIs are included in the Statement of Work (SOW) tasking. The FCA is accomplished to verify that the requirements in the system and CI performance specifications have been achieved in the design.  It does not focus on the results of the operational testing that is often accomplished by operational testing organizations in the services, although some of the findings from the operational testing may highlight performance requirements in the baselined specification that have not been achieved. Deficiencies in performance capability, as defined in the baselined specification, result in FCA action items requiring correction without a change to the contract.  Deficiencies in the operational capability, as defined in user-prepared need documents, usually result in ECPs and/or contract changes to incorporate revised requirements into the baselined specifications or to fund the development of new or revised designs to achieve the operational capability.

Since the final tested software design verified at the FCA normally becomes the production design, the PCAs for CSCI are normally included as a part of the SOW tasking for the SD&D phase.  CSCI FCAs and PCAs may be conducted simultaneously to conserve resources and to shorten schedules.

It is normal that the first production units in the Production and Deployment and Operational and Support Phases would be subjected to a PCA, which, depending on whether the acquisition strategy was performance or detail design based, would be conducted by the contractor or by the Government, respectively. This PCA allow the establishment of a Product Baseline for the CI reflecting the design that will be delivered to the field and will require support. From a logistics support standpoint, it is essential that the support activity have an accurate picture of the exact configuration.  If it does not, it is likely that the wrong spares will be acquired or redesign of the CI will be based on inaccurate information, leading to problems in the operation and/or support of the CI.

During a PCA, the deliverable item (hardware or software) is compared to the product configuration documentation to ensure that the documentation matches the design. This ensures that the exact design that will require support is documented.  The intent is that an exact record of the configuration will be maintained as various repair and modification actions are completed. The basic goal is sometimes compromised in the actual operation and maintenance environment.  Expediency, unauthorized changes, cannibalization, overwork, failure to complete paperwork, and carelessness can cause the record of the configuration of operational software or hardware to become inaccurate.  In some situations, a unit cannot be maintained or modified until its configuration is determined. In these kinds of circumstances, it is often necessary to inspect the unit against approved product configuration documentation, as in a PCA, to determine where differences exist. Then the unit can be brought back into conformance with the documentation, or the records corrected to reflect the actual unit configuration.

Auditing in the Performance-based Acquisition Environment

As discussed above, configuration audits address two major concerns:

  • The ability of the developed design to meet the specified performance requirements. The FCA addresses this concern.
  • The accuracy of the documentation reflecting the production design. This concern is addressed by the PCA.

Over the years prior to acquisition reform, the DoD developed hardware and software audit topics that were to be addressed by the FCA and the PCA, respectively. To document acceptability of a contractor’s accomplishments in the FCA topic area, a series of certifications were established. Similarly another series of certifications were established for the PCA topic areas. The audit teams completed the certifications that were applicable to the type of audit they were performing. Because the Government typically took control of the detail design, it conducted both FCA and PCA for each CI. The Government teams eventually addressed all the audit topic areas that were applicable to the type of item (hardware or software) being audited.

Acquisition reform policy requires acquisition of deliverable products based on performance specifications, rather than detail specifications unless it is essential to buy an identical item. Using the certifications as they existed before acquisition reform would mean that:

  • The Government would normally conduct FCAs for the System and CIs with Government controlled performance specifications and would thus address (and certify) the FCA topics
  • The Contractor would normally conduct PCAs without any Government involvement. Thus the Government would not address (and certify) any of the Government’s PCA concerns. Therefore, because some PCA topics have applicability even in a performance-based acquisition, this handbook no longer attributes the topics of concern, and the certifications specifically to either an FCA or a PCA.

 


[1] Go to Defense Standardization Program Office, ASSIST Database for the most up-to-date and ONLY official version of MIL-HDBK-61A. (http://assist2.daps.dla.mil/quicksearch/)

 

(Last Updated December 08, 2011)    Web Publisher | Privacy Policy | USMC Home Page