Configuration Audit Guide
PURPOSE: This guide prescribes the requirements for conducting both
Functional Configuration Audits and
Physical Configuration Audits on Systems,
Equipment and Computer Software Programs.
CLASSIFICATIONS: Configuration audit is a formal examination of the
functional and/or physical characteristics of a configuration item for verification
of conformance to its documented requirements. There are two types of audits
within the configuration management process covered in this template. They
are described as follows:
- Functional Configuration Audit (FCA). A formal examination of the "as tested"
characteristics of a configuration item (CI) focuses on design and performance
suitability. An FCA validates the performance characteristics specified in
the configuration item's functional or allocated configuration identification
documents and it's operational and support documents are complete and satisfactory.
The FCA also demonstrates that System Development & Demonstration is sufficiently
mature for entrance into Low Rate Initial Production.
- Physical Configuration Audit (PCA). A formal examination of the "as built"
physical configuration or "as coded" total system software conforms to the
design and construction process or technical documentation that defines it.
A PCA verifies the product configuration documentation to include design specifications,
drawings, and code listings. It is performed during the Production and Deployment
phase and focuses on production suitability. Satisfactory completion of the
audits is necessary to establish the product baseline.
APPLICATION: This guide is applicable for the performance of configuration
audits as specified in the contract, Statement of Work (SOW), and the Contract
Data Requirements Lists (CDRL).
The configuration item or system shall not be audited separately without
prior Government approval of the functional baseline and allocated baseline
for the CI, or system, involved. Recommendation to approve or disapprove
an audit shall be based upon and governed by the procedures and requirements
delineated in this guide.
Audit performance ultimately resides with the Program Management Office.
The Program Manager (PM) has overall disposition authority on audit results
and reports. The PM will designate a representative to be the focal point
for audits issues. The designee will serve as audit team leader, develop an
audit plan (Attachment 1), and write an FCA/PCA Review Report (Attachment
2) within two weeks of the audit completion to identify findings and any action
items and the plans for their resolution. The plan and review report will
be submitted as recommendations for PM approval.
- Audit Plan
- Format of Audit Report
- General Guidance for Conducting Configuration Audits
- FCA Checklist for information needed prior to or at audit
- PCA Checklist for information needed prior to or at audit
- Criteria for conducting the FCA or PCA
ATTACHMENT 1: CONFIGURATION AUDIT PLAN
- Background and Objectives: This section describes the background circumstances for FCA or PCA
Audits. It should identify the contract, SOW and CDRL references, program/system, and tentative
dates as indicated in the program schedule.
- Purpose: This section identifies the purpose of the audit, the Configuration Item(s) and
the principal result (validation of requirements or design).
- Review Procedures: This section should identify specific events that have to be accomplished
to ensure a successful audit.
- The FCA or PCA date should be established at least 60 days prior to the audit. This date
should be coordinated with the contractor and approved by the PM.
FCA/PCA Date: ______________________________.
- The designated representative should formally solicit support from the concerned disciplines
and functional areas experts. This should be accomplished once a specific date is approved. Date
letters sent: _____________________________.
- The contract should be reviewed at least 45 days prior to the audit to determine what
requirements are on contract, what data items are due prior to FCA or PCA, whether they require
approval action (agendas, specifications, drawings, etc.), and the status of delivery/approval
of the data. Date of review: __________________.
- All team members should be identified by name at least three weeks prior to the audit.
- A preparatory meeting should be held at least one week before the scheduled audit. At this
meeting, team members will be assigned specific responsibilities to be accomplished at the FCA or PCA.
- The audit shall be conducted using EIA-649 or MIL-STD-973 as guidance. Minutes are to be
provided by the contractor with action items.
- Explain how action items will be tracked for completion.
- A formal FCA/PCA review report with a copy of the minutes of the audit, in accordance with
Attachment 2, will be written NLT ten days after the audit.
- A final status letter will be provided to the Program Manager when all action items have
ATTACHMENT 2: FORMAT OF AUDIT REPORT
- Background: This section describes the background circumstances of the FCA/PCA. It should identify the
location and dates and provide a brief synopsis of events leading to the requirements of the audit.
- Contractual Requirements: This section identifies the specific contractual requirements.
- PURPOSE: This section identifies the purpose of the audit. Audits are essentially a validation
that the Configuration Item (CI) achieves the performance and functional characteristics and the "As Built"
conforms to the technical documentation.
- AUDITS: This section documents in some detail the thoroughness and extent of the audit. Areas not
investigated will be discussed, with reasons provided. This section also identifies the methodology used in
conduction the audit. Team members will be identified along with their functional specialties. Specific
methods and depth will be described. Formal minutes with action items shall be attached. This section
should identify how the action items will be corrected and tracked for completion.
- SUMMARY OF FINDINGS: Summarize the findings of the audit. Identify those areas having an impact
on production. Include a general discussion of the over-all evaluation of the contractor's efforts.
- RECOMMENDATIONS: The report will contain a recommendation to the PM for approval/disapproval of
the audit. The recommendation will suggest specific methods to correct identified deficiencies. If applicable,
make recommendations concerning the necessity for any audit follow-ups. The report is signed by the team
leader and is submitted to the PM with a summary of the recommendations.
ATTACHMENT 3: GENERAL GUIDANCE FOR
CONDUCTING A CONFIGURATION AUDIT
- On-Site Introduction (Team Orientation)
- Given by contractor host
- Identification of items to be audited
- Specification number
- Configuration identifier
- Serial number(s)
- Assembly/computer program/software identification numbers
- Conducting the Audit
- Contractor shall provide:
- Documentation described in EIA-649 or MIL-STD-973
- Facilities for team members to work
- Personnel to go through the audit exercise with the team members, with access to contractor's
expertise or other contractor areas (facilities) or organizations as necessary to obtain
resolutions or provide answers.
- Formal minutes based on mutually agreed-to rough draft provided by the team.
- Start the meeting by introducing your team, describing your FCA or PCA plans, and assure that the
contractor understands that you have no authority to change the contract
- Break into groups after the on-site introduction and the start of the audit.
- Suggest disciplines requested for each group (but not limited to):
Final Actions of the Audit
- Audit Co-Chairperson:
- Design Specifications and Test Data Review:
- Hardware Program Manager
- Software Program Manager
- Hardware Responsible Engineer
- Software Responsible Engineer
- Hardware Quality Assurance
- Software Quality Assurance
- Hardware Configuration Management
- Software Configuration Management
- Logistics/Technical Specialties (includes Safety, Human Factors, Reliability, Equipment Qualification):
- Specialty Engineer
- Quality Assurance
- Configuration Management
- Drawings and Associated Documentation/Controls/Records:
- Program Management
- Responsible Engineer
- Each audit day is concluded with a short program office team caucus
Generate Audit Minutes
- Members identify subjects that they feel need attention by team members
- Identify suspect areas, data, and activity and suggest corrective action, as appropriate,
or request direction.
- Discuss problems, identify next day's activities/events, and provide information of interest
or use to team members.
- Receive any instructions/assignments/suggestions necessary for successful task accomplishment.
Government representatives are observers at subcontract audits.
- Certification sheets similar to examples in EIA-649 or MIL-STD-973 will be completed for the entire
audit, plus any action item sheets generated as part of the audit. Assure that the minutes include the
disclaimer of any authority to change the contract.
- Upon completion of the audit, minutes will be prepared and distributed to the chairperson. Minutes
prepared in final format will be signed by both the government and contractor representatives and
delivered to the government in accordance with the Contract Data Requirements List.
ATTACHMENT 4: FUNCTIONAL
CONFIGURATION AUDIT CHECKLIST
Note: Items herein may vary according to
the provisions of the contract.
- Verify the Hardware Configuration Item (HWCI) and/or Computer Software Configuration Item (CSCI)
performance complies with its performance specification (e.g. B or development specification).
- Verify the compatibility/completeness of specification requirements versus test
procedures/plans/results and analyses.
- Information Needed Prior to or at Audit
- Approved performance specification and draft design specification (e.g. approved
development specification and draft of design specification.
- A current listing of all non-conformances (e.g. deviations, waivers, etc) either
approved or requested.
- Status of test programs to test configured items with automatic test equipment.
- Test plans/procedures and available acceptance test plans/procedures.
- A complete list of successfully accomplished functional tests during which pre-acceptance
data was recorded.
- A complete list of functional tests even if detailed test data are not recorded.
- A complete list of functional tests required by the specification but not yet performed (to be performed
as a subsystem or system test).
- Test reports (Validated data).
- A list of tests, which failed (see E, F, and G); action items are generated from this data.
- Drawings to identify items to be audited, to include:
- Specification identification number
- Configuration identifier
- Serial number(s)
- Assembly/computer program/software identification numbers
- Review Preliminary Design Review (PDR)
and/or Critical Design Review (CDR) minutes and
action items for closure and items requiring further checks and/or verification. Review pre-FCA
- For CSCI the following additional items should be checked:
- The contractor should provide the FCA team with a briefing for each CSCI being audited and
should delineate the test results and findings for each CSCI. As a minimum, the discussion should
include CSCI requirements that were not met, including a proposed solution to each item, an account
of the Engineering Change Proposals (ECP) incorporated
and tested as well as proposed, and a general presentation of the entire CSCI test effort
delineating problem areas as well as accomplishments.
- Audit the formal test plans/descriptions/procedures and compare them against the official
test data. Check results for completeness and accuracy. Deficiencies should be documented and
made a part of the FCA minutes. Establish and document completion dates for all discrepancies.
- Perform an audit of the Software Test Reports to validate that the reports are accurate and
completely describe the CSCI tests.
- Review all approved ECPs to ensure that they have been technically incorporated and verified.
- Review all updates to previously delivered documents to ensure accuracy and consistency throughout
the documentation set.
- Review the PDR and CDR to ensure that all findings have been incorporated and completed.
- Review the interface requirements and the testing of these requirements for each CSCI.
- Review database characteristics, storage allocation data and timing, and sequencing characteristics
for compliance with specified requirements.
ATTACHMENT 5: PHYSICAL CONFIGURATION AUDIT
Note: All items herein may vary according
to the provisions of the contract.
- Objectives--The Physical Configuration Audit is:
- A formal examination of the as-built version of a configuration item against its design documentation
in order to establish the product baseline. (A successful PCA is necessary to establish the product baseline.)
- A determination that the as-built configuration is reflected by its released engineering documentation
and quality assurance records.
- A determination that the acceptance test requirements prescribed by the documentation are adequate
for acceptance of production units of a configuration item (CI) (or end item) by quality assurance activities.
- A detailed audit of engineering drawings, specifications, technical data, and tests utilized in production
of hardware configuration items (HWCI) (or end items).
- A detailed audit of specifications (or technical descriptions, flow charts/Program Design Language,
code listing, manuals/handbooks) for CSCIs (or software end items).
- Information Needed Prior to or at Audit:
- Successful accomplishment of a PCA:
- May be conditional; e.g., higher-level tests/actions to be completed, as long as this will not impact
PCA accomplishment. The FCA and PCA are frequently combined.
- A copy of FCA minutes and associated data (attachments), if FCA and PCA are not combined.
- An approved performance specification and the final draft design specification. (The design
specification should be submitted sufficiently in advance of the PCA to permit government review.)
- Includes top level and any sub-tier specifications.
- For computer programs/software this shall include diagrams, code listings, etc which comprise part
of the design specification.
- A current listing of all non-conformances (e.g. deviations, waivers, etc) either approved or requested.
- Identify any changes actually made during test.
- Identify changes not completed.
- Identification of items to be audited, to include:
- Specification identification number
- Configuration Item identifier
- Serial number(s)
- Assembly and/or computer program/software identification number(s).
- Engineering drawings, associated lists and related technical data.
- Includes engineering drawing index containing revision letters assembled by top drawing number (may
be indentured listing).
- Approved nomenclature and nameplate(s).
- Operating maintenance and illustrated parts breakdown manuals (as applicable).
- Computer Program/Software manuals; e.g., users' manuals, positional handbooks, programming manuals,
etc. as applicable.
- Version Description Document.
- Media for delivery of software; e.g., card decks, tapes, etc.
- Acceptance Test Plan(s), procedures and test data (report(s)) for each item.
- A complete list of functional tests required by the CI (or end item) specification but not yet performed.
- A review of PDR and CDR minutes and action items for closure and items for further check/verification.
- A complete shortage list.
- To include both hardware and software shortages (may be on separate lists).
- A proposed DD 250 (or subcontractor documents).
- Manufacturing instruction sheets must be available on site (on request).
- A list of approved and outstanding changes against a CI (or end item).
- Logistics data to include spares and repair parts lists or recommendations, common and peculiar
tools, and other support requirements data as applicable shall be available on site.
ATTACHMENT 6: CRITERIA USED IN
CONDUCTING FUNCTIONAL AND PHYSICAL CONFIGURATION AUDITS
In conducting a Functional Configuration Audit (FCA) or a Physical Configuration
Audit (PCA), evaluate the contractor's policies, methods, and techniques against
the following general criteria and other specifically developed criteria.
You may not need all of these criteria, or you may have to add to them. Exercise
discretion; make sure to document any discrepancies not resolved. Group efforts
are to be oriented toward FCA/PCA objectives.
* Items apply to FCA only.
- Compare configuration of the CI (or end item) as documented with the design specification.
Compare configuration of CI as documented with the performance specification.
- Is the configuration of the CI (or end item) the same as the "as built" product configuration?
- Review the list of contractor's internal documentation for hardware configuration items (HWCI) and computer
software configuration items (CSCI).
Compare configuration audited at FCA with that to be audited at the PCA.
- Does the documentation reflect the physical configuration of the items for which test data are
verified? (For software this includes documents such as code listings, version description documents,
manuals, media identification; e.g., card decks, tapes, etc. as appropriate).
Differences between product and specification and those between FCA and PCA configurations tested
shall be recorded in the minutes.
- Review FCA Minutes for recorded discrepancies that require action (Has proper closure been
achieved and tested?)
Review all approved non-conformances (e.g. deviations, waivers, etc) to determine extent of variance from
applicable specifications and standards.
- Differences that are due to approved and tested changes that are compatible with approved specifications
and reflected in the engineering data are acceptable.
- Determine impact on and validity of the previous FCA and the current PCA activity. Establish
course of action (how to proceed or call it off). Differences due only to test instrumentation
Review list of shortages and un-incorporated design changes.
- Does a basis for compliance with the specifications and standards exist?
- Record any part of the PCA that fails to meet specifications or standards but is not an approved
- Identify and record any deviation/waiver found which is not on the list supplied. (Initiate
action item/corrective action.)
* Verify all requirements of the performance specification,
whether qualification testing is required or not for a given requirement.
- Accomplish as coordinated engineering/Quality Assurance effort.
- Determine impact on PCA results and validity of any shortages/changes.
- Determine where and by whom any necessary configuration changes will be made
(before or after delivery).
Verify all requirements of the design specification, whether acceptance testing is required or not
for a given requirement.
- When qualification testing does not cover or does not adequately cover a test/design requirement,
review contractor internal/engineering test data to verify all design requirements have been met.
Review test plans, procedures, and descriptions, and review test reports for compliance with specification
- When acceptance testing does not cover or does not adequately cover a test/design requirement,
review FCA or contractor internal/engineering test data to verify all design requirements have been met.
10. Review CSCI (or end item) for format and completeness.
- Confirm inspection and test of contractor equipment end items. (Review contractor records to confirm. Data
should support proper buy-off.)
- For software this includes Preliminary Qualification Test (PQT) and Formal Qualification Test (FQT)
test plans/procedures/results audit versus specification. (Objective is to verify PCA results are still
exactly applicable or that any discrepancies are authorized, properly tested and documented in the engineering
data and the test results are compatible).
- Check completeness, accuracy, tolerances, and compatibility.
- Are the tests properly witnessed?
Review the Version Description Document for completeness and compatibility with data audited.
- Compare top-level (CSCI) flow charts with sub-tier computer software component (CSC) flow charts.
- Review CSC flow charts and description.
- Review flow charts for proper entries, symbols, labels, tags, and exits.
- Review CSC interface requirements.
- Compare detailed CSC flow charts with coded program for accuracy and completeness.
- Crosscheck a current listing of instructions with the listing in the design specification.
- Review database characteristics, storage allocation charts, and timing and sequencing characteristics.
- Examine an actual tape, card deck or other media for conformance to the design specification.
(Packaging and marking requirements shall be complete and compatible with all documents, material requirements
conform, and serials are compatible with documentation of items audited.)
Check positional handbooks, user and computer programmer manuals for format and conformance with
applicable data items.
Review microprocessor software portion of hardware design specifications for the characteristics
noted herein for completeness and adequacy of software requirements and compatibility with the hardware
design and interface.
- Formal acceptance/verification of handbooks/manuals is not part of PCA but must wait until system testing
to insure procedural content is correct.
Engineering Change Proposals (ECP)/Changes
- Do all requirements interface adequately?
- Are the test results acceptable? (Including hardware/software or software/software interfaces.)
- Have all ECPs/Changes been included in the development and acceptance tests, and are the results
compatible with the requirements?
- Are any outstanding items documented and appropriate actions and schedules established?
Verify the validity and adequacy of the acceptance test requirements defined in the specifications.
- Have all PDR/CDR findings been incorporated and tested?
Final HWCI/CSCI (or end item) test report.
- Are all tests clearly applicable?
- Are test requirements completely defined?
- Are all tests supported by applicable data requirements?
- Make a direct comparison of test methods/procedures and test results with the performance/design
requirements of the deliverable HWCI/CSCI (or end item) as required. (Must be in the detail necessary to
establish the methods and instrumentation, as required by the contractor's internal procedures, satisfy the
design specification and be adequate to verify the performance quality of the HWCI/CSCI (or end item)).
- If acceptance test requirements changes are required, their validity must be reconciled with the results
of the performance tests conducted at FCA. (Determine impact on PCA objectives and audit completion, recovery,
- Are any changes made during the test identified? (Are they properly authorized; are they acceptable in
terms of test results/retest procedure?)
Test reports, procedures, and analyses and other data.
- Audit for accuracy and completeness.
Review HWCI/CSCI (or end item) specification(s) to assure that they have been validated as adequate to
define the product.
- Generally made attachments to the certification and worksheets filled out as an audit record.
Audit a representative number of drawings.
- Is the document complete, and is it in agreement with the approved issue?
- Does the spec contain the necessary testing, mobility/transportability and packaging requirements?
- Are the drawings accurate, and do they include authorized changes?
- List drawings, revision letter, title, and approval date of drawing.
- Are the drawings accurate and complete, and do they describe the equipment?
- Do the drawings conform to contractual requirements?
- Are engineering documents and/or release records capable of:
Review drawings of parts/assemblies to be provisioned.
- Determining composition of any part at any level?
- Determining the composition of the end item (or CI) with respect to other end items (CIs)?
- Determining the CI (end item) and associated serial number on which subordinate parts are used?
- Determining accountability of Class I and Class II changes which have been partially or completely
released against the CI (end item)?
- Determining the CI (end item) and serial number effectivity of any change?
- Determining the standard spec or part number used with any non-standard part number?
- Determining the contractor spec document and specification control number associated with any sub-contractor,
vendor or supplier part number?
- Identifying changes and retaining records of superseded configuration formally accepted by the procuring activity?
- Determining the next higher using assembly?
- Identifying all Class I and Class II engineering changes released for production incorporation?
- Determining the configuration released for each CI (end item) at the time of formal acceptance?
Audit samples of drawings versus Manufacturing instruction sheets.
- How is the referenced test data to be furnished?
Audit engineering release and change control records.
- Check drawings to assure accuracy of translation and conformance with hardware "as built."
- Do the drawing numbers, revision letters and approval dates comply (are compatible) with Manufacturing sheets?
- Are materials called for the same?
- Are instructions complete and compatible?
- Are processes called on Manufacturing sheets compatible?
- Are all marking requirements in conformance?
- Are all changes incorporated in Manufacturing sheets?
- Are there more than 5 Engineering Orders attached to the drawing?
- Are all proprietary markings justified?
- Are all drawings released and identified in release records?
- Are all drawings accounted for in a selected black box? Is there drawing continuity?
- Do part numbers agree with the drawings?
Review Logistics data for adequacy, completeness and compatibility with CI configuration.
- Are engineering change control numbers unique?
- Are drawing/part number systems controlled, and do they satisfy requirements?
- Are serial number assignments controlled?
- Verify there is one release record for each drawing number and that it contains as
applicable: Drawing number, revision, title, number of sheets, date of release, date of
change release(s), ECO (EO) number, serial numbers, top drawing number and spec number.
- Is control adequate to control processing and formal release of engineering changes?
Review prepared back-up data for correct types and quantities to insure adequate coverage at the
time of shipment to the user.
- Review Logistics Support planning documents, plans, lists (e.g. provisioned or to be provisioned).
- Are manuals/training documentation/data acceptable?
- Review long lead-time items.
- Are all long lead or provisioned items acquired before PCA of current configuration?
- Are interim releases of spares of the current configuration? (Do change/release control methods control this?)
- Review data of parts/assemblies to be provisioned for adequacy.
- Review proposed DD 250 (or comparable shipping data) to insure that it adequately defines the
equipment/computer programs. (Are open items/tasks listed as deficiencies/shortages?)
- Review any shortages and un-incorporated design changes. (Assure compatibility of DD 250 (or
comparable documents) and audit deficiency list and supporting action items or other back-up data.)
- Does the DD 250 or equivalent document include any discrepancies uncovered at the PCA? (Specify any
such items on the DD 250.)
- Has a determination been made as to where any required configuration changes should be made? (Before or
after delivery or in the field.)
- Do quality control records reflect the configuration under audit?
[Give credit to Mr. M. Ucchino, DSN 986-0815 for the guidelines for audits.]