Overview
In this activity, cost data and software metric reports provided by the software engineering process group (SEPG) are reviewed and analyzed. Actual performance results are evaluated against planned rates of progress. Risks are identified and corrective actions are taken.
Roles and Responsibilities
The SEPG is responsible for providing the software metric reports to the project software manager, and may be asked to participate in analyzing and evaluating the software metrics.
The project software manager is responsible for analyzing the project metrics to determine the project’s status.
The project manager may be consulted on potential changes to the software schedules and budget.
The software engineering manager and software test manager will assist in evaluating the software metrics.
Controls
The organization’s software baseline (see activity 4.2.3 Analyze And Develop Software Baseline) which documents the organizational norms for the metrics collected.
Inputs
Software metric reports received from the SEPG.
Procedures
Throughout the software project’s life cycle, software metrics are entered into the software metrics database which is maintained by the SEPG. The SEPG supplies reports to the project software manager. When analyzing the software metric data, the project software manager should compare it to the organizational norms as defined in the organization’s software baseline. SEPG members also may be consulted when analyzing the software project metrics. Potential problems identified during the metrics analysis should be discussed with the software project personnel and identified as risks (see 1.2.1.1 Perform Risk Management). The current set of software metrics collected and reports generated are in the Langley Research Center’s (LaRC) Software Metrics Database User’s Guide. For security reasons, cost data is maintained separately. The following steps give general guidance on interpreting the software metrics. No order to the steps should be assumed.
1) The number of Software Change Requests (SCR) (see Appendix C) impacting a software product does not give insight into its quality, but does provide information on its stability. SCRs are to be expected initially after a software product is baselined, but their number should decrease over time. A software product with no SCRs indicates it is not being used or maintained. Of special concern are SCRs for the software requirements that may indicate the instability of project-level requirements or premature baselining. If the software requirements are being changed based on changes to the system requirements (as can be determined by reviewing the SCR forms), the project software manager and software engineering manager should review the software project’s scope to determine if the software project is still feasible based upon the current estimated schedule and budget. SCR information is currently being provided on the Project Change Profile Report.
2) When analyzing cost data, it is necessary to understand what work was completed, and to base the cost analysis on easily identifiable milestones such as completing a product or task. When one of these milestones is reached, the following cost analysis should be performed.
A) Determine the actual cost of the work performed (ACWP) to date. This information should be readily available from the cost database or spreadsheet.
B) Determine the budgeted cost of the work performed (BCWP). This is the previously budgeted cost of completing the work up to and including the newly reached milestone. This can be obtained from the software project’s cost estimate.
C) Determine the budgeted cost of the work scheduled (BCWS). This is the cost of the work which was to be completed by the current date. This can be obtained from the software project schedule and cost estimate.
D) Determine the Schedule Performance Index (SPI) for Efficiency. This is defined as BCWP divided by BCWS.
E) Compute the new Estimate at Completion (EAC) where:
EAC = ACWP + ((1/SPI) * (Total budget for software project - BCWP))
This EAC assumes that progress will continue at the current rate. In certain cases, this may be pessimistic. If a non-recurring event caused a cost overrun, the following formula may be more appropriate:
EAC = ACWP + (BAC-BCWP)
The computed EAC (and the values used to compute it) should be discussed with the project manager to determine if additional funding is required for the software effort. The updated EAC should be recorded in the cost database.
3) The number of Software Trouble Reports (see Appendix C) received indicates the number of problems encountered by the testers and/or users. A high number of the Software Trouble Reports that are determined not to be errors but user or tester mistakes indicates poor user or test documentation. The number of these reports should decline as the testing progresses. A possible sign of inadequate testing is having an overall low number of Software Trouble Reports, but the rate of these reports does not decline as testing progresses [SEL-84-101, p. 6-8]. Software Trouble Report information is currently being provided in the Project Error Reports, and should be discussed with the software test manager.
4) The hours spent on the software project are a good indicator of the software project’s health. Spending fewer hours than expected, even if the schedule and milestones are being met, may indicate that the software project is understaffed. Problems may show up later due to costly learning curves or the poor quality of products developed earlier. Spending more hours than expected may indicate the project is stalled and falling behind, or that problems were encountered. Effort information is currently being provided in the Project Activity Hours Report.
5) Change in size estimates is another key indicator of project stability [SEL-81-305, p. 53]. When size estimates increase, cost estimates and schedule estimates should be reconsidered. Growth in size estimates after baselining the software requirements also may indicate "requirement growth" or poor designs. Size information is currently being provided in the Project Status Report.
6) Based upon the project software manager’s review of the metric reports, risks may be identified (see 1.2.1.1 Perform Risk Manager) or software project plans may need to be updated or modified (see 1.2.1.4 Maintain Software Project Plans).
Outputs
Risks identified based upon the software project metrics.
The updated cost estimate recorded in the cost database.