Major Project Reviews
II.1 Spaceflight Experiments Initiatives Review
Purpose
The Spaceflight Experiments Initiatives Review (SEIR) provides upper management with their first detailed look at new proposed work, and assesses the project’s reasonableness. As part of this review, the presentations for all non-advocacy reviews for spaceflight experiments are reviewed. For this review, the Head of the Space Projects Office serves as the chairperson, who selects the secretary and review panel members. The principal investigator should request that the review committee meet a minimum of four weeks prior to the response date for an Announcement of Opportunity or other request for proposals (RFP). For an unsolicited proposal, the review should be held as early as possible prior to a major commitment by the NASA Langley Research Center (LaRC).
Major Topics to be Reviewed
- The project’s identified science and mission objectives.
- The project’s scientific and technical merits, and its alignment with LaRC’s mission.
- Preliminary cost estimates and project schedule.
- Institutional support estimates.
Software Support Needed
- High-level software cost estimates.
- High-level software development schedule.
II.2 System Requirements Review
Purpose
The System Requirements Review (SRR) is performed during Phase B of the project’s life cycle to define its objectives and verify that the system requirements are sufficient. The major subsystems and requirements allocated to them should be clearly identified. Upon successfully completing this review, the system requirements are considered baselined.
Major Topics to be Reviewed
The system requirements.
- The identified major subsystems (including software) and their allocated system requirements.
- The system design approach.
- System requirements verification and validation approach.
- Project planning artifacts (e.g., work breakdown structure (WBS), and estimated cost, schedule, personnel resources).
- Project-level
Software Support Needed
- High-level software cost estimates.
- High-level software development schedule.
- Identification of high risk requirements allocated to software.
II.3 Preliminary Design Review
Purpose
The Preliminary Design Review (PDR), held at the end of Phase B, is a formal technical review of the system design, including the various subsystems (e.g., hardware configuration items (HWCI), computer software configuration items (CSCI)) and their components. The hardware and software designs should be approximately 70% complete. All system requirements should be allocated to the various subsystems, and the subsystem and component interfaces should be well-defined. The PDR confirms that:
- All system requirements are allocated to the subsystem and component levels.
- The system and subsystem designs are expected to meet all performance and functional requirements.
- The system and subsystem designs will not pose major problems (e.g., schedule delays, cost overruns).
Major Topics to be Reviewed
- System and subsystem schedules, and cost and labor estimates.
- System, subsystem, and component interfaces are completely defined.
- Required resources for developing all subsystems are available.
- System procurement plans.
- Mission operation plans.
- System development plans.
- Logistics plans.
- System reliability analysis.
- Project-level Configuration Management Plan (CMP).
- Project risk management approach.
Software Support Needed
- The MIL-STD-498 (preliminary) Software Design Document (SDD) (see Appendix E) showing the software structure and specifications of all software interfaces (at the CSCI and computer software component (CSC) levels) between other CSCIs, hardware, and users. Traceability from the design to the system requirements allocated to software should be demonstrated.
- Data storage needs for each CSCI.
- The selected software engineering and test environments.
- Software cost estimates and the development schedule for each CSCI.
- Software size estimates for each CSCI.
- MIL-STD-498 Software Development Plan(s) (SDP) (see Appendix E) (including the software life cycle selected, builds, design approach, standards selected, software engineering environment, and reuse plans).
- MIL-STD-498 Software Test Plan(s) (STP) (see Appendix E) explaining the approach to software testing.
- Identified high risk areas and open issues from formal inspections.
- Software procurement approach, including contractors selected, and commercial-off-the-shelf (COTS) hardware and software being purchased in support of the project.
Purpose
The Critical Design Review (CDR) ensures that the detailed design of each subsystem (e.g., hardware configuration items (HWCI), computer software configuration items (CSCI)) will meet the allocated system requirements, and that the project is ready to proceed with fabricating the flight system. The CDR is scheduled toward the end of Phase C when the detailed design of the hardware and software is approximately 95% complete.
Major Topics to be Reviewed
- The detailed design of each HWCI and CSCI (with all interfaces) is identified and described. All changes since the Preliminary Design Review (PDR), including fabrication and design drawings, are reviewed.
- Project status in terms of cost, schedule, and labor, with estimates to completion.
- Procurement status (e.g., contractors, materials to be purchased).
- Mission operations plan.
- Logistics planning (e.g., spares to be purchased, transportation arrangements, facilities acquired or reserved).
- Project verification and validation approach (e.g., test schedule, test plans and procedures, test organization identified).
- Project-level Configuration Management Plans (CMP).
- Project-level risk management approach.
Software Support Needed
- MIL-STD-498 (Detailed) Software Design Description (SDD) (see Appendix E) with supporting detailed-design diagrams.
- Detailed software cost estimates.
- Software size estimates for each CSCI.
- Detailed software project schedules for each CSCI.
- Results of any software simulations or prototyping results.
- Identification of software design changes since the PDR.
- Identified high risk areas and open issues from formal inspections.
Purpose
The System Acceptance Review (SAR) ensures that there is a high level of confidence that the flight system complies with the system requirements. The SAR also ensures that the flight system and its ground support equipment (GSE) will be safely transported to their destination, and that they will correctly operate upon arrival. The review is held after system integration and testing are completed in Phase D.
Major Topics to be Reviewed
- Changes in the system design since the Critical Design Review (CDR).
- Project cost (actuals and estimate-to-completion).
- System test results.
- Mission operations plan and contingency plans.
- Plans for transporting the equipment.
- Approach for data retrieval and analysis.
Software Support Required
- Software development costs (actuals and estimate-to-completion).
- Schedule for any remaining software work.
- Overview of software developed (or being developed) to support data retrieval and analysis.
- List of changes in the software design since CDR.
- System-level test results relating to software.
Purpose
The Flight Readiness Review (FRR), held at the end of Phase D, determines the overall readiness of the flight system to perform its science and mission objectives. It is held as close to the flight date as possible. The Mission Operations Review (MOR) may be held as part of this review or as a separate review.
Major Topics to be Reviewed
- The project’s science and mission objectives.
- The minimum mission success criteria.
- The public information plan.
- System-level test results.
- Significant events since the last review.
Software Support Required
- System-level test results relating to software.
- Information on software developed (or being developed) to support data retrieval and analysis.
II.7 Mission Operations Review
Purpose
The Mission Operations Review (MOR) may be held as part of the Flight Readiness Review (FRR) or as a separate subreview. The MOR determines the overall readiness of the mission operations system to support the flight system. It is held as close to the flight date as possible.
Major Topics to be Reviewed
- Planned ground support operations.
- Planned flight operations.
- Interfaces between the flight system and ground.
Software Support Needed
- Information on software-supported flight-to-ground interfaces.
- Information on the functionality of mission operations software.
- Mission operations software test results.
II.8 Operational Readiness Review (ORR)
Purpose
The Operational Readiness Review (ORR), held at the beginning of Phase E, verifies that the system’s operational support elements (including the mission operations software) are ready to support system operations.
Topics to be Reviewed
- System operating procedures.
- Contingency plans.
- Operating crews personnel, responsibilities, training.
- Safety issues.
- Operating facilities which are required.
Software Support Needed
- Software user’s guides.
- Identification of software personnel who will be involved in system operations.
- Mission operations software test results.
Purpose
The Lessons Learned Review (LLR) collects and disseminates information on experiences gained during the project. These experiences should address technical, managerial, and process issues. The review panel is provided with an overview of the lessons learned. The review normally is held soon after launch and the initialization of operations.
Major Topics to be Reviewed
- Science/mission objective lessons learned (explain how the objectives changed over the course of the project and why).
- Project planning missions lessons learned (explain how the project plan changed over the course of the project, how did the original schedule and cost estimates compare with the actuals).
- Design lessons learned (e.g., actual performance of the design, improvements that could have been made).
- Technical, public relations, and logistics lessons learned.
- Process lessons learned (e.g., effectiveness of review process, potential management and development activity improvements).
- Interorganizational lessons learned e.g., (relationships with other organizations, institutions, customers).
Software Support Needed
- Software lessons learned (both management and technical) (see activity 4.1.2 Identify Lessons Learned).