Overview
This activity describes general plans for the testing, including test levels, general test conditions, test progression, data recording and analysis, and the total scope of the planned testing.
Roles and Responsibilities
The software test manager completes the Software Test Plan (STP) (see Appendix E) sections defining the software test approach.
Controls
See parent activity 1.1.3 Plan Software Testing.
Inputs
The software requirements as documented in the Software Requirements Specification (SRS).
The software test environment as determined in activity 1.1.3.1 Identify Software Test Environment.
Procedures
The most difficult testing issue is deciding how much testing to do (i.e., how much is enough). Although the answer to this question should not depend on cost or budget constraints, it usually does. This activity defines the approach to testing, and the main testing parameters or boundaries. The software test manager iteratively performs the following steps:
1) Reads the Software Development Plan (SDP) (see Appendix E) to gain an understanding of how the software development effort is expected to proceed.
2) Identifies the test levels (e.g., computer software configuration item (CSCI) level, system level) and the classes of tests that will be performed (e.g., timing tests for real-time flight software, erroneous input tests for command queues in flight software, erroneous user input tests for ground support software, maximum capacity tests for flight data recording software). This information is documented in the Test Levels and Test Classes section of the STP.
3) Defines the general test conditions. These conditions apply to all tests or a group of tests. This is best explained with examples.
Example:
Each test for the command queues for the flight software shall include command parameter values with nominal, maximum, and minimum values.
Each test for the ground support, real-time displays shall use live data.
Execution timing shall be done for each CSCI in the flight software.
The STP Data Item Description (DID) requires a statement describing the extent of testing to be performed and the rationale for the extent selected. This also is best explained with an example:
Example:
The testing of the command queues for the flight software shall include 100% of the available commands. Of those commands, each command which has an associated parameter shall be tested with nominal, maximum, and minimum values for the parameter. Because any command can be sent from ground at any time during the flight, the software must be able to appropriately handle it.
In addition, the general approach for regression testing is identified. All of the information described in this step is documented in the General Test Conditions section of the STP.
In progressive or cumulative tests, which are common for flight systems, this sequence or progression is documented in the Test Progression section of the STP.
4) Considers data recording, reduction, and analysis procedures for the testing. This includes: automated data recording, post-processing software, and software analysis packages which may manipulate the data into a form appropriate for evaluation. Decisions regarding data recording and processing are documented in the Data Recording, Reduction, and Analysis section of the STP.
5) Makes an entry in the Planned Tests section of the STP for each item (i.e., CSCI, subsystem, system) to be tested. A complete specification of the item, including name and a project-unique identifier, is defined. The format specified by the STP DID is followed to include information about the test, such as test objective, qualification methods, special requirements (e.g., the flight software must be run for 90 minutes to test a complete orbit), type of data, etc.
Outputs
The Test Identification section of the STP.