GoalWrite a test plan with expected output worked out before the test, fill in the actual output, and find a fault from the results.
New words
test plan · a table that lists the test data to use and the output expected for each testexpected output · what the program should do, worked out from the requirements before testingactual output · what the program really does when it is run with the test data
Explanation
A test plan decides what to test before you start. Each row shows the kind of data, the input and the expected output. Work out the expected output from the requirements, not from the program. If you run the program first and copy its answer, a wrong program will seem right. Then run the program, or trace it by hand, and write down the actual output. If the actual output is the same as the expected output, the test passes. If not, the test fails and the program has an error. The plan should include normal, invalid and, when there is a range, extreme data. A plan with many tests of the same kind is weaker than one with a mix.
Read the requirements→Choose the test data→Write the expected output→Run and note the actual output→Compare: pass or fail
Worked examples
Test
Type of data
Input
Expected output
Actual output
Pass / fail
1
Normal
12
Locker found
2
Normal
30
Locker found
3
Extreme
1
Locker found
4
Extreme
36
Locker found
5
Invalid
0
No such locker
6
Invalid
37
No such locker
7
Invalid
−5
No such locker
A test plan for the locker program. The requirements say that numbers 1 to 36 are accepted and all other numbers are rejected. The plan has normal, extreme and invalid data. The expected outputs are worked out from these requirements. The last two columns are empty, because nothing has been run yet.
Test
Type of data
Input
Expected output
Actual output
Pass / fail
1
Normal
12
Locker found
Locker found
pass
2
Normal
30
Locker found
Locker found
pass
3
Extreme
1
Locker found
No such locker
FAIL
4
Extreme
36
Locker found
Locker found
pass
5
Invalid
0
No such locker
No such locker
pass
6
Invalid
37
No such locker
No such locker
pass
The plan after testing a faulty version, in which the first condition is locker > 1 instead of locker >= 1. Normal data and the high extreme pass. Only test 3, the extreme value 1, fails, because the program prints No such locker. Without the extreme data, this error could be missed, because all the normal tests pass.
Reading the code that was tested. This is the faulty version. The condition locker > 1 is false when the user types 1, so the program takes the else branch. The error is a logic error: the program runs without any message but gives the wrong answer. The fix is to change > to >=.
Class activity
Plan and test (no computer). Take the sticker rule in the first practice task (1 to 5 stickers are allowed; the program prints OK or Not allowed). Write the program on paper first. Then make a test plan with seven tests: two normal, the two extreme values, and three invalid. Write the expected outputs before you look at the program. Trace your program by hand for each test and fill in the actual output and pass / fail. Ask a partner to change one comparison in the program secretly, for example < for <=, and trace again to see which test now fails.
Practice
1
The requirement is: a pupil may choose from 1 to 5 stickers, and the program prints OK or Not allowed. Complete the expected output for each test.
Test
Type of data
Input
Expected output
1
Normal
3
?
2
Extreme
1
?
3
Extreme
5
?
4
Invalid
0
?
5
Invalid
6
?
Test 1 (3): OK. Test 2 (1): OK. Test 3 (5): OK. Test 4 (0): Not allowed. Test 5 (6): Not allowed. The expected outputs come from the requirement, which allows 1 to 5.
2
A learner writes the program shown, with the upper condition stickers < 5. What is the actual output for each test input 3, 1, 5, 0 and 6? Which test fails? What kind of error is it?
The actual outputs are OK, OK, Not allowed, Not allowed and Not allowed. The test with input 5 fails, because the expected output is OK. This is a logic error: the comparison should be <= 5. The failing test is an extreme value.
3
A program saves a score if it is from 0 to 40 inclusive, and prints Score saved. Otherwise it prints Not a valid score. Write a test plan with at least two normal tests, both extreme values and two invalid tests.
Normal: 20 gives Score saved, and 33 gives Score saved. Extreme: 0 gives Score saved, and 40 gives Score saved. Invalid: −1 gives Not a valid score, and 41 gives Not a valid score. The extreme and invalid values next to the edge are the most useful.
4
Why must you work out the expected output before you run the program?
If you run the program first and copy its output, a program with an error will give an answer that you accept as correct. The expected output must come from the requirements, so that wrong behaviour shows up as a failed test.