GoalWrite a test plan, test a program against it, and evaluate the result.
New words
test plan · a table of tests: the data, the expected outcome and, after testing, the actual outcomeexpected outcome · what the program should do, worked out by hand before the testevaluate · to judge how well a program does its job and decide what to improve
Explanation
A test plan makes testing fair and complete. First, work out the expected outcome for each test by hand, from the rules. Never copy it from the program’s output. Then run the program and write the actual outcome. If the two match, the test passes. If not, it fails, and the program or the plan must be corrected. After testing you evaluate. Does the program do everything it was meant to do? Which tests failed and why? What could be improved? For example, it might need to reject words and negative numbers. A good evaluation gives reasons, such as “It passed all boundary tests but crashed on a word, so it needs a check on the input.”
Worked examples
Python
1km = int(input("Kilometres? "))
2if km <= 3:
3 fare = 14elif km <= 10:
5 fare = 26else:
7 fare = 38print("Fare:", fare)
Output
Fare: 2
A bus fare program. The fare is 1 for up to 3 km, 2 for up to 10 km, and 3 for longer journeys. The flowchart in the picture above shows the same three branches. This is the program in Python, with 10 typed.
Type
Test data
Expected outcome
Actual outcome
Pass/fail
normal
2
Fare: 1
Fare: 1
pass
boundary
3
Fare: 1
Fare: 1
pass
boundary
4
Fare: 2
Fare: 2
pass
normal
7
Fare: 2
Fare: 2
pass
boundary
10
Fare: 2
Fare: 2
pass
boundary
11
Fare: 3
Fare: 3
pass
normal
25
Fare: 3
Fare: 3
pass
invalid
ten
a message and no crash
ValueError: invalid literal for int() with base 10: 'ten'
fail
The test plan. Each expected outcome came from the fare rules, not from the program. Boundary values 3, 4, 10 and 11 are on both sides of the edges. All the number tests passed, but the word ten made the program stop with an error.
Question
Answer
Did every normal test pass?
yes
Did every boundary test pass?
yes
Did the invalid test pass?
no, it crashed
What should change?
check the input before using it
The evaluation. The program does what it should for every whole number tested, so it meets its main purpose. It fails for invalid data: it stops with an error for a word, and it would give a fare of 1 for −2 kilometres, which makes no sense. Improvement: add validation so that only whole numbers from 1 upwards are accepted.
Class activity
Plan a test (no computer). A school canteen charges 1 for a snack when you buy up to 2, and 3 for 3 or more. Write the rules on a card. Draw a test plan table with the five columns, and fill in the first three columns: type, test data and expected outcome. Include normal, boundary and invalid data. Swap with a partner, then work through their plan by hand with the rules and fill in the actual outcome. Write two sentences to evaluate the rules: what works and what to improve. If you have a computer or tablet, ask an adult to help you run your tests.
Practice
1
Fill in the expected outcome column for the bus fare program. Work it out from the rules.
Type
Test data
Expected outcome
normal
1
?
normal
5
?
normal
12
?
1 km gives Fare: 1. 5 km gives Fare: 2. 12 km gives Fare: 3.
2
A different version uses km < 3 for the first branch. Which test in the plan would fail and what would the program print instead?
Python
1km = int(input("Kilometres? "))
2if km < 3:
3 fare = 14elif km <= 10:
5 fare = 26else:
7 fare = 38print("Fare:", fare)
The boundary test with 3 fails. The expected outcome is Fare: 1, but the program prints Fare: 2, because 3 < 3 is false and 3 <= 10 is true.
3
Write one sentence to evaluate the bus fare program, with a reason.
For example: it passes all tests for whole numbers, including the boundaries, but it crashes if a word is typed, so it needs a check on the input.
4
Why must the expected outcome be worked out before running the program?
If it were copied from the output, a mistake in the program would look correct. Working it out from the rules first means the test can show that the program is wrong.