Week 9 · Unit 7.2: Decomposing problems · 3 lessons
3
Test it and trace it
Computing 7: Unit 7.2, pp. 44–81
GoalWrite and complete a test plan with good test data, and use it to find a bug.
New words
test plan · a table of tests that says what to try and what should happentest data · the values used in a testexpected outcome · what should happen, worked out from the plan before the testactual outcome · what really happens when the test is done
Explanation
Testing shows whether a program does what the plan says. A test plan is a table with the test number, the test data, the expected outcome, the actual outcome and a pass or fail. The expected outcome is worked out from the flowchart before the test, never copied from the program. The actual outcome comes from running the program or tracing its code. If they are the same, the test passes. If not, it fails, and you debug. Choose test data carefully: a low value, a high value and values around the threshold, for example one below, exactly on it and one above. Tracing a flowchart with a value is the quickest way to find the expected outcome.
Column
What you write
Test
the number of the test
Test data
the value you try
Expected outcome
what the flowchart says should happen
Actual outcome
what really happens
Pass/fail
pass if the two outcomes match, fail if not
Worked examples
Test data
Why it is chosen
0
the lowest possible value
79
just below the threshold
80
exactly on the threshold
81
just above the threshold
255
the highest possible value
Here are the five tests Ravi chooses for his plant shelf (a cross if light_level < 80). Each value has a job.
Test
Test data
Expected outcome
Actual outcome
Pass/fail
1
0
cross
cross
pass
2
79
cross
cross
pass
3
80
tick
tick
pass
4
81
tick
tick
pass
5
255
tick
tick
pass
Ravi traces the flowchart for each value and writes the expected outcomes before running the program. The actual outcomes match, so every test passes.
Test
Test data
Expected outcome
Actual outcome
Pass/fail
1
0
cross
cross
pass
2
79
cross
cross
pass
3
80
tick
cross
fail
4
81
tick
tick
pass
Now a mistake: the program has if light_level <= 80 instead of < 80. Test 3 fails, because the program shows a cross at exactly 80. The other tests do not show the bug. This is why the value on the threshold is the most important test.
Class activity
Be the tester (no computer). Your partner draws a flowchart for a fish-tank sign: “temperature > 28? yes: cross, no: tick”. You write a test plan with five tests, filling in the expected outcomes first. Your partner then secretly changes the rule to “temperature >= 28”, and reads out the actual outcomes for your five test values. Which test fails? What does it tell you about the bug?
Practice
1
Complete the expected outcomes (tick or cross) for the temperatures 10, 25, 26, 27 and 40 using this flowchart.
A rule says: show a tick if light_level > 150. Choose three test values that check the boundary and say what each should give.
Good choices are 149 or 150 (cross), and 151 (tick). 150 matters most: it is exactly on the threshold, and > does not include it, so the outcome is a cross. Values such as 0 or 255 are also useful.
3
The plan says: tick if the light level is 200 or more. Look at the results. Which test fails, and how should the program be corrected?
Test
Test data
Expected outcome
Actual outcome
Pass/fail
1
255
tick
tick
?
2
200
tick
cross
?
3
199
cross
cross
?
4
0
cross
cross
?
Test 2 fails: at 200 a tick is expected but a cross appears. The program must use >= 200 instead of > 200.
4
Why must the expected outcome be worked out before the program is run?
If you copy the program’s result, you would always say it is correct, even if it is wrong. Working it out from the plan first makes the test fair.