GoalReview testing, errors, validation, trace tables and iterative development: plan tests, find and repair faults, and trace a short program.
New words
boundary data · test data on the edges of a rule: the extreme accepted values and the first rejected value on each siderun-time error · an error that stops a program while it is running, because it is asked to do something impossiblevalidation · a check that data is reasonable before the program uses ititerative development · building a program in small steps: make a little, test it, improve it and repeat
Explanation
Testing starts from the requirements. Good test data mixes normal data, extreme data (the smallest and largest accepted values), boundary data (the extremes and the first rejected value on each side) and invalid data (rejected values). A test plan lists each test with its expected output, worked out before the program is run. A syntax error stops a program from starting, a logic error lets it run with a wrong result, and a run-time error stops it part-way, for example when it asks for a position that does not exist. A validation check, such as a range, length or presence check, keeps bad data away from a risky line. A trace table records each variable, with a new row for every change. On a micro:bit, build a prototype of one requirement, test it, add the next requirement and keep a log of each version. At the end, evaluate: what went well, what did not and what you would change.
Kind of error
When it shows
Example
syntax error
before the program starts
a missing colon after if
logic error
the program finishes with a wrong answer
< used where <= is needed
run-time error
the program stops while it is running
asking for a position that is not there
Worked examples
Test
Kind of data
Age typed
Expected output
1
normal
11
Welcome
2
extreme (and boundary)
6
Welcome
3
extreme (and boundary)
16
Welcome
4
invalid (and boundary)
5
Sorry
5
invalid (and boundary)
17
Sorry
6
invalid
40
Sorry
A test plan. A roller-skating rink admits skaters aged 6 to 16 inclusive. The program prints Welcome or Sorry. The expected outputs come from the rule, before any test is run. The extreme values 6 and 16 and the first rejected values 5 and 17 are the boundary data, and a wrong comparison such as > for >= shows up there.
A run-time error and its repair. The user types oar. Without the check, the line print(word[4]) asks for the fifth letter of a three-letter word and the program stops with the error IndexError: string index out of range. The length check below keeps the risky line away from words that are too short. With canoe the program prints e, the fifth letter.
Building a micro:bit in small steps. Each press of A means one cup of water given to a plant. Version 1 only counts the presses. Version 2 adds button B, which shows a yes icon once more than 4 cups have been given, and otherwise shows the number of cups. Testing the values 4 and 5 checks the edge of the rule: after 4 presses, B shows 4, and after 5 presses it shows the yes icon.
Class activity
Fault finders (no computer). Each partner writes a rule for a program, such as ‘accepts a whole number from 2 to 9’, and four lines of Python or pseudocode that check it. Swap papers. Write a test plan with at least five tests and the expected outputs, and trace the other partner’s code for each test. Then each of you secretly changes one comparison (for example > for >=) and swaps back: which test finds the fault?
Practice
1
A canoe trip takes 3 to 8 people inclusive. Say whether each number of people is normal, extreme, invalid or boundary data. A number can be more than one kind.
Number of people
Kind of data
3
?
5
?
9
?
20
?
3 is normal, extreme and boundary, because it is the smallest accepted value; 5 is normal; 9 is invalid and boundary, because it is the first rejected value above the range; 20 is invalid.
2
Complete a trace table with the columns fuel, trips and Output. What does the program print?
Start: fuel 25, trips 0. After the first pass: fuel 19, trips 1. After the second: fuel 13, trips 2. After the third: fuel 7, trips 3. Now 7 >= 8 is false, so the loop ends. It prints 3.
3
Two micro:bits are in the same radio group. The user presses A on the sender once, and then presses B once. What does the receiver show after each press?
After A, the sender sends 3. Since 3 < 6 is true, the receiver shows the yes icon. After B, the sender sends 9. Since 9 < 6 is false, the receiver shows the number 9.
4
Write a Python program that accepts a ticket code only if it has exactly 6 characters. It prints Accepted, or Wrong length. Give one test input for normal data and two for invalid data.
Model answer: code = input("Ticket code? ") / if len(code) == 6: / [indent] print("Accepted") / else: / [indent] print("Wrong length"). Normal test: AB12CD, which gives Accepted. Invalid tests: AB12C (5 characters) and AB12CDE (7 characters), which give Wrong length.