The formulas for b and a from the two normal equations, and writing the regression line from a small table
Statistics Sec 3 Term 1, Unit 1, Lesson 2
GoalFind b and a from the normal equations or the formulas, and write the regression line from a small table.
New words
normal equations · Σy = na + bΣx and Σxy = aΣx + bΣx², the result of least squaresb (slope) · the regression coefficient of y on x: the change of ŷ for one more unit of xa (intercept) · the value of ŷ at x = 0
Explanation
The least squares line ŷ = a + bx is given by two equations, the normal equations: Σy = na + bΣx and Σxy = aΣx + bΣx². We take them as the result of the least squares idea and solve them for the two unknowns a and b. From the first, a = (Σy − bΣx) ÷ n. Put this into the second and multiply by n: nΣxy = ΣxΣy − b(Σx)² + b·nΣx², so b = (nΣxy − Σx·Σy) ÷ (nΣx² − (Σx)²). The top of b is the numerator of Pearson’s r, and the bottom is positive, so b and r always have the same sign. Dividing the first normal equation by n gives ȳ = a + b·x̄: the regression line always passes through the point of the two means (x̄, ȳ), which is a good check. For the throw data: Σx = 35, Σy = 60, Σxy = 472, Σx² = 285, n = 5. Then b = (5 × 472 − 35 × 60) ÷ (5 × 285 − 35²) = 260 ÷ 200 = 1.3 and a = (60 − 1.3 × 35) ÷ 5 = 2.9. The line is ŷ = 2.9 + 1.3x: each extra week of training adds about 1.3 m to the predicted throw.
Weeks x
Throw y (m)
xy
x²
y²
3
8
24
9
64
5
8
40
25
64
7
11
77
49
121
9
16
144
81
256
11
17
187
121
289
Σx = 35
Σy = 60
Σxy = 472
Σx² = 285
Σy² = 794
Worked examples
By elimination. The normal equations are 60 = 5a + 35b and 472 = 35a + 285b. Multiply the first by 7: 420 = 35a + 245b. Subtract it from the second: 52 = 40b, so b = 1.3. Then 5a = 60 − 45.5 = 14.5, so a = 2.9.
Checks and use. The means are x̄ = 35 ÷ 5 = 7 and ȳ = 60 ÷ 5 = 12, and 2.9 + 1.3 × 7 = 12, so the line passes through (7, 12). A pupil trained 8 weeks: ŷ = 2.9 + 1.3 × 8 = 13.3 m. At x = 9 the pupil threw 16 m and the line gives 14.6 m, so the error |16 − 14.6| is 1.4 m.
Meaning of a and b. b = 1.3 m per week is the slope: it carries the units “metres per week”, unlike r, which has no unit. a = 2.9 m is the value of the line at x = 0 weeks; it need not be a real throw, because x = 0 is outside the weeks we measured (3 to 11).
Class activity
Take the four pairs (1, 2), (2, 3), (3, 3), (4, 5). Find the sums, then b and a, then check that your line passes through the point of the means. Draw the dots and the line on squared paper. (Answer: ŷ = 1 + 0.9x.)
Practice
1
For the pairs (1, 4), (2, 4), (3, 6), (4, 8) find the regression line of y on x.
Σx = 10, Σy = 22, Σxy = 62, Σx² = 30. b = (4 × 62 − 10 × 22) ÷ (4 × 30 − 100) = 28 ÷ 20 = 1.4; a = (22 − 14) ÷ 4 = 2. The line is ŷ = 2 + 1.4x.
2
Check that this line passes through the point of the means, and predict y at x = 5.
x̄ = 2.5, ȳ = 5.5 and 2 + 1.4 × 2.5 = 5.5. At x = 5: ŷ = 2 + 7 = 9.
3
Used motorbikes: age x = 1, 3, 5, 9, 12 years and price y = 18, 13, 13, 10, 8 thousand LE. Find the line.
Σx = 30, Σy = 62, Σxy = 308, Σx² = 260. b = (1540 − 1860) ÷ (1300 − 900) = −320 ÷ 400 = −0.8; a = (62 + 24) ÷ 5 = 17.2. The line is ŷ = 17.2 − 0.8x.
4
What does b = −0.8 mean for the motorbikes?
Each extra year of age lowers the predicted price by 0.8 thousand LE, that is 800 LE.
5
The means of a data set are x̄ = 6 and ȳ = 20, and b = 3. Find a.
The line passes through (6, 20): a = ȳ − b·x̄ = 20 − 18 = 2.
