Week 4 · Regression: predicting y from x, simple linear regression y = a + bx and the least squares method · 3 lessons
1
From correlation to prediction: what regression is and its three kinds (simple linear, multiple and non-linear)
Statistics Sec 3 Term 1, Unit 1, Lesson 2
GoalSay what regression is, name its three kinds, and see why we study only the simple linear one here.
New words
regression · finding a rule that estimates one variable from anotherdependent and independent variable · y is predicted (dependent); x is used to predict it (independent)regression line · the straight line ŷ = a + bx used for prediction
Explanation
Correlation says how closely two variables go together. Often we want more: to predict. How many marks can a pupil expect after four hours of revision? Regression finds a rule, an equation, that estimates one variable from the other. The variable we predict is the dependent variable y; the one we use is the independent variable x. There are three kinds. Simple linear regression uses one x and a straight line, ŷ = a + bx. Multiple regression predicts y from several variables together (the mark from revision hours and sleep hours). Non-linear regression uses a curve (quadratic, exponential and so on) when the dots bend. In this course we study only the simple linear kind, which fits well when the dots form a narrow strip, that is when |r| is near 1. The picture shows eight pupils and a line that fits them well; we shall learn how to find such a line in the next two lessons. Read it like this: from x = 4 hours go up to the line and across to y, about 13.7 marks. Take care with the direction: this line predicts y from x. To predict x from y we need another line.
Worked examples
Reading predictions. For the line in the picture, ŷ = 5.8525 + 1.9672x. A pupil who revised 4 hours is predicted to score 5.8525 + 1.9672 × 4 = 13.72 marks, and one who revised 2.5 hours, 10.77 marks. These are estimates: real pupils scatter around the line.
Three kinds, three problems. (a) The marks of a pupil from revision hours alone: simple linear. (b) The marks from revision hours and sleep hours: multiple. (c) The height of a bean plant over 30 days, when the dots bend upward like a curve: non-linear.
Direction matters. For the same eight pupils, the line that predicts the mark from the hours is line A (y = 5.8525 + 1.9672x). The line that predicts the hours from the mark is x = −2.6197 + 0.4787y; drawn as y against x it is line B (y = 5.4722 + 2.0889x), a different line. For a mark of 14, line B’s equation gives x = 4.08 hours, while line A at x = 4.08 gives 13.88 marks, not 14.
Class activity
Think of a quantity you would like to predict (the time to walk to school, the cups of tea in a day). Name the variable you would use to predict it, say which is x and which is y, and say whether one variable or several would be needed.
Practice
1
A bakery predicts the number of loaves sold in a day from the temperature only. Which kind of regression is it?
Simple linear regression (one x, a straight line).
2
The dots of the height of a plant against its age bend upward steeply. Is a straight line a good model?
No. A curve (non-linear regression) would fit better.
3
A school predicts a pupil’s final mark from the revision hours and the attendance. What kind is this?
Multiple regression (two variables together).
4
The dots form a shapeless cloud with r = 0.05. Would you use a regression line to predict y?
No. The line would predict badly because x tells almost nothing about y.
5
When we predict the distance a pupil can throw from the weeks of training, which is the dependent variable?
The distance (y). The weeks of training are the independent variable x.
