Why this matters
Regression is how a pattern in past data becomes a prediction about tomorrow. It is also the single easiest place in this course to fool yourself, because an equation that fits the data beautifully can still be telling you nothing about cause.
The idea
Regression analysis writes the relationship between variables as an equation and uses it to predict. The response variable is the result you want (sales); the explanatory variable is what you predict it from (temperature). The equation is the model, a simplified picture of a real relationship.
Simple regression uses one explanatory variable, giving a regression line
y = ax + b. The least-squares method picks a and b by minimising the sum of
the squared residuals, each residual being the gap between an actual value and the
line.
Multiple regression uses several, because real results rarely have one cause. House
prices depend on size, age and distance from a station together, giving
y = a1x1 + a2x2 + b and so on, with each coefficient showing that variable's
influence.
| Aspect | Simple | Multiple |
|---|---|---|
| Explanatory variables | One | Two or more |
| Character | Simple, easy to interpret | Expresses complex relationships |
| Suits | Capturing a relationship between two variables | A response driven by several factors |
Three indicators judge a model:
| Indicator | What it is | Reading it |
|---|---|---|
| R squared | How much of the variability the model explains, on 0 to 1 | Closer to 1 is better |
| RMSE | Root of the mean squared error | Smaller is better; in the response's own units; punishes big misses |
| MAE | Mean of the absolute errors | Smaller is better; less swayed by outliers |
RMSE and MAE both measure error size, but RMSE weights large discrepancies more heavily while MAE averages them evenly. Which you want depends on whether a few big misses matter more than many small ones.
Four cautions. Correlation is not causation, and a well-fitting equation does not establish it; causation needs the cause to precede the effect, other factors ruled out, and a reason why. A model is not finished when built: check its accuracy, and do not use it outside the range of data it was fitted to. Do not judge on R squared alone. And adding explanatory variables does not automatically improve a model.
Picture it
flowchart TD
D[Past data] --> F["Fit a model: least squares minimises the residuals"]
F --> E{"Evaluate: R squared, RMSE, MAE"}
E -- poor --> F
E -- good --> U["Use it to predict, within the data's range"]
U --> C["Still not proof of causation"]Worked example
Ice cream sales correlate strongly with drowning deaths. R squared is high. Should beaches ban ice cream?
The model is not wrong about the data. Those two numbers really do rise and fall together, and the fit really is good. What the model cannot tell you is why, because regression only ever describes how variables move together.
Here a third variable, temperature, drives both: hot days bring more ice cream sales and more people swimming. Remove temperature's influence and the relationship between ice cream and drownings collapses. That is why "exclusion of other factors' influence" is one of the requirements for claiming causation, alongside the cause coming first in time and there being a theoretical reason.
It also shows why a high R squared is not a safety check. It tells you the model explains the variability in this data. It says nothing about whether the explanatory variable is doing the explaining or merely riding along with something that is.
The practical rule: use regression to predict, and be far more careful before using it to claim a cause.
Your turn
Try the practice questions for this lesson. Judge statements about regression and its indicators, match each indicator to its definition, and identify what separates simple from multiple regression.
Take it further
Your canteen wants to predict daily cold-drink sales so it can order the right amount. Should it trust one obvious cause, or model it and test the model?
- Set up the model. Name the response variable, then the explanatory variables you would try, such as temperature, students present, and whether it is an exam day. Say whether this is simple or multiple regression and why.
- Evaluate it. Say which indicators you would report, and what a high R squared with a large RMSE would tell you.
- Decide. Recommend whether the canteen should order from the model's prediction alone, and name one thing the model must not be used to claim.
Temperature and sales moving together does not establish that temperature causes the sales, and the canteen only needs the prediction anyway.
Recap
- Regression predicts a response variable from one or more explanatory variables.
- Simple regression fits
y = ax + bby least squares; multiple regression uses several variables. - R squared measures explained variability; RMSE punishes large errors, MAE averages them.
- A good fit is never proof of causation, and R squared alone is never the whole judgment.