How do you find the sum of the squared residuals? You calculate the difference between each observed value and its predicted value, square each difference, and add the results. In regression analysis, this measure is commonly called the sum of squared errors (SSE), sum of squared residuals (SSR), or residual sum of squares (RSS). The calculation shows how much variation in the dependent variable remains unexplained by the model.
Understanding Residuals
A residual is the vertical distance between an observed data point and the value predicted by a regression model. For observation (i):
[ e_i = y_i - \hat{y}_i ]
where:
- (y_i) is the observed or actual value;
- (\hat{y}_i) is the predicted value;
- (e_i) is the residual.
The predicted value comes from the regression equation. For a simple linear regression:
[ \hat{y}_i = b_0 + b_1x_i ]
where (b_0) is the intercept and (b_1) is the slope.
A positive residual means the model predicted a value lower than the actual observation. A negative residual means the model predicted a value higher than the actual observation.
Why Are the Residuals Squared?
Residuals can be positive or negative. If they were added directly, positive and negative errors could cancel each other, producing an artificially small result. Squaring prevents this cancellation and ensures that every error contributes positively to the total That's the whole idea..
The sum of the squared residuals is:
[ \boxed{SSE = \sum_{i=1}^{n}(y_i-\hat{y}_i)^2} ]
A smaller SSE generally indicates that the model’s predictions are closer to the observed values. An SSE of zero means the model fits every observation
Continuing the discussion, an SSE of zero would imply that every observed data point lies exactly on the regression line—an ideal scenario that rarely occurs in practice because real‑world relationships are seldom perfectly linear. In most situations the magnitude of each residual varies, so the summed squared deviations provide a quantitative gauge of how far the observations deviate from the fitted model.
In ordinary least‑squares (OLS) regression the goal is precisely to minimise this quantity. By solving the normal equations one obtains the slope (b_1) and intercept (b_0) that make (\sum_{i=1}^{n}(y_i-\hat y_i)^2) as small as possible. This optimisation step transforms the abstract notion of “unexplained variation” into concrete numerical values that can be compared across models or datasets.
A closely related decomposition splits the total variability in the dependent variable, denoted (TSS), into two parts:
[ TSS = \underbrace{\sum_{i=1}^{n}(y_i-\bar y)^2}{\text{total sum of squares}} = \underbrace{\sum{i=1}^{n}(\hat y_i-y_i)^2}{\text{residual sum of squares (RSS)}} + \underbrace{\sum{i=1}^{n} w_i e_i}^2}_{\text{explained sum of squares (ESS)}} ]
where (\bar y) is the mean of the observations and (w_i) represents weights (commonly all equal to 1). The ratio
[ R^{2}=1-\frac{RSS}{TSS} ]
captures the proportion of variability that the regression explains; values close to 1 indicate a strong linear association between (x) and (y).
Because the RSS serves as the objective function in the OLS algorithm, its computational efficiency stems from the fact that differentiation under the summation sign reduces to simple algebraic operations. Once the coefficients are estimated, the resulting RSS guides diagnostic checks such as:
- Outlier detection: Large residuals often signal points that lie far from the fitted curve;
- Model adequacy: A rapidly increasing RSS when new data are added may reveal heteroscedasticity or non‑linear patterns;
- Statistical inference: The standard error of any coefficient is derived from (RSS); larger RSS typically inflates these errors, weakening confidence intervals.
When conducting formal hypothesis tests—such as the overall F‑test for the model’s significance or t‑tests for individual predictors—the magnitude of the RSS appears explicitly in the test statistics. To give you an idea, the F‑statistic compares ((RSS_{\text{full}}-RSS_{\text{reduced}})/\text{df}_{\text{red}}) against an (\chi^2) distribution with degrees of freedom dictated by the reduction in parameters Easy to understand, harder to ignore..
It is also worth noting that while the RSS measures vertical dispersion around the line of best fit, other metrics exist (e.Plus, g. In real terms, , mean absolute deviation, median absolute error) that focus on scale invariance or robustness to extreme observations. Nonetheless, the SSE remains the cornerstone of classical linear modelling because it directly quantifies prediction error and underpins both the estimation procedure and subsequent statistical inference.
Simply put, computing the sum of squared residuals provides a clear, numeric summary of how
well a linear model captures the structure present in the data. And by translating the abstract concept of unexplained variation into a single, interpretable quantity, the RSS enables practitioners to compare competing specifications, diagnose potential deficiencies, and quantify the reliability of estimated relationships. Its central role in ordinary least squares estimation ensures that it remains an indispensable tool in the statistician’s toolkit That's the whole idea..
