Linear Regression

December 8, 2025

On this page

Linear Regression: OLS, Ridge, Lasso & Elastic Net

Linear regression specifies a conditional mean that is linear in the coefficients. We estimate those coefficients by ordinary least squares (OLS), then show why the same estimate maximizes the likelihood under the Gaussian error model used here.

The normal equations also explain why coefficient estimates can be unstable: when predictors are strongly correlated, small changes in the observed responses can produce large changes in the fitted coefficients. Ridge, lasso, and elastic net address this problem by adding penalties to the fitting objective. We derive these objectives, then discuss the assumptions and diagnostic checks for the model.

Recap: Differentiating a squared residual

The vector-derivative reference supplies the calculation used below. For fixed $\mathbf X\in\mathbb R^{n\times(p+1)}$ and $\mathbf y\in\mathbb R^n$ ,

\nabla_{\boldsymbol\beta}\|\mathbf y-\mathbf X\boldsymbol\beta\|_2^2 =-2\mathbf X^\top(\mathbf y-\mathbf X\boldsymbol\beta).

The gradient is a $(p+1)$ -vector. Setting it to zero gives the normal equations; obtaining a unique coefficient vector additionally requires the rank condition stated below.

Key Regression Framework

Frequentist Approach

In the frequentist approach, we treat the parameters $\boldsymbol{\beta}$ as fixed but unknown. For the likelihood derivation in this note, we use a linear mean model with normally distributed errors:

y_i = \beta_0 + \beta_1 x_{i1} + \dots + \beta_p x_{ip} + \epsilon_i,

where the errors are independent with $\epsilon_i \sim N(0,\, \sigma^2)$ . In compact matrix notation, this becomes:

\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\epsilon},

with $\mathbf{y}$ as an $n \times 1$ response vector, $\mathbf{X}$ as an $n \times (p+1)$ design matrix, $\boldsymbol{\beta}$ as the $(p+1) \times 1$ vector of unknown coefficients, and the errors $\boldsymbol{\epsilon} \sim N(\mathbf{0}, \sigma^2 \mathbf{I})$ . We write $\mathbf{x}_i^T$ for row $i$ of the design matrix, including its intercept entry.

We keep the design $\mathbf X$ fixed throughout. With $\boldsymbol\beta$ and $\sigma^2$ also fixed, the sampling distribution is $\mathbf Y\mid\mathbf X\sim N(\mathbf X\boldsymbol\beta,\sigma^2\mathbf I)$ . Means and covariances under this distribution average over the errors, not over the coefficients or design.

After observing the responses, we write their values as lowercase $\mathbf y$ and keep them fixed while fitting the coefficients. We omit the fixed $\mathbf X$ from the likelihood's conditioning notation to keep the formulas shorter.

Estimating Parameters (OLS & MLE)

We estimate $\boldsymbol{\beta}$ by minimizing the residual sum of squares (RSS), also called the sum of squared residuals (SSR). For a candidate coefficient vector, this criterion measures the discrepancy between the observed responses and the model's predictions:

RSS(\boldsymbol{\beta}) = \sum_{i=1}^{n} (y_i - \mathbf{x}_i^T\boldsymbol{\beta})^2 = (\mathbf{y} - \mathbf{X}\boldsymbol{\beta})^T(\mathbf{y} - \mathbf{X}\boldsymbol{\beta}) = \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|_2^2.

Diagram illustrating the Residual Sum of Squares (RSS) for a linear regression

Image credit: Article by wallstreetmojo

In the plot, each data point $(x_i, y_i)$ has a corresponding predicted value $\hat{y}_i$ . The vertical lines (labeled $e_1, e_2, \dots$ ) represent the residuals, mathematically given by $e_i = y_i - \hat{y}_i$ . The Residual Sum of Squares (RSS) is the sum of the squares of these residuals, i.e., $RSS = \sum_{i=1}^{n} e_i^2.$ Minimizing the RSS via Ordinary Least Squares (OLS) gives us the estimate for $\beta$ .

Under the stated Gaussian error model, maximizing the likelihood over the coefficients at a fixed positive noise variance gives the same coefficient estimates as minimizing the RSS. This is a point-estimation approach for $\boldsymbol{\beta}$ ; a Bayesian analysis instead describes uncertainty through a posterior distribution for $\boldsymbol{\beta}$ .

Deriving OLS and MLE Estimates for Linear Regression

Model Setup

We consider $\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\epsilon}$ , where $\boldsymbol{\epsilon} \sim N(0, \sigma^2 \mathbf{I})$ .

Here, $\mathbf{X}$ is an $n \times (p+1)$ design matrix, $\mathbf{y}$ is an $n \times 1$ vector, and $\boldsymbol{\beta}$ is a $(p+1) \times 1$ vector of unknown parameters.

For the inverse formulas below, assume $\mathbf{X}$ has full column rank, so $\mathbf{X}^T\mathbf{X}$ is invertible. Without this condition, least-squares fitted values remain well defined, but the coefficient vector need not be unique. Gaussian errors are needed for the likelihood argument here, not for defining or minimizing the squared-error criterion.

1. OLS Estimation (Loss Function Approach)

Define the Loss Function

S(\boldsymbol{\beta}) = \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2 = (\mathbf{y} - \mathbf{X}\boldsymbol{\beta})^T(\mathbf{y} - \mathbf{X}\boldsymbol{\beta})

Take the Derivative

\frac{\partial S(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}} = -2\mathbf{X}^T\mathbf{y} + 2\mathbf{X}^T\mathbf{X}\boldsymbol{\beta} = 0

Solve for $\boldsymbol{\beta}$

\begin{aligned} \mathbf{X}^T\mathbf{X}\boldsymbol{\beta} &= \mathbf{X}^T\mathbf{y} \\ \hat{\boldsymbol{\beta}}_{\mathrm{OLS}} &= (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y} \end{aligned}

2. MLE Estimation (Likelihood Approach)

Likelihood Definition

Under $\boldsymbol{\epsilon} \sim N(0, \sigma^2 \mathbf{I})$ , the probability density of $\mathbf{y}$ given $\boldsymbol{\beta}$ and $\sigma^2$ is:

p(\mathbf{y} \mid \boldsymbol{\beta}, \sigma^2) = \frac{1}{(2\pi\sigma^2)^{n/2}} \exp\left(-\frac{1}{2\sigma^2} \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2\right)

Log-Likelihood

Taking the natural log:

\ell(\boldsymbol{\beta}, \sigma^2) = -\frac{n}{2}\ln(2\pi) - \frac{n}{2}\ln(\sigma^2) - \frac{1}{2\sigma^2} \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2

Maximizing with Respect to $\boldsymbol{\beta}$

Holding $\sigma^2>0$ fixed, we focus on the term

-\frac{1}{2\sigma^2}\|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2,

which is the only component dependent on $\boldsymbol{\beta}$ . Maximizing this expression is equivalent to minimizing its negative:

\frac{1}{2\sigma^2}\|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2.

Since $\frac{1}{2\sigma^2}$ is a positive constant, minimizing this term is exactly the same as minimizing

\|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2.

Thus, maximizing the term

-\frac{1}{2\sigma^2}\|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2

is equivalent to minimizing the residual sum of squares, $RSS = \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2$ . The criteria differ by constants and a positive scale factor, but they yield the same estimates for $\boldsymbol{\beta}$ under this Gaussian model. We therefore recover the OLS normal equations:

\begin{aligned} \frac{\partial}{\partial \boldsymbol{\beta}} \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2 &= -2\mathbf{X}^T\mathbf{y} + 2\mathbf{X}^T\mathbf{X}\boldsymbol{\beta} = 0 \\ \mathbf{X}^T\mathbf{X}\boldsymbol{\beta} &= \mathbf{X}^T\mathbf{y} \\ \hat{\boldsymbol{\beta}}_{\text{MLE}} &= (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y} \end{aligned}

The OLS criterion penalizes prediction errors but does not directly restrict coefficient size. When predictors are highly correlated, the resulting coefficients can be large and unstable. Regularization addresses this difficulty by constraining the coefficient vector or, through a Lagrangian formulation, adding a penalty to the objective.

The penalty determines the form of shrinkage. Ridge shrinks coefficients toward zero, whereas lasso can set some coefficients exactly to zero and thereby select variables. Elastic net combines the two penalties. These methods seek a useful bias–variance trade-off; an improvement in prediction depends on the data and the amount of regularization.

Diagram illustrating Ridge, Lasso, and Elastic Net regularization techniques for linear regression

Image credit: Article by Tavishi

Interpreting the Contours and Constrained Regions

Unconstrained Optimization (OLS Only):
Each contour joins coefficient values with the same residual sum of squares (RSS). In the two-dimensional illustration (for $\beta_1$ and $\beta_2$ ), the center of the contours is the unconstrained minimum.

Constrained Optimization (Regularization):
The constraint restricts the coefficient values we may choose. In two dimensions, the ridge constraint is circular and the lasso constraint is diamond-shaped; elastic net combines the two types of restriction. The constrained solution is the point in the allowed region with the smallest RSS. We can locate it by finding the lowest loss contour that reaches that region.

The same reasoning applies in higher dimensions: among the coefficient vectors allowed by the constraint, we choose one that minimizes the loss.

We can now express this geometry algebraically. For each penalty, we give the constrained formulation, the corresponding penalized objective, and its effect on the fitted coefficients.

We use nonnegative penalty parameters and positive constraint bounds. For simpler matrix formulas, we penalize the full coefficient vector, including the intercept. Common implementations instead leave the intercept unpenalized and standardize predictor columns. To use that convention, the intercept must be excluded from the norms and penalty matrix. A constraint bound and its corresponding penalty parameter describe related optimization problems, but they are not interchangeable numerical values.

Ridge Regression

Constrained Formulation:
$\min_{\boldsymbol{\beta}} \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2$ subject to $\|\boldsymbol{\beta}\|_2^2 \le t$ .
The associated Lagrangian gives the penalized objective below for a suitable multiplier.

Objective: Minimize

\|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2 + \lambda \|\boldsymbol{\beta}\|_2^2

Closed-form Estimate:

\hat{\boldsymbol{\beta}}_{\text{ridge}} = (\mathbf{X}^T\mathbf{X} + \lambda \mathbf{I})^{-1}\mathbf{X}^T\mathbf{y}

For $\lambda>0$ , the matrix in this formula is positive definite even if the design is rank deficient. At $\lambda=0$ , the inverse again requires full column rank.

Key Points for Ridge:

Coefficient Shrinkage: The quadratic penalty shrinks coefficients toward zero. Unlike lasso, ridge does not generally produce a sparse coefficient vector.
Handling Multicollinearity: The addition of $\lambda \mathbf{I}$ improves the conditioning of $\mathbf{X}^T\mathbf{X}$ , yielding more stable estimates in the presence of highly correlated predictors.
Prediction: Shrinkage can reduce the effect of noise on the fitted model, although the predictive benefit depends on the penalty strength and the data.

Lasso Regression

Constrained Formulation:
$\min_{\boldsymbol{\beta}} \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2$ subject to $\|\boldsymbol{\beta}\|_1 \le t$ .
The Lagrangian adds a nonnegative multiplier times the constraint function. Omitting its constant term gives the penalized objective below; the multiplier determines the penalty strength.

Objective: Minimize

\|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2 + \lambda \|\boldsymbol{\beta}\|_1

Estimation: In general, we solve the lasso objective numerically.

Key Points for Lasso:

Sparsity: The $L_1$ penalty can set coefficients exactly to zero, producing a model that uses fewer predictors.
Variable Selection: A zero fitted coefficient excludes that predictor from the fitted linear combination; it does not establish that the predictor is scientifically irrelevant.
Regularization: Lasso combines coefficient shrinkage with variable selection, with both controlled by the penalty strength.

Elastic Net Regression

Constrained Formulation:
$\min_{\boldsymbol{\beta}} \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2$ subject to $\|\boldsymbol{\beta}\|_1 \le t_1$ and $\|\boldsymbol{\beta}\|_2^2 \le t_2.$

Formulation Approach:
Elastic Net integrates constraints on both the $L_1$ and $L_2$ norms via the Lagrangian method, leading to a combined penalized objective.

Objective: Solve

\min_{\boldsymbol{\beta}} \; \|\mathbf{y} - \mathbf{X}\boldsymbol{\beta}\|^2 + \lambda_1 \|\boldsymbol{\beta}\|_1 + \lambda_2 \|\boldsymbol{\beta}\|_2^2

Key Points for Elastic Net:

Combined Penalties: Elastic net combines ridge shrinkage ( $L_2$ ) with the sparsity-inducing lasso penalty ( $L_1$ ).
Group Selection: Particularly useful when predictors are highly correlated, as it tends to select groups of related variables.
Tuning: The two penalty parameters control the strength of each type of regularization.

Model Assumptions & How to Check Them

Linearity

Assumption: The conditional mean of the response is linear in the included model terms. Those terms may include transformations or interactions of the original predictors.

How to Check: Plot residuals against fitted values and relevant predictors. Curvature suggests a missing pattern in the mean model. Component-plus-residual plots can help assess individual terms, but an unremarkable plot does not prove the model is correct.

Independence of Errors

Assumption: The model assumes independent errors across observations. Errors are unobserved model quantities; fitted residuals are estimates of them and are not generally independent, even when the errors are.

How to Check: Begin with the study design: repeated measures, clusters, and time ordering can create dependence. For meaningfully ordered observations, residual plots and suitable autocorrelation diagnostics can reveal structure. The Durbin–Watson test is not a generic check for arbitrarily ordered cross-sectional data.

Homoscedasticity

Assumption: The variance of the errors should be constant across all levels of the independent variables.

How to Check: Plot the residuals versus the fitted values. A random scatter with a constant spread (and no funnel shape) suggests homoscedasticity. Formal tests such as the Breusch-Pagan or White’s test can also be used to assess constant variance.

Normality of Errors

Assumption: Under the Gaussian model, the errors are normally distributed. Together with the other model conditions, this supports exact classical finite-sample tests and confidence intervals; normality is not required to compute OLS estimates.

How to Check: Compare standardized residuals with normal quantiles in a Q-Q plot. Approximate alignment with the reference line is consistent with normality, while systematic departures deserve investigation. Formal tests can supplement this assessment but do not replace consideration of sample size and the inferential question.

No Multicollinearity

Assumption: Unique OLS coefficients require no exact linear dependence among the design columns. Strong but imperfect dependence does not prevent estimation, although it can make individual coefficients imprecise and unstable.

How to Check: Evaluate the correlation matrix of the independent variables and calculate Variance Inflation Factors (VIF). High correlations or VIF values above 5 (or 10, depending on the context) indicate potential multicollinearity issues.

Correct Model Specification

Assumption: For the stated linear conditional-mean model, the error has mean zero conditional on the included predictors. Omitting a variable is a problem when it makes that mean specification inappropriate; adding variables does not automatically remove bias or establish causality.

How to Check: Combine subject-matter reasoning with residual patterns and comparisons of plausible specifications. Information criteria can compare candidate models, but neither a selection procedure nor a residual plot verifies that all relevant structure has been included.

Regression Diagnostics

In R, applying plot() to the object returned by lm() produces four standard diagnostic plots. Each examines a different aspect of the fit, so we interpret them together with the study design rather than as a single pass-or-fail test.

Four diagnostic plots for a fitted linear regression

Residuals vs Fitted Plot

This plot compares residuals with fitted values. Under an adequate linear conditional-mean model, we look for residuals centered near zero without a systematic curve. A visible pattern suggests that the specified mean function has missed part of the relationship.

Assumption Checked: Linearity. A random scatter supports the linearity assumption, whereas a curved or patterned distribution suggests that a non-linear relationship might exist and that the model may need additional terms.

Normal Q-Q Plot

This plot compares standardized residuals with normal quantiles. Approximate alignment with the reference line supports the Gaussian error approximation.

Assumption Checked: Normality of errors. Systematic departures may affect inference that relies on that assumption.

Scale-Location Plot

This plot shows the square root of the absolute standardized residuals against fitted values. A roughly horizontal trend supports constant error variance. The absolute value is essential because residuals can be negative; see the R documentation.

Assumption Checked: Homoscedasticity. A horizontal band with no clear pattern supports the constant variance assumption; any funnel shape or systematic pattern suggests heteroscedasticity.

Residuals vs Leverage Plot

This plot compares standardized residuals with leverage, which measures an observation's position in predictor space. High leverage alone does not establish influence; combining it with a large residual can identify observations worth examining.

Purpose: Assess potential influence on the fitted model, not independence of the errors. Independence must be considered through the design and relevant dependence diagnostics.

The derivations connect three choices: how we measure fit, how we constrain coefficients, and how we assess the resulting model. OLS minimizes squared residuals, while ridge, lasso, and elastic net modify that objective to control coefficient size or select variables. Diagnostic plots then help us assess whether the fitted relationship is adequate for the question at hand; regularization alone does not establish that the model's assumptions hold.