Gauss-Markov Theorem

TL;DR

Under the source’s stated assumptions (error terms normally distributed with zero mean and constant variance), OLS yields the most precise linear unbiased estimates of regression coefficients.
OLS finds estimates by minimizing the sum of squared residuals, producing unbiased coefficient estimates on average.
If the stated assumptions are violated in practice, estimates can become biased and unreliable.

Definition

The Gauss-Markov theorem is a fundamental result in statistics that states that the ordinary least squares (OLS) method of estimating regression coefficients is the best linear unbiased estimator (BLUE) among all linear unbiased estimators. Given the assumptions of normality and constant variance of the error terms, the OLS method has the smallest variance among all linear unbiased estimators.

Explanation

Context: In regression analysis the goal is to explain variation in a dependent variable (for example, income) using one or more independent variables (for example, education, experience). Regression coefficients (beta coefficients) represent the expected change in the dependent variable associated with a one unit change in an independent variable, holding other variables constant.
Error term assumptions (per source): The error terms, or residuals, are assumed to be normally distributed with zero mean and constant variance. These assumptions permit the use of statistical tests to assess coefficient significance.
OLS mechanics (per source): The OLS method estimates regression coefficients by minimizing the sum of squared residuals — the differences between observed values and values predicted by the model.
What “best” means (per source): Under the stated assumptions, OLS estimates are unbiased (on average equal to the true population coefficients) and have the smallest variance among all linear unbiased estimators, making them the most precise among that class.

Examples

Income and Education (sample of 100 individuals)

Using OLS to estimate the relationship between income and education with the linear model:

\text{Income} = \beta_0 + \beta_1 \times \text{Education}

The OLS estimates for β0 and β1 are obtained by minimizing the sum of squared residuals. The resulting estimates are unbiased on average according to the theorem.

Stock Returns and Market Risk (sample of 100 firms)

Using OLS to estimate the relationship between stock returns and market risk with the linear model:

\text{Stock returns} = \beta_0 + \beta_1 \times \text{Market risk}