Linear Estimator

TL;DR

• Predicts an output (dependent) variable from one or more input (independent) variables using a linear function.
• Appears as simple linear regression for a single input and multiple linear regression for multiple inputs.
• Simple to implement and interpret, but limited to linear relationships and can be sensitive to outliers.

Definition

A linear estimator is a mathematical model used to predict the value of a dependent variable (the output) based on one or more independent variables (the inputs). It makes predictions using a linear function of the inputs. For a single input the linear function has the form:

$y = ax + b$

where a and b are constants and x is the independent variable. For multiple inputs the linear function has the form:

$y = a1x1 + a2x2 + a3x3 + ... + b$

where x1, x2, x3, etc. are the independent variables, a1, a2, a3, etc. are the coefficients, and b is the intercept.

Explanation

A linear estimator uses a weighted sum of input variables plus an intercept to produce a prediction. The coefficients (for example a or a1, a2, a3, …) determine the influence of each input on the predicted output, and the intercept b is the predicted value when all inputs are zero. Linear estimators are straightforward to understand and implement and can produce good predictions when the relationship between inputs and output is approximately linear.

Examples

Simple linear regression

A simple linear regression model predicts a continuous dependent variable (for example, the price of a house) from a single independent variable (for example, the square footage of the house). In this case the linear function is:

$y = ax + b$

where y is the predicted dependent variable, x is the independent variable, a is the slope of the regression line, and b is the intercept (the value of the dependent variable when the independent variable is 0).

Multiple linear regression

A multiple linear regression model predicts a dependent variable from multiple independent variables. For example, predicting the price of a house based on the square footage, the number of bedrooms, the number of bathrooms, and the age of the house. The linear function is:

$y = a1x1 + a2x2 + a3x3 + ... + b$

where y is the predicted dependent variable, x1, x2, x3, etc. are the independent variables, a1, a2, a3, etc. are the coefficients of the regression model, and b is the intercept.

Use cases

• Data analysis
• Machine learning
• Statistical modeling

Notes or pitfalls

• Linear estimators can only model linear relationships between dependent and independent variables; they are not suitable for non-linear relationships.
• They can be sensitive to outliers, which may affect prediction accuracy.

Related terms

• Simple linear regression
• Multiple linear regression