Locally Weighted Regression

TL;DR

Fits local regression models instead of a single global line to capture non-linear relationships.
Achieves flexibility by weighting data points differently depending on their proximity or region.
Useful when the relationship between predictors and response varies across the input space.

Definition

Locally weighted regression (LWR) is a machine learning technique that fits non-linear regression models to data by weighting data points differently in a given region, allowing for more flexible model behavior.

Explanation

Rather than fitting one regression line to the entire dataset, LWR fits multiple regression lines tailored to specific regions. Data points near the region of interest receive higher weight, and points farther away receive lower weight. This localized weighting lets the model capture relationships that vary across different parts of the input space, improving the ability to model non-linear patterns.

Examples

Housing prices

LWR can model the relationship between a house’s square footage and its price by creating multiple local regression lines instead of a single global line. For example, one line may be fit to houses with square footage between 1,000 and 1,500, while another line may be fit to houses with square footage between 1,500 and 2,000. This accounts for the possibility that the square-footage–price relationship differs across those ranges.

Medical diagnosis

LWR can predict the likelihood of a patient having a disease based on symptoms and medical history by fitting separate local lines for subgroups of patients. For example, one line may be fit to patients within a certain age range and gender, while another line may be fit to a different age range and gender, accommodating subgroup-specific relationships between symptoms and disease likelihood.

Machine learning
Regression models
Non-linear regression