Lenth's method

TL;DR

• Compare each predictor’s explanatory power by fitting the full multiple regression and then single-predictor regressions.
• Compute each predictor’s relative efficiency as the ratio of the full-model RSS to the RSS from the single-predictor model.
• A larger relative efficiency indicates a more important predictor in the model.

Definition

Lenth’s method is a statistical method used to determine the relative importance of each predictor variable in a multiple regression analysis. It calculates the relative efficiency of each predictor variable as a measure of how well that predictor explains variance in the response variable.

Explanation

1. Fit a multiple regression model using all predictor variables and compute the residual sum of squares (RSS) for the full model, RSS_full.
2. For each predictor variable, fit a separate regression model containing only that predictor and calculate its RSS, denoted RSS_i.
3. Compute the relative efficiency for predictor i as the ratio of the full-model RSS to the single-predictor RSS:

$$\text{Relative Efficiency}_i = \frac{RSS_{\text{full}}}{RSS_i}$$

A higher relative efficiency indicates that the predictor explains more of the variance in the response variable and is therefore more important relative to others.

Examples

Simple numeric example

Suppose a multiple regression model has three predictor variables X1, X2, and X3. The full model has an RSS of 500, and the models with only X1, X2, and X3 as predictors have RSS values of 400, 300, and 200, respectively. The relative efficiencies are:

• X1: 500/400 = 1.25
• X2: 500/300 = 1.67
• X3: 500/200 = 2.5

This reveals that X3 is the most important predictor (highest relative efficiency) and X1 is the least important (lowest relative efficiency).

Predicting chicken weight

A researcher fits a multiple regression model to predict chicken weight from age, diet, and amount of exercise. Lenth’s method yields relative efficiencies: age = 1.8, diet = 1.5, exercise = 1.2. This indicates age is the primary factor, with diet and exercise having lesser impacts.

Marketing campaign example

A researcher models marketing success with predictors advertising spend, target audience, and product type. Lenth’s method shows relative efficiencies: advertising spend = 2.5, target audience = 1.7, product type = 1.3. This indicates advertising spend is the most influential predictor, with target audience and product type being less influential.