Local Weighted Regression

Assume that we have an independent variable $X$ and a dependent variable $Y$. The local weighted regression consists in finding some nearby data points around a specific data point $A(x, y)$, and computing a weighted least squares regression using those data points.

Having that regression, the predicted $\hat{y}$ is estimated. This method do the same with all data points in the dataset. Then, all predicted $\hat{y}$ values are connected to form a local weighted curve.

To assign a weight to each data point, a kernel function is used. It gives the lasgest weight to the closest observations and reduces the weight as the distance grows. Different kernel functions can be used. The tricube function is one of them:

$$K(z_i) = (1 - |z_i|^3)^3 \quad \text{if } |z_i| < 1$$

$$K(z_i) = 0 \quad \text{if } |z_i| \geq 1$$

Where:

$$ z_i = \frac{X_i - X_{\text{focal point}}}{h} $$

And $h$ is the width of the window.

A cost function for this method is given by:

$$ \mathscr{L} = \sum_{i = 1}^k K(z_i)(Y_i - \hat{Y}_i)^2 $$

Considerations:

• This method doesn't produces coefficients for the general problem, like OLS does. That makes harder to interpret its results.
• It requires an enough large dataset and enough data points to produce satisfactory results.