regression analysis

• Linear regression, often fitted using least squares approach
• Least squares (Legendre 1805, Gauss 1809)
• Least absolute deviation
• Nonparametric regression
• Bayesian methods (such as Bayesian linear regression)
• Percentage regression
• Quantile regression
• Distance metric learning

The terms linear regression and least squares are closely linked; but they're not synonyms.

β is the unknown parameter (scalar or vector)

X is the independent variable.

Y is the dependent variable.

Y ≈ f(X, β)

Non-robust regression models (such as ordinary least square) make good predictions if the underlying assumption is correct. However, if the underlying assumption is wrong, they give misleading information.

Robost regression models try not to be overly affected by wrong assumptions.

Robust regression might be used

• when the data contains outliers
• when there is a strong suspicion of heteroscedasticity.

Heteroscedasticity can invalidate statistical tests of significance.

Regression validation tries to decide whether the results of a model (that was obtained from regression analysis) is acceptable to describe data.

pattern identification