# Linear model

In

statistics the

**linear model** can be expressed by saying

where

`Y` is an

`nx1` column vector of random variables,

`X` is an

`nxp` matrix of "known" (i.e., observable and non-random) quanitities, whose rows correspond to statistical units,

`β` is a

`px1` vector of (unobservable) parameters, and

`ε` is an

`nx1` vector of "errors", which are uncorrelated random variables each with expected value 0 and variance

`σ`^{2}. Often one takes the components of the vector of errors to be

independent and

normally distributed. Having observed the values of

`X` and

`Y`, the statistician must estimate

`β` and

`σ`^{2}. Typically the parameters

`β` are estimated by the method of

least squares.

If, rather than taking the variance of `ε` to be `σ`^{2}I, where `I` is the `nxn` identity matrix, one assumes the variance is `σ`^{2}M,
where `M` is a known matrix other than the identity matrix, then one estimates `β` by the method of "generalized least squares", in which, instead of minimizing the sum of squares of the residuals, one minimizes a different quadratic form in the residuals -- the quadratic form being the one given by the matrix `M`^{-1}.
If all of the off-diagonal entries in the matrix `M` are 0, then one normally estimates `β` by the method of "weighted least squares", with weights proportional to the reciprocals of the diagonal entries.

Ordinary Linear regression is a very closely related topic.

"Generalized linear models", rather than saying

say

where

`f` is the "link function". An example is the "Poisson regression model", which says

`Y`_{i} has a Poisson distribution with expected value `e`^{γ+δxi}.

The link function is the natural logarithm function.
Having observed

`x`_{i} and

`Y`_{i} for

`i=1,...,n`, one can estimate γ and δ by the method of

maximum likelihood.