Linear Model Classification

It is useful to describe various classifications of linear models to better understand how each type of linear model (e.g. Response Surface Methodology) fits into the "big picture".

A very helpful guide, given by Matlab, that describes and illustrates various regression analysis techniques: http://www.mathworks.com/help/toolbox/stats/bq_676m-2.html

Generalized Linear Models

"Generalized linear models" are linear models that can account for arbitrary numbers of inputs and outputs. These models assume errors are Gaussian, use statistics and create statistical models for data analysis.

General form:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle \mathbf{Y} = \mathbf{B X} + \mathbf{U} }

where:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle Y} is the response of the function of interest

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle \mathbf{B}} is the coefficient vector/matrix for the linear model coefficients

$\mathbf {X}$ are the inputs to the function

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle \mathbf{U}} is the error

General linear model information:

http://en.wikipedia.org/wiki/Multivariate_regression_model
Nelder and Wedderburn (1972)

Matlab functions:

glmfit
- http://www.mathworks.com/help/toolbox/stats/glmfit.html
- Carries out the regression

glmval
- http://www.mathworks.com/help/toolbox/stats/glmval.html
- Evaluates the value of a generalized linear model

Multiple Linear Regression

"Multiple linear regression" is a model for one response variable ("y"), and multiple predictor variables ("X").

Linear regression information:

Generic multiple linear regression model looks like:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \dots + \beta_{k} x_{ik} + e_i \quad i=1 \dots n }

for Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle n} responses (and all responses treated independently). Here, Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle y_i} is the Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle i^{th}} response from the "real" (i.e. unknown) process, which is perfectly represented using the polynomial above.

It is desirable to create a surrogate model that creates as good an approximation as possible of the above polynomial.

This "real", unknown polynomial can also be written (similar to ANOVA model):

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle y_i = \mu_i + e_i }

where

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle \mu_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \dots + \beta_{k} x_{ik} }

If each of the coefficients Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle \beta_j} are approximated (using linear algebra, e.g. least squares), the mean can be approximated:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle \hat{\mu}_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \dots + \beta_{k} x_{ik} }

and using that, the "real" polynomial responses Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle y_i} can be approximated with surrogate (or predicted) polynomial responses Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle \hat{y}_i} :

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle \hat{y}_i = \hat{\mu}_i }

Multivariate Linear Regression

"Multivariate linear regression" broadens multiple linear regression to account for more than one response variable ("Y").

Multivariate regression/analysis information:

http://en.wikipedia.org/wiki/Multivariate_analysis

Polynomial Models (Univariate)

Polynomial models can be used to fit a univariate function of a single input paramter, e.g. Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://en.wikipedia.org/api/rest_v1/":): {\displaystyle y(x)}

This can be done using the following Matlab functions:

polyfit
- http://www.mathworks.com/help/techdoc/ref/polyfit.html
- Fits a polynomial of a given degree to a set of inputs x and outputs y

polyval
- http://www.mathworks.com/help/techdoc/ref/polyval.html
- Evaluates the value of a given polynomial model at given input variable value or values

polyconf
- http://www.mathworks.com/help/toolbox/stats/polyconf.html
- Can be used to construct confidence intervals for polynomial models

Response Surface Models (Multivariate)

Main article: Response Surface Methodology

Response surfaces may be created in a number of different ways:

regress

regstats

SurfaceFit (Statistics toolbox)

others???

Linear Models

From charlesreid1

Contents