naginterfaces.library.correg.linregm_​fit_​stepwise

naginterfaces.library.correg.linregm_fit_stepwise(n, wmean, c, sw, isx, fin=4.0, fout=None, tau=1e-06, monlev=0, monfun=None, data=None, io_manager=None)

linregm_fit_stepwise calculates a full stepwise selection from $p$ variables by using Clarke’s sweep algorithm on the correlation matrix of a design and data matrix, $Z$. The (weighted) variance-covariance, (weighted) means and sum of weights of $Z$ must be supplied.

For full information please refer to the NAG Library document for g02ef

https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/g02/g02eff.html

Parameters

nint

The number of observations used in the calculations.

wmeanfloat, array-like, shape $(m+1)$

The mean of the design matrix, $Z$.

cfloat, array-like, shape $((m+1)\times (m+2)/2)$

The upper-triangular variance-covariance matrix packed by column for the design matrix, $Z$. Because the function computes the correlation matrix $R$ from $c$, the variance-covariance matrix need only be supplied up to a scaling factor.

swfloat

If weights were used to calculate $c$ then $sw$ is the sum of positive weight values; otherwise $sw$ is the number of observations used to calculate $c$.

isxint, array-like, shape $\left(m\right)$

The value of $isx[\text{j}-1]$ determines the set of variables used to perform full stepwise model selection, for $\text{j}=1,2,\dots ,m$.

$isx[\text{j}-1]=-1$

To exclude the variable corresponding to the $j$th column of $X$ from the final model.

$isx[\text{j}-1]=1$

To consider the variable corresponding to the $j$th column of $X$ for selection in the final model.

$isx[\text{j}-1]=2$

To force the inclusion of the variable corresponding to the $j$th column of $X$ in the final model.

finfloat, optional

The value of the variance ratio which an explanatory variable must exceed to be included in a model.

foutNone or float, optional

Note: if this argument is None then a default value will be used, determined as follows: $fin$.

The explanatory variable in a model with the lowest variance ratio value is removed from the model if its value is less than $fout$. $fout$ is usually set equal to the value of $fin$; a value less than $fin$ is occasionally preferred.

taufloat, optional

The tolerance, $\tau$, for detecting collinearities between variables when adding or removing an explanatory variable from a model. Explanatory variables deemed to be collinear are excluded from the final model.

monlevint, optional

A value of $1$ for $monlev$ enables monitoring of the model selection process; a value of $0$ disables it.

monfunNone or callable monfun(flag, var, val, data=None), optional

Note: if this argument is None then a NAG-supplied facility will be used.

The function for monitoring the model selection process.

Parameters

flagstr, length 1

The value of $flag$ indicates the stage of the stepwise selection of explanatory variables.

$flag=\text{'A'}$

Variable $var$ was added to the current model.

$flag=\text{'B'}$

Beginning the backward elimination step.

$flag=\text{'C'}$

Variable $var$ failed the collinearity test and is excluded from the model.

$flag=\text{'D'}$

Variable $var$ was dropped from the current model.

$flag=\text{'F'}$

Beginning the forward selection step

$flag=\text{'K'}$

Backward elimination did not remove any variables from the current model.

$flag=\text{'S'}$

Starting stepwise selection procedure.

$flag=\text{'V'}$

The variance ratio for variable $var$ takes the value $val$.

$flag=\text{'X'}$

Finished stepwise selection procedure.

varint

The index of the explanatory variable in the design matrix $Z$ to which $flag$ pertains.

valfloat

If $flag=\text{'V'}$, $val$ is the variance ratio value for the coefficient associated with explanatory variable index $var$.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

dataarbitrary, optional

User-communication data for callback functions.

io_managerFileObjManager, optional

Manager for I/O in this routine.

Returns

isxint, ndarray, shape $\left(m\right)$

The value of $isx[\text{j}-1]$ indicates the status of the $j$th explanatory variable in the model.

$isx[\text{j}-1]=-1$

Forced exclusion.

$isx[\text{j}-1]=0$

Excluded.

$isx[\text{j}-1]=1$

Selected.

$isx[\text{j}-1]=2$

Forced selection.

bfloat, ndarray, shape $(m+1)$

$b\left[0\right]$ contains the estimate for the intercept term in the fitted model. If $isx[j-1]\ne 0$, then $b[j+1-1]$ contains the estimate for the $j$th explanatory variable in the fitted model; otherwise $b[j+1-1]=0$.

sefloat, ndarray, shape $(m+1)$

$se[\text{j}-1]$ contains the standard error for the estimate of $b[\text{j}-1]$, for $\text{j}=1,2,\dots ,m+1$.

rsqfloat

The $R^2$-statistic for the fitted regression model.

rmsfloat

The mean square of residuals for the fitted regression model.

dfint

The number of degrees of freedom for the sum of squares of residuals.

Raises

NagValueError

(errno $1$)

On entry, $m=⟨value⟩$.

Constraint: $m>1$.

(errno $1$)

On entry, $n=⟨value⟩$.

Constraint: $n>1$.

(errno $1$)

On entry, $sw=⟨value⟩$.

Constraint: $sw>1.0$.

(errno $1$)

On entry, $fin=⟨value⟩$.

Constraint: $fin>0.0$.

(errno $1$)

On entry, $fout=⟨value⟩$; $fin=⟨value⟩$.

Constraint: $0.0\le fout\le fin$.

(errno $1$)

On entry, $tau=⟨value⟩$.

Constraint: $tau>0.0$.

(errno $1$)

On entry, $monlev=⟨value⟩$.

Constraint: $monlev=0$ or $1$.

(errno $2$)

On entry at least one diagonal element of $c\le 0.0$.

Constraint: $c$ must be positive definite.

(errno $2$)

On entry, $isx[⟨value⟩]=⟨value⟩$.

Constraint: $isx[\text{j}-1]=-1$, $1$ or $2$, for $\text{j}=1,2,\dots ,m$.

(errno $2$)

On entry, $isx[i-1]\ne 1$, for all $i=1,2,\dots ,m$.

Constraint: there must be at least one free variable.

(errno $4$)

All variables are collinear, no model to select.

Warns

NagAlgorithmicWarning

(errno $3$)

Matrix not positive definite, results may be inaccurate.

Notes

The general multiple linear regression model is defined by

$$y={\beta }_0+X\beta +ϵ\text{,}$$

where

$y$ is a vector of $n$ observations on the dependent variable,

${\beta }_0$ is an intercept coefficient,

$X$ is an $n\times p$ matrix of $p$ explanatory variables,

$\beta$ is a vector of $p$ unknown coefficients, and

$ϵ$ is a vector of length $n$ of unknown, Normally distributed, random errors.

linregm_fit_stepwise employs a full stepwise regression to select a subset of explanatory variables from the $p$ available variables (the intercept is included in the model) and computes regression coefficients and their standard errors, and various other statistical quantities, by minimizing the sum of squares of residuals. The method applies repeatedly a forward selection step followed by a backward elimination step and halts when neither step updates the current model.

The criterion used to update a current model is the variance ratio of residual sum of squares. Let $s_1$ and $s_2$ be the residual sum of squares of the current model and this model after undergoing a single update, with degrees of freedom $q_1$ and $q_2$, respectively. Then the condition:

$$\frac{(s_2-s_1)/(q_2-q_1)}{s_1/q_1}>f_1\text{,}$$

must be satisfied if a variable $k$ will be considered for entry to the current model, and the condition:

$$\frac{(s_1-s_2)/(q_1-q_2)}{s_1/q_1}<f_2\text{,}$$

must be satisfied if a variable $k$ will be considered for removal from the current model, where $f_1$ and $f_2$ are user-supplied values and $f_2\le f_1$.

In the entry step the entry statistic is computed for each variable not in the current model. If no variable is associated with a test value that exceeds $f_1$ then this step is terminated; otherwise the variable associated with the largest value for the entry statistic is entered into the model.

In the removal step the removal statistic is computed for each variable in the current model. If no variable is associated with a test value less than $f_2$ then this step is terminated; otherwise the variable associated with the smallest value for the removal statistic is removed from the model.

The data values $X$ and $y$ are not provided as input to the function. Instead, summary statistics of the design and data matrix $Z=\left(X|y\right)$ are required.

Explanatory variables are entered into and removed from the current model by using sweep operations on the correlation matrix $R$ of $Z$, given by:

$$\begin{array}{c}R=\left(\begin{array}{cccc}1& \dots & r_{1p}& r_{1y}\\ ⋮& \ddots & ⋮& ⋮\\ r_{p1}& \dots & 1& r_{py}\\ r_{y1}& \dots & r_{yp}& 1\end{array}\right)\text{,}\end{array}$$

where $r_{\text{i}\text{j}}$ is the correlation between the explanatory variables $\text{i}$ and $\text{j}$, for $\text{j}=1,2,\dots ,p$, for $\text{i}=1,2,\dots ,p$, and $r_{yi}$ (and $r_{iy}$) is the correlation between the response variable $y$ and the $\text{i}$th explanatory variable, for $\text{i}=1,2,\dots ,p$.

A sweep operation on the $k$th row and column ($k\le p$) of $R$ replaces:

$$\begin{array}{c}\begin{array}{c}{r}_{kk}\text{by}-1/r_{kk}\text{;}\\ r_{ik}\text{by}{r}_{ik}/\left|r_{kk}\right|\text{,}i=1,2,\dots ,p+1(i\ne k)\text{;}\\ r_{kj}\text{by}{r}_{kj}/\left|r_{kk}\right|\text{,}j=1,2,\dots ,p+1(j\ne k)\text{;}\\ r_{ij}\text{by}{r}_{ij}-r_{ik}{r}_{kj}/\left|r_{kk}\right|\text{,}i=1,2,\dots ,p+1(i\ne k)\text{;}j=1,2,\dots ,p+1(j\ne k)\text{.}\end{array}\end{array}$$

The $k$th explanatory variable is eligible for entry into the current model if it satisfies the collinearity tests: $r_{kk}>\tau$ and

$$(r_{ii}-\frac{r_{ik}{r}_{ki}}{r_{kk}})\tau \le 1\text{,}$$

for a user-supplied value ($>0$) of $\tau$ and where the index $i$ runs over explanatory variables in the current model. The sweep operation is its own inverse, therefore, pivoting on an explanatory variable $k$ in the current model has the effect of removing it from the model.

Once the stepwise model selection procedure is finished, the function calculates:

1. the least squares estimate for the $i$th explanatory variable included in the fitted model;

2. standard error estimates for each coefficient in the final model;

3. the square root of the mean square of residuals and its degrees of freedom;

4. the multiple correlation coefficient.

The function makes use of the symmetry of the sweep operations and correlation matrix which reduces by almost one half the storage and computation required by the sweep algorithm, see Clarke (1981) for details.

References

Clarke, M R B, 1981, Algorithm AS 178: the Gauss–Jordan sweep operator with detection of collinearity, Appl. Statist. (31), 166–169

Dempster, A P, 1969, Elements of Continuous Multivariate Analysis, Addison–Wesley

Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley

See also

naginterfaces.library.examples.correg.linregm_fit_stepwise_ex.main()