G02 Chapter Contents
G02 Chapter Introduction
NAG Library Manual

# NAG Library Routine DocumentG02EEF

Note:  before using this routine, please read the Users' Note for your implementation to check the interpretation of bold italicised terms and other implementation-dependent details.

## 1  Purpose

G02EEF carries out one step of a forward selection procedure in order to enable the ‘best’ linear regression model to be found.

## 2  Specification

 SUBROUTINE G02EEF ( ISTEP, MEAN, WEIGHT, N, M, X, LDX, VNAME, ISX, MAXIP, Y, WT, FIN, ADDVAR, NEWVAR, CHRSS, F, MODEL, NTERM, RSS, IDF, IFR, FREE, EXSS, Q, LDQ, P, WK, IFAIL)
 INTEGER ISTEP, N, M, LDX, ISX(M), MAXIP, NTERM, IDF, IFR, LDQ, IFAIL REAL (KIND=nag_wp) X(LDX,M), Y(N), WT(*), FIN, CHRSS, F, RSS, EXSS(MAXIP), Q(LDQ,MAXIP+2), P(MAXIP+1), WK(2*MAXIP) LOGICAL ADDVAR CHARACTER(*) VNAME(M), NEWVAR, MODEL(MAXIP), FREE(MAXIP) CHARACTER(1) MEAN, WEIGHT

## 3  Description

One method of selecting a linear regression model from a given set of independent variables is by forward selection. The following procedure is used:
 (i) Select the best fitting independent variable, i.e., the independent variable which gives the smallest residual sum of squares. If the $F$-test for this variable is greater than a chosen critical value, ${F}_{\mathrm{c}}$, then include the variable in the model, else stop. (ii) Find the independent variable that leads to the greatest reduction in the residual sum of squares when added to the current model. (iii) If the $F$-test for this variable is greater than a chosen critical value, ${F}_{\mathrm{c}}$, then include the variable in the model and go to (ii), otherwise stop.
At any step the variables not in the model are known as the free terms.
G02EEF allows you to specify some independent variables that must be in the model, these are known as forced variables.
The computational procedure involves the use of $QR$ decompositions, the $R$ and the $Q$ matrices being updated as each new variable is added to the model. In addition the matrix ${Q}^{\mathrm{T}}{X}_{\mathrm{free}}$, where ${X}_{\mathrm{free}}$ is the matrix of variables not included in the model, is updated.
G02EEF computes one step of the forward selection procedure at a call. The results produced at each step may be printed or used as inputs to G02DDF, in order to compute the regression coefficients for the model fitted at that step. Repeated calls to G02EEF should be made until $F<{F}_{\mathrm{c}}$ is indicated.

## 4  References

Draper N R and Smith H (1985) Applied Regression Analysis (2nd Edition) Wiley
Weisberg S (1985) Applied Linear Regression Wiley

## 5  Parameters

Note:  after the initial call to G02EEF with ${\mathbf{ISTEP}}=0$ all parameters except FIN must not be changed by you between calls.
1:     ISTEP – INTEGERInput/Output
On entry: indicates which step in the forward selection process is to be carried out.
${\mathbf{ISTEP}}=0$
The process is initialized.
Constraint: ${\mathbf{ISTEP}}\ge 0$.
On exit: is incremented by $1$.
2:     MEAN – CHARACTER(1)Input
On entry: indicates if a mean term is to be included.
${\mathbf{MEAN}}=\text{'M'}$
A mean term, intercept, will be included in the model.
${\mathbf{MEAN}}=\text{'Z'}$
The model will pass through the origin, zero-point.
Constraint: ${\mathbf{MEAN}}=\text{'M'}$ or $\text{'Z'}$.
3:     WEIGHT – CHARACTER(1)Input
On entry: indicates if weights are to be used.
${\mathbf{WEIGHT}}=\text{'U'}$
Least squares estimation is used.
${\mathbf{WEIGHT}}=\text{'W'}$
Weighted least squares is used and weights must be supplied in array WT.
Constraint: ${\mathbf{WEIGHT}}=\text{'U'}$ or $\text{'W'}$.
4:     N – INTEGERInput
On entry: $n$, the number of observations.
Constraint: ${\mathbf{N}}\ge 2$.
5:     M – INTEGERInput
On entry: $m$, the total number of independent variables in the dataset.
Constraint: ${\mathbf{M}}\ge 1$.
6:     X(LDX,M) – REAL (KIND=nag_wp) arrayInput
On entry: ${\mathbf{X}}\left(\mathit{i},\mathit{j}\right)$ must contain the $\mathit{i}$th observation for the $\mathit{j}$th independent variable, for $\mathit{i}=1,2,\dots ,{\mathbf{N}}$ and $\mathit{j}=1,2,\dots ,{\mathbf{M}}$.
7:     LDX – INTEGERInput
On entry: the first dimension of the array X as declared in the (sub)program from which G02EEF is called.
Constraint: ${\mathbf{LDX}}\ge {\mathbf{N}}$.
8:     VNAME(M) – CHARACTER(*) arrayInput
On entry: ${\mathbf{VNAME}}\left(\mathit{j}\right)$ must contain the name of the independent variable in column $\mathit{j}$ of X, for $\mathit{j}=1,2,\dots ,{\mathbf{M}}$.
9:     ISX(M) – INTEGER arrayInput
On entry: indicates which independent variables could be considered for inclusion in the regression.
${\mathbf{ISX}}\left(j\right)\ge 2$
The variable contained in the $\mathit{j}$th column of X is automatically included in the regression model, for $\mathit{j}=1,2,\dots ,{\mathbf{M}}$.
${\mathbf{ISX}}\left(j\right)=1$
The variable contained in the $\mathit{j}$th column of X is considered for inclusion in the regression model, for $\mathit{j}=1,2,\dots ,{\mathbf{M}}$.
${\mathbf{ISX}}\left(j\right)=0$
The variable in the $\mathit{j}$th column is not considered for inclusion in the model, for $\mathit{j}=1,2,\dots ,{\mathbf{M}}$.
Constraint: ${\mathbf{ISX}}\left(\mathit{j}\right)\ge 0$ and at least one value of ${\mathbf{ISX}}\left(\mathit{j}\right)=1$, for $\mathit{j}=1,2,\dots ,{\mathbf{M}}$.
10:   MAXIP – INTEGERInput
On entry: the maximum number of independent variables to be included in the model.
Constraints:
• if ${\mathbf{MEAN}}=\text{'M'}$, ${\mathbf{MAXIP}}\ge 1+\text{}$ number of values of ${\mathbf{ISX}}>0$;
• if ${\mathbf{MEAN}}=\text{'Z'}$, ${\mathbf{MAXIP}}\ge \text{}$ number of values of ${\mathbf{ISX}}>0$.
11:   Y(N) – REAL (KIND=nag_wp) arrayInput
On entry: the dependent variable.
12:   WT($*$) – REAL (KIND=nag_wp) arrayInput
Note: the dimension of the array WT must be at least ${\mathbf{N}}$ if ${\mathbf{WEIGHT}}=\text{'W'}$.
On entry: if ${\mathbf{WEIGHT}}=\text{'W'}$, WT must contain the weights to be used in the weighted regression, $W$.
If ${\mathbf{WT}}\left(i\right)=0.0$, the $i$th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.
If ${\mathbf{WEIGHT}}=\text{'U'}$, WT is not referenced and the effective number of observations is N.
Constraint: if ${\mathbf{WEIGHT}}=\text{'W'}$, ${\mathbf{WT}}\left(\mathit{i}\right)\ge 0.0$, for $\mathit{i}=1,2,\dots ,{\mathbf{N}}$.
13:   FIN – REAL (KIND=nag_wp)Input
On entry: the critical value of the $F$ statistic for the term to be included in the model, ${F}_{\mathrm{c}}$.
Suggested value: $2.0$ is a commonly used value in exploratory modelling.
Constraint: ${\mathbf{FIN}}\ge 0.0$.
On exit: indicates if a variable has been added to the model.
${\mathbf{ADDVAR}}=\mathrm{.TRUE.}$
A variable has been added to the model.
${\mathbf{ADDVAR}}=\mathrm{.FALSE.}$
No variable had an $F$ value greater than ${F}_{\mathrm{c}}$ and none were added to the model.
15:   NEWVAR – CHARACTER(*)Output
On exit: if ${\mathbf{ADDVAR}}=\mathrm{.TRUE.}$, NEWVAR contains the name of the variable added to the model.
Constraint: the declared size of NEWVAR must be greater than or equal to the declared size of VNAME.
On exit: if ${\mathbf{ADDVAR}}=\mathrm{.TRUE.}$, CHRSS contains the change in the residual sum of squares due to adding variable NEWVAR.
17:   F – REAL (KIND=nag_wp)Output
On exit: if ${\mathbf{ADDVAR}}=\mathrm{.TRUE.}$, F contains the $F$ statistic for the inclusion of the variable in NEWVAR.
18:   MODEL(MAXIP) – CHARACTER(*) arrayInput/Output
On entry: if ${\mathbf{ISTEP}}=0$, MODEL need not be set.
If ${\mathbf{ISTEP}}\ne 0$, MODEL must contain the values returned by the previous call to G02EEF.
Constraint: the declared size of MODEL must be greater than or equal to the declared size of VNAME.
On exit: the names of the variables in the current model.
19:   NTERM – INTEGERInput/Output
On entry: if ${\mathbf{ISTEP}}=0$, NTERM need not be set.
If ${\mathbf{ISTEP}}\ne 0$, NTERM must contain the value returned by the previous call to G02EEF.
Constraint: if ${\mathbf{ISTEP}}\ne 0$, ${\mathbf{NTERM}}>0$.
On exit: the number of independent variables in the current model, not including the mean, if any.
On entry: if ${\mathbf{ISTEP}}=0$, RSS need not be set.
If ${\mathbf{ISTEP}}\ne 0$, RSS must contain the value returned by the previous call to G02EEF.
Constraint: if ${\mathbf{ISTEP}}\ne 0$, ${\mathbf{RSS}}>0.0$.
On exit: the residual sums of squares for the current model.
21:   IDF – INTEGERInput/Output
On entry: if ${\mathbf{ISTEP}}=0$, IDF need not be set.
If ${\mathbf{ISTEP}}\ne 0$, IDF must contain the value returned by the previous call to G02EEF.
On exit: the degrees of freedom for the residual sum of squares for the current model.
22:   IFR – INTEGERInput/Output
On entry: if ${\mathbf{ISTEP}}=0$, IFR need not be set.
If ${\mathbf{ISTEP}}\ne 0$, IFR must contain the value returned by the previous call to G02EEF.
On exit: the number of free independent variables, i.e., the number of variables not in the model that are still being considered for selection.
23:   FREE(MAXIP) – CHARACTER(*) arrayInput/Output
On entry: if ${\mathbf{ISTEP}}=0$, FREE need not be set.
If ${\mathbf{ISTEP}}\ne 0$, FREE must contain the values returned by the previous call to G02EEF.
Constraint: the declared size of FREE must be greater than or equal to the declared size of VNAME.
On exit: the first IFR values of FREE contain the names of the free variables.
24:   EXSS(MAXIP) – REAL (KIND=nag_wp) arrayOutput
On exit: the first IFR values of EXSS contain what would be the change in regression sum of squares if the free variables had been added to the model, i.e., the extra sum of squares for the free variables. ${\mathbf{EXSS}}\left(i\right)$ contains what would be the change in regression sum of squares if the variable ${\mathbf{FREE}}\left(i\right)$ had been added to the model.
25:   Q(LDQ,${\mathbf{MAXIP}}+2$) – REAL (KIND=nag_wp) arrayInput/Output
On entry: if ${\mathbf{ISTEP}}=0$, Q need not be set.
If ${\mathbf{ISTEP}}\ne 0$, Q must contain the values returned by the previous call to G02EEF.
On exit: the results of the $QR$ decomposition for the current model:
• the first column of Q contains $c={Q}^{\mathrm{T}}y$ (or ${Q}^{\mathrm{T}}{W}^{\frac{1}{2}}y$ where $W$ is the vector of weights if used);
• the upper triangular part of columns $2$ to $p+1$ contain the $R$ matrix;
• the strictly lower triangular part of columns $2$ to $p+1$ contain details of the $Q$ matrix;
• the remaining $p+1$ to $p+{\mathbf{IFR}}$ columns of contain ${Q}^{\mathrm{T}}{X}_{\mathit{free}}$ (or ${Q}^{\mathrm{T}}{W}^{\frac{1}{2}}{X}_{\mathit{free}}$),
where $p={\mathbf{NTERM}}$, or $p={\mathbf{NTERM}}+1$ if ${\mathbf{MEAN}}=\text{'M'}$.
26:   LDQ – INTEGERInput
On entry: the first dimension of the array Q as declared in the (sub)program from which G02EEF is called.
Constraint: ${\mathbf{LDQ}}\ge {\mathbf{N}}$.
27:   P(${\mathbf{MAXIP}}+1$) – REAL (KIND=nag_wp) arrayInput/Output
On entry: if ${\mathbf{ISTEP}}=0$, P need not be set.
If ${\mathbf{ISTEP}}\ne 0$, P must contain the values returned by the previous call to G02EEF.
On exit: the first $p$ elements of P contain details of the $QR$ decomposition, where $p={\mathbf{NTERM}}$, or $p={\mathbf{NTERM}}+1$ if ${\mathbf{MEAN}}=\text{'M'}$.
28:   WK($2×{\mathbf{MAXIP}}$) – REAL (KIND=nag_wp) arrayWorkspace
29:   IFAIL – INTEGERInput/Output
On entry: IFAIL must be set to $0$, $-1\text{​ or ​}1$. If you are unfamiliar with this parameter you should refer to Section 3.3 in the Essential Introduction for details.
For environments where it might be inappropriate to halt program execution when an error is detected, the value $-1\text{​ or ​}1$ is recommended. If the output of error messages is undesirable, then the value $1$ is recommended. Otherwise, if you are not familiar with this parameter, the recommended value is $0$. When the value $-\mathbf{1}\text{​ or ​}\mathbf{1}$ is used it is essential to test the value of IFAIL on exit.
On exit: ${\mathbf{IFAIL}}={\mathbf{0}}$ unless the routine detects an error or a warning has been flagged (see Section 6).

## 6  Error Indicators and Warnings

If on entry ${\mathbf{IFAIL}}={\mathbf{0}}$ or $-{\mathbf{1}}$, explanatory error messages are output on the current error message unit (as defined by X04AAF).
Errors or warnings detected by the routine:
${\mathbf{IFAIL}}=1$
 On entry, ${\mathbf{N}}<1$, or ${\mathbf{M}}<1$, or ${\mathbf{LDX}}<{\mathbf{N}}$, or ${\mathbf{LDQ}}<{\mathbf{N}}$, or ${\mathbf{ISTEP}}<0$, or ${\mathbf{ISTEP}}\ne 0$ and ${\mathbf{NTERM}}=0$, or ${\mathbf{ISTEP}}\ne 0$ and ${\mathbf{RSS}}\le 0.0$, or ${\mathbf{FIN}}<0.0$, or ${\mathbf{MEAN}}\ne \text{'M'}$ or $\text{'Z'}$, or ${\mathbf{WEIGHT}}\ne \text{'U'}$ or $\text{'W'}$.
${\mathbf{IFAIL}}=2$
 On entry, ${\mathbf{WEIGHT}}=\text{'W'}$ and a value of ${\mathbf{WT}}<0.0$.
${\mathbf{IFAIL}}=3$
On entry, the degrees of freedom will be zero if a variable is selected, i.e., the number of variables in the model plus $1$ is equal to the effective number of observations.
${\mathbf{IFAIL}}=4$
 On entry, a value of ${\mathbf{ISX}}<0$, or there are no forced or free variables, i.e., no element of ${\mathbf{ISX}}>0$, or the value of MAXIP is too small for number of variables indicated by ISX.
${\mathbf{IFAIL}}=5$
On entry, the variables forced into the model are not of full rank, i.e., some of these variables are linear combinations of others.
${\mathbf{IFAIL}}=6$
 On entry, there are no free variables, i.e., no element of ${\mathbf{ISX}}=0$.
${\mathbf{IFAIL}}=7$
The value of the change in the sum of squares is greater than the input value of RSS. This may occur due to rounding errors if the true residual sum of squares for the new model is small relative to the residual sum of squares for the previous model.

## 7  Accuracy

As G02EEF uses a $QR$ transformation the results will often be more accurate than traditional algorithms using methods based on the cross-products of the dependent and independent variables.

None.

## 9  Example

The data, from an oxygen uptake experiment, is given by Weisberg (1985). The names of the variables are as given in Weisberg (1985). The independent and dependent variables are read and G02EEF is repeatedly called until ${\mathbf{ADDVAR}}=\mathrm{.FALSE.}$. At each step the $F$ statistic, the free variables and their extra sum of squares are printed; also, except for when ${\mathbf{ADDVAR}}=\mathrm{.FALSE.}$, the new variable, the change in the residual sum of squares and the terms in the model are printed.

### 9.1  Program Text

Program Text (g02eefe.f90)

### 9.2  Program Data

Program Data (g02eefe.d)

### 9.3  Program Results

Program Results (g02eefe.r)