The routine may be called by the names g02def or nagf_correg_linregm_var_add.
3Description
A linear regression model may be built up by adding new independent variables to an existing model. g02def updates the $QR$ decomposition used in the computation of the linear regression model. The $QR$ decomposition may come from g02daf or a previous call to g02def. The general linear regression model is defined by
$$y=X\beta +\epsilon \text{,}$$
where
$y$ is a vector of $n$ observations on the dependent variable,
$X$ is an $n\times p$ matrix of the independent variables of column rank $k$,
$\beta $ is a vector of length $p$ of unknown parameters,
and
$\epsilon $ is a vector of length $n$ of unknown random errors such that $\mathrm{var}\epsilon =V{\sigma}^{2}$, where $V$ is a known diagonal matrix.
If $V=I$, the identity matrix, then least squares estimation is used. If $V\ne I$, then for a given weight matrix $W\propto {V}^{-1}$, weighted least squares estimation is used.
The least squares estimates, $\hat{\beta}$ of the parameters $\beta $ minimize ${(y-X\beta )}^{\mathrm{T}}(y-X\beta )$ while the weighted least squares estimates, minimize ${(y-X\beta )}^{\mathrm{T}}W(y-X\beta )$.
The parameter estimates may be found by computing a $QR$ decomposition of $X$ (or ${W}^{\frac{1}{2}}X$ in the weighted case), i.e.,
where ${R}^{*}=\left(\begin{array}{l}R\\ 0\end{array}\right)$ and $R$ is a $p\times p$ upper triangular matrix and $Q$ is an $n\times n$ orthogonal matrix.
If $R$ is of full rank, then $\hat{\beta}$ is the solution to
$$R\hat{\beta}={c}_{1}\text{,}$$
where $c={Q}^{\mathrm{T}}y$ (or ${Q}^{\mathrm{T}}{W}^{\frac{1}{2}}y$) and ${c}_{1}$ is the first $p$ elements of $c$.
If $R$ is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of $R$.
To add a new independent variable, ${x}_{p+1}$, $R$ and $c$ have to be updated. The matrix ${Q}_{p+1}$ is found such that ${Q}_{p+1}^{\mathrm{T}}[R:{Q}^{\mathrm{T}}{x}_{p+1}]$ (or ${Q}_{p+1}^{\mathrm{T}}[R:{Q}^{\mathrm{T}}{W}^{\frac{1}{2}}{x}_{p+1}]$) is upper triangular. The vector $c$ is then updated by multiplying by ${Q}_{p+1}^{\mathrm{T}}$.
The new independent variable is tested to see if it is linearly related to the existing independent variables by checking that at least one of the values ${\left({Q}^{\mathrm{T}}{x}_{p+1}\right)}_{\mathit{i}}$, for $\mathit{i}=p+2,\dots ,n$, is nonzero.
The new parameter estimates, $\hat{\beta}$, can then be obtained by a call to g02ddf.
The routine can be used with $p=0$, in which case $R$ and $c$ are initialized.
4References
Draper N R and Smith H (1985) Applied Regression Analysis (2nd Edition) Wiley
Golub G H and Van Loan C F (1996) Matrix Computations (3rd Edition) Johns Hopkins University Press, Baltimore
Hammarling S (1985) The singular value decomposition in multivariate statistics SIGNUM Newsl.20(3) 2–25
McCullagh P and Nelder J A (1983) Generalized Linear Models Chapman and Hall
Searle S R (1971) Linear Models Wiley
5Arguments
1: $\mathbf{weight}$ – Character(1)Input
On entry: indicates if weights are to be used.
${\mathbf{weight}}=\text{'U'}$
Least squares estimation is used.
${\mathbf{weight}}=\text{'W'}$
Weighted least squares is used and weights must be supplied in array wt.
Constraint:
${\mathbf{weight}}=\text{'U'}$ or $\text{'W'}$.
2: $\mathbf{n}$ – IntegerInput
On entry: $n$, the number of observations.
Constraint:
${\mathbf{n}}\ge 1$.
3: $\mathbf{ip}$ – IntegerInput
On entry: $p$, the number of independent variables already in the model.
Constraint:
${\mathbf{ip}}\ge 0$ and ${\mathbf{ip}}<{\mathbf{n}}$.
4: $\mathbf{q}({\mathbf{ldq}},{\mathbf{ip}}+2)$ – Real (Kind=nag_wp) arrayInput/Output
On entry: if ${\mathbf{ip}}\ne 0$, q must contain the results of the $QR$ decomposition for the model with $p$ parameters as returned by g02daf or a previous call to g02def.
If ${\mathbf{ip}}=0$, the first column of q should contain the $n$ values of the dependent variable, $y$.
On exit: the results of the $QR$ decomposition for the model with $p+1$ parameters:
the first column of q contains the updated value of $c$;
the columns $2$ to ${\mathbf{ip}}+1$ are unchanged;
the first ${\mathbf{ip}}+1$ elements of column ${\mathbf{ip}}+2$ contain the new column of $R$, while the remaining ${\mathbf{n}}-{\mathbf{ip}}-1$ elements contain details of the matrix ${Q}_{p+1}$.
5: $\mathbf{ldq}$ – IntegerInput
On entry: the first dimension of the array q as declared in the (sub)program from which g02def is called.
Constraint:
${\mathbf{ldq}}\ge {\mathbf{n}}$.
6: $\mathbf{p}\left({\mathbf{ip}}+1\right)$ – Real (Kind=nag_wp) arrayInput/Output
On entry: contains further details of the $QR$ decomposition used. The first ip elements of p must contain the zeta values for the $QR$ decomposition (see f08aef for details).
The first ip elements of array p are provided by g02daf or by previous calls to g02def.
On exit: the first ip elements of p are unchanged and the $({\mathbf{ip}}+1)$th element contains the zeta value for ${Q}_{p+1}$.
7: $\mathbf{wt}\left(*\right)$ – Real (Kind=nag_wp) arrayInput
Note: the dimension of the array wt
must be at least
${\mathbf{n}}$ if ${\mathbf{weight}}=\text{'W'}$.
On entry: if ${\mathbf{weight}}=\text{'W'}$wt must contain the weights to be used with the model.
If ${\mathbf{wt}}\left(i\right)=0.0$, the $i$th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.
If ${\mathbf{weight}}=\text{'U'}$, wt is not referenced and the effective number of observations is $n$.
Constraint:
if ${\mathbf{weight}}=\text{'W'}$, ${\mathbf{wt}}\left(\mathit{i}\right)\ge 0.0$, for $\mathit{i}=1,2,\dots ,n$.
8: $\mathbf{x}\left({\mathbf{n}}\right)$ – Real (Kind=nag_wp) arrayInput
On entry: $x$, the new independent variable.
9: $\mathbf{rss}$ – Real (Kind=nag_wp)Output
On exit: the residual sum of squares for the new fitted model.
Note: this will only be valid if the model is of full rank, see Section 9.
10: $\mathbf{tol}$ – Real (Kind=nag_wp)Input
On entry: the value of tol is used to decide if the new independent variable is linearly related to independent variables already included in the model. If the new variable is linearly related then $c$ is not updated. The smaller the value of tol the stricter the criterion for deciding if there is a linear relationship.
Suggested value:
${\mathbf{tol}}=0.000001$.
Constraint:
${\mathbf{tol}}>0.0$.
11: $\mathbf{ifail}$ – IntegerInput/Output
On entry: ifail must be set to $0$, $-1$ or $1$ to set behaviour on detection of an error; these values have no effect when no error is detected.
A value of $0$ causes the printing of an error message and program execution will be halted; otherwise program execution continues. A value of $-1$ means that an error message is printed while a value of $1$ means that it is not.
If halting is not appropriate, the value $-1$ or $1$ is recommended. If message printing is undesirable, then the value $1$ is recommended. Otherwise, the value $-1$ is recommended since useful values can be provided in some output arguments even when ${\mathbf{ifail}}\ne {\mathbf{0}}$ on exit. When the value $-\mathbf{1}$ or $\mathbf{1}$ is used it is essential to test the value of ifail on exit.
On exit: ${\mathbf{ifail}}={\mathbf{0}}$ unless the routine detects an error or a warning has been flagged (see Section 6).
6Error Indicators and Warnings
If on entry ${\mathbf{ifail}}=0$ or $-1$, explanatory error messages are output on the current error message unit (as defined by x04aaf).
Errors or warnings detected by the routine:
Note: in some cases g02def may return useful information.
${\mathbf{ifail}}=1$
On entry, ${\mathbf{ip}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{ip}}\ge 0$.
On entry, ${\mathbf{ip}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{ip}}<{\mathbf{n}}$.
On entry, ${\mathbf{ldq}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{ldq}}\ge {\mathbf{n}}$.
On entry, ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{n}}\ge 1$.
On entry, ${\mathbf{tol}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{tol}}>0.0$.
On entry, ${\mathbf{weight}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{weight}}=\text{'W'}$ or $\text{'U'}$.
${\mathbf{ifail}}=2$
On entry, ${\mathbf{wt}}\left(\u27e8\mathit{\text{value}}\u27e9\right)<0.0$.
Constraint: ${\mathbf{wt}}\left(i\right)\ge 0.0$, for $i=1,2,\dots ,n$.
${\mathbf{ifail}}=3$
x variable is a linear combination of existing model terms.
${\mathbf{ifail}}=-99$
An unexpected error has been triggered by this routine. Please
contact NAG.
See Section 7 in the Introduction to the NAG Library FL Interface for further information.
${\mathbf{ifail}}=-399$
Your licence key may have expired or may not have been installed correctly.
See Section 8 in the Introduction to the NAG Library FL Interface for further information.
${\mathbf{ifail}}=-999$
Dynamic memory allocation failed.
See Section 9 in the Introduction to the NAG Library FL Interface for further information.
7Accuracy
The accuracy is closely related to the accuracy of f08agf which should be consulted for further details.
8Parallelism and Performance
g02def is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
g02def makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this routine. Please also consult the Users' Note for your implementation for any additional implementation-specific information.
9Further Comments
It should be noted that the residual sum of squares produced by g02def may not be correct if the model to which the new independent variable is added is not of full rank. In such a case g02ddf should be used to calculate the residual sum of squares.
10Example
A dataset consisting of $12$ observations is read in. The four independent variables are stored in the array x while the dependent variable is read into the first column of q. If the character variable $\mathit{mean}$ indicates that a mean should be included in the model a variable taking the value $1.0$ for all observations is set up and fitted. Subsequently, one variable at a time is selected to enter the model as indicated by the input value of $\mathit{indx}$. After the variable has been added the parameter estimates are calculated by g02ddf and the results printed. This is repeated until the input value of $\mathit{indx}$ is $0$.