NAG CL Interface
g02dac (linregm_fit)
1
Purpose
g02dac performs a general multiple linear regression when the independent variables may be linearly dependent. Parameter estimates, standard errors, residuals and influence statistics are computed. g02dac may be used to perform a weighted regression.
2
Specification
void 
g02dac (Nag_IncludeMean mean,
Integer n,
const double x[],
Integer tdx,
Integer m,
const Integer sx[],
Integer ip,
const double y[],
const double wt[],
double *rss,
double *df,
double b[],
double se[],
double cov[],
double res[],
double h[],
double q[],
Integer tdq,
Nag_Boolean *svd,
Integer *rank,
double p[],
double tol,
double com_ar[],
NagError *fail) 

The function may be called by the names: g02dac, nag_correg_linregm_fit or nag_regsn_mult_linear.
3
Description
The general linear regression model is defined by
where
 $y$ is a vector of $n$ observations on the dependent variable,
 $X$ is an $n$ by $p$ matrix of the independent variables of column rank $k$,
 $\beta $ is a vector of length $p$ of unknown arguments, and
 $\epsilon $ is a vector of length $n$ of unknown random errors such that $\mathrm{var}\epsilon =V{\sigma}^{2}$, where $V$ is a known diagonal matrix.
Note: the $p$ independent variables may be selected from a set of $m$ potential independent variables.
If $V=I$, the identity matrix, then least squares estimation is used.
If $V\ne I$, then for a given weight matrix $W\propto {V}^{1}$, weighted least squares estimation is used.
The least squares estimates $\hat{\beta}$ of the arguments $\beta $ minimize ${\left(yX\beta \right)}^{\mathrm{T}}\left(yX\beta \right)$ while the weighted least squares estimates minimize ${\left(yX\beta \right)}^{\mathrm{T}}W\left(yX\beta \right)$.
g02dac finds a
$QR$ decomposition of
$X$ (or
${W}^{1/2}X$ in the weighted case), i.e.,
where
${R}^{*}=\left(\begin{array}{c}R\\ 0\end{array}\right)$ and
$R$ is a
$p$ by
$p$ upper triangular matrix and
$Q$ is an
$n$ by
$n$ orthogonal matrix.
If
$R$ is of full rank, then
$\hat{\beta}$ is the solution to
where
$c={Q}^{\mathrm{T}}y$ (or
${Q}^{\mathrm{T}}{W}^{1/2}y$) and
${c}_{1}$ is the first
$p$ elements of
$c$.
If
$R$ is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of
$R$,
where
$D$ is a
$k$ by
$k$ diagonal matrix with nonzero diagonal elements,
$k$ being the rank of
$R$ and
${Q}_{*}$ and
$P$ are
$p$ by
$p$ orthogonal matrices. This gives the solution
${P}_{1}$ being the first
$k$ columns of
$P$, i.e.,
$P=\left({P}_{1}{P}_{0}\right)$ and
${Q}_{{*}_{1}}$ being the first
$k$ columns of
${Q}_{*}$.
Details of the SVD are made available, in the form of the matrix
${P}^{*}$:
This will be only one of the possible solutions. Other estimates may be obtained by applying constraints to the arguments. These solutions can be obtained by using
g02dkc after using
g02dac. Only certain linear combinations of the arguments will have unique estimates; these are known as estimable functions.
The fit of the model can be examined by considering the residuals, ${r}_{i}={y}_{i}\hat{y}$, where $\hat{y}=X\hat{\beta}$ are the fitted values. The fitted values can be written as $Hy$ for an $n$ by $n$ matrix $H$. The $i$th diagonal element of $H$, ${h}_{i}$, gives a measure of the influence of the $i$th value of the independent variables on the fitted regression model. The values ${h}_{i}$ are sometimes known as leverages. Both ${r}_{i}$ and ${h}_{i}$ are provided by g02dac.
The output of g02dac also includes $\hat{\beta}$, the residual sum of squares and associated degrees of freedom, $\left(nk\right)$, the standard errors of the parameter estimates and the variancecovariance matrix of the parameter estimates.
In many linear regression models the first term is taken as a mean term or an intercept, i.e., ${X}_{\mathit{i},1}=1$, for $\mathit{i}=1,2,\dots ,n$. This is provided as an option. Also note that not all the potential independent variables need to be included in a model; a facility to select variables to be included in the model is provided.
Details of the
$QR$ decomposition and, if used, the SVD, are made available. These allow the regression to be updated by adding or deleting an observation using
g02dcc, adding or deleting a variable using
g02dec and
g02dfc or estimating and testing an estimable function using
g02dnc.
4
References
Cook R D and Weisberg S (1982) Residuals and Influence in Regression Chapman and Hall
Draper N R and Smith H (1985) Applied Regression Analysis (2nd Edition) Wiley
Golub G H and Van Loan C F (1996) Matrix Computations (3rd Edition) Johns Hopkins University Press, Baltimore
Hammarling S (1985) The singular value decomposition in multivariate statistics SIGNUM Newsl. 20(3) 2–25
McCullagh P and Nelder J A (1983) Generalized Linear Models Chapman and Hall
Searle S R (1971) Linear Models Wiley
5
Arguments

1:
$\mathbf{mean}$ – Nag_IncludeMean
Input

On entry: indicates if a mean term is to be included.
 ${\mathbf{mean}}=\mathrm{Nag\_MeanInclude}$
 A mean term, (intercept), will be included in the model.
 ${\mathbf{mean}}=\mathrm{Nag\_MeanZero}$
 The model will pass through the origin, zero point.
Constraint:
${\mathbf{mean}}=\mathrm{Nag\_MeanInclude}$ or $\mathrm{Nag\_MeanZero}$.

2:
$\mathbf{n}$ – Integer
Input

On entry: the number of observations, $n$.
Constraint:
${\mathbf{n}}\ge 2$.

3:
$\mathbf{x}\left[{\mathbf{n}}\times {\mathbf{tdx}}\right]$ – const double
Input

On entry: ${\mathbf{x}}\left[\left(\mathit{i}\right)\times {\mathbf{tdx}}+\mathit{j}\right]$ must contain the $\mathit{i}$th observation for the $\mathit{j}$th potential independent variable, for $\mathit{i}=0,1,\dots ,n1$ and $\mathit{j}=0,1,\dots ,m1$.

4:
$\mathbf{tdx}$ – Integer
Input

On entry: the stride separating matrix column elements in the array
x.
Constraint:
${\mathbf{tdx}}\ge {\mathbf{m}}$.

5:
$\mathbf{m}$ – Integer
Input

On entry: the total number of independent variables in the dataset, $m$.
Constraint:
${\mathbf{m}}\ge 1$.

6:
$\mathbf{sx}\left[{\mathbf{m}}\right]$ – const Integer
Input

On entry: indicates which of the potential independent variables are to be included in the model. If
${\mathbf{sx}}\left[j\right]>0$, then the variable contained in the corresponding column of
x is included in the regression model.
Constraints:
 ${\mathbf{sx}}\left[\mathit{j}\right]\ge 0$, for $\mathit{j}=0,1,\dots ,m1$;
 if ${\mathbf{mean}}=\mathrm{Nag\_MeanInclude}$, then exactly ${\mathbf{ip}}1$ values of sx must be $>0$;
 if ${\mathbf{mean}}=\mathrm{Nag\_MeanZero}$, then exactly ip values of sx must be $>0$.

7:
$\mathbf{ip}$ – Integer
Input

On entry: the number $p$ of independent variables in the model, including the mean or intercept if present.
Constraints:
 if ${\mathbf{mean}}=\mathrm{Nag\_MeanInclude}$, $1\le {\mathbf{ip}}\le {\mathbf{m}}+1$;
 if ${\mathbf{mean}}=\mathrm{Nag\_MeanZero}$, $1\le {\mathbf{ip}}\le {\mathbf{m}}$.

8:
$\mathbf{y}\left[{\mathbf{n}}\right]$ – const double
Input

On entry: observations on the dependent variable, $y$.

9:
$\mathbf{wt}\left[{\mathbf{n}}\right]$ – const double
Input

On entry: optionally, the weights to be used in the weighted regression.
If
${\mathbf{wt}}\left[i1\right]=0.0$, then the
$i$th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights. The values of
res and
h will be set to zero for observations with zero weights.
If weights are not provided then
wt must be set to
NULL and the effective number of observations is
n.
Constraint:
if ${\mathbf{wt}}\phantom{\rule{0.25em}{0ex}}\text{is not}\phantom{\rule{0.25em}{0ex}}\mathbf{NULL}$, ${\mathbf{wt}}\left[\mathit{i}1\right]=0.0$, for $\mathit{i}=1,2,\dots ,n$.

On exit: the residual sum of squares for the regression.

11:
$\mathbf{df}$ – double *
Output

On exit: the degrees of freedom associated with the residual sum of squares.

12:
$\mathbf{b}\left[{\mathbf{ip}}\right]$ – double
Output

On exit:
${\mathbf{b}}\left[\mathit{i}\right]$, for
$\mathit{i}=0,1,\dots ,{\mathbf{ip}}1$, contain the least squares estimates of the arguments of the regression model,
$\hat{\beta}$.
If
${\mathbf{mean}}=\mathrm{Nag\_MeanInclude}$, then
${\mathbf{b}}\left[0\right]$ will contain the estimate of the mean argument and
${\mathbf{b}}\left[i\right]$ will contain the coefficient of the variable contained in column
$j$ of
x, where
${\mathbf{sx}}\left[j\right]$ is the
$i$th positive value in the array
sx.
If
${\mathbf{mean}}=\mathrm{Nag\_MeanZero}$, then
${\mathbf{b}}\left[i1\right]$ will contain the coefficient of the variable contained in column
$j$ of
x, where
${\mathbf{sx}}\left[j\right]$ is the
$i$th positive value in the array
sx.

13:
$\mathbf{se}\left[{\mathbf{ip}}\right]$ – double
Output

On exit:
${\mathbf{se}}\left[\mathit{i}\right]$, for
$\mathit{i}=0,1,\dots ,{\mathbf{ip}}1$, contains the standard errors of the
ip parameter estimates given in
b.

14:
$\mathbf{cov}\left[{\mathbf{ip}}\times \left({\mathbf{ip}}+1\right)/2\right]$ – double
Output

On exit: the first
${\mathbf{ip}}\times \left({\mathbf{ip}}+1\right)/2$ elements of
cov contain the upper triangular part of the variancecovariance matrix of the
ip parameter estimates given in
b. They are stored packed by column, i.e., the covariance between the parameter estimate given in
${\mathbf{b}}\left[i\right]$ and the parameter estimate given in
${\mathbf{b}}\left[\mathit{j}\right]$,
$\mathit{j}\ge \mathit{i}$, is stored in
${\mathbf{cov}}\left[\mathit{j}\left(\mathit{j}+1\right)/2+\mathit{i}\right]$, for
$\mathit{i}=0,1,\dots ,{\mathbf{ip}}1$ and
$\mathit{j}=i,\dots ,{\mathbf{ip}}1$.

15:
$\mathbf{res}\left[{\mathbf{n}}\right]$ – double
Output

On exit: the (weighted) residuals, ${r}_{i}$.

16:
$\mathbf{h}\left[{\mathbf{n}}\right]$ – double
Output

On exit: the diagonal elements of $H$, ${h}_{i}$, the leverages.

17:
$\mathbf{q}\left[{\mathbf{n}}\times {\mathbf{tdq}}\right]$ – double
Output

Note: the $\left(i,j\right)$th element of the matrix $Q$ is stored in ${\mathbf{q}}\left[\left(i1\right)\times {\mathbf{tdq}}+j1\right]$.
On exit: the results of the
$QR$ decomposition: the first column of
q contains
$c$, the upper triangular part of columns 2 to
${\mathbf{ip}}+1$ contain the
$R$ matrix, the strictly lower triangular part of columns 2 to
${\mathbf{ip}}+1$ contain details of the
$Q$ matrix.

18:
$\mathbf{tdq}$ – Integer
Input

On entry: the stride separating matrix column elements in the array
q.
Constraint:
${\mathbf{tdq}}\ge {\mathbf{ip}}+1$.

19:
$\mathbf{svd}$ – Nag_Boolean *
Output

On exit: if a singular value decomposition has been performed then
svd will be Nag_TRUE, otherwise
svd will be Nag_FALSE.

20:
$\mathbf{rank}$ – Integer *
Output

On exit: the rank of the independent variables.
If ${\mathbf{svd}}=\mathrm{Nag\_FALSE}$, ${\mathbf{rank}}={\mathbf{ip}}$.
If
${\mathbf{svd}}=\mathrm{Nag\_TRUE}$,
rank is an estimate of the rank of the independent variables.
rank is calculated as the number of singular values greater than
tol (largest singular value). It is possible for the SVD to be carried out but
rank to be returned as
ip.

21:
$\mathbf{p}\left[2\times {\mathbf{ip}}+{\mathbf{ip}}\times {\mathbf{ip}}\right]$ – double
Output

On exit: details of the
$QR$ decomposition and SVD if used.
If
${\mathbf{svd}}=\mathrm{Nag\_FALSE}$, only the first
ip elements of
p are used, these will contain details of the Householder vector in the
$QR$ decomposition (see
Sections 2.2.1 and
3.4.6 in the
F08 Chapter Introduction).
If
${\mathbf{svd}}=\mathrm{Nag\_TRUE}$, the first
ip elements of
p will
contain details of the Householder vector in the
$QR$ decomposition and the next
ip elements of
p contain singular values. The following
ip by
ip elements contain the matrix
${P}^{*}$ stored by rows.

22:
$\mathbf{tol}$ – double
Input

On entry: the value of
tol is used to decide what is the rank of the independent variables. The smaller the value of
tol the stricter the criterion for selecting the singular value decomposition. If
${\mathbf{tol}}=0.0$, then the singular value decomposition will never be used, this may cause run time errors or inaccurate results if the independent variables are not of full rank.
Suggested value:
${\mathbf{tol}}=0.000001$.
Constraint:
${\mathbf{tol}}\ge 0.0$.

23:
$\mathbf{com\_ar}\left[{\mathbf{ip}}+{\mathbf{ip}}\times {\mathbf{ip}}\right]$ – double
Output

On exit: if
${\mathbf{svd}}=\mathrm{Nag\_TRUE}$,
com_ar contains information which is needed by
g02dgc.

24:
$\mathbf{fail}$ – NagError *
Input/Output

The NAG error argument (see
Section 7 in the Introduction to the NAG Library CL Interface).
6
Error Indicators and Warnings
 NE_2_INT_ARG_LT

On entry, ${\mathbf{n}}=\u2329\mathit{\text{value}}\u232a$ while ${\mathbf{ip}}=\u2329\mathit{\text{value}}\u232a$. These arguments must satisfy ${\mathbf{n}}\ge {\mathbf{ip}}$.
On entry, ${\mathbf{tdq}}=\u2329\mathit{\text{value}}\u232a$ while ${\mathbf{ip}}+1=\u2329\mathit{\text{value}}\u232a$. These arguments must satisfy ${\mathbf{tdq}}\ge {\mathbf{ip}}+1$.
On entry, ${\mathbf{tdx}}=\u2329\mathit{\text{value}}\u232a$ while ${\mathbf{m}}=\u2329\mathit{\text{value}}\u232a$. These arguments must satisfy ${\mathbf{tdx}}\ge {\mathbf{m}}$.
 NE_ALLOC_FAIL

Dynamic memory allocation failed.
 NE_BAD_PARAM

On entry, argument
mean had an illegal value.
 NE_BAD_SX_OR_IP

Either a value of
sx is
$<0$, or
ip is incompatible with
mean and
sx, or
${\mathbf{ip}}>$ the effective number of observations.
 NE_INT_ARG_LT

On entry, ${\mathbf{ip}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{ip}}\ge 1$.
On entry, ${\mathbf{m}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{m}}\ge 1$.
On entry, ${\mathbf{n}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{n}}\ge 2$.
On entry, ${\mathbf{sx}}\left[\u2329\mathit{\text{value}}\u232a\right]$ must not be less than 0: ${\mathbf{sx}}\left[\u2329\mathit{\text{value}}\u232a\right]=\u2329\mathit{\text{value}}\u232a$.
 NE_REAL_ARG_LT

On entry,
tol must not be less than 0.0:
${\mathbf{tol}}=\u2329\mathit{\text{value}}\u232a$.
On entry, ${\mathbf{wt}}\left[\u2329\mathit{\text{value}}\u232a\right]$ must not be less than 0.0: ${\mathbf{wt}}\left[\u2329\mathit{\text{value}}\u232a\right]=\u2329\mathit{\text{value}}\u232a$.
 NE_SVD_NOT_CONV

The singular value decomposition has failed to converge.
 NE_ZERO_DOF_RESID

The degrees of freedom for the residuals are zero, i.e., the designated number of arguments $\text{}=\text{}$ the effective number of observations. In this case the parameter estimates will be returned along with the diagonal elements of $H$, but neither standard errors nor the variancecovariance matrix will be calculated.
7
Accuracy
The accuracy of this function is closely related to the accuracy of the $QR$ decomposition.
8
Parallelism and Performance
g02dac is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
g02dac makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the
X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this function. Please also consult the
Users' Note for your implementation for any additional implementationspecific information.
Function
g02fac can be used to compute standardized residuals and further measures of influence.
g02dac requires, in particular, the results stored in
res and
h.
9.1
Internal Changes
Internal changes have been made to this function as follows:
 At Mark 26.1: The documented minimum length of the array argument com_ar was too large. The documented minimum length was given as ${\mathbf{ip}}\times {\mathbf{ip}}\times 5\times \left({\mathbf{ip}}1\right)$ but the actual minimum length is ${\mathbf{ip}}\times {\mathbf{ip}}+{\mathbf{ip}}$ which is much less for nontrivial cases, ${\mathbf{ip}}>1$.
In addition, provided example programs that called g02dac allocated lengths of ${\mathbf{ip}}\times {\mathbf{ip}}+5\times \left({\mathbf{ip}}1\right)$ for the array argument, which was also larger than necessary for nontrivial problems.
The
g02dac routine document has been updated to document the actual minimum length requirement for
com_ar, and those example programs that call
g02dac have been updated to allocate the actual minimum length required for
com_ar.
For details of all known issues which have been reported for the NAG Library please refer to the
Known Issues.
10
Example
For this function two examples are presented. There is a single example program for g02dac, with a main program and the code to solve the two example problems is given in the functions ex1 and ex2.
Example 1 (ex1)
Data from an experiment with four treatments and three observations per treatment are read in. The treatments are represented by dummy $\left(01\right)$ variables. An unweighted model is fitted with a mean included in the model.
Example 2 (ex2)
This example program uses
g02dac to find the coefficient of the
$n$ degree polynomial
that fits the data,
$p\left(x\left(i\right)\right)$ to
$y\left(i\right)$, in a least squares sense.
In this example g02dac is called with both ${\mathbf{mean}}=\mathrm{Nag\_MeanInclude}$ and ${\mathbf{mean}}=\mathrm{Nag\_MeanZero}$. The polynomial degree, the number of data points and the tolerance can be modified using the example data file.
10.1
Program Text
10.2
Program Data
10.3
Program Results