naginterfaces.library.correg.linregm_​var_​add

naginterfaces.library.correg.linregm_var_add(q, p, x, wt=None, tol=1e-06)

linregm_var_add adds a new independent variable to a general linear regression model.

For full information please refer to the NAG Library document for g02de

https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/g02/g02def.html

Parameters

qfloat, array-like, shape $(n,\text{ip}+2)$

If $\text{ip}\ne 0$, $q$ must contain the results of the $QR$ decomposition for the model with $p$ parameters as returned by linregm_fit() or a previous call to linregm_var_add.

If $\text{ip}=0$, the first column of $q$ should contain the $n$ values of the dependent variable, $y$.

pfloat, array-like, shape $(\text{ip}+1)$

Contains further details of the $QR$ decomposition used. The first $\text{ip}$ elements of $p$ must contain the zeta values for the $QR$ decomposition (see lapackeig.dgeqrf for details).

The first $\text{ip}$ elements of array $p$ are provided by linregm_fit() or by previous calls to linregm_var_add.

xfloat, array-like, shape $\left(n\right)$

$x$, the new independent variable.

wtNone or float, array-like, shape $\left(n\right)$, optional

If provided $wt$ must contain the weights to be used with the model.

If $wt[i-1]=0.0$, the $i$th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If $wt$ is not provided the effective number of observations is $n$.

tolfloat, optional

The value of $tol$ is used to decide if the new independent variable is linearly related to independent variables already included in the model. If the new variable is linearly related then $c$ is not updated. The smaller the value of $tol$ the stricter the criterion for deciding if there is a linear relationship.

Returns

qfloat, ndarray, shape $(n,\text{ip}+2)$

The results of the $QR$ decomposition for the model with $p+1$ parameters:

the first column of $q$ contains the updated value of $c$;

the columns $2$ to $\text{ip}+1$ are unchanged;

the first $\text{ip}+1$ elements of column $\text{ip}+2$ contain the new column of $R$, while the remaining $n-\text{ip}-1$ elements contain details of the matrix $Q_{p+1}$.

pfloat, ndarray, shape $(\text{ip}+1)$

The first $\text{ip}$ elements of $p$ are unchanged and the $(\text{ip}+1)$th element contains the zeta value for $Q_{p+1}$.

rssfloat

The residual sum of squares for the new fitted model.

Note: this will only be valid if the model is of full rank, see Further Comments.

Raises

NagValueError

(errno $1$)

On entry, $tol=⟨value⟩$.

Constraint: $tol>0.0$.

(errno $1$)

On entry, $\text{ip}=⟨value⟩$ and $n=⟨value⟩$.

Constraint: $\text{ip}<n$.

(errno $1$)

On entry, $\text{weight}=⟨value⟩$.

Constraint: $\text{weight}=\text{'W'}$ or $\text{'U'}$.

(errno $1$)

On entry, $\text{ip}=⟨value⟩$.

Constraint: $\text{ip}\ge 0$.

(errno $1$)

On entry, $n=⟨value⟩$.

Constraint: $n\ge 1$.

(errno $2$)

On entry, $wt[⟨value⟩]<0.0$.

Constraint: $wt[i-1]\ge 0.0$, for $i=1,2,\dots ,n$.

Warns

NagAlgorithmicWarning

(errno $3$)

$x$ variable is a linear combination of existing model terms.

Notes

In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

A linear regression model may be built up by adding new independent variables to an existing model. linregm_var_add updates the $QR$ decomposition used in the computation of the linear regression model. The $QR$ decomposition may come from linregm_fit() or a previous call to linregm_var_add. The general linear regression model is defined by

$$y=X\beta +ϵ\text{,}$$

where	$y$ is a vector of $n$ observations on the dependent variable,
	$X$ is an $n\times p$ matrix of the independent variables of column rank $k$,
	$\beta$ is a vector of length $p$ of unknown parameters,
and	$ϵ$ is a vector of length $n$ of unknown random errors such that $var\left(ϵ\right)=V{\sigma }^2$, where $V$ is a known diagonal matrix.

If $V=I$, the identity matrix, then least squares estimation is used. If $V\ne I$, then for a given weight matrix $W\propto V^{-1}$, weighted least squares estimation is used.

The least squares estimates, $\hat{\beta }$ of the parameters $\beta$ minimize ${(y-X\beta )}^T(y-X\beta )$ while the weighted least squares estimates, minimize ${(y-X\beta )}^{T}W(y-X\beta )$.

The parameter estimates may be found by computing a $QR$ decomposition of $X$ (or $W^{\frac{1}{2}}X$ in the weighted case), i.e.,

$$X=Q{R}^{\ast }(\text{or}{W}^{\frac{1}{2}}X=Q{R}^{\ast })\text{,}$$

where $R^{\ast }=\left(\begin{array}{c}R\\ 0\end{array}\right)$ and $R$ is a $p\times p$ upper triangular matrix and $Q$ is an $n\times n$ orthogonal matrix.

If $R$ is of full rank, then $\hat{\beta }$ is the solution to

$$R\hat{\beta }=c_1\text{,}$$

where $c=Q^{T}y$ (or $Q^T{W}^{\frac{1}{2}}y$) and $c_1$ is the first $p$ elements of $c$.

If $R$ is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of $R$.

To add a new independent variable, $x_{p+1}$, $R$ and $c$ have to be updated. The matrix $Q_{p+1}$ is found such that $Q_{p+1}^T[R:Q^T{x}_{p+1}]$ (or $Q_{p+1}^T[R:Q^T{W}^{\frac{1}{2}}{x}_{p+1}]$) is upper triangular. The vector $c$ is then updated by multiplying by $Q_{p+1}^T$.

The new independent variable is tested to see if it is linearly related to the existing independent variables by checking that at least one of the values ${\left(Q^T{x}_{p+1}\right)}_{\text{i}}$, for $\text{i}=p+2,\dots ,n$, is nonzero.

The new parameter estimates, $\hat{\beta }$, can then be obtained by a call to linregm_update().

The function can be used with $p=0$, in which case $R$ and $c$ are initialized.

References

Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley

Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2–25

McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall

Searle, S R, 1971, Linear Models, Wiley