# NAG C Library Function Document

## 1Purpose

nag_opt_lsq_covariance (e04ycc) returns estimates of elements of the variance-covariance matrix of the estimated regression coefficients for a nonlinear least squares problem. The estimates are derived from the Jacobian of the function $f\left(x\right)$ at the solution.
nag_opt_lsq_covariance (e04ycc) may be used following either of the NAG C Library nonlinear least squares functions nag_opt_lsq_no_deriv (e04fcc), nag_opt_lsq_deriv (e04gbc).

## 2Specification

 #include #include
 void nag_opt_lsq_covariance (Integer job, Integer m, Integer n, double fsumsq, double cj[], Nag_E04_Opt *options, NagError *fail)

## 3Description

nag_opt_lsq_covariance (e04ycc) is intended for use when the nonlinear least squares function, $F\left(x\right)={f}^{\mathrm{T}}\left(x\right)f\left(x\right)$, represents the goodness-of-fit of a nonlinear model to observed data. It assumes that the Hessian of $F\left(x\right)$, at the solution, can be adequately approximated by $2{J}^{\mathrm{T}}J$, where $J$ is the Jacobian of $f\left(x\right)$ at the solution. The estimated variance-covariance matrix $C$ is then given by
 $C = σ 2 JT J -1 JT J ​ nonsingular,$
where ${\sigma }^{2}$ is the estimated variance of the residual at the solution, $\stackrel{-}{x}$, given by
 $σ 2 = F x - m-n ,$
$m$ being the number of observations and $n$ the number of variables.
The diagonal elements of $C$ are estimates of the variances of the estimated regression coefficients. See the e04 Chapter Introduction, Bard (1974) and Wolberg (1967) for further information on the use of the matrix $C$.
When ${J}^{\mathrm{T}}J$ is singular then $C$ is taken to be
 $C = σ 2 JT J † ,$
where ${\left({J}^{\mathrm{T}}J\right)}^{†}$ is the pseudo-inverse of ${J}^{\mathrm{T}}J$, and ${\sigma }^{2}=\frac{F\left(\stackrel{-}{x}\right)}{m-k},k=\mathrm{rank}\left(J\right)$ but in this case the argument fail is returned with ${\mathbf{fail}}\mathbf{.}\mathbf{code}={\mathbf{NW_LIN_DEPEND}}$ as a warning to you that $J$ has linear dependencies in its columns. The assumed rank of $J$ can be obtained from ${\mathbf{fail}}\mathbf{.}\mathbf{errnum}$.
The function can be used to find either the diagonal elements of $C$, or the elements of the $j$th column of $C$, or the whole of $C$.
nag_opt_lsq_covariance (e04ycc) must be preceded by one of the nonlinear least squares functions mentioned in Section 1, and requires the arguments fsumsq and options to be supplied by those functions. fsumsq is the residual sum of squares $F\left(\stackrel{-}{x}\right)$ while the structure options contains the members $\mathbf{options}\mathbf{\to }\mathbf{s}$ and $\mathbf{options}\mathbf{\to }\mathbf{v}$ which give the singular values and right singular vectors respectively in the singular value decomposition of $J$.
Bard Y (1974) Nonlinear Parameter Estimation Academic Press
Wolberg J R (1967) Prediction Analysis Van Nostrand

## 5Arguments

1:    $\mathbf{job}$IntegerInput
On entry: indicates which elements of $C$ are returned as follows:
${\mathbf{job}}=-1$
The $n$ by $n$ symmetric matrix $C$ is returned.
${\mathbf{job}}=0$
The diagonal elements of $C$ are returned.
${\mathbf{job}}>0$
The elements of column job of $C$ are returned.
Constraint: $-1\le {\mathbf{job}}\le {\mathbf{n}}$.
2:    $\mathbf{m}$IntegerInput
On entry: the number $m$ of observations (residuals ${f}_{i}\left(x\right)$).
Constraint: ${\mathbf{m}}\ge {\mathbf{n}}$.
3:    $\mathbf{n}$IntegerInput
On entry: the number $n$ of variables $\left({x}_{j}\right)$.
Constraint: $1\le {\mathbf{n}}\le {\mathbf{m}}$.
4:    $\mathbf{fsumsq}$doubleInput
On entry: the sum of squares of the residuals, $F\left(\stackrel{-}{x}\right)$, at the solution $\stackrel{-}{x}$, as returned by the nonlinear least squares function.
Constraint: ${\mathbf{fsumsq}}\ge 0.0$.
5:    $\mathbf{cj}\left[{\mathbf{n}}\right]$doubleOutput
On exit: with ${\mathbf{job}}=0$, cj returns the $n$ diagonal elements of $C$. With ${\mathbf{job}}=j>0$, cj returns the $n$ elements of the $j$th column of $C$. When ${\mathbf{job}}=-1$, cj is not referenced.
6:    $\mathbf{options}$Nag_E04_Opt *Input/Output
On entry/exit: the structure used in the call to the nonlinear least squares function. The following members are relevant to nag_opt_lsq_covariance (e04ycc), their values should not be altered between the call to the least squares function and the call to nag_opt_lsq_covariance (e04ycc).
sdoubleInput
On entry: the pointer to the $n$ singular values of the Jacobian as returned by the nonlinear least squares function.
vdoubleInput/Output
On entry: the pointer to the $n$ by $n$ right-hand orthogonal matrix (the right singular vectors) of $J$ as returned by the nonlinear least squares function.
On exit: when ${\mathbf{job}}\ge 0$ then $\mathbf{v}$ is unchanged.
When ${\mathbf{job}}=-1$ then the leading $n$ by $n$ part of $\mathbf{v}$ is overwritten by the $n$ by $n$ matrix $C$. Matrix element $i,j$ is held in $\mathbf{v}\left[\left(i-1\right)×\mathbf{tdv}+j-1\right]$ for $i=1,2,\dots ,n$ and $j=1,2,\dots ,n$.
tdvIntegerInput
On entry: the trailing dimension used by $\mathbf{v}$.
7:    $\mathbf{fail}$NagError *Input/Output
The NAG error argument (see Section 3.7 in How to Use the NAG Library and its Documentation).

## 6Error Indicators and Warnings

NE_2_INT_ARG_GT
On entry, ${\mathbf{job}}=〈\mathit{\text{value}}〉$ while ${\mathbf{n}}=〈\mathit{\text{value}}〉$. These arguments must satisfy ${\mathbf{job}}\le {\mathbf{n}}$.
NE_2_INT_ARG_LT
On entry, ${\mathbf{m}}=〈\mathit{\text{value}}〉$ while ${\mathbf{n}}=〈\mathit{\text{value}}〉$. These arguments must satisfy ${\mathbf{m}}\ge {\mathbf{n}}$.
NE_INT_ARG_LT
On entry, job must not be less than $-1$: ${\mathbf{job}}=〈\mathit{\text{value}}〉$.
On entry, n must not be less than $1$: ${\mathbf{n}}=〈\mathit{\text{value}}〉$.
NE_REAL_ARG_LT
On entry, fsumsq must not be less than 0.0: ${\mathbf{fsumsq}}=〈\mathit{\text{value}}〉$.
NE_SINGULAR_VALUES
The singular values are all zero, so that at the solution the Jacobian matrix has rank 0.
NW_LIN_DEPEND
At the solution the Jacobian matrix contains linear, or near linear, dependencies amongst its columns. $J$ assumed to have rank $〈\mathit{\text{value}}〉$.
In this case the required elements of $C$ have still been computed based upon $J$ having an assumed rank given by ${\mathbf{fail}}\mathbf{.}\mathbf{errnum}$. The rank is computed by regarding singular values $\mathbf{options}\mathbf{.}\mathbf{s}\left[j\right]$ that are not larger than $10\epsilon ×\mathbf{options}\mathbf{.}\mathbf{s}\left[0\right]$ as zero, where $\epsilon$ is the machine precision (see nag_machine_precision (X02AJC)). If you expect near linear dependencies at the solution and are happy with this tolerance in determining rank you should not call nag_opt_lsq_covariance (e04ycc) with the null pointer NAGERR_DEFAULT as the argument fail but should specifically declare and initialize a NagError structure for the argument fail.
Overflow
If overflow occurs then either an element of $C$ is very large, or the singular values or singular vectors have been incorrectly supplied.

## 7Accuracy

The computed elements of $C$ will be the exact covariances corresponding to a closely neighbouring Jacobian matrix $J$.

## 8Parallelism and Performance

nag_opt_lsq_covariance (e04ycc) is not threaded in any implementation.

When ${\mathbf{job}}=-1$ the time taken by the function is approximately proportional to ${n}^{3}$. When ${\mathbf{job}}\ge 0$ the time taken by the function is approximately proportional to ${n}^{2}$.

## 10Example

This example estimates the variance-covariance matrix $C$ for the least squares estimates of ${x}_{1}$, ${x}_{2}$ and ${x}_{3}$ in the model
 $y = x 1 + t 1 x 2 t 2 + x 3 t 3$
using the 15 sets of data given in the following table:
 $y t 1 t 2 t 3 0.14 1.0 15.0 1.0 0.18 2.0 14.0 2.0 0.22 3.0 13.0 3.0 0.25 4.0 12.0 4.0 0.29 5.0 11.0 5.0 0.32 6.0 10.0 6.0 0.35 7.0 9.0 7.0 0.39 8.0 8.0 8.0 0.37 9.0 7.0 7.0 0.58 10.0 6.0 6.0 0.73 11.0 5.0 5.0 0.96 12.0 4.0 4.0 1.34 13.0 3.0 3.0 2.10 14.0 2.0 2.0 4.39 15.0 1.0 1.0$
The program uses (0.5,1.0,1.5) as the initial guess at the position of the minimum and computes the least squares solution using nag_opt_lsq_no_deriv (e04fcc). Note that the structure options is initialized by nag_opt_init (e04xxc) before calling nag_opt_lsq_no_deriv (e04fcc). See the function documents for nag_opt_lsq_no_deriv (e04fcc), nag_opt_init (e04xxc) and nag_opt_free (e04xzc) for further information.

### 10.1Program Text

Program Text (e04ycce.c)

### 10.2Program Data

Program Data (e04ycce.d)

### 10.3Program Results

Program Results (e04ycce.r)