NAG Toolbox: nag_correg_ridge_opt (g02ka)

Purpose

Syntax

Description

References

Parameters

Compulsory Input Parameters

1:     $\mathrm{x}\left(\mathit{ldx},\mathbf{m}\right)$ – double array

ldx, the first dimension of the array, must satisfy the constraint $\mathit{ldx}\ge \mathbf{n}$.

The values of independent variables in the data matrix $X$.

2:     $\mathrm{isx}\left(\mathbf{m}\right)$ – int64int32nag_int array

Indicates which $m$ independent variables are included in the model.

$\mathbf{isx}\left(j\right)=1$

The $j$th variable in x will be included in the model.

$\mathbf{isx}\left(j\right)=0$

Variable $j$ is excluded.

Constraint: $\mathbf{isx}\left(\mathit{j}\right)=0\text{ or }1$, for $\mathit{j}=1,2,\dots ,\mathbf{m}$.

3:     $\mathrm{ip}$ – int64int32nag_int scalar

$m$, the number of independent variables in the model.

Constraints:

• $1\le \mathbf{ip}\le \mathbf{m}$;
• Exactly ip elements of isx must be equal to $1$.

4:     $\mathrm{y}\left(\mathbf{n}\right)$ – double array

The $n$ values of the dependent variable $y$.

5:     $\mathrm{h}$ – double scalar

An initial value for the ridge regression parameter $h$; used as a starting point for the optimization.

Constraint: $\mathbf{h}>0.0$.

6:     $\mathrm{opt}$ – int64int32nag_int scalar

The measure of prediction error used to optimize the ridge regression parameter $h$. The value of opt must be set equal to one of:

$\mathbf{opt}=1$

Generalized cross-validation (GCV);

$\mathbf{opt}=2$

Unbiased estimate of variance (UEV)

$\mathbf{opt}=3$

Future prediction error (FPE)

$\mathbf{opt}=4$

Bayesian information criteron (BIC).

Constraint: $\mathbf{opt}=1$, $2$, $3$ or $4$.

7:     $\mathrm{niter}$ – int64int32nag_int scalar

The maximum number of iterations allowed to optimize the ridge regression parameter $h$.

Constraint: $\mathbf{niter}\ge 1$.

8:     $\mathrm{tol}$ – double scalar

Iterations of the ridge regression parameter $h$ will halt when consecutive values of $h$ lie within tol.

Constraint: $\mathbf{tol}>0.0$.

9:     $\mathrm{orig}$ – int64int32nag_int scalar

If $\mathbf{orig}=1$, the parameter estimates $b$ are calculated for the original data; otherwise $\mathbf{orig}=2$ and the parameter estimates $\tilde{b}$ are calculated for the standardized data.

Constraint: $\mathbf{orig}=1$ or $2$.

10:   $\mathrm{optloo}$ – int64int32nag_int scalar

If $\mathbf{optloo}=2$, the leave-one-out cross-validation estimate of prediction error is calculated; otherwise no such estimate is calculated and $\mathbf{optloo}=1$.

Constraint: $\mathbf{optloo}=1$ or $2$.

Optional Input Parameters

1:     $\mathrm{n}$ – int64int32nag_int scalar

Default: the dimension of the array y and the first dimension of the array x. (An error is raised if these dimensions are not equal.)

$n$, the number of observations.

Constraint: $\mathbf{n}>1$.

2:     $\mathrm{m}$ – int64int32nag_int scalar

Default: the dimension of the array isx and the second dimension of the array x. (An error is raised if these dimensions are not equal.)

The number of independent variables available in the data matrix $X$.

Constraint: $\mathbf{m}\le \mathbf{n}$.

3:     $\mathrm{tau}$ – double scalar

Default: $\mathbf{tau}=0.0$

Singular values less than tau of the SVD of the data matrix $X$ will be set equal to zero.

Constraint: $\mathbf{tau}\ge 0.0$.

Output Parameters

1:     $\mathrm{h}$ – double scalar

h is the optimized value of the ridge regression parameter $h$.

2:     $\mathrm{niter}$ – int64int32nag_int scalar

The number of iterations used to optimize the ridge regression parameter $h$ within tol.

3:     $\mathrm{nep}$ – double scalar

The number of effective parameters, $\gamma$, in the model.

4:     $\mathrm{b}\left(\mathbf{ip}+1\right)$ – double array

Contains the intercept and parameter estimates for the fitted ridge regression model in the order indicated by isx. The first element of b contains the estimate for the intercept; $\mathbf{b}\left(\mathit{j}+1\right)$ contains the parameter estimate for the $\mathit{j}$th independent variable in the model, for $\mathit{j}=1,2,\dots ,\mathbf{ip}$.

5:     $\mathrm{vif}\left(\mathbf{ip}\right)$ – double array

The variance inflation factors in the order indicated by isx. For the $\mathit{j}$th independent variable in the model, $\mathbf{vif}\left(\mathit{j}\right)$ is the value of $v_{\mathit{j}}$, for $\mathit{j}=1,2,\dots ,\mathbf{ip}$.

6:     $\mathrm{res}\left(\mathbf{n}\right)$ – double array

$\mathbf{res}\left(\mathit{i}\right)$ is the value of the $\mathit{i}$th residual for the fitted ridge regression model, for $\mathit{i}=1,2,\dots ,\mathbf{n}$.

7:     $\mathrm{rss}$ – double scalar

The sum of squares of residual values.

8:     $\mathrm{df}$ – int64int32nag_int scalar

The degrees of freedom for the residual sum of squares rss.

9:     $\mathrm{perr}\left(5\right)$ – double array

The first four elements contain, in this order, the measures of prediction error: GCV, UEV, FPE and BIC.

If $\mathbf{optloo}=2$, $\mathbf{perr}\left(5\right)$ is the LOOCV estimate of prediction error; otherwise $\mathbf{perr}\left(5\right)$ is not referenced.

10:   $\mathrm{ifail}$ – int64int32nag_int scalar

$\mathbf{ifail}=\mathbf{0}$ unless the function detects an error (see Error Indicators and Warnings).

Error Indicators and Warnings

   $\mathbf{ifail}=1$

Constraint: $\mathbf{h}>0.0$.

Constraint: $\mathbf{n}>1$.

Constraint: $\mathbf{niter}\ge 1$.

Constraint: $\mathbf{opt}=1$, $2$, $3$ or $4$.

Constraint: $\mathbf{optloo}=1$ or $2$.

Constraint: $\mathbf{orig}=1$ or $2$.

Constraint: $\mathbf{tau}\ge 0.0$.

Constraint: $\mathbf{tol}>0.0$.

   $\mathbf{ifail}=2$

Constraint: $1\le \mathbf{ip}\le \mathbf{m}$.

Constraint: $\mathbf{isx}\left(j\right)=0$ or $1$.

Constraint: $\mathit{ldx}\ge \mathbf{n}$.

Constraint: $\mathbf{m}\le \mathbf{n}$.

Constraint: $\mathrm{sum}\left(\mathbf{isx}\right)=\mathbf{ip}$.

   $\mathbf{ifail}=3$

SVD failed to converge.

   $\mathbf{ifail}=-1$

Maximum number of iterations used.

   $\mathbf{ifail}=-99$

An unexpected error has been triggered by this routine. Please contact NAG.

   $\mathbf{ifail}=-399$

Your licence key may have expired or may not have been installed correctly.

   $\mathbf{ifail}=-999$

Dynamic memory allocation failed.

Accuracy

Further Comments

Example

Open in the MATLAB editor: g02ka_example

function g02ka_example


fprintf('g02ka example results\n\n');

% Data
x = [19.5, 43.1, 29.1;
     24.7, 49.8, 28.2;
     30.7, 51.9, 37.0;
     29.8, 54.3, 31.1;
     19.1, 42.2, 30.9;
     25.6, 53.9, 23.7;
     31.4, 58.5, 27.6;
     27.9, 52.1, 30.6;
     22.1, 49.9, 23.2;
     25.5, 53.5, 24.8;
     31.1, 56.6, 30.0;
     30.4, 56.7, 28.3;
     18.7, 46.5, 23.0;
     19.7, 44.2, 28.6;
     14.6, 42.7, 21.3;
     29.5, 54.4, 30.1;
     27.7, 55.3, 25.7;
     30.2, 58.6, 24.6;
     22.7, 48.2, 27.1;
     25.2, 51.0, 27.5];

[n,m] = size(x);
isx   = ones(m,1,'int64');
ip    = int64(m);
y = [11.9; 22.8; 18.7; 20.1; 12.9; 21.7; 27.1; 25.4; 21.3; 19.3;
     25.4; 27.2; 11.7; 17.8; 12.8; 23.9; 22.6; 25.4; 14.8; 21.1];

% Parameters
h      = 0.5;
opt    = int64(1);
niter  = int64(25);
tol    = 0.0001;
orig   = int64(2);
optloo = int64(2);

% Fit ridge regression model
[h, niter, nep, b, vif, res, rss, df, perr, ifail] = ...
  g02ka( ...
         x, isx, ip, y, h, opt, niter, tol, orig, optloo);

% Display results
fprintf('Value of ridge parameter      : %10.4f\n\n', h);
fprintf('Sum of squares of residuals   : %14.4e\n', rss);
fprintf('Degrees of freedom            : %5d\n', df);
fprintf('Number of effective parameters: %10.4f\n', nep);
fprintf('\nParameter estimates\n');
ivar = double([1:ip+1]);
fprintf('%4d%11.4f\n',[ivar; b(ivar)']);
fprintf('\nNumber of iterations: %15d\n\n', niter);
if opt==1
  fprintf('Ridge parameter minimises GCV\n');
elseif opt==2
  fprintf('Ridge parameter minimises UEV\n');
elseif opt==3
  fprintf('Ridge parameter minimises FPE\n');
elseif opt==4
  fprintf('Ridge parameter minimises BIC\n');
end
fprintf('\nEstimated prediction errors:\n');
fprintf('GCV    = %10.4f\n', perr(1));
fprintf('UEV    = %10.4f\n', perr(2));
fprintf('FPE    = %10.4f\n', perr(3));
fprintf('BIC    = %10.4f\n', perr(4));
if optloo==2
  fprintf('LOO CV = %10.4f\n', perr(5));
end
fprintf('\nResiduals\n');
ivar = [1:n];
fprintf('%4d%11.4f\n',[ivar; res(ivar)']);
fprintf('\nVariance inflation factors\n');
ivar = double([1:ip]);
fprintf('%4d%11.4f\n',[ivar; vif(ivar)']);

g02ka example results

Value of ridge parameter      :     0.0712

Sum of squares of residuals   :     1.0917e+02
Degrees of freedom            :    16
Number of effective parameters:     2.9059

Parameter estimates
   1    20.1950
   2     9.7934
   3     9.9576
   4    -2.0125

Number of iterations:               6

Ridge parameter minimises GCV

Estimated prediction errors:
GCV    =     7.4718
UEV    =     6.3862
FPE    =     7.3141
BIC    =     8.2380
LOO CV =     7.5495

Residuals
   1    -1.9894
   2     3.5469
   3    -3.0392
   4    -3.0309
   5    -0.1899
   6    -0.3146
   7     0.9775
   8     4.0157
   9     2.5332
  10    -2.3560
  11     0.5446
  12     2.3989
  13    -4.0876
  14     3.2778
  15     0.2894
  16     0.7330
  17    -0.7116
  18    -0.6092
  19    -2.9995
  20     1.0110

Variance inflation factors
   1     0.2928
   2     0.4162
   3     0.8089