NAG Library Function Document

nag_robust_m_regsn_user_fn (g02hdc)

double psip0, double beta, Nag_RegType regtype, Nag_SigmaEst sigma_est, Integer n, Integer m, double x[], Integer pdx, double y[], double wgt[], double theta[], Integer *k, double *sigma, double rs[], double tol, double eps, Integer maxit, Integer nitmon, const char *outfile, Integer *nit, Nag_Comm *comm, NagError *fail)

3 Description

For the linear regression model

y = X θ + ε,

where	$y$ is a vector of length $n$ of the dependent variable,
	$X$ is a $n$ by $m$ matrix of independent variables of column rank $k$ ,
	$θ$ is a vector of length $m$ of unknown arguments,
and	$ε$ is a vector of length $n$ of unknown errors with var $(ε_{i}) = σ^{2}$ ,

nag_robust_m_regsn_user_fn (g02hdc) calculates the M-estimates given by the solution,

\hat{θ}

, to the equation

\sum_{i = 1}^{n} ψ (r_{i} / (σ w_{i})) w_{i} x_{i j} = 0, j = 1, 2, \dots, m,

(1)

where	$r_{i}$ is the $i$ th residual, i.e., the $i$ th element of the vector $r = y - X \hat{θ}$ ,
	$ψ$ is a suitable weight function,
	$w_{i}$ are suitable weights such as those that can be calculated by using output from nag_robust_m_regsn_wts (g02hbc),
and	$σ$ may be estimated at each iteration by the median absolute deviation of the residuals $\hat{σ} = {med}_{i} [\|r_{i}\|] / β_{1}$

or as the solution to

\sum_{i = 1}^{n} χ (r_{i} / (\hat{σ} w_{i})) w_{i}^{2} = (n - k) β_{2}

for a suitable weight function

χ

, where

β_{1}

and

β_{2}

are constants, chosen so that the estimator of

σ

is asymptotically unbiased if the errors,

ε_{i}

, have a Normal distribution. Alternatively

σ

may be held at a constant value.

The above describes the Schweppe type regression. If the

w_{i}

are assumed to equal

1

for all

i

, then Huber type regression is obtained. A third type, due to Mallows, replaces (1) by

\sum_{i = 1}^{n} ψ (r_{i} / σ) w_{i} x_{i j} = 0, j = 1, 2, \dots, m .

This may be obtained by use of the transformations

\begin{array}{l} w_{i}^{*} & \leftarrow \sqrt{w_{i}} \\ y_{i}^{*} & \leftarrow y_{i} \sqrt{w_{i}} \\ x_{i j}^{*} & \leftarrow x_{i j} \sqrt{w_{i}}, j = 1, 2, \dots, m \end{array}

(see Marazzi (1987)).

The calculation of the estimates of

θ

can be formulated as an iterative weighted least squares problem with a diagonal weight matrix

G

given by

G_{i i} = \{\begin{array}{cl} \frac{ψ (r_{i} / (σ w_{i}))}{(r_{i} / (σ w_{i}))}, & r_{i} \neq 0 \\ ψ^{'} (0), & r_{i} = 0 . \end{array} .

The value of

θ

at each iteration is given by the weighted least squares regression of

y

X

. This is carried out by first transforming the

y

and

X

\begin{array}{l} {\tilde{y}}_{i} & = y_{i} \sqrt{G_{i i}} \\ {\tilde{x}}_{i j} & = x_{i j} \sqrt{G_{i i}}, j = 1, 2, \dots, m \end{array}

and then using a least squares solver. If

X

is of full column rank then an orthogonal-triangular (

Q R

) decomposition is used; if not, a singular value decomposition is used.

Observations with zero or negative weights are not included in the solution.

Note: there is no explicit provision in the function for a constant term in the regression model. However, the addition of a dummy variable whose value is

1.0

for all observations will produce a value of

\hat{θ}

corresponding to the usual constant term.

nag_robust_m_regsn_user_fn (g02hdc) is based on routines in ROBETH, see Marazzi (1987).

4 References

Hampel F R, Ronchetti E M, Rousseeuw P J and Stahel W A (1986) Robust Statistics. The Approach Based on Influence Functions Wiley

Huber P J (1981) Robust Statistics Wiley

Marazzi A (1987) Subroutines for robust and bounded influence regression in ROBETH Cah. Rech. Doc. IUMSP, No. 3 ROB 2 Institut Universitaire de Médecine Sociale et Préventive, Lausanne

5 Arguments

1: order – Nag_OrderTypeInput

On entry: the order argument specifies the two-dimensional storage scheme being used, i.e., row-major ordering or column-major ordering. C language defined storage is specified by

order = Nag_RowMajor

. See Section 3.2.1.3 in the Essential Introduction for a more detailed explanation of the use of this argument.

Constraint:

order = Nag_RowMajor

or Nag_ColMajor.

2: chi – function, supplied by the userExternal Function

sigma_est = Nag_SigmaChi

, chi must return the value of the weight function

χ

for a given value of its argument. The value of

χ

must be non-negative.

The specification of chi is:

double

chi (double t, Nag_Comm *comm)

1: t – doubleInput

On entry: the argument for which chi must be evaluated.

2: comm – Nag_Comm *

Pointer to structure of type Nag_Comm; the following members are relevant to chi.

user – double *
iuser – Integer *
p – Pointer: The type Pointer will be void *. Before calling nag_robust_m_regsn_user_fn (g02hdc) you may allocate memory and initialize these pointers with various quantities for use by chi when called from nag_robust_m_regsn_user_fn (g02hdc) (see Section 3.2.1 in the Essential Introduction).

chi is required only if

sigma_est = Nag_SigmaConst

, otherwise it can be specified as a pointer with

0

value.

3: psi – function, supplied by the userExternal Function

psi must return the value of the weight function

ψ

for a given value of its argument.

The specification of psi is:

double

psi (double t, Nag_Comm *comm)

1: t – doubleInput

On entry: the argument for which psi must be evaluated.

2: comm – Nag_Comm *

Pointer to structure of type Nag_Comm; the following members are relevant to psi.

user – double *
iuser – Integer *
p – Pointer: The type Pointer will be void *. Before calling nag_robust_m_regsn_user_fn (g02hdc) you may allocate memory and initialize these pointers with various quantities for use by psi when called from nag_robust_m_regsn_user_fn (g02hdc) (see Section 3.2.1 in the Essential Introduction).

4: psip0 – doubleInput

On entry: the value of

ψ' (0)

5: beta – doubleInput

On entry: if

sigma_est = Nag_SigmaRes

, beta must specify the value of

β_{1}

For Huber and Schweppe type regressions,

β_{1}

is the

75

th percentile of the standard Normal distribution (see nag_deviates_normal (g01fac)). For Mallows type regression

β_{1}

is the solution to

\frac{1}{n} \sum_{i = 1}^{n} Φ (β_{1} / \sqrt{w_{i}}) = 0.75,

where

Φ

is the standard Normal cumulative distribution function (see nag_cumul_normal (s15abc)).

sigma_est = Nag_SigmaChi

, beta must specify the value of

β_{2}

\begin{array}{l} β_{2} = & \int_{- \infty}^{\infty} χ (z) ϕ (z) d z, & in the Huber case; \\ β_{2} = & \frac{1}{n} \sum_{i = 1}^{n} w_{i} \int_{- \infty}^{\infty} χ (z) ϕ (z) d z, & in the Mallows case; \\ β_{2} = & \frac{1}{n} \sum_{i = 1}^{n} w_{i}^{2} \int_{- \infty}^{\infty} χ (z / w_{i}) ϕ (z) d z, & in the Schweppe case; \end{array}

where

ϕ

is the standard normal density, i.e.,

\frac{1}{\sqrt{2 π}} \exp (- \frac{1}{2} x^{2})

sigma_est = Nag_SigmaConst

, beta is not referenced.

Constraint: if

sigma_est \neq Nag_SigmaConst

beta > 0.0

6: regtype – Nag_RegTypeInput

On entry: determines the type of regression to be performed.

$regtype = Nag_HuberReg$: Huber type regression.
$regtype = Nag_MallowsReg$: Mallows type regression.
$regtype = Nag_SchweppeReg$: Schweppe type regression.

Constraint:

regtype = Nag_MallowsReg

Nag_HuberReg

Nag_SchweppeReg

7: sigma_est – Nag_SigmaEstInput

On entry: determines how

σ

is to be estimated.

$sigma_est = Nag_SigmaConst$: $σ$ is held constant at its initial value.
$sigma_est = Nag_SigmaRes$: $σ$ is estimated by median absolute deviation of residuals.
$sigma_est = Nag_SigmaChi$: $σ$ is estimated using the $χ$ function.

Constraint:

sigma_est = Nag_SigmaRes

Nag_SigmaConst

Nag_SigmaChi

8: n – IntegerInput

On entry:

n

, the number of observations.

Constraint:

n > 1

9: m – IntegerInput

On entry:

m

, the number of independent variables.

Constraint:

1 \leq m < n

10: x[ $\dim$ ] – doubleInput/Output

Note: the dimension, dim, of the array x must be at least

$\max (1, pdx \times m)$ when $order = Nag_ColMajor$ ;
$\max (1, n \times pdx)$ when $order = Nag_RowMajor$ .

Where

X (i, j)

appears in this document, it refers to the array element

$x [(j - 1) \times pdx + i - 1]$ when $order = Nag_ColMajor$ ;
$x [(i - 1) \times pdx + j - 1]$ when $order = Nag_RowMajor$ .

On entry: the values of the

X

matrix, i.e., the independent variables.

X (i, j)

must contain the

i j

th element of

x

, for

i = 1, 2, \dots, n

and

j = 1, 2, \dots, m

regtype = Nag_MallowsReg

, during calculations the elements of x will be transformed as described in Section 3. Before exit the inverse transformation will be applied. As a result there may be slight differences between the input x and the output x.

On exit: unchanged, except as described above.

11: pdx – IntegerInput

On entry: the stride separating row or column elements (depending on the value of order) in the array x.

Constraints:

if $order = Nag_ColMajor$ , $pdx \geq n$ ;
if $order = Nag_RowMajor$ , $pdx \geq m$ .

12: y[n] – doubleInput/Output

On entry: the data values of the dependent variable.

y [i - 1]

must contain the value of

y

for the

i

th observation, for

i = 1, 2, \dots, n

regtype = Nag_MallowsReg

, during calculations the elements of y will be transformed as described in Section 3. Before exit the inverse transformation will be applied. As a result there may be slight differences between the input y and the output y.

On exit: unchanged, except as described above.

13: wgt[n] – doubleInput/Output

On entry: the weight for the

i

th observation, for

i = 1, 2, \dots, n

regtype = Nag_MallowsReg

, during calculations elements of wgt will be transformed as described in Section 3. Before exit the inverse transformation will be applied. As a result there may be slight differences between the input wgt and the output wgt.

wgt [i - 1] \leq 0

, the

i

th observation is not included in the analysis.

regtype = Nag_HuberReg

, wgt is not referenced.

On exit: unchanged, except as described above.

14: theta[m] – doubleInput/Output

On entry: starting values of the argument vector

θ

. These may be obtained from least squares regression. Alternatively if

sigma_est = Nag_SigmaRes

and

sigma = 1

or if

sigma_est = Nag_SigmaChi

and sigma approximately equals the standard deviation of the dependent variable,

y

, then

theta [i - 1] = 0.0

, for

i = 1, 2, \dots, m

may provide reasonable starting values.

On exit: the M-estimate of

θ_{i}

, for

i = 1, 2, \dots, m

15: k – Integer *Output

On exit: the column rank of the matrix

X

16: sigma – double *Input/Output

On entry: a starting value for the estimation of

σ

. sigma should be approximately the standard deviation of the residuals from the model evaluated at the value of

θ

given by theta on entry.

Constraint:

sigma > 0.0

On exit: the final estimate of

σ

sigma_est \neq Nag_SigmaConst

or the value assigned on entry if

sigma_est = Nag_SigmaConst

17: rs[n] – doubleOutput

On exit: the residuals from the model evaluated at final value of theta, i.e., rs contains the vector

(y - X \hat{θ})

18: tol – doubleInput

On entry: the relative precision for the final estimates. Convergence is assumed when both the relative change in the value of sigma and the relative change in the value of each element of theta are less than tol.

It is advisable for tol to be greater than

100 \times machine precision

Constraint:

tol > 0.0

19: eps – doubleInput

On entry: a relative tolerance to be used to determine the rank of

X

eps < machine precision

eps > 1.0

then machine precision will be used in place of tol.

A reasonable value for eps is

5.0 \times 10^{- 6}

where this value is possible.

20: maxit – IntegerInput

On entry: the maximum number of iterations that should be used during the estimation.

A value of

maxit = 50

should be adequate for most uses.

Constraint:

maxit > 0

21: nitmon – IntegerInput

On entry: determines the amount of information that is printed on each iteration.

$nitmon \leq 0$: No information is printed.
$nitmon > 0$: On the first and every nitmon iterations the values of sigma, theta and the change in theta during the iteration are printed.

22: outfile – const char *Input

On entry: a null terminated character string giving the name of the file to which results should be printed. If

outfile = NULL

or an empty string then the stdout stream is used. Note that the file will be opened in the append mode.

23: nit – Integer *Output

On exit: the number of iterations that were used during the estimation.

24: comm – Nag_Comm *Communication Structure

The NAG communication argument (see Section 3.2.1.1 in the Essential Introduction).

25: fail – NagError *Input/Output

The NAG error argument (see Section 3.6 in the Essential Introduction).

6 Error Indicators and Warnings

NE_ALLOC_FAIL: Dynamic memory allocation failed.
NE_BAD_PARAM: On entry, argument $〈value〉$ had an illegal value.
NE_CHI: Value given by chi function $< 0$ : $chi (〈value〉) = 〈value〉$ .
NE_CONVERGENCE_SOL: Iterations to solve weighted least squares equations failed to converge.
NE_CONVERGENCE_THETA: Iterations to calculate estimates of theta failed to converge in maxit iterations: $maxit = 〈value〉$ .
NE_FULL_RANK: Weighted least squares equations not of full rank: rank $= 〈value〉$ .
NE_INT: On entry, $m = 〈value〉$ .
Constraint: $m \geq 1$ .; On entry, $maxit = 〈value〉$ .
Constraint: $maxit > 0$ .; On entry, $n = 〈value〉$ .
Constraint: $n > 1$ .; On entry, $n = 〈value〉$ .
Constraint: $n \geq 2$ .; On entry, $pdx = 〈value〉$ .
Constraint: $pdx > 0$ .
NE_INT_2: On entry, $m = 〈value〉$ and $n = 〈value〉$ .
Constraint: $1 \leq m < n$ .; On entry, $n = 〈value〉$ and $m = 〈value〉$ .
Constraint: $n > m$ .; On entry, $n = 〈value〉$ and $pdx = 〈value〉$ .
Constraint: $pdx \geq n$ .; On entry, $pdx = 〈value〉$ and $m = 〈value〉$ .
Constraint: $pdx \geq m$ .
NE_INTERNAL_ERROR: An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact NAG for assistance.
NE_NOT_CLOSE_FILE: Cannot close file $〈value〉$ .
NE_NOT_WRITE_FILE: Cannot open file $〈value〉$ for writing.
NE_REAL: On entry, $beta = 〈value〉$ .
Constraint: $beta > 0.0$ .; On entry, $sigma = 〈value〉$ .
Constraint: $sigma > 0.0$ .; On entry, $tol = 〈value〉$ .
Constraint: $tol > 0.0$ .
NE_ZERO_DF: On entry, $n = 〈value〉$ and $k = 〈value〉$ .
Constraint: $n - k > 0$ .
NE_ZERO_VALUE: Estimated value of sigma is zero.

7 Accuracy

The accuracy of the results is controlled by tol.

8 Further Comments

In cases when

sigma_est \neq Nag_SigmaConst

it is important for the value of sigma to be of a reasonable magnitude. Too small a value may cause too many of the winsorized residuals, i.e.,

ψ (r_{i} / σ)

, to be zero, which will lead to convergence problems and may trigger the

fail . code =

NE_FULL_RANK error.

By suitable choice of the functions chi and psi this function may be used for other applications of iterative weighted least squares.

For the variance-covariance matrix of

θ

see nag_robust_m_regsn_param_var (g02hfc).

9 Example

Having input

X

Y

and the weights, a Schweppe type regression is performed using Huber's

ψ

function. The function BETCAL calculates the appropriate value of

β_{2}

NAG Library Function Documentnag_robust_m_regsn_user_fn (g02hdc)

+− Contents

1 Purpose

2 Specification

3 Description

4 References

5 Arguments

6 Error Indicators and Warnings

7 Accuracy

8 Further Comments

9 Example

9.1 Program Text

9.2 Program Data

9.3 Program Results

NAG Library Function Document

nag_robust_m_regsn_user_fn (g02hdc)