NAG Library Routine Document
G07GAF identifies outlying values using Peirce's criterion.
|SUBROUTINE G07GAF (
||N, P, Y, MEAN, VAR, IOUT, NIOUT, LDIFF, DIFF, LLAMB, IFAIL)
||N, P, IOUT(N), NIOUT, LDIFF, IFAIL
||Y(N), MEAN, VAR, DIFF(LDIFF), LLAMB(LDIFF)
G07GAF flags outlying values in data using Peirce's criterion. Let
- denote a vector of observations (for example the residuals) obtained from a model with parameters,
- denote the number of potential outlying values,
- and denote the mean and variance of respectively,
- denote a vector of length constructed by dropping the values from
with the largest value of ,
- denote the (unknown) variance of ,
denote the ratio of and with
Peirce's method flags
as a potential outlier if
is obtained from the solution of
is the cumulative distribution function for the standard Normal distribution.
is unknown an assumption is made that the relationship between
, depends only on the sum of squares of the rejected observations and the ratio estimated as
A value for the cutoff
is calculated iteratively. An initial value of
is used and a value of
is estimated using equation (1)
. Equation (3)
is then used to obtain an estimate of
and then equation (2)
is used to get a new estimate for
. This process is then repeated until the relative change in
between consecutive iterations is
is machine precision
By construction, the cutoff for testing for potential outliers is less than the cutoff for testing for potential outliers. Therefore Peirce's criterion is used in sequence with the existence of a single potential outlier being investigated first. If one is found, the existence of two potential outliers is investigated etc.
If one of a duplicate series of observations is flagged as an outlier, then all of them are flagged as outliers.
Gould B A (1855) On Peirce's criterion for the rejection of doubtful observations, with tables for facilitating its application The Astronomical Journal 45
Peirce B (1852) Criterion for the rejection of doubtful observations The Astronomical Journal 45
- 1: N – INTEGERInput
On entry: , the number of observations.
- 2: P – INTEGERInput
On entry: , the number of parameters in the model used in obtaining the . If is an observed set of values, as opposed to the residuals from fitting a model with parameters, then should be set to , i.e., as if a model just containing the mean had been used.
- 3: Y(N) – REAL (KIND=nag_wp) arrayInput
On entry: , the data being tested.
- 4: MEAN – REAL (KIND=nag_wp)Input
, the mean of
, otherwise MEAN
is not referenced and the mean is calculated from the data supplied in Y
- 5: VAR – REAL (KIND=nag_wp)Input
, the variance of
, otherwise the variance is calculated from the data supplied in Y
- 6: IOUT(N) – INTEGER arrayOutput
: the indices of the values in Y
sorted in descending order of the absolute difference from the mean, therefore
- 7: NIOUT – INTEGEROutput
: the number of potential outliers. The indices for these potential outliers are held in the first NIOUT
elements of IOUT
. By construction there can be at most
values flagged as outliers.
- 8: LDIFF – INTEGERInput
: the maximum number of values to be returned in arrays DIFF
, arrays DIFF
are not referenced.
- 9: DIFF(LDIFF) – REAL (KIND=nag_wp) arrayOutput
On exit: holds for observation , for .
- 10: LLAMB(LDIFF) – REAL (KIND=nag_wp) arrayOutput
On exit: holds for observation , for .
- 11: IFAIL – INTEGERInput/Output
must be set to
. If you are unfamiliar with this parameter you should refer to Section 3.3
in the Essential Introduction for details.
For environments where it might be inappropriate to halt program execution when an error is detected, the value
is recommended. If the output of error messages is undesirable, then the value
is recommended. Otherwise, if you are not familiar with this parameter, the recommended value is
. When the value is used it is essential to test the value of IFAIL on exit.
unless the routine detects an error or a warning has been flagged (see Section 6
6 Error Indicators and Warnings
If on entry
, explanatory error messages are output on the current error message unit (as defined by X04AAF
Errors or warnings detected by the routine:
On entry, .
On entry, or .
One problem with Peirce's algorithm as implemented in G07GAF is the assumed relationship between
, the variance using the full dataset, and
, the variance with the potential outliers removed. In some cases, for example if the data
were the residuals from a linear regression, this assumption may not hold as the regression line may change significantly when outlying values have been dropped resulting in a radically different set of residuals. In such cases G07GBF
should be used instead.
This example reads in a series of data and flags any potential outliers.
The dataset used is from Peirce's original paper and consists of fifteen observations on the vertical semidiameter of Venus.
9.1 Program Text
Program Text (g07gafe.f90)
9.2 Program Data
Program Data (g07gafe.d)
9.3 Program Results
Program Results (g07gafe.r)