g02buf calculates the sample means and sums of squares and cross-products, or sums of squares and cross-products of deviations from the mean, in a single pass for a set of data. The data may be weighted.
The routine may be called by the names g02buf or nagf_correg_ssqmat.
3Description
g02buf is an adaptation of West's WV2 algorithm; see West (1979). This routine calculates the (optionally weighted) sample means and (optionally weighted) sums of squares and cross-products or sums of squares and cross-products of deviations from the (weighted) mean for a sample of $n$ observations on $m$ variables ${X}_{j}$, for $\mathit{j}=1,2,\dots ,m$. The algorithm makes a single pass through the data.
For the first $i-1$ observations let the mean of the $j$th variable be ${\overline{x}}_{j}(i-1)$, the cross-product about the mean for the $j$th and $k$th variables be ${c}_{jk}(i-1)$ and the sum of weights be ${W}_{i-1}$. These are updated by the $i$th observation, ${x}_{ij}$, for $\mathit{j}=1,2,\dots ,m$, with weight ${w}_{i}$ as follows:
The algorithm is initialized by taking ${\overline{x}}_{j}\left(1\right)={x}_{1j}$, the first observation, and ${c}_{ij}\left(1\right)=0.0$.
For the unweighted case ${w}_{i}=1$ and ${W}_{i}=i$ for all $i$.
Note that only the upper triangle of the matrix is calculated and returned packed by column.
4References
Chan T F, Golub G H and Leveque R J (1982) Updating Formulae and a Pairwise Algorithm for Computing Sample Variances Compstat, Physica-Verlag
West D H D (1979) Updating mean and variance estimates: An improved method Comm. ACM22 532–555
5Arguments
1: $\mathbf{mean}$ – Character(1)Input
On entry: indicates whether g02buf is to calculate sums of squares and cross-products, or sums of squares and cross-products of deviations about the mean.
${\mathbf{mean}}=\text{'M'}$
The sums of squares and cross-products of deviations about the mean are calculated.
${\mathbf{mean}}=\text{'Z'}$
The sums of squares and cross-products are calculated.
Constraint:
${\mathbf{mean}}=\text{'M'}$ or $\text{'Z'}$.
2: $\mathbf{weight}$ – Character(1)Input
On entry: indicates whether the data is weighted or not.
${\mathbf{weight}}=\text{'U'}$
The calculations are performed on unweighted data.
${\mathbf{weight}}=\text{'W'}$
The calculations are performed on weighted data.
Constraint:
${\mathbf{weight}}=\text{'W'}$ or $\text{'U'}$.
3: $\mathbf{n}$ – IntegerInput
On entry: $n$, the number of observations in the dataset.
Constraint:
${\mathbf{n}}\ge 1$.
4: $\mathbf{m}$ – IntegerInput
On entry: $m$, the number of variables.
Constraint:
${\mathbf{m}}\ge 1$.
5: $\mathbf{x}({\mathbf{ldx}},{\mathbf{m}})$ – Real (Kind=nag_wp) arrayInput
On entry: ${\mathbf{x}}(\mathit{i},\mathit{j})$ must contain the $\mathit{i}$th observation on the $\mathit{j}$th variable, for $\mathit{i}=1,2,\dots ,n$ and $\mathit{j}=1,2,\dots ,m$.
6: $\mathbf{ldx}$ – IntegerInput
On entry: the first dimension of the array x as declared in the (sub)program from which g02buf is called.
Constraint:
${\mathbf{ldx}}\ge {\mathbf{n}}$.
7: $\mathbf{wt}\left(*\right)$ – Real (Kind=nag_wp) arrayInput
Note: the dimension of the array wt
must be at least
${\mathbf{n}}$ if ${\mathbf{weight}}=\text{'W'}$.
On entry: the optional weights of each observation.
If ${\mathbf{weight}}=\text{'U'}$, wt is not referenced.
If ${\mathbf{weight}}=\text{'W'}$, ${\mathbf{wt}}\left(i\right)$ must contain the weight for the $i$th observation.
Constraint:
if ${\mathbf{weight}}=\text{'W'}$, ${\mathbf{wt}}\left(\mathit{i}\right)\ge 0.0$, for $\mathit{i}=1,2,\dots ,n$.
8: $\mathbf{sw}$ – Real (Kind=nag_wp)Output
On exit: the sum of weights.
If ${\mathbf{weight}}=\text{'U'}$, sw contains the number of observations, $n$.
9: $\mathbf{wmean}\left({\mathbf{m}}\right)$ – Real (Kind=nag_wp) arrayOutput
On exit: the sample means. ${\mathbf{wmean}}\left(j\right)$ contains the mean for the $j$th variable.
10: $\mathbf{c}\left(({\mathbf{m}}\times {\mathbf{m}}+{\mathbf{m}})/2\right)$ – Real (Kind=nag_wp) arrayOutput
On exit: the cross-products.
If ${\mathbf{mean}}=\text{'M'}$, c contains the upper triangular part of the matrix of (weighted) sums of squares and cross-products of deviations about the mean.
If ${\mathbf{mean}}=\text{'Z'}$, c contains the upper triangular part of the matrix of (weighted) sums of squares and cross-products.
These are stored packed by columns, i.e., the cross-product between the $j$th and $k$th variable, $k\ge j$, is stored in ${\mathbf{c}}\left(k\times (k-1)/2+j\right)$.
11: $\mathbf{ifail}$ – IntegerInput/Output
On entry: ifail must be set to $0$, $\mathrm{-1}$ or $1$ to set behaviour on detection of an error; these values have no effect when no error is detected.
A value of $0$ causes the printing of an error message and program execution will be halted; otherwise program execution continues. A value of $\mathrm{-1}$ means that an error message is printed while a value of $1$ means that it is not.
If halting is not appropriate, the value $\mathrm{-1}$ or $1$ is recommended. If message printing is undesirable, then the value $1$ is recommended. Otherwise, the value $0$ is recommended. When the value $-\mathbf{1}$ or $\mathbf{1}$ is used it is essential to test the value of ifail on exit.
On exit: ${\mathbf{ifail}}={\mathbf{0}}$ unless the routine detects an error or a warning has been flagged (see Section 6).
6Error Indicators and Warnings
If on entry ${\mathbf{ifail}}=0$ or $\mathrm{-1}$, explanatory error messages are output on the current error message unit (as defined by x04aaf).
Errors or warnings detected by the routine:
${\mathbf{ifail}}=1$
On entry, ${\mathbf{ldx}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{ldx}}\ge {\mathbf{n}}$.
On entry, ${\mathbf{m}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{m}}\ge 1$.
On entry, ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{n}}\ge 1$.
${\mathbf{ifail}}=2$
On entry, ${\mathbf{mean}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{mean}}=\text{'M'}$ or $\text{'Z'}$.
${\mathbf{ifail}}=3$
On entry, ${\mathbf{weight}}=\u27e8\mathit{\text{value}}\u27e9$.
Constraint: ${\mathbf{weight}}=\text{'W'}$ or $\text{'U'}$.
${\mathbf{ifail}}=4$
On entry, ${\mathbf{wt}}\left(\u27e8\mathit{\text{value}}\u27e9\right)<0.0$.
Constraint: ${\mathbf{wt}}\left(i\right)\ge 0.0$, for $i=1,2,\dots ,n$.
${\mathbf{ifail}}=-99$
An unexpected error has been triggered by this routine. Please
contact NAG.
See Section 7 in the Introduction to the NAG Library FL Interface for further information.
${\mathbf{ifail}}=-399$
Your licence key may have expired or may not have been installed correctly.
See Section 8 in the Introduction to the NAG Library FL Interface for further information.
${\mathbf{ifail}}=-999$
Dynamic memory allocation failed.
See Section 9 in the Introduction to the NAG Library FL Interface for further information.
g02buf makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this routine. Please also consult the Users' Note for your implementation for any additional implementation-specific information.
9Further Comments
g02bwf may be used to calculate the correlation coefficients from the cross-products of deviations about the mean. The cross-products of deviations about the mean may be scaled
using f06edforf06fdf
to give a variance-covariance matrix.
The means and cross-products produced by g02buf may be updated by adding or removing observations using g02btf.
Two sets of means and cross-products, as produced by g02buf, can be combined using g02bzf.
10Example
A program to calculate the means, the required sums of squares and cross-products matrix, and the variance matrix for a set of $3$ observations of $3$ variables.