naginterfaces.library.correg.ssqmat

naginterfaces.library.correg.ssqmat(x, mean='M', wt=None)

ssqmat calculates the sample means and sums of squares and cross-products, or sums of squares and cross-products of deviations from the mean, in a single pass for a set of data. The data may be weighted.

For full information please refer to the NAG Library document for g02bu

https://www.nag.com/numeric/nl/nagdoc_29.3/flhtml/g02/g02buf.html

Parameters

xfloat, array-like, shape $(n,m)$

$x[\text{i}-1,\text{j}-1]$ must contain the $\text{i}$th observation on the $\text{j}$th variable, for $\text{j}=1,2,\dots ,m$, for $\text{i}=1,2,\dots ,n$.

meanstr, length 1, optional

Indicates whether ssqmat is to calculate sums of squares and cross-products, or sums of squares and cross-products of deviations about the mean.

$mean=\text{'M'}$

The sums of squares and cross-products of deviations about the mean are calculated.

$mean=\text{'Z'}$

The sums of squares and cross-products are calculated.

wtNone or float, array-like, shape $\left(n\right)$, optional

The optional weights of each observation. If weights are not provided then $wt$ must be set to None, otherwise $wt[i-1]$ must contain the weight for the $i$th observation.

Returns

swfloat

The sum of weights.

If $wt\text{is}None$, $sw$ contains the number of observations, $n$.

wmeanfloat, ndarray, shape $\left(m\right)$

The sample means. $wmean[j-1]$ contains the mean for the $j$th variable.

cfloat, ndarray, shape $\left((m\times m+m)/2\right)$

The cross-products.

If $mean=\text{'M'}$, $c$ contains the upper triangular part of the matrix of (weighted) sums of squares and cross-products of deviations about the mean.

If $mean=\text{'Z'}$, $c$ contains the upper triangular part of the matrix of (weighted) sums of squares and cross-products.

These are stored packed by columns, i.e., the cross-product between the $j$th and $k$th variable, $k\ge j$, is stored in $c[k\times (k-1)/2+j-1]$.

Raises

NagValueError

(errno $1$)

On entry, $m=⟨value⟩$.

Constraint: $m\ge 1$.

(errno $1$)

On entry, $n=⟨value⟩$.

Constraint: $n\ge 1$.

(errno $2$)

On entry, $mean=⟨value⟩$.

Constraint: $mean=\text{'M'}$ or $\text{'Z'}$.

(errno $3$)

On entry, $\text{weight}=⟨value⟩$.

Constraint: $\text{weight}=\text{'W'}$ or $\text{'U'}$.

(errno $4$)

On entry, $wt[⟨value⟩]<0.0$.

Constraint: $wt[i-1]\ge 0.0$, for $i=1,2,\dots ,n$.

Notes

ssqmat is an adaptation of West’s WV2 algorithm; see West (1979). This function calculates the (optionally weighted) sample means and (optionally weighted) sums of squares and cross-products or sums of squares and cross-products of deviations from the (weighted) mean for a sample of $n$ observations on $m$ variables $X_j$, for $\text{j}=1,2,\dots ,m$. The algorithm makes a single pass through the data.

For the first $i-1$ observations let the mean of the $j$th variable be ${\overline{x}}_j(i-1)$, the cross-product about the mean for the $j$th and $k$th variables be $c_{jk}(i-1)$ and the sum of weights be $W_{i-1}$. These are updated by the $i$th observation, $x_{ij}$, for $\text{j}=1,2,\dots ,m$, with weight $w_i$ as follows:

$$\begin{array}{c}\begin{array}{c}{W}_i=W_{i-1}+w_i\\ {\overline{x}}_j\left(i\right)={\overline{x}}_j(i-1)+\frac{w_i}{W_i}(x_j-{\overline{x}}_j(i-1))\text{,}j=1,2,\dots ,m\end{array}\end{array}$$

and

$$c_{jk}\left(i\right)=c_{jk}(i-1)+\frac{w_i}{W_i}(x_j-{\overline{x}}_j(i-1))(x_k-{\overline{x}}_k(i-1))W_{i-1}\text{,}j=1,2,\dots ,m\text{and}k=j,j+1,\dots ,m\text{.}$$

The algorithm is initialized by taking ${\overline{x}}_j\left(1\right)=x_{1j}$, the first observation, and $c_{ij}\left(1\right)=0.0$.

For the unweighted case $w_i=1$ and $W_i=i$ for all $i$.

Note that only the upper triangle of the matrix is calculated and returned packed by column.

References

Chan, T F, Golub, G H and Leveque, R J, 1982, Updating Formulae and a Pairwise Algorithm for Computing Sample Variances, Compstat, Physica-Verlag

West, D H D, 1979, Updating mean and variance estimates: An improved method, Comm. ACM (22), 532–555

See also

naginterfaces.library.examples.correg.lars_param_ex.main()

naginterfaces.library.examples.correg.linregm_fit_stepwise_ex.main()