hide long namesshow long names
hide short namesshow short names
Integer type:  int32  int64  nag_int  show int32  show int32  show int64  show int64  show nag_int  show nag_int

PDF version (NAG web site, 64-bit version, 64-bit version)
Chapter Contents
Chapter Introduction
NAG Toolbox

NAG Toolbox: nag_mv_prin_comp (g03aa)

Purpose

nag_mv_prin_comp (g03aa) performs a principal component analysis on a data matrix; both the principal component loadings and the principal component scores are returned.

Syntax

[s, e, p, v, ifail] = g03aa(matrix, std, x, isx, s, nvar, 'n', n, 'm', m, 'wt', wt)
[s, e, p, v, ifail] = nag_mv_prin_comp(matrix, std, x, isx, s, nvar, 'n', n, 'm', m, 'wt', wt)
Note: the interface to this routine has changed since earlier releases of the toolbox:
Mark 22: n has been made optional
Mark 24: drop weight, wt optional
.

Description

Let XX be an nn by pp data matrix of nn observations on pp variables x1,x2,,xpx1,x2,,xp and let the pp by pp variance-covariance matrix of x1,x2,,xpx1,x2,,xp be SS. A vector a1a1 of length pp is found such that:
a1TSa1  is maximized subject to  a1Ta1 = 1.
a1TSa1  is maximized subject to  a1Ta1=1.
The variable z1 = i = 1pa1ixiz1=i=1pa1ixi is known as the first principal component and gives the linear combination of the variables that gives the maximum variation. A second principal component, z2 = i = 1pa2ixiz2=i=1pa2ixi, is found such that:
a2TSa2  is maximized subject to ​a2Ta2 = 1and ​a2Ta1 = 0.
a2TSa2  is maximized subject to ​a2Ta2=1and ​a2Ta1=0.
This gives the linear combination of variables that is orthogonal to the first principal component that gives the maximum variation. Further principal components are derived in a similar way.
The vectors a1,a2,,apa1,a2,,ap, are the eigenvectors of the matrix SS and associated with each eigenvector is the eigenvalue, λi2λi2. The value of λi2 / λi2λi2/λi2 gives the proportion of variation explained by the iith principal component. Alternatively, the aiai's can be considered as the right singular vectors in a singular value decomposition with singular values λiλi of the data matrix centred about its mean and scaled by 1 / sqrt((n1))1/(n-1), XsXs. This latter approach is used in nag_mv_prin_comp (g03aa), with
Xs = VΛP
Xs=VΛP
where ΛΛ is a diagonal matrix with elements λiλi, PP is the pp by pp matrix with columns aiai and VV is an nn by pp matrix with VV = IVV=I, which gives the principal component scores.
Principal component analysis is often used to reduce the dimension of a dataset, replacing a large number of correlated variables with a smaller number of orthogonal variables that still contain most of the information in the original dataset.
The choice of the number of dimensions required is usually based on the amount of variation accounted for by the leading principal components. If kk principal components are selected, then a test of the equality of the remaining pkp-k eigenvalues is
(n − (2p + 5) / 6) ( p ) − ∑ log(λi2) + (p − k)
log(p ) ∑ λi2 / (p − k)i = k + 1
i = k + 1
(n-(2p+5)/6) {-i=k+1plog(λi2)+(p-k)log(i=k+1pλi2/(p-k)) }
which has, asymptotically, a χ2χ2-distribution with (1/2)(pk1)(pk + 2)12(p-k-1)(p-k+2) degrees of freedom.
Equality of the remaining eigenvalues indicates that if any more principal components are to be considered then they all should be considered.
Instead of the variance-covariance matrix the correlation matrix, the sums of squares and cross-products matrix or a standardized sums of squares and cross-products matrix may be used. In the last case SS is replaced by σ(1/2)Sσ(1/2)σ-12Sσ-12 for a diagonal matrix σσ with positive elements. If the correlation matrix is used, the χ2χ2 approximation for the statistic given above is not valid.
The principal component scores, FF, are the values of the principal component variables for the observations. These can be standardized so that the variance of these scores for each principal component is 1.01.0 or equal to the corresponding eigenvalue.
Weights can be used with the analysis, in which case the matrix XX is first centred about the weighted means then each row is scaled by an amount sqrt(wi)wi, where wiwi is the weight for the iith observation.

References

Chatfield C and Collins A J (1980) Introduction to Multivariate Analysis Chapman and Hall
Cooley W C and Lohnes P R (1971) Multivariate Data Analysis Wiley
Hammarling S (1985) The singular value decomposition in multivariate statistics SIGNUM Newsl. 20(3) 2–25
Kendall M G and Stuart A (1969) The Advanced Theory of Statistics (Volume 1) (3rd Edition) Griffin
Morrison D F (1967) Multivariate Statistical Methods McGraw–Hill

Parameters

Compulsory Input Parameters

1:     matrix – string (length ≥ 1)
Indicates for which type of matrix the principal component analysis is to be carried out.
matrix = 'C'matrix='C'
It is for the correlation matrix.
matrix = 'S'matrix='S'
It is for a standardized matrix, with standardizations given by s.
matrix = 'U'matrix='U'
It is for the sums of squares and cross-products matrix.
matrix = 'V'matrix='V'
It is for the variance-covariance matrix.
Constraint: matrix = 'C'matrix='C', 'S''S', 'U''U' or 'V''V'.
2:     std – string (length ≥ 1)
Indicates if the principal component scores are to be standardized.
std = 'S'std='S'
The principal component scores are standardized so that FF = IFF=I, i.e., F = XsPΛ1 = VF=XsPΛ-1=V.
std = 'U'std='U'
The principal component scores are unstandardized, i.e., F = XsP = VΛF=XsP=VΛ.
std = 'Z'std='Z'
The principal component scores are standardized so that they have unit variance.
std = 'E'std='E'
The principal component scores are standardized so that they have variance equal to the corresponding eigenvalue.
Constraint: std = 'E'std='E', 'S''S', 'U''U' or 'Z''Z'.
3:     x(ldx,m) – double array
ldx, the first dimension of the array, must satisfy the constraint ldxnldxn.
x(i,j)xij must contain the iith observation for the jjth variable, for i = 1,2,,ni=1,2,,n and j = 1,2,,mj=1,2,,m.
4:     isx(m) – int64int32nag_int array
m, the dimension of the array, must satisfy the constraint m1m1.
isx(j)isxj indicates whether or not the jjth variable is to be included in the analysis.
If isx(j) > 0isxj>0, the variable contained in the jjth column of x is included in the principal component analysis, for j = 1,2,,mj=1,2,,m.
Constraint: isx(j) > 0isxj>0 for nvar values of jj.
5:     s(m) – double array
m, the dimension of the array, must satisfy the constraint m1m1.
The standardizations to be used, if any.
If matrix = 'S'matrix='S', the first mm elements of s must contain the standardization coefficients, the diagonal elements of σσ.
Constraint: if isx(j) > 0isxj>0, s(j) > 0.0sj>0.0, for j = 1,2,,mj=1,2,,m.
6:     nvar – int64int32nag_int scalar
pp, the number of variables in the principal component analysis.
Constraint: 1nvarmin (n1,m)1nvarmin(n-1,m).

Optional Input Parameters

1:     n – int64int32nag_int scalar
Default: The first dimension of the array x.
nn, the number of observations.
Constraint: n2n2.
2:     m – int64int32nag_int scalar
Default: The dimension of the arrays isx, s and the second dimension of the array x. (An error is raised if these dimensions are not equal.)
mm, the number of variables in the data matrix.
Constraint: m1m1.
3:     wt( : :) – double array
Note: the dimension of the array wt must be at least nn if weight = 'W'weight='W', and at least 11 otherwise.
If weight = 'W'weight='W', the first nn elements of wt must contain the weights to be used in the principal component analysis.
If wt(i) = 0.0wti=0.0, the iith observation is not included in the analysis. The effective number of observations is the sum of the weights.
If weight = 'U'weight='U', wt is not referenced and the effective number of observations is nn.
Constraints:
  • wt(i)0.0wti0.0, for i = 1,2,,ni=1,2,,n;
  • the sum of weights nvar + 1nvar+1.

Input Parameters Omitted from the MATLAB Interface

weight ldx lde ldp ldv wk

Output Parameters

1:     s(m) – double array
If matrix = 'S'matrix='S', s is unchanged on exit.
If matrix = 'C'matrix='C', s contains the variances of the selected variables. s(j)sj contains the variance of the variable in the jjth column of x if isx(j) > 0isxj>0.
If matrix = 'U'matrix='U' or 'V''V', s is not referenced.
2:     e(lde,66) – double array
ldenvarldenvar.
The statistics of the principal component analysis.
e(i,1)ei1
The eigenvalues associated with the iith principal component, λi2λi2, for i = 1,2,,pi=1,2,,p.
e(i,2)ei2
The proportion of variation explained by the iith principal component, for i = 1,2,,pi=1,2,,p.
e(i,3)ei3
The cumulative proportion of variation explained by the first iith principal components, for i = 1,2,,pi=1,2,,p.
e(i,4)ei4
The χ2χ2 statistics, for i = 1,2,,pi=1,2,,p.
e(i,5)ei5
The degrees of freedom for the χ2χ2 statistics, for i = 1,2,,pi=1,2,,p.
If matrix'C'matrix'C', e(i,6)ei6 contains significance level for the χ2χ2 statistic, for i = 1,2,,pi=1,2,,p.
If matrix = 'C'matrix='C', e(i,6)ei6 is returned as zero.
3:     p(ldp,nvar) – double array
ldpnvarldpnvar.
The first nvar columns of p contain the principal component loadings, aiai. The jjth column of p contains the nvar coefficients for the jjth principal component.
4:     v(ldv,nvar) – double array
ldvnldvn.
The first nvar columns of v contain the principal component scores. The jjth column of v contains the n scores for the jjth principal component.
If weight = 'W'weight='W', any rows for which wt(i)wti is zero will be set to zero.
5:     ifail – int64int32nag_int scalar
ifail = 0ifail=0 unless the function detects an error (see [Error Indicators and Warnings]).

Error Indicators and Warnings

Errors or warnings detected by the function:

Cases prefixed with W are classified as warnings and do not generate an error of type NAG:error_n. See nag_issue_warnings.

  ifail = 1ifail=1
On entry,m < 1m<1,
orn < 2n<2,
ornvar < 1nvar<1,
ornvar > mnvar>m,
ornvarnnvarn,
orldx < nldx<n,
orldv < nldv<n,
orldp < nvarldp<nvar,
orlde < nvarlde<nvar,
ormatrix'C'matrix'C', 'S''S', 'U''U' or 'V''V',
orstd'S'std'S', 'U''U', 'Z''Z' or 'E''E',
orweight'U'weight'U' or 'W''W'.
  ifail = 2ifail=2
On entry,weight = 'W'weight='W' and a value of wt < 0.0wt<0.0.
  ifail = 3ifail=3
On entry,there are not nvar values of isx > 0isx>0,
orweight = 'W'weight='W' and the effective number of observations is less than nvar + 1nvar+1.
  ifail = 4ifail=4
On entry,s(j)0.0sj0.0 for some j = 1,2,,mj=1,2,,m, when matrix = 'S'matrix='S' and isx(j) > 0isxj>0.
  ifail = 5ifail=5
The singular value decomposition has failed to converge. This is an unlikely error exit.
W ifail = 6ifail=6
All eigenvalues/singular values are zero. This will be caused by all the variables being constant.

Accuracy

As nag_mv_prin_comp (g03aa) uses a singular value decomposition of the data matrix, it will be less affected by ill-conditioned problems than traditional methods using the eigenvalue decomposition of the variance-covariance matrix.

Further Comments

None.

Example

function nag_mv_prin_comp_example
matrix = 'V';
std = 'E';
x = [7, 4, 3;
     4, 1, 8;
     6, 3, 5;
     8, 6, 1;
     8, 5, 7;
     7, 2, 9;
     5, 3, 3;
     9, 5, 8;
     7, 4, 5;
     8, 2, 2];
isx = [int64(1);1;1];
s = [-5.04677090184712e-39;
     -5.04512289241806e-39;
     -1.790699005126953];
nvar = int64(3);
[sOut, e, p, v, ifail] = nag_mv_prin_comp(matrix, std, x, isx, s, nvar)
 

sOut =

   -0.0000
   -0.0000
   -1.7907


e =

    8.2739    0.6515    0.6515    8.6127    5.0000    0.1255
    3.6761    0.2895    0.9410    4.1183    2.0000    0.1276
    0.7499    0.0590    1.0000         0         0         0


p =

   -0.1376    0.6990   -0.7017
   -0.2505    0.6609    0.7075
    0.9583    0.2731    0.0842


v =

   -2.1514   -0.1731    0.1068
    3.8042   -2.8875    0.5104
    0.1532   -0.9869    0.2694
   -4.7065    1.3015    0.6517
    1.2938    2.2791    0.4492
    4.0993    0.1436   -0.8031
   -1.6258   -2.2321    0.8028
    2.1145    3.2512   -0.1684
   -0.2348    0.3730    0.2751
   -2.7464   -1.0689   -2.0940


ifail =

                    0


function g03aa_example
matrix = 'V';
std = 'E';
x = [7, 4, 3;
     4, 1, 8;
     6, 3, 5;
     8, 6, 1;
     8, 5, 7;
     7, 2, 9;
     5, 3, 3;
     9, 5, 8;
     7, 4, 5;
     8, 2, 2];
isx = [int64(1);1;1];
s = [-5.04677090184712e-39;
     -5.04512289241806e-39;
     -1.790699005126953];
nvar = int64(3);
[sOut, e, p, v, ifail] = g03aa(matrix, std, x, isx, s, nvar)
 

sOut =

   -0.0000
   -0.0000
   -1.7907


e =

    8.2739    0.6515    0.6515    8.6127    5.0000    0.1255
    3.6761    0.2895    0.9410    4.1183    2.0000    0.1276
    0.7499    0.0590    1.0000         0         0         0


p =

   -0.1376    0.6990   -0.7017
   -0.2505    0.6609    0.7075
    0.9583    0.2731    0.0842


v =

   -2.1514   -0.1731    0.1068
    3.8042   -2.8875    0.5104
    0.1532   -0.9869    0.2694
   -4.7065    1.3015    0.6517
    1.2938    2.2791    0.4492
    4.0993    0.1436   -0.8031
   -1.6258   -2.2321    0.8028
    2.1145    3.2512   -0.1684
   -0.2348    0.3730    0.2751
   -2.7464   -1.0689   -2.0940


ifail =

                    0



PDF version (NAG web site, 64-bit version, 64-bit version)
Chapter Contents
Chapter Introduction
NAG Toolbox

© The Numerical Algorithms Group Ltd, Oxford, UK. 2009–2013