NAG CL Interface
F01 (Matop)
Matrix Operations, Including Inversion

1 Scope of the Chapter

This chapter provides facilities for four types of problem:
  1. (i)Matrix Inversion
  2. (ii)Matrix Factorizations
  3. (iii)Matrix Arithmetic and Manipulation
  4. (iv)Matrix Functions
See Sections 2.1, 2.2, 2.3 and 2.4 where these problems are discussed.

2 Background to the Problems

2.1 Matrix Inversion

  1. (i)Nonsingular square matrices of order n.
    If A, a square matrix of order n, is nonsingular (has rank n), then its inverse X exists and satisfies the equations AX=XA=I (the identity or unit matrix).
    It is worth noting that if AX-I=R, so that R is the ‘residual’ matrix, then a bound on the relative error is given by R, i.e.,
    X-A-1 A-1 R.  
  2. (ii)General real rectangular matrices.
    A real matrix A has no inverse if it is square (n by n) and singular (has rank <n), or if it is of shape (m by n) with mn, but there is a Generalized or Pseudo-inverse A+ which satisfies the equations
    AA+A=A,  A+AA+=A+,  AA+T=AA+,  A+AT=A+A  
    (which of course are also satisfied by the inverse X of A if A is square and nonsingular).
    1. (a)if mn and rankA=n then A can be factorized using a QR factorization, given by
      A=Q R 0 ,  
      where Q is an m by m orthogonal matrix and R is an n by n, nonsingular, upper triangular matrix. The pseudo-inverse of A is then given by
      A+=R-1Q~T,  
      where Q~ consists of the first n columns of Q.
    2. (b)if mn and rankA=m then A can be factorized using an RQ factorization, given by
      A=R0QT  
      where Q is an n by n orthogonal matrix and R is an m by m, nonsingular, upper triangular matrix. The pseudo-inverse of A is then given by
      A+ = Q~R-1 ,  
      where Q~ consists of the first m columns of Q.
    3. (c)if mn and rankA=rn then A can be factorized using a QR factorization, with column interchanges, as
      A=Q R 0 PT,  
      where Q is an m by m orthogonal matrix, R is an r by n upper trapezoidal matrix and P is an n by n permutation matrix. The pseudo-inverse of A is then given by
      A+=PRTRRT-1Q~T,  
      where Q~ consists of the first r columns of Q.
    4. (d)if rankA=rk=minm,n, then A can be factorized as the singular value decomposition
      A=UΣVT,  
      where U is an m by m orthogonal matrix, V is an n by n orthogonal matrix and Σ is an m by n diagonal matrix with non-negative diagonal elements σ. The first k columns of U and V are the left- and right-hand singular vectors of A respectively and the k diagonal elements of Σ are the singular values of A. Σ may be chosen so that
      σ1σ2σk0  
      and in this case if rankA=r then
      σ1σ2σr>0,  σr+1==σk=0.  
      If U~ and V~ consist of the first r columns of U and V respectively and Σ~ is an r by r diagonal matrix with diagonal elements σ1,σ2,,σr then A is given by
      A=U~Σ~V~T  
      and the pseudo-inverse of A is given by
      A+=V~Σ~-1U~T.  
      Notice that
      ATA=VΣTΣVT  
      which is the classical eigenvalue (spectral) factorization of ATA.
    5. (e)if A is complex then the above relationships are still true if we use ‘unitary’ in place of ‘orthogonal’ and conjugate transpose in place of transpose. For example, the singular value decomposition of A is
      A=UΣVH,  
      where U and V are unitary, VH the conjugate transpose of V and Σ is as in (ii)(d) above.

2.2 Matrix Factorizations

The functions in this section perform matrix factorizations which are required for the solution of systems of linear equations with various special structures. A few functions which perform associated computations are also included.
Other functions for matrix factorizations are to be found in Chapters F07, F08 and F11.
This section also contains a few functions associated with eigenvalue problems (see Chapter F02). (Historical note: this section used to contain many more such functions, but they have now been superseded by functions in Chapter F08.)
Finally, this section contains functions for computing non-negative matrix factorizations, which are used for dimensional reduction and classification in data analysis. Given a rectangular m×n matrix, A, with non-negative elements, a non-negative matrix factorization of A is an approximate factorization of A into the product of a m×k non-negative matrix W and a k×n non-negative matrix H, so that AWH. Typically k is chosen so that kminm,n. The matrices W and H are then computed to minimize A-WHF. The factorization is not unique.

2.3 Matrix Arithmetic and Manipulation

The intention of functions in this section (f01v) is to cater for some of the commonly occurring operations in matrix manipulation, i.e., conversion between different storage formats,such as conversion between rectangular band matrix storage and packed band matrix storage. For vector or matrix-vector or matrix-matrix operations refer to Chapters F06 and F16.

2.4 Matrix Functions

Given a square matrix A, the matrix function fA is a matrix with the same dimensions as A which provides a generalization of the scalar function f.
If A has a full set of eigenvectors V then A can be factorized as
A = V D V-1 ,  
where D is the diagonal matrix whose diagonal elements, di, are the eigenvalues of A. fA is given by
fA = V fD V-1 ,  
where fD is the diagonal matrix whose ith diagonal element is fdi.
In general, A may not have a full set of eigenvectors. The matrix function can then be defined via a Cauchy integral. For An×n,
fA = 1 2π i Γ fz zI-A-1 dz ,  
where Γ is a closed contour surrounding the eigenvalues of A, and f is analytic within Γ.
Some matrix functions are defined implicitly. A matrix logarithm is a solution X to the equation
eX=A .  
In general X is not unique, but if A has no eigenvalues on the closed negative real line then a unique principal logarithm exists whose eigenvalues have imaginary part between π and -π. Similarly, a matrix square root is a solution X to the equation
X2=A .  
If A has no eigenvalues on the closed negative real line then a unique principal square root exists with eigenvalues in the right half-plane. If A has a vanishing eigenvalue then logA cannot be computed. If the vanishing eigenvalue is defective (its algebraic multiplicity exceeds its geometric multiplicity, or equivalently it occurs in a Jordan block of size greater than 1) then the square root cannot be computed. If the vanishing eigenvalue is semisimple (its algebraic and geometric multiplicities are equal, or equivalently it occurs only in Jordan blocks of size 1) then a square root can be computed.
Algorithms for computing matrix functions are usually tailored to a specific function. Currently Chapter F01 contains routines for calculating the exponential, logarithm, sine, cosine, sinh, cosh, square root and general real power of both real and complex matrices. In addition there are routines to compute a general function of real symmetric and complex Hermitian matrices and a general function of general real and complex matrices.
The Fréchet derivative of a matrix function fA in the direction of the matrix E is the linear function mapping E to LfA,E such that
fA+E - fA - LfA,E = OE .  
The Fréchet derivative measures the first-order effect on fA of perturbations in A. Chapter F01 contains functions for calculating the Fréchet derivative of the exponential, logarithm and real powers of both real and complex matrices.
The condition number of a matrix function is a measure of its sensitivity to perturbations in the data. The absolute condition number measures these perturbations in an absolute sense, and is defined by
condabs f,A lim ε0 sup E0 fA+E - fA ε .  
The relative condition number, which is usually of more interest, measures these perturbations in a relative sense, and is defined by
condrel f,A = condabs f,A A fA .  
The absolute and relative condition numbers can be expressed in terms of the norm of the Fréchet derivative by
condabs f,A = max E0 LA,E E ,  
condrel f,A = A fA max E0 LA,E E .  
Chapter F01 contains routines for calculating the condition number of the matrix exponential, logarithm, sine, cosine, sinh, cosh, square root and general real power of both real and complex matrices. It also contains routines for estimating the condition number of a general function of a real or complex matrix.

3 Recommendations on Choice and Use of Available Functions

3.1 Matrix Inversion

Note:  before using any function for matrix inversion, consider carefully whether it is really needed.
Although the solution of a set of linear equations Ax=b can be written as x=A-1b, the solution should never be computed by first inverting A and then computing A-1b; the functions in Chapters F04 or F07 should always be used to solve such sets of equations directly; they are faster in execution, and numerically more stable and accurate. Similar remarks apply to the solution of least squares problems which again should be solved by using the functions in Chapters F04 and F08 rather than by computing a pseudo-inverse.
  1. (a)Nonsingular square matrices of order n
    This chapter describes techniques for inverting a general real matrix A and matrices which are positive definite (have all eigenvalues positive) and are either real and symmetric or complex and Hermitian. It is wasteful and uneconomical not to use the appropriate function when a matrix is known to have one of these special forms. A general function must be used when the matrix is not known to be positive definite. In most functions the inverse is computed by solving the linear equations Axi=ei, for i=1,2,,n, where ei is the ith column of the identity matrix.
    The residual matrix R=AX-I, where X is a computed inverse of A, conveys useful information. Firstly R is a bound on the relative error in X and secondly R<12 guarantees the convergence of the iterative process in the ‘corrected’ inverse functions.
    The decision trees for inversion show which functions in Chapter F07 should be used for the inversion of other special types of matrices not treated in the chapter.
  2. (b)General real rectangular matrices
    For real matrices f08aec returns the QR factorization of the matrix and f08bfc returns the QR factorization with column interchanges. The corresponding complex functions are f08asc and f08btc respectively. Functions are also provided to form the orthogonal matrices and transform by the orthogonal matrices following the use of the above functions.
    f08kbc and f08kpc compute the singular value decomposition as described in Section 2 for real and complex matrices respectively. If A has rank rk=minm,n then the k-r smallest singular values will be negligible and the pseudo-inverse of A can be obtained as A+=VΣ-1UT as described in Section 2. If the rank of A is not known in advance it can be estimated from the singular values (see Section 2.4 in the F04 Chapter Introduction). For large sparse matrices, leading terms in the singular value decomposition can be computed using functions from Chapter F12.

3.2 Matrix Factorizations

Most of these functions serve a special purpose required for the solution of sets of simultaneous linear equations or the eigenvalue problem. For further details you should consult Sections 3 or 4 in the F02 Chapter Introduction or Sections 3 or 4 in the F04 Chapter Introduction.
f11dac is provided for factorizing general real sparse matrices. A more recent algorithm for the same problem is available through f11mec. For factorizing real symmetric positive definite sparse matrices, see f11jac. These functions should be used only when A is not banded and when the total number of nonzero elements is less than 10% of the total number of elements. In all other cases either the band functions or the general functions should be used.
The functions f01sac (for dense matrices) and f01sbc (sparse matrices, using a reverse communication interface) are provided for computing non-negative matrix factorizations.

3.3 Matrix Arithmetic and Manipulation

The functions in the f01v (LAPACK) section are designed to allow conversion between full storage format and one of the packed storage schemes required by some of the functions in Chapters F02, F04, F06, F07 and F08.

3.3.1 NAG Names and LAPACK Names

Functions with NAG short name beginning f01v may be called either by their NAG short names or by their NAG long names. The NAG long names for a function is simply the LAPACK name (in lower case) prepended by nag_, for example, nag_matop_dtrttf is the long name for f01vec.
When using the NAG Library, the double precision form of the LAPACK name must be used (beginning with d- or z-).
References to Chapter F01 functions in the manual normally include the LAPACK double precision names, for example, f01vec.
The LAPACK function names follow a simple scheme (which is similar to that used for the BLAS in Chapter F16). Most names have the structure nag_xyytzz, where the components have the following meanings:
– the initial letter, x, indicates the data type (real or complex) and precision:
– the fourth letter, t, indicates that the function is performing a storage scheme transformation (conversion)
– the letters yy indicate the original storage scheme used to store a triangular part of the matrix A, while the letters zz indicate the target storage scheme of the conversion (yy cannot equal zz since this would do nothing):

3.4 Matrix Functions

f01ecc and f01fcc compute the matrix exponential, eA, of a real and complex square matrix A respectively. If estimates of the condition number of the matrix exponential are required then f01jgc and f01kgc should be used. If Fréchet derivatives are required then f01jhc and f01khc should be used.
f01edc and f01fdc compute the matrix exponential, eA, of a real symmetric and complex Hermitian matrix respectively. If the matrix is real symmetric, or complex Hermitian then it is recommended that f01edc, or f01fdc be used as they are more efficient and, in general, more accurate than f01ecc and f01fcc.
f01ejc and f01fjc compute the principal matrix logarithm, logA, of a real and complex square matrix A respectively. If estimates of the condition number of the matrix logarithm are required then f01jjc and f01kjc should be used. If Fréchet derivatives are required then f01jkc and f01kkc should be used.
f01ekc and f01fkc compute the matrix exponential, sine, cosine, sinh or cosh of a real and complex square matrix A respectively. If the matrix exponential is required then it is recommended that f01ecc or f01fcc be used as they are, in general, more accurate than f01ekc and f01fkc. If estimates of the condition number of the matrix function are required then f01jac and f01kac should be used.
f01elc and f01emc compute the matrix function, fA, of a real square matrix. f01flc and f01fmc compute the matrix function of a complex square matrix. The derivatives of f are required for these computations. f01elc and f01flc use numerical differentiation to obtain the derivatives of f. f01emc and f01fmc use derivatives you have supplied. If estimates of the condition number are required but you are not supplying derivatives then f01jbc and f01kbc should be used. If estimates of the condition number of the matrix function are required and you are supplying derivatives of f then f01jcc and f01kcc should be used.
If the matrix A is real symmetric or complex Hermitian then it is recommended that to compute the matrix function, fA, f01efc and f01ffc are used respectively as they are more efficient and, in general, more accurate than f01elc, f01emc, f01flc and f01fmc.
f01gac and f01hac compute the matrix function etAB for explicitly stored dense real and complex matrices A and B respectively while f01gbc and f01hbc compute the same using reverse communication. In the latter case, control is returned to you. You should calculate any required matrix-matrix products and then call the function again. See Section 7 in How to Use the NAG Library for further information.
f01enc and f01fnc compute the principal square root A1/2 of a real and complex square matrix A respectively. If A is complex and upper triangular then f01fpc should be used. If A is real and upper quasi-triangular then f01epc should be used. If estimates of the condition number of the matrix square root are required then f01jdc and f01kdc should be used.
f01eqc and f01fqc compute the matrix power Ap, where p, of real and complex matrices respectively. If estimates of the condition number of the matrix power are required then f01jec and f01kec should be used. If Fréchet derivatives are required then f01jfc and f01kfc should be used.

4 Decision Trees

The decision trees show the functions in this chapter and in Chapter F07 and Chapter F08 that should be used for inverting matrices of various types. They also show which function should be used to calculate various matrix functions.
(i) Matrix Inversion:

Tree 1

Is A an n by n matrix of rank n?   Is A a real matrix?   see Tree 2
yesyes
  no   no
see Tree 3
see Tree 4

Tree 2: Inverse of a real n by n matrix of full rank

Is A a band matrix?   See Note 1.
yes
  no
Is A symmetric?   Is A positive definite?   Is one triangle of A stored as a linear array?   f07gdc and f07gjc
yesyesyes
  no   no   no
f07fdc and f07fjc
Is one triangle of A stored as a linear array?   f07pdc and f07pjc
yes
  no
f07mdc and f07mjc
Is A triangular?   Is A stored as a linear array?   f07ujc
yesyes
  no   no
f07tjc
f07adc and f07ajc

Tree 3: Inverse of a complex n by n matrix of full rank

Is A a band matrix?   See Note 1.
yes
  no
Is A Hermitian?   Is A positive definite?   Is one triangle of A stored as a linear array?   f07grc and f07gwc
yesyesyes
  no   no   no
f07frc and f07fwc
Is one triangle A stored as a linear array?   f07prc and f07pwc
yes
  no
f07mrc and f07mwc
Is A symmetric?   Is one triangle of A stored as a linear array?   f07qrc and f07qwc
yesyes
  no   no
f07nrc and f07nwc
Is A triangular?   Is A stored as a linear array?   f07uwc
yesyes
  no   no
f07twc
f07anc or f07arc and f07awc

Tree 4: Pseudo-inverses

Is A a complex matrix?   Is A of full rank?   Is A an m by n matrix with m<n?   f08avc and f08awc or f08axc
yesyesyes
  no   no   no
f08asc and f08auc or f08atc
f08kpc
Is A of full rank?   Is A an m by n matrix with m<n?   f08ahc and f08ajc or f08akc
yesyes
  no   no
f08aec and f08agc or f08afc
f08kbc
Note 1: the inverse of a band matrix A does not in general have the same shape as A, and no functions are provided specifically for finding such an inverse. The matrix must either be treated as a full matrix, or the equations AX=B must be solved, where B has been initialized to the identity matrix I. In the latter case, see the decision trees in Section 4 in the F04 Chapter Introduction.
Note 2: by ‘guaranteed accuracy’ we mean that the accuracy of the inverse is improved by use of the iterative refinement technique using additional precision.
(ii) Matrix Factorizations: see the decision trees in Section 4 in the F02 and F04 Chapter Introductions.
(iii) Matrix Arithmetic and Manipulation: not appropriate.
(iv) Matrix Functions:

Tree 5: Matrix functions fA of an n by n real matrix A

Is etAB required?   Is A stored in dense format?   f01gac
yesyes
  no   no
f01gbc
Is A real symmetric?   Is eA required?   f01edc
yesyes
  no   no
f01efc
Is cosA or coshA or sinA or sinhA required?   Is the condition number of the matrix function required?   f01jac
yesyes
  no   no
f01ekc
Is logA required?   Is the condition number of the matrix logarithm required?   f01jjc
yesyes
  no   no
Is the Fréchet derivative of the matrix logarithm required?   f01jkc
yes
  no
f01ejc
Is expA required?   Is the condition number of the matrix exponential required?   f01jgc
yesyes
  no   no
Is the Fréchet derivative of the matrix exponential required?   f01jhc
yes
  no
f01ecc
Is A1/2 required?   Is the condition number of the matrix square root required?   f01jdc
yesyes
  no   no
Is the matrix upper quasi-triangular?   f01epc
yes
  no
f01enc
Is Ap required?   Is the condition number of the matrix power required?   f01jec
yesyes
  no   no
Is the Fréchet derivative of the matrix power required?   f01jfc
yes
  no
f01eqc
fA will be computed. Will derivatives of f be supplied by the user?   Is the condition number of the matrix function required?   f01jcc
yesyes
  no   no
f01emc
Is the condition number of the matrix function required?   f01jbc
yes
  no
f01elc

Tree 6: Matrix functions fA of an n by n complex matrix A

Is etAB required?   Is A stored in dense format?   f01hac
yesyes
  no   no
f01hbc
Is A complex Hermitian?   Is eA required?   f01fdc
yesyes
  no   no
f01ffc
Is cosA or coshA or sinA or sinhA required?   Is the condition number of the matrix function required?   f01kac
yesyes
  no   no
f01fkc
Is logA required?   Is the condition number of the matrix logarithm required?   f01kjc
yesyes
  no   no
Is the Fréchet derivative of the matrix logarithm required?   f01kkc
yes
  no
f01fjc
Is expA required?   Is the condition number of the matrix exponential required?   f01kgc
yesyes
  no   no
Is the Fréchet derivative of the matrix exponential required?   f01khc
yes
  no
f01fcc
Is A1/2 required?   Is the condition number of the matrix square root required?   f01kdc
yesyes
  no   no
Is the matrix upper triangular?   f01fpc
yes
  no
f01fnc
Is Ap required?   Is the condition number of the matrix power required?   f01kec
yesyes
  no   no
Is the Fréchet derivative of the matrix power required?   f01kfc
yes
  no
f01fqc
fA will be computed. Will derivatives of f be supplied by the user?   Is the condition number of the matrix function required?   f01kcc
yesyes
  no   no
f01fmc
Is the condition number of the matrix function required?   f01kbc
yes
  no
f01flc

5 Functionality Index

Action of the matrix exponential on a complex matrix   f01hac
Action of the matrix exponential on a complex matrix (reverse communication)   f01hbc
Action of the matrix exponential on a real matrix   f01gac
Action of the matrix exponential on a real matrix (reverse communication)   f01gbc
Matrix Arithmetic and Manipulation,  
matrix multiplication,  
triangular matrices,  
in-place,  
complex matrices   f01duc
real matrices   f01dgc
matrix storage conversion,  
full to packed triangular storage,  
complex matrices   f01vbc
real matrices   f01vac
full to Rectangular Full Packed storage,  
complex matrix   f01vfc
real matrix   f01vec
packed triangular to full storage,  
complex matrices   f01vdc
real matrices   f01vcc
packed triangular to Rectangular Full Packed storage,  
complex matrices   f01vkc
real matrices   f01vjc
Rectangular Full Packed to full storage,  
complex matrices   f01vhc
real matrices   f01vgc
Rectangular Full Packed to packed triangular storage,  
complex matrices   f01vmc
real matrices   f01vlc
Matrix function,  
complex Hermitian n by n matrix,  
matrix exponential   f01fdc
matrix function   f01ffc
complex n by n matrix,  
condition number for a matrix exponential   f01kgc
condition number for a matrix exponential, logarithm, sine, cosine, sinh or cosh   f01kac
condition number for a matrix function, using numerical differentiation   f01kbc
condition number for a matrix function, using user-supplied derivatives   f01kcc
condition number for a matrix logarithm   f01kjc
condition number for a matrix power   f01kec
condition number for the matrix square root, logarithm, sine, cosine, sinh or cosh   f01kdc
Fréchet derivative  
matrix exponential   f01khc
matrix logarithm   f01kkc
matrix power   f01kfc
general power  
matrix   f01fqc
matrix exponential   f01fcc
matrix exponential, sine, cosine, sinh or cosh   f01fkc
matrix function, using numerical differentiation   f01flc
matrix function, using user-supplied derivatives   f01fmc
matrix logarithm   f01fjc
matrix square root   f01fnc
upper triangular  
matrix square root   f01fpc
real n by n matrix,  
condition number for a matrix exponential   f01jgc
condition number for a matrix function, using numerical differentiation   f01jbc
condition number for a matrix function, using user-supplied derivatives   f01jcc
condition number for a matrix logarithm   f01jjc
condition number for a matrix power   f01jec
condition number for the matrix exponential, logarithm, sine, cosine, sinh or cosh   f01jac
condition number for the matrix square root, logarithm, sine, cosine, sinh or cosh   f01jdc
Fréchet derivative  
matrix exponential   f01jhc
matrix logarithm   f01jkc
matrix power   f01jfc
general power  
matrix exponential   f01eqc
matrix exponential   f01ecc
matrix exponential, sine, cosine, sinh or cosh   f01ekc
matrix function, using numerical differentiation   f01elc
matrix function, using user-supplied derivatives   f01emc
matrix logarithm   f01ejc
matrix square root   f01enc
upper quasi-triangular  
matrix square root   f01epc
real symmetric n by n matrix,  
matrix exponential   f01edc
matrix function   f01efc
Matrix Transformations,  
non-negative matrix factorization   f01sac
non-negative matrix factorization, reverse communication   f01sbc
real band symmetric positive definite matrix,  
variable bandwidth, LDLT factorization   f01mcc

6 Auxiliary Functions Associated with Library Function Arguments

None.

7 Withdrawn or Deprecated Functions

The following lists all those functions that have been withdrawn since Mark 23 of the Library or are in the Library, but deprecated.
Function Status Replacement Function(s)
f01bnc Withdrawn at Mark 25 f07frc
f01qcc Withdrawn at Mark 25 f08aec
f01qdc Withdrawn at Mark 25 f08agc
f01qec Withdrawn at Mark 25 f08afc
f01rcc Withdrawn at Mark 25 f08asc
f01rdc Withdrawn at Mark 25 f08auc
f01rec Withdrawn at Mark 25 f08atc

8 References

Golub G H and Van Loan C F (1996) Matrix Computations (3rd Edition) Johns Hopkins University Press, Baltimore
Higham N J (2008) Functions of Matrices: Theory and Computation SIAM, Philadelphia, PA, USA
Wilkinson J H (1965) The Algebraic Eigenvalue Problem Oxford University Press, Oxford
Wilkinson J H (1977) Some recent advances in numerical linear algebra The State of the Art in Numerical Analysis (ed D A H Jacobs) Academic Press
Wilkinson J H and Reinsch C (1971) Handbook for Automatic Computation II, Linear Algebra Springer–Verlag