F01 Chapter Contents
F01 Chapter Introduction (PDF version)
NAG Library Manual

NAG Library Chapter Introduction

F01 – Matrix Operations, Including Inversion

+ Contents

1  Scope of the Chapter

This chapter provides facilities for four types of problem:
(i) Matrix Inversion
(ii) Matrix Factorizations
(iii) Matrix Arithmetic and Manipulation
(iv) Matrix Functions
These problems are discussed separately in Section 2.1, Section 2.2, Section 2.3 and Section 2.4.

2  Background to the Problems

2.1  Matrix Inversion

(i) Non-singular square matrices of order n.
If A, a square matrix of order n, is nonsingular (has rank n), then its inverse X exists and satisfies the equations AX=XA=I (the identity or unit matrix).
It is worth noting that if AX-I=R, so that R is the ‘residual’ matrix, then a bound on the relative error is given by R, i.e.,
X-A-1 A-1 R.
(ii) General real rectangular matrices.
A real matrix A has no inverse if it is square (n by n) and singular (has rank <n), or if it is of shape (m by n) with mn, but there is a Generalized or Pseudo Inverse Z which satisfies the equations
AZA=A,  ZAZ=Z,  AZT=AZ,  ZAT=ZA
(which of course are also satisfied by the inverse X of A if A is square and nonsingular).
(a) if mn and rankA=n then A can be factorized using a QR factorization, given by
A=Q R 0 ,
where Q is an m by m orthogonal matrix and R is an n by n, nonsingular, upper triangular matrix. The pseudo-inverse of A is then given by
Z=R-1Q~T,
where Q~ consists of the first n columns of Q.
(b) if mn and rankA=m then A can be factorized using an RQ factorization, given by
A=R0PT
where P is an n by n orthogonal matrix and R is an m by m, nonsingular, upper triangular matrix. The pseudo-inverse of A is then given by
Z=P~R-1,
where P~ consists of the first m columns of P.
(c) if mn and rankA=rn then A can be factorized using a QR factorization, with column interchanges, as
A=Q R 0 PT,
where Q is an m by m orthogonal matrix, R is an r by n upper trapezoidal matrix and P is an n by n permutation matrix. The pseudo-inverse of A is then given by
Z=PRTRRT-1Q~T,
where Q~ consists of the first r columns of Q.
(d) if rankA=rk=minm,n, then A can be factorized as the singular value decomposition
A=QDPT,
where Q is an m by m orthogonal matrix, P is an n by n orthogonal matrix and D is an m by n diagonal matrix with non-negative diagonal elements. The first k columns of Q and P are the left- and right-hand singular vectors of A respectively and the k diagonal elements of D are the singular values of A. D may be chosen so that
d1d2dk0
and in this case if rankA=r then
d1d2dr>0,  dr+1==dk=0.
If Q~ and P~ consist of the first r columns of Q and P respectively and Σ is an r by r diagonal matrix with diagonal elements d1,d2,,dr then A is given by
A=Q~ΣP~T
and the pseudo-inverse of A is given by
Z=P~Σ-1Q~T.
Notice that
ATA=PDTDPT
which is the classical eigenvalue (spectral) factorization of ATA.
(e) if A is complex then the above relationships are still true if we use ‘unitary’ in place of ‘orthogonal’ and conjugate transpose in place of transpose. For example, the singular value decomposition of A is
A=QDPH,
where Q and P are unitary, PH the conjugate transpose of P and D is as in (d) above.

2.2  Matrix Factorizations

The routines in this section perform matrix factorizations which are required for the solution of systems of linear equations with various special structures. A few routines which perform associated computations are also included.
Other routines for matrix factorizations are to be found in Chapters F07, F08 and F11.
This section also contains a few routines associated with eigenvalue problems (see Chapter F02). (Historical note: this section used to contain many more such routines, but they have now been superseded by routines in Chapter F08.)

2.3  Matrix Arithmetic and Manipulation

The intention of routines in this section (sub-chapters F01C, F01V and F01Z) is to cater for some of the commonly occurring operations in matrix manipulation, e.g., transposing a matrix or adding part of one matrix to another, and for conversion between different storage formats, e.g., conversion between rectangular band matrix storage and packed band matrix storage. For vector or matrix-vector or matrix-matrix operations refer to Chapters F06 and F16.

2.4  Matrix Functions

The routines in this section compute functions of matrices. Currently there are routines to compute the matrix exponential, as well as routines to compute a general function of real symmetric and complex Hermitian matrices.

3  Recommendations on Choice and Use of Available Routines

3.1  Matrix Inversion

Note:  before using any routine for matrix inversion, consider carefully whether it is really needed.
Although the solution of a set of linear equations Ax=b can be written as x=A-1b, the solution should never be computed by first inverting A and then computing A-1b; the routines in Chapters F04 or F07 should always be used to solve such sets of equations directly; they are faster in execution, and numerically more stable and accurate. Similar remarks apply to the solution of least squares problems which again should be solved by using the routines in Chapters F04 and F08 rather than by computing a pseudo-inverse.
(a) Non-singular square matrices of order n 
This chapter describes techniques for inverting a general real matrix A and matrices which are positive definite (have all eigenvalues positive) and are either real and symmetric or complex and Hermitian. It is wasteful and uneconomical not to use the appropriate routine when a matrix is known to have one of these special forms. A general routine must be used when the matrix is not known to be positive definite. In most routines the inverse is computed by solving the linear equations Axi=ei, for i=1,2,,n, where ei is the ith column of the identity matrix.
Routines are given for calculating the approximate inverse, that is solving the linear equations just once, and also for obtaining the accurate inverse by successive iterative corrections of this first approximation. The latter, of course, are more costly in terms of time and storage, since each correction involves the solution of n sets of linear equations and since the original A and its LU decomposition must be stored together with the first and successively corrected approximations to the inverse. In practice the storage requirements for the ‘corrected’ inverse routines are about double those of the ‘approximate’ inverse routines, though the extra computer time is not prohibitive since the same matrix and the same LU decomposition is used in every linear equation solution.
Despite the extra work of the ‘corrected’ inverse routines they are superior to the ‘approximate’ inverse routines. A correction provides a means of estimating the number of accurate figures in the inverse or the number of ‘meaningful’ figures relating to the degree of uncertainty in the coefficients of the matrix.
The residual matrix R=AX-I, where X is a computed inverse of A, conveys useful information. Firstly R is a bound on the relative error in X and secondly R<12  guarantees the convergence of the iterative process in the ‘corrected’ inverse routines.
The decision trees for inversion show which routines in Chapter F04 and Chapter F07 should be used for the inversion of other special types of matrices not treated in the chapter.
(b) General real rectangular matrices
For real matrices F08AEF (DGEQRF) and F01QJF return QR and RQ factorizations of A respectively and F08BFF (DGEQP3) returns the QR factorization with column interchanges. The corresponding complex routines are F08ASF (ZGEQRF), F01RJF and F08BTF (ZGEQP3) respectively. Routines are also provided to form the orthogonal matrices and transform by the orthogonal matrices following the use of the above routines. F01QGF and F01RGF form the RQ factorization of an upper trapezoidal matrix for the real and complex cases respectively.
F01BLF uses the QR factorization as described in Section 2.1(ii)(a) and is the only routine that explicitly returns a pseudo-inverse. If mn, then the routine will calculate the pseudo-inverse Z of the matrix A. If m<n, then the n by m matrix AT should be used. The routine will calculate the pseudo-inverse Z of AT and the required pseudo-inverse will be ZT. The routine also attempts to calculate the rank, r, of the matrix given a tolerance to decide when elements can be regarded as zero. However, should this routine fail due to an incorrect determination of the rank, the singular value decomposition method (described below) should be used.
F08KBF (DGESVD) and F08KPF (ZGESVD) compute the singular value decomposition as described in Section 2 for real and complex matrices respectively. If A has rank rk=minm,n then the k-r smallest singular values will be negligible and the pseudo-inverse of A can be obtained as Z=PΣ-1QT as described in Section 2. If the rank of A is not known in advance it can be estimated from the singular values (see Section 2.4 in the F04 Chapter Introduction). In the real case with mn, F02WDF provides details of the QR factorization or the singular value decomposition depending on whether or not A is of full rank and for some problems provides an attractive alternative to F08KBF (DGESVD). For large sparse matrices, leading terms in the singular value decomposition can be computed using routines from Chapter F12.

3.2  Matrix Factorizations

Each of these routines serves a special purpose required for the solution of sets of simultaneous linear equations or the eigenvalue problem. For further details you should consult Sections 3 or 4 in the F02 Chapter Introduction or Sections 3 or 4 in the F04 Chapter Introduction.
F01BRF and F01BSF are provided for factorizing general real sparse matrices. A more recent algorithm for the same problem is available through F11MEF. For factorizing real symmetric positive definite sparse matrices, see F11JAF. These routines should be used only when A is not banded and when the total number of nonzero elements is less than 10% of the total number of elements. In all other cases either the band routines or the general routines should be used.

3.3  Matrix Arithmetic and Manipulation

The routines in the F01C section are designed for the general handling of m by n matrices. Emphasis has been placed on flexibility in the parameter specifications and on avoiding, where possible, the use of internally declared arrays. They are therefore suited for use with large matrices of variable row and column dimensions. Routines are included for the addition and subtraction of sub-matrices of larger matrices, as well as the standard manipulations of full matrices. Those routines involving matrix multiplication may use additional-precision arithmetic for the accumulation of inner products. See also Chapter F06.
The routines in the F01V (LAPACK) and F01Z section are designed to allow conversion between full storage format and one of the packed storage schemes required by some of the routines in Chapters F02, F04, F06, F07 and F08.

3.3.1  NAG Names and LAPACK Names

Routines with NAG name beginning F01V may be called either by their NAG names or by their LAPACK names. When using the NAG Library, the double precision form of the LAPACK name must be used (beginning with D- or Z-).
References to Chapter F01 routines in the manual normally include the LAPACK double precision names, for example, F01VEF (DTRTTF).
The LAPACK routine names follow a simple scheme (which is similar to that used for the BLAS in Chapter F06). Most names have the structure XYYTZZ, where the components have the following meanings:
– the initial letter, X, indicates the data type (real or complex) and precision:
– the fourth letter, T, indicates that the routine is performing a storage scheme transformation (conversion)
– the letters YY indicate the original storage scheme used to store a triangular part of the matrix A, while the letters ZZ indicate the target storage scheme of the conversion (YY cannot equal ZZ since this would do nothing):

3.4  Matrix Functions

F01ECF and F01FCF compute the matrix exponential, eA, for a real and complex square matrix A respectively. F01EDF and F01FDF compute the matrix exponential for a real symmetric and complex Hermitian matrix respectively. If the matrix is real symmetric, or complex Hermitian then it is recommended that F01EDF, or F01FDF be used as they are more efficient and, in general, more accurate than F01ECF and F01FCF.
Routines F01EFF and F01FFF compute the matrix function, fA, of a real symmetric and complex Hermitian matrix A respectively.

4  Decision Trees

The decision trees show the routines in this chapter and in Chapter F04 that should be used for inverting matrices of various types.
(i) Matrix Inversion:

Tree 1

Is A an n by n matrix of rank n? _
yes
Is A a real matrix? _
yes
see Tree 2
| no
|
| see Tree 3
no
|
see Tree 4

Tree 2: Inverse of a real n by n matrix of full rank

Is A a band matrix? _
yes
See Note 1.
no
|
Is A symmetric? _
yes
Is A positive definite? _
yes
Do you want guaranteed accuracy? (See Note 2) _
yes
F01ABF
| | no
|
| | Is one triangle of A stored as a linear array? _
yes
F07GDF and F07GJF
| | no
|
| | F01ADF or F07FDF and F07FJF
| no
|
| Is one triangle of A stored as a linear array? _
yes
F07PDF and F07PJF
| no
|
| F07MDF and F07MJF
no
|
Is A triangular? _
yes
Is A stored as a linear array? _
yes
F07UJF
| no
|
| F07TJF
no
|
Do you want guaranteed accuracy? (See Note 2) _
yes
F04AEF
no
|
F07ADF and F07AJF

Tree 3: Inverse of a complex n by n matrix of full rank

Is A a band matrix? _
yes
See Note 1.
no
|
Is A Hermitian? _
yes
Is A positive definite? _
yes
Is one triangle of A stored as a linear array? _
yes
F07GRF and F07GWF
| | no
|
| | F07FRF and F07FWF
| no
|
| Is one triangle A stored as a linear array? _
yes
F07PRF and F07PWF
| no
|
| F07MRF and F07MWF
no
|
Is A symmetric? _
yes
Is one triangle of A stored as a linear array? _
yes
F07QRF and F07QWF
| no
|
| F07NRF and F07NWF
no
|
Is A triangular? _
yes
Is A stored as a linear array? _
yes
F07UWF
| no
|
| F07TWF
no
|
F07ANF or F07ARF and F07AWF

Tree 4: Pseudo-inverses

Is A a complex matrix? _
yes
Is A of full rank? _
yes
Is A an m by n matrix with m<n? _
yes
F01RJF and F01RKF
| | no
|
| | F08ASF and F08AUF or F08ATF
| no
|
| F08KPF
no
|
Is A of full rank? _
yes
Is A an m by n matrix with m<n? _
yes
F01QJF and F01QKF
| no
|
| F08AEF and F08AGF or F08AFF
no
|
Is A an m by n matrix with m<n? _
yes
F08KBF
no
|
Is reliability more important than efficiency? _
yes
F08KBF
no
|
F01BLF
(ii) Matrix Factorizations: see the decision trees in Section 4 in the F02 and F04 Chapter Introductions.
(iii) Matrix Arithmetic and Manipulation: not appropriate.
(iv) Matrix Functions: not required.
Note 1: the inverse of a band matrix A does not in general have the same shape as A, and no routines are provided specifically for finding such an inverse. The matrix must either be treated as a full matrix, or the equations AX=B must be solved, where B has been initialized to the identity matrix I. In the latter case, see the decision trees in Section 4 in the F04 Chapter Introduction.
Note 2: by ‘guaranteed accuracy’ we mean that the accuracy of the inverse is improved by use of the iterative refinement technique using additional precision.

5  Functionality Index

Inversion (also see Chapter F07), 
    real m by n matrix, 
        pseudo inverse F01BLF
    real symmetric positive definite matrix, 
        accurate inverse F01ABF
        approximate inverse F01ADF
Matrix Arithmetic and Manipulation, 
    matrix addition, 
        complex matrices F01CWF
        real matrices F01CTF
    matrix multiplication F01CKF
    matrix storage conversion, 
        packed band  ↔  rectangular storage, 
            complex matrices F01ZDF
            real matrices F01ZCF
        packed triangular  ↔  square storage, 
            complex matrices F01ZBF
            real matrices F01ZAF
    matrix subtraction, 
        complex matrices F01CWF
        real matrices F01CTF
    matrix transpose F01CRF
    packed triangular to Rectangular Full Packed storage, 
        complex matrices F01VKF (ZTPTTF)
        real matrices F01VJF (DTPTTF)
    packed triangular to square storage, 
        complex matrices F01VDF (ZTPTTR)
        real matrices F01VCF (DTPTTR)
    Rectangular Full Packed to packed triangular storage, 
        complex matrices F01VMF (ZTFTTP)
        real matrices F01VLF (DTFTTP)
    rectangular packed to square storage, 
        complex matrices F01VHF (ZTFTTR)
        real matrices F01VGF (DTFTTR)
    square to packed triangular storage, 
        complex matrices F01VBF (ZTRTTP)
        real matrices F01VAF (DTRTTP)
    square to Rectangular Full Packed storage, 
        complex matrix F01VFF (ZTRTTF)
        real matrix F01VEF (DTRTTF)
Matrix Function 
    complex Hermitian n by n matrix 
        matrix exponential F01FDF
        matrix function F01FFF
    complex n by n matrix 
        matrix exponential F01FCF
    real n by n matrix 
        matrix exponential F01ECF
    real symmetric n by n matrix 
        matrix exponential F01EDF
        matrix function F01EFF
Matrix Transformations, 
    complex matrix, form unitary matrix F01RKF
    complex m by n(m ≤ n) matrix, 
        RQ factorization F01RJF
    complex upper trapezoidal matrix, 
        RQ factorization F01RGF
    eigenproblem Ax = λBx, A, B banded, 
        reduction to standard symmetric problem F01BVF
    real almost block-diagonal matrix, 
        LU factorization F01LHF
    real band symmetric positive definite matrix, 
        ULDLTUT factorization F01BUF
        variable bandwidth, LDLT factorization F01MCF
    real matrix, 
        form orthogonal matrix F01QKF
    real m by n(m  ≤  n) matrix, 
        RQ factorization F01QJF
    real sparse matrix, 
        factorization F01BRF
        factorization, known sparsity pattern F01BSF
    real upper trapezoidal matrix, 
        RQ factorization F01QGF
    tridiagonal matrix, 
        LU factorization F01LEF

6  Auxiliary Routines Associated with Library Routine Parameters

None.

7  Routines Withdrawn or Scheduled for Withdrawal

Withdrawn
Routine
Mark of
Withdrawal

Replacement Routine(s)
F01AAF17F07ADF (DGETRF) and F07AJF (DGETRI)
F01ACF16F01ABF
F01AEF18F07FDF (DPOTRF), F08SEF (DSYGST) and F06EGF (DSWAP)
F01AFF18F06EGF (DSWAP) and F06YJF (DTRSM)
F01AGF18F08FEF (DSYTRD)
F01AHF18F08FGF (DORMTR)
F01AJF18F08FEF (DSYTRD) and F08FFF (DORGTR)
F01AKF18F08NEF (DGEHRD)
F01ALF18F08NGF (DORMHR)
F01AMF18F08NSF (ZGEHRD)
F01ANF18F08NUF (ZUNMHR)
F01APF18F06QFF and F08NFF (DORGHR)
F01ATF18F08NHF (DGEBAL)
F01AUF18F08NJF (DGEBAK)
F01AVF18F08NVF (ZGEBAL)
F01AWF18F08NWF (ZGEBAK)
F01AXF18F06EFF (DCOPY) and F08BEF (DGEQPF)
F01AYF18F08GEF (DSPTRD)
F01AZF18F08GGF (DOPMTR)
F01BCF18F08FSF (ZHETRD) and F08FTF (ZUNGTR)
F01BDF18F07FDF (DPOTRF), F08SEF (DSYGST) and F06EGF (DSWAP)
F01BEF18F06YFF (DTRMM) and F06EGF (DSWAP)
F01BNF17F07FRF (ZPOTRF)
F01BPF17F07FRF (ZPOTRF) and F07FWF (ZPOTRI)
F01BQF16F07GDF (DPPTRF) or F07PDF (DSPTRF)
F01BTF18F07ADF (DGETRF)
F01BWF18F08HEF (DSBTRD)
F01BXF17F07FDF (DPOTRF)
F01CLF16F06YAF (DGEMM)
F01LBF18F07BDF (DGBTRF)
F01MAF19F11JAF
F01NAF17F07BRF (ZGBTRF)
F01QCF18F08AEF (DGEQRF)
F01QDF18F08AGF (DORMQR)
F01QEF18F08AFF (DORGQR)
F01QFF18F08BEF (DGEQPF)
F01RCF18F08ASF (ZGEQRF)
F01RDF18F08AUF (ZUNMQR)
F01REF18F08ATF (ZUNGQR)
F01RFF18F08BSF (ZGEQPF)

8  References

Golub G H and Van Loan C F (1996) Matrix Computations (3rd Edition) Johns Hopkins University Press, Baltimore
Wilkinson J H (1965) The Algebraic Eigenvalue Problem Oxford University Press, Oxford
Wilkinson J H (1977) Some recent advances in numerical linear algebra The State of the Art in Numerical Analysis (ed D A H Jacobs) Academic Press
Wilkinson J H and Reinsch C (1971) Handbook for Automatic Computation II, Linear Algebra Springer–Verlag

F01 Chapter Contents
F01 Chapter Introduction (PDF version)
NAG Library Manual

© The Numerical Algorithms Group Ltd, Oxford, UK. 2011