NAG Library Routine Document
E02GBF
1 Purpose
E02GBF calculates an l1 solution to an over-determined system of linear equations, possibly subject to linear inequality constraints.
2 Specification
SUBROUTINE E02GBF ( |
M, N, MPL, E, LDE, F, X, MXS, MONIT, IPRINT, K, EL1N, INDX, W, IW, IFAIL) |
INTEGER |
M, N, MPL, LDE, MXS, IPRINT, K, INDX(MPL), IW, IFAIL |
REAL (KIND=nag_wp) |
E(LDE,MPL), F(MPL), X(N), EL1N, W(IW) |
EXTERNAL |
MONIT |
|
3 Description
Given a matrix
A with
m rows and
n columns
m≥n and a vector
b with
m elements, the routine calculates an
l1 solution to the over-determined system of equations
That is to say, it calculates a vector
x, with
n elements, which minimizes the
l1-norm (the sum of the absolute values) of the residuals
where the residuals
ri are given by
Here
aij is the element in row
i and column
j of
A,
bi is the
ith element of
b and
xj the
jth element of
x.
If, in addition, a matrix C with l rows and n columns and a vector d with l elements, are given, the vector x computed by the routine is such as to minimize the l1-norm rx subject to the set of inequality constraints Cx≥d.
The matrices A and C need not be of full rank.
Typically in applications to data fitting, data consisting of
m points with coordinates
ti,yi is to be approximated by a linear combination of known functions
ϕit,
in the
l1-norm, possibly subject to linear inequality constraints on the coefficients
αj of the form
Cα≥d where
α is the vector of the
αj and
C and
d are as in the previous paragraph. This is equivalent to finding an
l1 solution to the over-determined system of equations
subject to
Cα≥d.
Thus if, for each value of i and j, the element aij of the matrix A above is set equal to the value of ϕjti and bi is equal to yi and C and d are also supplied to the routine, the solution vector x will contain the required values of the αj. Note that the independent variable t above can, instead, be a vector of several independent variables (this includes the case where each of ϕi is a function of a different variable, or set of variables).
The algorithm follows the Conn–Pietrzykowski approach (see
Bartels et al. (1978) and
Conn and Pietrzykowski (1977)), which is via an exact penalty function
where
γ is a penalty parameter,
ciT
is the
ith row of the matrix
C, and
di is the
ith element of the vector
d. It proceeds in a step-by-step manner much like the simplex method for linear programming but does not move from vertex to vertex and does not require the problem to be cast in a form containing only non-negative unknowns. It uses stable procedures to update an orthogonal factorization of the current set of active equations and constraints.
4 References
Bartels R H, Conn A R and Charalambous C (1976) Minimisation techniques for piecewise Differentiable functions – the
l∞ solution to an overdetermined linear system
Technical Report No. 247, CORR 76/30 Mathematical Sciences Department, The John Hopkins University
Bartels R H, Conn A R and Sinclair J W (1976) A Fortran program for solving overdetermined systems of linear equations in the
l1 Sense
Technical Report No. 236, CORR 76/7 Mathematical Sciences Department, The John Hopkins University
Bartels R H, Conn A R and Sinclair J W (1978) Minimisation techniques for piecewise differentiable functions – the
l1 solution to an overdetermined linear system
SIAM J. Numer. Anal. 15 224–241
Conn A R and Pietrzykowski T (1977) A penalty-function method converging directly to a constrained optimum
SIAM J. Numer. Anal. 14 348–375
5 Parameters
- 1: M – INTEGERInput
On entry: the number of equations in the over-determined system, m (i.e., the number of rows of the matrix A).
Constraint:
M≥2.
- 2: N – INTEGERInput
On entry: the number of unknowns, n (the number of columns of the matrix A).
Constraint:
N≥2.
- 3: MPL – INTEGERInput
On entry: m+l, where l is the number of constraints (which may be zero).
Constraint:
MPL≥M.
- 4: E(LDE,MPL) – REAL (KIND=nag_wp) arrayInput/Output
On entry: the equation and constraint matrices stored in the following manner.
The first m columns contain the m rows of the matrix A; element
Eij specifying the element aji in the jth row and ith column of A (the coefficient of the ith unknown in the jth equation), for i=1,2,…,n and j=1,2,…,m. The next l columns contain the l rows of the constraint matrix C; element
Eij+m containing the element cji in the jth row and ith column of C (the coefficient of the ith unknown in the jth constraint), for i=1,2,…,n and j=1,2,…,l.
On exit: unchanged, except possibly to the extent of a small multiple of the
machine precision. (See
Section 8.)
- 5: LDE – INTEGERInput
On entry: the first dimension of the array
E as declared in the (sub)program from which E02GBF is called.
Constraint:
LDE≥N.
- 6: F(MPL) – REAL (KIND=nag_wp) arrayInput
On entry: Fi, for i=1,2,…,m, must contain
bi (the ith element of the right-hand side vector of the over-determined system of equations) and Fm+i, for i=1,2,…,l, must contain di (the ith element of the right-hand side vector of the constraints), where l is the number of constraints.
- 7: X(N) – REAL (KIND=nag_wp) arrayInput/Output
On entry: Xi must contain an estimate of the ith unknown, for i=1,2,…,n. If no better initial estimate for Xi is available, set Xi=0.0.
On exit: the latest estimate of the
ith unknown, for i=1,2,…,n. If IFAIL=0 on exit, these are the solution values.
- 8: MXS – INTEGERInput
On entry: the maximum number of steps to be allowed for the solution of the unconstrained problem. Typically this may be a modest multiple of
n. If, on entry,
MXS is zero or negative, the value returned by
X02BBF is used.
- 9: MONIT – SUBROUTINE, supplied by the user.External Procedure
MONIT can be used to print out the current values of any selection of its parameters. The frequency with which
MONIT is called in E02GBF is controlled by
IPRINT.
The specification of
MONIT is:
INTEGER |
N, NITER, K |
REAL (KIND=nag_wp) |
X(N), EL1N |
|
- 1: N – INTEGERInput
On entry: the number n of unknowns (the number of columns of the matrix A).
- 2: X(N) – REAL (KIND=nag_wp) arrayInput
On entry: the latest estimate of the unknowns.
- 3: NITER – INTEGERInput
On entry: the number of iterations so far carried out.
- 4: K – INTEGERInput
On entry: the total number of equations and constraints which are currently active (i.e., the number of equations with zero residuals plus the number of constraints which are satisfied as equations).
- 5: EL1N – REAL (KIND=nag_wp)Input
On entry: the l1-norm of the current residuals of the over-determined system of equations.
MONIT must either be a module subprogram USEd by, or declared as EXTERNAL in, the (sub)program from which E02GBF is called. Parameters denoted as
Input must
not be changed by this procedure.
- 10: IPRINT – INTEGERInput
On entry: the frequency of iteration print out.
- IPRINT>0
- MONIT is called every IPRINT iterations and at the solution.
- IPRINT=0
- Information is printed out at the solution only. Otherwise MONIT is not called (but a dummy routine must still be provided).
- 11: K – INTEGEROutput
On exit: the total number of equations and constraints which are then active (i.e., the number of equations with zero residuals plus the number of constraints which are satisfied as equalities).
- 12: EL1N – REAL (KIND=nag_wp)Output
On exit: the l1-norm (sum of absolute values) of the equation residuals.
- 13: INDX(MPL) – INTEGER arrayOutput
On exit: specifies which columns of
E relate to the inactive equations and constraints.
INDX1 up to
INDXK number the active columns and
INDXK+1 up to
INDXMPL number the inactive columns.
- 14: W(IW) – REAL (KIND=nag_wp) arrayWorkspace
- 15: IW – INTEGERInput
On entry: the dimension of the array
W as declared in the (sub)program from which E02GBF is called.
Constraint:
IW≥3×MPL+5×N+N2+N+1×N+2/2.
- 16: IFAIL – INTEGERInput/Output
-
On entry:
IFAIL must be set to
0,
-1 or 1. If you are unfamiliar with this parameter you should refer to
Section 3.3 in the Essential Introduction for details.
For environments where it might be inappropriate to halt program execution when an error is detected, the value
-1 or 1 is recommended. If the output of error messages is undesirable, then the value
1 is recommended. Otherwise, if you are not familiar with this parameter, the recommended value is
0.
When the value -1 or 1 is used it is essential to test the value of IFAIL on exit.
On exit:
IFAIL=0 unless the routine detects an error or a warning has been flagged (see
Section 6).
6 Error Indicators and Warnings
If on entry
IFAIL=0 or
-1, explanatory error messages are output on the current error message unit (as defined by
X04AAF).
Errors or warnings detected by the routine:
- IFAIL=1
The constraints cannot all be satisfied simultaneously: they are not compatible with one another. Hence no solution is possible.
- IFAIL=2
The limit imposed by
MXS has been reached without finding a solution. Consider restarting from the current point by simply calling E02GBF again without changing the parameters.
- IFAIL=3
The routine has failed because of numerical difficulties; the problem is too ill-conditioned. Consider rescaling the unknowns.
- IFAIL=4
On entry, one or more of the following conditions are violated:
- M≥N≥2,
- or MPL≥M,
- or IW≥3×MPL+5×N+N2+N+1×N+2/2,
- or LDE≥N.
Alternatively elements
1 to
M of one of the first
MPL columns of the array
E are all zero – this corresponds to a zero row in either of the matrices
A or
C.
7 Accuracy
The method is stable.
8 Further Comments
The effect of m and n on the time and on the number of iterations varies from problem to problem, but typically the number of iterations is a small multiple of n and the total time taken is approximately proportional to mn2.
Linear dependencies among the rows or columns of
A and
C are not necessarily a problem to the algorithm. Solutions can be obtained from rank-deficient
A and
C. However, the algorithm requires that at every step the currently active columns of
E form a linearly independent set. If this is not the case at any step, small, random perturbations of the order of rounding error are added to the appropriate columns of
E. Normally this perturbation process will not affect the solution significantly. It does mean, however, that results may not be exactly reproducible.
9 Example
Suppose we wish to approximate in
0,1 a set of data by a curve of the form
which has non-negative slope at the data points. Given points
ti,yi we may form the equations
for
i=1,2,…,6, for the
6 data points. The requirement of a non-negative slope at the data points demands
for each
ti and these form the constraints.
(Note that, for fitting with polynomials, it would usually be advisable to work with the polynomial expressed in Chebyshev series form (see the
E02 Chapter Introduction). The power series form is used here for simplicity of exposition.)
9.1 Program Text
Program Text (e02gbfe.f90)
9.2 Program Data
Program Data (e02gbfe.d)
9.3 Program Results
Program Results (e02gbfe.r)