f01sbc computes a non-negative matrix factorization for a real non-negative $m\times n$ matrix $A$. It uses reverse communication for evaluating matrix products, so that the matrix $A$ is not accessed explicitly.
The function may be called by the names: f01sbc or nag_matop_real_nmf_rcomm.
3Description
The matrix $A$ is factorized into the product of an $m\times k$ matrix $W$ and a $k\times n$ matrix $H$, both with non-negative elements. The factorization is approximate, $A\approx WH$, with $W$ and $H$ chosen to minimize the functional
$$f(W,H)={\Vert A-WH\Vert}_{F}^{2}\text{.}$$
You are free to choose any value for $k$, provided $k<\mathrm{min}\phantom{\rule{0.125em}{0ex}}(m,n)$. The product $WH$ will then be a low-rank approximation to $A$, with rank at most $k$.
f01sbc finds $W$ and $H$ using an iterative method known as the Hierarchical Alternating Least Squares algorithm. You may specify initial values for $W$ and $H$, or you may provide a seed value for f01sbc to generate the initial values using a random number generator.
f01sbc does not explicitly need to access the elements of $A$; it only requires the result of matrix multiplications of the form $AX$ or ${A}^{\mathrm{T}}Y$. A reverse communication interface is used, in which control is returned to the calling program whenever a matrix product is required.
4References
Cichocki A and Phan A–H (2009) Fast local algorithms for large scale nonnegative matrix and tensor factorizations IEICE Transactions on Fundamentals of Electronics, Communications and Computer SciencesE92–A 708–721
Cichocki A, Zdunek R and Amari S–I (2007) Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization Lecture Notes in Computer Science4666 Springer 169–176
Ho N–D (2008) Nonnegative matrix factorization algorithms and applications PhD Thesis Univ. Catholique de Louvain
5Arguments
Note: this function uses reverse communication. Its use involves an initial entry, intermediate exits and re-entries, and a final exit, as indicated by the argument irevcm. Between intermediate exits and re-entries, all arguments other thanw and ht must remain unchanged.
1: $\mathbf{irevcm}$ – Integer *Input/Output
On initial entry: must be set to $0$.
On intermediate exit:
specifies what action you must take before re-entering f01sbc with irevcm unchanged. The value of irevcm should be interpreted as follows:
${\mathbf{irevcm}}=1$
Indicates the start of a new iteration. No action is required by you, but w and h are available for printing, and a limit on the number of iterations can be applied.
${\mathbf{irevcm}}=2$
Indicates that before re-entry to f01sbc, the product ${A}^{\mathrm{T}}W$ must be computed and stored in ht.
${\mathbf{irevcm}}=3$
Indicates that before re-entry to f01sbc, the product $A{H}^{\mathrm{T}}$ must be computed and stored in w.
On final exit: ${\mathbf{irevcm}}=0$.
Note: any values you return to f01sbc as part of the reverse communication procedure should not include floating-point NaN (Not a Number) or infinity values, since these are not handled by f01sbc. If your code inadvertently does return any NaNs or infinities, f01sbc is likely to produce unexpected results.
2: $\mathbf{m}$ – IntegerInput
On entry: $m$, the number of rows of the matrix $A$. Also the number of rows of the matrix $W$.
Constraint:
${\mathbf{m}}\ge 2$.
3: $\mathbf{n}$ – IntegerInput
On entry: $n$, the number of columns of the matrix $A$. Also the number of columns of the matrix $H$.
Constraint:
${\mathbf{n}}\ge 2$.
4: $\mathbf{k}$ – IntegerInput
On entry: $k$, the number of columns of the matrix $W$. Also, the number of rows of the matrix $H$. See Section 9.2 for further details.
On intermediate exit:
if ${\mathbf{irevcm}}=3$, ht contains the $n\times k$ non-negative matrix ${H}^{\mathrm{T}}$, which is required in order to form $A{H}^{\mathrm{T}}$.
On intermediate re-entry: if ${\mathbf{irevcm}}=2$, ht must contain ${A}^{\mathrm{T}}W$.
If ${\mathbf{irevcm}}=0$, $1$ or $3$, ht must not be changed.
On entry: the stride separating matrix row elements in the array ht.
Constraint:
${\mathbf{pdht}}\ge {\mathbf{n}}$.
11: $\mathbf{seed}$ – IntegerInput
On initial entry:
if ${\mathbf{seed}}\le 0$, the supplied values of $W$ and $H$ are used for the initial iterate.
If ${\mathbf{seed}}\ge 1$, the value of seed is used to seed a random number generator for the initial iterates $W$ and $H$. See Section 9.3 for further details.
12: $\mathbf{errtol}$ – doubleInput
On entry: the convergence tolerance for when the Hierarchical Alternating Least Squares iteration has reached a stationary point. If ${\mathbf{errtol}}\le 0.0$, $\mathrm{max}\phantom{\rule{0.125em}{0ex}}({\mathbf{m}},{\mathbf{n}})\times \sqrt{\mathit{machineprecision}}$ is used.
The NAG error argument (see Section 7 in the Introduction to the NAG Library CL Interface).
6Error Indicators and Warnings
NE_ALLOC_FAIL
Dynamic memory allocation failed.
See Section 3.1.2 in the Introduction to the NAG Library CL Interface for further information.
NE_BAD_PARAM
On entry, argument $\u27e8\mathit{\text{value}}\u27e9$ had an illegal value.
NE_INIT_ESTIMATE
An internal error occurred when generating initial values for w and h. Please contact NAG.
NE_INT
On entry, ${\mathbf{m}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{m}}\ge 2$.
On entry, ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{n}}\ge 2$.
On initial entry, ${\mathbf{irevcm}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{irevcm}}=0$.
On intermediate re-entry, ${\mathbf{irevcm}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{irevcm}}=1$, $2$ or $3$.
NE_INT_2
On entry, ${\mathbf{pdh}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{k}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{pdh}}\ge {\mathbf{k}}$.
On entry, ${\mathbf{pdht}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{pdht}}\ge {\mathbf{n}}$.
On entry, ${\mathbf{pdw}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{m}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: ${\mathbf{pdw}}\ge {\mathbf{m}}$.
NE_INT_3
On entry, ${\mathbf{k}}=\u27e8\mathit{\text{value}}\u27e9$, ${\mathbf{m}}=\u27e8\mathit{\text{value}}\u27e9$ and ${\mathbf{n}}=\u27e8\mathit{\text{value}}\u27e9$. Constraint: $1\le {\mathbf{k}}<\mathrm{min}\phantom{\rule{0.125em}{0ex}}({\mathbf{m}},{\mathbf{n}})$.
NE_INTERNAL_ERROR
An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact NAG for assistance.
See Section 7.5 in the Introduction to the NAG Library CL Interface for further information.
NE_INVALID_ARRAY
On entry, one of more of the elements of w or h were negative.
NE_NO_LICENCE
Your licence key may have expired or may not have been installed correctly.
See Section 8 in the Introduction to the NAG Library CL Interface for further information.
7Accuracy
The Hierarchical Alternating Least Squares algorithm used by f01sbc is locally convergent; it is guaranteed to converge to a stationary point of $f(W,H)$, but this may not be the global minimum. The iteration is deemed to have converged if the gradient of $f(W,H)$ is less than errtol times the gradient at the initial values of $W$ and $H$.
Due to the local convergence property, you may wish to run f01sbc multiple times with different starting iterates. This can be done by explicitly providing the starting values of $W$ and $H$ each time, or by choosing a different random seed each time.
Note that even if f01sbc exits with ${\mathbf{fail}}\mathbf{.}\mathbf{code}=$NE_INT_2, the factorization given by $W$ and $H$ may still be a good enough approximation to be useful.
8Parallelism and Performance
Background information to multithreading can be found in the Multithreading documentation.
f01sbc is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
f01sbc makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this function. Please also consult the Users' Note for your implementation for any additional implementation-specific information.
9Further Comments
f01sbc is designed to be used when $A$ is large and sparse. Whenever a matrix multiplication is required, the function will return control to the calling program so that the multiplication can be done in the most efficient way possible. Note that $W$ and $H$ will not, in general, be sparse even if $A$ is sparse.
If $A$ is small and dense, then f01sac can be used to compute $W$ and $H$ without the use of a reverse communication interface.
9.1Uniqueness
Note that non-negative matrix factorization is not unique. For a factorization given by the matrices $W$ and $H$, an equally good solution is given by $WD$ and ${D}^{\mathrm{-1}}H$, where $D$ is any real non-negative $k\times k$ matrix whose inverse is also non-negative. In f01sbc, $W$ and $H$ are normalized so that the columns of $W$ have unit length.
9.2Choice of $\mathit{k}$
The most appropriate choice of the factorization rank, $k$, is often problem dependent. Details of your particular application may help in guiding your choice of $k$, for example, it may be known a priori that the data in $A$ naturally falls into a certain number of categories.
Alternatively, trial and error can be used. Compute non-negative matrix factorizations for several different values of $k$ (typically with $k\ll \mathrm{min}\phantom{\rule{0.125em}{0ex}}(m,n)$) and select the one that performs the best.
Finally, it is also possible to use a singular value decomposition of $A$ to guide your choice of $k$, by looking for an abrupt decay in the size of the singular values of $A$. The singular value decomposition can be computed using f12fbc.
9.3Generating Random Initial Iterates
If ${\mathbf{seed}}\ge 1$ on entry, then f01sbc uses the functions g05kfcandg05sac, with the NAG basic generator, to populate w and h. For further information on this random number generator see Section 2.1.1 in the G05 Chapter Introduction.
Note that this generator gives a repeatable sequence of random numbers, so if the value of seed is not changed between function calls, then the same initial iterates will be generated.
9.4Use in Conjunction with NAG Library Functions
To compute the non-negative matrix factorization, the following
skeleton code can normally be used:
do {
f01sbc(&irevcm,m,n,k,w,ldw,h,ldh,ht,ldht,seed,errtol,comm,icomm,&ifail)
if (irevcm == 1) {
.. Print W and H if required and check number of iterations ..
}
else if (irevcm == 2) {
.. Compute A^TW and store in ht..
}
else if (irevcm == 3) {
.. Compute AH^T and store in w ..
}
} (while irevcm !=0)
The code used to compute the matrix products will vary depending on the way $A$ is stored. If all the elements of $A$ are stored explicitly, then f16yac) can be used. If $A$ is triangular, then f16yfc should be used. If $A$ is symmetric, then f16ycc should be used. For sparse $A$ stored in coordinate storage format f11xacandf11xec can be used. Alternatively, if $A$ is stored in compressed column format f11mkc can be used.
10Example
This example finds a non-negative matrix factorization for the matrix