NAG CL Interface
f01sac (real_nmf)
1
Purpose
f01sac computes a nonnegative matrix factorization for a real nonnegative $m$ by $n$ matrix $A$.
2
Specification
void 
f01sac (Integer m,
Integer n,
Integer k,
const double a[],
Integer pda,
double w[],
Integer pdw,
double h[],
Integer pdh,
Integer seed,
double errtol,
Integer maxit,
NagError *fail) 

The function may be called by the names: f01sac or nag_matop_real_nmf.
3
Description
The matrix
$A$ is factorized into the product of an
$m$ by
$k$ matrix
$W$ and a
$k$ by
$n$ matrix
$H$, both with nonnegative elements. The factorization is approximate,
$A\approx WH$, with
$W$ and
$H$ chosen to minimize the functional
You are free to choose any value for $k$, provided $k<\mathrm{min}\phantom{\rule{0.125em}{0ex}}\left(m,n\right)$. The product $WH$ will then be a lowrank approximation to $A$, with rank at most $k$.
f01sac finds $W$ and $H$ using an iterative method known as the Hierarchical Alternating Least Squares algorithm. You may specify initial values for $W$ and $H$, or you may provide a seed value for f01sac to generate the initial values using a random number generator.
4
References
Cichocki A and Phan A–H (2009) Fast local algorithms for large scale nonnegative matrix and tensor factorizations IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E92–A 708–721
Cichocki A, Zdunek R and Amari S–I (2007) Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization Lecture Notes in Computer Science 4666 Springer 169–176
Ho N–D (2008) Nonnegative matrix factorization algorithms and applications PhD Thesis Univ. Catholique de Louvain
5
Arguments

1:
$\mathbf{m}$ – Integer
Input

On entry: $m$, the number of rows of the matrix $A$. Also the number of rows of the matrix $W$.
Constraint:
${\mathbf{m}}\ge 2$.

2:
$\mathbf{n}$ – Integer
Input

On entry: $n$, the number of columns of the matrix $A$. Also the number of columns of the matrix $H$.
Constraint:
${\mathbf{n}}\ge 2$.

3:
$\mathbf{k}$ – Integer
Input

On entry:
$k$, the number of columns of the matrix
$W$; the number of rows of the matrix
$H$. See
Section 9.2 for further details.
Constraint:
$1\le {\mathbf{k}}<\mathrm{min}\phantom{\rule{0.125em}{0ex}}\left({\mathbf{m}},{\mathbf{n}}\right)$.

4:
$\mathbf{a}\left[\mathit{dim}\right]$ – const double
Input

Note: the dimension,
dim, of the array
a
must be at least
${\mathbf{pda}}\times {\mathbf{n}}$.
The $\left(i,j\right)$th element of the matrix $A$ is stored in ${\mathbf{a}}\left[\left(j1\right)\times {\mathbf{pda}}+i1\right]$.
On entry: the $m$ by $n$ nonnegative matrix $A$.

5:
$\mathbf{pda}$ – Integer
Input

On entry: the stride separating matrix row elements in the array
a.
Constraint:
${\mathbf{pda}}\ge {\mathbf{m}}$.

6:
$\mathbf{w}\left[\mathit{dim}\right]$ – double
Input/Output

Note: the dimension,
dim, of the array
w
must be at least
${\mathbf{pdw}}\times {\mathbf{k}}$.
The $\left(i,j\right)$th element of the matrix $W$ is stored in ${\mathbf{w}}\left[\left(j1\right)\times {\mathbf{pdw}}+i1\right]$.
On entry:
 if ${\mathbf{seed}}\le 0$, w should be set to an initial iterate for the nonnegative matrix factor, $W$.
 If ${\mathbf{seed}}\ge 1$, w need not be set. f01sac will generate a random initial iterate.
On exit: the nonnegative matrix factor, $W$.

7:
$\mathbf{pdw}$ – Integer
Input

On entry: the stride separating matrix row elements in the array
w.
Constraint:
${\mathbf{pdw}}\ge {\mathbf{m}}$.

8:
$\mathbf{h}\left[\mathit{dim}\right]$ – double
Input/Output

Note: the dimension,
dim, of the array
h
must be at least
${\mathbf{pdh}}\times {\mathbf{n}}$.
The $\left(i,j\right)$th element of the matrix $H$ is stored in ${\mathbf{h}}\left[\left(j1\right)\times {\mathbf{pdh}}+i1\right]$.
On entry:
 if ${\mathbf{seed}}\le 0$, h should be set to an initial iterate for the nonnegative matrix factor, $H$.
 If ${\mathbf{seed}}\ge 1$, h need not be set. f01sac will generate a random initial iterate.
On exit: the nonnegative matrix factor, $H$.

9:
$\mathbf{pdh}$ – Integer
Input

On entry: the stride separating matrix row elements in the array
h.
Constraint:
${\mathbf{pdh}}\ge {\mathbf{k}}$.

10:
$\mathbf{seed}$ – Integer
Input

On entry:
 if ${\mathbf{seed}}\le 0$, the supplied values of $W$ and $H$ are used for the initial iterate.
 If ${\mathbf{seed}}\ge 1$, the value of seed is used to seed a random number generator for the initial iterates $W$ and $H$. See Section 9.3 for further details.

11:
$\mathbf{errtol}$ – double
Input

On entry: the convergence tolerance for when the Hierarchical Alternating Least Squares iteration has reached a stationary point. If ${\mathbf{errtol}}\le 0.0$, $\mathrm{max}\phantom{\rule{0.125em}{0ex}}\left({\mathbf{m}},{\mathbf{n}}\right)\times \sqrt{\mathit{machineprecision}}$ is used.

12:
$\mathbf{maxit}$ – Integer
Input

On entry: specifies the maximum number of iterations to be used. If ${\mathbf{maxit}}\le 0$, $200$ is used.

13:
$\mathbf{fail}$ – NagError *
Input/Output

The NAG error argument (see
Section 7 in the Introduction to the NAG Library CL Interface).
6
Error Indicators and Warnings
 NE_ALLOC_FAIL

Dynamic memory allocation failed.
See
Section 3.1.2 in the Introduction to the NAG Library CL Interface for further information.
 NE_BAD_PARAM

On entry, argument $\u2329\mathit{\text{value}}\u232a$ had an illegal value.
 NE_CONVERGENCE

The function has failed to converge after
$\u2329\mathit{\text{value}}\u232a$ iterations. The factorization given by
w and
h may still be a good enough approximation to be useful. Alternatively an improved factorization may be obtained by increasing
maxit or using different initial choices of
w and
h.
 NE_INIT_ESTIMATE

An internal error occurred when generating initial values for
w and
h. Please contact
NAG.
 NE_INT

On entry, ${\mathbf{m}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{m}}\ge 2$.
On entry, ${\mathbf{n}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{n}}\ge 2$.
 NE_INT_2

On entry, ${\mathbf{pda}}=\u2329\mathit{\text{value}}\u232a$ and ${\mathbf{m}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{pda}}\ge {\mathbf{m}}$.
On entry, ${\mathbf{pdh}}=\u2329\mathit{\text{value}}\u232a$ and ${\mathbf{k}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{pdh}}\ge {\mathbf{k}}$.
On entry, ${\mathbf{pdw}}=\u2329\mathit{\text{value}}\u232a$ and ${\mathbf{m}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: ${\mathbf{pdw}}\ge {\mathbf{m}}$.
 NE_INT_3

On entry, ${\mathbf{k}}=\u2329\mathit{\text{value}}\u232a$, ${\mathbf{m}}=\u2329\mathit{\text{value}}\u232a$ and ${\mathbf{n}}=\u2329\mathit{\text{value}}\u232a$.
Constraint: $1\le {\mathbf{k}}<\mathrm{min}\phantom{\rule{0.125em}{0ex}}\left({\mathbf{m}},{\mathbf{n}}\right)$.
 NE_INTERNAL_ERROR

An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact
NAG for assistance.
See
Section 7.5 in the Introduction to the NAG Library CL Interface for further information.
 NE_INVALID_ARRAY

On entry, one of more of the elements of
a,
w or
h were negative.
 NE_NO_LICENCE

Your licence key may have expired or may not have been installed correctly.
See
Section 8 in the Introduction to the NAG Library CL Interface for further information.
7
Accuracy
The Hierarchical Alternating Least Squares algorithm used by
f01sac is locally convergent; it is guaranteed to converge to a stationary point of
$f\left(W,H\right)$, but this may not be the global minimum. The iteration is deemed to have converged if the gradient of
$f\left(W,H\right)$ is less than
errtol times the gradient at the initial values of
$W$ and
$H$.
Due to the local convergence property, you may wish to run f01sac multiple times with different starting iterates. This can be done by explicitly providing the starting values of $W$ and $H$ each time, or by choosing a different random seed for each function call.
Note that even if
f01sac exits with
${\mathbf{fail}}\mathbf{.}\mathbf{code}=$ NE_CONVERGENCE, the factorization given by
$W$ and
$H$ may still be a good enough approximation to be useful.
8
Parallelism and Performance
f01sac is threaded by NAG for parallel execution in multithreaded implementations of the NAG Library.
f01sac makes calls to BLAS and/or LAPACK routines, which may be threaded within the vendor library used by this implementation. Consult the documentation for the vendor library for further information.
Please consult the
X06 Chapter Introduction for information on how to control and interrogate the OpenMP environment used within this function. Please also consult the
Users' Note for your implementation for any additional implementationspecific information.
Each iteration of the Hierarchical Alternating Least Squares algorithm requires $\mathit{O}\left(mnk\right)$ floatingpoint operations.
The real allocatable memory required is $m\times n+k\left(m+n\right)$.
If
$A$ is large and sparse, then
f01sbc should be used to compute a nonnegative matrix factorization.
9.1
Uniqueness
Note that nonnegative matrix factorization is not unique. For a factorization given by the matrices $W$ and $H$, an equally good solution is given by $WD$ and ${D}^{1}H$, where $D$ is any real nonnegative $k\times k$ matrix whose inverse is also nonnegative. In f01sac, $W$ and $H$ are normalized so that the columns of $W$ have unit length.
9.2
Choice of $k$
The most appropriate choice of the factorization rank, $k$, is often problem dependent. Details of your particular application may help in guiding your choice of $k$, for example, it may be known a priori that the data in $A$ naturally falls into a certain number of categories.
Alternatively, trial and error can be used. Compute nonnegative matrix factorizations for several different values of $k$ (typically with $k\ll \mathrm{min}\phantom{\rule{0.125em}{0ex}}\left(m,n\right)$) and select the one that performs the best.
Finally, it is also possible to use a singular value decomposition of
$A$ to guide your choice of
$k$, by looking for an abrupt decay in the size of the singular values of
$A$. The singular value decomposition can be computed using
f08kbc.
9.3
Generating Random Initial Iterates
If
${\mathbf{seed}}\ge 1$ on entry, then
f01sac uses the functions
g05kfc and
g05sac, with the NAG basic generator, to populate
w and
h. For further information on this random number generator see
Section 2.1.1 in the
G05 Chapter Introduction.
Note that this generator gives a repeatable sequence of random numbers, so if the value of
seed is not changed between function calls, then the same initial iterates will be generated.
10
Example
This example finds a nonnegative matrix factorization for the matrix
10.1
Program Text
10.2
Program Data
10.3
Program Results