# NAG C Library Function Document

## 1Purpose

nag_mv_kmeans_cluster_analysis (g03efc) performs $K$-means cluster analysis.

## 2Specification

 #include #include
 void nag_mv_kmeans_cluster_analysis (Integer n, Integer m, const double x[], Integer tdx, const Integer isx[], Integer nvar, Integer k, double cmeans[], Integer tdc, const double wt[], Integer inc[], Integer nic[], double css[], double csw[], Integer maxit, NagError *fail)

## 3Description

Given $n$ objects with $p$ variables measured on each object, ${x}_{ij}$ for $i=1,2,\dots ,n$ and $j=1,2,\dots ,p$, nag_mv_kmeans_cluster_analysis (g03efc) allocates each object to one of $K$ groups or clusters to minimize the within-cluster sum of squares:
 $∑ k=1 K ∑ i ∈ S k ∑ j=1 p x ij - x - kj 2 ,$
where ${S}_{k}$ is the set of objects in the $k$th cluster and ${\stackrel{-}{x}}_{kj}$ is the mean for the variable $j$ over cluster $k$. This is often known as $K$-means clustering.
In addition to the data matrix, a $K$ by $p$ matrix giving the initial cluster centres for the $K$ clusters is required. The objects are then initially allocated to the cluster with the nearest cluster mean. Given the initial allocation, the procedure is to iteratively search for the $K$-partition with locally optimal within-cluster sum of squares by moving points from one cluster to another.
Optionally, weights for each object, ${w}_{i}$, can be used so that the clustering is based on within-cluster weighted sums of squares:
 $∑ k=1 K ∑ i ∈ S k ∑ j=1 p w i x ij - x ~ kj 2 ,$
where ${\stackrel{~}{x}}_{kj}$ is the weighted mean for variable $j$ over cluster $k$.
The function is based on the algorithm of Hartigan and Wong (1979).
Everitt B S (1974) Cluster Analysis Heinemann
Hartigan J A and Wong M A (1979) Algorithm AS 136: A K-means clustering algorithm Appl. Statist. 28 100–108
Kendall M G and Stuart A (1976) The Advanced Theory of Statistics (Volume 3) (3rd Edition) Griffin
Krzanowski W J (1990) Principles of Multivariate Analysis Oxford University Press

## 5Arguments

1:    $\mathbf{n}$IntegerInput
On entry: the number of observations, $n$.
Constraint: ${\mathbf{n}}\ge 2$.
2:    $\mathbf{m}$IntegerInput
On entry: the number of variables in the array x.
Constraint: ${\mathbf{m}}\ge {\mathbf{nvar}}$.
3:    $\mathbf{x}\left[{\mathbf{n}}×{\mathbf{tdx}}\right]$const doubleInput
On entry: ${\mathbf{x}}\left[\left(\mathit{i}-1\right)×{\mathbf{tdx}}+\mathit{j}-1\right]$ must contain the value of $\mathit{j}$th variable for the $\mathit{i}$th object, for $\mathit{i}=1,2,\dots ,n$ and $\mathit{j}=1,2,\dots ,{\mathbf{m}}$.
4:    $\mathbf{tdx}$IntegerInput
On entry: the stride separating matrix column elements in the array x.
Constraint: ${\mathbf{tdx}}\ge {\mathbf{m}}$.
5:    $\mathbf{isx}\left[{\mathbf{m}}\right]$const IntegerInput
On entry: ${\mathbf{isx}}\left[j-1\right]$ indicates whether or not the $j$th variable is to be included in the analysis.
If ${\mathbf{isx}}\left[\mathit{j}-1\right]>0$, then the $\mathit{j}$th variable contained in the $\mathit{j}$th column of x is included, for $\mathit{j}=1,2,\dots ,{\mathbf{m}}$.
Constraint: ${\mathbf{isx}}\left[j-1\right]>0$ for nvar values of $j$.
6:    $\mathbf{nvar}$IntegerInput
On entry: the number of variables included in the sum of squares calculations, $p$.
Constraint: $1\le {\mathbf{nvar}}\le {\mathbf{m}}$.
7:    $\mathbf{k}$IntegerInput
On entry: the number of clusters, $K$.
Constraint: ${\mathbf{k}}\ge 2$.
8:    $\mathbf{cmeans}\left[{\mathbf{k}}×{\mathbf{tdc}}\right]$doubleInput/Output
On entry: ${\mathbf{cmeans}}\left[\left(\mathit{i}-1\right)×{\mathbf{tdc}}+\mathit{j}-1\right]$ must contain the value of the $\mathit{j}$th variable for the $\mathit{i}$th initial cluster centre, for $\mathit{i}=1,2,\dots ,K$ and $\mathit{j}=1,2,\dots ,p$.
On exit: ${\mathbf{cmeans}}\left[\left(\mathit{i}-1\right)×{\mathbf{tdc}}+\mathit{j}-1\right]$ contains the value of the $\mathit{j}$th variable for the $\mathit{i}$th computed cluster centre, for $\mathit{i}=1,2,\dots ,K$ and $\mathit{j}=1,2,\dots ,p$.
9:    $\mathbf{tdc}$IntegerInput
On entry: the stride separating matrix column elements in the array cmeans.
Constraint: ${\mathbf{tdc}}\ge {\mathbf{nvar}}$.
10:  $\mathbf{wt}\left[{\mathbf{n}}\right]$const doubleInput
On entry: the elements of wt must contain the weights to be used in the analysis. The effective number of observations is the sum of the weights. If ${\mathbf{wt}}\left[i-1\right]=0.0$ then the $i$th observation is not included in the analysis.
If weights are not provided then wt must be set to NULL and the effective number of observations is n.
Constraints:
• ${\mathbf{wt}}\left[\mathit{i}-1\right]\ge 0.0$, for $\mathit{i}=1,2,\dots ,n$;
• ${\mathbf{wt}}\left[i-1\right]>0.0$ for at least two values of $i$.
11:  $\mathbf{inc}\left[{\mathbf{n}}\right]$IntegerOutput
On exit: ${\mathbf{inc}}\left[\mathit{i}-1\right]$ contains the cluster to which the $\mathit{i}$th object has been allocated, for $\mathit{i}=1,2,\dots ,n$.
12:  $\mathbf{nic}\left[{\mathbf{k}}\right]$IntegerOutput
On exit: ${\mathbf{nic}}\left[\mathit{i}-1\right]$ contains the number of objects in the $\mathit{i}$th cluster, for $\mathit{i}=1,2,\dots ,K$.
13:  $\mathbf{css}\left[{\mathbf{k}}\right]$doubleOutput
On exit: ${\mathbf{css}}\left[\mathit{i}-1\right]$ contains the within-cluster (weighted) sum of squares of the $\mathit{i}$th cluster, for $\mathit{i}=1,2,\dots ,K$.
14:  $\mathbf{csw}\left[{\mathbf{k}}\right]$doubleOutput
On exit: ${\mathbf{csw}}\left[\mathit{i}-1\right]$ contains the within-cluster sum of weights of the $\mathit{i}$th cluster, for $\mathit{i}=1,2,\dots ,K$. If ${\mathbf{wt}}=\mathbf{NULL}$ the sum of weights is the number of objects in the cluster.
15:  $\mathbf{maxit}$IntegerInput
On entry: the maximum number of iterations allowed in the analysis.
Suggested value: ${\mathbf{maxit}}=10$.
Constraint: ${\mathbf{maxit}}>0$.
16:  $\mathbf{fail}$NagError *Input/Output
The NAG error argument (see Section 3.7 in How to Use the NAG Library and its Documentation).

## 6Error Indicators and Warnings

NE_2_INT_ARG_LT
On entry, ${\mathbf{m}}=〈\mathit{\text{value}}〉$ while ${\mathbf{nvar}}=〈\mathit{\text{value}}〉$. These arguments must satisfy ${\mathbf{m}}\ge {\mathbf{nvar}}$.
On entry, ${\mathbf{tdc}}=〈\mathit{\text{value}}〉$ while ${\mathbf{nvar}}=〈\mathit{\text{value}}〉$. These arguments must satisfy ${\mathbf{tdc}}\ge {\mathbf{nvar}}$.
On entry, ${\mathbf{tdx}}=〈\mathit{\text{value}}〉$ while ${\mathbf{m}}=〈\mathit{\text{value}}〉$. These arguments must satisfy ${\mathbf{tdx}}\ge {\mathbf{m}}$.
NE_ALLOC_FAIL
Dynamic memory allocation failed.
NE_CLUSTER_EMPTY
At least one cluster is empty after the initial assignment.
Try a different set of initial cluster centres in cmeans and also consider decreasing the value of k. The empty clusters may be found by examining the values in nic.
NE_INT_ARG_LE
On entry, ${\mathbf{maxit}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{maxit}}>0$.
NE_INT_ARG_LT
On entry, ${\mathbf{k}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{k}}\ge 2$.
On entry, ${\mathbf{n}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{n}}\ge 2$.
On entry, ${\mathbf{nvar}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{nvar}}\ge 1$.
NE_INTERNAL_ERROR
An internal error has occurred in this function. Check the function call and any array sizes. If the call is correct then please contact NAG for assistance.
NE_NEG_WEIGHT_ELEMENT
On entry, ${\mathbf{wt}}\left[〈\mathit{\text{value}}〉\right]=〈\mathit{\text{value}}〉$.
Constraint: When referenced, all elements of wt must be non-negative.
NE_TOO_MANY
Too many iterations ($〈\mathit{\text{value}}〉$). Convergence has not been achieved within the maximum number of iterations given by maxit. Try increasing maxit and, if possible, use the returned values in cmeans as the initial cluster centres.
NE_VAR_INCL_INDICATED
The number of variables, nvar in the analysis $\text{}=〈\mathit{\text{value}}〉$, while number of variables included in the analysis via array ${\mathbf{isx}}=〈\mathit{\text{value}}〉$.
Constraint: these two numbers must be the same.
NE_WT_ZERO
At least two elements of wt must be greater than zero.

## 7Accuracy

nag_mv_kmeans_cluster_analysis (g03efc) produces clusters that are locally optimal; the within-cluster sum of squares may not be decreased by transferring a point from one cluster to another, but different partitions may have the same or smaller within-cluster sum of squares.

## 8Parallelism and Performance

nag_mv_kmeans_cluster_analysis (g03efc) is not threaded in any implementation.

The time per iteration is approximately proportional to $npK$.

## 10Example

The data consists of observations of five variables on twenty soils (Kendall and Stuart (1976)). The data is read in, the $K$-means clustering performed and the results printed.

### 10.1Program Text

Program Text (g03efce.c)

### 10.2Program Data

Program Data (g03efce.d)

### 10.3Program Results

Program Results (g03efce.r)