# Source code for naginterfaces.library.correg

# -*- coding: utf-8 -*-
r"""
Module Summary
--------------
Interfaces for the NAG Mark 28.7 correg Chapter.

correg - Correlation and Regression Analysis

This module is concerned with two techniques

(i) correlation analysis and

(#) regression modelling,

both of which are concerned with determining the inter-relationships among two or more variables.

Other modules of the NAG Library which cover similar problems are submodule :mod:~naginterfaces.library.fit and submodule :mod:~naginterfaces.library.opt. Submodule :mod:~naginterfaces.library.fit functions may be used to fit linear models by criteria other than least squares and also for polynomial regression; submodule :mod:~naginterfaces.library.opt functions may be used to fit nonlinear models and linearly constrained linear models.

--------
naginterfaces.library.examples.correg :
This subpackage contains examples for the correg module.
See also the :ref:library_correg_ex subsection.

Functionality Index
-------------------

**Correlation-like coefficients**

all variables

casewise treatment of missing values: :meth:coeffs_zero_miss_case

no missing values: :meth:coeffs_zero

pairwise treatment of missing values: :meth:coeffs_zero_miss_pair

subset of variables

casewise treatment of missing values: :meth:coeffs_zero_subset_miss_case

no missing values: :meth:coeffs_zero_subset

pairwise treatment of missing values: :meth:coeffs_zero_subset_miss_pair

**Generalized linear models**

binomial errors: :meth:glm_binomial

computes estimable function: :meth:glm_estfunc

gamma errors: :meth:glm_gamma

Normal errors: :meth:glm_normal

Poisson errors: :meth:glm_poisson

prediction: :meth:glm_predict

transform model parameters: :meth:glm_constrain

**Hierarchical mixed effects regression**

initiation: :meth:mixeff_hier_init

using maximum likelihood: :meth:mixeff_hier_ml

using restricted maximum likelihood: :meth:mixeff_hier_reml

**Least angle regression (includes LASSO)**

Additional parameter calculation: :meth:lars_param

Model fitting

Cross-product matrix: :meth:lars_xtx

Raw data: :meth:lars

**Linear mixed effects regression**

fitting (via REML or ML): :meth:lmm_fit

initiation: :meth:lmm_init

initiation, combine: :meth:lmm_init_combine

via maximum likelihood (ML): :meth:mixeff_ml

via restricted maximum likelihood (REML): :meth:mixeff_reml

**Multiple linear regression**

from correlation coefficients: :meth:linregm_coeffs_const

from correlation-like coefficients: :meth:linregm_coeffs_noconst

**Multiple linear regression/General linear model**

add/delete observation from model: :meth:linregm_obs_edit

add independent variable to model: :meth:linregm_var_add

computes estimable function: :meth:linregm_estfunc

delete independent variable from model: :meth:linregm_var_del

general linear regression model: :meth:linregm_fit

regression for new dependent variable: :meth:linregm_fit_newvar

regression parameters from updated model: :meth:linregm_update

transform model parameters: :meth:linregm_constrain

**Nearest correlation matrix**

fixed elements: :meth:corrmat_fixed

fixed submatrix: :meth:corrmat_shrinking

:math:k-factor structure: :meth:corrmat_nearest_kfactor

method of Qi and Sun

element-wise weights: :meth:corrmat_h_weight

unweighted, unbounded: :meth:corrmat_nearest

weighted norm: :meth:corrmat_nearest_bounded

rank-constrained: :meth:corrmat_nearest_rank

shrinkage method: :meth:corrmat_target

**Non-parametric rank correlation (Kendall and/or Spearman)**

missing values

casewise treatment of missing values

overwriting input data: :meth:coeffs_kspearman_miss_case_overwrite

preserving input data: :meth:coeffs_kspearman_miss_case

pairwise treatment of missing values: :meth:coeffs_kspearman_miss_pair

no missing values

overwriting input data: :meth:coeffs_kspearman_overwrite

preserving input data: :meth:coeffs_kspearman

**Partial least squares**

calculates predictions given an estimated PLS model: :meth:pls_pred

fits a PLS model for a given number of factors: :meth:pls_fit

orthogonal scores using SVD: :meth:pls_svd

orthogonal scores using Wold's method: :meth:pls_wold

**Product-moment correlation**

correlation coefficients, all variables

casewise treatment of missing values: :meth:coeffs_pearson_miss_case

no missing values: :meth:coeffs_pearson

pairwise treatment of missing values: :meth:coeffs_pearson_miss_pair

correlation coefficients, subset of variables

casewise treatment of missing values: :meth:coeffs_pearson_subset_miss_case

no missing values: :meth:coeffs_pearson_subset

pairwise treatment of missing values: :meth:coeffs_pearson_subset_miss_pair

correlation matrix

compute correlation and covariance matrices: :meth:corrmat

compute from sum of squares matrix: :meth:ssqmat_to_corrmat

compute partial correlation and covariance matrices: :meth:corrmat_partial

sum of squares matrix

combine: :meth:ssqmat_combine

compute: :meth:ssqmat

update: :meth:ssqmat_update

**Quantile regression**

linear

comprehensive: :meth:quantile_linreg

simple: :meth:quantile_linreg_easy

**Residuals**

Durbin--Watson test: :meth:linregm_stat_durbwat

standardized residuals and influence statistics: :meth:linregm_stat_resinf

**Ridge regression**

ridge parameter(s) supplied: :meth:ridge

ridge parameter optimized: :meth:ridge_opt

**Robust correlation**

Huber's method: :meth:robustm_corr_huber

user-supplied weight function only: :meth:robustm_corr_user

user-supplied weight function plus derivatives: :meth:robustm_corr_user_deriv

**Robust regression**

compute weights for use with :meth:robustm_user: :meth:robustm_wts

standard :math:M-estimates: :meth:robustm

user-supplied weight functions: :meth:robustm_user

variance-covariance matrix following :meth:robustm_user: :meth:robustm_user_varmat

**Selecting regression model**

all possible regressions: :meth:linregm_rssq

forward selection: :meth:linregm_fit_onestep

:math:R^2 and :math:C_p statistics: :meth:linregm_rssq_stat

**Service functions**

for multiple linear regression

reorder elements from vectors and matrices: :meth:linregm_service_reorder

select elements from vectors and matrices: :meth:linregm_service_select

general option getting function: :meth:optget

general option setting function: :meth:optset

**Simple linear regression**

no intercept: :meth:linregs_noconst

no intercept with missing values: :meth:linregs_noconst_miss

with intercept: :meth:linregs_const

with intercept and with missing values: :meth:linregs_const_miss

**Stepwise linear regression**

Clarke's sweep algorithm: :meth:linregm_fit_stepwise

For full information please refer to the NAG Library document

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02intro.html
"""

[docs]def corrmat_nearest(g, errtol=0.0, maxits=0, maxit=0):
r"""
corrmat_nearest computes the nearest correlation matrix, in the Frobenius norm, to a given square, input matrix.

.. _g02aa-py2-py-doc:

For full information please refer to the NAG Library document for g02aa

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02aaf.html

.. _g02aa-py2-py-parameters:

**Parameters**
**g** : float, array-like, shape :math:\left(n, n\right)
:math:G, the initial matrix.

**errtol** : float, optional
The termination tolerance for the Newton iteration. If :math:\mathrm{errtol}\leq 0.0, :math:n\times \sqrt{\text{machine precision}} is used.

**maxits** : int, optional
:math:\mathrm{maxits} specifies the maximum number of iterations used for the iterative scheme used to solve the linear algebraic equations at each Newton step.

If :math:\mathrm{maxits}\leq 0, :math:100 is used.

**maxit** : int, optional
Specifies the maximum number of Newton iterations.

If :math:\mathrm{maxit}\leq 0, :math:200 is used.

**Returns**
**x** : float, ndarray, shape :math:\left(n, n\right)
Contains the nearest correlation matrix.

**itera** : int
The number of Newton steps taken.

**feval** : int
The number of function evaluations of the dual problem.

**nrmgrd** : float
The norm of the gradient of the last Newton step.

.. _g02aa-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 0.

(errno :math:2)
Newton iteration fails to converge in :math:\langle\mathit{\boldsymbol{value}}\rangle iterations.

(errno :math:4)
An intermediate eigenproblem could not be solved. This should not occur. Please contact NAG <https://www.nag.com>__ with details of your call.

**Warns**
**NagAlgorithmicWarning**
(errno :math:3)
Machine precision is limiting convergence.

The array returned in :math:\mathrm{x} may still be of interest.

.. _g02aa-py2-py-notes:

**Notes**
A correlation matrix may be characterised as a real square matrix that is symmetric, has a unit diagonal and is positive semidefinite.

corrmat_nearest applies an inexact Newton method to a dual formulation of the problem, as described by Qi and Sun (2006).
It applies the improvements suggested by Borsdorf and Higham (2010).

.. _g02aa-py2-py-references:

**References**
Borsdorf, R and Higham, N J, 2010, A preconditioned (Newton) algorithm for the nearest correlation matrix, IMA Journal of Numerical Analysis (30(1)), 94--107

Qi, H and Sun, D, 2006, A quadratically convergent Newton method for computing the nearest correlation matrix, SIAM J. Matrix AnalAppl (29(2)), 360--385

--------
:meth:naginterfaces.library.examples.correg.corrmat_nearest_ex.main
"""
raise NotImplementedError

[docs]def corrmat_nearest_bounded(g, opt, alpha=None, w=None, errtol=0.0, maxits=0, maxit=0):
r"""
corrmat_nearest_bounded computes the nearest correlation matrix, in the Frobenius norm or weighted Frobenius norm, and optionally with bounds on the eigenvalues, to a given square, input matrix.

.. _g02ab-py2-py-doc:

For full information please refer to the NAG Library document for g02ab

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02abf.html

.. _g02ab-py2-py-parameters:

**Parameters**
**g** : float, array-like, shape :math:\left(n, n\right)
:math:G, the initial matrix.

**opt** : str, length 1
Indicates the problem to be solved.

:math:\mathrm{opt} = \texttt{'A'}

The lower bound problem is solved.

:math:\mathrm{opt} = \texttt{'W'}

The weighted norm problem is solved.

:math:\mathrm{opt} = \texttt{'B'}

Both problems are solved.

**alpha** : None or float, optional
The value of :math:\alpha.

If :math:\mathrm{opt} = \texttt{'W'}, :math:\mathrm{alpha} need not be set.

**w** : None or float, array-like, shape :math:\left(:\right), optional
Note: the required length for this argument is determined as follows: if :math:\mathrm{opt}\neq\texttt{'A'}: :math:n; otherwise: :math:0.

The square roots of the diagonal elements of :math:W, that is the diagonal of :math:W^{\frac{1}{2}}.

If :math:\mathrm{opt} = \texttt{'A'}, :math:\mathrm{w} is not referenced and may be **None**.

**errtol** : float, optional
The termination tolerance for the Newton iteration. If :math:\mathrm{errtol}\leq 0.0, :math:n\times \sqrt{\text{machine precision}} is used.

**maxits** : int, optional
Specifies the maximum number of iterations to be used by the iterative scheme to solve the linear algebraic equations at each Newton step.

If :math:\mathrm{maxits}\leq 0, :math:2\times n is used.

**maxit** : int, optional
Specifies the maximum number of Newton iterations.

If :math:\mathrm{maxit}\leq 0, :math:200 is used.

**Returns**
**x** : float, ndarray, shape :math:\left(n, n\right)
Contains the nearest correlation matrix.

**itera** : int
The number of Newton steps taken.

**feval** : int
The number of function evaluations of the dual problem.

**nrmgrd** : float
The norm of the gradient of the last Newton step.

.. _g02ab-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 0.

(errno :math:1)
On entry, the value of :math:\mathrm{opt} is invalid.

Constraint: :math:\mathrm{opt} = \texttt{'A'}, :math:\texttt{'W'} or :math:\texttt{'B'}.

(errno :math:1)
On entry, :math:\mathrm{alpha} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0.0 < \mathrm{alpha} < 1.0.

(errno :math:1)
On entry, all elements of :math:\mathrm{w} were not positive.

Constraint: :math:\mathrm{w}[\textit{i}-1] > 0.0, for all :math:i.

(errno :math:2)
Newton iteration fails to converge in :math:\langle\mathit{\boldsymbol{value}}\rangle iterations. Increase :math:\mathrm{maxit} or check the call to the function.

**Warns**
**NagAlgorithmicWarning**
(errno :math:3)
The machine precision is limiting convergence. In this instance the returned value of :math:\mathrm{x} may be useful.

(errno :math:4)
An intermediate eigenproblem could not be solved. This should not occur. Please contact NAG <https://www.nag.com>__ with details of your call.

.. _g02ab-py2-py-notes:

**Notes**
Finds the nearest correlation matrix :math:X by minimizing :math:\frac{1}{2}\left\lVert G-X\right\rVert^2 where :math:G is an approximate correlation matrix.

The norm can either be the Frobenius norm or the weighted Frobenius norm :math:\frac{1}{2}\left\lVert W^{\frac{1}{2}}\left(G-X\right)W^{\frac{1}{2}}\right\rVert_F^2.

You can optionally specify a lower bound on the eigenvalues, :math:\alpha, of the computed correlation matrix, forcing the matrix to be positive definite, :math:0 < \alpha < 1.

Note that if the weights vary by several orders of magnitude from one another the algorithm may fail to converge.

.. _g02ab-py2-py-references:

**References**
Borsdorf, R and Higham, N J, 2010, A preconditioned (Newton) algorithm for the nearest correlation matrix, IMA Journal of Numerical Analysis (30(1)), 94--107

Qi, H and Sun, D, 2006, A quadratically convergent Newton method for computing the nearest correlation matrix, SIAM J. Matrix AnalAppl (29(2)), 360--385
"""
raise NotImplementedError

[docs]def corrmat_nearest_kfactor(g, k, errtol=0.0, maxit=0):
r"""
corrmat_nearest_kfactor computes the factor loading matrix associated with the nearest correlation matrix with :math:k-factor structure, in the Frobenius norm, to a given square, input matrix.

.. _g02ae-py2-py-doc:

For full information please refer to the NAG Library document for g02ae

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02aef.html

.. _g02ae-py2-py-parameters:

**Parameters**
**g** : float, array-like, shape :math:\left(n, n\right)
:math:G, the initial matrix.

**k** : int
:math:k, the number of factors and columns of :math:X.

**errtol** : float, optional
The termination tolerance for the projected gradient norm. See references for further details. If :math:\mathrm{errtol}\leq 0.0, :math:0.01 is used. This is often a suitable default value.

**maxit** : int, optional
Specifies the maximum number of iterations in the spectral projected gradient method.

If :math:\mathrm{maxit}\leq 0, :math:40000 is used.

**Returns**
**x** : float, ndarray, shape :math:\left(n, \mathrm{k}\right)
Contains the matrix :math:X.

**itera** : int
The number of steps taken in the spectral projected gradient method.

**feval** : int
The number of evaluations of :math:\left\lVert G-XX^\mathrm{T}+\mathrm{diag}\left(XX^\mathrm{T}-I\right)\right\rVert_F.

**nrmpgd** : float
The norm of the projected gradient at the final iteration.

.. _g02ae-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 0.

(errno :math:1)
On entry, :math:\mathrm{k} = \langle\mathit{\boldsymbol{value}}\rangle and :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0 < \mathrm{k}\leq n.

(errno :math:2)
Spectral gradient method fails to converge in :math:\langle\mathit{\boldsymbol{value}}\rangle iterations.

.. _g02ae-py2-py-notes:

**Notes**
A correlation matrix :math:C with :math:k-factor structure may be characterised as a real square matrix that is symmetric, has a unit diagonal, is positive semidefinite and can be written as :math:C = XX^\mathrm{T}+\mathrm{diag}\left(I-XX^\mathrm{T}\right), where :math:I is the identity matrix and :math:X has :math:n rows and :math:k columns. :math:X is often referred to as the factor loading matrix.

corrmat_nearest_kfactor applies a spectral projected gradient method to the modified problem :math:\mathrm{min}\left(\left\lVert G-XX^\mathrm{T}+\mathrm{diag}\left(XX^\mathrm{T}-I\right)\right\rVert \right)_F such that :math:\left\lVert x_{\textit{i}}^\mathrm{T}\right\rVert_2\leq 1, for :math:\textit{i} = 1,2,\ldots,n, where :math:x_i is the :math:i\ th row of the factor loading matrix, :math:X, which gives us the solution.

.. _g02ae-py2-py-references:

**References**
Birgin, E G, Martínez, J M and Raydan, M, 2001, Algorithm 813: SPG--software for convex-constrained optimization, ACM Trans. Math. Software (27), 340--349

Borsdorf, R, Higham, N J and Raydan, M, 2010, Computing a nearest correlation matrix with factor structure, SIAM J. Matrix Anal. Appl. (31(5)), 2603--2622
"""
raise NotImplementedError

[docs]def corrmat_h_weight(g, alpha, h, errtol=0.0, maxit=0):
r"""
corrmat_h_weight computes the nearest correlation matrix, using element-wise weighting in the Frobenius norm and optionally with bounds on the eigenvalues, to a given square, input matrix.

.. _g02aj-py2-py-doc:

For full information please refer to the NAG Library document for g02aj

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02ajf.html

.. _g02aj-py2-py-parameters:

**Parameters**
**g** : float, array-like, shape :math:\left(n, n\right)
:math:G, the initial matrix.

**alpha** : float
The value of :math:\alpha.

If :math:\mathrm{alpha} < 0.0, :math:0.0 is used.

**h** : float, array-like, shape :math:\left(n, n\right)
The matrix of weights :math:H.

**errtol** : float, optional
The termination tolerance for the iteration. If :math:\mathrm{errtol}\leq 0.0, :math:n\times \sqrt{\text{machine precision}} is used. See Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02ajf.html#accuracy>__ for further details.

**maxit** : int, optional
Specifies the maximum number of iterations to be used.

If :math:\mathrm{maxit}\leq 0, :math:200 is used.

**Returns**
**x** : float, ndarray, shape :math:\left(n, n\right)
Contains the nearest correlation matrix.

**itera** : int
The number of iterations taken.

**norm** : float
The value of :math:\left\lVert H\circ \left(G-X\right)\right\rVert_F after the final iteration.

.. _g02aj-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 0.

(errno :math:5)
On entry, :math:\mathrm{alpha} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{alpha} < 1.0.

(errno :math:6)
On entry, one or more of the off-diagonal elements of :math:H were negative.

(errno :math:7)
Function failed to converge in :math:\langle\mathit{\boldsymbol{value}}\rangle iterations.

Increase :math:\mathrm{maxit} or check the call to the function.

**Warns**
**NagAlgorithmicWarning**
(errno :math:8)
Failure to solve intermediate eigenproblem. This should not occur. Please contact NAG <https://www.nag.com>__ with details of your call.

.. _g02aj-py2-py-notes:

**Notes**
corrmat_h_weight finds the nearest correlation matrix, :math:X, to an approximate correlation matrix, :math:G, using element-wise weighting, this minimizes :math:\left\lVert H\circ \left(G-X\right)\right\rVert_F, where :math:C = A\circ B denotes the matrix :math:C with elements :math:C_{{ij}} = A_{{ij}}\times B_{{ij}}.

You can optionally specify a lower bound on the eigenvalues, :math:\alpha, of the computed correlation matrix, forcing the matrix to be strictly positive definite, if :math:0 < \alpha < 1.

Zero elements in :math:H should be used when you wish to put no emphasis on the corresponding element of :math:G.
The algorithm scales :math:H so that the maximum element is :math:1.
It is this scaled matrix that is used in computing the norm above and for the stopping criteria described in Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02ajf.html#accuracy>__.

Note that if the elements in :math:H vary by several orders of magnitude from one another the algorithm may fail to converge.

.. _g02aj-py2-py-references:

**References**
Borsdorf, R and Higham, N J, 2010, A preconditioned (Newton) algorithm for the nearest correlation matrix, IMA Journal of Numerical Analysis (30(1)), 94--107

Jiang, K, Sun, D and Toh, K-C, 2012, An inexact accelerated proximal gradient method for large scale linearly constrained convex SDP, SIAM J. Optim. (22(3)), 1042--1064

Qi, H and Sun, D, 2006, A quadratically convergent Newton method for computing the nearest correlation matrix, SIAM J. Matrix AnalAppl (29(2)), 360--385
"""
raise NotImplementedError

[docs]def corrmat_nearest_rank(g, rank, errtol=0.0, ranktol=0.0, maxits=0, maxit=0):
r"""
corrmat_nearest_rank computes the nearest correlation matrix of maximum prescribed rank, in the Frobenius norm, to a given square, input matrix.

.. _g02ak-py2-py-doc:

For full information please refer to the NAG Library document for g02ak

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02akf.html

.. _g02ak-py2-py-parameters:

**Parameters**
**g** : float, array-like, shape :math:\left(n, n\right)
:math:\tilde{G}, the initial matrix.

**rank** : int
:math:r, the upper bound for the rank of :math:X.

**errtol** : float, optional
The termination tolerance for the convergence measure of the objective function value.

If :math:\mathrm{errtol}\leq 0.0, then a value of :math:1.0e-5 is used.

See Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02akf.html#algdetails>__ for further details.

**ranktol** : float, optional
The feasibility tolerance for the rank constraint.

If :math:\mathrm{ranktol}\leq 0.0, :math:\sqrt{\text{machine precision}} is used.

See Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02akf.html#algdetails>__ for further details.

**maxits** : int, optional
Specifies the maximum number of iterations used for the majorization approach to solve penalized problems at each level of penalty parameter.

If :math:\mathrm{maxits}\leq 0, then a value of :math:100 is used.

**maxit** : int, optional
Specifies the maximum number of iterations for the penalty method, i.e., the maximum level of penalty parameter.

If :math:\mathrm{maxit}\leq 0, then a value of :math:200 is used.

**Returns**
**x** : float, ndarray, shape :math:\left(n, n\right)
:math:X, the nearest correlation matrix of rank :math:r.

**f** : float
The difference between :math:X and :math:G given by :math:\frac{1}{2}\left\lVert X-G\right\rVert_F^2.

**rankerr** : float
The rank error of :math:X, defined as :math:\sum_{{i = r+1}}^n\left(\lambda_i\right), given that :math:\lambda_i denote eigenvalues of :math:X sorted in non-increasing order.

**nsub** : int
The total number of majorized problems that have been solved, i.e., the total number of calls for :meth:corrmat_nearest.

.. _g02ak-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 0.

(errno :math:4)
On entry, :math:\mathrm{rank} = \langle\mathit{\boldsymbol{value}}\rangle and :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0 < \mathrm{rank}\leq n.

(errno :math:5)
Majorized penalty approach fails to converge in :math:\mathrm{maxit} level of penalty iterations.

**Warns**
**NagAlgorithmicWarning**
(errno :math:6)
Convergence is limited by machine precision. The objective function value or rank is decreasing very slowly.

The array returned in :math:\mathrm{x} may still be of interest.

.. _g02ak-py2-py-notes:

**Notes**
corrmat_nearest_rank finds the nearest correlation matrix :math:X of maximum prescribed rank :math:r to an approximate correlation matrix :math:G in the Frobenius norm.

The solver is based on the Majorized Penalty Approach (MPA) proposed by Gao and Sun (2010).
One of the key elements in this type of method is that the subproblems are similar to the nearest correlation matrix problem without rank constraint, and can be solved efficiently by :meth:corrmat_nearest.
The total number of subproblems solved is controlled by the arguments :math:\mathrm{maxit} and :math:\mathrm{maxits}.
The algorithm behaviour and solver accuracy can be modified by these and other input arguments.
The default values for these arguments are chosen to work well in the general case but it is recommended that you tune them to your particular problem.
For a detailed description of the algorithm see Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02akf.html#algdetails>__.

.. _g02ak-py2-py-references:

**References**
Bai, S, Qi, H--D and Xiu, N, 2015, Constrained best Euclidean distance embedding on a sphere: A matrix optimization approach, SIAM J. Optim. (25(1)), 439--467

Gao, Y and Sun, D, 2010, A majorized penalty approach for calibrating rank constrained correlation matrix problems, Technical report, Department of Mathematics, National University of Singapore

Qi, H--D and Yuan, X, 2014, Computing the nearest Euclidean distance matrix with low embedding dimensions, Mathematical Programming (147(1--2)), 351--389
"""
raise NotImplementedError

[docs]def corrmat_shrinking(g, k, errtol=0.0, eigtol=0.0):
r"""
corrmat_shrinking computes a correlation matrix, subject to preserving a leading principal submatrix and applying the smallest relative perturbation to the remainder of the approximate input matrix.

.. _g02an-py2-py-doc:

For full information please refer to the NAG Library document for g02an

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02anf.html

.. _g02an-py2-py-parameters:

**Parameters**
**g** : float, array-like, shape :math:\left(n, n\right)
:math:G, the initial matrix.

**k** : int
:math:k, the order of the leading principal submatrix :math:A.

**errtol** : float, optional
The termination tolerance for the iteration.

If :math:\mathrm{errtol}\leq 0.0, :math:\sqrt{\text{machine precision}} is used.

See Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02anf.html#accuracy>__ for further details.

**eigtol** : float, optional
The tolerance used in determining the definiteness of :math:A.

If :math:\lambda_{\mathrm{min}}\left(A\right) > n\times λ_{\mathrm{max}}\left(A\right)\times \mathrm{eigtol}, where :math:\lambda_{\mathrm{min}}\left(A\right) and :math:\lambda_{\mathrm{max}}\left(A\right) denote the minimum and maximum eigenvalues of :math:A respectively, :math:A is positive definite.

If :math:\mathrm{eigtol}\leq 0, machine precision is used.

**Returns**
**x** : float, ndarray, shape :math:\left(n, n\right)
Contains the matrix :math:X.

**alpha** : float
:math:\alpha.

**itera** : int
The number of iterations taken.

**eigmin** : float
The smallest eigenvalue of the leading principal submatrix :math:A.

**norm** : float
The value of :math:\left\lVert G-X\right\rVert_F after the final iteration.

.. _g02an-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 0.

(errno :math:3)
On entry, :math:\mathrm{k} = \langle\mathit{\boldsymbol{value}}\rangle and :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq {\mathrm{k} > 0}.

(errno :math:5)
The :math:k\times k principal leading submatrix of the initial matrix :math:G is not positive definite.

(errno :math:6)
Failure to solve intermediate eigenproblem. This should not occur. Please contact NAG <https://www.nag.com>__.

.. _g02an-py2-py-notes:

**Notes**
corrmat_shrinking finds a correlation matrix, :math:X, starting from an approximate correlation matrix, :math:G, with positive definite leading principal submatrix of order :math:k.
The returned correlation matrix, :math:X, has the following structure:

.. math::
X = \alpha \begin{pmatrix}A&0\\0&I\end{pmatrix}+\left(1-α\right)G

where :math:A is the :math:k\times k leading principal submatrix of the input matrix :math:G and positive definite, and :math:\alpha \in \left[0, 1\right].

corrmat_shrinking utilizes a shrinking method to find the minimum value of :math:\alpha such that :math:X is positive definite with unit diagonal.

.. _g02an-py2-py-references:

**References**
Higham, N J, Strabić, N and Šego, V, 2014, Restoring definiteness via shrinking, with an application to correlation matrices with a fixed block, MIMS EPrint 2014.54, Manchester Institute for Mathematical Sciences, The University of Manchester, UK
"""
raise NotImplementedError

[docs]def corrmat_target(g, theta, h, errtol=0.0, eigtol=0.0):
r"""
corrmat_target computes a correlation matrix, by using a positive definite **target** matrix derived from weighting the approximate input matrix, with an optional bound on the minimum eigenvalue.

.. _g02ap-py2-py-doc:

For full information please refer to the NAG Library document for g02ap

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02apf.html

.. _g02ap-py2-py-parameters:

**Parameters**
**g** : float, array-like, shape :math:\left(n, n\right)
:math:G, the initial matrix.

**theta** : float
The value of :math:\theta. If :math:\mathrm{theta} < 0.0, :math:0.0 is used.

**h** : float, array-like, shape :math:\left(n, n\right)
The matrix of weights :math:H.

**errtol** : float, optional
The termination tolerance for the iteration.

If :math:\mathrm{errtol}\leq 0.0, :math:\sqrt{\text{machine precision}} is used.

See Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02apf.html#accuracy>__ for further details.

**eigtol** : float, optional
The tolerance used in determining the definiteness of the target matrix :math:T = H\circ G.

If :math:\lambda_{\mathrm{min}}\left(T\right) > n\times λ_{\mathrm{max}}\left(T\right)\times \mathrm{eigtol}, where :math:\lambda_{\mathrm{min}}\left(T\right) and :math:\lambda_{\mathrm{max}}\left(T\right) denote the minimum and maximum eigenvalues of :math:T respectively, :math:T is positive definite.

If :math:\mathrm{eigtol}\leq 0, machine precision is used.

**Returns**
**h** : float, ndarray, shape :math:\left(n, n\right)
A symmetric matrix :math:\frac{1}{2}\left(H+H^\mathrm{T}\right) with its diagonal elements set to :math:1.0.

**x** : float, ndarray, shape :math:\left(n, n\right)
Contains the matrix :math:X.

**alpha** : float
The constant :math:\alpha used in the formation of :math:X.

**itera** : int
The number of iterations taken.

**eigmin** : float
The smallest eigenvalue of the target matrix :math:T.

**norm** : float
The value of :math:\left\lVert G-X\right\rVert_F after the final iteration.

.. _g02ap-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 0.

(errno :math:3)
On entry, :math:\mathrm{theta} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{theta} < 1.0.

(errno :math:6)
The target matrix is not positive definite.

(errno :math:7)
Failure to solve intermediate eigenproblem. This should not occur. Please contact NAG <https://www.nag.com>__.

.. _g02ap-py2-py-notes:

**Notes**
Starting from an approximate correlation matrix, :math:G, corrmat_target finds a correlation matrix, :math:X, which has the form

.. math::
X = \alpha T+\left(1-\alpha \right)G\text{,}

where :math:\alpha \in \left[0, 1\right] and :math:T = H\circ G is a target matrix. :math:C = A\circ B denotes the matrix :math:C with elements :math:C_{{ij}} = A_{{ij}}\times B_{{ij}}. :math:H is a matrix of weights that defines the target matrix.
The target matrix must be positive definite and thus have off-diagonal elements less than :math:1 in magnitude.
A value of :math:1 in :math:H essentially fixes an element in :math:G so it is unchanged in :math:X.

corrmat_target utilizes a shrinking method to find the minimum value of :math:\alpha such that :math:X is positive definite with unit diagonal and with a smallest eigenvalue of at least :math:\theta \in \left[0, 1\right) times the smallest eigenvalue of the target matrix.

.. _g02ap-py2-py-references:

**References**
Higham, N J, Strabić, N and Šego, V, 2014, Restoring definiteness via shrinking, with an application to correlation matrices with a fixed block, MIMS EPrint 2014.54, Manchester Institute for Mathematical Sciences, The University of Manchester, UK
"""
raise NotImplementedError

[docs]def corrmat_fixed(g, alpha, h, m, errtol=0.0, maxit=0):
r"""
corrmat_fixed computes the nearest correlation matrix, in the Frobenius norm, while fixing elements and optionally with bounds on the eigenvalues, to a given square input matrix.

.. _g02as-py2-py-doc:

For full information please refer to the NAG Library document for g02as

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02asf.html

.. _g02as-py2-py-parameters:

**Parameters**
**g** : float, array-like, shape :math:\left(n, n\right)
:math:\tilde{G}, the initial matrix.

**alpha** : float
The value of :math:\alpha.

If :math:\mathrm{alpha} < 0.0, a value of :math:0.0 is used.

**h** : int, array-like, shape :math:\left(n, n\right)
The symmetric matrix :math:H. If an element of :math:H is :math:1 then the corresponding element in :math:G is fixed in the output :math:X. Only the strictly lower triangular part of :math:H need be set.

**m** : int
The number of previous iterates to use in the Anderson acceleration. If :math:\mathrm{m} = 0, Anderson acceleration is not used. See Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02asf.html#accuracy>__ for further details.

If :math:\mathrm{m} < 0, a value of :math:4 is used.

**errtol** : float, optional
The termination tolerance for the iteration.

If :math:\mathrm{errtol}\leq 0.0, :math:\sqrt{\text{machine precision}} is used.

See Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02asf.html#accuracy>__ for further details.

**maxit** : int, optional
Specifies the maximum number of iterations.

If :math:\mathrm{maxit}\leq 0, a value of :math:200 is used.

**Returns**
**x** : float, ndarray, shape :math:\left(n, n\right)
Contains the matrix :math:X.

**its** : int
The number of iterations taken.

**fnorm** : float
The value of :math:\left\lVert G-X\right\rVert_F after the final iteration.

.. _g02as-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 0.

(errno :math:4)
On entry, :math:\mathrm{alpha} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{alpha} < 1.0.

(errno :math:5)
On entry, :math:\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle and :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{m} \leq {n\times n}.

(errno :math:7)
Function failed to converge in :math:\langle\mathit{\boldsymbol{value}}\rangle iterations.

A solution may not exist, however, try increasing :math:\mathrm{maxit}.

(errno :math:8)
Failure during Anderson acceleration.

Consider setting :math:\mathrm{m} = 0 and recomputing.

(errno :math:9)
The fixed element :math:G_{{ij}}, lies outside the interval :math:\left[-1, 1\right], for :math:i = \langle\mathit{\boldsymbol{value}}\rangle and :math:j = \langle\mathit{\boldsymbol{value}}\rangle.

.. _g02as-py2-py-notes:

**Notes**
corrmat_fixed finds the nearest correlation matrix, :math:X, to a matrix, :math:G, in the Frobenius norm.
It uses an alternating projections algorithm with Anderson acceleration.
Elements in the input matrix can be fixed by supplying the value :math:1 in the corresponding element of the matrix :math:H.
However, note that the algorithm may fail to converge if the fixed elements do not form part of a valid correlation matrix.
You can optionally specify a lower bound, :math:\alpha, on the eigenvalues of the computed correlation matrix, forcing the matrix to be positive definite with :math:0\leq \alpha < 1.

.. _g02as-py2-py-references:

**References**
Anderson, D G, 1965, Iterative Procedures for Nonlinear Integral Equations, J. Assoc. Comput. Mach. (12), 547--560

Higham, N J and Strabić, N, 2016, Anderson acceleration of the alternating projections method for computing the nearest correlation matrix, Numer. Algor. (72), 1021--1042
"""
raise NotImplementedError

[docs]def coeffs_pearson(x):
r"""
coeffs_pearson computes means and standard deviations of variables, sums of squares and cross-products of deviations from means, and Pearson product-moment correlation coefficients for a set of data.

.. _g02ba-py2-py-doc:

For full information please refer to the NAG Library document for g02ba

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02baf.html

.. _g02ba-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**Returns**
**xbar** : float, ndarray, shape :math:\left(m\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**std** : float, ndarray, shape :math:\left(m\right)
The standard deviation, :math:s_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**ssp** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{ssp}[\textit{j}-1,\textit{k}-1] is the cross-product of deviations :math:S_{{\textit{j}\textit{k}}}, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**r** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{r}[\textit{j}-1,\textit{k}-1] is the product-moment correlation coefficient :math:R_{{\textit{j}\textit{k}}} between the :math:\textit{j}\ th and :math:\textit{k}\ th variables, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

.. _g02ba-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

.. _g02ba-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consist of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{1}{n}\sum_{{i = 1}}^nx_{{ij}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{1}{{n-1}}\sum_{{i = 1}}^n\left(x_{{ij}}-\bar{x}_j\right)^2}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Sums of squares and cross-products of deviations from means:

.. math::
S_{{jk}} = \sum_{{i = 1}}^n\left(x_{{ij}}-\bar{x}_j\right)\left(x_{{ik}}-\bar{x}_k\right)\text{, }\quad j,k = 1,2,\ldots,m\text{.}

(#) Pearson product-moment correlation coefficients:

.. math::
R_{{jk}} = \frac{S_{{jk}}}{{\sqrt{S_{{jj}}S_{{kk}}}}}\text{, }\quad j,k = 1,2,\ldots,m\text{.}

If :math:S_{{jj}} or :math:S_{{kk}} is zero, :math:R_{{jk}} is set to zero.
"""
raise NotImplementedError

[docs]def coeffs_pearson_miss_case(x, miss, xmiss):
r"""
coeffs_pearson_miss_case computes means and standard deviations of variables, sums of squares and cross-products of deviations from means, and Pearson product-moment correlation coefficients for a set of data omitting completely any cases with a missing observation for any variable.

.. _g02bb-py2-py-doc:

For full information please refer to the NAG Library document for g02bb

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bbf.html

.. _g02bb-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bbf.html#accuracy>__).

**Returns**
**xbar** : float, ndarray, shape :math:\left(m\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**std** : float, ndarray, shape :math:\left(m\right)
The standard deviation, :math:s_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**ssp** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{ssp}[\textit{j}-1,\textit{k}-1] is the cross-product of deviations :math:S_{{\textit{j}\textit{k}}}, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**r** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{r}[\textit{j}-1,\textit{k}-1] is the product-moment correlation coefficient :math:R_{{\textit{j}\textit{k}}} between the :math:\textit{j}\ th and :math:\textit{k}\ th variables, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**ncases** : int
The number of cases actually used in the calculations (when cases involving missing values have been eliminated).

.. _g02bb-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

(errno :math:4)
After observations with missing values were omitted, no cases remained.

(errno :math:5)
After observations with missing values were omitted, only one case remained.

.. _g02bb-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consist of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),{j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.
In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:xm_j.
Missing values need not be specified for all variables.

Let :math:w_i = 0 if observation :math:i contains a missing value for any of those variables for which missing values have been declared, i.e., if :math:x_{{ij}} = xm_j for any :math:j for which an :math:xm_j has been assigned (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bbf.html#accuracy>__); and :math:w_i = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{{\sum_{{i = 1}}^nw_ix_{{ij}}}}{{\sum_{{i = 1}}^nw_i}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{{\sum_{{i = 1}}^nw_i\left(x_{{ij}}-\bar{x}_j\right)^2}}{{\sum_{{i = 1}}^nw_i-1}}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Sums of squares and cross-products of deviations from means:

.. math::
S_{{jk}} = \sum_{{i = 1}}^nw_i\left(x_{{ij}}-\bar{x}_j\right)\left(x_{{ik}}-\bar{x}_k\right)\text{, }\quad j,k = 1,2,\ldots,m\text{.}

(#) Pearson product-moment correlation coefficients:

.. math::
R_{{jk}} = \frac{S_{{jk}}}{{\sqrt{S_{{jj}}S_{{kk}}}}}\text{, }\quad j,k = 1,2,\ldots,m\text{.}

If :math:S_{{jj}} or :math:S_{{kk}} is zero, :math:R_{{jk}} is set to zero.
"""
raise NotImplementedError

[docs]def coeffs_pearson_miss_pair(n, x, miss, xmiss):
r"""
coeffs_pearson_miss_pair computes means and standard deviations of variables, sums of squares and cross-products of deviations from means, and Pearson product-moment correlation coefficients for a set of data omitting cases with missing values from only those calculations involving the variables for which the values are missing.

.. _g02bc-py2-py-doc:

For full information please refer to the NAG Library document for g02bc

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bcf.html

.. _g02bc-py2-py-parameters:

**Parameters**
**n** : int
:math:n, the number of observations or cases.

**x** : float, array-like, shape :math:\left(\mathrm{n}, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bcf.html#accuracy>__).

**Returns**
**xbar** : float, ndarray, shape :math:\left(m\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**std** : float, ndarray, shape :math:\left(m\right)
The standard deviation, :math:s_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**ssp** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{ssp}[\textit{j}-1,\textit{k}-1] is the cross-product of deviations :math:S_{{\textit{j}\textit{k}}}, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**r** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{r}[\textit{j}-1,\textit{k}-1] is the product-moment correlation coefficient :math:R_{{\textit{j}\textit{k}}} between the :math:\textit{j}\ th and :math:\textit{k}\ th variables, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**ncases** : int
The minimum number of cases used in the calculation of any of the sums of squares and cross-products and correlation coefficients (when cases involving missing values have been eliminated).

**cnt** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{cnt}[\textit{j}-1,\textit{k}-1] is the number of cases, :math:c_{{\textit{j}\textit{k}}}, actually used in the calculation of :math:S_{{\textit{j}\textit{k}}}, and :math:R_{{\textit{j}\textit{k}}}, the sum of cross-products and correlation coefficient for the :math:\textit{j}\ th and :math:\textit{k}\ th variables, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

.. _g02bc-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{n} \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

**Warns**
**NagAlgorithmicWarning**
(errno :math:4)
After observations with missing values were omitted, fewer than two cases remained for at least one pair of variables. (The pairs of variables involved can be determined by examination of the contents of the array :math:\mathrm{cnt}). All means, standard deviations, sums of squares and cross-products, and correlation coefficients based on two or more cases are returned by the function even if :math:\mathrm{errno} = 4.

.. _g02bc-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consist of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.
In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.

Let :math:w_{{\textit{i}\textit{j}}} = 0 if the :math:\textit{i}\ th observation for the :math:\textit{j}\ th variable is a missing value, i.e., if a missing value, :math:\textit{xm}_{\textit{j}}, has been declared for the :math:\textit{j}\ th variable, and :math:x_{{\textit{i}\textit{j}}} = \textit{xm}_{\textit{j}} (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bcf.html#accuracy>__); and :math:w_{{\textit{i}\textit{j}}} = 1 otherwise, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{{\sum_{{i = 1}}^nw_{{ij}}x_{{ij}}}}{{\sum_{{i = 1}}^nw_{{ij}}}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{{\sum_{{i = 1}}^nw_{{ij}}\left(x_{{ij}}-\bar{x}_j\right)^2}}{{\left(\sum_{{i = 1}}^nw_{{ij}}\right)-1}}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Sums of squares and cross-products of deviations from means:

.. math::
S_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}\left(x_{{ij}}-\bar{x}_{{j\left(k\right)}}\right)\left(x_{{ik}}-\bar{x}_{{k\left(j\right)}}\right)\text{, }\quad j,k = 1,2,\ldots,m\text{,}

where

.. math::
\bar{x}_{{j\left(k\right)}} = \frac{{\sum_{{i = 1}}^nw_{{ij}}w_{{ik}}x_{{ij}}}}{{\sum_{{i = 1}}^nw_{{ij}}w_{{ik}}}}\quad \text{ and }\quad \bar{x}_{{k\left(j\right)}} = \frac{{\sum_{{i = 1}}^nw_{{ik}}w_{{ij}}x_{{ik}}}}{{\sum_{{i = 1}}^nw_{{ik}}w_{{ij}}}}\text{,}

(i.e., the means used in the calculation of the sums of squares and cross-products of deviations are based on the same set of observations as are the cross-products.)

(#) Pearson product-moment correlation coefficients:

.. math::
R_{{jk}} = \frac{S_{{jk}}}{{\sqrt{S_{{jj\left(k\right)}}S_{{kk\left(j\right)}}}}}\text{, }\quad j,k, = 1,2,\ldots,m\text{,}

where :math:S_{{jj\left(k\right)}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}{\left(x_{{ij}}-\bar{x}_{{j\left(k\right)}}\right)}^2 and :math:S_{{kk\left(j\right)}} = \sum_{{i = 1}}^nw_{{ik}}w_{{ij}}{\left(x_{{ik}}-\bar{x}_{{k\left(j\right)}}\right)}^2 and :math:\bar{x}_{{j\left(k\right)}} and :math:\bar{x}_{{k\left(j\right)}} are as defined in (c) above

(i.e., the sums of squares of deviations used in the denominator are based on the same set of observations as are used in the calculation of the numerator).

If :math:S_{{jj\left(k\right)}} or :math:S_{{kk\left(j\right)}} is zero, :math:R_{{jk}} is set to zero.

(#) The number of cases used in the calculation of each of the correlation coefficients:

.. math::
c_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}\text{, }\quad j,k = 1,2,\ldots,m\text{.}

(The diagonal terms, :math:c_{{\textit{j}\textit{j}}}, for :math:\textit{j} = 1,2,\ldots,m, also give the number of cases used in the calculation of the means, :math:\bar{x}_{\textit{j}}, and the standard deviations, :math:s_{\textit{j}}.)
"""
raise NotImplementedError

[docs]def coeffs_zero(x):
r"""
coeffs_zero computes means and standard deviations of variables, sums of squares and cross-products about zero, and correlation-like coefficients for a set of data.

.. _g02bd-py2-py-doc:

For full information please refer to the NAG Library document for g02bd

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bdf.html

.. _g02bd-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to the value of :math:x_{{\textit{i}\textit{j}}}, the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**Returns**
**xbar** : float, ndarray, shape :math:\left(m\right)
:math:\mathrm{xbar}[\textit{j}-1] contains the mean value, :math:\bar{x}_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**std** : float, ndarray, shape :math:\left(m\right)
The standard deviation, :math:s_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**sspz** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{sspz}[\textit{j}-1,\textit{k}-1] is the cross-product about zero, :math:\tilde{S}_{{\textit{j}\textit{k}}}, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**rz** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{rz}[\textit{j}-1,\textit{k}-1] is the correlation-like coefficient, :math:\tilde{R}_{{\textit{j}\textit{k}}}, between the :math:\textit{j}\ th and :math:\textit{k}\ th variables, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

.. _g02bd-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

.. _g02bd-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{1}{n}\sum_{{i = 1}}^nx_{{ij}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{1}{{n-1}}\sum_{{i = 1}}^n\left(x_{{ij}}-\bar{x}_j\right)^2}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Sums of squares and cross-products about zero:

.. math::
\tilde{S}_{{jk}} = \sum_{{i = 1}}^nx_{{ij}}x_{{ik}}\text{, }\quad j,k = 1,2,\ldots,m\text{.}

(#) Correlation-like coefficients:

.. math::
\tilde{R}_{{jk}} = \frac{\tilde{S}_{{jk}}}{{\sqrt{\tilde{S}_{{jj}}\tilde{S}_{{kk}}}}}\text{, }\quad j,k = 1,2,\ldots,m\text{.}

If :math:\tilde{S}_{{jj}} or :math:\tilde{S}_{{kk}} is zero, :math:\tilde{R}_{{jk}} is set to zero.
"""
raise NotImplementedError

[docs]def coeffs_zero_miss_case(x, miss, xmiss):
r"""
coeffs_zero_miss_case computes means and standard deviations of variables, sums of squares and cross-products about zero, and correlation-like coefficients for a set of data omitting completely any cases with a missing observation for any variable.

.. _g02be-py2-py-doc:

For full information please refer to the NAG Library document for g02be

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bef.html

.. _g02be-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bef.html#accuracy>__).

**Returns**
**xbar** : float, ndarray, shape :math:\left(m\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**std** : float, ndarray, shape :math:\left(m\right)
The standard deviation, :math:s_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**sspz** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{sspz}[\textit{j}-1,\textit{k}-1] is the cross-product about zero, :math:\tilde{S}_{{\textit{j}\textit{k}}}, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**rz** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{rz}[\textit{j}-1,\textit{k}-1] is the correlation-like coefficient, :math:\tilde{R}_{{\textit{j}\textit{k}}}, between the :math:\textit{j}\ th and :math:\textit{k}\ th variables, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**ncases** : int
The number of cases actually used in the calculations (when cases involving missing values have been eliminated).

.. _g02be-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

(errno :math:4)
After observations with missing values were omitted, no cases remained.

(errno :math:5)
After observations with missing values were omitted, only one case remained.

.. _g02be-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.
In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.

Let :math:w_i = 0 if observation :math:i contains a missing value for any of those variables for which missing values have been declared, i.e., if :math:x_{{ij}} = \textit{xm}_j for any :math:j for which an :math:\textit{xm}_j has been assigned (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bef.html#accuracy>__); and :math:w_i = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{{\sum_{{i = 1}}^nw_ix_{{ij}}}}{{\sum_{{i = 1}}^nw_i}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{{\sum_{{i = 1}}^nw_i\left(x_{{ij}}-\bar{x}_j\right)^2}}{{\sum_{{i = 1}}^nw_i-1}}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Sums of squares and cross-products about zero:

.. math::
\tilde{S}_{{jk}} = \sum_{{i = 1}}^nw_ix_{{ij}}x_{{ik}}\text{, }\quad j,k = 1,2,\ldots,m\text{.}

(#) Correlation-like coefficients:

.. math::
\tilde{R}_{{jk}} = \frac{\tilde{S}_{{jk}}}{{\sqrt{\tilde{S}_{{jj}}\tilde{S}_{{kk}}}}}\text{, }\quad j,k = 1,2,\ldots,m\text{.}

If :math:\tilde{S}_{{jj}} or :math:\tilde{S}_{{kk}} is zero, :math:\tilde{R}_{{jk}} is set to zero.
"""
raise NotImplementedError

[docs]def coeffs_zero_miss_pair(x, miss, xmiss):
r"""
coeffs_zero_miss_pair computes means and standard deviations of variables, sums of squares and cross-products about zero and correlation-like coefficients for a set of data omitting cases with missing values from only those calculations involving the variables for which the values are missing.

.. _g02bf-py2-py-doc:

For full information please refer to the NAG Library document for g02bf

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bff.html

.. _g02bf-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bff.html#accuracy>__).

**Returns**
**xbar** : float, ndarray, shape :math:\left(m\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**std** : float, ndarray, shape :math:\left(m\right)
The standard deviation, :math:s_{\textit{j}}, of the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m.

**sspz** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{sspz}[\textit{j}-1,\textit{k}-1] is the cross-product about zero, :math:\tilde{S}_{{\textit{j}\textit{k}}}, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**rz** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{rz}[\textit{j}-1,\textit{k}-1] is the correlation-like coefficient, :math:\tilde{R}_{{\textit{j}\textit{k}}}, between the :math:\textit{j}\ th and :math:\textit{k}\ th variables, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**ncases** : int
The minimum number of cases used in the calculation of any of the sums of squares and cross-products and correlation-like coefficients (when cases involving missing values have been eliminated).

**cnt** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{cnt}[\textit{j}-1,\textit{k}-1] is the number of cases, :math:c_{{\textit{j}\textit{k}}}, actually used in the calculation of :math:\tilde{S}_{{\textit{j}\textit{k}}}, and :math:\tilde{R}_{{\textit{j}\textit{k}}}, the sum of cross-products and correlation-like coefficient for the :math:\textit{j}\ th and :math:\textit{k}\ th variables, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

.. _g02bf-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

**Warns**
**NagAlgorithmicWarning**
(errno :math:4)
After observations with missing values were omitted, fewer than two cases remained for at least one pair of variables. (The pairs of variables involved can be determined by examination of the contents of the array :math:\mathrm{cnt}). All means, standard deviations, sums of squares and cross-products, and correlation-like coefficients based on two or more cases are returned by the function even if :math:\mathrm{errno} = 4.

.. _g02bf-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.
In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.

Let :math:w_{{\textit{i}\textit{j}}} = 0 if the :math:\textit{i}\ th observation for the :math:\textit{j}\ th variable is a missing value, i.e., if a missing value, :math:\textit{xm}_{\textit{j}}, has been declared for the :math:\textit{j}\ th variable, and :math:x_{{\textit{i}\textit{j}}} = \textit{xm}_{\textit{j}} (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bff.html#accuracy>__); and :math:w_{{\textit{i}\textit{j}}} = 1 otherwise, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{{\sum_{{i = 1}}^nw_{{ij}}x_{{ij}}}}{{\sum_{{i = 1}}^nw_{{ij}}}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{{\sum_{{i = 1}}^nw_{{ij}}\left(x_{{ij}}-\bar{x}_j\right)^2}}{{\sum_{{i = 1}}^nw_{{ij}}-1}}}\text{, }\quad j = 1,2,\ldots,m\text{.}

(#) Sums of squares and cross-products about zero:

.. math::
\tilde{S}_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}x_{{ij}}x_{{ik}}\text{, }\quad j,k = 1,2,\ldots,m\text{.}

(#) Correlation-like coefficients:

.. math::
\tilde{R}_{{jk}} = \frac{\tilde{S}_{{jk}}}{{\sqrt{\tilde{S}_{{jj\left(k\right)}}j\left(k\right)\tilde{S}_{{kk\left(j\right)}}}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

where :math:\tilde{S}_{{jj\left(k\right)}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}x_{{ij}}^2 and :math:\tilde{S}_{{kk\left(j\right)}} = \sum_{{i = 1}}^nw_{{ik}}w_{{ij}}x_{{ik}}^2

(i.e., the sums of squares about zero are based on the same set of observations as are used in the calculation of the numerator).

If :math:\tilde{S}_{{jj\left(k\right)}} or :math:\tilde{S}_{{kk\left(j\right)}} is zero, :math:\tilde{R}_{{jk}} is set to zero.

(#) The number of cases used in the calculation of each of the correlation-like coefficients:

.. math::
c_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}\text{, }\quad j,k = 1,2,\ldots,m\text{.}

(The diagonal terms, :math:c_{{\textit{j}\textit{j}}}, for :math:\textit{j} = 1,2,\ldots,m, also give the number of cases used in the calculation of the means :math:\bar{x}_{\textit{j}} and the standard deviations :math:s_{\textit{j}}.)
"""
raise NotImplementedError

[docs]def coeffs_pearson_subset(x, kvar):
r"""
coeffs_pearson_subset computes means and standard deviations, sums of squares and cross-products of deviations from means, and Pearson product-moment correlation coefficients for selected variables.

.. _g02bg-py2-py-doc:

For full information please refer to the NAG Library document for g02bg

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bgf.html

.. _g02bg-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**kvar** : int, array-like, shape :math:\left(\textit{nvars}\right)
:math:\mathrm{kvar}[\textit{j}] must be set to the column number in :math:\mathrm{x} of the :math:\textit{j}\ th variable for which information is required, for :math:\textit{j} = 0,\ldots,p-1.

**Returns**
**xbar** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**std** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The standard deviation, :math:s_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**ssp** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{ssp}[\textit{j}-1,\textit{k}-1] is the cross-product of deviations, :math:S_{{\textit{j}\textit{k}}}, for the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**r** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{r}[\textit{j}-1,\textit{k}-1] is the product-moment correlation coefficient, :math:R_{{\textit{j}\textit{k}}}, between the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

.. _g02bg-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\textit{nvars} = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nvars} \geq 2 and :math:\textit{nvars} \leq m.

(errno :math:4)
On entry, :math:\textit{i} = \langle\mathit{\boldsymbol{value}}\rangle, :math:\mathrm{kvar}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{kvar}[\textit{i}-1]\leq m.

.. _g02bg-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consist of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable, together with the subset of these variables, :math:v_1,v_2,\ldots,v_p, for which information is required.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{1}{n}\sum_{{i = 1}}^nx_{{ij}}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{1}{{n-1}}\sum_{{i = 1}}^n\left(x_{{ij}}-\bar{x}_j\right)^2}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Sums of squares and cross-products of deviations from zero:

.. math::
S_{{jk}} = \sum_{{i = 1}}^n\left(x_{{ij}}-\bar{x}_j\right)\left(x_{{ik}}-\bar{x}_k\right)\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{.}

(#) Pearson product-moment correlation coefficients:

.. math::
R_{{jk}} = \frac{S_{{jk}}}{{\sqrt{S_{{jj}}S_{{kk}}}}}\text{, }\quad j,k = v_1,v_2,\ldots v_p\text{.}

If :math:S_{{jj}} or :math:S_{{kk}} is zero, :math:R_{{jk}} is set to zero.
"""
raise NotImplementedError

[docs]def coeffs_pearson_subset_miss_case(x, miss, xmiss, mistyp, kvar):
r"""
coeffs_pearson_subset_miss_case computes means and standard deviations, sums of squares and cross-products of deviations from means, and Pearson product-moment correlation coefficients for selected variables omitting completely any cases with a missing observation for any variable (either over all variables in the dataset or over only those variables in the selected subset).

.. _g02bh-py2-py-doc:

For full information please refer to the NAG Library document for g02bh

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bhf.html

.. _g02bh-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bhf.html#accuracy>__).

**mistyp** : int
Indicates the manner in which missing observations are to be treated.

:math:\mathrm{mistyp} = 1

A case is excluded if it contains a missing value for any of the variables :math:1,2,\ldots,m.

:math:\mathrm{mistyp} = 0

A case is excluded if it contains a missing value for any of the :math:p\left(\leq m\right) variables specified in the array :math:\mathrm{kvar}.

**kvar** : int, array-like, shape :math:\left(\textit{nvars}\right)
:math:\mathrm{kvar}[\textit{j}] must be set to the column number in :math:\mathrm{x} of the :math:\textit{j}\ th variable for which information is required, for :math:\textit{j} = 0,\ldots,p-1.

**Returns**
**xbar** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The mean value, of :math:\bar{x}_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**std** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The standard deviation, :math:s_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**ssp** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{ssp}[\textit{j}-1,\textit{k}-1] is the cross-product of deviations, :math:S_{{\textit{j}\textit{k}}}, for the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**r** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{r}[\textit{j}-1,\textit{k}-1] is the product-moment correlation coefficient, :math:R_{{\textit{j}\textit{k}}}, between the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**ncases** : int
The number of cases actually used in the calculations (when cases involving missing values have been eliminated).

.. _g02bh-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\textit{nvars} = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nvars} \geq 2 and :math:\textit{nvars} \leq m.

(errno :math:4)
On entry, :math:\textit{i} = \langle\mathit{\boldsymbol{value}}\rangle, :math:\mathrm{kvar}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{kvar}[\textit{i}-1]\leq m.

(errno :math:5)
On entry, :math:\mathrm{mistyp} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mistyp} = 0 or :math:1.

(errno :math:6)
After observations with missing values were omitted, no cases remained.

(errno :math:7)
After observations with missing values were omitted, only one case remained.

.. _g02bh-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable, together with the subset of these variables, :math:v_1,v_2,\ldots,v_p, for which information is required.

In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.
The missing values can be utilized in two slightly different ways; you can indicate which scheme is required.

Firstly, let :math:w_i = 0 if observation :math:i contains a missing value for any of those variables in the set :math:1,2,\ldots,m for which missing values have been declared, i.e., if :math:x_{{ij}} = \textit{xm}_j for any :math:j (:math:j = 1,2,\ldots,m) for which an :math:\textit{xm}_j has been assigned (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bhf.html#accuracy>__); and :math:w_i = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

Secondly, let :math:w_i = 0 if observation :math:i contains a missing value for any of those variables in the selected subset :math:v_1,v_2,\ldots,v_p for which missing values have been declared, i.e., if :math:x_{{ij}} = \textit{xm}_j for any :math:j (:math:j = v_1,v_2,\ldots,v_p) for which an :math:\textit{xm}_j has been assigned (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bhf.html#accuracy>__); and :math:w_i = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{{\sum_{{i = 1}}^nw_ix_{{ij}}}}{{\sum_{{i = 1}}^nw_i}}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{{\sum_{{i = 1}}^nw_i\left(x_{{ij}}-\bar{x}_j\right)^2}}{{\sum_{{i = 1}}^nw_i-1}}}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Sums of squares and cross-products of deviations from means:

.. math::
S_{{jk}} = \sum_{{i = 1}}^nw_i\left(x_{{ij}}-\bar{x}_j\right)\left(x_{{ik}}-\bar{x}_k\right)\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{.}

(#) Pearson product-moment correlation coefficients:

.. math::
R_{{jk}} = \frac{S_{{jk}}}{{\sqrt{S_{{jj}}S_{{kk}}}}}\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{.}

If :math:S_{{jj}} or :math:S_{{kk}} is zero, :math:R_{{jk}} is set to zero.
"""
raise NotImplementedError

[docs]def coeffs_pearson_subset_miss_pair(x, miss, xmiss, kvar):
r"""
coeffs_pearson_subset_miss_pair computes means and standard deviations, sums of squares and cross-products of deviations from means, and Pearson product-moment correlation coefficients for selected variables omitting cases with missing values from only those calculations involving the variables for which the values are missing.

.. _g02bj-py2-py-doc:

For full information please refer to the NAG Library document for g02bj

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bjf.html

.. _g02bj-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bjf.html#accuracy>__).

**kvar** : int, array-like, shape :math:\left(\textit{nvars}\right)
:math:\mathrm{kvar}[\textit{j}] must be set to the column number in :math:\mathrm{x} of the :math:\textit{j}\ th variable for which information is required, for :math:\textit{j} = 0,\ldots,p-1.

**Returns**
**xbar** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**std** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The standard deviation, :math:s_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**ssp** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{ssp}[\textit{j}-1,\textit{k}-1] is the cross-product of deviations, :math:S_{{\textit{j}\textit{k}}}, for the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**r** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{r}[\textit{j}-1,\textit{k}-1] is the product-moment correlation coefficient, :math:R_{{\textit{j}\textit{k}}}, between the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**ncases** : int
The minimum number of cases used in the calculation of any of the sums of squares and cross-products and correlation coefficients (when cases involving missing values have been eliminated).

**cnt** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{cnt}[\textit{j}-1,\textit{k}-1] is the number of cases, :math:c_{{\textit{j}\textit{k}}}, actually used in the calculation of :math:S_{{\textit{j}\textit{k}}}, and :math:R_{{\textit{j}\textit{k}}}, the sum of cross-products and correlation coefficient for the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

.. _g02bj-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\textit{nvars} = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nvars} \geq 2 and :math:\textit{nvars} \leq m.

(errno :math:4)
On entry, :math:\textit{i} = \langle\mathit{\boldsymbol{value}}\rangle, :math:\mathrm{kvar}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{kvar}[\textit{i}-1]\leq m.

**Warns**
**NagAlgorithmicWarning**
(errno :math:5)
After observations with missing values were omitted, fewer than two cases remained.

.. _g02bj-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable, together with the subset of these variables, :math:v_1,v_2,\ldots,v_p, for which information is required.

In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.

Let :math:w_{{\textit{i}\textit{j}}} = 0 if the :math:\textit{i}\ th observation for the :math:\textit{j}\ th variable is a missing value, i.e., if a missing value, :math:\textit{xm}_{\textit{j}}, has been declared for the :math:\textit{j}\ th variable, and :math:x_{{\textit{i}\textit{j}}} = \textit{xm}_{\textit{j}} (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bjf.html#accuracy>__); and :math:w_{{\textit{i}\textit{j}}} = 1 otherwise, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{{\sum_{{i = 1}}^nw_{{ij}}x_{{ij}}}}{{\sum_{{i = 1}}^nw_{{ij}}}}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{{\sum_{{i = 1}}^nw_{{ij}}\left(x_{{ij}}-\bar{x}_j\right)^2}}{{\sum_{{i = 1}}^nw_{{ij}}-1}}}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Sums of squares and cross-products of deviations from means:

.. math::
S_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}\left(x_{{ij}}-\bar{x}_{{j\left(k\right)}}\right)\left(x_{{ik}}-\bar{x}_{{k\left(j\right)}}\right)\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{,}

where

.. math::
\bar{x}_{{j\left(k\right)}} = \frac{{\sum_{{i = 1}}^nw_{{ij}}w_{{ik}}x_{{ij}}}}{{\sum_{{i = 1}}^nw_{{ij}}w_{{ik}}}}\quad \text{ and }\quad \bar{x}_{{k\left(j\right)}} = \frac{{\sum_{{i = 1}}^nw_{{ik}}w_{{ij}}x_{{ik}}}}{{\sum_{{i = 1}}^nw_{{ik}}w_{{ij}}}}\text{,}

(i.e., the means used in the calculation of the sum of squares and cross-products of deviations are based on the same set of observations as are the cross-products).

(#) Pearson product-moment correlation coefficients:

.. math::
R_{{jk}} = \frac{S_{{jk}}}{{\sqrt{S_{{jj\left(k\right)}}S_{{kk\left(j\right)}}}}}\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{,}

where

.. math::
S_{{jj\left(k\right)}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}{\left(x_{{ij}}-\bar{x}_{{j\left(k\right)}}\right)}^2\quad \text{ and }\quad S_{{kk\left(j\right)}} = \sum_{{i = 1}}^nw_{{ik}}w_{{ij}}{\left(x_{{ik}}-\bar{x}_{{k\left(j\right)}}\right)}^2\text{,}

(i.e., the sums of squares of deviations used in the denominator are based on the same set of observations as are used in the calculation of the numerator).

If :math:S_{{jj\left(k\right)}} or :math:S_{{kk\left(j\right)}} is zero, :math:R_{{jk}} is set to zero.

(#) The number of cases used in the calculation of each of the correlation coefficients:

.. math::
c_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{.}

(The diagonal terms, :math:c_{{jj}}, for :math:j = v_1,v_2,\ldots,v_p, also give the number of cases used in the calculation of the means, :math:\bar{x}_j, and the standard deviations, :math:s_j.)
"""
raise NotImplementedError

[docs]def coeffs_zero_subset(x, kvar):
r"""
coeffs_zero_subset computes means and standard deviations, sums of squares and cross-products about zero, and correlation-like coefficients for selected variables.

.. _g02bk-py2-py-doc:

For full information please refer to the NAG Library document for g02bk

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bkf.html

.. _g02bk-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**kvar** : int, array-like, shape :math:\left(\textit{nvars}\right)
:math:\mathrm{kvar}[\textit{j}] must be set to the column number in :math:\mathrm{x} of the :math:\textit{j}\ th variable for which information is required, for :math:\textit{j} = 0,\ldots,p-1.

**Returns**
**xbar** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**std** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The standard deviation, :math:s_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**sspz** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{sspz}[\textit{j}-1,\textit{k}-1] is the cross-product about zero, :math:\tilde{S}_{{\textit{j}\textit{k}}}, for the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**rz** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{rz}[\textit{j}-1,\textit{k}-1] is the correlation-like coefficient, :math:\tilde{R}_{{\textit{j}\textit{k}}}, between the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

.. _g02bk-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\textit{nvars} = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nvars} \geq 2 and :math:\textit{nvars} \leq m.

(errno :math:4)
On entry, :math:\textit{i} = \langle\mathit{\boldsymbol{value}}\rangle, :math:\mathrm{kvar}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{kvar}[\textit{i}-1]\leq m.

.. _g02bk-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable, together with the subset of these variables, :math:v_1,v_2,\ldots,v_p, for which information is required.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{{\sum_{{i = 1}}^nx_{{ij}}}}{n}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{1}{{n-1}}\sum_{{i = 1}}^n\left(x_{{ij}}-\bar{x}_j\right)^2}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Sums of squares and cross-products about zero:

.. math::
\tilde{S}_{{jk}} = \sum_{{i = 1}}^nx_{{ij}}x_{{ik}}\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{.}

(#) Correlation-like coefficients:

.. math::
\tilde{R}_{{jk}} = \frac{\tilde{S}_{{jk}}}{{\sqrt{\tilde{S}_{{jj}}\tilde{S}_{{kk}}}}}\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{.}

If :math:\tilde{S}_{{jj}} or :math:\tilde{S}_{{kk}} is zero, :math:\tilde{R}_{{jk}} is set to zero.
"""
raise NotImplementedError

[docs]def coeffs_zero_subset_miss_case(x, miss, xmiss, mistyp, kvar):
r"""
coeffs_zero_subset_miss_case computes means and standard deviations, sums of squares and cross-products about zero, and correlation-like coefficients for selected variables omitting completely any cases with a missing observation for any variable (either over all variables in the dataset or over only those variables in the selected subset).

.. _g02bl-py2-py-doc:

For full information please refer to the NAG Library document for g02bl

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02blf.html

.. _g02bl-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02blf.html#accuracy>__).

**mistyp** : int
Indicates the manner in which missing observations are to be treated.

:math:\mathrm{mistyp} = 1

A case is excluded if it contains a missing value for any of the variables :math:1,2,\ldots,m.

:math:\mathrm{mistyp} = 0

A case is excluded if it contains a missing value for any of the :math:p\left(\leq m\right) variables specified in the array :math:\mathrm{kvar}.

**kvar** : int, array-like, shape :math:\left(\textit{nvars}\right)
:math:\mathrm{kvar}[\textit{j}] must be set to the column number in :math:\mathrm{x} of the :math:\textit{j}\ th variable for which information is required, for :math:\textit{j} = 0,\ldots,p-1.

**Returns**
**xbar** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**std** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The standard deviation, :math:s_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**sspz** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{sspz}[\textit{j}-1,\textit{k}-1] is the cross-product about zero, :math:\tilde{S}_{{\textit{j}\textit{k}}}, for the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**rz** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{rz}[\textit{j}-1,\textit{k}-1] is the correlation-like coefficient, :math:\tilde{R}_{{\textit{j}\textit{k}}}, between the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**ncases** : int
The number of cases actually used in the calculations (when cases involving missing values have been eliminated).

.. _g02bl-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\textit{nvars} = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nvars} \geq 2 and :math:\textit{nvars} \leq m.

(errno :math:4)
On entry, :math:\textit{i} = \langle\mathit{\boldsymbol{value}}\rangle, :math:\mathrm{kvar}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{kvar}[\textit{i}-1]\leq m.

(errno :math:5)
On entry, :math:\mathrm{mistyp} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mistyp} = 0 or :math:1.

(errno :math:6)
After observations with missing values were omitted, no cases remained.

(errno :math:7)
After observations with missing values were omitted, only one case remained.

.. _g02bl-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consist of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right)\text{ and }j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable, together with the subset of these variables, :math:v_1,v_2,\ldots,v_p, for which information is required.

In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.

The missing values can be utilized in two slightly different ways, you can indicate which scheme is required.

Firstly, let :math:w_i = 0 if observation :math:i contains a missing value for any of those variables in the set :math:1,2,\ldots,m for which missing values have been declared, i.e., if :math:x_{{ij}} = \textit{xm}_j for any :math:j (:math:j = 1,2,\ldots,m) for which an :math:\textit{xm}_j has been assigned (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02blf.html#accuracy>__); and :math:w_i = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

Secondly, let :math:w_i = 0 if observation :math:i contains a missing value for any of those variables in the selected subset :math:v_1,v_2,\ldots,v_p for which missing values have been declared, i.e., if :math:x_{{ij}} = \textit{xm}_j for any :math:j\left({j = v_1}, v_2, \ldots, v_p\right) for which an :math:\textit{xm}_j has been assigned (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02blf.html#accuracy>__); and :math:w_i = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{{\sum_{{i = 1}}^nw_ix_{{ij}}}}{{\sum_{{i = 1}}^nw_i}}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Standard deviations:

.. math::
S_j = \sqrt{\frac{{\sum_{{i = 1}}^nw_i\left(x_{{ij}}-\bar{x}_j\right)^2}}{{\sum_{{i = 1}}^nw_i-1}}}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Sums of squares and cross-products about zero:

.. math::
\tilde{S}_{{jk}} = \sum_{{i = 1}}^nw_ix_{{ij}}x_{{ik\text{, }\quad j,k}} = v_1,v_2,\ldots,v_p\text{.}

(#) Correlation-like coefficients:

.. math::
\tilde{R}_{{jk}} = \frac{\tilde{S}_{{jk}}}{{\sqrt{\tilde{S}_{{jj}}\tilde{S}_{{kk}}}}}\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{.}

If :math:\tilde{S}_{{jj}} or :math:\tilde{S}_{{kk}} is zero, :math:\tilde{R}_{{jk}} is set to zero.
"""
raise NotImplementedError

[docs]def coeffs_zero_subset_miss_pair(x, miss, xmiss, kvar):
r"""
coeffs_zero_subset_miss_pair computes means and standard deviations, sums of squares and cross-products about zero, and correlation-like coefficients for selected variables omitting cases with missing values from only those calculations involving the variables for which the values are missing.

.. _g02bm-py2-py-doc:

For full information please refer to the NAG Library document for g02bm

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bmf.html

.. _g02bm-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bmf.html#accuracy>__).

**kvar** : int, array-like, shape :math:\left(\textit{nvars}\right)
:math:\mathrm{kvar}[\textit{j}] must be set to the column number in :math:\mathrm{x} of the :math:\textit{j}\ th variable for which information is required, for :math:\textit{j} = 0,\ldots,p-1.

**Returns**
**xbar** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The mean value, :math:\bar{x}_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**std** : float, ndarray, shape :math:\left(\textit{nvars}\right)
The standard deviation, :math:s_{\textit{j}}, of the variable specified in :math:\mathrm{kvar}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,p.

**sspz** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{sspz}[\textit{j}-1,\textit{k}-1] is the cross-product about zero, :math:\tilde{S}_{{\textit{j}\textit{k}}}, for the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**rz** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{rz}[\textit{j}-1,\textit{k}-1] is the correlation-like coefficient, :math:\tilde{R}_{{\textit{j}\textit{k}}}, between the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

**ncases** : int
The minimum number of cases used in the calculation of any of the sums of squares and cross-products and correlation-like coefficients (when cases involving missing values have been eliminated).

**cnt** : float, ndarray, shape :math:\left(\textit{nvars}, \textit{nvars}\right)
:math:\mathrm{cnt}[\textit{j}-1,\textit{k}-1] is the number of cases, :math:c_{{\textit{j}\textit{k}}}, actually used in the calculation of the sum of cross-product and correlation-like coefficient for the variables specified in :math:\mathrm{kvar}[\textit{j}-1] and :math:\mathrm{kvar}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,p, for :math:\textit{j} = 1,2,\ldots,p.

.. _g02bm-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\textit{nvars} = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nvars} \geq 2 and :math:\textit{nvars} \leq m.

(errno :math:4)
On entry, :math:\textit{i} = \langle\mathit{\boldsymbol{value}}\rangle, :math:\mathrm{kvar}[\textit{i}-1] = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{kvar}[\textit{i}-1]\leq m.

**Warns**
**NagAlgorithmicWarning**
(errno :math:5)
After observations with missing values were omitted, fewer than two cases remained for at least one pair of variables. (The pairs of variables involved can be determined by examination of the contents of the array :math:\mathrm{cnt}). All means, standard deviations, sums of squares and cross-products, and correlation-like coefficients based on two or more cases are returned by the function even if :math:\mathrm{errno} = 5.

.. _g02bm-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable, together with the subset of these variables, :math:v_1,v_2,\ldots,v_p, for which information is required.

In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.

Let :math:w_{{\textit{i}\textit{j}}} = 0, if the :math:\textit{i}\ th observation for the :math:\textit{j}\ th variable is a missing value, i.e., if a missing value, :math:\textit{xm}_{\textit{j}}, has been declared for the :math:\textit{j}\ th variable, and :math:x_{{\textit{i}\textit{j}}} = \textit{xm}_{\textit{j}} (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bmf.html#accuracy>__); and :math:w_{{\textit{i}\textit{j}}} = 1 otherwise, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x}_j = \frac{{\sum_{{i = 1}}^nw_{{ij}}x_{{ij}}}}{{\sum_{{i = 1}}^nw_{{ij}}}}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Standard deviations:

.. math::
s_j = \sqrt{\frac{{\sum_{{i = 1}}^nw_{{ij}}\left(x_{{ij}}-\bar{x}_j\right)^2}}{{\sum_{{i = 1}}^nw_{{ij}}-1}}}\text{, }\quad j = v_1,v_2,\ldots,v_p\text{.}

(#) Sums of squares and cross-products about zero:

.. math::
\tilde{S}_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}x_{{ij}}x_{{ik}}\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{.}

(#) Correlation-like coefficients:

.. math::
\tilde{R}_{{jk}} = \frac{\tilde{S}_{{jk}}}{{\sqrt{\tilde{S}_{{jj\left(k\right)}}\tilde{S}_{{kk\left(j\right)}}}}}\text{, }\quad j,k = v_{{1,}}v_2,\ldots,v_p\text{,}

where :math:\tilde{S}_{{jj\left(k\right)}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}x_{{ij}}^2 and :math:\tilde{S}_{{kk\left(j\right)}} = \sum_{{i = 1}}^nw_{{ik}}w_{{ij}}x_{{ik}}^2

(i.e., the sums of squares about zero are based on the same set of observations as are used in the calculation of the numerator).

If :math:\tilde{S}_{{jj\left(k\right)}} or :math:\tilde{S}_{{kk\left(j\right)}} is zero, :math:\tilde{R}_{{jk}} is set to zero.

(#) The number of cases used in the calculation of each of the correlation-like coefficients:

.. math::
c_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}\text{, }\quad j,k = v_1,v_2,\ldots,v_p\text{.}

(The diagonal terms, :math:c_{{\textit{j}\textit{j}}}, for :math:\textit{j} = 1,2,\ldots,n, also give the number of cases used in the calculation of the means :math:\bar{x}_{\textit{j}} and the standard deviations :math:s_{\textit{j}}.)
"""
raise NotImplementedError

[docs]def coeffs_kspearman_overwrite(x, itype):
r"""
coeffs_kspearman_overwrite computes Kendall and/or Spearman nonparametric rank correlation coefficients for a set of data; the data array is overwritten with the ranks of the observations.

.. _g02bn-py2-py-doc:

For full information please refer to the NAG Library document for g02bn

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bnf.html

.. _g02bn-py2-py-parameters:

**Parameters**
**x** : float, ndarray, shape :math:\left(n, m\right), modified in place
On entry: :math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

On exit: :math:\mathrm{x}[\textit{i}-1,\textit{j}-1] contains the rank :math:y_{{\textit{i}\textit{j}}} of the observation :math:x_{{\textit{i}\textit{j}}}, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**itype** : int
The type of correlation coefficients which are to be calculated.

:math:\mathrm{itype} = -1

Only Kendall's tau coefficients are calculated.

:math:\mathrm{itype} = 0

Both Kendall's tau and Spearman's coefficients are calculated.

:math:\mathrm{itype} = 1

Only Spearman's coefficients are calculated.

**Returns**
**rr** : float, ndarray, shape :math:\left(m, m\right)
The requested correlation coefficients.

If only Kendall's tau coefficients are requested (:math:\mathrm{itype} = -1), :math:\mathrm{rr}[j-1,k-1] contains Kendall's tau for the :math:j\ th and :math:k\ th variables.

If only Spearman's coefficients are requested (:math:\mathrm{itype} = 1), :math:\mathrm{rr}[j-1,k-1] contains Spearman's rank correlation coefficient for the :math:j\ th and :math:k\ th variables.

If both Kendall's tau and Spearman's coefficients are requested (:math:\mathrm{itype} = 0), the upper triangle of :math:\mathrm{rr} contains the Spearman coefficients and the lower triangle the Kendall coefficients.

That is, for the :math:\textit{j}\ th and :math:\textit{k}\ th variables, where :math:\textit{j} is less than :math:\textit{k}, :math:\mathrm{rr}[\textit{j}-1,\textit{k}-1] contains the Spearman rank correlation coefficient, and :math:\mathrm{rr}[\textit{k}-1,\textit{j}-1] contains Kendall's tau, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

(Diagonal terms, :math:\mathrm{rr}[j-1,j-1], are unity for all three values of :math:\mathrm{itype}.)

.. _g02bn-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

(errno :math:4)
On entry, :math:\mathrm{itype} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{itype} = -1 or :math:1.

.. _g02bn-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation of the :math:j\ th variable.

The quantities calculated are:

(a) Ranks

For a given variable, :math:j say, each of the :math:n observations, :math:x_{{1j}},x_{{2j}},\ldots,x_{{nj}}, has associated with it an additional number, the 'rank' of the observation, which indicates the magnitude of that observation relative to the magnitudes of the other :math:n-1 observations on that same variable.

The smallest observation for variable :math:j is assigned the rank :math:1, the second smallest observation for variable :math:j the rank :math:2, the third smallest the rank :math:3, and so on until the largest observation for variable :math:j is given the rank :math:n.

If a number of cases all have the same value for the given variable, :math:j, then they are each given an 'average' rank, e.g., if in attempting to assign the rank :math:h+1, :math:k observations were found to have the same value, then instead of giving them the ranks

.. math::
h+1,h+2,\ldots,h+k\text{,}

all :math:k observations would be assigned the rank

.. math::
\frac{{2h+k+1}}{2}

and the next value in ascending order would be assigned the rank

.. math::
h+k+1\text{.}

The process is repeated for each of the :math:m variables.

Let :math:y_{{ij}} be the rank assigned to the observation :math:x_{{ij}} when the :math:j\ th variable is being ranked.
The actual observations :math:x_{{ij}} are replaced by the ranks :math:y_{{ij}}.

(#) Nonparametric rank correlation coefficients

(i) Kendall's tau:

.. math::
R_{{jk}} = \frac{{\sum_{{h = 1}}^n\sum_{{i = 1}}^n\mathrm{sign}\left(y_{{hj}}-y_{{ij}}\right)\mathrm{sign}\left(y_{{hk}}-y_{{ik}}\right)}}{{\sqrt{\left[n\left(n-1\right)-T_j\right]\left[n\left(n-1\right)-T_k\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+----------------------------------------------------------+
|where|:math:\mathrm{sign}\left(u\right) = 1 if :math:u > 0, |
+-----+----------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = 0 if :math:u = 0, |
+-----+----------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = -1 if :math:u < 0,|
+-----+----------------------------------------------------------+

and :math:T_j = \sum t_j\left(t_j-1\right), where :math:t_j is the number of ties of a particular value of variable :math:j, and the summation is over all tied values of variable :math:j.

(#) Spearman's:

.. math::
R_{{jk}}^* = \frac{{n\left(n^2-1\right)-6\sum_{{i = 1}}^n\left(y_{{ij}}-y_{{ik}}\right)^2-\frac{1}{2}\left(T_j^*+T_k^*\right)}}{{\sqrt{\left[n\left(n^2-1\right)-T_j^*\right]\left[n\left(n^2-1\right)-T_k^*\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

where :math:T_j^* = \sum t_j\left(t_j^2-1\right), :math:t_j being the number of ties of a particular value of variable :math:j, and the summation being over all tied values of variable :math:j.

.. _g02bn-py2-py-references:

**References**
Siegel, S, 1956, Non-parametric Statistics for the Behavioral Sciences, McGraw--Hill
"""
raise NotImplementedError

[docs]def coeffs_kspearman_miss_case_overwrite(x, miss, xmiss, itype):
r"""
coeffs_kspearman_miss_case_overwrite computes Kendall and/or Spearman nonparametric rank correlation coefficients for a set of data omitting completely any cases with a missing observation for any variable; the data array is overwritten with the ranks of the observations.

.. _g02bp-py2-py-doc:

For full information please refer to the NAG Library document for g02bp

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bpf.html

.. _g02bp-py2-py-parameters:

**Parameters**
**x** : float, ndarray, shape :math:\left(n, m\right), modified in place
On entry: :math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

On exit: :math:\mathrm{x}[\textit{i}-1,\textit{j}-1] contains the rank :math:y_{{\textit{i}\textit{j}}} of the observation :math:x_{{\textit{i}\textit{j}}}, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n. (For those observations containing missing values, and, therefore, excluded from the calculation, :math:y_{{\textit{i}\textit{j}}} = 0, for :math:\textit{j} = 1,2,\ldots,m.)

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bpf.html#accuracy>__).

**itype** : int
The type of correlation coefficients which are to be calculated.

:math:\mathrm{itype} = -1

Only Kendall's tau coefficients are calculated.

:math:\mathrm{itype} = 0

Both Kendall's tau and Spearman's coefficients are calculated.

:math:\mathrm{itype} = 1

Only Spearman's coefficients are calculated.

**Returns**
**rr** : float, ndarray, shape :math:\left(m, m\right)
The requested correlation coefficients.

If only Kendall's tau coefficients are requested (:math:\mathrm{itype} = -1), :math:\mathrm{rr}[j-1,k-1] contains Kendall's tau for the :math:j\ th and :math:k\ th variables.

If only Spearman's coefficients are requested (:math:\mathrm{itype} = 1), :math:\mathrm{rr}[j-1,k-1] contains Spearman's rank correlation coefficient for the :math:j\ th and :math:k\ th variables.

If both Kendall's tau and Spearman's coefficients are requested (:math:\mathrm{itype} = 0), the upper triangle of :math:\mathrm{rr} contains the Spearman coefficients and the lower triangle the Kendall coefficients.

That is, for the :math:\textit{j}\ th and :math:\textit{k}\ th variables, where :math:\textit{j} is less than :math:\textit{k}, :math:\mathrm{rr}[\textit{j}-1,\textit{k}-1] contains the Spearman rank correlation coefficient, and :math:\mathrm{rr}[\textit{k}-1,\textit{j}-1] contains Kendall's tau, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

(Diagonal terms, :math:\mathrm{rr}[j-1,j-1], are unity for all three values of :math:\mathrm{itype}.)

**ncases** : int
The number of cases, :math:n_{\mathrm{c}}, actually used in the calculations (when cases involving missing values have been eliminated).

**incase** : int, ndarray, shape :math:\left(n\right)
:math:\mathrm{incase}[\textit{i}-1] holds the value :math:1 if the :math:\textit{i}\ th case was included in the calculations, and the value :math:0 if the :math:\textit{i}\ th case contained a missing value for at least one variable. That is, :math:\mathrm{incase}[\textit{i}-1] = w_{\textit{i}} (see :ref:Notes <g02bp-py2-py-notes>), for :math:\textit{i} = 1,2,\ldots,n.

.. _g02bp-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

(errno :math:4)
On entry, :math:\mathrm{itype} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{itype} = -1 or :math:1.

(errno :math:5)
After observations with missing values were omitted, fewer than two cases remained.

.. _g02bp-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.
In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.

Let :math:w_i = 0 if observation :math:i contains a missing value for any of those variables for which missing values have been declared; i.e., if :math:x_{{ij}} = \textit{xm}_j for any :math:j for which an :math:\textit{xm}_j has been assigned (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bpf.html#accuracy>__); and :math:w_i = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Ranks

For a given variable, :math:j say, each of the observations :math:x_{{ij}} for which :math:w_i = 1, for :math:\textit{i} = 1,2,\ldots,n, has associated with it an additional number, the 'rank' of the observation, which indicates the magnitude of that observation relative to the magnitudes of the other observations on that same variable for which :math:w_i = 1.

The smallest of these valid observations for variable :math:j is assigned the rank :math:1, the second smallest observation for variable :math:j the rank :math:2, the third smallest the rank :math:3, and so on until the largest such observation is given the rank :math:n_c, where :math:n_c = \sum_{{i = 1}}^nw_i.

If a number of cases all have the same value for the given variable, :math:j, then they are each given an 'average' rank, e.g., if in attempting to assign the rank :math:h+1, :math:k observations for which :math:w_i = 1 were found to have the same value, then instead of giving them the ranks

.. math::
h+1,h+2,\ldots,h+k\text{,}

all :math:k observations would be assigned the rank

.. math::
\frac{{2h+k+1}}{2}

and the next value in ascending order would be assigned the rank

.. math::
h+k+1\text{.}

The process is repeated for each of the :math:m variables.

Let :math:y_{{ij}} be the rank assigned to the observation :math:x_{{ij}} when the :math:j\ th variable is being ranked.
For those observations, :math:i, for which :math:w_i = 0, :math:y_{{ij}} = 0, for :math:j = 1,2,\ldots,m.

The actual observations :math:x_{{ij}} are replaced by the ranks :math:y_{{ij}}, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

(#) Nonparametric rank correlation coefficients

(i) Kendall's tau:

.. math::
R_{{jk}} = \frac{{\sum_{{h = 1}}^n\sum_{{i = 1}}^nw_hw_i\mathrm{sign}\left(y_{{hj}}-y_{{ij}}\right)\mathrm{sign}\left(y_{{hk}}-y_{{ik}}\right)}}{{\sqrt{\left[n_c\left(n_c-1\right)-T_j\right]\left[n_c\left(n_c-1\right)-T_k\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+---------------------------------------------------------+
|where|:math:n_c = \sum_{{i = 1}}^nw_i                        |
+-----+---------------------------------------------------------+
|and  |:math:\mathrm{sign}\left(u\right) = 1 if :math:u > 0 |
+-----+---------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = 0 if :math:u = 0 |
+-----+---------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = -1 if :math:u < 0|
+-----+---------------------------------------------------------+

and :math:T_j = \sum t_j\left(t_j-1\right) where :math:t_j is the number of ties of a particular value of variable :math:j, and the summation is over all tied values of variable :math:j.

(#) Spearman's:

.. math::
R_{{jk}}^* = \frac{{n_c\left(n_c^2-1\right)-6\sum_{{i = 1}}^nw_i\left(y_{{ij}}-y_{{ik}}\right)^2-\frac{1}{2}\left(T_j^*+T_k^*\right)}}{{\sqrt{\left[n_c\left(n_c^2-1\right)-T_j^*\right]\left[n_c\left(n_c^2-1\right)-T_k^*\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

where :math:n_c = \sum_{{i = 1}}^nw_i

and :math:T_j^* = \sum t_j\left(t_j^2-1\right) where :math:t_j is the number of ties of a particular value of variable :math:j, and the summation is over all tied values of variable :math:j.

.. _g02bp-py2-py-references:

**References**
Siegel, S, 1956, Non-parametric Statistics for the Behavioral Sciences, McGraw--Hill
"""
raise NotImplementedError

[docs]def coeffs_kspearman(x, itype):
r"""
coeffs_kspearman computes Kendall and/or Spearman nonparametric rank correlation coefficients for a set of data; the data array is preserved, and the ranks of the observations are not available on exit from the function.

.. _g02bq-py2-py-doc:

For full information please refer to the NAG Library document for g02bq

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bqf.html

.. _g02bq-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to data value :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**itype** : int
The type of correlation coefficients which are to be calculated.

:math:\mathrm{itype} = -1

Only Kendall's tau coefficients are calculated.

:math:\mathrm{itype} = 0

Both Kendall's tau and Spearman's coefficients are calculated.

:math:\mathrm{itype} = 1

Only Spearman's coefficients are calculated.

**Returns**
**rr** : float, ndarray, shape :math:\left(m, m\right)
The requested correlation coefficients.

If only Kendall's tau coefficients are requested (:math:\mathrm{itype} = -1), :math:\mathrm{rr}[j-1,k-1] contains Kendall's tau for the :math:j\ th and :math:k\ th variables.

If only Spearman's coefficients are requested (:math:\mathrm{itype} = 1), :math:\mathrm{rr}[j-1,k-1] contains Spearman's rank correlation coefficient for the :math:j\ th and :math:k\ th variables.

If both Kendall's tau and Spearman's coefficients are requested (:math:\mathrm{itype} = 0), the upper triangle of :math:\mathrm{rr} contains the Spearman coefficients and the lower triangle the Kendall coefficients.

That is, for the :math:\textit{j}\ th and :math:\textit{k}\ th variables, where :math:\textit{j} is less than :math:\textit{k}, :math:\mathrm{rr}[\textit{j}-1,\textit{k}-1] contains the Spearman rank correlation coefficient, and :math:\mathrm{rr}[\textit{k}-1,\textit{j}-1] contains Kendall's tau, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

(Diagonal terms, :math:\mathrm{rr}[j-1,j-1], are unity for all three values of :math:\mathrm{itype}.)

.. _g02bq-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

(errno :math:4)
On entry, :math:\mathrm{itype} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{itype} = -1 or :math:1.

.. _g02bq-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right),j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.

The observations are first ranked, as follows.

For a given variable, :math:j say, each of the :math:n observations, :math:x_{{1j}},x_{{2j}},\ldots,x_{{nj}}, has associated with it an additional number, the 'rank' of the observation, which indicates the magnitude of that observation relative to the magnitude of the other :math:n-1 observations on that same variable.

The smallest observation for variable :math:j is assigned the rank :math:1, the second smallest observation for variable :math:j the rank :math:2, the third smallest the rank :math:3, and so on until the largest observation for variable :math:j is given the rank :math:n.

If a number of cases all have the same value for the given variable, :math:j, then they are each given an 'average' rank -- e.g., if in attempting to assign the rank :math:h+1, :math:k observations were found to have the same value, then instead of giving them the ranks

.. math::
h+1,h+2,\ldots,h+k\text{,}

all :math:k observations would be assigned the rank

.. math::
\frac{{2h+k+1}}{2}

and the next value in ascending order would be assigned the rank

.. math::
h+k+1\text{.}

The process is repeated for each of the :math:m variables.

Let :math:y_{{ij}} be the rank assigned to the observation :math:x_{{ij}} when the :math:j\ th variable is being ranked.

The quantities calculated are:

(a) Kendall's tau rank correlation coefficients:

.. math::
R_{{jk}} = \frac{{\sum_{{h = 1}}^n\sum_{{i = 1}}^n\mathrm{sign}\left(y_{{hj}}-y_{{ij}}\right)\mathrm{sign}\left(y_{{hk}}-y_{{ik}}\right)}}{{\sqrt{\left[n\left(n-1\right)-T_j\right]\left[n\left(n-1\right)-T_k\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+---------------------------------------------------------+
|where|:math:\mathrm{sign}\left(u\right) = 1 if :math:u > 0 |
+-----+---------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = 0 if :math:u = 0 |
+-----+---------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = -1 if :math:u < 0|
+-----+---------------------------------------------------------+

and :math:T_j = \sum t_j\left(t_j-1\right), :math:t_j being the number of ties of a particular value of variable :math:j, and the summation being over all tied values of variable :math:j.

(#) Spearman's rank correlation coefficients:

.. math::
R_{{jk}}^* = \frac{{n\left(n^2-1\right)-6\sum_{{i = 1}}^n\left(y_{{ij}}-y_{{ik}}\right)^2-\frac{1}{2}\left(T_j^*+T_k^*\right)}}{{\sqrt{\left[n\left(n^2-1\right)-T_j^*\right]\left[n\left(n^2-1\right)-T_k^*\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

where :math:T_j^* = \sum t_j\left(t_j^2-1\right) where :math:t_j is the number of ties of a particular value of variable :math:j, and the summation is over all tied values of variable :math:j.

.. _g02bq-py2-py-references:

**References**
Siegel, S, 1956, Non-parametric Statistics for the Behavioral Sciences, McGraw--Hill
"""
raise NotImplementedError

[docs]def coeffs_kspearman_miss_case(x, miss, xmiss, itype):
r"""
coeffs_kspearman_miss_case computes Kendall and/or Spearman nonparametric rank correlation coefficients for a set of data, omitting completely any cases with a missing observation for any variable; the data array is preserved, and the ranks of the observations are not available on exit from the function.

.. _g02br-py2-py-doc:

For full information please refer to the NAG Library document for g02br

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02brf.html

.. _g02br-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[i-1,j-1] must be set to :math:x_{{ij}}, the value of the :math:i\ th observation on the :math:j\ th variable, where :math:i = 1,2,\ldots,n and :math:j = 1,2,\ldots,m\text{.}

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02brf.html#accuracy>__).

**itype** : int
The type of correlation coefficients which are to be calculated.

:math:\mathrm{itype} = -1

Only Kendall's tau coefficients are calculated.

:math:\mathrm{itype} = 0

Both Kendall's tau and Spearman's coefficients are calculated.

:math:\mathrm{itype} = 1

Only Spearman's coefficients are calculated.

**Returns**
**rr** : float, ndarray, shape :math:\left(m, m\right)
The requested correlation coefficients.

If only Kendall's tau coefficients are requested (:math:\mathrm{itype} = -1), :math:\mathrm{rr}[j-1,k-1] contains Kendall's tau for the :math:j\ th and :math:k\ th variables.

If only Spearman's coefficients are requested (:math:\mathrm{itype} = 1), :math:\mathrm{rr}[j-1,k-1] contains Spearman's rank correlation coefficient for the :math:j\ th and :math:k\ th variables.

If both Kendall's tau and Spearman's coefficients are requested (:math:\mathrm{itype} = 0), the upper triangle of :math:\mathrm{rr} contains the Spearman coefficients and the lower triangle the Kendall coefficients.

That is, for the :math:\textit{j}\ th and :math:\textit{k}\ th variables, where :math:\textit{j} is less than :math:\textit{k}, :math:\mathrm{rr}[\textit{j}-1,\textit{k}-1] contains the Spearman rank correlation coefficient, and :math:\mathrm{rr}[\textit{k}-1,\textit{j}-1] contains Kendall's tau, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

(Diagonal terms, :math:\mathrm{rr}[j-1,j-1], are unity for all three values of :math:\mathrm{itype}.)

**ncases** : int
The number of cases, :math:n_{\mathrm{c}}, actually used in the calculations (when cases involving missing values have been eliminated).

**incase** : int, ndarray, shape :math:\left(n\right)
:math:\mathrm{incase}[\textit{i}-1] holds the value :math:1 if the :math:\textit{i}\ th case was included in the calculations, and the value :math:0 if the :math:\textit{i}\ th case contained a missing value for at least one variable. That is, :math:\mathrm{incase}[\textit{i}-1] = w_{\textit{i}} (see :ref:Notes <g02br-py2-py-notes>), for :math:\textit{i} = 1,2,\ldots,n.

.. _g02br-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

(errno :math:4)
On entry, :math:\mathrm{itype} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{itype} = -1 or :math:1.

**Warns**
**NagAlgorithmicWarning**
(errno :math:5)
After observations with missing values were omitted, fewer than two cases remained.

.. _g02br-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\text{}\left(n\geq 2\right), j = 1,2,\ldots,m\text{}\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.
In addition, each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.

Let :math:w_i = 0 if observation :math:i contains a missing value for any of those variables for which missing values have been declared, i.e., if :math:x_{{ij}} = \textit{xm}_j for any :math:j for which an :math:\textit{xm}_j has been assigned (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02brf.html#accuracy>__); and :math:w_i = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

The observations are first ranked as follows.

For a given variable, :math:j say, each of the observations :math:x_{{ij}} for which :math:w_i = 1, (:math:i = 1,2,\ldots,n) has associated with it an additional number, the 'rank' of the observation, which indicates the magnitude of that observation relative to the magnitudes of the other observations on that same variable for which :math:w_i = 1.

The smallest of these valid observations for variable :math:j is assigned the rank :math:1, the second smallest observation for variable :math:j the rank :math:2, the third smallest the rank :math:3, and so on until the largest such observation is given the rank :math:n_c, where :math:n_c = \sum_{{i = 1}}^nw_i.

If a number of cases all have the same value for the given variable, :math:j, then they are each given an 'average' rank, e.g., if in attempting to assign the rank :math:h+1, :math:k observations for which :math:w_i = 1 were found to have the same value, then instead of giving them the ranks

.. math::
h+1,h+2,\ldots,h+k\text{,}

all :math:k observations would be assigned the rank

.. math::
\frac{{2h+k+1}}{2}

and the next value in ascending order would be assigned the rank

.. math::
h+k+1\text{.}

The process is repeated for each of the :math:m variables.

Let :math:y_{{ij}} be the rank assigned to the observation :math:x_{{ij}} when the :math:j\ th variable is being ranked.
For those observations, :math:i, for which :math:w_i = 0, :math:y_{{ij}} = 0, for :math:j = 1,2,\ldots,m.

The quantities calculated are:

(a) Kendall's tau rank correlation coefficients:

.. math::
R_{{jk}} = \frac{{\sum_{{h = 1}}^n\sum_{{i = 1}}^nw_hw_i\mathrm{sign}\left(y_{{hj}}-y_{{ij}}\right)\mathrm{sign}\left(y_{{hk}}-y_{{ik}}\right)}}{{\sqrt{\left[n_c\left(n_c-1\right)-T_j\right]\left[n_c\left(n_c-1\right)-T_k\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+---------------------------------------------------------+
|where|:math:n_c = \sum_{{i = 1}}^nw_i                        |
+-----+---------------------------------------------------------+
|and  |:math:\mathrm{sign}\left(u\right) = 1 if :math:u > 0 |
+-----+---------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = 0 if :math:u = 0 |
+-----+---------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = -1 if :math:u < 0|
+-----+---------------------------------------------------------+

and :math:T_j = \sum t_j\left(t_j-1\right) where :math:t_j is the number of ties of a particular value of variable :math:j, and the summation is over all tied values of variable :math:j.

(#) Spearman's rank correlation coefficients:

.. math::
R_{{jk}}^* = \frac{{n_c\left(n_c^2-1\right)-6\sum_{{i = 1}}^nw_i\left(y_{{ij}}-y_{{ik}}\right)^2-\frac{1}{2}\left(T_j^*+T_k^*\right)}}{{\sqrt{\left[n_c\left(n_c^2-1\right)-T_j^*\right]\left[n_c\left(n_c^2-1\right)-T_k^*\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

where :math:n_c = \sum_{{i = 1}}^nw_i and :math:T_j^* = \sum t_j\left(t_j^2-1\right) where :math:t_j is the number of ties of a particular value of variable :math:j, and the summation is over all tied values of variable :math:j.

.. _g02br-py2-py-references:

**References**
Siegel, S, 1956, Non-parametric Statistics for the Behavioral Sciences, McGraw--Hill

--------
:meth:naginterfaces.library.examples.correg.coeffs_kspearman_miss_case_ex.main
"""
raise NotImplementedError

[docs]def coeffs_kspearman_miss_pair(x, miss, xmiss, itype):
r"""
coeffs_kspearman_miss_pair computes Kendall and/or Spearman nonparametric rank correlation coefficients for a set of data omitting cases with missing values from only those calculations involving the variables for which the values are missing; the data array is preserved, and the ranks of the observations are not available on exit from the function.

.. _g02bs-py2-py-doc:

For full information please refer to the NAG Library document for g02bs

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bsf.html

.. _g02bs-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must be set to :math:x_{{\textit{i}\textit{j}}}, the value of the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**miss** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{miss}[j-1] must be set equal to :math:1 if a missing value, :math:xm_j, is to be specified for the :math:j\ th variable in the array :math:\mathrm{x}, or set equal to :math:0 otherwise. Values of :math:\mathrm{miss} must be given for all :math:m variables in the array :math:\mathrm{x}.

**xmiss** : float, array-like, shape :math:\left(m\right)
:math:\mathrm{xmiss}[j-1] must be set to the missing value, :math:xm_j, to be associated with the :math:j\ th variable in the array :math:\mathrm{x}, for those variables for which missing values are specified by means of the array :math:\mathrm{miss} (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bsf.html#accuracy>__).

**itype** : int
The type of correlation coefficients which are to be calculated.

:math:\mathrm{itype} = -1

Only Kendall's tau coefficients are calculated.

:math:\mathrm{itype} = 0

Both Kendall's tau and Spearman's coefficients are calculated.

:math:\mathrm{itype} = 1

Only Spearman's coefficients are calculated.

**Returns**
**rr** : float, ndarray, shape :math:\left(m, m\right)
The requested correlation coefficients.

If only Kendall's tau coefficients are requested (:math:\mathrm{itype} = -1), :math:\mathrm{rr}[j-1,k-1] contains Kendall's tau for the :math:j\ th and :math:k\ th variables.

If only Spearman's coefficients are requested (:math:\mathrm{itype} = 1), :math:\mathrm{rr}[j-1,k-1] contains Spearman's rank correlation coefficient for the :math:j\ th and :math:k\ th variables.

If both Kendall's tau and Spearman's coefficients are requested (:math:\mathrm{itype} = 0), the upper triangle of :math:\mathrm{rr} contains the Spearman coefficients and the lower triangle the Kendall coefficients.

That is, for the :math:\textit{j}\ th and :math:\textit{k}\ th variables, where :math:\textit{j} is less than :math:\textit{k}, :math:\mathrm{rr}[\textit{j}-1,\textit{k}-1] contains the Spearman rank correlation coefficient, and :math:\mathrm{rr}[\textit{k}-1,\textit{j}-1] contains Kendall's tau, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

(Diagonal terms, :math:\mathrm{rr}[j-1,j-1], are unity for all three values of :math:\mathrm{itype}.)

**ncases** : int
The minimum number of cases used in the calculation of any of the correlation coefficients (when cases involving missing values have been eliminated).

**cnt** : float, ndarray, shape :math:\left(m, m\right)
The number of cases, :math:n_{{\textit{j}\textit{k}}}, actually used in the calculation of the rank correlation coefficient for the :math:\textit{j}\ th and :math:\textit{k}\ th variables, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

.. _g02bs-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

(errno :math:4)
On entry, :math:\mathrm{itype} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{itype} = -1 or :math:1.

**Warns**
**NagAlgorithmicWarning**
(errno :math:5)
After observations with missing values were omitted, fewer than two cases remained for at least one pair of variables. (The pairs of variables involved can be determined by examination of the contents of the array :math:\mathrm{cnt}). All means, standard deviations, sums of squares and cross-products, and correlation-like coefficients based on two or more cases are returned by the function even if :math:\mathrm{errno} = 5.

.. _g02bs-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

The input data consists of :math:n observations for each of :math:m variables, given as an array

.. math::
\left[x_{{ij}}\right]\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right)\text{ and }j = 1,2,\ldots,m\left(m\geq 2\right)\text{,}

where :math:x_{{ij}} is the :math:i\ th observation on the :math:j\ th variable.
In addition each of the :math:m variables may optionally have associated with it a value which is to be considered as representing a missing observation for that variable; the missing value for the :math:j\ th variable is denoted by :math:\textit{xm}_j.
Missing values need not be specified for all variables.

Let :math:w_{{\textit{i}\textit{j}}} = 0 if the :math:\textit{i}\ th observation for the :math:\textit{j}\ th variable is a missing value, i.e., if a missing value, :math:\textit{xm}_{\textit{j}}, has been declared for the :math:\textit{j}\ th variable, and :math:x_{{\textit{i}\textit{j}}} = \textit{xm}_{\textit{j}} (see also Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bsf.html#accuracy>__); and :math:w_{{\textit{i}\textit{j}}} = 1 otherwise, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

The observations are first ranked, a pair of variables at a time as follows:

For a given pair of variables, :math:j and :math:l say, each of the observations :math:x_{{\textit{i}j}} for which the product :math:w_{{\textit{i}j}}w_{{\textit{i}l}} = 1, for :math:\textit{i} = 1,2,\ldots,n, has associated with it an additional number, the 'rank' of the observation, which indicates the magnitude of that observation relative to the magnitude of the other observations on variable :math:j for which :math:w_{{ij}}w_{{il}} = 1.

The smallest of these valid observations for variable :math:j is assigned to rank :math:1, the second smallest valid observation for variable :math:j the rank :math:2, the third smallest rank :math:3, and so on until the largest such observation is given the rank :math:n_{{jl}}, where

.. math::
n_{{jl}} = \sum_{{i = 1}}^nw_{{ij}}w_{{il}}\text{.}

If a number of cases all have the same value for the variable :math:j, then they are each given an 'average' rank, e.g., if in attempting to assign the rank :math:h+1, :math:k observations for which :math:w_{{ij}}w_{{il}} = 1 were found to have the same value, then instead of giving them the ranks

.. math::
h+1,h+2,\ldots,h+k\text{,}

all :math:k observations would be assigned the rank

.. math::
\frac{{2h+k+1}}{2}

and the next value in ascending order would be assigned the rank

.. math::
h+k+1\text{.}

The variable :math:\textit{l} is then ranked in a similar way. The process is then repeated for all pairs of variables :math:\textit{j} and :math:\textit{l}, for :math:\textit{l} = \textit{j},\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.
Let :math:y_{{\textit{i}\textit{j}\left(\textit{l}\right)}} be the rank assigned to the observation :math:x_{{\textit{i}\textit{j}}} when the :math:\textit{j}\ th and :math:\textit{l}\ th variables are being ranked, and :math:y_{{\textit{i}\textit{l}\left(\textit{j}\right)}} be the rank assigned to the observation :math:x_{{\textit{i}\textit{l}}} during the same process, for :math:\textit{l} = j,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Kendall's tau rank correlation coefficients:

.. math::
R_{{jk}} = \frac{{\sum_{{h = 1}}^n\sum_{{i = 1}}^nw_{{hj}}w_{{hk}}w_{{ij}}w_{{ik}}\mathrm{sign}\left(y_{{hj\left(k\right)}}-y_{{ij\left(k\right)}}\right)\mathrm{sign}\left(y_{{hk\left(j\right)}}-y_{{ik\left(j\right)}}\right)}}{{\sqrt{\left[n_{{jk}}\left(n_{{jk}}-1\right)-T_{{j\left(k\right)}}\right]\left[n_{{jk}}\left(n_{{jk}}-1\right)-T_{{k\left(j\right)}}\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+---------------------------------------------------------+
|where|:math:n_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}      |
+-----+---------------------------------------------------------+
|and  |:math:\mathrm{sign}\left(u\right) = 1 if :math:u > 0 |
+-----+---------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = 0 if :math:u = 0 |
+-----+---------------------------------------------------------+
|     |:math:\mathrm{sign}\left(u\right) = -1 if :math:u < 0|
+-----+---------------------------------------------------------+

and :math:T_{{j\left(k\right)}} = \sum t_j\left(t_j-1\right) where :math:t_j is the number of ties of a particular value of variable :math:j when the :math:j\ th and :math:k\ th variables are being ranked, and the summation is over all tied values of variable :math:j.

(#) Spearman's rank correlation coefficients:

.. math::
R_{{jk}}^* = \frac{{n_{{jk}}\left(n_{{jk}}^2-1\right)-6\sum_{{i = 1}}^nw_{{ij}}w_{{ik}}{\left(y_{{ij\left(k\right)}}-y_{{ik\left(j\right)}}\right)}^2-\frac{1}{2}\left(T_{{j\left(k\right)}}^*+T_{{k\left(j\right)}}^*\right)}}{{\sqrt{\left[n_{{jk}}\left(n_{{jk}}^2-1\right)-T_{{j\left(k\right)}}^*\right]\left[n_{{jk}}\left(n_{{jk}}^2-1\right)-T_{{k\left(j\right)}}^*\right]}}}\text{, }\quad j,k = 1,2,\ldots,m\text{,}

where :math:n_{{jk}} = \sum_{{i = 1}}^nw_{{ij}}w_{{ik}}

and :math:T_{{j\left(k\right)}}^* = \sum t_j\left(t_j^2-1\right), where :math:t_j is the number of ties of a particular value of variable :math:j when the :math:j\ th and :math:k\ th variables are being ranked, and the summation is over all tied values of variable :math:j.

.. _g02bs-py2-py-references:

**References**
Siegel, S, 1956, Non-parametric Statistics for the Behavioral Sciences, McGraw--Hill
"""
raise NotImplementedError

[docs]def ssqmat_update(wt, x, sw, xbar, c, mean='M'):
r"""
ssqmat_update updates the sample means and sums of squares and cross-products, or sums of squares and cross-products of deviations about the mean, for a new observation.
The data may be weighted.

.. _g02bt-py2-py-doc:

For full information please refer to the NAG Library document for g02bt

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02btf.html

.. _g02bt-py2-py-parameters:

**Parameters**
**wt** : float
The weight to use for the current observation, :math:w_i.

For unweighted means and cross-products set :math:\mathrm{wt} = 1.0.

The use of a suitable negative value of :math:\mathrm{wt}, e.g., :math:{-w_i} will have the effect of deleting the observation.

**x** : float, array-like, shape :math:\left(m\times \textit{incx}\right)
:math:\mathrm{x}[\left(j-1\right)\times \textit{incx}] must contain the value of the :math:j\ th variable for the current observation, :math:j = 1,2,\ldots,m.

**sw** : float
The sum of weights for the previous observations, :math:W_{{i-1}}.

:math:\mathrm{sw} = 0.0

The update procedure is initialized.

:math:\mathrm{sw}+\mathrm{wt} = 0.0

All elements of :math:\mathrm{xbar} and :math:\mathrm{c} are set to zero.

**xbar** : float, array-like, shape :math:\left(m\right)
If :math:\mathrm{sw} = 0.0, :math:\mathrm{xbar} is initialized, otherwise :math:\mathrm{xbar}[\textit{j}-1] must contain the weighted mean of the :math:\textit{j}\ th variable for the previous :math:\left(\textit{i}-1\right) observations, :math:\bar{x}_{\textit{j}}\left(\textit{i}-1\right), for :math:\textit{j} = 1,2,\ldots,m.

**c** : float, array-like, shape :math:\left(\left(m\times m+m\right)/2\right)
If :math:\mathrm{sw}\neq 0.0, :math:\mathrm{c} must contain the upper triangular part of the matrix of weighted sums of squares and cross-products or weighted sums of squares and cross-products of deviations about the mean. It is stored packed form by column, i.e., the cross-product between the :math:j\ th and :math:k\ th variable, :math:k\geq j, is stored in :math:\mathrm{c}[k\times \left(k-1\right)/2+j-1].

**mean** : str, length 1, optional
Indicates whether ssqmat_update is to calculate sums of squares and cross-products, or sums of squares and cross-products of deviations about the mean.

:math:\mathrm{mean} = \texttt{'M'}

The sums of squares and cross-products of deviations about the mean are calculated.

:math:\mathrm{mean} = \texttt{'Z'}

The sums of squares and cross-products are calculated.

**Returns**
**sw** : float
Contains the updated sum of weights, :math:W_i.

**xbar** : float, ndarray, shape :math:\left(m\right)
:math:\mathrm{xbar}[\textit{j}-1] contains the weighted mean of the :math:\textit{j}\ th variable, :math:\bar{x}_{\textit{j}}\left(\textit{i}\right), for :math:\textit{j} = 1,2,\ldots,m.

**c** : float, ndarray, shape :math:\left(\left(m\times m+m\right)/2\right)
The update sums of squares and cross-products stored as on input.

.. _g02bt-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{incx} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{incx} \geq 1.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:2)
On entry, :math:\mathrm{sw} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{sw}\geq 0.0.

(errno :math:3)
On entry, :math:{\left(\mathrm{sw}+\mathrm{wt}\right)} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:{\left(\mathrm{sw}+\mathrm{wt}\right)} \geq 0.0.

(errno :math:4)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

.. _g02bt-py2-py-notes:

**Notes**
ssqmat_update is an adaptation of West's WV2 algorithm; see West (1979).
This function updates the weighted means of variables and weighted sums of squares and cross-products or weighted sums of squares and cross-products of deviations about the mean for observations on :math:m variables :math:X_j, for :math:j = 1,2,\ldots,m.
For the first :math:i-1 observations let the mean of the :math:j\ th variable be :math:\bar{x}_j\left(i-1\right), the cross-product about the mean for the :math:j\ th and :math:k\ th variables be :math:c_{{jk}}\left(i-1\right) and the sum of weights be :math:W_{{i-1}}.
These are updated by the :math:i\ th observation, :math:x_{{ij}}, for :math:\textit{j} = 1,2,\ldots,m, with weight :math:w_i as follows:

.. math::

and

.. math::
c_{{jk}}\left(i\right) = c_{{jk}}\left(i-1\right)+\frac{w_i}{W_i}\left(x_j-\bar{x}_j\left(i-1\right)\right)\left(x_k-\bar{x}_k\left(i-1\right)\right)W_{{i-1}}\text{, }\quad j = 1,2,\ldots,m\text{;}k = j,j+1,2,\ldots,m\text{.}

The algorithm is initialized by taking :math:\bar{x}_j\left(1\right) = x_{{1j}}, the first observation and :math:c_{{ij}}\left(1\right) = 0.0.

For the unweighted case :math:w_i = 1 and :math:W_i = i for all :math:i.

.. _g02bt-py2-py-references:

**References**
Chan, T F, Golub, G H and Leveque, R J, 1982, Updating Formulae and a Pairwise Algorithm for Computing Sample Variances, Compstat, Physica-Verlag

West, D H D, 1979, Updating mean and variance estimates: An improved method, Comm. ACM (22), 532--555
"""
raise NotImplementedError

[docs]def ssqmat(x, mean='M', wt=None):
r"""
ssqmat calculates the sample means and sums of squares and cross-products, or sums of squares and cross-products of deviations from the mean, in a single pass for a set of data.
The data may be weighted.

.. _g02bu-py2-py-doc:

For full information please refer to the NAG Library document for g02bu

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02buf.html

.. _g02bu-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**mean** : str, length 1, optional
Indicates whether ssqmat is to calculate sums of squares and cross-products, or sums of squares and cross-products of deviations about the mean.

:math:\mathrm{mean} = \texttt{'M'}

The sums of squares and cross-products of deviations about the mean are calculated.

:math:\mathrm{mean} = \texttt{'Z'}

The sums of squares and cross-products are calculated.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
The optional weights of each observation. If weights are not provided then :math:\mathrm{wt} must be set to **None**, otherwise :math:\mathrm{wt}[i-1] must contain the weight for the :math:i\ th observation.

**Returns**
**sw** : float
The sum of weights.

If :math:\mathrm{wt}\text{ is }\mathbf{None}, :math:\mathrm{sw} contains the number of observations, :math:n.

**wmean** : float, ndarray, shape :math:\left(m\right)
The sample means. :math:\mathrm{wmean}[j-1] contains the mean for the :math:j\ th variable.

**c** : float, ndarray, shape :math:\left(\left(m\times m+m\right)/2\right)
The cross-products.

If :math:\mathrm{mean} = \texttt{'M'}, :math:\mathrm{c} contains the upper triangular part of the matrix of (weighted) sums of squares and cross-products of deviations about the mean.

If :math:\mathrm{mean} = \texttt{'Z'}, :math:\mathrm{c} contains the upper triangular part of the matrix of (weighted) sums of squares and cross-products.

These are stored packed by columns, i.e., the cross-product between the :math:j\ th and :math:k\ th variable, :math:k\geq j, is stored in :math:\mathrm{c}[k\times \left(k-1\right)/2+j-1].

.. _g02bu-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 1.

(errno :math:2)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:3)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'W'} or :math:\texttt{'U'}.

(errno :math:4)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1] \geq 0.0, for :math:i = 1,2,\ldots,n.

.. _g02bu-py2-py-notes:

**Notes**
ssqmat is an adaptation of West's WV2 algorithm; see West (1979).
This function calculates the (optionally weighted) sample means and (optionally weighted) sums of squares and cross-products or sums of squares and cross-products of deviations from the (weighted) mean for a sample of :math:n observations on :math:m variables :math:X_j, for :math:\textit{j} = 1,2,\ldots,m.
The algorithm makes a single pass through the data.

For the first :math:i-1 observations let the mean of the :math:j\ th variable be :math:\bar{x}_j\left(i-1\right), the cross-product about the mean for the :math:j\ th and :math:k\ th variables be :math:c_{{jk}}\left(i-1\right) and the sum of weights be :math:W_{{i-1}}.
These are updated by the :math:i\ th observation, :math:x_{{ij}}, for :math:\textit{j} = 1,2,\ldots,m, with weight :math:w_i as follows:

.. math::
\begin{array}{c} W_i = W_{{i-1}} + w_i \\ \bar{x}_j \left(i\right) = \bar{x}_j \left(i-1\right) + \frac{w_i}{W_i} \left(x_j-\bar{x}_j\left(i-1\right)\right) \text{, }\quad j = 1,2,\ldots,m \end{array}

and

.. math::
c_{{jk}}\left(i\right) = c_{{jk}}\left(i-1\right)+\frac{w_i}{W_i}\left(x_j-\bar{x}_j\left(i-1\right)\right)\left(x_k-\bar{x}_k\left(i-1\right)\right)W_{{i-1}}\text{, }\quad j = 1,2,\ldots,m\text{ and }k = j,j+1,\ldots,m\text{.}

The algorithm is initialized by taking :math:\bar{x}_j\left(1\right) = x_{{1j}}, the first observation, and :math:c_{{ij}}\left(1\right) = 0.0.

For the unweighted case :math:w_i = 1 and :math:W_i = i for all :math:i.

Note that only the upper triangle of the matrix is calculated and returned packed by column.

.. _g02bu-py2-py-references:

**References**
Chan, T F, Golub, G H and Leveque, R J, 1982, Updating Formulae and a Pairwise Algorithm for Computing Sample Variances, Compstat, Physica-Verlag

West, D H D, 1979, Updating mean and variance estimates: An improved method, Comm. ACM (22), 532--555

--------
:meth:naginterfaces.library.examples.correg.lars_param_ex.main
:meth:naginterfaces.library.examples.correg.linregm_fit_stepwise_ex.main
"""
raise NotImplementedError

[docs]def ssqmat_to_corrmat(m, r):
r"""
ssqmat_to_corrmat calculates a matrix of Pearson product-moment correlation coefficients from sums of squares and cross-products of deviations about the mean.

.. _g02bw-py2-py-doc:

For full information please refer to the NAG Library document for g02bw

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bwf.html

.. _g02bw-py2-py-parameters:

**Parameters**
**m** : int
:math:m, the number of variables.

**r** : float, array-like, shape :math:\left(\left(\mathrm{m}\times \mathrm{m}+\mathrm{m}\right)/2\right)
Contains the upper triangular part of the sums of squares and cross-products matrix of deviations from the mean. These are stored packed by column, i.e., the cross-product between variable :math:j and :math:k, :math:k\geq j, is stored in :math:\mathrm{r}[\left(k\times \left(k-1\right)/2+j\right)-1].

**Returns**
**r** : float, ndarray, shape :math:\left(\left(\mathrm{m}\times \mathrm{m}+\mathrm{m}\right)/2\right)
The Pearson product-moment correlation coefficients.

These are stored packed by column corresponding to the input cross-products.

.. _g02bw-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{m} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{m} \geq 1.

**Warns**
**NagAlgorithmicWarning**
(errno :math:2)
A variable has a zero variance. All correlations involving the variable with zero variance will be returned as zero.

.. _g02bw-py2-py-notes:

**Notes**
ssqmat_to_corrmat calculates a matrix of Pearson product-moment correlation coefficients from sums of squares and cross-products about the mean for observations on :math:m variables which can be computed by a single call to :meth:ssqmat or a series of calls to :meth:ssqmat_update.
The sums of squares and cross-products are stored in an array packed by column and are overwritten by the correlation coefficients.

Let :math:c_{{jk}} be the cross-product of deviations from the mean, for :math:\textit{k} = j,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m, then the product-moment correlation coefficient, :math:r_{{jk}} is given by

.. math::
r_{{jk}} = \frac{c_{{jk}}}{{\sqrt{c_{{jj}}c_{{kk}}}}}\text{.}
"""
raise NotImplementedError

[docs]def corrmat(x, nonzwt='W', wt=None):
r"""
corrmat calculates the sample means, the standard deviations, the variance-covariance matrix, and the matrix of Pearson product-moment correlation coefficients for a set of data.
Weights may be used.

.. _g02bx-py2-py-doc:

For full information please refer to the NAG Library document for g02bx

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bxf.html

.. _g02bx-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation for the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**nonzwt** : str, length 1, optional
The variance calculation uses a divisor which is either the number of weights or the number of nonzero weights.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
:math:w, the optional frequency weighting for each observation, with :math:\mathrm{wt}[i-1] = w_i. Usually :math:w_i will be an integral value corresponding to the number of observations associated with the :math:i\ th data value, or zero if the :math:i\ th data value is to be ignored. If :math:\mathrm{wt}\text{ is }\mathbf{None}, :math:w_i is set to :math:1 for all :math:i.

**Returns**
**xbar** : float, ndarray, shape :math:\left(m\right)
The sample means. :math:\mathrm{xbar}[j-1] contains the mean of the :math:j\ th variable.

**std** : float, ndarray, shape :math:\left(m\right)
The standard deviations. :math:\mathrm{std}[j-1] contains the standard deviation for the :math:j\ th variable.

**v** : float, ndarray, shape :math:\left(m, m\right)
The variance-covariance matrix. :math:\mathrm{v}[\textit{j}-1,\textit{k}-1] contains the covariance between variables :math:\textit{j} and :math:\textit{k}, for :math:\textit{k} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m.

**r** : float, ndarray, shape :math:\left(m, m\right)
The matrix of Pearson product-moment correlation coefficients. :math:\mathrm{r}[j-1,k-1] contains the correlation coefficient between variables :math:j and :math:k.

.. _g02bx-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 1.

(errno :math:2)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'U'}, :math:\texttt{'V'} or :math:\texttt{'W'}

(errno :math:3)
On entry, at least one value of :math:\mathrm{wt} is negative.

Constraint: :math:\mathrm{wt}[i-1] \geq 0, for :math:i = 1,2,\ldots,n.

(errno :math:4)
On entry, :math:\langle\mathit{\boldsymbol{value}}\rangle observations have nonzero weight.

Constraint: at least two observations must have a nonzero weight.

(errno :math:4)
On entry, Sum of the weights is :math:\langle\mathit{\boldsymbol{value}}\rangle.

Constraint: Sum of the weights must be greater than :math:1.

**Warns**
**NagAlgorithmicWarning**
(errno :math:5)
A variable has a zero variance. In this case :math:\mathrm{v} and :math:\mathrm{std} are returned as calculated but :math:\mathrm{r} will contain zero for any correlation involving a variable with zero variance.

.. _g02bx-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

For :math:n observations on :math:m variables the one-pass algorithm of West (1979) as implemented in :meth:ssqmat is used to compute the means, the standard deviations, the variance-covariance matrix, and the Pearson product-moment correlation matrix for :math:p selected variables.
Suitables weights may be used to indicate multiple observations and to remove missing values.
The quantities are defined by:

(a) The means

.. math::
\bar{ x }_j = \frac{{\sum_{{i = 1}}^nw_ix_{{ij}}}}{{\sum_{{i = 1}}^nw_i}}\quad \text{ }\quad j = 1,\ldots,p

(b) The variance-covariance matrix

.. math::
C_{{jk}} = \frac{{\sum_{{i = 1}}^nw_i\left(x_{{ij}}-\bar{ x }_j\right)\left(x_{{ik}}-\bar{ x }_k\right)}}{{\sum_{{i = 1}}^nw_i-1}}\quad \text{ }\quad j,k = 1,\ldots,p

(c) The standard deviations

.. math::

(d) The Pearson product-moment correlation coefficients

.. math::

where :math:x_{{ij}} is the value of the :math:i\ th observation on the :math:j\ th variable and :math:w_i is the weight for the :math:i\ th observation which will be :math:1 in the unweighted case.

Note that the denominator for the variance-covariance is :math:\sum_{{i = 1}}^nw_i-1, so the weights should be scaled so that the sum of weights reflects the true sample size.

.. _g02bx-py2-py-references:

**References**
Chan, T F, Golub, G H and Leveque, R J, 1982, Updating Formulae and a Pairwise Algorithm for Computing Sample Variances, Compstat, Physica-Verlag

West, D H D, 1979, Updating mean and variance estimates: An improved method, Comm. ACM (22), 532--555
"""
raise NotImplementedError

[docs]def corrmat_partial(ny, nx, isz, r):
r"""
corrmat_partial computes a partial correlation/variance-covariance matrix from a correlation or variance-covariance matrix computed by :meth:corrmat.

.. _g02by-py2-py-doc:

For full information please refer to the NAG Library document for g02by

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02byf.html

.. _g02by-py2-py-parameters:

**Parameters**
**ny** : int
The number of :math:Y variables, :math:n_y, for which partial correlation coefficients are to be computed.

**nx** : int
The number of :math:X variables, :math:n_x, which are to be considered as fixed.

**isz** : int, array-like, shape :math:\left(m\right)
Indicates which variables belong to set :math:X and :math:Y.

:math:\mathrm{isz}[i-1] < 0

The :math:\textit{i}\ th variable is a :math:Y variable, for :math:\textit{i} = 1,2,\ldots,m.

:math:\mathrm{isz}[i-1] > 0

The :math:i\ th variable is a :math:X variable.

:math:\mathrm{isz}[i-1] = 0

The :math:i\ th variable is not included in the computations.

**r** : float, array-like, shape :math:\left(m, m\right)
The variance-covariance or correlation matrix for the :math:\textit{m} variables as given by :meth:corrmat. Only the upper triangle need be given.

**Note:** the matrix must be a full rank variance-covariance or correlation matrix and so be positive definite.

This condition is not directly checked by the function.

**Returns**
**p** : float, ndarray, shape :math:\left(\mathrm{ny}, \mathrm{ny}\right)
The strict upper triangle of :math:\mathrm{p} contains the strict upper triangular part of the :math:n_y\times n_y partial correlation matrix. The lower triangle contains the lower triangle of the :math:n_y\times n_y partial variance-covariance matrix if the matrix given in :math:\mathrm{r} is a variance-covariance matrix. If the matrix given in :math:\mathrm{r} is a partial correlation matrix then the variance-covariance matrix is for standardized variables.

.. _g02by-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{ny} = \langle\mathit{\boldsymbol{value}}\rangle, :math:\mathrm{nx} = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:{\mathrm{ny}+\mathrm{nx}} \leq m.

(errno :math:1)
On entry, :math:\mathrm{nx} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{nx} \geq 1.

(errno :math:1)
On entry, :math:\mathrm{ny} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{ny} > 1.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m > 2.

(errno :math:2)
On entry, :math:\langle\mathit{\boldsymbol{value}}\rangle values of :math:\mathrm{isz} are greater than zero and :math:\mathrm{nx} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: exactly :math:\mathrm{nx} values of :math:\mathrm{isz} must be greater than zero.

(errno :math:2)
On entry, :math:\langle\mathit{\boldsymbol{value}}\rangle values of :math:\mathrm{isz} are less than zero and :math:\mathrm{ny} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: exactly :math:\mathrm{ny} values of :math:\mathrm{isz} must be less than zero.

(errno :math:3)
On entry, the square root of the correlation matrix of the independent variables is singular.

Try removing some of the :math:X variables by setting the appropriate element of :math:\mathrm{isz} = 0.

(errno :math:3)
On entry, the correlation matrix of the independent variables is singular.

Try removing some of the :math:X variables by setting the appropriate element of :math:\mathrm{isz} = 0.

(errno :math:4)
On entry, an element of the partial correlation matrix is greater than :math:1.

Constraint: :math:\mathrm{r} must be positive definite.

(errno :math:4)
On entry, a diagonal element of the partial covariance matrix is zero.

Constraint: :math:\mathrm{r} must be positive definite.

(errno :math:4)
On entry, a diagonal element of the partial covariance matrix is zero and an element of the partial correlation matrix is greater than :math:1.

Constraint: :math:\mathrm{r} must be positive definite.

.. _g02by-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

Partial correlation can be used to explore the association between pairs of random variables in the presence of other variables.
For three variables, :math:y_1, :math:y_2 and :math:x_3, the partial correlation coefficient between :math:y_1 and :math:y_2 given :math:x_3 is computed as:

.. math::
\frac{{r_{12}-r_{13}r_{23}}}{{\sqrt{\left(1-r_{13}^2\right)\left(1-r_{23}^2\right)}}}\text{,}

where :math:r_{{ij}} is the product-moment correlation coefficient between variables with subscripts :math:i and :math:j.
The partial correlation coefficient is a measure of the linear association between :math:y_1 and :math:y_2 having eliminated the effect due to both :math:y_1 and :math:y_2 being linearly associated with :math:x_3.
That is, it is a measure of association between :math:y_1 and :math:y_2 conditional upon fixed values of :math:x_3.
Like the full correlation coefficients the partial correlation coefficient takes a value in the range (:math:-1,1) with the value :math:0 indicating no association.

In general, let a set of variables be partitioned into two groups :math:Y and :math:X with :math:n_y variables in :math:Y and :math:n_x variables in :math:X and let the variance-covariance matrix of all :math:n_y+n_x variables be partitioned into,

.. math::
\left[\begin{array}{ll}\Sigma_{{xx}}&\Sigma_{{xy}}\\\Sigma_{{yx}}&\Sigma_{{yy}}\end{array}\right]\text{.}

The variance-covariance of :math:Y conditional on fixed values of the :math:X variables is given by:

.. math::
\Sigma_{{y | x}} = \Sigma_{{yy}}-\Sigma_{{yx}}\Sigma_{{xx}}^{-1}\Sigma_{{xy}}\text{.}

The partial correlation matrix is then computed by standardizing :math:\Sigma_{{y | x}},

.. math::
\mathrm{diag}\left({\left(\Sigma_{{y | x}}\right)}^{{-\frac{1}{2}}}\right)\Sigma_{{y | x}}\mathrm{diag}\left({\left(\Sigma_{{y | x}}\right)}^{{-\frac{1}{2}}}\right)\text{.}

To test the hypothesis that a partial correlation is zero under the assumption that the data has an approximately Normal distribution a test similar to the test for the full correlation coefficient can be used.
If :math:r is the computed partial correlation coefficient then the appropriate :math:t statistic is

.. math::
r\sqrt{\frac{{n-n_x-2}}{{1-r^2}}}\text{,}

which has approximately a Student's :math:t-distribution with :math:n-n_x-2 degrees of freedom, where :math:n is the number of observations from which the full correlation coefficients were computed.

.. _g02by-py2-py-references:

**References**
Krzanowski, W J, 1990, Principles of Multivariate Analysis, Oxford University Press

Morrison, D F, 1967, Multivariate Statistical Methods, McGraw--Hill

Osborn, J F, 1979, Statistical Exercises in Medical Research, Blackwell

Snedecor, G W and Cochran, W G, 1967, Statistical Methods, Iowa State University Press
"""
raise NotImplementedError

[docs]def ssqmat_combine(xsw, xmean, xc, ysw, ymean, yc, mean='M'):
r"""
ssqmat_combine combines two sets of sample means and sums of squares and cross-products matrices.
It is designed to be used in conjunction with :meth:ssqmat to allow large datasets to be summarised.

.. _g02bz-py2-py-doc:

For full information please refer to the NAG Library document for g02bz

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02bzf.html

.. _g02bz-py2-py-parameters:

**Parameters**
**xsw** : float
:math:W_x, the sum of weights, from the first set of data, :math:X. If the data is unweighted then this will be the number of observations in the first dataset.

**xmean** : float, array-like, shape :math:\left(m\right)
:math:\mu_x, the sample means for the first set of data, :math:X.

**xc** : float, array-like, shape :math:\left(\left(m\times m+m\right)/2\right)
:math:C_x, the sums of squares and cross-products matrix for the first set of data, :math:X, as returned by :meth:ssqmat.

:meth:ssqmat, returns this matrix packed by columns, i.e., the cross-product between the :math:j\ th and :math:k\ th variable, :math:k\geq j, is stored in :math:\mathrm{xc}[k\times \left(k-1\right)/2+j-1].

No check is made that :math:C_x is a valid cross-products matrix.

**ysw** : float
:math:W_y, the sum of weights, from the second set of data, :math:Y. If the data is unweighted then this will be the number of observations in the second dataset.

**ymean** : float, array-like, shape :math:\left(m\right)
:math:\mu_y, the sample means for the second set of data, :math:Y.

**yc** : float, array-like, shape :math:\left(\left(m\times m+m\right)/2\right)
:math:C_y, the sums of squares and cross-products matrix for the second set of data, :math:Y, as returned by :meth:ssqmat.

:meth:ssqmat, returns this matrix packed by columns, i.e., the cross-product between the :math:j\ th and :math:k\ th variable, :math:k\geq j, is stored in :math:\mathrm{yc}[k\times \left(k-1\right)/2+j-1].

No check is made that :math:C_y is a valid cross-products matrix.

**mean** : str, length 1, optional
Indicates whether the matrices supplied in :math:\mathrm{xc} and :math:\mathrm{yc} are sums of squares and cross-products, or sums of squares and cross-products of deviations about the mean.

:math:\mathrm{mean} = \text{‘M'}

Sums of squares and cross-products of deviations about the mean have been supplied.

:math:\mathrm{mean} = \text{‘Z'}

Sums of squares and cross-products have been supplied.

**Returns**
**xsw** : float
:math:W_z, the sum of weights, from the combined dataset, :math:Z. If both datasets are unweighted then this will be the number of observations in the combined dataset.

**xmean** : float, ndarray, shape :math:\left(m\right)
:math:\mu_z, the sample means for the combined data, :math:Z.

**xc** : float, ndarray, shape :math:\left(\left(m\times m+m\right)/2\right)
:math:C_z, the sums of squares and cross-products matrix for the combined dataset, :math:Z.

This matrix is again stored packed by columns.

.. _g02bz-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:11)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \text{‘M'} \text{ or } \text{‘Z'}.

(errno :math:21)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:31)
On entry, :math:\mathrm{xsw} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{xsw}\geq 0.0.

(errno :math:61)
On entry, :math:\mathrm{ysw} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{ysw}\geq 0.0.

.. _g02bz-py2-py-notes:

**Notes**
Let :math:X and :math:Y denote two sets of data, each with :math:m variables and :math:n_x and :math:n_y observations respectively.
Let :math:\mu_x denote the (optionally weighted) vector of :math:m means for the first dataset and :math:C_x denote either the sums of squares and cross-products of deviations from :math:\mu_x

.. math::
C_x = \left(X-e\mu_x^\mathrm{T}\right)^\mathrm{T}D_x\left(X-e\mu_x^\mathrm{T}\right)

or the sums of squares and cross-products, in which case

.. math::
C_x = X^\mathrm{T}D_xX

where :math:e is a vector of :math:n_x ones and :math:D_x is a diagonal matrix of (optional) weights and :math:W_x is defined as the sum of the diagonal elements of :math:D.
Similarly, let :math:\mu_y, :math:C_y and :math:W_y denote the same quantities for the second dataset.

Given :math:\mu_x,\mu_y,C_x,C_y,W_x and :math:W_y ssqmat_combine calculates :math:\mu_z, :math:C_z and :math:W_z as if a dataset :math:Z, with :math:m variables and :math:n_x+n_y observations were supplied to :meth:ssqmat, with :math:Z constructed as

.. math::
Z = \left(\begin{array}{c}X\\Y\end{array}\right)\text{.}

ssqmat_combine has been designed to combine the results from two calls to :meth:ssqmat allowing large datasets, or cases where all the data is not available at the same time, to be summarised.

.. _g02bz-py2-py-references:

**References**
Bennett, J, Pebay, P, Roe, D and Thompson, D, 2009, Numerically stable, single-pass, parallel statistics algorithms, Proceedings of IEEE International Conference on Cluster Computing
"""
raise NotImplementedError

[docs]def linregs_const(x, y):
r"""
linregs_const performs a simple linear regression with dependent variable :math:y and independent variable :math:x.

.. _g02ca-py2-py-doc:

For full information please refer to the NAG Library document for g02ca

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02caf.html

.. _g02ca-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{x}[\textit{i}-1] must contain :math:x_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**y** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{y}[\textit{i}-1] must contain :math:y_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**Returns**
**result** : float, ndarray, shape :math:\left(20\right)
The following information:

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[0] |:math:\bar{x}, the mean value of the independent variable, :math:x;                                                           |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[1] |:math:\bar{y}, the mean value of the dependent variable, :math:y;                                                             |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[2] |:math:s_x the standard deviation of the independent variable, :math:x;                                                        |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[3] |:math:s_y the standard deviation of the dependent variable, :math:y;                                                          |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[4] |:math:r, the Pearson product-moment correlation between the independent variable :math:x and the dependent variable :math:y;|
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[5] |:math:b, the regression coefficient;                                                                                            |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[6] |:math:a, the regression constant;                                                                                               |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[7] |:math:se\left(b\right), the standard error of the regression coefficient;                                                       |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[8] |:math:se\left(a\right), the standard error of the regression constant;                                                          |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[9] |:math:t\left(b\right), the :math:t value for the regression coefficient;                                                      |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[10]|:math:t\left(a\right), the :math:t value for the regression constant;                                                         |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[11]|:math:\mathrm{SSR}, the sum of squares attributable to the regression;                                                          |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[12]|:math:\mathrm{DFR}, the degrees of freedom attributable to the regression;                                                      |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[13]|:math:\mathrm{MSR}, the mean square attributable to the regression;                                                             |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[14]|:math:F, the :math:F value for the analysis of variance;                                                                      |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[15]|:math:\mathrm{SSD}, the sum of squares of deviations about the regression;                                                      |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[16]|:math:\mathrm{DFD}, the degrees of freedom of deviations about the regression                                                   |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[17]|:math:\mathrm{MSD}, the mean square of deviations about the regression;                                                         |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[18]|:math:\mathrm{SST}, the total sum of squares;                                                                                   |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[19]|DFT, the total degrees of freedom.                                                                                                |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+

.. _g02ca-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 2.

(errno :math:2)
On entry, all :math:\textit{n} values of at least one of :math:\mathrm{x} and :math:\mathrm{y} are identical.

.. _g02ca-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

linregs_const fits a straight line of the form

.. math::
y = a+bx

to the data points

.. math::
\left(x_1, y_1\right),\left(x_2, y_2\right),\ldots,\left(x_n, y_n\right)\text{,}

such that

.. math::
y_i = a+bx_i+e_i\text{, }\quad i = 1,2,\ldots,n\left(n > 2\right)\text{.}

The function calculates the regression coefficient, :math:b, the regression constant, :math:a (and various other statistical quantities) by minimizing

.. math::
\sum_{{i = 1}}^ne_i^2\text{.}

The input data consist of the :math:n pairs of observations

.. math::
\left(x_1, y_1\right),\left(x_2, y_2\right),\ldots,\left(x_n, y_n\right)

on the independent variable :math:x and the dependent variable :math:y.

The quantities calculated are:

(a) Means:

.. math::
\bar{x} = \frac{1}{n}\sum_{{i = 1}}^nx_i\text{; }\quad \bar{y} = \frac{1}{n}\sum_{{i = 1}}^ny_i\text{.}

(#) Standard deviations:

.. math::
s_x = \sqrt{\frac{1}{{n-1}}\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)^2}\text{; }\quad s_y = \sqrt{\frac{1}{{n-1}}\sum_{{i = 1}}^n\left(y_i-\bar{y}\right)^2}\text{.}

(#) Pearson product-moment correlation coefficient:

.. math::
r = \frac{{\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)}}{{\sqrt{\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)^2\sum_{{i = 1}}^n\left(y_i-\bar{y}\right)^2}}}\text{.}

(#) The regression coefficient, :math:b, and the regression constant, :math:a:

.. math::
b = \frac{{\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)}}{{\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)^2}}\text{;}a = \bar{y}-b\bar{x}\text{.}

(#) The sum of squares attributable to the regression, :math:\mathrm{SSR}, the sum of squares of deviations about the regression, :math:\mathrm{SSD}, and the total sum of squares, :math:\mathrm{SST}:

.. math::
\mathrm{SST} = \sum_{{i = 1}}^n\left(y_i-\bar{y}\right)^2\text{;}\mathrm{SSD} = \sum_{{i = 1}}^n\left(y_i-a-bx_i\right)^2\text{;}\mathrm{SSR} = \mathrm{SST}-\mathrm{SSD}\text{.}

(#) The degrees of freedom attributable to the regression, :math:\mathrm{DFR}, the degrees of freedom of deviations about the regression, :math:\mathrm{DFD}, and the total degrees of freedom, :math:\mathrm{DFT}:

.. math::
\mathrm{DFT} = n-1\text{; }\mathrm{DFD} = n-2\text{; }\mathrm{DFR} = 1\text{.}

(#) The mean square attributable to the regression, :math:\mathrm{MSR}, and the mean square of deviations about the regression, :math:\mathrm{MSD}:

.. math::
\mathrm{MSR} = \mathrm{SSR}/\mathrm{DFR}\text{;}\mathrm{MSD} = \mathrm{SSD}/\mathrm{DFD}\text{.}

(#) The :math:F value for the analysis of variance:

.. math::
F = \mathrm{MSR}/\mathrm{MSD}\text{.}

(#) The standard error of the regression coefficient, :math:se\left(b\right), and the standard error of the regression constant, :math:se\left(a\right):

.. math::
se\left(b\right) = \sqrt{\frac{\mathrm{MSD}}{{\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)^2}}}\text{; }\quad se\left(a\right) = \sqrt{\mathrm{MSD}\left(\frac{1}{n}+\frac{{\bar{x}^2}}{{\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)^2}}\right)}\text{.}

(#) The :math:t value for the regression coefficient, :math:t\left(b\right), and the :math:t value for the regression constant, :math:t\left(a\right):

.. math::
t\left(b\right) = \frac{b}{{se\left(b\right)}}\text{; }\quad t\left(a\right) = \frac{a}{{se\left(a\right)}}\text{.}

.. _g02ca-py2-py-references:

**References**
Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley
"""
raise NotImplementedError

[docs]def linregs_noconst(x, y):
r"""
linregs_noconst performs a simple linear regression with no constant, with dependent variable :math:y and independent variable :math:x.

.. _g02cb-py2-py-doc:

For full information please refer to the NAG Library document for g02cb

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02cbf.html

.. _g02cb-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{x}[\textit{i}-1] must contain :math:x_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**y** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{y}[\textit{i}-1] must contain :math:y_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**Returns**
**result** : float, ndarray, shape :math:\left(20\right)
The following information:

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[0] |:math:\bar{x}, the mean value of the independent variable, :math:x;                                                           |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[1] |:math:\bar{y}, the mean value of the dependent variable, :math:y;                                                             |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[2] |:math:s_x, the standard deviation of the independent variable, :math:x;                                                       |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[3] |:math:s_y, the standard deviation of the dependent variable, :math:y;                                                         |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[4] |:math:r, the Pearson product-moment correlation between the independent variable :math:x and the dependent variable :math:y;|
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[5] |:math:b, the regression coefficient;                                                                                            |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[6] |the value :math:0.0;                                                                                                            |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[7] |:math:se\left(b\right), the standard error of the regression coefficient;                                                       |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[8] |the value :math:0.0;                                                                                                            |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[9] |:math:t\left(b\right), the :math:t value for the regression coefficient;                                                      |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[10]|the value :math:0.0;                                                                                                            |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[11]|:math:SSR, the sum of squares attributable to the regression;                                                                   |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[12]|:math:DFR, the degrees of freedom attributable to the regression;                                                               |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[13]|:math:MSR, the mean square attributable to the regression;                                                                      |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[14]|:math:F, the :math:F value for the analysis of variance;                                                                      |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[15]|:math:SSD, the sum of squares of deviations about the regression;                                                               |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[16]|:math:DFD, the degrees of freedom of deviations about the regression;                                                           |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[17]|:math:MSD, the mean square of deviations about the regression;                                                                  |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[18]|:math:SST, the total sum of squares;                                                                                            |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[19]|:math:DFT, the total degrees of freedom.                                                                                        |
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------+

.. _g02cb-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 2.

(errno :math:2)
On entry, all :math:\textit{n} values of at least one of :math:\mathrm{x} and :math:\mathrm{y} are identical.

.. _g02cb-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

linregs_noconst fits a straight line of the form

.. math::
y = bx

to the data points

.. math::
\left(x_1, y_1\right),\left(x_2, y_2\right),\ldots,\left(x_n, y_n\right)\text{,}

such that

.. math::
y_i = bx_i+e_i\text{, }\quad i = 1,2,\ldots,n\left(n\geq 2\right)\text{.}

The function calculates the regression coefficient, :math:b, and the various other statistical quantities by minimizing

.. math::
\sum_{{i = 1}}^ne_i^2\text{.}

The input data consists of the :math:n pairs of observations :math:\left(x_1, y_1\right),\left(x_2, y_2\right),\ldots,\left(x_n, y_n\right) on the independent variable :math:x and the dependent variable :math:y.

The quantities calculated are:

(a) Means:

.. math::
\bar{x} = \frac{1}{n}\sum_{{i = 1}}^nx_i\text{; }\quad \bar{y} = \frac{1}{n}\sum_{{i = 1}}^ny_i\text{.}

(#) Standard deviations:

.. math::
s_x = \sqrt{\frac{1}{{n-1}}\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)^2}\text{; }\quad s_y = \sqrt{\frac{1}{{n-1}}\sum_{{i = 1}}^n\left(y_i-\bar{y}\right)^2}\text{.}

(#) Pearson product-moment correlation coefficient:

.. math::
r = \frac{{\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)}}{{\sqrt{\sum_{{i = 1}}^n\left(x_i-\bar{x}\right)^2\sum_{{i = 1}}^n\left(y_i-\bar{y}\right)^2}}}\text{.}

(#) The regression coefficient, :math:b:

.. math::
b = \frac{{\sum_{{i = 1}}^nx_iy_i}}{{\sum_{{i = 1}}^nx_i^2}}\text{.}

(#) The sum of squares attributable to the regression, :math:SSR, the sum of squares of deviations about the regression, :math:SSD, and the total sum of squares, :math:SST:

.. math::
SST = \sum_{{i = 1}}^ny_i^2\text{; }\quad SSD = \sum_{{i = 1}}^n\left(y_i-bx_i\right)^2\text{, }\quad SSR = SST-SSD\text{.}

(#) The degrees of freedom attributable to the regression, :math:DFR, the degrees of freedom of deviations about the regression, :math:DFD, and the total degrees of freedom, :math:DFT:

.. math::

(#) The mean square attributable to the regression, :math:MSR, and the mean square of deviations about the regression, :math:MSD\text{.}

.. math::
MSR = SSR/DFR\text{; }\quad MSD = SSD/DFD\text{.}

(#) The :math:F value for the analysis of variance:

.. math::
F = MSR/MSD\text{.}

(#) The standard error of the regression coefficient:

.. math::
se\left(b\right) = \sqrt{\frac{{MSD}}{{\sum_{{i = 1}}^nx_i^2}}}\text{.}

(#) The :math:t value for the regression coefficient:

.. math::
t\left(b\right) = \frac{b}{{se\left(b\right)}}\text{.}

.. _g02cb-py2-py-references:

**References**
Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley
"""
raise NotImplementedError

[docs]def linregs_const_miss(x, y, xmiss, ymiss):
r"""
linregs_const_miss performs a simple linear regression with dependent variable :math:y and independent variable :math:x, omitting cases involving missing values.

.. _g02cc-py2-py-doc:

For full information please refer to the NAG Library document for g02cc

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02ccf.html

.. _g02cc-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{x}[\textit{i}-1] must contain :math:x_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**y** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{y}[\textit{i}-1] must contain :math:y_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**xmiss** : float
The value :math:xm which is to be taken as the missing value for the variable :math:x. See Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02ccf.html#accuracy>__.

**ymiss** : float
The value :math:ym which is to be taken as the missing value for the variable :math:y. See Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02ccf.html#accuracy>__.

**Returns**
**result** : float, ndarray, shape :math:\left(21\right)
The following information:

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[0] |:math:\bar{x}, the mean value of the independent variable, :math:x;                                                          |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[1] |:math:\bar{y}, the mean value of the dependent variable, :math:y;                                                            |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[2] |:math:s_x, the standard deviation of the independent variable, :math:x;                                                      |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[3] |:math:s_y, the standard deviation of the dependent variable, :math:y;                                                        |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[4] |:math:r, the Pearson product-moment correlation between the independent variable :math:x and the dependent variable :math:y|
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[5] |:math:b, the regression coefficient;                                                                                           |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[6] |:math:a, the regression constant;                                                                                              |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[7] |:math:se\left(b\right), the standard error of the regression coefficient;                                                      |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[8] |:math:se\left(a\right), the standard error of the regression constant;                                                         |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[9] |:math:t\left(b\right), the :math:t value for the regression coefficient;                                                     |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[10]|:math:t\left(a\right), the :math:t value for the regression constant;                                                        |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[11]|:math:SSR, the sum of squares attributable to the regression;                                                                  |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[12]|:math:DFR, the degrees of freedom attributable to the regression;                                                              |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[13]|:math:MSR, the mean square attributable to the regression;                                                                     |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[14]|:math:F, the :math:F value for the analysis of variance;                                                                     |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[15]|:math:SSD, the sum of squares of deviations about the regression;                                                              |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[16]|:math:DFD, the degrees of freedom of deviations about the regression;                                                          |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[17]|:math:MSD, the mean square of deviations about the regression;                                                                 |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[18]|:math:SST, the total sum of squares;                                                                                           |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[19]|:math:DFT, the total degrees of freedom;                                                                                       |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[20]|:math:n_c, the number of observations used in the calculations.                                                                |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------+

.. _g02cc-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > 2.

(errno :math:2)
After observations with missing values were omitted, two or fewer cases remained.

(errno :math:3)
After observations with missing values were omitted, all remaining values of at least one of :math:\mathrm{x} and :math:\mathrm{y} were identical.

.. _g02cc-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

linregs_const_miss fits a straight line of the form

.. math::
y = a+bx

to those of the data points

.. math::
\left(x_1, y_1\right),\left(x_2, y_2\right),\ldots,\left(x_n, y_n\right)

that do not include missing values, such that

.. math::
y_i = a+bx_i+e_i

for those :math:\left(x_i, y_i\right), :math:i = 1,2,\ldots,n\quad \text{ }\quad \left(n > 2\right) which do not include missing values.

The function eliminates all pairs of observations :math:\left(x_i, y_i\right) which contain a missing value for either :math:x or :math:y, and then calculates the regression coefficient, :math:b, the regression constant, :math:a, and various other statistical quantities, by minimizing the sum of the :math:e_i^2 over those cases remaining in the calculations.

The input data consists of the :math:n pairs of observations :math:\left(x_1, y_1\right),\left(x_2, y_2\right),\ldots,\left(x_n, y_n\right) on the independent variable :math:x and the dependent variable :math:y.

In addition two values, :math:\textit{xm} and :math:\textit{ym}, are given which are considered to represent missing observations for :math:x and :math:y respectively. (See Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02ccf.html#accuracy>__).

Let :math:w_i = 0 if the :math:i\ th observation of either :math:x or :math:y is missing, i.e., if :math:x_i = \textit{xm} and/or :math:y_i = \textit{ym}; and :math:w_i = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x} = \frac{{\sum_{{i = 1}}^nw_ix_i}}{{\sum_{{i = 1}}^nw_i}}\text{; }\quad \bar{y} = \frac{{\sum_{{i = 1}}^nw_iy_i}}{{\sum_{{i = 1}}^nw_i}}\text{.}

(#) Standard deviations:

.. math::
s_x = \sqrt{\frac{{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)^2}}{{\sum_{{i = 1}}^nw_i-1}}}\text{; }\quad s_y = \sqrt{\frac{{\sum_{{i = 1}}^nw_i\left(y_i-\bar{y}\right)^2}}{{\sum_{{i = 1}}^nw_i-1}}}\text{.}

(#) Pearson product-moment correlation coefficient:

.. math::
r = \frac{{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)}}{{\sqrt{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)^2\sum_{{i = 1}}^nw_i\left(y_i-\bar{y}\right)^2}}}\text{.}

(#) The regression coefficient, :math:b, and the regression constant, :math:a:

.. math::
b = \frac{{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)}}{{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)^2}}\text{, }\quad a = \bar{y}-b\bar{x}\text{.}

(#) The sum of squares attributable to the regression, :math:SSR, the sum of squares of deviations about the regression, :math:SSD, and the total sum of squares, :math:SST:

.. math::
SST = \sum_{{i = 1}}^nw_i\left(y_i-\bar{y}\right)^2\text{; }\quad SSD = \sum_{{i = 1}}^nw_i\left(y_i-a-bx_i\right)^2\text{; }\quad SSR = SST-SSD\text{.}

(#) The degrees of freedom attributable to the regression, :math:DFR, the degrees of freedom of deviations about the regression, :math:DFD, and the total degrees of freedom, :math:DFT:

.. math::
DFT = \sum_{{i = 1}}^nw_i-1\text{; }\quad DFD = \sum_{{i = 1}}^nw_i-2\text{; }\quad DFR = 1\text{.}

(#) The mean square attributable to the regression, :math:MSR, and the mean square of deviations about the regression, :math:MSD:

.. math::
MSR = SSR/DFR\text{; }\quad MSD = SSD/DFD\text{.}

(#) The :math:F value for the analysis of variance:

.. math::
F = MSR/MSD\text{.}

(#) The standard error of the regression coefficient, :math:se\left(b\right), and the standard error of the regression constant, :math:se\left(a\right):

.. math::
se\left(b\right) = \sqrt{\frac{{MSD}}{{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)^2}}}\text{; }\quad se\left(a\right) = \sqrt{MSD\left(\frac{1}{{\sum_{{i = 1}}^nw_i}}+\frac{{\bar{x}^2}}{{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)^2}}\right)}\text{.}

(#) The :math:t value for the regression coefficient, :math:t\left(b\right), and the :math:t value for the regression constant, :math:t\left(a\right):

.. math::
t\left(b\right) = \frac{b}{{se\left(b\right)}}\text{; }\quad t\left(a\right) = \frac{a}{{se\left(a\right)}}\text{.}

(#) The number of observations used in the calculations:

.. math::
n_c = \sum_{{i = 1}}^nw_i\text{.}

.. _g02cc-py2-py-references:

**References**
Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley
"""
raise NotImplementedError

[docs]def linregs_noconst_miss(x, y, xmiss, ymiss):
r"""
linregs_noconst_miss performs a simple linear regression with no constant, with dependent variable :math:y and independent variable :math:x, omitting cases involving missing values.

.. _g02cd-py2-py-doc:

For full information please refer to the NAG Library document for g02cd

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02cdf.html

.. _g02cd-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{x}[\textit{i}-1] must contain :math:x_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**y** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{y}[\textit{i}-1] must contain :math:y_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**xmiss** : float
The value :math:xm, which is to be taken as the missing value for the variable :math:x (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02cdf.html#accuracy>__).

**ymiss** : float
The value :math:ym, which is to be taken as the missing value for the variable :math:y (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02cdf.html#accuracy>__).

**Returns**
**result** : float, ndarray, shape :math:\left(21\right)
The following information:

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[0] |:math:\bar{x}, the mean value of the independent variable, :math:x;                                                            |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[1] |:math:\bar{y}, the mean value of the dependent variable, :math:y;                                                              |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[2] |:math:s_x, the standard deviation of the independent variable, :math:x;                                                        |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[3] |:math:s_y, the standard deviation of the dependent variable, :math:y;                                                          |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[4] |:math:r, the Pearson product-moment correlation between the independent variable :math:x and the dependent variable, :math:y;|
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[5] |:math:b, the regression coefficient;                                                                                             |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[6] |the value :math:0.0;                                                                                                             |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[7] |:math:se\left(b\right), the standard error of the regression coefficient;                                                        |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[8] |the value :math:0.0;                                                                                                             |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[9] |:math:t\left(b\right), the :math:t value for the regression coefficient;                                                       |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[10]|the value :math:0.0;                                                                                                             |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[11]|:math:SSR, the sum of squares attributable to the regression;                                                                    |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[12]|:math:DFR, the degrees of freedom attributable to the regression;                                                                |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[13]|:math:MSR, the mean square attributable to the regression;                                                                       |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[14]|:math:F, the :math:F value for the analysis of variance;                                                                       |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[15]|:math:SSD, the sum of squares of deviations about the regression;                                                                |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[16]|:math:DFD, the degrees of freedom of deviations about the regression;                                                            |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[17]|:math:MSD, the mean square of deviations about the regression;                                                                   |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[18]|:math:SST, the total sum of squares                                                                                              |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[19]|:math:DFT, the total degrees of freedom;                                                                                         |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[20]|:math:n_c, the number of observations used in the calculations.                                                                  |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------+

.. _g02cd-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
After observations with missing values were omitted, fewer than two cases remained.

(errno :math:3)
After observations with missing values were omitted, all remaining values of at least one of :math:\mathrm{x} and :math:\mathrm{y} were identical.

.. _g02cd-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

linregs_noconst_miss fits a straight line of the form

.. math::
y = bx

to those of the data points

.. math::
\left(x_1, y_1\right),\left(x_2, y_2\right),\ldots,\left(x_n, y_n\right)

that do not include missing values, such that

.. math::
y_i = bx_i+e_i

for those :math:\left(x_i, y_i\right), for :math:i = 1,2,\ldots,n\quad \text{ }\quad \left(n\geq 2\right) which do not include missing values.

The function eliminates all pairs of observations :math:\left(x_i, y_i\right) which contain a missing value for either :math:x or :math:y, and then calculates the regression coefficient, :math:b, and various other statistical quantities by minimizing the sum of the :math:e_i^2 over those cases remaining in the calculations.

The input data consists of the :math:n pairs of observations :math:\left(x_1, y_1\right),\left(x_2, y_2\right),\ldots,\left(x_n, y_n\right) on the independent variable :math:x and the dependent variable :math:y.

In addition two values, :math:\textit{xm} and :math:\textit{ym}, are given which are considered to represent missing observations for :math:x and :math:y respectively. (See Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02cdf.html#accuracy>__).

Let :math:w_{\textit{i}} = 0, if the :math:\textit{i}\ th observation of either :math:x or :math:y is missing, i.e., if :math:x_{\textit{i}} = \textit{xm} and/or :math:y_{\textit{i}} = \textit{ym}; and :math:w_{\textit{i}} = 1 otherwise, for :math:\textit{i} = 1,2,\ldots,n.

The quantities calculated are:

(a) Means:

.. math::
\bar{x} = \frac{{\sum_{{i = 1}}^nw_ix_i}}{{\sum_{{i = 1}}^nw_i}}\text{; }\quad \bar{y} = \frac{{\sum_{{i = 1}}^nw_iy_i}}{{\sum_{{i = 1}}^nw_i}}\text{.}

(#) Standard deviations:

.. math::
s_x = \sqrt{\frac{{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)^2}}{{\sum_{{i = 1}}^nw_i-1}}}\text{; }\quad s_y = \sqrt{\frac{{\sum_{{i = 1}}^nw_i\left(y_i-\bar{y}\right)^2}}{{\sum_{{i = 1}}^nw_i-1}}}\text{.}

(#) Pearson product-moment correlation coefficient:

.. math::
r = \frac{{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)}}{{\sum_{{i = 1}}^nw_i\left(x_i-\bar{x}\right)^2\sum_{{i = 1}}^nw_i\left(y_i-\bar{y}\right)^2}}\text{.}

(#) The regression coefficient, :math:b:

.. math::
b = \frac{{\sum_{{i = 1}}^nw_ix_iy_i}}{{\sum_{{i = 1}}^nw_ix_i^2}}\text{.}

(#) The sum of squares attributable to the regression, :math:SSR, the sum of squares of deviations about the regression, :math:SSD, and the total sum of squares, :math:SST:

.. math::
SST = \sum_{{i = 1}}^nw_iy_i^2\text{; }\quad SSD = \sum_{{i = 1}}^nw_i\left(y_i-bx_i\right)^2\text{; }\quad SSR = SST-SSD\text{.}

(#) The degrees of freedom attributable to the regression, :math:DFR, the degrees of freedom of deviations about the regression, :math:DFD, and the total degrees of freedom, :math:DFT:

.. math::
DFT = \sum_{{i = 1}}^nw_i\text{; }\quad DFD = \sum_{{i = 1}}^nw_i-1\text{; }\quad DFR = 1\text{.}

(#) The mean square attributable to the regression, :math:MSR, and the mean square of deviations about the regression, :math:MSD:

.. math::
MSR = SSR/DFR\text{; }\quad MSD = SSD/DFD\text{.}

(#) The :math:F value for the analysis of variance:

.. math::
F = MSR/MSD\text{.}

(#) The standard error of the regression coefficient:

.. math::
se\left(b\right) = \sqrt{\frac{{MSD}}{{\sum_{{i = 1}}^nw_ix_i^2}}}\text{.}

(#) The :math:t value for the regression coefficient:

.. math::
t\left(b\right) = \frac{b}{{se\left(b\right)}}\text{.}

(#) The number of observations used in the calculations:

.. math::
n_c = \sum_{{i = 1}}^nw_i\text{.}

.. _g02cd-py2-py-references:

**References**
Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley
"""
raise NotImplementedError

[docs]def linregm_service_select(xbar, std, ssp, r, korder):
r"""
linregm_service_select takes selected elements from two vectors (typically vectors of means and standard deviations) to form two smaller vectors, and selected rows and columns from two matrices (typically either matrices of sums of squares and cross-products of deviations from means and Pearson product-moment correlation coefficients, or matrices of sums of squares and cross-products about zero and correlation-like coefficients) to form two smaller matrices, allowing reordering of elements in the process.

.. _g02ce-py2-py-doc:

For full information please refer to the NAG Library document for g02ce

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02cef.html

.. _g02ce-py2-py-parameters:

**Parameters**
**xbar** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{xbar}[\textit{i}-1] must be set to :math:\bar{x}_{\textit{i}}, the mean of variable :math:\textit{i}, for :math:\textit{i} = 1,2,\ldots,n.

**std** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{std}[\textit{i}-1] must be set to :math:s_{\textit{i}}, the standard deviation of variable :math:\textit{i}, for :math:\textit{i} = 1,2,\ldots,n.

**ssp** : float, array-like, shape :math:\left(n, n\right)
:math:\mathrm{ssp}[\textit{i}-1,\textit{j}-1] must be set to the sum of cross-products of deviations from means :math:S_{{\textit{i}\textit{j}}} (or about zero, :math:\tilde{S}_{{\textit{i}\textit{j}}}) for variables :math:\textit{i} and :math:\textit{j}, for :math:\textit{j} = 1,2,\ldots,n, for :math:\textit{i} = 1,2,\ldots,n.

**r** : float, array-like, shape :math:\left(n, n\right)
:math:\mathrm{r}[\textit{i}-1,\textit{j}-1] must be set to the Pearson product-moment correlation coefficient :math:R_{{\textit{i}\textit{j}}} (or the correlation-like coefficient, :math:\tilde{R}_{{\textit{i}\textit{j}}}) for variables :math:\textit{i} and :math:\textit{j}, for :math:\textit{j} = 1,2,\ldots,n, for :math:\textit{i} = 1,2,\ldots,n.

**korder** : int, array-like, shape :math:\left(m\right)
:math:\mathrm{korder}[\textit{i}-1] must be set to the number of the original variable which is to be the :math:\textit{i}\ th variable in the output vectors and matrices, for :math:\textit{i} = 1,2,\ldots,m.

**Returns**
**xbar2** : float, ndarray, shape :math:\left(m\right)
The mean of variable :math:i, :math:\mathrm{xbar}[i-1], where :math:i = \mathrm{korder}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,m. (The array :math:\mathrm{xbar2} must differ from :math:\mathrm{xbar} and :math:\mathrm{std}.)

**std2** : float, ndarray, shape :math:\left(m\right)
The standard deviation of variable :math:i, :math:\mathrm{std}[i-1], where :math:i = \mathrm{korder}[\textit{k}-1], for :math:\textit{k} = 1,2,\ldots,m. (The array :math:\mathrm{std2} must differ from both :math:\mathrm{xbar} and :math:\mathrm{std}.)

**ssp2** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{ssp2}[\textit{k}-1,\textit{l}-1] contains the value of :math:\mathrm{ssp}[i-1,j-1], where :math:i = \mathrm{korder}[\textit{k}-1] and :math:j = \mathrm{korder}[\textit{l}-1], for :math:\textit{l} = 1,2,\ldots,m, for :math:\textit{k} = 1,2,\ldots,m. (The array :math:\mathrm{ssp2} must differ from both :math:\mathrm{ssp} and :math:\mathrm{r}.)

That is to say: on exit, :math:\mathrm{ssp2}[k-1,l-1] contains the sum of cross-products of deviations from means :math:S_{{ij}} (or about zero, :math:\tilde{S}_{{ij}}).

**r2** : float, ndarray, shape :math:\left(m, m\right)
:math:\mathrm{r2}[\textit{k}-1,\textit{l}-1] contains the value of :math:\mathrm{r}[i-1,j-1], where :math:i = \mathrm{korder}[\textit{k}-1] and :math:j = \mathrm{korder}[\textit{l}-1], for :math:\textit{l} = 1,2,\ldots,m, for :math:\textit{k} = 1,2,\ldots,m. (The array :math:\mathrm{r2} must differ from both :math:\mathrm{ssp} and :math:\mathrm{r}.)

That is to say: on exit, :math:\mathrm{r2}[k-1,l-1] contains the Pearson product-moment coefficient :math:R_{{ij}} (or the correlation-like coefficient, :math:\tilde{R}_{{ij}}).

.. _g02ce-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constratint: :math:n \geq m.

(errno :math:4)
On entry, :math:\mathrm{korder}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle and :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{korder}[i-1]\leq n, for :math:i = 1,2,\ldots,m.

.. _g02ce-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

Input to the function consists of:

(a) A vector of means:

.. math::
\left({\bar{x}_1}, {\bar{x}_2}, {\bar{x}_3}, \ldots, {\bar{x}_n}\right)\text{,}

where :math:n is the number of input variables.

(#) A vector of standard deviations:

.. math::
\left(s_1, s_2, s_3, \ldots, s_n\right)\text{.}

(#) A matrix of sums of squares and cross-products of deviations from means:

.. math::
\begin{pmatrix}S_{11}&S_{12}&S_{13}&.&.&.&S_{{1n}}\\S_{21}&S_{22}&&&&&S_{{2n}}\\S_{31}&&&&&&.\\.&&&&&&.\\.&&&&&&.\\.&&&&&&.\\S_{{n1}}&S_{{n2}}&.&.&.&.&S_{{nn}}\end{pmatrix}\text{.}

(#) A matrix of correlation coefficients:

.. math::
\begin{pmatrix}R_{11}&R_{12}&R_{13}&.&.&.&R_{{1n}}\\R_{21}&R_{22}&&&&&R_{{2n}}\\R_{31}&&&&&&.\\.&&&&&&.\\.&&&&&&.\\.&&&&&&.\\R_{{n1}}&R_{{n2}}&.&.&.&.&R_{{nn}}\end{pmatrix}\text{.}

(#) The number of variables, :math:m, in the required subset, and their row/column numbers in the input data, :math:i_1,i_2,i_3,\ldots,i_m,

.. math::
i\leq i_k\leq n\quad \text{ for }k = 1,2,\ldots,m\quad \text{ }\quad \left({n\geq 2}, {m\geq 1\text{ and }m\leq n}\right)\text{.}

New vectors and matrices are output containing the following information:

(i) A vector of means:

.. math::
\left({\bar{x}_{i_1}}, {\bar{x}_{i_2}}, {\bar{x}_{i_3}}, \ldots, {\bar{x}_{i_m}}\right)\text{.}

(#) A vector of standard deviations:

.. math::
\left({s_{i_1}}, {s_{i_2}}, {s_{i_3}}, \ldots, {s_{i_m}}\right)\text{.}

(#) A matrix of sums of squares and cross-products of deviations from means:

.. math::
\begin{pmatrix}S_{{i_1i_1}}&S_{{i_1i_2}}&S_{{i_1i_3}}&.&.&.&S_{{i_1i_m}}\\S_{{i_2i_1}}&S_{{i_2i_2}}&&&&&.\\S_{{i_3i_1}}&&&&&&.\\.&&&&&&.\\.&&&&&&.\\.&&&&&&.\\S_{{i_mi_1}}&S_{{i_mi_2}}&.&.&.&.&S_{{i_mi_m}}\end{pmatrix}\text{.}

(#) A matrix of correlation coefficients:

.. math::
\begin{pmatrix}R_{{i_1i_1}}&R_{{i_1i_2}}&R_{{i_1i_3}}&.&.&.&R_{{i_1i_m}}\\R_{{i_2i_1}}&R_{{i_2i_2}}&&&&&.\\R_{{i_3i_1}}&&&&&&.\\.&&&&&&.\\.&&&&&&.\\.&&&&&&.\\R_{{i_mi_1}}&R_{{i_mi_2}}&.&.&.&.&R_{{i_mi_m}}\end{pmatrix}\text{.}

**Note:** for sums of squares of cross-products of deviations about zero and correlation-like coefficients :math:S_{{ij}} and :math:R_{{ij}} should be replaced by :math:\tilde{S}_{{ij}} and :math:\tilde{R}_{{ij}} in the description of the input and output above.
"""
raise NotImplementedError

[docs]def linregm_service_reorder(korder, xbar, std, ssp, r):
r"""
linregm_service_reorder reorders the elements in two vectors (typically vectors of means and standard deviations), and the rows and columns in two matrices (typically either matrices of sums of squares and cross-products of deviations from means and Pearson product-moment correlation coefficients, or matrices of sums of squares and cross-products about zero and correlation-like coefficients).

.. _g02cf-py2-py-doc:

For full information please refer to the NAG Library document for g02cf

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02cff.html

.. _g02cf-py2-py-parameters:

**Parameters**
**korder** : int, array-like, shape :math:\left(n\right)
:math:\mathrm{korder}[\textit{i}-1] must be set to the number of the original variable which is to be the :math:\textit{i}\ th variable in the re-arranged data, for :math:\textit{i} = 1,2,\ldots,n.

**xbar** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{xbar}[\textit{i}-1] must be set to the mean of variable :math:\textit{i}, for :math:\textit{i} = 1,2,\ldots,n.

**std** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{std}[\textit{i}-1] must be set to the standard deviation of variable :math:\textit{i}, for :math:\textit{i} = 1,2,\ldots,n.

**ssp** : float, array-like, shape :math:\left(n, n\right)
:math:\mathrm{ssp}[\textit{i}-1,\textit{j}-1] must be set to the sum of cross-products of deviations from means :math:S_{{\textit{i}\textit{j}}} (or about zero :math:\tilde{S}_{{\textit{i}\textit{j}}}) for variables :math:\textit{i} and :math:\textit{j}, for :math:\textit{j} = 1,2,\ldots,n, for :math:\textit{i} = 1,2,\ldots,n.

**r** : float, array-like, shape :math:\left(n, n\right)
:math:\mathrm{r}[\textit{i}-1,\textit{j}-1] must be set to the Pearson product-moment correlation coefficient :math:R_{{\textit{i}\textit{j}}} (or the correlation-like coefficient :math:\tilde{R}_{{\textit{i}\textit{j}}}) for variables :math:\textit{i} and :math:\textit{j}, for :math:\textit{j} = 1,2,\ldots,n, for :math:\textit{i} = 1,2,\ldots,n.

**Returns**
**xbar** : float, ndarray, shape :math:\left(n\right)
:math:\mathrm{xbar}[\textit{i}-1] contains the mean of variable :math:k where :math:k = \mathrm{korder}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,n.

**std** : float, ndarray, shape :math:\left(n\right)
:math:\mathrm{std}[\textit{i}-1] contains the standard deviation of variable :math:k where :math:k = \mathrm{korder}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,n.

**ssp** : float, ndarray, shape :math:\left(n, n\right)
:math:\mathrm{ssp}[i-1,j-1] contains the sum of cross-products of deviations from means :math:S_{{kl}} (or about zero :math:\tilde{S}_{{kl}}) for variables :math:k and :math:l, where :math:k = \mathrm{korder}[i-1], and :math:l = \mathrm{korder}[j-1], :math:i,j = 1,2,\ldots,n.

**r** : float, ndarray, shape :math:\left(n, n\right)
:math:\mathrm{r}[\textit{i}-1,\textit{j}-1] contains the Pearson product-moment correlation coefficient :math:R_{{kl}} (or the correlation-like coefficient :math:\tilde{R}_{{kl}}) for variables :math:k and :math:l, where :math:k = \mathrm{korder}[\textit{i}-1] and :math:l = \mathrm{korder}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,n, for :math:\textit{i} = 1,2,\ldots,n.

.. _g02cf-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:3)
On entry, :math:\mathrm{korder}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle and :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{korder}[i-1]\leq n, for :math:i = 1,2,\ldots,m.

(errno :math:4)
On entry, there is not a one-to-one correspondence between the old variables and the new variables; at least one of the original variables is not included in the new set, and consequently at least one other variable has been included more than once.

.. _g02cf-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

Input to the function consists of:

(a) A list of the order in which the :math:n variables are to be arranged on exit:

.. math::
i_1,i_2,i_3,\ldots,i_n\text{.}

(#) A vector of means:

.. math::
\left({\bar{x}_1}, {\bar{x}_2}, {\bar{x}_3}, \ldots, {\bar{x}_n}\right)\text{.}

(#) A vector of standard deviations:

.. math::
\left(s_1, s_2, s_3, \ldots, s_n\right)\text{.}

(#) A matrix of sums of squares and cross-products of deviations from means:

.. math::
\begin{pmatrix}S_{11}&S_{12}&S_{13}&.&.&.&S_{{1n}}\\S_{21}&S_{22}&&&&&.\\S_{31}&&&&&&.\\.&&&&&&.\\.&&&&&&.\\.&&&&&&.\\S_{{n1}}&S_{{n2}}&.&.&.&.&S_{{nn}}\end{pmatrix}\text{.}

(#) A matrix of correlation coefficients:

.. math::
\begin{pmatrix}R_{11}&R_{12}&R_{13}&.&.&.&R_{{1n}}\\R_{21}&R_{22}&&&&&.\\R_{31}&&&&&&.\\.&&&&&&.\\.&&&&&&.\\.&&&&&&.\\R_{{n1}}&R_{{n2}}&.&.&.&.&R_{{nn}}\end{pmatrix}\text{.}

On exit from the function, these same vectors and matrices are reordered, in the manner specified, and contain the following information:

(i) The vector of means:

.. math::
\left({\bar{x}_{i_1}}, {\bar{x}_{i_2}}, {\bar{x}_{i_3}}, \ldots, {\bar{x}_{i_n}}\right)\text{.}

(#) The vector of standard deviations:

.. math::
\left({s_{i_1}}, {s_{i_2}}, {s_{i_3}}, {\ldots s_{i_n}}\right)\text{.}

(#) The matrix of sums of squares and cross-products of deviations from means:

.. math::
\begin{pmatrix}S_{{i_1i_1}}&S_{{i_1i_2}}&S_{{i_1i_3}}&.&.&.&S_{{i_1i_n}}\\S_{{i_2i_1}}&S_{{i_2i_2}}&&&&&.\\S_{{i_3i_1}}&&&&&&.\\.&&&&&&.\\.&&&&&&.\\.&&&&&&.\\S_{{i_ni_1}}&S_{{i_ni_2}}&.&.&.&.&S_{{i_ni_n}}\end{pmatrix}\text{.}

(#) The matrix of correlation coefficients:

.. math::
\begin{pmatrix}R_{{i_1i_1}}&R_{{i_1i_2}}&R_{{i_1i_3}}&.&.&.&R_{{i_1i_n}}\\R_{{i_2i_1}}&R_{{i_2i_2}}&&&&&.\\R_{{i_3i_1}}&&&&&&.\\.&&&&&&.\\.&&&&&&.\\.&&&&&&.\\R_{{i_ni_1}}&R_{{i_ni_2}}&.&.&.&.&R_{{i_ni_n}}\end{pmatrix}\text{.}

**Note:** for sums of squares of cross-products of deviations about zero and correlation-like coefficients :math:S_{{ij}} and :math:R_{{ij}} should be replaced by :math:\tilde{S}_{{ij}} and :math:\tilde{R}_{{ij}} in the description of the input and output above.
"""
raise NotImplementedError

[docs]def linregm_coeffs_const(n, xbar, ssp, r):
r"""
linregm_coeffs_const performs a multiple linear regression on a set of variables whose means, sums of squares and cross-products of deviations from means, and Pearson product-moment correlation coefficients are given.

.. _g02cg-py2-py-doc:

For full information please refer to the NAG Library document for g02cg

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02cgf.html

.. _g02cg-py2-py-parameters:

**Parameters**
**n** : int
The number of cases :math:n, used in calculating the sums of squares and cross-products and correlation coefficients.

**xbar** : float, array-like, shape :math:\left(k+1\right)
:math:\mathrm{xbar}[\textit{i}-1] must be set to :math:\bar{x}_{\textit{i}}, the mean value of the :math:\textit{i}\ th variable, for :math:\textit{i} = 1,2,\ldots,k+1; the mean of the dependent variable must be contained in :math:\mathrm{xbar}[k].

**ssp** : float, array-like, shape :math:\left(k+1, k+1\right)
:math:\mathrm{ssp}[\textit{i}-1,\textit{j}-1] must be set to :math:S_{{\textit{i}\textit{j}}}, the sum of cross-products of deviations from means for the :math:\textit{i}\ th and :math:\textit{j}\ th variables, for :math:\textit{j} = 1,2,\ldots,k+1, for :math:\textit{i} = 1,2,\ldots,k+1; terms involving the dependent variable appear in row :math:k+1 and column :math:k+1.

**r** : float, array-like, shape :math:\left(k+1, k+1\right)
:math:\mathrm{r}[\textit{i}-1,\textit{j}-1] must be set to :math:R_{{\textit{i}\textit{j}}}, the Pearson product-moment correlation coefficient for the :math:\textit{i}\ th and :math:\textit{j}\ th variables, for :math:\textit{j} = 1,2,\ldots,k+1, for :math:\textit{i} = 1,2,\ldots,k+1; terms involving the dependent variable appear in row :math:k+1 and column :math:k+1.

**Returns**
**result** : float, ndarray, shape :math:\left(13\right)
The following information:

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[0] |:math:SSR, the sum of squares attributable to the regression;                                   |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[1] |:math:DFR, the degrees of freedom attributable to the regression;                               |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[2] |:math:MSR, the mean square attributable to the regression;                                      |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[3] |:math:F, the :math:F value for the analysis of variance;                                      |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[4] |:math:SSD, the sum of squares of deviations about the regression;                               |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[5] |:math:DFD, the degrees of freedom of deviations about the regression;                           |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[6] |:math:MSD, the mean square of deviations about the regression;                                  |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[7] |:math:SST, the total sum of squares;                                                            |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[8] |:math:DFT, the total degrees of freedom;                                                        |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[9] |:math:s, the standard error estimate;                                                           |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[10]|:math:R, the coefficient of multiple correlation;                                               |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[11]|:math:R^2, the coefficient of multiple determination;                                           |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[12]|:math:\bar{R}^2, the coefficient of multiple determination corrected for the degrees of freedom.|
+---------------------------+--------------------------------------------------------------------------------------------------+

**coef** : float, ndarray, shape :math:\left(k, 3\right)
For :math:i = 1,2,\ldots,k, the following information:

:math:\mathrm{coef}[i-1,0]

:math:b_i, the regression coefficient for the :math:i\ th variable.

:math:\mathrm{coef}[i-1,1]

:math:se\left(b_i\right), the standard error of the regression coefficient for the :math:i\ th variable.

:math:\mathrm{coef}[i-1,2]

:math:t\left(b_i\right), the :math:t value of the regression coefficient for the :math:i\ th variable.

**con** : float, ndarray, shape :math:\left(3\right)
The following information:

.. rst-class:: nag-rules-none nag-align-left

+-----------------------+-------------------------------------------------------------------------+
|:math:\mathrm{con}[0]|:math:a, the regression constant;                                      |
+-----------------------+-------------------------------------------------------------------------+
|:math:\mathrm{con}[1]|:math:se\left(a\right), the standard error of the regression constant; |
+-----------------------+-------------------------------------------------------------------------+
|:math:\mathrm{con}[2]|:math:t\left(a\right), the :math:t value for the regression constant.|
+-----------------------+-------------------------------------------------------------------------+

**rinv** : float, ndarray, shape :math:\left(k, k\right)
The inverse of the matrix of correlation coefficients for the independent variables; that is, the inverse of the matrix consisting of the first :math:k rows and columns of :math:\mathrm{r}.

**c** : float, ndarray, shape :math:\left(k, k\right)
The modified inverse matrix, where

:math:\mathrm{c}[\textit{i}-1,\textit{j}-1] = \mathrm{r}[\textit{i}-1,\textit{j}-1]\times \mathrm{rinv}[\textit{i}-1,\textit{j}-1]/\mathrm{ssp}[\textit{i}-1,\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,k, for :math:\textit{i} = 1,2,\ldots,k.

.. _g02cg-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:k = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:k \geq 1.

(errno :math:3)
On entry, :math:\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle and :math:k = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{n} > {k+1}.

(errno :math:5)
The :math:\textit{k} by :math:\textit{k} partition of :math:\mathrm{r} which requires inversion is not positive definite.

(errno :math:6)
The refinement following inversion has failed.

.. _g02cg-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

linregm_coeffs_const fits a curve of the form

.. math::
y = a+b_1x_1+b_2x_2 + \cdots +b_kx_k

to the data points

.. math::
\begin{array}{c}\left(x_{11}, x_{21}, \ldots, x_{{k1}}, y_1\right)\\\left(x_{12}, x_{22}, \ldots, x_{{k2}}, y_2\right)\\ \vdots \\\left(x_{{1n}}, x_{{2n}}, \ldots, x_{{kn}}, y_n\right)\end{array}

such that

.. math::
y_i = a+b_1x_{{1i}}+b_2x_{{2i}} + \cdots +b_kx_{{ki}}+e_i\text{, }\quad i = 1,2,\ldots,n\text{.}

The function calculates the regression coefficients, :math:b_1,b_2,\ldots,b_k, the regression constant, :math:a, and various other statistical quantities by minimizing

.. math::
\sum_{{i = 1}}^ne_i^2\text{.}

The actual data values :math:\left(x_{{1i}}, x_{{2i}}, \ldots, x_{{ki}}, y_i\right) are not provided as input to the function.

(i) The number of cases, :math:n, on which the regression is based.

(#) The total number of variables, dependent and independent, in the regression, :math:\left(k+1\right).

(#) The number of independent variables in the regression, :math:k.

(#) The means of all :math:k+1 variables in the regression, both the independent variables :math:\left(x_1, x_2, \ldots, x_k\right) and the dependent variable :math:\left(y\right), which is the :math:\left(k+1\right)\ th variable: i.e., :math:\bar{x}_1,\bar{x}_2,\ldots,\bar{x}_k,\bar{y}.

(#) The :math:\left(k+1\right)\times \left(k+1\right) matrix [:math:S_{{ij}}] of sums of squares and cross-products of deviations from means of all the variables in the regression; the terms involving the dependent variable, :math:y, appear in the :math:\left(k+1\right)\ th row and column.

(#) The :math:\left(k+1\right)\times \left(k+1\right) matrix [:math:R_{{ij}}] of the Pearson product-moment correlation coefficients for all the variables in the regression; the correlations involving the dependent variable, :math:y, appear in the :math:\left(k+1\right)\ th row and column.

The quantities calculated are:

(a) The inverse of the :math:k\times k partition of the matrix of correlation coefficients, [:math:R_{{ij}}], involving only the independent variables. The inverse is obtained using an accurate method which assumes that this sub-matrix is positive definite.

(#) The modified inverse matrix, :math:C = \left[c_{{ij}}\right], where

.. math::
c_{{ij}} = \frac{{R_{{ij}}r_{{ij}}}}{S_{{ij}}}\text{, }\quad i,j = 1,2,\ldots,k\text{,}

where :math:r_{{ij}} is the :math:\left(i, j\right)\ th element of the inverse matrix of [:math:R_{{ij}}] as described in \(a) above. Each element of :math:C is thus the corresponding element of the matrix of correlation coefficients multiplied by the corresponding element of the inverse of this matrix, divided by the corresponding element of the matrix of sums of squares and cross-products of deviations from means.

(#) The regression coefficients:

.. math::
b_i = \sum_{{j = i}}^kc_{{ij}}S_{{j\left(k+1\right)}}\text{, }\quad i = 1,2,\ldots,k\text{,}

where :math:S_{{j\left(k+1\right)}} is the sum of cross-products of deviations from means for the independent variable :math:x_j and the dependent variable :math:y.

(#) The sum of squares attributable to the regression, :math:SSR, the sum of squares of deviations about the regression, :math:SSD, and the total sum of squares, :math:SST:

:math:SST = S_{{\left(k+1\right)\left(k+1\right)}}, the sum of squares of deviations from the mean for the dependent variable, :math:y;

:math:SSR = \sum_{{j = 1}}^kb_jS_{{j\left(k+1\right)}}\text{; }\quad SSD = SST-SSR

(#) The degrees of freedom attributable to the regression, :math:DFR, the degrees of freedom of deviations about the regression, :math:DFD, and the total degrees of freedom, :math:DFT:

.. math::

(#) The mean square attributable to the regression, :math:MSR, and the mean square of deviations about the regression, :math:MSD:

.. math::
MSR = SSR/DFR\text{; }\quad MSD = SSD/DFD\text{.}

(#) The :math:F values for the analysis of variance:

.. math::
F = MSR/MSD\text{.}

(#) The standard error estimate:

.. math::
s = \sqrt{MSD}\text{.}

(#) The coefficient of multiple correlation, :math:R, the coefficient of multiple determination, :math:R^2 and the coefficient of multiple determination corrected for the degrees of freedom, :math:\bar{R}^2;

.. math::
R = \sqrt{1-\frac{{SSD}}{{SST}}}\text{; }\quad R^2 = 1-\frac{{SSD}}{{SST}}\text{; }\quad \bar{R}^2 = 1-\frac{{SSD\times DFT}}{{SST\times DFD}}\text{.}

(#) The standard error of the regression coefficients:

.. math::
se\left(b_i\right) = \sqrt{MSD\times c_{{ii}}}\text{, }\quad i = 1,2,\ldots,k\text{.}

(#) The :math:t values for the regression coefficients:

.. math::
t\left(b_i\right) = \frac{{b_i}}{{se\left(b_i\right)}}\text{, }\quad i = 1,2,\ldots,k\text{.}

(#) The regression constant, :math:a, its standard error, :math:se\left(a\right), and its :math:t value, :math:t\left(a\right):

.. math::
a = \bar{y}-\sum_{{i = 1}}^kb_i\bar{x}_i\text{; }\quad se\left(a\right) = \sqrt{MSD\times \left(\frac{1}{n}+\sum_{{i = 1}}^k\sum_{{j = 1}}^k\bar{x}_ic_{{ij}}\bar{x}_j\right)}\text{; }\quad t\left(a\right) = \frac{a}{{se\left(a\right)}}\text{.}

.. _g02cg-py2-py-references:

**References**
Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley
"""
raise NotImplementedError

[docs]def linregm_coeffs_noconst(n, sspz, rz):
r"""
linregm_coeffs_noconst performs a multiple linear regression with no constant on a set of variables whose sums of squares and cross-products about zero and correlation-like coefficients are given.

.. _g02ch-py2-py-doc:

For full information please refer to the NAG Library document for g02ch

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02chf.html

.. _g02ch-py2-py-parameters:

**Parameters**
**n** : int
:math:n, the number of cases used in calculating the sums of squares and cross-products and correlation-like coefficients.

**sspz** : float, array-like, shape :math:\left(k+1, k+1\right)
:math:\mathrm{sspz}[\textit{i}-1,\textit{j}-1] must be set to :math:\tilde{S}_{{\textit{i}\textit{j}}}, the sum of cross-products about zero for the :math:\textit{i}\ th and :math:\textit{j}\ th variables, for :math:\textit{j} = 1,2,\ldots,k+1, for :math:\textit{i} = 1,2,\ldots,k+1; terms involving the dependent variable appear in row :math:k+1 and column :math:k+1.

**rz** : float, array-like, shape :math:\left(k+1, k+1\right)
:math:\mathrm{rz}[\textit{i}-1,\textit{j}-1] must be set to :math:\tilde{R}_{{\textit{i}\textit{j}}}, the correlation-like coefficient for the :math:\textit{i}\ th and :math:\textit{j}\ th variables, for :math:\textit{j} = 1,2,\ldots,k+1, for :math:\textit{i} = 1,2,\ldots,k+1; coefficients involving the dependent variable appear in row :math:k+1 and column :math:k+1.

**Returns**
**result** : float, ndarray, shape :math:\left(13\right)
The following information:

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[0] |:math:SSR, the sum of squares attributable to the regression;                                   |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[1] |:math:DFR, the degrees of freedom attributable to the regression;                               |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[2] |:math:MSR, the mean square attributable to the regression;                                      |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[3] |:math:F, the :math:F value for the analysis of variance;                                      |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[4] |:math:SSD, the sum of squares of deviations about the regression;                               |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[5] |:math:DFD, the degrees of freedom of deviations about the regression;                           |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[6] |:math:MSD, the mean square of deviations about the regression;                                  |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[7] |:math:SST, the total sum of squares;                                                            |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[8] |:math:DFT, the total degrees of freedom;                                                        |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[9] |:math:s, the standard error estimate;                                                           |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[10]|:math:R, the coefficient of multiple correlation;                                               |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[11]|:math:R^2, the coefficient of multiple determination;                                           |
+---------------------------+--------------------------------------------------------------------------------------------------+
|:math:\mathrm{result}[12]|:math:\bar{R}^2, the coefficient of multiple determination corrected for the degrees of freedom.|
+---------------------------+--------------------------------------------------------------------------------------------------+

**coef** : float, ndarray, shape :math:\left(k, 3\right)
For :math:i = 1,2,\ldots,k, the following information:

:math:\mathrm{coef}[i-1,0]

:math:b_i, the regression coefficient for the :math:i\ th variable.

:math:\mathrm{coef}[i-1,1]

:math:se\left(b_i\right), the standard error of the regression coefficient for the :math:i\ th variable.

:math:\mathrm{coef}[i-1,2]

:math:t\left(b_i\right), the :math:t value of the regression coefficient for the :math:i\ th variable.

**rznv** : float, ndarray, shape :math:\left(k, k\right)
The inverse of the matrix of correlation-like coefficients for the independent variables; that is, the inverse of the matrix consisting of the first :math:k rows and columns of :math:\mathrm{rz}.

**cz** : float, ndarray, shape :math:\left(k, k\right)
The modified inverse matrix, :math:C, where

.. math::
\mathrm{cz}[i-1,j-1] = \frac{{\mathrm{rz}[i-1,j-1]\times \mathrm{rznv}[i-1,j-1]}}{{\mathrm{sspz}[i-1,j-1]}}\text{, }\quad i,j = 1,2,\ldots,k\text{.}

.. _g02ch-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:k = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:k \geq 1.

(errno :math:3)
On entry, :math:\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle and :math:k = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{n} \geq {k+1}.

(errno :math:5)
The :math:\textit{K}\times \textit{K} partition of :math:\mathrm{rz} which requires inversion is not positive definite.

(errno :math:6)
The refinement following the actual inversion has failed.

.. _g02ch-py2-py-notes:

**Notes**
No equivalent traditional C interface for this routine exists in the NAG Library.

linregm_coeffs_noconst fits a curve of the form

.. math::
y = b_1x_1+b_2x_2 + \cdots +b_kx_k

to the data points

.. math::
\begin{array}{c}\left(x_{11}, x_{21}, \ldots, x_{{k1}}, y_1\right)\\\left(x_{12}, x_{22}, \ldots, x_{{k2}}, y_2\right)\\ \vdots \\\left(x_{{1n}}, x_{{2n}}, \ldots, x_{{kn}}, y_n\right)\end{array}

such that

.. math::
y_i = b_1x_{{1i}}+b_2x_{{2i}} + \cdots +b_kx_{{ki}}+e_i\text{, }\quad i = 1,2,\ldots,n\text{.}

The function calculates the regression coefficients, :math:b_1,b_2,\ldots,b_k, (and various other statistical quantities) by minimizing

.. math::
\sum_{{i = 1}}^ne_i^2\text{.}

The actual data values :math:\left(x_{{1i}}, x_{{2i}}, \ldots, x_{{ki}}, y_i\right) are not provided as input to the function.
Instead, input to the function consists of:

(i) The number of cases, :math:n, on which the regression is based.

(#) The total number of variables, dependent and independent, in the regression, :math:\left(k+1\right).

(#) The number of independent variables in the regression, :math:k.

(#) The :math:\left(k+1\right)\times \left(k+1\right) matrix :math:\left[\tilde{S}_{{ij}}\right] of sums of squares and cross-products about zero of all the variables in the regression; the terms involving the dependent variable, :math:y, appear in the :math:\left(k+1\right)\ th row and column.

(#) The :math:\left(k+1\right)\times \left(k+1\right) matrix :math:\left[\tilde{R}_{{ij}}\right] of correlation-like coefficients for all the variables in the regression; the correlations involving the dependent variable, :math:y, appear in the :math:\left(k+1\right)\ th row and column.

The quantities calculated are:

(a) The inverse of the :math:k\times k partition of the matrix of correlation-like coefficients, :math:\left[\tilde{R}_{{ij}}\right], involving only the independent variables. The inverse is obtained using an accurate method which assumes that this sub-matrix is positive definite (see Further Comments <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02chf.html#fcomments>__).

(#) The modified matrix, :math:C = \left[c_{{ij}}\right], where

.. math::
c_{{ij}} = \frac{{\tilde{R}_{{ij}}\tilde{r}^{{ij}}}}{\tilde{S}_{{ij}}}\text{, }\quad i,j = 1,2,\ldots,k\text{,}

where :math:\tilde{r}^{{ij}} is the :math:\left(i, j\right)\ th element of the inverse matrix of :math:\left[\tilde{R}_{{ij}}\right] as described in \(a) above. Each element of :math:C is thus the corresponding element of the matrix of correlation-like coefficients multiplied by the corresponding element of the inverse of this matrix, divided by the corresponding element of the matrix of sums of squares and cross-products about zero.

(#) The regression coefficients:

.. math::
b_i = \sum_{{j = 1}}^kc_{{ij}}\tilde{S}_{{j\left(k+1\right)}}\text{, }\quad i = 1,2,\ldots,k\text{,}

where :math:\tilde{S}_{{j\left(k+1\right)}} is the sum of cross-products about zero for the independent variable :math:x_j and the dependent variable :math:y.

(#) The sum of squares attributable to the regression, :math:SSR, the sum of squares of deviations about the regression, :math:SSD, and the total sum of squares, :math:SST:

:math:SST = \tilde{S}_{{\left(k+1\right)\left(k+1\right)}}, the sum of squares about zero for the dependent variable, :math:y;

:math:SSR = \sum_{{j = 1}}^kb_j\tilde{S}_{{j\left(k+1\right)}}\text{; }\quad SSD = SST-SSR.

(#) The degrees of freedom attributable to the regression, :math:DFR, the degrees of freedom of deviations about the regression, :math:DFD, and the total degrees of freedom, :math:DFT:

.. math::

(#) The mean square attributable to the regression, :math:MSR, and the mean square of deviations about the regression, :math:MSD:

.. math::
MSR = SSR/DFR\text{; }\quad MSD = SSD/DFD\text{.}

(#) The :math:F value for the analysis of variance:

.. math::
F = MSR/MSD\text{.}

(#) The standard error estimate:

.. math::
s = \sqrt{MSD}\text{.}

(#) The coefficient of multiple correlation, :math:R, the coefficient of multiple determination, :math:R^2, and the coefficient of multiple determination corrected for the degrees of freedom, :math:\bar{R}^2:

.. math::
R = \sqrt{1-\frac{{SSD}}{{SST}}}\text{; }\quad R^2 = 1-\frac{{SSD}}{{SST}}\text{; }\quad \bar{R}^2 = 1-\frac{{SSD\times DFT}}{{SST\times DFD}}\text{.}

(#) The standard error of the regression coefficients:

.. math::
se\left(b_i\right) = \sqrt{MSD\times c_{{ii}}}\text{, }\quad i = 1,2,\ldots,k\text{.}

(#) The :math:t values for the regression coefficients:

.. math::
t\left(b_i\right) = \frac{{b_i}}{{se\left(b_i\right)}}\text{, }\quad i = 1,2,\ldots,k\text{.}

.. _g02ch-py2-py-references:

**References**
Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley
"""
raise NotImplementedError

[docs]def linregm_fit(x, isx, y, mean='M', wt=None, tol=0.000001):
r"""
linregm_fit performs a general multiple linear regression when the independent variables may be linearly dependent. Parameter estimates, standard errors, residuals and influence statistics are computed. linregm_fit may be used to perform a weighted regression.

.. _g02da-py2-py-doc:

For full information please refer to the NAG Library document for g02da

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02daf.html

.. _g02da-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation for the :math:\textit{j}\ th independent variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**isx** : int, array-like, shape :math:\left(m\right)
Indicates which independent variables are to be included in the model.

:math:\mathrm{isx}[j-1] > 0

The variable contained in the :math:j\ th column of :math:\mathrm{x} is included in the regression model.

**y** : float, array-like, shape :math:\left(n\right)
:math:y, the observations on the dependent variable.

**mean** : str, length 1, optional
Indicates if a mean term is to be included.

:math:\mathrm{mean} = \texttt{'M'}

A mean term, intercept, will be included in the model.

:math:\mathrm{mean} = \texttt{'Z'}

The model will pass through the origin, zero-point.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
If provided :math:\mathrm{wt} must contain the weights to be used with the model.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

The values of :math:\mathrm{res} and :math:\mathrm{h} will be set to zero for observations with zero weights.

If :math:\mathrm{wt} is not provided the effective number of observations is :math:n.

**tol** : float, optional
The value of :math:\mathrm{tol} is used to decide if the independent variables are of full rank and if not what is the rank of the independent variables. The smaller the value of :math:\mathrm{tol} the stricter the criterion for selecting the singular value decomposition. If :math:\mathrm{tol} = 0.0, the singular value decomposition will never be used; this may cause run time errors or inaccurate results if the independent variables are not of full rank.

**Returns**
The residual sum of squares for the regression.

**idf** : int
The degrees of freedom associated with the residual sum of squares.

**b** : float, ndarray, shape :math:\left(\textit{ip}\right)
:math:\mathrm{b}[i-1], :math:i = 1,2,\ldots,\textit{ip} contains the least squares estimates of the parameters of the regression model, :math:\hat{\beta }.

If :math:\mathrm{mean} = \texttt{'M'}, :math:\mathrm{b}[0] will contain the estimate of the mean parameter and :math:\mathrm{b}[i] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

If :math:\mathrm{mean} = \texttt{'Z'}, :math:\mathrm{b}[i-1] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

**se** : float, ndarray, shape :math:\left(\textit{ip}\right)
:math:\mathrm{se}[i-1], :math:i = 1,2,\ldots,\textit{ip} contains the standard errors of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}.

**cov** : float, ndarray, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The first :math:\textit{ip}\times \left(\textit{ip}+1\right)/2 elements of :math:\mathrm{cov} contain the upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored packed by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[j\times \left(j-1\right)/2+i-1].

**res** : float, ndarray, shape :math:\left(n\right)
The (weighted) residuals, :math:r_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**h** : float, ndarray, shape :math:\left(n\right)
The diagonal elements of :math:H, :math:h_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**q** : float, ndarray, shape :math:\left(n, \textit{ip}+1\right)
The results of the :math:QR decomposition:

the first column of :math:\mathrm{q} contains :math:c;

the upper triangular part of columns :math:2 to :math:\textit{ip}+1 contain the :math:R matrix;

the strictly lower triangular part of columns :math:2 to :math:\textit{ip}+1 contain details of the :math:Q matrix.

**svd** : bool
If a singular value decomposition has been performed then :math:\mathrm{svd} will be :math:\mathbf{True}, otherwise :math:\mathrm{svd} will be :math:\mathbf{False}.

**irank** : int
The rank of the independent variables.

If :math:\mathrm{svd} = \mathbf{False}, :math:\mathrm{irank} = \textit{ip}.

If :math:\mathrm{svd} = \mathbf{True}, :math:\mathrm{irank} is an estimate of the rank of the independent variables.

:math:\mathrm{irank} is calculated as the number of singular values greater that :math:\mathrm{tol}\times \text{} (largest singular value).

It is possible for the SVD to be carried out but :math:\mathrm{irank} to be returned as :math:\textit{ip}.

**p** : float, ndarray, shape :math:\left(2\times \textit{ip}+\textit{ip}\times \textit{ip}\right)
Details of the :math:QR decomposition and SVD if used.

If :math:\mathrm{svd} = \mathbf{False}, only the first :math:\textit{ip} elements of :math:\mathrm{p} are used these will contain the zeta values for the :math:QR decomposition (see :meth:lapackeig.dgeqrf <naginterfaces.library.lapackeig.dgeqrf> for details).

If :math:\mathrm{svd} = \mathbf{True}, the first :math:\textit{ip} elements of :math:\mathrm{p} will contain the zeta values for the :math:QR decomposition (see :meth:lapackeig.dgeqrf <naginterfaces.library.lapackeig.dgeqrf> for details) and the next :math:\textit{ip} elements of :math:\mathrm{p} contain singular values.

The following :math:\textit{ip} by :math:\textit{ip} elements contain the matrix :math:P^* stored by columns.

**wk** : float, ndarray, shape :math:\left(\max\left(2,{ 5 \times \left(\textit{ip}-1\right) + \textit{ip} \times \textit{ip} }\right)\right)
If on exit :math:\mathrm{svd} = \mathbf{True}, :math:\mathrm{wk} contains information which is needed by :meth:linregm_fit_newvar; otherwise :math:\mathrm{wk} is used as workspace.

.. _g02da-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:1)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol}\geq 0.0.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle and :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \leq n.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'U'} or :math:\texttt{'W'}.

(errno :math:2)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:3)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1]\geq 0.0, for :math:i = 1,2,\ldots,n.

(errno :math:4)
On entry, :math:\mathrm{isx}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.

Constraint: :math:\mathrm{isx}[i-1]\geq 0.0, for :math:i = 1,2,\ldots,m.

(errno :math:4)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} must be compatible with the number of nonzero elements in :math:\mathrm{isx}.

(errno :math:6)
SVD solution failed to converge.

**Warns**
**NagAlgorithmicWarning**
(errno :math:5)
The degrees of freedom for the residuals are zero, i.e., the designated number of arguments is equal to the effective number of observations. In this case the parameter estimates will be returned along with the diagonal elements of :math:H, but neither standard errors nor the variance-covariance matrix will be calculated.

.. _g02da-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

The general linear regression model is defined by

.. math::
y = X\beta +\epsilon \text{,}

where

:math:y is a vector of :math:n observations on the dependent variable,

:math:X is an :math:n\times p matrix of the independent variables of column rank :math:k,

:math:\beta is a vector of length :math:p of unknown parameters, and

:math:\epsilon is a vector of length :math:n of unknown random errors such that :math:\mathrm{var}\left(\epsilon \right) = V\sigma^2, where :math:V is a known diagonal matrix.

If :math:V = I, the identity matrix, then least squares estimation is used.
If :math:V\neq I, then for a given weight matrix :math:W\propto V^{-1}, weighted least squares estimation is used.

The least squares estimates :math:\hat{\beta } of the parameters :math:\beta minimize :math:\left(y-X\beta \right)^\mathrm{T}\left(y-X\beta \right) while the weighted least squares estimates minimize :math:\left(y-X\beta \right)^\mathrm{T}W\left(y-X\beta \right).

linregm_fit finds a :math:QR decomposition of :math:X (or :math:W^{{1/2}}X in weighted case), i.e.,

.. math::

where :math:R^* = \begin{pmatrix}R\\0\end{pmatrix} and :math:R is a :math:p\times p upper triangular matrix and :math:Q is an :math:n\times n orthogonal matrix.
If :math:R is of full rank, then :math:\hat{\beta } is the solution to

.. math::
R\hat{\beta } = c_1\text{,}

where :math:c = Q^\mathrm{T}y (or :math:Q^\mathrm{T}W^{{1/2}}y) and :math:c_1 is the first :math:p elements of :math:c.
If :math:R is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of :math:R,

.. math::
R = Q_*\begin{pmatrix}D&0\\0&0\end{pmatrix}P^\mathrm{T}\text{,}

where :math:D is a :math:k\times k diagonal matrix with nonzero diagonal elements, :math:k being the rank of :math:R, and :math:Q_* and :math:P are :math:p\times p orthogonal matrices.
This gives the solution

.. math::
\hat{\beta } = P_1D^{-1}Q_{*_1}^\mathrm{T}c_1\text{,}

:math:P_1 being the first :math:k columns of :math:P, i.e., :math:P = \begin{pmatrix}P_1&P_0\end{pmatrix}, and :math:Q_{*_1} being the first :math:k columns of :math:Q_*.

Details of the SVD, are made available, in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}\text{.}

This will be only one of the possible solutions.
Other estimates may be obtained by applying constraints to the parameters.
These solutions can be obtained by using :meth:linregm_constrain after using linregm_fit.
Only certain linear combinations of the parameters will have unique estimates; these are known as estimable functions.

The fit of the model can be examined by considering the residuals, :math:r_i = y_i-\hat{y}, where :math:\hat{y} = X\hat{\beta } are the fitted values.
The fitted values can be written as :math:Hy for an :math:n\times n matrix :math:H.
The :math:i\ th diagonal elements of :math:H, :math:h_i, give a measure of the influence of the :math:i\ th values of the independent variables on the fitted regression model.
The values :math:h_i are sometimes known as leverages.
Both :math:r_i and :math:h_i are provided by linregm_fit.

The output of linregm_fit also includes :math:\hat{\beta }, the residual sum of squares and associated degrees of freedom, :math:\left(n-k\right), the standard errors of the parameter estimates and the variance-covariance matrix of the parameter estimates.

In many linear regression models the first term is taken as a mean term or an intercept, i.e., :math:X_{{i,1}} = 1, for :math:i = 1,2,\ldots,n.
This is provided as an option.
Also only some of the possible independent variables are required to be included in a model, a facility to select variables to be included in the model is provided.

Details of the :math:QR decomposition and, if used, the SVD, are made available.
These allow the regression to be updated by adding or deleting an observation using :meth:linregm_obs_edit, adding or deleting a variable using :meth:linregm_var_add and :meth:linregm_var_del or estimating and testing an estimable function using :meth:linregm_estfunc.

.. _g02da-py2-py-references:

**References**
Cook, R D and Weisberg, S, 1982, Residuals and Influence in Regression, Chapman and Hall

Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley

Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2--25

McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall

Searle, S R, 1971, Linear Models, Wiley

--------
:meth:naginterfaces.library.examples.correg.linregm_fit_ex.main
"""
raise NotImplementedError

[docs]def linregm_obs_edit(update, isx, q, x, ix, y, rss, mean='M', wt=None):
r"""
linregm_obs_edit adds or deletes an observation from a general regression model fitted by :meth:linregm_fit.

.. _g02dc-py2-py-doc:

For full information please refer to the NAG Library document for g02dc

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02dcf.html

.. _g02dc-py2-py-parameters:

**Parameters**
**update** : str, length 1
Indicates if an observation is to be added or deleted.

:math:\mathrm{update} = \texttt{'A'}

:math:\mathrm{update} = \texttt{'D'}

The observation is deleted.

**isx** : int, array-like, shape :math:\left(m\right)
If :math:\mathrm{isx}[\textit{j}-1] is greater than :math:0, the value contained in :math:\mathrm{x}[\left(\textit{j}-1\right)\times \mathrm{ix}] is to be included as a value of :math:x^\mathrm{T}, for :math:\textit{j} = 1,2,\ldots,m.

**q** : float, array-like, shape :math:\left(\textit{ip}, \textit{ip}+1\right)
Must be array :math:\mathrm{q} as output by :meth:linregm_fit, :meth:linregm_var_add, :meth:linregm_var_del or :meth:linregm_fit_onestep, or a previous call to linregm_obs_edit.

**x** : float, array-like, shape :math:\left(\left(m-1\right)\times \mathrm{ix}+1\right)
The :math:\textit{ip} values for the dependent variables of the new observation, :math:x^\mathrm{T}. The positions will depend on the value of :math:\mathrm{ix}.

**ix** : int
The increment for elements of :math:\mathrm{x}. Two situations are common:

:math:\mathrm{ix} = 1

The values of :math:x are to be chosen from consecutive locations in :math:\mathrm{x}, i.e., :math:\mathrm{x}[0],\mathrm{x}[1],\ldots,\mathrm{x}[m-1].

:math:\mathrm{ix} = {\textit{ldx}}

The values of :math:x are to be chosen from a row of a two-dimensional array with first dimension :math:\textit{ldx}, i.e., :math:\mathrm{x}[0],\mathrm{x}[{\textit{ldx}}],\ldots,\mathrm{x}[\left(m-1\right){\textit{ldx}}].

**y** : float
The value of the dependent variable for the new observation, :math:y_{\text{new}}.

The value of the residual sums of squares for the original set of observations.

**mean** : str, length 1, optional
Indicates if a mean has been used in the model.

:math:\mathrm{mean} = \texttt{'M'}

A mean term or intercept will have been included in the model by :meth:linregm_fit.

:math:\mathrm{mean} = \texttt{'Z'}

A model with no mean term or intercept will have been fitted by :meth:linregm_fit.

**wt** : None or float, optional
If provided, :math:\mathrm{wt} must contain the weight to be used with the new observation.

If :math:\mathrm{wt} is **None**, the observation is not included in the model.

**Returns**
**q** : float, ndarray, shape :math:\left(\textit{ip}, \textit{ip}+1\right)
The first :math:\textit{ip} elements of the first column of :math:\mathrm{q} will contain :math:c_1^* the upper triangular part of columns :math:2 to :math:\textit{ip}+1 will contain :math:R^* the remainder is unchanged.

The updated values of the residual sums of squares.

**Note:** this will only be valid if the model is of full rank.

.. _g02dc-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\langle\mathit{\boldsymbol{value}}\rangle elements of :math:\mathrm{isx} > 0 instead of :math:\textit{ip}-1 (for mean) :math:\text{} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:1)
On entry, :math:\mathrm{ix} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{ix} \geq 1.

(errno :math:1)
On entry, :math:\langle\mathit{\boldsymbol{value}}\rangle elements of :math:\mathrm{isx} > 0 instead of :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:1)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:1)
On entry, :math:\mathrm{rss} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{rss}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{update} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{update} = \texttt{'A'} or :math:\texttt{'D'}.

(errno :math:1)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'U'} or :math:\texttt{'W'}.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:2)
On entry, :math:\mathrm{wt} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{wt}\geq 0.0.

(errno :math:3)
The :math:R matrix could not be updated.

(errno :math:4)
The residual sums of squares cannot be updated.

.. _g02dc-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

:meth:linregm_fit fits a general linear regression model to a dataset.
You may wish to change the model by either adding or deleting an observation from the dataset. linregm_obs_edit takes the results from :meth:linregm_fit and makes the required changes to the vector :math:c and the upper triangular matrix :math:R produced by :meth:linregm_fit.
The regression coefficients, standard errors and the variance-covariance matrix of the regression coefficients can be obtained from :meth:linregm_update after all required changes to the dataset have been made.

:meth:linregm_fit performs a :math:QR decomposition on the (weighted) :math:X matrix of independent variables.
To add a new observation to a model with :math:p parameters, the upper triangular matrix :math:R and vector :math:c_1 (the first :math:p elements of :math:c) are augmented by the new observation on independent variables in :math:x^\mathrm{T} and dependent variable :math:y_{\text{new}}.
Givens rotations are then used to restore the upper triangular form.

.. math::
\begin{pmatrix}R:c_1\\x:y_{\text{new}}\end{pmatrix}→\begin{pmatrix}R^*:c_1^*\\0:y_{\text{new}}^*\end{pmatrix}\text{.}

**Note:** only :math:R and the upper part of :math:c are updated the remainder of the :math:Q matrix is unchanged.

.. _g02dc-py2-py-references:

**References**
Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2--25
"""
raise NotImplementedError

r"""
linregm_update calculates the regression parameters for a general linear regression model.
It is intended to be called after :meth:linregm_obs_edit, :meth:linregm_var_add or :meth:linregm_var_del.

.. _g02dd-py2-py-doc:

For full information please refer to the NAG Library document for g02dd

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02ddf.html

.. _g02dd-py2-py-parameters:

**Parameters**
**q** : float, array-like, shape :math:\left(:, \textit{ip}+1\right)
Note: the required extent for this argument in dimension 1 is determined as follows: if :math:\mathrm{rss}\leq 0.0: :math:n; otherwise: :math:\textit{ip}.

Must be the array :math:\mathrm{q} as output by :meth:linregm_obs_edit, :meth:linregm_var_add, :meth:linregm_var_del or :meth:linregm_fit_onestep. If on entry :math:\mathrm{rss}\leq 0.0 then all :math:\textit{n} elements of :math:c are needed. This is provided by functions :meth:linregm_var_add, :meth:linregm_var_del or :meth:linregm_fit_onestep.

Either the residual sum of squares or a value less than or equal to :math:0.0 to indicate that the residual sum of squares is to be calculated by the function.

**p** : float, array-like, shape :math:\left(\textit{ip}\times \textit{ip}+2\times \textit{ip}\right)
Must be array :math:\mathrm{p} as output by :meth:linregm_fit, :meth:linregm_var_add or :meth:linregm_fit_onestep, or a previous call to linregm_update.

**tol** : float, optional
The value of :math:\mathrm{tol} is used to decide if the independent variables are of full rank and, if not, what is the rank of the independent variables. The smaller the value of :math:\mathrm{tol} the stricter the criterion for selecting the singular value decomposition. If :math:\mathrm{tol} = 0.0, the singular value decomposition will never be used, this may cause run time errors or inaccuracies if the independent variables are not of full rank.

**Returns**
If :math:\mathrm{rss}\leq 0.0 on entry, then on exit :math:\mathrm{rss} will contain the residual sum of squares as calculated by linregm_update.

If :math:\mathrm{rss} was positive on entry, it will be unchanged.

**idf** : int
The degrees of freedom associated with the residual sum of squares.

**b** : float, ndarray, shape :math:\left(\textit{ip}\right)
The estimates of the :math:p parameters, :math:\hat{\beta }.

**se** : float, ndarray, shape :math:\left(\textit{ip}\right)
The standard errors of the :math:p parameters given in :math:\mathrm{b}.

**cov** : float, ndarray, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:p parameter estimates given in :math:\mathrm{b}. They are stored packed by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[j\times \left(j-1\right)/2+i-1].

**svd** : bool
If a singular value decomposition has been performed, :math:\mathrm{svd} = \mathbf{True}, otherwise :math:\mathrm{svd} = \mathbf{False}.

**irank** : int
The rank of the independent variables.

If :math:\mathrm{svd} = \mathbf{False}, :math:\mathrm{irank} = \textit{ip}.

If :math:\mathrm{svd} = \mathbf{True}, :math:\mathrm{irank} is an estimate of the rank of the independent variables.

:math:\mathrm{irank} is calculated as the number of singular values greater than :math:\mathrm{tol}\times \text{} (largest singular value).

It is possible for the SVD to be carried out but :math:\mathrm{irank} to be returned as :math:\textit{ip}.

**p** : float, ndarray, shape :math:\left(\textit{ip}\times \textit{ip}+2\times \textit{ip}\right)
Contains details of the singular value decomposition if used.

If :math:\mathrm{svd} = \mathbf{False}, :math:\mathrm{p} is not referenced.

If :math:\mathrm{svd} = \mathbf{True}, the first :math:\textit{ip} elements of :math:\mathrm{p} are unchanged, the next :math:\textit{ip} values contain the singular values.

The following :math:\textit{ip}\times \textit{ip} values contain the matrix :math:P^* stored by columns.

.. _g02dd-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol}\geq 0.0.

(errno :math:1)
On entry, :math:\textit{ldq} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: if :math:\mathrm{rss}\leq 0.0, :math:\textit{ldq} \geq n.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 1.

(errno :math:2)
The degrees of freedom for error are less than or equal to :math:0. In this case the estimates of :math:\beta are returned but not the standard errors or covariances.

(errno :math:3)
SVD solution failed to converge.

.. _g02dd-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

A general linear regression model fitted by :meth:linregm_fit may be adjusted by adding or deleting an observation using :meth:linregm_obs_edit, adding a new independent variable using :meth:linregm_var_add or deleting an existing independent variable using :meth:linregm_var_del.
Alternatively a model may be constructed by a forward selection procedure using :meth:linregm_fit_onestep.
These functions compute the vector :math:c and the upper triangular matrix :math:R. linregm_update takes these basic results and computes the regression coefficients, :math:\hat{\beta }, their standard errors and their variance-covariance matrix.

If :math:R is of full rank, then :math:\hat{\beta } is the solution to

.. math::
R\hat{\beta } = c_1\text{,}

where :math:c_1 is the first :math:p elements of :math:c.

If :math:R is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of :math:R,

.. math::
R = Q_*\begin{pmatrix}D&0\\0&0\end{pmatrix}P^\mathrm{T}\text{,}

where :math:D is a :math:k\times k diagonal matrix with nonzero diagonal elements, :math:k being the rank of :math:R, and :math:Q_* and :math:P are :math:p\times p orthogonal matrices.
This gives the solution

.. math::
\hat{\beta } = P_1D^{-1}Q_{*_1}^\mathrm{T}c_1\text{.}

:math:P_1 being the first :math:k columns of :math:P, i.e., :math:P = \left(P_1P_0\right), and :math:Q_{*_1} being the first :math:k columns of :math:Q_*.

Details of the SVD are made available in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}\text{.}

This will be only one of the possible solutions.
Other estimates may be obtained by applying constraints to the parameters.
These solutions can be obtained by calling :meth:linregm_constrain after calling linregm_update.
Only certain linear combinations of the parameters will have unique estimates; these are known as estimable functions.
These can be estimated using :meth:linregm_estfunc.

The residual sum of squares required to calculate the standard errors and the variance-covariance matrix can either be input or can be calculated if additional information on :math:c for the whole sample is provided.

.. _g02dd-py2-py-references:

**References**
Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2--25

Searle, S R, 1971, Linear Models, Wiley
"""
raise NotImplementedError

[docs]def linregm_var_add(q, p, x, wt=None, tol=0.000001):
r"""
linregm_var_add adds a new independent variable to a general linear regression model.

.. _g02de-py2-py-doc:

For full information please refer to the NAG Library document for g02de

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02def.html

.. _g02de-py2-py-parameters:

**Parameters**
**q** : float, array-like, shape :math:\left(n, \textit{ip}+2\right)
If :math:\textit{ip}\neq 0, :math:\mathrm{q} must contain the results of the :math:QR decomposition for the model with :math:p parameters as returned by :meth:linregm_fit or a previous call to linregm_var_add.

If :math:\textit{ip} = 0, the first column of :math:\mathrm{q} should contain the :math:n values of the dependent variable, :math:y.

**p** : float, array-like, shape :math:\left(\textit{ip}+1\right)
Contains further details of the :math:QR decomposition used. The first :math:\textit{ip} elements of :math:\mathrm{p} must contain the zeta values for the :math:QR decomposition (see :meth:lapackeig.dgeqrf <naginterfaces.library.lapackeig.dgeqrf> for details).

The first :math:\textit{ip} elements of array :math:\mathrm{p} are provided by :meth:linregm_fit or by previous calls to linregm_var_add.

**x** : float, array-like, shape :math:\left(n\right)
:math:x, the new independent variable.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
If provided :math:\mathrm{wt} must contain the weights to be used with the model.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If :math:\mathrm{wt} is not provided the effective number of observations is :math:n.

**tol** : float, optional
The value of :math:\mathrm{tol} is used to decide if the new independent variable is linearly related to independent variables already included in the model. If the new variable is linearly related then :math:c is not updated. The smaller the value of :math:\mathrm{tol} the stricter the criterion for deciding if there is a linear relationship.

**Returns**
**q** : float, ndarray, shape :math:\left(n, \textit{ip}+2\right)
The results of the :math:QR decomposition for the model with :math:p+1 parameters:

the first column of :math:\mathrm{q} contains the updated value of :math:c;

the columns :math:2 to :math:\textit{ip}+1 are unchanged;

the first :math:\textit{ip}+1 elements of column :math:\textit{ip}+2 contain the new column of :math:R, while the remaining :math:n-\textit{ip}-1 elements contain details of the matrix :math:Q_{{p+1}}.

**p** : float, ndarray, shape :math:\left(\textit{ip}+1\right)
The first :math:\textit{ip} elements of :math:\mathrm{p} are unchanged and the :math:\left(\textit{ip}+1\right)\ th element contains the zeta value for :math:Q_{{p+1}}.

The residual sum of squares for the new fitted model.

**Note:** this will only be valid if the model is of full rank, see Further Comments <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02def.html#fcomments>__.

.. _g02de-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol} > 0.0.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle and :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} < n.

(errno :math:1)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'W'} or :math:\texttt{'U'}.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 0.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 1.

(errno :math:2)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1]\geq 0.0, for :math:i = 1,2,\ldots,n.

**Warns**
**NagAlgorithmicWarning**
(errno :math:3)
:math:\mathrm{x} variable is a linear combination of existing model terms.

.. _g02de-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

A linear regression model may be built up by adding new independent variables to an existing model. linregm_var_add updates the :math:QR decomposition used in the computation of the linear regression model.
The :math:QR decomposition may come from :meth:linregm_fit or a previous call to linregm_var_add.
The general linear regression model is defined by

.. math::
y = X\beta +\epsilon \text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|where|:math:y is a vector of :math:n observations on the dependent variable,                                                                                                             |
+-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     |:math:X is an :math:n\times p matrix of the independent variables of column rank :math:k,                                                                                        |
+-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     |:math:\beta is a vector of length :math:p of unknown parameters,                                                                                                                   |
+-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|and  |:math:\epsilon is a vector of length :math:n of unknown random errors such that :math:\mathrm{var}\left(\epsilon \right) = V\sigma^2, where :math:V is a known diagonal matrix.|
+-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

If :math:V = I, the identity matrix, then least squares estimation is used.
If :math:V\neq I, then for a given weight matrix :math:W\propto V^{-1}, weighted least squares estimation is used.

The least squares estimates, :math:\hat{\beta } of the parameters :math:\beta minimize :math:\left(y-X\beta \right)^\mathrm{T}\left(y-X\beta \right) while the weighted least squares estimates, minimize :math:\left(y-X\beta \right)^\mathrm{T}W\left(y-X\beta \right).

The parameter estimates may be found by computing a :math:QR decomposition of :math:X (or :math:W^{{\frac{1}{2}}}X in the weighted case), i.e.,

.. math::

where :math:R^* = \begin{pmatrix}R\\0\end{pmatrix} and :math:R is a :math:p\times p upper triangular matrix and :math:Q is an :math:n\times n orthogonal matrix.

If :math:R is of full rank, then :math:\hat{\beta } is the solution to

.. math::
R\hat{\beta } = c_1\text{,}

where :math:c = Q^\mathrm{T}y (or :math:Q^\mathrm{T}W^{{\frac{1}{2}}}y) and :math:c_1 is the first :math:p elements of :math:c.

If :math:R is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of :math:R.

To add a new independent variable, :math:x_{{p+1}}, :math:R and :math:c have to be updated.
The matrix :math:Q_{{p+1}} is found such that :math:Q_{{p+1}}^\mathrm{T}\left[R:Q^\mathrm{T}x_{{p+1}}\right] (or :math:Q_{{p+1}}^\mathrm{T}\left[R:Q^\mathrm{T}W^{{\frac{1}{2}}}x_{{p+1}}\right]) is upper triangular.
The vector :math:c is then updated by multiplying by :math:Q_{{p+1}}^\mathrm{T}.

The new independent variable is tested to see if it is linearly related to the existing independent variables by checking that at least one of the values :math:\left(Q^\mathrm{T}x_{{p+1}}\right)_{\textit{i}}, for :math:\textit{i} = p+2,\ldots,n, is nonzero.

The new parameter estimates, :math:\hat{\beta }, can then be obtained by a call to :meth:linregm_update.

The function can be used with :math:p = 0, in which case :math:R and :math:c are initialized.

.. _g02de-py2-py-references:

**References**
Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley

Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2--25

McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall

Searle, S R, 1971, Linear Models, Wiley
"""
raise NotImplementedError

r"""
linregm_var_del deletes an independent variable from a general linear regression model.

.. _g02df-py2-py-doc:

For full information please refer to the NAG Library document for g02df

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02dff.html

.. _g02df-py2-py-parameters:

**Parameters**
**q** : float, array-like, shape :math:\left(\textit{ip}, \textit{ip}+1\right)
The results of the :math:QR decomposition as returned by functions :meth:linregm_fit, :meth:linregm_obs_edit, :meth:linregm_var_add or :meth:linregm_fit_onestep, or previous calls to linregm_var_del.

**indx** : int
Indicates which independent variable is to be deleted from the model.

The residual sum of squares for the full regression.

**Returns**
**q** : float, ndarray, shape :math:\left(\textit{ip}, \textit{ip}+1\right)
The updated :math:QR decomposition.

The residual sum of squares with the (:math:\mathrm{indx})th variable removed. Note that the residual sum of squares will only be valid if the regression is of full rank, otherwise the residual sum of squares should be obtained using :meth:linregm_update.

.. _g02df-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{rss} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{rss}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{indx} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{indx} \geq 1 and :math:\mathrm{indx} \leq \textit{ip}.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:2)
On entry, :math:\mathrm{q}[\langle\mathit{\boldsymbol{value}}\rangle,\langle\mathit{\boldsymbol{value}}\rangle] = 0.0.

.. _g02df-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

When selecting a linear regression model it is sometimes useful to drop independent variables from the model and to examine the resulting sub-model. linregm_var_del updates the :math:QR decomposition used in the computation of the linear regression model.
The :math:QR decomposition may come from :meth:linregm_fit or :meth:linregm_var_add, or a previous call to linregm_var_del.

For the general linear regression model with :math:p independent variables fitted :meth:linregm_fit or :meth:linregm_var_add compute a :math:QR decomposition of the (weighted) independent variables and form an upper triangular matrix :math:R and a vector :math:c.
To remove an independent variable :math:R and :math:c have to be updated.
The column of :math:R corresponding to the variable to be dropped is removed and the matrix is then restored to upper triangular form by applying a series of Givens rotations.
The rotations are then applied to :math:c.
Note only the first :math:p elements of :math:c are affected.

The method used means that while the updated values of :math:R and :math:c are computed an updated value of :math:Q from the :math:QR decomposition is not available so a call to :meth:linregm_var_add cannot be made after a call to linregm_var_del.

:meth:linregm_update can be used to calculate the parameter estimates, :math:\hat{\beta }, from the information provided by linregm_var_del.

.. _g02df-py2-py-references:

**References**
Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2--25
"""
raise NotImplementedError

[docs]def linregm_fit_newvar(rss, irank, cov, q, svd, p, y, wk, wt=None):
r"""
linregm_fit_newvar calculates the estimates of the parameters of a general linear regression model for a new dependent variable after a call to :meth:linregm_fit.

.. _g02dg-py2-py-doc:

For full information please refer to the NAG Library document for g02dg

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02dgf.html

.. _g02dg-py2-py-parameters:

**Parameters**
The residual sum of squares for the original dependent variable.

**irank** : int
The rank of the independent variables, as given by :meth:linregm_fit.

**cov** : float, array-like, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The covariance matrix of the parameter estimates as given by :meth:linregm_fit.

**q** : float, array-like, shape :math:\left(n, \textit{ip}+1\right)
The results of the :math:QR decomposition as returned by :meth:linregm_fit.

**svd** : bool
Indicates if a singular value decomposition was used by :meth:linregm_fit.

:math:\mathrm{svd} = \mathbf{True}

A singular value decomposition was used by :meth:linregm_fit.

:math:\mathrm{svd} = \mathbf{False}

A singular value decomposition was not used by :meth:linregm_fit.

**p** : float, array-like, shape :math:\left(:\right)
Note: the required length for this argument is determined as follows: if :math:\mathrm{not}\left(\mathrm{svd}\right): :math:\textit{ip}; otherwise: :math:{ \textit{ip} \times \textit{ip} + 2 \times \textit{ip} }.

Details of the :math:QR decomposition and SVD, if used, as returned in array :math:\mathrm{p} by :meth:linregm_fit.

If :math:\mathrm{svd} = \mathbf{False}, only the first :math:\textit{ip} elements of :math:\mathrm{p} are used; these contain the zeta values for the :math:QR decomposition (see :meth:lapackeig.dgeqrf <naginterfaces.library.lapackeig.dgeqrf> for details).

If :math:\mathrm{svd} = \mathbf{True}, the first :math:\textit{ip} elements of :math:\mathrm{p} contain the zeta values for the :math:QR decomposition (see :meth:lapackeig.dgeqrf <naginterfaces.library.lapackeig.dgeqrf> for details) and the next :math:\textit{ip}\times \textit{ip}+\textit{ip} elements of :math:\mathrm{p} contain details of the singular value decomposition.

**y** : float, array-like, shape :math:\left(n\right)
The new dependent variable, :math:y_{\text{new}}.

**wk** : float, array-like, shape :math:\left(5\times \left(\textit{ip}-1\right)+\textit{ip}\times \textit{ip}\right)
If :math:\mathrm{svd} = \mathbf{True}, :math:\mathrm{wk} must be unaltered from the previous call to :meth:linregm_fit or linregm_fit_newvar.

If :math:\mathrm{svd} = \mathbf{False}, :math:\mathrm{wk} is used as workspace.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
If provided :math:\mathrm{wt} must contain the weights to be used with the model.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If :math:\mathrm{wt} is not provided the effective number of observations is :math:n.

**Returns**
The residual sum of squares for the new dependent variable.

**cov** : float, ndarray, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored packed by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1].

**q** : float, ndarray, shape :math:\left(n, \textit{ip}+1\right)
The first column of :math:\mathrm{q} contains the new values of :math:c, the remainder of :math:\mathrm{q} will be unchanged.

**b** : float, ndarray, shape :math:\left(\textit{ip}\right)
The least squares estimates of the parameters of the regression model, :math:\hat{\beta }.

**se** : float, ndarray, shape :math:\left(\textit{ip}\right)
The standard error of the estimates of the parameters.

**res** : float, ndarray, shape :math:\left(n\right)
The residuals for the new regression model.

.. _g02dg-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{irank} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{irank} > 0.

(errno :math:1)
On entry, :math:\mathrm{irank} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: if :math:\mathrm{svd} = \mathbf{False}, :math:\mathrm{irank} = \textit{ip}.

(errno :math:1)
On entry, :math:\mathrm{irank} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: if :math:\mathrm{svd} = \mathbf{True}, :math:\mathrm{irank} \leq \textit{ip}.

(errno :math:1)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'W'} or :math:\texttt{'U'}.

(errno :math:1)
On entry, :math:\mathrm{rss} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{rss} > 0.0.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq \textit{ip}.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:2)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1]\geq 0.0, for :math:i = 1,2,\ldots,n.

.. _g02dg-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

linregm_fit_newvar uses the results given by :meth:linregm_fit to fit the same set of independent variables to a new dependent variable.

:meth:linregm_fit computes a :math:QR decomposition of the matrix of :math:p independent variables and also, if the model is not of full rank, a singular value decomposition (SVD).
These results can be used to compute estimates of the parameters for a general linear model with a new dependent variable.
The :math:QR decomposition leads to the formation of an upper triangular :math:p\times p matrix :math:R and an :math:n\times n orthogonal matrix :math:Q.
In addition the vector :math:c = Q^\mathrm{T}y (or :math:Q^\mathrm{T}W^{{1/2}}y) is computed.
For a new dependent variable, :math:y_{\mathrm{new}}, linregm_fit_newvar computes a new value of :math:c = Q^\mathrm{T}y_{\text{new}} or :math:Q^\mathrm{T}W^{{1/2}}y_{\text{new}}.

If :math:R is of full rank, then the least squares parameter estimates, :math:\hat{\beta }, are the solution to

.. math::
R\hat{\beta } = c_1\text{,}

where :math:c_1 is the first :math:p elements of :math:c.

If :math:R is not of full rank, then :meth:linregm_fit will have computed an SVD of :math:R,

.. math::
R = Q_*\begin{pmatrix}D&0\\0&0\end{pmatrix}P^\mathrm{T}\text{,}

where :math:D is a :math:k\times k diagonal matrix with nonzero diagonal elements, :math:k being the rank of :math:R, and :math:Q_* and :math:P are :math:p\times p orthogonal matrices.
This gives the solution

.. math::
\hat{\beta } = P_1D^{-1}Q_{*_1}^\mathrm{T}c_1\text{,}

:math:P_1 being the first :math:k columns of :math:P, i.e., :math:P = \left(P_1P_0\right), and :math:Q_{*_1} being the first :math:k columns of :math:Q_*.
Details of the SVD are made available by :meth:linregm_fit in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}\text{.}

The matrix :math:Q_* is made available through the workspace of :meth:linregm_fit.

In addition to parameter estimates, the new residuals are computed and the variance-covariance matrix of the parameter estimates are found by scaling the variance-covariance matrix for the original regression.

.. _g02dg-py2-py-references:

**References**
Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2--25

Searle, S R, 1971, Linear Models, Wiley
"""
raise NotImplementedError

[docs]def linregm_constrain(p, c, b, rss, idf):
r"""
linregm_constrain calculates the estimates of the parameters of a general linear regression model for given constraints from the singular value decomposition results.

.. _g02dk-py2-py-doc:

For full information please refer to the NAG Library document for g02dk

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02dkf.html

.. _g02dk-py2-py-parameters:

**Parameters**
**p** : float, array-like, shape :math:\left(\textit{ip}\times \textit{ip}+2\times \textit{ip}\right)
As returned by :meth:linregm_fit and :meth:linregm_update.

**c** : float, array-like, shape :math:\left(\textit{ip}, \textit{iconst}\right)
The :math:\textit{iconst} constraints stored by column, i.e., the :math:i\ th constraint is stored in the :math:i\ th column of :math:\mathrm{c}.

**b** : float, array-like, shape :math:\left(\textit{ip}\right)
The parameter estimates computed by using the singular value decomposition, :math:\hat{\beta }_{\text{svd}}.

The residual sum of squares as returned by :meth:linregm_fit or :meth:linregm_update.

**idf** : int
The degrees of freedom associated with the residual sum of squares as returned by :meth:linregm_fit or :meth:linregm_update.

**Returns**
**b** : float, ndarray, shape :math:\left(\textit{ip}\right)
The parameter estimates of the parameters with the constraints imposed, :math:\hat{\beta }_{\mathrm{c}}.

**se** : float, ndarray, shape :math:\left(\textit{ip}\right)
The standard error of the parameter estimates in :math:\mathrm{b}.

**cov** : float, ndarray, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored packed by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1].

.. _g02dk-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{idf} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{idf} > 0.

(errno :math:1)
On entry, :math:\textit{iconst} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{iconst} > 0.

(errno :math:1)
On entry, :math:\mathrm{rss} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{rss} > 0.0.

(errno :math:1)
On entry, :math:\textit{iconst} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{iconst} < \textit{ip}.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:2)
:math:\mathrm{c} does not give a model of full rank.

.. _g02dk-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

linregm_constrain computes the estimates given a set of linear constraints for a general linear regression model which is not of full rank.
It is intended for use after a call to :meth:linregm_fit or :meth:linregm_update.

In the case of a model not of full rank the functions use a singular value decomposition (SVD) to find the parameter estimates, :math:\hat{\beta }_{\text{svd}}, and their variance-covariance matrix.
Details of the SVD are made available in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}\text{,}

as described by :meth:linregm_fit and :meth:linregm_update.

Alternative solutions can be formed by imposing constraints on the parameters.
If there are :math:p parameters and the rank of the model is :math:k, then :math:n_c = p-k constraints will have to be imposed to obtain a unique solution.

Let :math:C be a :math:p\times n_c matrix of constraints, such that

.. math::
C^{\mathrm{T}}\beta = 0

then the new parameter estimates :math:\hat{\beta }_c are given by

.. math::
\begin{array}{cc}\hat{\beta }_c& = A\hat{\beta }_{\mathrm{svd}}\text{;}\\\\& = \left(I-P_0\left(C^\mathrm{T}P_0\right)^{-1}\right)\hat{\beta }_{\mathrm{svd}}\text{,}\end{array}

where :math:I is the identity matrix, and the variance-covariance matrix is given by

.. math::
AP_1D^{-2}P_1^{\mathrm{T}}A^{\mathrm{T}}\text{,}

provided :math:\left(C^{\mathrm{T}}P_0\right)^{-1} exists.

.. _g02dk-py2-py-references:

**References**
Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2--25

Searle, S R, 1971, Linear Models, Wiley
"""
raise NotImplementedError

[docs]def linregm_estfunc(irank, b, cov, p, f, tol=0.0):
r"""
linregm_estfunc gives the estimate of an estimable function along with its standard error.

.. _g02dn-py2-py-doc:

For full information please refer to the NAG Library document for g02dn

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02dnf.html

.. _g02dn-py2-py-parameters:

**Parameters**
**irank** : int
:math:k, the rank of the independent variables.

**b** : float, array-like, shape :math:\left(\textit{ip}\right)
The :math:\textit{ip} values of the estimates of the parameters of the model, :math:\hat{\beta }.

**cov** : float, array-like, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored packed by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1].

**p** : float, array-like, shape :math:\left(\textit{ip}\times \textit{ip}+2\times \textit{ip}\right)
As returned by :meth:linregm_fit and :meth:linregm_update.

**f** : float, array-like, shape :math:\left(\textit{ip}\right)
:math:f, the linear function to be estimated.

**tol** : float, optional
:math:\eta, the tolerance value used in the check for estimability.

If :math:\mathrm{tol}\leq 0.0 then :math:\sqrt{\epsilon }, where :math:\epsilon is the machine precision, is used instead.

**Returns**
**est** : bool
Indicates if the function was estimable.

:math:\mathrm{est} = \mathbf{True}

The function is estimable.

:math:\mathrm{est} = \mathbf{False}

The function is not estimable and :math:\mathrm{stat}, :math:\mathrm{sestat} and :math:\mathrm{t} are not set.

**stat** : float
If :math:\mathrm{est} = \mathbf{True}, :math:\mathrm{stat} contains the estimate of the function, :math:f^\mathrm{T}\hat{\beta }.

**sestat** : float
If :math:\mathrm{est} = \mathbf{True}, :math:\mathrm{sestat} contains the standard error of the estimate of the function, :math:\mathrm{se}\left(F\right).

**t** : float
If :math:\mathrm{est} = \mathbf{True}, :math:\mathrm{t} contains the :math:t-statistic for the test of the function being equal to zero.

.. _g02dn-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{irank} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{irank} \leq \textit{ip}.

(errno :math:1)
On entry, :math:\mathrm{irank} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{irank} \geq 1.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:3)
Standard error of statistic :math:\text{} = 0.0; this may be due to rounding errors if the standard error is very small or due to mis-specified inputs :math:\mathrm{cov} and :math:\mathrm{f}.

**Warns**
**NagAlgorithmicWarning**
(errno :math:2)
On entry, :math:\mathrm{irank} = \textit{ip}, i.e., model of full rank. In this case :math:\mathrm{est} is returned as true and all statistics are calculated.

.. _g02dn-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

linregm_estfunc computes the estimates of an estimable function for a general linear regression model which is not of full rank.
It is intended for use after a call to :meth:linregm_fit or :meth:linregm_update.
An estimable function is a linear combination of the parameters such that it has a unique estimate.
For a full rank model all linear combinations of parameters are estimable.

In the case of a model not of full rank the functions use a singular value decomposition (SVD) to find the parameter estimates, :math:\hat{\beta }, and their variance-covariance matrix.
Given the upper triangular matrix :math:R obtained from the :math:QR decomposition of the independent variables the SVD gives

.. math::
R = Q_*\begin{pmatrix}D&0\\0&0\end{pmatrix}P^\mathrm{T}\text{,}

where :math:D is a :math:k\times k diagonal matrix with nonzero diagonal elements, :math:k being the rank of :math:R, and :math:Q_* and :math:P are :math:p\times p orthogonal matrices.
This gives the solution

.. math::
\hat{\beta } = P_1D^{-1}Q_{*_1}^\mathrm{T}c_1\text{,}

:math:P_1 being the first :math:k columns of :math:P, i.e., :math:P = \left(P_1P_0\right), :math:Q_{*_1} being the first :math:k columns of :math:Q_*, and :math:c_1 being the first :math:p elements of :math:c.

Details of the SVD are made available in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}\text{,}

as given by :meth:linregm_fit and :meth:linregm_update.

A linear function of the parameters, :math:F = f^\mathrm{T}\beta, can be tested to see if it is estimable by computing :math:\zeta = P_0^\mathrm{T}f.
If :math:\zeta is zero, then the function is estimable; if not, the function is not estimable.
In practice :math:\left\lvert \zeta \right\rvert is tested against some small quantity :math:\eta.

Given that :math:F is estimable it can be estimated by :math:f^\mathrm{T}\hat{\beta } and its standard error calculated from the variance-covariance matrix of :math:\hat{\beta }, :math:C_{\beta }, as

.. math::
\mathrm{se}\left(F\right) = \sqrt{f^\mathrm{T}C_{\beta }f}\text{.}

Also a :math:t-statistic,

.. math::
t = \frac{{f^\mathrm{T}\hat{\beta }}}{{\mathrm{se}\left(F\right)}}\text{,}

can be computed.
The :math:t-statistic will have a Student's :math:t-distribution with degrees of freedom as given by the degrees of freedom for the residual sum of squares for the model.

.. _g02dn-py2-py-references:

**References**
Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

Hammarling, S, 1985, The singular value decomposition in multivariate statistics, SIGNUM Newsl. (20(3)), 2--25

Searle, S R, 1971, Linear Models, Wiley
"""
raise NotImplementedError

[docs]def linregm_rssq(x, vname, isx, y, mean='M', wt=None):
r"""
linregm_rssq calculates the residual sums of squares for all possible linear regressions for a given set of independent variables.

.. _g02ea-py2-py-doc:

For full information please refer to the NAG Library document for g02ea

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02eaf.html

.. _g02ea-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation for the :math:\textit{j}\ th independent variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**vname** : str, array-like, shape :math:\left(m\right)
:math:\mathrm{vname}[\textit{j}-1] must contain the name of the variable in column :math:\textit{j} of :math:\mathrm{x}, for :math:\textit{j} = 1,2,\ldots,m.

**isx** : int, array-like, shape :math:\left(m\right)
Indicates which independent variables are to be considered in the model.

:math:\mathrm{isx}[j-1]\geq 2

The variable contained in the :math:j\ th column of :math:\mathrm{x} is included in all regression models, i.e., is a forced variable.

:math:\mathrm{isx}[j-1] = 1

The variable contained in the :math:j\ th column of :math:\mathrm{x} is included in the set from which the regression models are chosen, i.e., is a free variable.

:math:\mathrm{isx}[j-1] = 0

The variable contained in the :math:j\ th column of :math:\mathrm{x} is not included in the models.

**y** : float, array-like, shape :math:\left(n\right)
:math:\mathrm{y}[\textit{i}-1] must contain the :math:\textit{i}\ th observation on the dependent variable, :math:y_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**mean** : str, length 1, optional
Indicates if a mean term is to be included.

:math:\mathrm{mean} = \texttt{'M'}

A mean term, intercept, will be included in the model.

:math:\mathrm{mean} = \texttt{'Z'}

The model will pass through the origin, zero-point.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
If provided :math:\mathrm{wt} must contain the weights to be used with the model.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If :math:\mathrm{wt} is not provided the effective number of observations is :math:n.

**Returns**
**nmod** : int
The total number of models for which residual sums of squares have been calculated.

**modl** : str, ndarray, shape :math:\left(\max\left(2^{\textit{k}},m\right), m\right)
The first :math:\mathrm{nterms}[i-1] elements of the :math:i\ th row of :math:\mathrm{modl} contain the names of the independent variables, as given in :math:\mathrm{vname}, that are included in the :math:i\ th model.

**rss** : float, ndarray, shape :math:\left(\max\left(2^{\textit{k}},m\right)\right)
:math:\mathrm{rss}[\textit{i}-1] contains the residual sum of squares for the :math:\textit{i}\ th model, for :math:\textit{i} = 1,2,\ldots,\mathrm{nmod}.

**nterms** : int, ndarray, shape :math:\left(\max\left(2^{\textit{k}},m\right)\right)
:math:\mathrm{nterms}[\textit{i}-1] contains the number of independent variables in the :math:\textit{i}\ th model, not including the mean if one is fitted, for :math:\textit{i} = 1,2,\ldots,\mathrm{nmod}.

**mrank** : int, ndarray, shape :math:\left(\max\left(2^{\textit{k}},m\right)\right)
:math:\mathrm{mrank}[i-1] contains the rank of the residual sum of squares for the :math:i\ th model.

.. _g02ea-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{ldmodl} = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ldmodl} \geq m.

(errno :math:1)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'W'} or :math:\texttt{'U'}.

(errno :math:1)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 2.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1] \geq 0.0, for :math:i = 1,2,\ldots,n.

(errno :math:3)
On entry, :math:\mathrm{isx}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.

Constraint: :math:\mathrm{isx}[i-1] \geq 0, for :math:i = 1,2,\ldots,m.

(errno :math:3)
There are no free variables, i.e., no element of :math:\mathrm{isx} = 1.

(errno :math:4)
On entry, :math:\textit{ldmodl} = \langle\mathit{\boldsymbol{value}}\rangle and number of possible models is :math:\langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ldmodl}\geq the number of possible models.

(errno :math:5)
On entry, the number of independent variables to be considered (forced plus free plus mean if included) is greater or equal to the effective number of observations.

(errno :math:6)
The full model is not of full rank, i.e., some of the independent variables may be linear combinations of other independent variables. Variables must be excluded from the model in order to give full rank.

.. _g02ea-py2-py-notes:

**Notes**
For a set of :math:\textit{k} possible independent variables there are :math:2^{\textit{k}} linear regression models with from zero to :math:\textit{k} independent variables in each model.
For example if :math:\textit{k} = 3 and the variables are :math:A, :math:B and :math:C then the possible models are:

(i) null model

(#) :math:A

(#) :math:B

(#) :math:C

(#) :math:A and :math:B

(#) :math:A and :math:C

(#) :math:B and :math:C

(#) :math:A, :math:B and :math:C.

linregm_rssq calculates the residual sums of squares from each of the :math:2^{\textit{k}} possible models.
The method used involves a :math:QR decomposition of the matrix of possible independent variables.
Independent variables are then moved into and out of the model by a series of Givens rotations and the residual sums of squares computed for each model; see Clark (1981) and Smith and Bremner (1989).

The computed residual sums of squares are then ordered first by increasing number of terms in the model, then by decreasing size of residual sums of squares.
So the first model will always have the largest residual sum of squares and the :math:2^{\textit{k}}\ th will always have the smallest.
This aids you in selecting the best possible model from the given set of independent variables.

linregm_rssq allows you to specify some independent variables that must be in the model, the forced variables.
The other independent variables from which the possible models are to be formed are the free variables.

.. _g02ea-py2-py-references:

**References**
Clark, M R B, 1981, A Givens algorithm for moving from one linear model to another without going back to the data, Appl. Statist. (30), 198--203

Smith, D M and Bremner, J M, 1989, All possible subset regressions using the :math:{QR} decomposition, Comput. Statist. Data Anal. (7), 217--236

Weisberg, S, 1985, Applied Linear Regression, Wiley
"""
raise NotImplementedError

r"""
linregm_rssq_stat calculates :math:R^2 and :math:C_p-values from the residual sums of squares for a series of linear regression models.

.. _g02ec-py2-py-doc:

For full information please refer to the NAG Library document for g02ec

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02ecf.html

.. _g02ec-py2-py-parameters:

**Parameters**
**n** : int
:math:n, the number of observations used in the regression model.

**sigsq** : float
The best estimate of true variance of the errors, :math:\hat{\sigma }^2.

**tss** : float
The total sum of squares for the regression model.

**nterms** : int, array-like, shape :math:\left(\textit{nmod}\right)
:math:\mathrm{nterms}[\textit{i}-1] must contain the number of independent variables (not counting the mean) fitted to the :math:\textit{i}\ th model, for :math:\textit{i} = 1,2,\ldots,\textit{nmod}.

**rss** : float, array-like, shape :math:\left(\textit{nmod}\right)
:math:\mathrm{rss}[i-1] must contain the residual sum of squares for the :math:i\ th model.

**mean** : str, length 1, optional
Indicates if a mean term is to be included.

:math:\mathrm{mean} = \texttt{'M'}

A mean term, intercept, will be included in the model.

:math:\mathrm{mean} = \texttt{'Z'}

The model will pass through the origin, zero-point.

**Returns**
**rsq** : float, ndarray, shape :math:\left(\textit{nmod}\right)
:math:\mathrm{rsq}[\textit{i}-1] contains the :math:R^2-value for the :math:\textit{i}\ th model, for :math:\textit{i} = 1,2,\ldots,\textit{nmod}.

**cp** : float, ndarray, shape :math:\left(\textit{nmod}\right)
:math:\mathrm{cp}[\textit{i}-1] contains the :math:C_p-value for the :math:\textit{i}\ th model, for :math:\textit{i} = 1,2,\ldots,\textit{nmod}.

.. _g02ec-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:1)
On entry, :math:\mathrm{tss} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tss} > 0.0.

(errno :math:1)
On entry, :math:\mathrm{sigsq} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{sigsq} > 0.0.

(errno :math:1)
On entry, :math:\textit{nmod} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nmod} > 0.

(errno :math:2)
On entry: the number of parameters, :math:p, is :math:\langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{n} \geq {2p}.

(errno :math:3)
On entry, :math:\mathrm{rss}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{tss} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{rss}[i] \leq \mathrm{tss}, for all :math:i.

(errno :math:4)
A value of :math:C_p is less than :math:0.0. This may occur if :math:\mathrm{sigsq} is too large or if :math:\mathrm{rss}, :math:\mathrm{n} or IP are incorrect.

.. _g02ec-py2-py-notes:

**Notes**
When selecting a linear regression model for a set of :math:n observations a balance has to be found between the number of independent variables in the model and fit as measured by the residual sum of squares.
The more variables included the smaller will be the residual sum of squares.
Two statistics can help in selecting the best model.

(a) :math:R^2 represents the proportion of variation in the dependent variable that is explained by the independent variables.

.. math::
R^2 = \frac{\text{Regression Sum of Squares}}{\text{Total Sum of Squares}}\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|where|:math:\text{Total Sum of Squares} = \mathrm{tss} = \sum \left(y-\bar{y}\right)^2 (if mean is fitted, otherwise :math:\mathrm{tss} = \sum y^2) and                            |
+-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     |:math:\text{Regression Sum of Squares} = \text{RegSS} = \mathrm{tss}-\mathrm{rss}, where :math:\mathrm{rss} = \text{residual sum of squares} = \sum \left(y-\hat{y}\right)^2.|
+-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

The :math:R^2-values can be examined to find a model with a high :math:R^2-value but with small number of independent variables.

(#) :math:C_p statistic.

.. math::

where :math:p is the number of parameters (including the mean) in the model and :math:\hat{\sigma }^2 is an estimate of the true variance of the errors. This can often be obtained from fitting the full model.

A well fitting model will have :math:C_p\simeq p. :math:C_p is often plotted against :math:p to see which models are closest to the :math:C_p = p line.

linregm_rssq_stat may be called after :meth:linregm_rssq which calculates the residual sums of squares for all possible linear regression models.

.. _g02ec-py2-py-references:

**References**
Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley

Weisberg, S, 1985, Applied Linear Regression, Wiley
"""
raise NotImplementedError

[docs]def linregm_fit_onestep(istep, x, vname, isx, y, model, nterm, rss, idf, ifr, free, q, p, mean='M', wt=None, fin=2.0):
r"""
linregm_fit_onestep carries out one step of a forward selection procedure in order to enable the 'best' linear regression model to be found.

.. _g02ee-py2-py-doc:

For full information please refer to the NAG Library document for g02ee

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02eef.html

.. _g02ee-py2-py-parameters:

**Parameters**
**istep** : int
Indicates which step in the forward selection process is to be carried out.

:math:\mathrm{istep} = 0

The process is initialized.

**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation for the :math:\textit{j}\ th independent variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**vname** : str, array-like, shape :math:\left(m\right)
:math:\mathrm{vname}[\textit{j}-1] must contain the name of the independent variable in column :math:\textit{j} of :math:\mathrm{x}, for :math:\textit{j} = 1,2,\ldots,m.

**isx** : int, array-like, shape :math:\left(m\right)
Indicates which independent variables could be considered for inclusion in the regression.

:math:\mathrm{isx}[j-1]\geq 2

The variable contained in the :math:\textit{j}\ th column of :math:\mathrm{x} is automatically included in the regression model, for :math:\textit{j} = 1,2,\ldots,m.

:math:\mathrm{isx}[j-1] = 1

The variable contained in the :math:\textit{j}\ th column of :math:\mathrm{x} is considered for inclusion in the regression model, for :math:\textit{j} = 1,2,\ldots,m.

:math:\mathrm{isx}[j-1] = 0

The variable in the :math:\textit{j}\ th column is not considered for inclusion in the model, for :math:\textit{j} = 1,2,\ldots,m.

**y** : float, array-like, shape :math:\left(n\right)
The dependent variable.

**model** : str, array-like, shape :math:\left(\textit{maxip}\right)
If :math:\mathrm{istep} = 0, :math:\mathrm{model} need not be set.

If :math:\mathrm{istep}\neq 0, :math:\mathrm{model} must contain the values returned by the previous call to linregm_fit_onestep.

**nterm** : int
If :math:\mathrm{istep} = 0, :math:\mathrm{nterm} need not be set.

If :math:\mathrm{istep}\neq 0, :math:\mathrm{nterm} must contain the value returned by the previous call to linregm_fit_onestep.

If :math:\mathrm{istep} = 0, :math:\mathrm{rss} need not be set.

If :math:\mathrm{istep}\neq 0, :math:\mathrm{rss} must contain the value returned by the previous call to linregm_fit_onestep.

**idf** : int
If :math:\mathrm{istep} = 0, :math:\mathrm{idf} need not be set.

If :math:\mathrm{istep}\neq 0, :math:\mathrm{idf} must contain the value returned by the previous call to linregm_fit_onestep.

**ifr** : int
If :math:\mathrm{istep} = 0, :math:\mathrm{ifr} need not be set.

If :math:\mathrm{istep}\neq 0, :math:\mathrm{ifr} must contain the value returned by the previous call to linregm_fit_onestep.

**free** : str, array-like, shape :math:\left(\textit{maxip}\right)
If :math:\mathrm{istep} = 0, :math:\mathrm{free} need not be set.

If :math:\mathrm{istep}\neq 0, :math:\mathrm{free} must contain the values returned by the previous call to linregm_fit_onestep.

**q** : float, array-like, shape :math:\left(n, \textit{maxip}+2\right)
If :math:\mathrm{istep} = 0, :math:\mathrm{q} need not be set.

If :math:\mathrm{istep}\neq 0, :math:\mathrm{q} must contain the values returned by the previous call to linregm_fit_onestep.

**p** : float, array-like, shape :math:\left(\textit{maxip}+1\right)
If :math:\mathrm{istep} = 0, :math:\mathrm{p} need not be set.

If :math:\mathrm{istep}\neq 0, :math:\mathrm{p} must contain the values returned by the previous call to linregm_fit_onestep.

**mean** : str, length 1, optional
Indicates if a mean term is to be included.

:math:\mathrm{mean} = \texttt{'M'}

A mean term, intercept, will be included in the model.

:math:\mathrm{mean} = \texttt{'Z'}

The model will pass through the origin, zero-point.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
If provided :math:\mathrm{wt} must contain the weights to be used with the model.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If :math:\mathrm{wt} is not provided the effective number of observations is :math:n.

**fin** : float, optional
The critical value of the :math:F statistic for the term to be included in the model, :math:F_{\mathrm{c}}.

**Returns**
**istep** : int
Is incremented by :math:1.

Indicates if a variable has been added to the model.

:math:\mathrm{addvar} = \mathbf{True}

A variable has been added to the model.

:math:\mathrm{addvar} = \mathbf{False}

No variable had an :math:F value greater than :math:F_{\mathrm{c}} and none were added to the model.

**newvar** : str
If :math:\mathrm{addvar} = \mathbf{True}, :math:\mathrm{newvar} contains the name of the variable added to the model.

If :math:\mathrm{addvar} = \mathbf{True}, :math:\mathrm{chrss} contains the change in the residual sum of squares due to adding variable :math:\mathrm{newvar}.

**f** : float
If :math:\mathrm{addvar} = \mathbf{True}, :math:\mathrm{f} contains the :math:F statistic for the inclusion of the variable in :math:\mathrm{newvar}.

**model** : str, ndarray, shape :math:\left(\textit{maxip}\right)
The names of the variables in the current model.

**nterm** : int
The number of independent variables in the current model, not including the mean, if any.

The residual sums of squares for the current model.

**idf** : int
The degrees of freedom for the residual sum of squares for the current model.

**ifr** : int
The number of free independent variables, i.e., the number of variables not in the model that are still being considered for selection.

**free** : str, ndarray, shape :math:\left(\textit{maxip}\right)
The first :math:\mathrm{ifr} values of :math:\mathrm{free} contain the names of the free variables.

**exss** : float, ndarray, shape :math:\left(\textit{maxip}\right)
The first :math:\mathrm{ifr} values of :math:\mathrm{exss} contain what would be the change in regression sum of squares if the free variables had been added to the model, i.e., the extra sum of squares for the free variables. :math:\mathrm{exss}[i-1] contains what would be the change in regression sum of squares if the variable :math:\mathrm{free}[i-1] had been added to the model.

**q** : float, ndarray, shape :math:\left(n, \textit{maxip}+2\right)
The results of the :math:QR decomposition for the current model:

the first column of :math:\mathrm{q} contains :math:c = Q^\mathrm{T}y (or :math:Q^\mathrm{T}W^{{\frac{1}{2}}}y where :math:W is the vector of weights if used);

the upper triangular part of columns :math:2 to :math:\textit{p}+1 contain the :math:R matrix;

the strictly lower triangular part of columns :math:2 to :math:\textit{p}+1 contain details of the :math:Q matrix;

the remaining :math:\textit{p}+1 to :math:\textit{p}+\mathrm{ifr} columns of contain :math:Q^\mathrm{T}X_{\textit{free}} (or :math:Q^\mathrm{T}W^{{\frac{1}{2}}}X_{\textit{free}}),

where :math:\textit{p} = \mathrm{nterm}, or :math:\textit{p} = \mathrm{nterm}+1 if :math:\mathrm{mean} = \texttt{'M'}.

**p** : float, ndarray, shape :math:\left(\textit{maxip}+1\right)
The first :math:\textit{p} elements of :math:\mathrm{p} contain details of the :math:QR decomposition, where :math:\textit{p} = \mathrm{nterm}, or :math:\textit{p} = \mathrm{nterm}+1 if :math:\mathrm{mean} = \texttt{'M'}.

.. _g02ee-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{fin} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{fin}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{rss} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{rss} > 0.0.

(errno :math:1)
On entry, :math:\mathrm{istep} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{nterm} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: if :math:\mathrm{istep} \neq 0, :math:\mathrm{nterm} > 0.

(errno :math:1)
On entry, :math:\mathrm{istep} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{istep} \geq 0.

(errno :math:1)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'W'} or :math:\texttt{'U'}.

(errno :math:1)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1]\geq 0.0, for :math:i = 1,2,\ldots,n.

(errno :math:3)
On entry, number of forced variables :math:\text{}\geq n.

(errno :math:3)
Degrees of freedom for error will equal :math:0 if new variable is added, i.e., the number of variables in the model plus :math:1 is equal to the effective number of observations.

(errno :math:4)
On entry, :math:\textit{maxip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{maxip} must be large enough to accommodate the number of terms given by :math:\mathrm{isx}.

(errno :math:4)
On entry, :math:\mathrm{isx}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.

Constraint: :math:\mathrm{isx}[i-1] \geq 0, for :math:i = 1,2,\ldots,m.

(errno :math:4)
On entry, :math:\mathrm{isx}[i-1] = 0, for all :math:i = 1,2,\ldots,m.

Constraint: at least one value of :math:\mathrm{isx} must be nonzero.

**Warns**
**NagAlgorithmicWarning**
(errno :math:5)
On entry, the variables forced into the model are not of full rank, i.e., some of these variables are linear combinations of others.

(errno :math:6)
There are no free variables, i.e., no element of :math:\mathrm{isx} = 0.

(errno :math:7)
The value of the change in the sum of squares is greater than the input value of :math:\mathrm{rss}. This may occur due to rounding errors if the true residual sum of squares for the new model is small relative to the residual sum of squares for the previous model.

.. _g02ee-py2-py-notes:

**Notes**
One method of selecting a linear regression model from a given set of independent variables is by forward selection.
The following procedure is used:

(i) Select the best fitting independent variable, i.e., the independent variable which gives the smallest residual sum of squares. If the :math:F-test for this variable is greater than a chosen critical value, :math:F_{\mathrm{c}}, then include the variable in the model, else stop.

(#) Find the independent variable that leads to the greatest reduction in the residual sum of squares when added to the current model.

(#) If the :math:F-test for this variable is greater than a chosen critical value, :math:F_{\mathrm{c}}, then include the variable in the model and go to \(2), otherwise stop.

At any step the variables not in the model are known as the free terms.

linregm_fit_onestep allows you to specify some independent variables that must be in the model, these are known as forced variables.

The computational procedure involves the use of :math:QR decompositions, the :math:R and the :math:Q matrices being updated as each new variable is added to the model.
In addition the matrix :math:Q^\mathrm{T}X_{\mathrm{free}}, where :math:X_{\mathrm{free}} is the matrix of variables not included in the model, is updated.

linregm_fit_onestep computes one step of the forward selection procedure at a call.
The results produced at each step may be printed or used as inputs to :meth:linregm_update, in order to compute the regression coefficients for the model fitted at that step.
Repeated calls to linregm_fit_onestep should be made until :math:F < F_{\mathrm{c}} is indicated.

.. _g02ee-py2-py-references:

**References**
Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley

Weisberg, S, 1985, Applied Linear Regression, Wiley
"""
raise NotImplementedError

[docs]def linregm_fit_stepwise(n, wmean, c, sw, isx, fin=4.0, fout=None, tau=0.000001, monlev=0, monfun=None, data=None, io_manager=None):
r"""
linregm_fit_stepwise calculates a full stepwise selection from :math:p variables by using Clarke's sweep algorithm on the correlation matrix of a design and data matrix, :math:Z.
The (weighted) variance-covariance, (weighted) means and sum of weights of :math:Z must be supplied.

.. _g02ef-py2-py-doc:

For full information please refer to the NAG Library document for g02ef

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02eff.html

.. _g02ef-py2-py-parameters:

**Parameters**
**n** : int
The number of observations used in the calculations.

**wmean** : float, array-like, shape :math:\left(m+1\right)
The mean of the design matrix, :math:Z.

**c** : float, array-like, shape :math:\left(\left(m+1\right)\times \left(m+2\right)/2\right)
The upper-triangular variance-covariance matrix packed by column for the design matrix, :math:Z. Because the function computes the correlation matrix :math:R from :math:\mathrm{c}, the variance-covariance matrix need only be supplied up to a scaling factor.

**sw** : float
If weights were used to calculate :math:\mathrm{c} then :math:\mathrm{sw} is the sum of positive weight values; otherwise :math:\mathrm{sw} is the number of observations used to calculate :math:\mathrm{c}.

**isx** : int, array-like, shape :math:\left(m\right)
The value of :math:\mathrm{isx}[\textit{j}-1] determines the set of variables used to perform full stepwise model selection, for :math:\textit{j} = 1,2,\ldots,m.

:math:\mathrm{isx}[\textit{j}-1] = -1

To exclude the variable corresponding to the :math:j\ th column of :math:X from the final model.

:math:\mathrm{isx}[\textit{j}-1] = 1

To consider the variable corresponding to the :math:j\ th column of :math:X for selection in the final model.

:math:\mathrm{isx}[\textit{j}-1] = 2

To force the inclusion of the variable corresponding to the :math:j\ th column of :math:X in the final model.

**fin** : float, optional
The value of the variance ratio which an explanatory variable must exceed to be included in a model.

**fout** : None or float, optional
Note: if this argument is **None** then a default value will be used, determined as follows: :math:\mathrm{fin}.

The explanatory variable in a model with the lowest variance ratio value is removed from the model if its value is less than :math:\mathrm{fout}. :math:\mathrm{fout} is usually set equal to the value of :math:\mathrm{fin}; a value less than :math:\mathrm{fin} is occasionally preferred.

**tau** : float, optional
The tolerance, :math:\tau, for detecting collinearities between variables when adding or removing an explanatory variable from a model. Explanatory variables deemed to be collinear are excluded from the final model.

**monlev** : int, optional
A value of :math:1 for :math:\mathrm{monlev} enables monitoring of the model selection process; a value of :math:0 disables it.

**monfun** : None or callable monfun(flag, var, val, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.

The function for monitoring the model selection process.

**Parameters**
**flag** : str, length 1
The value of :math:\mathrm{flag} indicates the stage of the stepwise selection of explanatory variables.

:math:\mathrm{flag} = \texttt{'A'}

Variable :math:\mathrm{var} was added to the current model.

:math:\mathrm{flag} = \texttt{'B'}

Beginning the backward elimination step.

:math:\mathrm{flag} = \texttt{'C'}

Variable :math:\mathrm{var} failed the collinearity test and is excluded from the model.

:math:\mathrm{flag} = \texttt{'D'}

Variable :math:\mathrm{var} was dropped from the current model.

:math:\mathrm{flag} = \texttt{'F'}

Beginning the forward selection step

:math:\mathrm{flag} = \texttt{'K'}

Backward elimination did not remove any variables from the current model.

:math:\mathrm{flag} = \texttt{'S'}

Starting stepwise selection procedure.

:math:\mathrm{flag} = \texttt{'V'}

The variance ratio for variable :math:\mathrm{var} takes the value :math:\mathrm{val}.

:math:\mathrm{flag} = \texttt{'X'}

Finished stepwise selection procedure.

**var** : int
The index of the explanatory variable in the design matrix :math:Z to which :math:\mathrm{flag} pertains.

**val** : float
If :math:\mathrm{flag} = \texttt{'V'}, :math:\mathrm{val} is the variance ratio value for the coefficient associated with explanatory variable index :math:\mathrm{var}.

**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.

**data** : arbitrary, optional
User-communication data for callback functions.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**isx** : int, ndarray, shape :math:\left(m\right)
The value of :math:\mathrm{isx}[\textit{j}-1] indicates the status of the :math:j\ th explanatory variable in the model.

:math:\mathrm{isx}[\textit{j}-1] = -1

Forced exclusion.

:math:\mathrm{isx}[\textit{j}-1] = 0

Excluded.

:math:\mathrm{isx}[\textit{j}-1] = 1

Selected.

:math:\mathrm{isx}[\textit{j}-1] = 2

Forced selection.

**b** : float, ndarray, shape :math:\left(m+1\right)
:math:\mathrm{b}[0] contains the estimate for the intercept term in the fitted model. If :math:\mathrm{isx}[j-1]\neq 0, then :math:\mathrm{b}[{j+1}-1] contains the estimate for the :math:j\ th explanatory variable in the fitted model; otherwise :math:\mathrm{b}[{j+1}-1] = 0.

**se** : float, ndarray, shape :math:\left(m+1\right)
:math:\mathrm{se}[\textit{j}-1] contains the standard error for the estimate of :math:\mathrm{b}[\textit{j}-1], for :math:\textit{j} = 1,2,\ldots,m+1.

**rsq** : float
The :math:R^2-statistic for the fitted regression model.

**rms** : float
The mean square of residuals for the fitted regression model.

**df** : int
The number of degrees of freedom for the sum of squares of residuals.

.. _g02ef-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m > 1.

(errno :math:1)
On entry, :math:\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{n} > 1.

(errno :math:1)
On entry, :math:\mathrm{sw} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{sw} > 1.0.

(errno :math:1)
On entry, :math:\mathrm{fin} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{fin} > 0.0.

(errno :math:1)
On entry, :math:\mathrm{fout} = \langle\mathit{\boldsymbol{value}}\rangle; :math:\mathrm{fin} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0.0\leq \mathrm{fout}\leq \mathrm{fin}.

(errno :math:1)
On entry, :math:\mathrm{tau} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tau} > 0.0.

(errno :math:1)
On entry, :math:\mathrm{monlev} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{monlev} = 0 or :math:1.

(errno :math:2)
On entry at least one diagonal element of :math:\mathrm{c}\leq 0.0.

Constraint: :math:\mathrm{c} must be positive definite.

(errno :math:2)
On entry, :math:\mathrm{isx}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{isx}[\textit{j}-1] = -1, :math:1 or :math:2, for :math:\textit{j} = 1,2,\ldots,m.

(errno :math:2)
On entry, :math:\mathrm{isx}[i-1]\neq 1, for all :math:i = 1,2,\ldots,m.

Constraint: there must be at least one free variable.

(errno :math:4)
All variables are collinear, no model to select.

**Warns**
**NagAlgorithmicWarning**
(errno :math:3)
Matrix not positive definite, results may be inaccurate.

.. _g02ef-py2-py-notes:

**Notes**
The general multiple linear regression model is defined by

.. math::
y = \beta_0+X\beta +\epsilon \text{,}

where

:math:y is a vector of :math:n observations on the dependent variable,

:math:\beta_0 is an intercept coefficient,

:math:X is an :math:n\times p matrix of :math:p explanatory variables,

:math:\beta is a vector of :math:p unknown coefficients, and

:math:\epsilon is a vector of length :math:n of unknown, Normally distributed, random errors.

linregm_fit_stepwise employs a full stepwise regression to select a subset of explanatory variables from the :math:p available variables (the intercept is included in the model) and computes regression coefficients and their standard errors, and various other statistical quantities, by minimizing the sum of squares of residuals.
The method applies repeatedly a forward selection step followed by a backward elimination step and halts when neither step updates the current model.

The criterion used to update a current model is the variance ratio of residual sum of squares.
Let :math:s_1 and :math:s_2 be the residual sum of squares of the current model and this model after undergoing a single update, with degrees of freedom :math:q_1 and :math:q_2, respectively.
Then the condition:

.. math::
\frac{{\left(s_2-s_1\right)/\left(q_2-q_1\right)}}{{s_1/q_1}} > f_1\text{,}

must be satisfied if a variable :math:k will be considered for entry to the current model, and the condition:

.. math::
\frac{{\left(s_1-s_2\right)/\left(q_1-q_2\right)}}{{s_1/q_1}} < f_2\text{,}

must be satisfied if a variable :math:k will be considered for removal from the current model, where :math:f_1 and :math:f_2 are user-supplied values and :math:f_2\leq f_1.

In the entry step the entry statistic is computed for each variable not in the current model.
If no variable is associated with a test value that exceeds :math:f_1 then this step is terminated; otherwise the variable associated with the largest value for the entry statistic is entered into the model.

In the removal step the removal statistic is computed for each variable in the current model.
If no variable is associated with a test value less than :math:f_2 then this step is terminated; otherwise the variable associated with the smallest value for the removal statistic is removed from the model.

The data values :math:X and :math:y are not provided as input to the function.
Instead, summary statistics of the design and data matrix :math:Z = \left(X | y\right) are required.

Explanatory variables are entered into and removed from the current model by using sweep operations on the correlation matrix :math:R of :math:Z, given by:

.. math::
R = \begin{pmatrix}1&\ldots &r_{{1p}}&r_{{1y}}\\ \vdots &⋱& \vdots & \vdots \\r_{{p1}}&\ldots &1&r_{{py}}\\r_{{y1}}&\ldots &r_{{yp}}&1\end{pmatrix}\text{,}

where :math:r_{{\textit{i}\textit{j}}} is the correlation between the explanatory variables :math:\textit{i} and :math:\textit{j}, for :math:\textit{j} = 1,2,\ldots,p, for :math:\textit{i} = 1,2,\ldots,p, and :math:r_{{yi}} (and :math:r_{{iy}}) is the correlation between the response variable :math:y and the :math:\textit{i}\ th explanatory variable, for :math:\textit{i} = 1,2,\ldots,p.

A sweep operation on the :math:k\ th row and column (:math:k\leq p) of :math:R replaces:

.. math::
\begin{array}{l} r_{{kk}} \text{ by } -1 / r_{{kk}} \text{;} \\ r_{{ik}} \text{ by } r_{{ik}} / \left\lvert r_{{kk}}\right\rvert \text{, }\quad i = 1,2,\ldots,p+1 \text{ } \left(i\neq k\right) \text{;} \\ r_{{kj}} \text{ by } r_{{kj}} / \left\lvert r_{{kk}}\right\rvert \text{, }\quad j = 1,2,\ldots,p+1 \text{ } \left(j\neq k\right) \text{;} \\ r_{{ij}} \text{ by } r_{{ij}} - r_{{ik}} r_{{kj}} / \left\lvert r_{{kk}}\right\rvert \text{, } i = 1,2,\ldots,p+1 \text{ } \left(i\neq k\right) \text{; } j = 1,2,\ldots,p+1 \text{ } \left(j\neq k\right) \text{.} \end{array}

The :math:k\ th explanatory variable is eligible for entry into the current model if it satisfies the collinearity tests: :math:r_{{kk}} > \tau and

.. math::
\left(r_{{ii}}-\frac{{r_{{ik}}r_{{ki}}}}{{r_{{kk}}}}\right)\tau \leq 1\text{,}

for a user-supplied value (:math:> 0) of :math:\tau and where the index :math:i runs over explanatory variables in the current model.
The sweep operation is its own inverse, therefore, pivoting on an explanatory variable :math:k in the current model has the effect of removing it from the model.

Once the stepwise model selection procedure is finished, the function calculates:

(a) the least squares estimate for the :math:i\ th explanatory variable included in the fitted model;

(#) standard error estimates for each coefficient in the final model;

(#) the square root of the mean square of residuals and its degrees of freedom;

(#) the multiple correlation coefficient.

The function makes use of the symmetry of the sweep operations and correlation matrix which reduces by almost one half the storage and computation required by the sweep algorithm, see Clarke (1981) for details.

.. _g02ef-py2-py-references:

**References**
Clarke, M R B, 1981, Algorithm AS 178: the Gauss--Jordan sweep operator with detection of collinearity, Appl. Statist. (31), 166--169

Dempster, A P, 1969, Elements of Continuous Multivariate Analysis, Addison--Wesley

Draper, N R and Smith, H, 1985, Applied Regression Analysis, (2nd Edition), Wiley

--------
:meth:naginterfaces.library.examples.correg.linregm_fit_stepwise_ex.main
"""
raise NotImplementedError

[docs]def linregm_stat_resinf(n, ip, res, h, rms):
r"""
linregm_stat_resinf calculates two types of standardized residuals and two measures of influence for a linear regression.

.. _g02fa-py2-py-doc:

For full information please refer to the NAG Library document for g02fa

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02faf.html

.. _g02fa-py2-py-parameters:

**Parameters**
**n** : int
:math:n, the number of observations included in the regression.

**ip** : int
:math:p, the number of linear parameters estimated in the regression model.

**res** : float, array-like, shape :math:\left(\textit{nres}\right)
The residuals, :math:r_i.

**h** : float, array-like, shape :math:\left(\textit{nres}\right)
The diagonal elements of :math:H, :math:h_i, corresponding to the residuals in :math:\mathrm{res}.

**rms** : float
The estimate of :math:\sigma^2 based on all :math:n observations, :math:s^2, i.e., the residual mean square.

**Returns**
**sres** : float, ndarray, shape :math:\left(\textit{nres}, 4\right)
The standardized residuals and influence statistics.

For the observation with residual, :math:r_i, given in :math:\mathrm{res}[i-1].

:math:\mathrm{sres}[i-1,0]

Is the internally standardized residual, :math:\mathrm{RI}_i.

:math:\mathrm{sres}[i-1,1]

Is the externally standardized residual, :math:\mathrm{RE}_i.

:math:\mathrm{sres}[i-1,2]

Is Cook's :math:D statistic, :math:D_i.

:math:\mathrm{sres}[i-1,3]

Is Atkinson's :math:T statistic, :math:T_i.

.. _g02fa-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{nres} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nres} \leq \mathrm{n}.

(errno :math:1)
On entry, :math:\mathrm{rms} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{rms} > 0.0.

(errno :math:1)
On entry, :math:\mathrm{ip} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{n} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:{\mathrm{n}-1} > \mathrm{ip}.

(errno :math:1)
On entry, :math:\mathrm{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{ip} \geq 1.

(errno :math:1)
On entry, :math:\textit{nres} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nres} \geq 1.

(errno :math:2)
On entry, :math:\mathrm{h}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0.0 < \mathrm{h}[i] < 1.0, for all :math:i.

(errno :math:3)
On entry, a value in :math:\mathrm{res} is too large for given :math:\mathrm{rms}. :math:\mathrm{res}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{rms} = \langle\mathit{\boldsymbol{value}}\rangle.

.. _g02fa-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

For the general linear regression model

.. math::
y = X\beta +\epsilon \text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+-------------------------------------------------------------------------------------------------------------------------------------------+
|where|:math:y is a vector of length :math:n of the dependent variable,                                                                       |
+-----+-------------------------------------------------------------------------------------------------------------------------------------------+
|     |:math:X is an :math:n\times p matrix of the independent variables,                                                                     |
+-----+-------------------------------------------------------------------------------------------------------------------------------------------+
|     |:math:\beta is a vector of length :math:p of unknown parameters,                                                                       |
+-----+-------------------------------------------------------------------------------------------------------------------------------------------+
|and  |:math:\epsilon is a vector of length :math:n of unknown random errors such that :math:\mathrm{var}\left(\epsilon \right) = \sigma^2I.|
+-----+-------------------------------------------------------------------------------------------------------------------------------------------+

The residuals are given by

.. math::
r = y-\hat{y} = y-X\hat{\beta }

and the fitted values, :math:\hat{y} = X\hat{\beta }, can be written as :math:Hy for an :math:n\times n matrix :math:H.
The :math:i\ th diagonal elements of :math:H, :math:h_i, give a measure of the influence of the :math:i\ th values of the independent variables on the fitted regression model.
The values of :math:r and the :math:h_i are returned by :meth:linregm_fit.

linregm_stat_resinf calculates statistics which help to indicate if an observation is extreme and having an undue influence on the fit of the regression model.
Two types of standardized residual are calculated:

(i) The :math:i\ th residual is standardized by its variance when the estimate of :math:\sigma^2, :math:s^2, is calculated from all the data; this is known as internal Studentization.

.. math::
RI_i = \frac{r_i}{{s\sqrt{1-h_i}}}\text{.}

(#) The :math:i\ th residual is standardized by its variance when the estimate of :math:\sigma^2, :math:s_{{-i}}^2 is calculated from the data excluding the :math:i\ th observation; this is known as external Studentization.

.. math::
RE_i = \frac{r_i}{{s_{{-i}}\sqrt{1-h_i}}} = r_i\sqrt{\frac{{n-p-1}}{{n-p-RI_i^2}}}\text{.}

The two measures of influence are:

(i) Cook's :math:D

.. math::
D_i = \frac{1}{p}RE_i^2\frac{h_i}{{1-h_i}}\text{.}

(#) Atkinson's :math:T

.. math::
T_i = \left\lvert RE_i\right\rvert \sqrt{\left(\frac{{n-p}}{p}\right)\left(\frac{h_i}{{1-h_i}}\right)}\text{.}

.. _g02fa-py2-py-references:

**References**
Atkinson, A C, 1981, Two graphical displays for outlying and influential observations in regression, Biometrika (68), 13--20

Cook, R D and Weisberg, S, 1982, Residuals and Influence in Regression, Chapman and Hall
"""
raise NotImplementedError

[docs]def linregm_stat_durbwat(ip, res):
r"""
linregm_stat_durbwat calculates the Durbin--Watson statistic, for a set of residuals, and the upper and lower bounds for its significance.

.. _g02fc-py2-py-doc:

For full information please refer to the NAG Library document for g02fc

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02fcf.html

.. _g02fc-py2-py-parameters:

**Parameters**
**ip** : int
:math:p, the number of independent variables in the regression model, including the mean.

**res** : float, array-like, shape :math:\left(n\right)
The residuals, :math:r_1,r_2,\ldots,r_n.

**Returns**
**d** : float
The Durbin--Watson statistic, :math:d.

**pdl** : float
Lower bound for the significance of the Durbin--Watson statistic, :math:p_{\mathrm{l}}.

**pdu** : float
Upper bound for the significance of the Durbin--Watson statistic, :math:p_{\mathrm{u}}.

.. _g02fc-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{ip} \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > \mathrm{ip}.

(errno :math:2)
On entry, mean of :math:\mathrm{res} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: the mean of the residuals :math:\text{}\leq \sqrt{\epsilon }, where :math:\epsilon = \text{machine precision}.

(errno :math:3)
On entry, all residuals are identical.

.. _g02fc-py2-py-notes:

**Notes**
For the general linear regression model

.. math::
y = X\beta +\epsilon \text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|where|:math:y is a vector of length :math:n of the dependent variable, :math:X is an :math:n\times p matrix of the independent variables, :math:\beta is a vector of length :math:p of unknown parameters,|
+-----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|and  |:math:\epsilon is a vector of length :math:n of unknown random errors.                                                                                                                                      |
+-----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

The residuals are given by

.. math::
r = y-\hat{y} = y-X\hat{\beta }

and the fitted values, :math:\hat{y} = X\hat{\beta }, can be written as :math:Hy for an :math:n\times n matrix :math:H.
Note that when a mean term is included in the model the sum of the residuals is zero.
If the observations have been taken serially, that is :math:y_1,y_2,\ldots,y_n can be considered as a time series, the Durbin--Watson test can be used to test for serial correlation in the :math:\epsilon_i, see Durbin and Watson (1950), Durbin and Watson (1951) and Durbin and Watson (1971).

The Durbin--Watson statistic is

.. math::
d = \frac{{\sum_{{i = 1}}^{{n-1}}\left(r_{{i+1}}-r_i\right)^2}}{{\sum_{{i = 1}}^nr_i^2}}\text{.}

Positive serial correlation in the :math:\epsilon_i will lead to a small value of :math:d while for independent errors :math:d will be close to :math:2.
Durbin and Watson show that the exact distribution of :math:d depends on the eigenvalues of the matrix :math:HA where the matrix :math:A is such that :math:d can be written as

.. math::
d = \frac{{r^\mathrm{T}Ar}}{{r^\mathrm{T}r}}

and the eigenvalues of the matrix :math:A are :math:\lambda_j = \left(1-\cos\left(\pi j/n\right)\right), for :math:j = 1,2,\ldots,n-1.

However bounds on the distribution can be obtained, the lower bound being

.. math::
d_{\mathrm{l}} = \frac{{\sum_{{i = 1}}^{{n-p}}\lambda_iu_i^2}}{{\sum_{{i = 1}}^{{n-p}}u_i^2}}

and the upper bound being

.. math::
d_{\mathrm{u}} = \frac{{\sum_{{i = 1}}^{{n-p}}\lambda_{{i-1+p}}u_i^2}}{{\sum_{{i = 1}}^{{n-p}}u_i^2}}\text{,}

where the :math:u_i are independent standard Normal variables.
The lower tail probabilities associated with these bounds, :math:p_{\mathrm{l}} and :math:p_{\mathrm{u}}, are computed by :meth:stat.prob_durbin_watson <naginterfaces.library.stat.prob_durbin_watson>.
The interpretation of the bounds is that, for a test of size (significance) :math:\alpha, if :math:p_l\leq \alpha the test is significant, if :math:p_u > \alpha the test is not significant, while if :math:p_{\mathrm{l}} > \alpha and :math:p_{\mathrm{u}}\leq \alpha no conclusion can be reached.

The above probabilities are for the usual test of positive auto-correlation.
If the alternative of negative auto-correlation is required, then a call to :meth:stat.prob_durbin_watson <naginterfaces.library.stat.prob_durbin_watson> should be made with the argument :math:\mathrm{d} taking the value of :math:4-d; see Newbold (1988).

.. _g02fc-py2-py-references:

**References**
Durbin, J and Watson, G S, 1950, Testing for serial correlation in least squares regression. I, Biometrika (37), 409--428

Durbin, J and Watson, G S, 1951, Testing for serial correlation in least squares regression. II, Biometrika (38), 159--178

Durbin, J and Watson, G S, 1971, Testing for serial correlation in least squares regression. III, Biometrika (58), 1--19

Granger, C W J and Newbold, P, 1986, Forecasting Economic Time Series, (2nd Edition), Academic Press

Newbold, P, 1988, Statistics for Business and Economics, Prentice--Hall
"""
raise NotImplementedError

[docs]def glm_normal(x, isx, y, link='I', mean='M', wt=None, s=0.0, a=0.0, v=None, tol=0.0, maxit=10, iprint=0, eps=0.0, io_manager=None):
r"""
glm_normal fits a generalized linear model with normal errors.

.. _g02ga-py2-py-doc:

For full information please refer to the NAG Library document for g02ga

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02gaf.html

.. _g02ga-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation for the :math:\textit{j}\ th independent variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**isx** : int, array-like, shape :math:\left(m\right)
Indicates which independent variables are to be included in the model.

If :math:\mathrm{isx}[j-1] > 0, the variable contained in the :math:j\ th column of :math:\mathrm{x} is included in the regression model.

**y** : float, array-like, shape :math:\left(n\right)
The observations on the dependent variable, :math:y_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**link** : str, length 1, optional
Indicates which link function is to be used.

:math:\mathrm{link} = \texttt{'E'}

:math:\mathrm{link} = \texttt{'I'}

:math:\mathrm{link} = \texttt{'L'}

:math:\mathrm{link} = \texttt{'S'}

A square root link is used.

:math:\mathrm{link} = \texttt{'R'}

**mean** : str, length 1, optional
Indicates if a mean term is to be included.

:math:\mathrm{mean} = \texttt{'M'}

A mean term, intercept, will be included in the model.

:math:\mathrm{mean} = \texttt{'Z'}

The model will pass through the origin, zero-point.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
If provided :math:\mathrm{wt} must contain the weights to be used with the model.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If :math:\mathrm{wt} is not provided the effective number of observations is :math:n.

**s** : float, optional
The scale parameter for the model, :math:\sigma^2.

If :math:\mathrm{s} = 0.0, the scale parameter is estimated with the function using the residual mean square.

**a** : float, optional
If :math:\mathrm{link} = \texttt{'E'}, :math:\mathrm{a} must contain the power of the exponential.

If :math:\mathrm{link} \neq \texttt{'E'}, :math:\mathrm{a} is not referenced.

**v** : None or float, array-like, shape :math:\left(n, \textit{ip}+7\right), optional
If :math:\mathrm{v}\text{ is not }\mathbf{None}, :math:\mathrm{v}[\textit{i}-1,6], for :math:\textit{i} = 1,2,\ldots,n, must contain the offset values :math:o_{\textit{i}}.

All other values need not be set.

**tol** : float, optional
Indicates the accuracy required for the fit of the model.

The iterative weighted least squares procedure is deemed to have converged if the absolute change in deviance between interactions is less than :math:\mathrm{tol}\times \left(1.0+\text{current residual sum of squares}\right).

This is approximately an absolute precision if the residual sum of squares is small and a relative precision if the residual sum of squares is large.

If :math:0.0\leq \mathrm{tol} < \text{machine precision}, glm_normal will use :math:10\times \text{machine precision}.

**maxit** : int, optional
The maximum number of iterations for the iterative weighted least squares.

If :math:\mathrm{maxit} = 0, a default value of :math:10 is used.

**iprint** : int, optional
Indicates if the printing of information on the iterations is required.

:math:\mathrm{iprint}\leq 0

There is no printing.

:math:\mathrm{iprint} > 0

Every :math:\mathrm{iprint} iteration, the following is printed:

the deviance,

the current estimates,

and if the weighted least squares equations are singular, then this is indicated.

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see :class:~naginterfaces.base.utils.FileObjManager).

**eps** : float, optional
The value of :math:\mathrm{eps} is used to decide if the independent variables are of full rank and, if not, what is the rank of the independent variables. The smaller the value of :math:\mathrm{eps} the stricter the criterion for selecting the singular value decomposition.

If :math:0.0\leq \mathrm{eps} < \text{machine precision}, the function will use machine precision instead.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**s** : float
If on input :math:\mathrm{s} = 0.0, :math:\mathrm{s} contains the estimated value of the scale parameter, :math:\hat{\sigma }^2.

If on input :math:\mathrm{s}\neq 0.0, :math:\mathrm{s} is unchanged on exit.

The residual sum of squares for the fitted model.

**idf** : int
The degrees of freedom associated with the residual sum of squares for the fitted model.

**b** : float, ndarray, shape :math:\left(\textit{ip}\right)
The estimates of the parameters of the generalized linear model, :math:\hat{\beta }.

If :math:\mathrm{mean} = \texttt{'M'}, :math:\mathrm{b}[0] will contain the estimate of the mean parameter and :math:\mathrm{b}[i] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

If :math:\mathrm{mean} = \texttt{'Z'}, :math:\mathrm{b}[i-1] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

**irank** : int
The rank of the independent variables.

If the model is of full rank, :math:\mathrm{irank} = \textit{ip}.

If the model is not of full rank, :math:\mathrm{irank} is an estimate of the rank of the independent variables. :math:\mathrm{irank} is calculated as the number of singular values greater than :math:\mathrm{eps}\times \text{} (largest singular value).

It is possible for the SVD to be carried out but for :math:\mathrm{irank} to be returned as :math:\textit{ip}.

**se** : float, ndarray, shape :math:\left(\textit{ip}\right)
The standard errors of the linear parameters.

:math:\mathrm{se}[\textit{i}-1] contains the standard error of the parameter estimate in :math:\mathrm{b}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,\textit{ip}.

**cov** : float, ndarray, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored packed by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1].

**v** : float, ndarray, shape :math:\left(n, \textit{ip}+7\right)
Auxiliary information on the fitted model.

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,0]  |contains the linear predictor value, :math:\eta_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                 |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,1]  |contains the fitted value, :math:\hat{\mu }_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                     |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,2]  |is only included for consistency with other functions. :math:\mathrm{v}[\textit{i}-1,2] = 1.0, for :math:\textit{i} = 1,2,\ldots,n.|
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,3]  |contains the square root of the working weight, :math:w_{\textit{i}}^{\frac{1}{2}}, for :math:\textit{i} = 1,2,\ldots,n.           |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,4]  |contains the residual, :math:r_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                  |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,5]  |contains the leverage, :math:h_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                  |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,6]  |contains the offset, for :math:i = 1,2,\ldots,n. If :math:\mathrm{v} is **None** on entry, all values will be zero.                |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,j-1]|for :math:j = 8,\ldots,\textit{ip}+7, contains the results of the :math:QR decomposition or the singular value decomposition.      |
+---------------------------+---------------------------------------------------------------------------------------------------------------------------------------+

If the model is not of full rank, i.e., :math:\mathrm{irank} < \textit{ip}, the first :math:\textit{ip} rows of columns :math:8 to :math:\textit{ip}+7 contain the :math:P^* matrix.

.. _g02ga-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:1)
On entry, :math:\mathrm{eps} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{eps}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} \geq 0.

(errno :math:1)
On entry, :math:\textit{offset} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{offset} = \texttt{'Y'} or :math:\texttt{'N'}.

(errno :math:1)
On entry, :math:\mathrm{link} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{link} = \texttt{'E'}, :math:\texttt{'I'}, :math:\texttt{'L'}, :math:\texttt{'S'} or :math:\texttt{'R'}.

(errno :math:1)
On entry, :math:\mathrm{a} = 0.0 and :math:\mathrm{link} = \texttt{'E'}.

Constraint: if :math:\mathrm{link} = \texttt{'E'}, :math:\mathrm{a}\neq 0.0.

(errno :math:1)
On entry, :math:\mathrm{s} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{s}\geq 0.0.

(errno :math:1)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'W'} or :math:\texttt{'U'}.

(errno :math:1)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1] \geq 0.0, for :math:i = 1,2,\ldots,n.

(errno :math:3)
On entry, :math:\mathrm{isx}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.

Constraint: :math:\mathrm{isx}[j-1] \geq 0.0, for :math:j = 1,2,\ldots,m.

(errno :math:3)
On entry, :math:\textit{ip} incompatible with number of nonzero values of :math:\mathrm{isx}: :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:3)
Number of requested x-variables greater than :math:\textit{n}.

(errno :math:4)
A fitted value is at a boundary. This will only occur with :math:\mathrm{link} = \texttt{'L'}, :math:\texttt{'R'} or :math:\texttt{'E'}. This may occur if there are small values of :math:y and the model is not suitable for the data. The model should be reformulated with, perhaps, some observations dropped.

(errno :math:5)
SVD solution failed to converge.

(errno :math:6)
The iterative weighted least squares has failed to converge in :math:\mathrm{maxit} (or default :math:10) iterations. The value of :math:\mathrm{maxit} could be increased but it may be advantageous to examine the convergence using the :math:\mathrm{iprint} option. This may indicate that the convergence is slow because the solution is at a boundary in which case it may be better to reformulate the model.

**Warns**
**NagAlgorithmicWarning**
(errno :math:7)
The rank of the model has changed during the weighted least squares iterations. The estimate for :math:\beta returned may be reasonable, but you should check how the deviance has changed during iterations.

(errno :math:8)
The degrees of freedom for error are :math:0. A saturated model has been fitted.

.. _g02ga-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

A generalized linear model with Normal errors consists of the following elements:

(a) a set of :math:n observations, :math:y_i, from a Normal distribution with probability density function:

.. math::
\frac{1}{{\sqrt{2\pi }\sigma }}\mathrm{exp}\left(-\frac{\left(y-\mu \right)^2}{{2\sigma^2}}\right)\text{,}

where :math:\mu is the mean and :math:\sigma^2 is the variance.

(#) :math:X, a set of :math:p independent variables for each observation, :math:x_1,x_2,\ldots,x_p.

(#) a linear model:

.. math::
\eta = \sum \beta_jx_j\text{.}

(#) a link between the linear predictor, :math:\eta, and the mean of the distribution, :math:\mu, i.e., :math:\eta = g\left(\mu \right). The possible link functions are:

(i) exponent link: :math:\eta = \mu^a, for a constant :math:a,

(#) identity link: :math:\eta = \mu,

(#) log link: :math:\eta = \log\left(\mu \right),

(#) square root link: :math:\eta = \sqrt{\mu },

(#) reciprocal link: :math:\eta = \frac{1}{\mu }.

(#) a measure of fit, the residual sum of squares :math:\text{} = \sum \left(y_i-\hat{\mu }_i\right)^2.

The linear parameters are estimated by iterative weighted least squares.
An adjusted dependent variable, :math:z, is formed:

.. math::
z = \eta +\left(y-\mu \right)\frac{{d\eta }}{{d\mu }}

and a working weight, :math:w,

.. math::
w^{-1} = \left(\frac{{d\eta }}{{d\mu }}\right)^2\text{.}

At each iteration an approximation to the estimate of :math:\beta, :math:\hat{\beta }, is found by the weighted least squares regression of :math:z on :math:X with weights :math:w.

glm_normal finds a :math:QR decomposition of :math:w^{{\frac{1}{2}}}X, i.e., :math:w^{{\frac{1}{2}}}X = QR where :math:R is a :math:p\times p triangular matrix and :math:Q is an :math:n\times p column orthogonal matrix.

If :math:R is of full rank, then :math:\hat{\beta } is the solution to

.. math::
R\hat{\beta } = Q^\mathrm{T}w^{{\frac{1}{2}}}z\text{.}

If :math:R is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of :math:R.

.. math::
R = Q_*\begin{pmatrix}D&0\\0&0\end{pmatrix}P^\mathrm{T}\text{,}

where :math:D is a :math:k\times k diagonal matrix with nonzero diagonal elements, :math:k being the rank of :math:R and :math:w^{{\frac{1}{2}}}X.

This gives the solution

.. math::
\hat{\beta } = P_1D^{-1}\begin{pmatrix}Q_*&0\\0&I\end{pmatrix}Q^\mathrm{T}w^{{\frac{1}{2}}}z

:math:P_1 being the first :math:k columns of :math:P, i.e., :math:P = \left(P_1P_0\right).

The iterations are continued until there is only a small change in the residual sum of squares.

The initial values for the algorithm are obtained by taking

.. math::
\hat{\eta } = g\left(y\right)\text{.}

The fit of the model can be assessed by examining and testing the residual sum of squares, in particular comparing the difference in residual sums of squares between nested models, i.e., when one model is a sub-model of the other.

Let :math:\mathrm{RSS}_f be the residual sum of squares for the full model with degrees of freedom :math:\nu_f and let :math:\mathrm{RSS}_s be the residual sum of squares for the sub-model with degrees of freedom :math:\nu_s then:

.. math::

has, approximately, an :math:F-distribution with (:math:\nu_s-\nu_f), :math:\nu_f degrees of freedom.

The parameter estimates, :math:\hat{\beta }, are asymptotically Normally distributed with variance-covariance matrix:

:math:C = R^{-1}{R^{-1}}^\mathrm{T}\sigma^2 in the full rank case,

otherwise :math:C = P_1D^{-2}P_1^\mathrm{T}\sigma^2

The residuals and influence statistics can also be examined.

The estimated linear predictor :math:\hat{\eta } = X\hat{\beta }, can be written as :math:Hw^{{\frac{1}{2}}}z for an :math:n\times n matrix :math:H.
The :math:i\ th diagonal elements of :math:H, :math:h_i, give a measure of the influence of the :math:i\ th values of the independent variables on the fitted regression model.
These are sometimes known as leverages.

The fitted values are given by :math:\hat{\mu } = g^{-1}\left(\hat{\eta }\right).

glm_normal also computes the residuals, :math:r:

.. math::
r_i = y_i-\hat{\mu }_i\text{.}

An option allows prior weights :math:\omega_i to be used; this gives a model with:

.. math::
\sigma_i^2 = \frac{\sigma^2}{\omega_i}\text{.}

In many linear regression models the first term is taken as a mean term or an intercept, i.e., :math:x_{{\textit{i},1}} = 1, for :math:\textit{i} = 1,2,\ldots,n; this is provided as an option.

Often only some of the possible independent variables are included in a model, the facility to select variables to be included in the model is provided.

If part of the linear predictor can be represented by a variable with a known coefficient, then this can be included in the model by using an offset, :math:o:

.. math::
\eta = o+\sum \beta_jx_j\text{.}

If the model is not of full rank the solution given will be only one of the possible solutions.
Other estimates may be obtained by applying constraints to the parameters.
These solutions can be obtained by using :meth:glm_constrain after using glm_normal.
Only certain linear combinations of the parameters will have unique estimates; these are known as estimable functions and can be estimated and tested using :meth:glm_estfunc.

Details of the SVD are made available, in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}\text{.}

.. _g02ga-py2-py-references:

**References**
Cook, R D and Weisberg, S, 1982, Residuals and Influence in Regression, Chapman and Hall

McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall

--------
:meth:naginterfaces.library.examples.correg.glm_normal_ex.main
"""
raise NotImplementedError

[docs]def glm_binomial(x, isx, y, t, link='G', mean='M', wt=None, v=None, tol=0.0, maxit=10, iprint=0, eps=0.0, io_manager=None):
r"""
glm_binomial fits a generalized linear model with binomial errors.

.. _g02gb-py2-py-doc:

For full information please refer to the NAG Library document for g02gb

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02gbf.html

.. _g02gb-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation for the :math:\textit{j}\ th independent variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**isx** : int, array-like, shape :math:\left(m\right)
Indicates which independent variables are to be included in the model.

If :math:\mathrm{isx}[j-1] > 0, the variable contained in the :math:j\ th column of :math:\mathrm{x} is included in the regression model.

**y** : float, array-like, shape :math:\left(n\right)
The observations on the dependent variable, :math:y_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.

**t** : float, array-like, shape :math:\left(n\right)
:math:t, the binomial denominator.

**link** : str, length 1, optional
Indicates which link function is to be used.

:math:\mathrm{link} = \texttt{'G'}

:math:\mathrm{link} = \texttt{'P'}

:math:\mathrm{link} = \texttt{'C'}

A complementary log-log link is used.

**mean** : str, length 1, optional
Indicates if a mean term is to be included.

:math:\mathrm{mean} = \texttt{'M'}

A mean term, intercept, will be included in the model.

:math:\mathrm{mean} = \texttt{'Z'}

The model will pass through the origin, zero-point.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
If provided :math:\mathrm{wt} must contain the weights to be used with the model.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If :math:\mathrm{wt} is not provided the effective number of observations is :math:n.

**v** : None or float, array-like, shape :math:\left(n, \textit{ip}+7\right), optional
If :math:\mathrm{v}\text{ is not }\mathbf{None}, :math:\mathrm{v}[\textit{i}-1,6], for :math:\textit{i} = 1,2,\ldots,n, must contain the offset values :math:o_{\textit{i}}.

All other values need not be set.

**tol** : float, optional
Indicates the accuracy required for the fit of the model.

The iterative weighted least squares procedure is deemed to have converged if the absolute change in deviance between iterations is less than :math:\mathrm{tol}\times \left(1.0+\text{Current Deviance}\right).

This is approximately an absolute precision if the deviance is small and a relative precision if the deviance is large.

If :math:0.0\leq \mathrm{tol} < \text{machine precision}, the function will use :math:10\times \text{machine precision} instead.

**maxit** : int, optional
The maximum number of iterations for the iterative weighted least squares.

If :math:\mathrm{maxit} = 0, a default value of :math:10 is used.

**iprint** : int, optional
Indicates if the printing of information on the iterations is required.

:math:\mathrm{iprint}\leq 0

There is no printing.

:math:\mathrm{iprint} > 0

The following is printed every :math:\mathrm{iprint} iterations:

the deviance,

the current estimates,

and if the weighted least squares equations are singular, then this is indicated.

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see :class:~naginterfaces.base.utils.FileObjManager).

**eps** : float, optional
The value of :math:\mathrm{eps} is used to decide if the independent variables are of full rank and, if not, what is the rank of the independent variables. The smaller the value of :math:\mathrm{eps} the stricter the criterion for selecting the singular value decomposition.

If :math:0.0\leq \mathrm{eps} < \text{machine precision}, the function will use machine precision instead.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**dev** : float
The deviance for the fitted model.

**idf** : int
The degrees of freedom associated with the deviance for the fitted model.

**b** : float, ndarray, shape :math:\left(\textit{ip}\right)
The estimates of the parameters of the generalized linear model, :math:\hat{\beta }.

If :math:\mathrm{mean} = \texttt{'M'}, the first element of :math:\mathrm{b} will contain the estimate of the mean parameter and :math:\mathrm{b}[i] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

If :math:\mathrm{mean} = \texttt{'Z'}, :math:\mathrm{b}[i-1] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

**irank** : int
The rank of the independent variables.

If the model is of full rank, :math:\mathrm{irank} = \textit{ip}.

If the model is not of full rank, :math:\mathrm{irank} is an estimate of the rank of the independent variables. :math:\mathrm{irank} is calculated as the number of singular values greater that :math:\mathrm{eps}\times \text{}\ (largest singular value).

It is possible for the SVD to be carried out but for :math:\mathrm{irank} to be returned as :math:\textit{ip}.

**se** : float, ndarray, shape :math:\left(\textit{ip}\right)
The standard errors of the linear parameters.

:math:\mathrm{se}[\textit{i}-1] contains the standard error of the parameter estimate in :math:\mathrm{b}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,\textit{ip}.

**cov** : float, ndarray, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored in packed form by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1].

**v** : float, ndarray, shape :math:\left(n, \textit{ip}+7\right)
Auxiliary information on the fitted model.

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,0]  |contains the linear predictor value, :math:\eta_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                  |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,1]  |contains the fitted value, :math:\hat{\mu }_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                      |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,2]  |contains the variance standardization, :math:\frac{1}{\tau_{\textit{i}}}, for :math:\textit{i} = 1,2,\ldots,n.                                      |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,3]  |contains the square root of the working weight, :math:w_{\textit{i}}^{\frac{1}{2}}, for :math:\textit{i} = 1,2,\ldots,n.                            |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,4]  |contains the deviance residual, :math:r_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                          |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,5]  |contains the leverage, :math:h_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                                   |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,6]  |contains the offset, :math:o_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n. If :math:\mathrm{v} is **None** on entry, all values will be zero.|
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,j-1]|for :math:j = 8,\ldots,\textit{ip}+7, contains the results of the :math:QR decomposition or the singular value decomposition.                       |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+

If the model is not of full rank, i.e., :math:\mathrm{irank} < \textit{ip}, the first :math:\textit{ip} rows of columns :math:8 to :math:\textit{ip}+7 contain the :math:P^* matrix.

.. _g02gb-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:1)
On entry, :math:\mathrm{eps} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{eps}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} \geq 0.

(errno :math:1)
On entry, :math:\textit{offset} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{offset} = \texttt{'Y'} or :math:\texttt{'N'}.

(errno :math:1)
On entry, :math:\mathrm{link} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{link} = \texttt{'G'}, :math:\texttt{'P'} or :math:\texttt{'C'}.

(errno :math:1)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'W'} or :math:\texttt{'U'}.

(errno :math:1)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1] \geq 0.0, for :math:i = 1,2,\ldots,n.

(errno :math:3)
On entry, :math:\mathrm{isx}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.

Constraint: :math:\mathrm{isx}[j-1] \geq 0.0, for :math:j = 1,2,\ldots,m.

(errno :math:3)
On entry, :math:\textit{ip} incompatible with number of nonzero values of :math:\mathrm{isx}: :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:3)
Number of requested x-variables greater than :math:\textit{n}.

(errno :math:4)
On entry, :math:\mathrm{t}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{t}[i-1] \geq 0, for :math:i = 1,2,\ldots,n.

(errno :math:5)
On entry, :math:\mathrm{y}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{t}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0.0\leq \mathrm{y}[i-1]\leq \mathrm{t}[i-1], for :math:i = 1,2,\ldots,n.

(errno :math:6)
A fitted value is at the boundary, i.e., :math:0.0 or :math:1.0. This may occur if there are :math:y values of :math:0.0 or :math:t and the model is too complex for the data. The model should be reformulated with, perhaps, some observations dropped.

(errno :math:7)
SVD solution failed to converge.

(errno :math:8)
The iterative weighted least squares has failed to converge in :math:\mathrm{maxit} (or default :math:10) iterations. The value of :math:\mathrm{maxit} could be increased but it may be advantageous to examine the convergence using the :math:\mathrm{iprint} option. This may indicate that the convergence is slow because the solution is at a boundary in which case it may be better to reformulate the model.

**Warns**
**NagAlgorithmicWarning**
(errno :math:9)
The rank of the model has changed during the weighted least squares iterations. The estimate for :math:\beta returned may be reasonable, but you should check how the deviance has changed during iterations.

(errno :math:10)
Degrees of freedom for error are :math:0.

.. _g02gb-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

A generalized linear model with binomial errors consists of the following elements:

(a) a set of :math:n observations, :math:y_i, from a binomial distribution:

.. math::
\begin{pmatrix}t\\y\end{pmatrix}\pi^y\left(1-\pi \right)^{{t-y}}\text{.}

(#) :math:X, a set of :math:p independent variables for each observation, :math:x_1,x_2,\ldots,x_p.

(#) a linear model:

.. math::
\eta = \sum \beta_jx_j\text{.}

(#) a link between the linear predictor, :math:\eta, and the mean of the distribution, :math:\mu = \pi t, the link function, :math:\eta = g\left(\mu \right). The possible link functions are:

(i) logistic link: :math:\eta = \log\left(\frac{\mu }{{t-\mu }}\right),

(#) probit link: :math:\eta = \Phi^{-1}\left(\frac{\mu }{t}\right),

(#) complementary log-log link: :math:\log\left(-\log\left(1-\frac{\mu }{t}\right)\right)\text{.}

(#) a measure of fit, the deviance:

.. math::
\sum_{{i = 1}}^n\mathrm{dev}\left(y_i,\hat{\mu }_i\right) = \sum_{{i = 1}}^n2\left(y_i\log\left(\frac{y_i}{\hat{\mu }_i}\right)+\left(t_i-y_i\right)\log\left(\frac{\left(t_i-y_i\right)}{\left(t_i-\hat{\mu }_i\right)}\right)\right)\text{.}

The linear parameters are estimated by iterative weighted least squares.
An adjusted dependent variable, :math:z, is formed:

.. math::
z = \eta +\left(y-\mu \right)\frac{{d\eta }}{{d\mu }}

and a working weight, :math:w,

.. math::
w^{-1} = \left(\tau \frac{{d\eta }}{{d\mu }}\right)^2\text{, where }\tau = \sqrt{\frac{t}{{\mu \left(t-\mu \right)}}}\text{.}

At each iteration an approximation to the estimate of :math:\beta, :math:\hat{\beta }, is found by the weighted least squares regression of :math:z on :math:X with weights :math:w.

glm_binomial finds a :math:QR decomposition of :math:w^{{1/2}}X, i.e., :math:w^{{1/2}}X = QR where :math:R is a :math:p\times p triangular matrix and :math:Q is an :math:n\times p column orthogonal matrix.

If :math:R is of full rank, then :math:\hat{\beta } is the solution to

.. math::
R\hat{\beta } = Q^\mathrm{T}w^{{1/2}}z\text{.}

If :math:R is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of :math:R.

.. math::
R = Q_*\begin{pmatrix}D&0\\0&0\end{pmatrix}P^\mathrm{T}\text{,}

where :math:D is a :math:k\times k diagonal matrix with nonzero diagonal elements, :math:k being the rank of :math:R and :math:w^{{1/2}}X.

This gives the solution

.. math::
\hat{\beta } = P_1D^{-1}\begin{pmatrix}Q_*&0\\0&I\end{pmatrix}Q^\mathrm{T}w^{{1/2}}z\text{,}

:math:P_1 being the first :math:k columns of :math:P, i.e., :math:P = \left(P_1P_0\right).

The iterations are continued until there is only a small change in the deviance.

The initial values for the algorithm are obtained by taking

.. math::
\hat{\eta } = g\left(y\right)\text{.}

The fit of the model can be assessed by examining and testing the deviance, in particular by comparing the difference in deviance between nested models, i.e., when one model is a sub-model of the other.
The difference in deviance between two nested models has, asymptotically, a :math:\chi^2-distribution with degrees of freedom given by the difference in the degrees of freedom associated with the two deviances.

The parameters estimates, :math:\hat{\beta }, are asymptotically Normally distributed with variance-covariance matrix

:math:C = R^{-1}{R^{-1}}^\mathrm{T} in the full rank case, otherwise

:math:C = P_1D^{-2}P_1^\mathrm{T}.

The residuals and influence statistics can also be examined.

The estimated linear predictor :math:\hat{\eta } = X\hat{\beta }, can be written as :math:Hw^{{1/2}}z for an :math:n\times n matrix :math:H.
The :math:i\ th diagonal elements of :math:H, :math:h_i, give a measure of the influence of the :math:i\ th values of the independent variables on the fitted regression model.
These are sometimes known as leverages.

The fitted values are given by :math:\hat{\mu } = g^{-1}\left(\hat{\eta }\right).

glm_binomial also computes the deviance residuals, :math:r:

.. math::
r_i = \mathrm{sign}\left(y_i-\hat{\mu }_i\right)\sqrt{\mathrm{dev}\left(y_i,\hat{\mu }_i\right)}\text{.}

An option allows the use of prior weights in the model.

In many linear regression models the first term is taken as a mean term or an intercept, i.e., :math:x_{{i,1}} = 1, for :math:i = 1,2,\ldots,n.
This is provided as an option.

Often only some of the possible independent variables are included in a model; the facility to select variables to be included in the model is provided.

If part of the linear predictor can be represented by variables with a known coefficient then this can be included in the model by using an offset, :math:o:

.. math::
\eta = o+\sum \beta_jx_j\text{.}

If the model is not of full rank the solution given will be only one of the possible solutions.
Other estimates may be obtained by applying constraints to the parameters.
These solutions can be obtained by using :meth:glm_constrain after using glm_binomial.
Only certain linear combinations of the parameters will have unique estimates, these are known as estimable functions and can be estimated and tested using :meth:glm_estfunc.

Details of the SVD are made available in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}\text{.}

.. _g02gb-py2-py-references:

**References**
Cook, R D and Weisberg, S, 1982, Residuals and Influence in Regression, Chapman and Hall

Cox, D R, 1983, Analysis of Binary Data, Chapman and Hall

McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall

--------
:meth:naginterfaces.library.examples.correg.glm_binomial_ex.main
"""
raise NotImplementedError

[docs]def glm_poisson(x, isx, y, link='L', mean='M', wt=None, a=0.0, v=None, tol=0.0, maxit=10, iprint=0, eps=0.0, io_manager=None):
r"""
glm_poisson fits a generalized linear model with Poisson errors.

.. _g02gc-py2-py-doc:

For full information please refer to the NAG Library document for g02gc

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02gcf.html

.. _g02gc-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
The matrix of all possible independent variables. :math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\textit{j}\ th element of :math:\mathrm{x}, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**isx** : int, array-like, shape :math:\left(m\right)
Indicates which independent variables are to be included in the model.

If :math:\mathrm{isx}[j-1] > 0, the variable contained in the :math:j\ th column of :math:\mathrm{x} is included in the regression model.

**y** : float, array-like, shape :math:\left(n\right)
:math:y, observations on the dependent variable.

**link** : str, length 1, optional
Indicates which link function is to be used.

:math:\mathrm{link} = \texttt{'E'}

:math:\mathrm{link} = \texttt{'I'}

:math:\mathrm{link} = \texttt{'L'}

:math:\mathrm{link} = \texttt{'S'}

A square root link is used.

:math:\mathrm{link} = \texttt{'R'}

**mean** : str, length 1, optional
Indicates if a mean term is to be included.

:math:\mathrm{mean} = \texttt{'M'}

A mean term, intercept, will be included in the model.

:math:\mathrm{mean} = \texttt{'Z'}

The model will pass through the origin, zero-point.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
If provided :math:\mathrm{wt} must contain the weights to be used with the model.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If :math:\mathrm{wt} is not provided the effective number of observations is :math:n.

**a** : float, optional
If :math:\mathrm{link} = \texttt{'E'}, :math:\mathrm{a} must contain the power of the exponential.

If :math:\mathrm{link}\neq \texttt{'E'}, :math:\mathrm{a} is not referenced.

**v** : None or float, array-like, shape :math:\left(n, \textit{ip}+7\right), optional
If :math:\mathrm{v}\text{ is not }\mathbf{None}, :math:\mathrm{v}[\textit{i}-1,6], for :math:\textit{i} = 1,2,\ldots,n, must contain the offset values :math:o_{\textit{i}}.

All other values need not be set.

**tol** : float, optional
Indicates the accuracy required for the fit of the model.

The iterative weighted least squares procedure is deemed to have converged if the absolute change in deviance between iterations is less than :math:\mathrm{tol}\times \left(1.0+\text{Current Deviance}\right).

This is approximately an absolute precision if the deviance is small and a relative precision if the deviance is large.

If :math:0.0\leq \mathrm{tol} < \text{machine precision}, the function will use :math:10\times \text{machine precision} instead.

**maxit** : int, optional
The maximum number of iterations for the iterative weighted least squares.

If :math:\mathrm{maxit} = 0, a default value of :math:10 is used.

**iprint** : int, optional
Indicates if the printing of information on the iterations is required.

:math:\mathrm{iprint}\leq 0

There is no printing.

:math:\mathrm{iprint} > 0

Every :math:\mathrm{iprint} iteration, the following are printed:

the deviance;

the current estimates;

and if the weighted least squares equations are singular, then this is indicated.

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see :class:~naginterfaces.base.utils.FileObjManager).

**eps** : float, optional
The value of :math:\mathrm{eps} is used to decide if the independent variables are of full rank and, if not, what is the rank of the independent variables. The smaller the value of :math:\mathrm{eps} the stricter the criterion for selecting the singular value decomposition.

If :math:0.0\leq \mathrm{eps} < \text{machine precision}, the function will use machine precision instead.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**dev** : float
The deviance for the fitted model.

**idf** : int
The degrees of freedom asociated with the deviance for the fitted model.

**b** : float, ndarray, shape :math:\left(\textit{ip}\right)
The estimates of the parameters of the generalized linear model, :math:\hat{\beta }.

If :math:\mathrm{mean} = \texttt{'M'}, the first element of :math:\mathrm{b} will contain the estimate of the mean parameter and :math:\mathrm{b}[i] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

If :math:\mathrm{mean} = \texttt{'Z'}, :math:\mathrm{b}[i-1] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

**irank** : int
The rank of the independent variables.

If the model is of full rank, :math:\mathrm{irank} = \textit{ip}.

If the model is not of full rank, :math:\mathrm{irank} is an estimate of the rank of the independent variables. :math:\mathrm{irank} is calculated as the number of singular values greater that :math:\mathrm{eps}\times \text{}\ (largest singular value).

It is possible for the SVD to be carried out but for :math:\mathrm{irank} to be returned as :math:\textit{ip}.

**se** : float, ndarray, shape :math:\left(\textit{ip}\right)
The standard errors of the linear parameters.

:math:\mathrm{se}[\textit{i}-1] contains the standard error of the parameter estimate in :math:\mathrm{b}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,\textit{ip}.

**cov** : float, ndarray, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored packed by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1].

**v** : float, ndarray, shape :math:\left(n, \textit{ip}+7\right)
Auxiliary information on the fitted model.

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,0]  |contains the linear predictor value, :math:\eta_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                  |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,1]  |contains the fitted value, :math:\hat{\mu }_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                      |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,2]  |contains the variance standardization, :math:\frac{1}{\tau_{\textit{i}}}, for :math:\textit{i} = 1,2,\ldots,n.                                      |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,3]  |contains the square root of the working weight, :math:w_{\textit{i}}^{\frac{1}{2}}, for :math:\textit{i} = 1,2,\ldots,n.                            |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,4]  |contains the deviance residual, :math:r_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                          |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,5]  |contains the leverage, :math:h_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                                   |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,6]  |contains the offset, :math:o_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n. If :math:\mathrm{v} is **None** on entry, all values will be zero.|
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,j-1]|for :math:j = 8,\ldots,\textit{ip}+7, contains the results of the :math:QR decomposition or the singular value decomposition.                       |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+

If the model is not of full rank, i.e., :math:\mathrm{irank} < \textit{ip}, the first :math:\textit{ip} rows of columns :math:8 to :math:\textit{ip}+7 contain the :math:P^* matrix.

.. _g02gc-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:1)
On entry, :math:\mathrm{eps} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{eps}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} \geq 0.

(errno :math:1)
On entry, :math:\textit{offset} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{offset} = \texttt{'Y'} or :math:\texttt{'N'}.

(errno :math:1)
On entry, :math:\mathrm{link} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{link} = \texttt{'E'}, :math:\texttt{'I'}, :math:\texttt{'L'}, :math:\texttt{'S'} or :math:\texttt{'R'}.

(errno :math:1)
On entry, :math:\mathrm{a} = 0.0 and :math:\mathrm{link} = \texttt{'E'}.

Constraint: if :math:\mathrm{link} = \texttt{'E'}, :math:\mathrm{a}\neq 0.0.

(errno :math:1)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'W'} or :math:\texttt{'U'}.

(errno :math:1)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1] \geq 0.0, for :math:i = 1,2,\ldots,n.

(errno :math:3)
On entry, :math:\mathrm{isx}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.

Constraint: :math:\mathrm{isx}[j-1] \geq 0.0, for :math:j = 1,2,\ldots,m.

(errno :math:3)
On entry, :math:\textit{ip} incompatible with number of nonzero values of :math:\mathrm{isx}: :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:3)
Number of requested x-variables greater than :math:\textit{n}.

(errno :math:4)
On entry, :math:\mathrm{y}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{y}[i-1] \leq 0.0, for :math:i = 1,2,\ldots,n.

(errno :math:5)
A fitted value is at the boundary, i.e., :math:\hat{\mu } = 0.0. This may occur if there are :math:y values of :math:0.0 and the model is too complex for the data. The model should be reformulated with, perhaps, some observations dropped.

(errno :math:6)
SVD solution failed to converge.

(errno :math:7)
The iterative weighted least squares has failed to converge in :math:\mathrm{maxit} (or default :math:10) iterations. The value of :math:\mathrm{maxit} could be increased but it may be advantageous to examine the convergence using the :math:\mathrm{iprint} option. This may indicate that the convergence is slow because the solution is at a boundary in which case it may be better to reformulate the model.

**Warns**
**NagAlgorithmicWarning**
(errno :math:8)
The rank of the model has changed during the weighted least squares iterations. The estimate for :math:\beta returned may be reasonable, but you should check how the deviance has changed during iterations.

(errno :math:9)
The degrees of freedom for error are :math:0. A saturated model has been fitted.

.. _g02gc-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

A generalized linear model with Poisson errors consists of the following elements:

(a) a set of :math:n observations, :math:y_i, from a Poisson distribution:

.. math::
\frac{{\mu^ye^{{-\mu }}}}{{y!}}\text{.}

(#) :math:X, a set of :math:p independent variables for each observation, :math:x_1,x_2,\ldots,x_p.

(#) a linear model:

.. math::
\eta = \sum \beta_jx_j\text{.}

(#) a link between the linear predictor, :math:\eta, and the mean of the distribution, :math:\mu, :math:\eta = g\left(\mu \right). The possible link functions are:

(i) exponent link: :math:\eta = \mu^a, for a constant :math:a,

(#) identity link: :math:\eta = \mu,

(#) log link: :math:\eta = \log\left(\mu \right),

(#) square root link: :math:\eta = \sqrt{\mu },

(#) reciprocal link: :math:\eta = \frac{1}{\mu }.

(#) a measure of fit, the deviance:

.. math::
\sum_{{i = 1}}^n\mathrm{dev}\left(y_i,\hat{\mu }_i\right) = \sum_{{i = 1}}^n2\left(y_i\log\left(\frac{y_i}{\hat{\mu }_i}\right)-\left(y_i-\hat{\mu }_i\right)\right)\text{.}

The linear parameters are estimated by iterative weighted least squares.
An adjusted dependent variable, :math:z, is formed:

.. math::
z = \eta +\left(y-\mu \right)\frac{{\mathrm{d}\eta }}{{\mathrm{d}\mu }}

and a working weight, :math:w,

.. math::
w^{-1} = \left(\tau d\frac{{\mathrm{d}\eta }}{{\mathrm{d}\mu }}\right)^2\text{,}

where :math:\tau = \sqrt{\mu }.

At each iteration an approximation to the estimate of :math:\beta, :math:\hat{\beta }, is found by the weighted least squares regression of :math:z on :math:X with weights :math:w.

glm_poisson finds a :math:QR decomposition of :math:w^{{1/2}}X, i.e., :math:w^{{1/2}}X = QR where :math:R is a :math:p\times p triangular matrix and :math:Q is an :math:n\times p column orthogonal matrix.

If :math:R is of full rank, then :math:\hat{\beta } is the solution to:

.. math::
R\hat{\beta } = Q^\mathrm{T}w^{{1/2}}z\text{.}

If :math:R is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of :math:R.

.. math::
R = Q_*\begin{pmatrix}D&0\\0&0\end{pmatrix}P^\mathrm{T}\text{,}

where :math:D is a :math:k\times k diagonal matrix with nonzero diagonal elements, :math:k being the rank of :math:R and :math:w^{{1/2}}X.

This gives the solution

.. math::
\hat{\beta } = P_1D^{-1}\begin{pmatrix}Q_*&0\\0&I\end{pmatrix}Q^\mathrm{T}w^{{1/2}}z\text{,}

:math:P_1 being the first :math:k columns of :math:P, i.e., :math:P = \left(P_1P_0\right).

The iterations are continued until there is only a small change in the deviance.

The initial values for the algorithm are obtained by taking

.. math::
\hat{\eta } = g\left(y\right)\text{.}

The fit of the model can be assessed by examining and testing the deviance, in particular by comparing the difference in deviance between nested models, i.e., when one model is a sub-model of the other.
The difference in deviance between two nested models has, asymptotically, a :math:\chi^2-distribution with degrees of freedom given by the difference in the degrees of freedom associated with the two deviances.

The parameters estimates, :math:\hat{\beta }, are asymptotically Normally distributed with variance-covariance matrix

:math:C = R^{-1}{R^{-1}}^\mathrm{T} in the full rank case, otherwise

:math:C = P_1D^{-2}P_1^\mathrm{T}.

The residuals and influence statistics can also be examined.

The estimated linear predictor :math:\hat{\eta } = X\hat{\beta }, can be written as :math:Hw^{{1/2}}z for an :math:n\times n matrix :math:H.
The :math:i\ th diagonal elements of :math:H, :math:h_i, give a measure of the influence of the :math:i\ th values of the independent variables on the fitted regression model.
These are known as leverages.

The fitted values are given by :math:\hat{\mu } = g^{-1}\left(\hat{\eta }\right).

glm_poisson also computes the deviance residuals, :math:r:

.. math::
r_i = \mathrm{sign}\left(y_i-\hat{\mu }_i\right)\sqrt{\mathrm{dev}\left(y_i,\hat{\mu }_i\right)}\text{.}

An option allows prior weights to be used with the model.

In many linear regression models the first term is taken as a mean term or an intercept, i.e., :math:x_{{i,1}} = 1, for :math:i = 1,2,\ldots,n.
This is provided as an option.

Often only some of the possible independent variables are included in a model; the facility to select variables to be included in the model is provided.

If part of the linear predictor can be represented by a variables with a known coefficient then this can be included in the model by using an offset, :math:o:

.. math::
\eta = o+\sum \beta_jx_j\text{.}

If the model is not of full rank the solution given will be only one of the possible solutions.
Other estimates may be obtained by applying constraints to the parameters.
These solutions can be obtained by using :meth:glm_constrain after using glm_poisson.
Only certain linear combinations of the parameters will have unique estimates, these are known as estimable functions, these can be estimated and tested using :meth:glm_estfunc.

Details of the SVD are made available in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}\text{.}

The generalized linear model with Poisson errors can be used to model contingency table data; see Cook and Weisberg (1982) and McCullagh and Nelder (1983).

.. _g02gc-py2-py-references:

**References**
Cook, R D and Weisberg, S, 1982, Residuals and Influence in Regression, Chapman and Hall

McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall

Plackett, R L, 1974, The Analysis of Categorical Data, Griffin

--------
:meth:naginterfaces.library.examples.surviv.coxmodel_ex.main
"""
raise NotImplementedError

[docs]def glm_gamma(x, isx, y, link='R', mean='M', wt=None, s=0, a=0.0, v=None, tol=0.0, maxit=10, iprint=0, eps=0.0, io_manager=None):
r"""
glm_gamma fits a generalized linear model with gamma errors.

.. _g02gd-py2-py-doc:

For full information please refer to the NAG Library document for g02gd

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02gdf.html

.. _g02gd-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation for the :math:\textit{j}\ th independent variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**isx** : int, array-like, shape :math:\left(m\right)
Indicates which independent variables are to be included in the model.

If :math:\mathrm{isx}[j-1] > 0, the variable contained in the :math:j\ th column of :math:\mathrm{x} is included in the regression model.

**y** : float, array-like, shape :math:\left(n\right)
:math:y, the dependent variable.

**link** : str, length 1, optional
Indicates which link function is to be used.

:math:\mathrm{link} = \texttt{'E'}

:math:\mathrm{link} = \texttt{'I'}

:math:\mathrm{link} = \texttt{'L'}

:math:\mathrm{link} = \texttt{'S'}

A square root link is used.

:math:\mathrm{link} = \texttt{'R'}

**mean** : str, length 1, optional
Indicates if a mean term is to be included.

:math:\mathrm{mean} = \texttt{'M'}

A mean term, intercept, will be included in the model.

:math:\mathrm{mean} = \texttt{'Z'}

The model will pass through the origin, zero-point.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
If provided :math:\mathrm{wt} must contain the weights to be used with the model.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If :math:\mathrm{wt} is not provided the effective number of observations is :math:n.

**s** : float, optional
The scale parameter for the gamma model, :math:\nu^{-1}.

:math:\mathrm{s} = 0.0

The scale parameter is estimated with the function using the formula described in :ref:Notes <g02gd-py2-py-notes>.

**a** : float, optional
If :math:\mathrm{link} = \texttt{'E'}, :math:\mathrm{a} must contain the power of the exponential.

If :math:\mathrm{link} \neq \texttt{'E'}, :math:\mathrm{a} is not referenced.

**v** : None or float, array-like, shape :math:\left(n, \textit{ip}+7\right), optional
The offset values :math:o_{\textit{i}}, otherwise :math:\mathrm{v} need not be set and no offset is used.

**tol** : float, optional
Indicates the accuracy required for the fit of the model.

The iterative weighted least squares procedure is deemed to have converged if the absolute change in deviance between iterations is less than :math:\mathrm{tol}\times \left(1.0+\text{Current Deviance}\right).

This is approximately an absolute precision if the deviance is small and a relative precision if the deviance is large.

If :math:0.0\leq \mathrm{tol} < \text{machine precision} then the function will use :math:10\times \text{machine precision} instead.

**maxit** : int, optional
The maximum number of iterations for the iterative weighted least squares.

:math:\mathrm{maxit} = 0

A default value of :math:10 is used.

**iprint** : int, optional
Indicates if the printing of information on the iterations is required.

:math:\mathrm{iprint}\leq 0

There is no printing.

:math:\mathrm{iprint} > 0

Every :math:\mathrm{iprint} iteration, the following are printed:

the deviance;

the current estimates;

and if the weighted least squares equations are singular, then this is indicated.

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see :class:~naginterfaces.base.utils.FileObjManager).

**eps** : float, optional
The value of :math:\mathrm{eps} is used to decide if the independent variables are of full rank and, if not, what is the rank of the independent variables. The smaller the value of :math:\mathrm{eps} the stricter the criterion for selecting the singular value decomposition.

If :math:0.0\leq \mathrm{eps} < \text{machine precision}, the function will use machine precision instead.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**s** : float
If on input :math:\mathrm{s} = 0.0, :math:\mathrm{s} contains the estimated value of the scale parameter, :math:\hat{\nu }^{-1}.

If on input :math:\mathrm{s}\neq 0.0, :math:\mathrm{s} is unchanged on exit.

**dev** : float
The adjusted deviance for the fitted model.

**idf** : int
The degrees of freedom asociated with the deviance for the fitted model.

**b** : float, ndarray, shape :math:\left(\textit{ip}\right)
The estimates of the parameters of the generalized linear model, :math:\hat{\beta }.

If :math:\mathrm{mean} = \texttt{'M'}, the first element of :math:\mathrm{b} will contain the estimate of the mean parameter and :math:\mathrm{b}[i] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

If :math:\mathrm{mean} = \texttt{'Z'}, :math:\mathrm{b}[i-1] will contain the coefficient of the variable contained in column :math:j of :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

**irank** : int
The rank of the independent variables.

If the model is of full rank, :math:\mathrm{irank} = \textit{ip}.

If the model is not of full rank, :math:\mathrm{irank} is an estimate of the rank of the independent variables. :math:\mathrm{irank} is calculated as the number of singular values greater that :math:\mathrm{eps}\times \text{}\ (largest singular value).

It is possible for the SVD to be carried out but for :math:\mathrm{irank} to be returned as :math:\textit{ip}.

**se** : float, ndarray, shape :math:\left(\textit{ip}\right)
The standard errors of the linear parameters.

:math:\mathrm{se}[\textit{i}-1] contains the standard error of the parameter estimate in :math:\mathrm{b}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,\textit{ip}.

**cov** : float, ndarray, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored in packed form by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1].

**v** : float, ndarray, shape :math:\left(n, \textit{ip}+7\right)
Auxiliary information on the fitted model.

.. rst-class:: nag-rules-none nag-align-left

+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,0]  |contains the linear predictor value, :math:\eta_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                     |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,1]  |contains the fitted value, :math:\hat{\mu }_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                         |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,2]  |contains the variance standardization, :math:\frac{1}{\tau_{\textit{i}}}, for :math:\textit{i} = 1,2,\ldots,n.                                         |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,3]  |contains the square root of the working weight, :math:w_{\textit{i}}^{\frac{1}{2}}, for :math:\textit{i} = 1,2,\ldots,n.                               |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,4]  |contains the Anscombe residual, :math:r_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                             |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,5]  |contains the leverage, :math:h_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n.                                                                      |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,6]  |contains the offset, :math:o_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,n. If :math:\mathrm{v}\text{ is }\mathbf{None}, all values will be zero.|
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
|:math:\mathrm{v}[i-1,j-1]|for :math:j = 8,\ldots,\textit{ip}+7, contains the results of the :math:QR decomposition or the singular value decomposition.                          |
+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+

If the model is not of full rank, i.e., :math:\mathrm{irank} < \textit{ip}, the first :math:\textit{ip} rows of columns :math:8 to :math:\textit{ip}+7 contain the :math:P^* matrix.

.. _g02gd-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:1)
On entry, :math:\mathrm{eps} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{eps}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} \geq 0.

(errno :math:1)
On entry, :math:\textit{offset} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{offset} = \texttt{'Y'} or :math:\texttt{'N'}.

(errno :math:1)
On entry, :math:\mathrm{link} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{link} = \texttt{'E'}, :math:\texttt{'I'}, :math:\texttt{'L'}, :math:\texttt{'S'} or :math:\texttt{'R'}.

(errno :math:1)
On entry, :math:\mathrm{a} = 0.0 and :math:\mathrm{link} = \texttt{'E'}.

Constraint: if :math:\mathrm{link} = \texttt{'E'}, :math:\mathrm{a}\neq 0.0.

(errno :math:1)
On entry, :math:\mathrm{s} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{s}\geq 0.0.

(errno :math:1)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{weight} = \texttt{'W'} or :math:\texttt{'U'}.

(errno :math:1)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.0.

Constraint: :math:\mathrm{wt}[i-1] \geq 0.0, for :math:i = 1,2,\ldots,n.

(errno :math:3)
On entry, :math:\mathrm{isx}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.

Constraint: :math:\mathrm{isx}[j-1] \geq 0.0, for :math:j = 1,2,\ldots,m.

(errno :math:3)
On entry, :math:\textit{ip} incompatible with number of nonzero values of :math:\mathrm{isx}: :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:3)
Number of requested x-variables greater than :math:\textit{n}.

(errno :math:4)
On entry, :math:\mathrm{y}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{y}[i-1] \leq 0.0, for :math:i = 1,2,\ldots,n.

(errno :math:5)
A fitted value is at the boundary, i.e., :math:\hat{\mu } = 0.0. This may occur if there are :math:y values of :math:0.0 and the model is too complex for the data. The model should be reformulated with, perhaps, some observations dropped.

(errno :math:6)
SVD solution failed to converge.

(errno :math:7)
The iterative weighted least squares has failed to converge in :math:\mathrm{maxit} (or default :math:10) iterations. The value of :math:\mathrm{maxit} could be increased but it may be advantageous to examine the convergence using the :math:\mathrm{iprint} option. This may indicate that the convergence is slow because the solution is at a boundary in which case it may be better to reformulate the model.

**Warns**
**NagAlgorithmicWarning**
(errno :math:8)
The rank of the model has changed during the weighted least squares iterations. The estimate for :math:\beta returned may be reasonable, but you should check how the deviance has changed during iterations.

(errno :math:9)
The degrees of freedom for error are :math:0. A saturated model has been fitted.

.. _g02gd-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

A generalized linear model with gamma errors consists of the following elements:

(a) a set of :math:n observations, :math:y_i, from a gamma distribution with probability density function:

.. math::
\frac{1}{{\Gamma \left(\nu \right)}}\left(\frac{{\nu y}}{\mu }\right)^{\nu }\mathrm{exp}\left(-\frac{{\nu y}}{\mu }\right)\frac{1}{y}

:math:\nu being constant for the sample.

(#) :math:X, a set of :math:p independent variables for each observation, :math:x_1,x_2,\ldots,x_p.

(#) a linear model:

.. math::
\eta = \sum \beta_jx_j\text{.}

(#) a link between the linear predictor, :math:\eta, and the mean of the distribution, :math:\mu, :math:\eta = g\left(\mu \right). The possible link functions are:

(i) exponent link: :math:\eta = \mu^a, for a constant :math:a,

(#) identity link: :math:\eta = \mu,

(#) log link: :math:\eta = \log\left(\mu \right),

(#) square root link: :math:\eta = \sqrt{\mu },

(#) reciprocal link: :math:\eta = \frac{1}{\mu }.

(#) a measure of fit, an adjusted deviance. This is a function related to the deviance, but defined for :math:y = 0:

.. math::
\sum_{{i = 1}}^n\mathrm{dev}^*\left(y_i,\hat{\mu }_i\right) = \sum_{{i = 1}}^n2\left(\log\left(\hat{\mu }_i\right)+\left(\frac{y_i}{\hat{\mu }_i}\right)\right)\text{.}

The linear parameters are estimated by iterative weighted least squares.
An adjusted dependent variable, :math:z, is formed:

.. math::
z = \eta +\left(y-\mu \right)\frac{{d\eta }}{{d\mu }}

and a working weight, :math:w,

.. math::
w^{-1} = \left(\tau \frac{{d\eta }}{{d\mu }}\right)^2\text{, where }\quad \tau = \frac{1}{\mu }\text{.}

At each iteration an approximation to the estimate of :math:\beta, :math:\hat{\beta } is found by the weighted least squares regression of :math:z on :math:X with weights :math:w.

glm_gamma finds a :math:QR decomposition of :math:w^{{\frac{1}{2}}}X, i.e.,

:math:w^{{\frac{1}{2}}}X = QR where :math:R is a :math:p\times p triangular matrix and :math:Q is an :math:n\times p column orthogonal matrix.

If :math:R is of full rank then :math:\hat{\beta } is the solution to:

:math:R\hat{\beta } = Q^\mathrm{T}w^{{\frac{1}{2}}}z

If :math:R is not of full rank a solution is obtained by means of a singular value decomposition (SVD) of :math:R.

.. math::
R = Q_*\begin{pmatrix}D&0\\0&0\end{pmatrix}P^\mathrm{T}\text{.}

where :math:D is a :math:k\times k diagonal matrix with nonzero diagonal elements, :math:k being the rank of :math:R and :math:w^{{\frac{1}{2}}}X.

This gives the solution

.. math::
\hat{\beta } = P_1D^{-1}\begin{pmatrix}Q_*&0\\0&I\end{pmatrix}Q^\mathrm{T}w^{{\frac{1}{2}}}z\text{,}

where :math:P_1 is the first :math:k columns of :math:P, i.e., :math:P = \left(P_1P_0\right).

The iterations are continued until there is only a small change in the deviance.

The initial values for the algorithm are obtained by taking

.. math::
\hat{\eta } = g\left(y\right)\text{.}

The scale parameter, :math:\nu^{-1} is estimated by a moment estimator:

.. math::
\hat{\nu }^{-1} = \sum_{{i = 1}}^n\frac{{{\left[\left(y_i-\hat{\mu }_i\right)/\hat{\mu }_i\right]}^2}}{\left(n-k\right)}\text{.}

The fit of the model can be assessed by examining and testing the deviance, in particular, by comparing the difference in deviance between nested models, i.e., when one model is a sub-model of the other.
The difference in deviance or adjusted deviance between two nested models with known :math:\nu has, asymptotically, a :math:\chi^2-distribution with degrees of freedom given by the difference in the degrees of freedom associated with the two deviances.

The parameters estimates, :math:\hat{\beta }, are asymptotically Normally distributed with variance-covariance matrix:

:math:C = R^{-1}{R^{-1}}^\mathrm{T}\nu^{-1} in the full rank case, otherwise

:math:C = P_1D^{-2}P_1^\mathrm{T}\nu^{-1}.

The residuals and influence statistics can also be examined.

The estimated linear predictor :math:\hat{\eta } = X\hat{\beta }, can be written as :math:Hw^{{\frac{1}{2}}}z for an :math:n\times n matrix :math:H.
The :math:i\ th diagonal elements of :math:H, :math:h_i, give a measure of the influence of the :math:i\ th values of the independent variables on the fitted regression model.
These are known as leverages.

The fitted values are given by :math:\hat{\mu } = g^{-1}\left(\hat{\eta }\right).

glm_gamma also computes the Anscombe residuals, :math:r:

.. math::
r_i = \frac{{3\left(y_i^{{\frac{1}{3}}}-\hat{\mu }_i^{{\frac{1}{3}}}\right)}}{{\hat{\mu }_i^{{\frac{1}{3}}}}}\text{.}

An option allows the use of prior weights, :math:\omega_i.
This gives a model with:

.. math::
\nu_i = \nu \omega_i\text{.}

In many linear regression models the first term is taken as a mean term or an intercept, i.e., :math:x_{{i,1}} = 1, for :math:i = 1,2,\ldots,n.
This is provided as an option.

Often only some of the possible independent variables are included in a model, the facility to select variables to be included in the model is provided.

If part of the linear predictor can be represented by a variables with a known coefficient then this can be included in the model by using an offset, :math:o:

.. math::
\eta = o+\sum \beta_jx_j\text{.}

If the model is not of full rank the solution given will be only one of the possible solutions.
Other estimates may be obtained by applying constraints to the parameters.
These solutions can be obtained by using :meth:glm_constrain after using glm_gamma.
Only certain linear combinations of the parameters will have unique estimates, these are known as estimable functions, and can be estimated and tested using :meth:glm_estfunc.

Details of the SVD are made available in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix} D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}\text{.}

.. _g02gd-py2-py-references:

**References**
Cook, R D and Weisberg, S, 1982, Residuals and Influence in Regression, Chapman and Hall

McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall
"""
raise NotImplementedError

[docs]def glm_constrain(v, c, b, s):
r"""
glm_constrain calculates the estimates of the parameters of a generalized linear model for given constraints from the singular value decomposition results.

.. _g02gk-py2-py-doc:

For full information please refer to the NAG Library document for g02gk

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02gkf.html

.. _g02gk-py2-py-parameters:

**Parameters**
**v** : float, array-like, shape :math:\left(\textit{ip}, \textit{ip}+7\right)
The array :math:\mathrm{v} as returned by :meth:glm_normal, :meth:glm_binomial, :meth:glm_poisson or :meth:glm_gamma.

**c** : float, array-like, shape :math:\left(\textit{ip}, \textit{iconst}\right)
Contains the :math:\textit{iconst} constraints stored by column, i.e., the :math:i\ th constraint is stored in the :math:i\ th column of :math:\mathrm{c}.

**b** : float, array-like, shape :math:\left(\textit{ip}\right)
The parameter estimates computed by using the singular value decomposition, :math:\hat{\beta }_{\text{svd}}.

**s** : float
The estimate of the scale parameter.

For results from :meth:glm_normal and :meth:glm_gamma then :math:\mathrm{s} is the scale parameter for the model.

For results from :meth:glm_binomial and :meth:glm_poisson then :math:\mathrm{s} should be set to :math:1.0.

**Returns**
**b** : float, ndarray, shape :math:\left(\textit{ip}\right)
The parameter estimates of the parameters with the constraints imposed, :math:\hat{\beta }_{\mathrm{c}}.

**se** : float, ndarray, shape :math:\left(\textit{ip}\right)
The standard error of the parameter estimates in :math:\mathrm{b}.

**cov** : float, ndarray, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored packed by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1].

.. _g02gk-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{iconst} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{iconst} > 0.

(errno :math:1)
On entry, :math:\mathrm{s} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{s} > 0.0.

(errno :math:1)
On entry, :math:\textit{iconst} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{iconst} < \textit{ip}.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

(errno :math:2)
:math:\mathrm{c} does not give a model of full rank.

.. _g02gk-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

glm_constrain computes the estimates given a set of linear constraints for a generalized linear model which is not of full rank.
It is intended for use after a call to :meth:glm_normal, :meth:glm_binomial, :meth:glm_poisson or :meth:glm_gamma.

In the case of a model not of full rank the functions use a singular value decomposition to find the parameter estimates, :math:\hat{\beta }_{\text{svd}}, and their variance-covariance matrix.
Details of the SVD are made available in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}

as described by :meth:glm_normal, :meth:glm_binomial, :meth:glm_poisson and :meth:glm_gamma.
Alternative solutions can be formed by imposing constraints on the parameters.
If there are :math:p parameters and the rank of the model is :math:k then :math:n_{\mathrm{c}} = p-k constraints will have to be imposed to obtain a unique solution.

Let :math:C be a :math:p\times n_{\mathrm{c}} matrix of constraints, such that

.. math::
C^\mathrm{T}\beta = 0\text{,}

then the new parameter estimates :math:\hat{\beta }_{\mathrm{c}} are given by:

.. math::
\begin{array}{ll}\hat{\beta }_{\mathrm{c}}& = A\hat{\beta }_{\mathrm{svd}}\\& = \left(I-P_0\left(C^\mathrm{T}P_0\right)^{-1}\right)\hat{\beta }_{\mathrm{svd}}\text{, where }I\text{ is the identity matrix,}\end{array}

and the variance-covariance matrix is given by

.. math::
AP_1D^{-2}P_1^\mathrm{T}A^\mathrm{T}

provided :math:\left(C^\mathrm{T}P_0\right)^{-1} exists.

.. _g02gk-py2-py-references:

**References**
Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall

Searle, S R, 1971, Linear Models, Wiley
"""
raise NotImplementedError

[docs]def glm_estfunc(irank, b, cov, v, f, tol=0.0):
r"""
glm_estfunc gives the estimate of an estimable function along with its standard error from the results from fitting a generalized linear model.

.. _g02gn-py2-py-doc:

For full information please refer to the NAG Library document for g02gn

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02gnf.html

.. _g02gn-py2-py-parameters:

**Parameters**
**irank** : int
:math:k, the rank of the dependent variables.

**b** : float, array-like, shape :math:\left(\textit{ip}\right)
The :math:\textit{ip} values of the estimates of the parameters of the model, :math:\hat{\beta }.

**cov** : float, array-like, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix of the :math:\textit{ip} parameter estimates given in :math:\mathrm{b}. They are stored packed by column, i.e., the covariance between the parameter estimate given in :math:\mathrm{b}[i-1] and the parameter estimate given in :math:\mathrm{b}[j-1], :math:j\geq i, is stored in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1].

**v** : float, array-like, shape :math:\left(\textit{ip}, \textit{ip}+7\right)
As returned by :meth:glm_normal, :meth:glm_binomial, :meth:glm_poisson and :meth:glm_gamma.

**f** : float, array-like, shape :math:\left(\textit{ip}\right)
:math:f, the linear function to be estimated.

**tol** : float, optional
The tolerance value used in the check for estimability, :math:\eta.

If :math:\mathrm{tol}\leq 0.0 then :math:\sqrt{\epsilon }, where :math:\epsilon is the machine precision, is used instead.

**Returns**
**est** : bool
Indicates if the function was estimable.

:math:\mathrm{est} = \mathbf{True}

The function is estimable.

:math:\mathrm{est} = \mathbf{False}

The function is not estimable and :math:\mathrm{stat}, :math:\mathrm{sestat} and :math:\mathrm{z} are not set.

**stat** : float
If :math:\mathrm{est} = \mathbf{True}, :math:\mathrm{stat} contains the estimate of the function, :math:f^\mathrm{T}\hat{\beta }

**sestat** : float
If :math:\mathrm{est} = \mathbf{True}, :math:\mathrm{sestat} contains the standard error of the estimate of the function, :math:\mathrm{se}\left(F\right).

**z** : float
If :math:\mathrm{est} = \mathbf{True}, :math:\mathrm{z} contains the :math:z statistic for the test of the function being equal to zero.

.. _g02gn-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{irank} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{irank} \leq \textit{ip}.

(errno :math:1)
On entry, :math:\mathrm{irank} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{irank} \geq 1.

(errno :math:1)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} \geq 1.

**Warns**
**NagAlgorithmicWarning**
(errno :math:2)
:math:\mathrm{irank} = \textit{ip}. In this case :math:\mathrm{est} is returned as true and all statistics are calculated.

(errno :math:3)
Standard error of statistic :math:\text{} = 0.0; this may be due to rounding errors if the standard error is very small or due to mis-specified inputs :math:\mathrm{cov} and :math:\mathrm{f}.

.. _g02gn-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

glm_estfunc computes the estimates of an estimable function for a generalized linear model which is not of full rank.
It is intended for use after a call to :meth:glm_normal, :meth:glm_binomial, :meth:glm_poisson or :meth:glm_gamma.
An estimable function is a linear combination of the parameters such that it has a unique estimate.
For a full rank model all linear combinations of parameters are estimable.

In the case of a model not of full rank the functions use a singular value decomposition (SVD) to find the parameter estimates, :math:\hat{\beta }, and their variance-covariance matrix.
Given the upper triangular matrix :math:R obtained from the :math:QR decomposition of the independent variables the SVD gives

.. math::
R = Q_*\begin{pmatrix}D&0\\0&0\end{pmatrix}P^\mathrm{T}\text{,}

where :math:D is a :math:k\times k diagonal matrix with nonzero diagonal elements, :math:k being the rank of :math:R, and :math:Q_* and :math:P are :math:p\times p orthogonal matrices.

.. math::
\hat{\beta } = P_1D^{-1}Q_{*_1}^\mathrm{T}c_1\text{,}

:math:P_1 being the first :math:k columns of :math:P, i.e., :math:P = \left(P_1P_0\right); :math:Q_{*_1} being the first :math:k columns of :math:Q_*, and :math:c_1 being the first :math:p elements of :math:c.

Details of the SVD are made available in the form of the matrix :math:P^*:

.. math::
P^* = \begin{pmatrix}D^{-1} P_1^\mathrm{T} \\ P_0^\mathrm{T} \end{pmatrix}

as described by :meth:glm_normal, :meth:glm_binomial, :meth:glm_poisson and :meth:glm_gamma.

A linear function of the parameters, :math:F = f^\mathrm{T}\beta, can be tested to see if it is estimable by computing :math:\zeta = P_0^\mathrm{T}f.
If :math:\zeta is zero, then the function is estimable, if not; the function is not estimable.
In practice :math:\left\lvert \zeta \right\rvert is tested against some small quantity :math:\eta.

Given that :math:F is estimable it can be estimated by :math:f^\mathrm{T}\hat{\beta } and its standard error calculated from the variance-covariance matrix of :math:\hat{\beta }, :math:C_{\beta }, as

.. math::
\mathrm{se}\left(F\right) = \sqrt{f^\mathrm{T}C_{\beta }f}\text{.}

Also a :math:z statistic

.. math::
z = \frac{{f^\mathrm{T}\hat{\beta }}}{{\mathrm{se}\left(F\right)}}\text{,}

can be computed.
The distribution of :math:z will be approximately Normal.

.. _g02gn-py2-py-references:

**References**
Golub, G H and Van Loan, C F, 1996, Matrix Computations, (3rd Edition), Johns Hopkins University Press, Baltimore

McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall

Searle, S R, 1971, Linear Models, Wiley
"""
raise NotImplementedError

[docs]def glm_predict(errfn, x, isx, b, cov, vfobs, link=None, mean='M', t=None, off=None, wt=None, s=0.0, a=0.0):
r"""
glm_predict allows prediction from a generalized linear model fit via :meth:glm_normal, :meth:glm_binomial, :meth:glm_poisson or :meth:glm_gamma or a linear model fit via :meth:linregm_fit.

.. _g02gp-py2-py-doc:

For full information please refer to the NAG Library document for g02gp

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02gpf.html

.. _g02gp-py2-py-parameters:

**Parameters**
**errfn** : str, length 1
Indicates the distribution used to model the dependent variable, :math:y.

:math:\mathrm{errfn} = \texttt{'B'}

The binomial distribution is used.

:math:\mathrm{errfn} = \texttt{'G'}

The gamma distribution is used.

:math:\mathrm{errfn} = \texttt{'N'}

The Normal (Gaussian) distribution is used.

:math:\mathrm{errfn} = \texttt{'P'}

The Poisson distribution is used.

**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation for the :math:\textit{j}\ th independent variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**isx** : int, array-like, shape :math:\left(m\right)
Indicates which independent variables are to be included in the model.

If :math:\mathrm{isx}[j-1] > 0, the variable contained in the :math:j\ th column of :math:\mathrm{x} is included in the regression model.

**b** : float, array-like, shape :math:\left(\textit{ip}\right)
The model parameters, :math:\beta.

If :math:\mathrm{mean} = \texttt{'M'}, :math:\mathrm{b}[0] must contain the mean parameter and :math:\mathrm{b}[i] the coefficient of the variable contained in the :math:j\ th independent :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

If :math:\mathrm{mean} = \texttt{'Z'}, :math:\mathrm{b}[i-1] must contain the coefficient of the variable contained in the :math:j\ th independent :math:\mathrm{x}, where :math:\mathrm{isx}[j-1] is the :math:i\ th positive value in the array :math:\mathrm{isx}.

**cov** : float, array-like, shape :math:\left(\textit{ip}\times \left(\textit{ip}+1\right)/2\right)
The upper triangular part of the variance-covariance matrix, :math:C, of the model parameters. This matrix should be supplied packed by column, i.e., the covariance between parameters :math:\beta_i and :math:\beta_j, that is the values stored in :math:\mathrm{b}[i-1] and :math:\mathrm{b}[j-1], should be supplied in :math:\mathrm{cov}[\textit{j}\times \left(\textit{j}-1\right)/2+\textit{i}-1], for :math:\textit{j} = \textit{i},\ldots,\textit{ip}, for :math:\textit{i} = 1,2,\ldots,\textit{ip}.

**vfobs** : bool
If :math:\mathrm{vfobs} = \mathbf{True}, the variance of future observations is included in the standard error of the predicted variable (i.e., :math:I_{\mathrm{fobs}} = 1), otherwise :math:I_{\mathrm{fobs}} = 0.

**link** : None or str, length 1, optional
Note: if this argument is **None** then a default value will be used, determined as follows: if :math:\mathrm{errfn} = \texttt{'B'}: :math:{ \texttt{'G'} }; if :math:\mathrm{errfn} = \texttt{'G'}: :math:{ \texttt{'R'} }; if :math:\mathrm{errfn} = \texttt{'P'}: :math:{ \texttt{'L'} }; otherwise: :math:{ \texttt{'I'} }.

Indicates which link function is to be used.

:math:\mathrm{link} = \texttt{'C'}

A complementary log-log link is used.

:math:\mathrm{link} = \texttt{'E'}

:math:\mathrm{link} = \texttt{'G'}

:math:\mathrm{link} = \texttt{'I'}

:math:\mathrm{link} = \texttt{'L'}

:math:\mathrm{link} = \texttt{'P'}

:math:\mathrm{link} = \texttt{'R'}

:math:\mathrm{link} = \texttt{'S'}

A square root link is used.

Details on the functional form of the different links can be found in the G02 Introduction <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02intro.html>__.

**mean** : str, length 1, optional
Indicates if a mean term is to be included.

:math:\mathrm{mean} = \texttt{'M'}

A mean term, intercept, will be included in the model.

:math:\mathrm{mean} = \texttt{'Z'}

The model will pass through the origin, zero-point.

**t** : None or float, array-like, shape :math:\left(:\right), optional
Note: the required length for this argument is determined as follows: if :math:\mathrm{errfn}=\texttt{'B'}: :math:n; otherwise: :math:0.

If :math:\mathrm{errfn} = \texttt{'B'}, :math:\mathrm{t}[i-1] must contain the binomial denominator, :math:t_i, for the :math:i\ th observation.

Otherwise :math:\mathrm{t} is not referenced and may be **None**.

**off** : None or float, array-like, shape :math:\left(:\right), optional
Note: the required length for this argument is determined as follows: if :math:\mathrm{off}\text{ is not }\mathbf{None}: :math:n; otherwise: :math:0.

If an offset is required, then :math:\mathrm{off}[i-1] must contain the value of the offset :math:o_i, for the :math:i\ th observation. Otherwise :math:\mathrm{off} must be **None**.

**wt** : None or float, array-like, shape :math:\left(:\right), optional
Note: the required length for this argument is determined as follows: if :math:\mathrm{wt}\text{ is not }\mathbf{None}\text{ and }\mathrm{vfobs}= \mathbf{True}: :math:n; otherwise: :math:0.

If weighted estimates are required then :math:\mathrm{wt}[i-1] must contain the weight, :math:\omega_i for the :math:i\ th observation. Otherwise :math:\mathrm{wt} must be supplied as **None**.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with positive weights.

If :math:\mathrm{wt} = \text{} **None**, the effective number of observations is :math:n.

If the variance of future observations is not included in the standard error of the predicted variable, :math:\mathrm{wt} is not referenced.

**s** : float, optional
If :math:\mathrm{errfn} = \texttt{'N'} or :math:\texttt{'G'} and :math:\mathrm{vfobs} = \mathbf{True}, the scale parameter, :math:\phi.

Otherwise :math:\mathrm{s} is not referenced and :math:\phi = 1.

**a** : float, optional
If :math:\mathrm{link} = \texttt{'E'}, :math:\mathrm{a} must contain the power of the exponential.

If :math:\mathrm{link} \neq \texttt{'E'}, :math:\mathrm{a} is not referenced.

**Returns**
**eta** : float, ndarray, shape :math:\left(n\right)
The linear predictor, :math:\eta.

**seeta** : float, ndarray, shape :math:\left(n\right)
The standard error of the linear predictor, :math:\mathrm{se}\left(\eta \right).

**pred** : float, ndarray, shape :math:\left(n\right)
The predicted value, :math:\hat{y}.

**sepred** : float, ndarray, shape :math:\left(n\right)
The standard error of the predicted value, :math:\mathrm{se}\left(\hat{y}\right). If :math:\mathrm{pred}[i-1] could not be calculated, glm_predict returns :math:\mathrm{errno} = 22, and :math:\mathrm{sepred}[i-1] is set to :math:{-99.0}.

.. _g02gp-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{errfn} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{errfn} = \texttt{'B'}, :math:\texttt{'G'}, :math:\texttt{'N'} or :math:\texttt{'P'}.

(errno :math:2)
On entry, :math:\mathrm{link} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: if :math:\mathrm{errfn} = \texttt{'B'}, :math:\mathrm{link} = \texttt{'C'}, :math:\texttt{'G'} or :math:\texttt{'P'},

otherwise, :math:\mathrm{link} = \texttt{'E'}, :math:\texttt{'I'}, :math:\texttt{'L'}, :math:\texttt{'R'} or :math:\texttt{'S'}.

(errno :math:2)
On entry, :math:\mathrm{errfn} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{link} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: if :math:\mathrm{errfn} = \texttt{'B'}, :math:\mathrm{link} = \texttt{'C'}, :math:\texttt{'G'} or :math:\texttt{'P'},

otherwise, :math:\mathrm{link} = \texttt{'E'}, :math:\texttt{'I'}, :math:\texttt{'L'}, :math:\texttt{'R'} or :math:\texttt{'S'}.

(errno :math:3)
On entry, :math:\mathrm{mean} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{mean} = \texttt{'M'} or :math:\texttt{'Z'}.

(errno :math:5)
On entry, :math:\textit{weight} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: if :math:\mathrm{vfobs} = \mathbf{True}, :math:\textit{weight} = \texttt{'U'} or :math:\texttt{'W'}.

(errno :math:6)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 1.

(errno :math:9)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:10)
On entry, :math:\mathrm{isx}[\langle\mathit{\boldsymbol{value}}\rangle] < 0.

Constraint: :math:\mathrm{isx}[j-1] \geq 0.0, for :math:j = 1,2,\ldots,m.

(errno :math:11)
On entry, :math:\textit{ip} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ip} > 0.

(errno :math:12)
On entry, :math:\mathrm{t}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{t}[i-1] \geq 0.0, for all :math:i.

(errno :math:14)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{wt}[i-1] \geq 0.0, for all :math:i.

(errno :math:15)
On entry, :math:\mathrm{s} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{s} > 0.0.

(errno :math:16)
On entry, :math:\mathrm{a} = 0.0.

Constraint: if :math:\mathrm{link} = \texttt{'E'}, :math:\mathrm{a}\neq 0.0.

(errno :math:18)
On entry, :math:\mathrm{cov}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{cov}[i] \geq 0.0 for at least one diagonal element.

**Warns**
**NagAlgorithmicWarning**
(errno :math:22)
At least one predicted value could not be calculated as required. :math:\mathrm{sepred} is set to :math:-99.0 for affected predicted values.

.. _g02gp-py2-py-notes:

**Notes**
A generalized linear model consists of the following elements:

(i) A suitable distribution for the dependent variable :math:y.

(#) A linear model, with linear predictor :math:\eta = X\beta, where :math:X is a matrix of independent variables and :math:\beta a column vector of :math:p parameters.

(#) A link function :math:g\left(.\right) between the expected value of :math:y and the linear predictor, that is :math:E\left(y\right) = \mu = g\left(\eta \right).

In order to predict from a generalized linear model, that is estimate a value for the dependent variable, :math:y, given a set of independent variables :math:X, the matrix :math:X must be supplied, along with values for the parameters :math:\beta and their associated variance-covariance matrix, :math:C.
Suitable values for :math:\beta and :math:C are usually estimated by first fitting the prediction model to a training dataset with known responses, using for example :meth:glm_normal, :meth:glm_binomial, :meth:glm_poisson or :meth:glm_gamma.
The predicted variable, and its standard error can then be obtained from:

.. math::
\hat{y} = g^{-1}\left(\eta \right)\text{, }\quad \mathrm{se}\left(\hat{y}\right) = \sqrt{{\left(\frac{{\delta g^{-1}\left(x\right)}}{{\delta x}}\right)}_{\eta }}\mathrm{se}\left(\eta \right)+I_{\mathrm{fobs}}\mathrm{Var}\left(y\right)

where

.. math::
\eta = o+X\beta \text{, }\quad \mathrm{se}\left(\eta \right) = \mathrm{diag}\left(\sqrt{XCX^\mathrm{T}}\right)\text{,}

:math:o is a vector of offsets and :math:I_{\mathrm{fobs}} = 0, if the variance of future observations is not taken into account, and :math:1 otherwise.
Here :math:\mathrm{diag}\left(A\right) indicates the diagonal elements of matrix :math:A.

If required, the variance for the :math:i\ th future observation, :math:\mathrm{Var}\left(y_i\right), can be calculated as:

.. math::
\mathrm{Var}\left(y_i\right) = \frac{{\phi V\left(\theta \right)}}{{w_i}}

where :math:w_i is a weight, :math:\phi is the scale (or dispersion) parameter, and :math:V\left(\theta \right) is the variance function.
Both the scale parameter and the variance function depend on the distribution used for the :math:y, with:

.. rst-class:: nag-rules-none nag-align-left

+--------+---------------------------------------------------------------------------------------------+
|Poisson |:math:V\left(\theta \right) = \mu_i, :math:\phi = 1                                      |
+--------+---------------------------------------------------------------------------------------------+
|binomial|:math:V\left(\theta \right) = \frac{{\mu_i\left(t_i-\mu_i\right)}}{{t_i}}, :math:\phi = 1|
+--------+---------------------------------------------------------------------------------------------+
|Normal  |:math:V\left(\theta \right) = 1                                                            |
+--------+---------------------------------------------------------------------------------------------+
|gamma   |:math:V\left(\theta \right) = \mu_i^2                                                      |
+--------+---------------------------------------------------------------------------------------------+

In the cases of a Normal and gamma error structure, the scale parameter (:math:\phi), is supplied by you.
This value is usually obtained from the function used to fit the prediction model.
In many cases, for a Normal error structure, :math:\phi = \hat{\sigma }^2, i.e., the estimated variance.

.. _g02gp-py2-py-references:

**References**
McCullagh, P and Nelder, J A, 1983, Generalized Linear Models, Chapman and Hall

--------
:meth:naginterfaces.library.examples.correg.glm_binomial_ex.main
"""
raise NotImplementedError

[docs]def robustm(indw, ipsi, isigma, indc, x, y, cpsi, h1, h2, h3, cucv, dchi, theta, sigma, tol=5e-5, maxit=50, nitmon=0, io_manager=None):
r"""
robustm performs bounded influence regression (:math:M-estimates).
Several standard methods are available.

.. _g02ha-py2-py-doc:

For full information please refer to the NAG Library document for g02ha

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02haf.html

.. _g02ha-py2-py-parameters:

**Parameters**
**indw** : int
Specifies the type of regression to be performed.

:math:\mathrm{indw} < 0

Mallows type regression with Maronna's proposed weights.

:math:\mathrm{indw} = 0

Huber type regression.

:math:\mathrm{indw} > 0

Schweppe type regression with Krasker--Welsch weights.

**ipsi** : int
Specifies which :math:\psi function is to be used.

:math:\mathrm{ipsi} = 0

:math:\psi \left(t\right) = t, i.e., least squares.

:math:\mathrm{ipsi} = 1

Huber's function.

:math:\mathrm{ipsi} = 2

Hampel's piecewise linear function.

:math:\mathrm{ipsi} = 3

Andrew's sine wave.

:math:\mathrm{ipsi} = 4

Tukey's bi-weight.

**isigma** : int
Specifies how :math:\sigma is to be estimated.

:math:\mathrm{isigma} < 0

:math:\sigma is estimated by median absolute deviation of residuals.

:math:\mathrm{isigma} = 0

:math:\sigma is held constant at its initial value.

:math:\mathrm{isigma} > 0

:math:\sigma is estimated using the :math:\chi function.

**indc** : int
If :math:\mathrm{indw} \neq 0, :math:\mathrm{indc} specifies the approximations used in estimating the covariance matrix of :math:\hat{\theta }.

:math:\mathrm{indc} = 1

Averaging over residuals.

:math:\mathrm{indc} \neq 1

Replacing expected by observed.

:math:\mathrm{indw} = 0

:math:\mathrm{indc} is not referenced.

**x** : float, array-like, shape :math:\left(n, m\right)
The values of the :math:X matrix, i.e., the independent variables. :math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\textit{j}\ th element of :math:X, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

If :math:\mathrm{indw} < 0, then during calculations the elements of :math:\mathrm{x} will be transformed as described in :ref:Notes <g02ha-py2-py-notes>.

Before exit the inverse transformation will be applied.

As a result there may be slight differences between the input :math:\mathrm{x} and the output :math:\mathrm{x}.

**y** : float, array-like, shape :math:\left(n\right)
The data values of the dependent variable.

:math:\mathrm{y}[\textit{i}-1] must contain the value of :math:y for the :math:\textit{i}\ th observation, for :math:\textit{i} = 1,2,\ldots,n.

If :math:\mathrm{indw} < 0, then during calculations the elements of :math:\mathrm{y} will be transformed as described in :ref:Notes <g02ha-py2-py-notes>.

Before exit the inverse transformation will be applied.

As a result there may be slight differences between the input :math:\mathrm{y} and the output :math:\mathrm{y}.

**cpsi** : float
If :math:\mathrm{ipsi} = 1, :math:\mathrm{cpsi} must specify the parameter, :math:c, of Huber's :math:\psi function.

If :math:\mathrm{ipsi} \neq 1 on entry, :math:\mathrm{cpsi} is not referenced.

**h1** : float
If :math:\mathrm{ipsi} = 2, :math:\mathrm{h1}, :math:\mathrm{h2}, and :math:\mathrm{h3} must specify the parameters :math:h_1, :math:h_2, and :math:h_3, of Hampel's piecewise linear :math:\psi function. :math:\mathrm{h1}, :math:\mathrm{h2}, and :math:\mathrm{h3} are not referenced if :math:\mathrm{ipsi}\neq 2.

**h2** : float
If :math:\mathrm{ipsi} = 2, :math:\mathrm{h1}, :math:\mathrm{h2}, and :math:\mathrm{h3} must specify the parameters :math:h_1, :math:h_2, and :math:h_3, of Hampel's piecewise linear :math:\psi function. :math:\mathrm{h1}, :math:\mathrm{h2}, and :math:\mathrm{h3} are not referenced if :math:\mathrm{ipsi}\neq 2.

**h3** : float
If :math:\mathrm{ipsi} = 2, :math:\mathrm{h1}, :math:\mathrm{h2}, and :math:\mathrm{h3} must specify the parameters :math:h_1, :math:h_2, and :math:h_3, of Hampel's piecewise linear :math:\psi function. :math:\mathrm{h1}, :math:\mathrm{h2}, and :math:\mathrm{h3} are not referenced if :math:\mathrm{ipsi}\neq 2.

**cucv** : float
If :math:\mathrm{indw} < 0, must specify the value of the constant, :math:c, of the function :math:u for Maronna's proposed weights.

If :math:\mathrm{indw} > 0, must specify the value of the function :math:u for the Krasker--Welsch weights.

If :math:\mathrm{indw} = 0, is not referenced.

**dchi** : float
:math:d, the constant of the :math:\chi function. :math:\mathrm{dchi} is not referenced if :math:\mathrm{ipsi} = 0, or if :math:\mathrm{isigma} \leq 0.

**theta** : float, array-like, shape :math:\left(m\right)
Starting values of the parameter vector :math:\theta. These may be obtained from least squares regression. Alternatively if :math:\mathrm{isigma} < 0 and :math:\mathrm{sigma} = 1 or if :math:\mathrm{isigma} > 0 and :math:\mathrm{sigma} approximately equals the standard deviation of the dependent variable, :math:y, then :math:\mathrm{theta}[\textit{i}-1] = 0.0, for :math:\textit{i} = 1,2,\ldots,m may provide reasonable starting values.

**sigma** : float
A starting value for the estimation of :math:\sigma. :math:\mathrm{sigma} should be approximately the standard deviation of the residuals from the model evaluated at the value of :math:\theta given by :math:\mathrm{theta} on entry.

**tol** : float, optional
The relative precision for the calculation of :math:A (if :math:\mathrm{indw} \neq 0), the estimates of :math:\theta and the estimate of :math:\sigma (if :math:\mathrm{isigma} \neq 0). Convergence is assumed when the relative change in all elements being considered is less than :math:\mathrm{tol}.

If :math:\mathrm{indw} < 0 and :math:\mathrm{isigma} < 0, :math:\mathrm{tol} is also used to determine the precision of :math:\beta_1.

It is advisable for :math:\mathrm{tol} to be greater than :math:100\times \text{machine precision}.

**maxit** : int, optional
The maximum number of iterations that should be used in the calculation of :math:A (if :math:\mathrm{indw} \neq 0), and of the estimates of :math:\theta and :math:\sigma, and of :math:\beta_1 (if :math:\mathrm{indw} < 0 and :math:\mathrm{isigma} < 0).

A value of :math:\mathrm{maxit} = 50 should be adequate for most uses.

**nitmon** : int, optional
The amount of information that is printed on each iteration.

:math:\mathrm{nitmon} = 0

No information is printed.

:math:\mathrm{nitmon}\neq 0

The current estimate of :math:\theta, the change in :math:\theta during the current iteration and the current value of :math:\sigma are printed on the first and every :math:\mathrm{abs}\left(\mathrm{nitmon}\right) iterations.

Also, if :math:\mathrm{indw} \neq 0 and :math:\mathrm{nitmon} > 0, then information on the iterations to calculate :math:A is printed.

This is the current estimate of :math:A and the maximum value of :math:S_{{ij}} (see :ref:Notes <g02ha-py2-py-notes>).

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see :class:~naginterfaces.base.utils.FileObjManager).

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**x** : float, ndarray, shape :math:\left(n, m\right)
Unchanged, except as described above.

**y** : float, ndarray, shape :math:\left(n\right)
Unchanged, except as described above.

**theta** : float, ndarray, shape :math:\left(m\right)
:math:\mathrm{theta}[\textit{i}-1] contains the M-estimate of :math:\theta_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,m.

**sigma** : float
Contains the final estimate of :math:\sigma if :math:\mathrm{isigma} \neq 0 or the value assigned on entry if :math:\mathrm{isigma} = 0.

**c** : float, ndarray, shape :math:\left(m, m\right)
The diagonal elements of :math:\mathrm{c} contain the estimated asymptotic standard errors of the estimates of :math:\theta, i.e., :math:\mathrm{c}[i-1,i-1] contains the estimated asymptotic standard error of the estimate contained in :math:\mathrm{theta}[i-1].

The elements above the diagonal contain the estimated asymptotic correlation between the estimates of :math:\theta, i.e., :math:\mathrm{c}[i-1,j-1], :math:1\leq i < j\leq m contains the asymptotic correlation between the estimates contained in :math:\mathrm{theta}[i-1] and :math:\mathrm{theta}[j-1].

The elements below the diagonal contain the estimated asymptotic covariance between the estimates of :math:\theta, i.e., :math:\mathrm{c}[i-1,j-1], :math:1\leq j < i\leq m contains the estimated asymptotic covariance between the estimates contained in :math:\mathrm{theta}[i-1] and :math:\mathrm{theta}[j-1].

**rs** : float, ndarray, shape :math:\left(n\right)
The residuals from the model evaluated at final value of :math:\mathrm{theta}, i.e., :math:\mathrm{rs} contains the vector :math:\left(y-X\hat{\theta }\right).

**wgt** : float, ndarray, shape :math:\left(n\right)
The vector of weights. :math:\mathrm{wgt}[\textit{i}-1] contains the weight for the :math:\textit{i}\ th observation, for :math:\textit{i} = 1,2,\ldots,n.

**stat** : float, ndarray, shape :math:\left(4\right)
The following values are assigned to :math:\mathrm{stat}:

:math:\mathrm{stat}[0] = \beta_1 if :math:\mathrm{isigma} < 0, or :math:\mathrm{stat}[0] = \beta_2 if :math:\mathrm{isigma} > 0.

:math:\mathrm{stat}[1] = \text{} number of iterations used to calculate :math:A.

:math:\mathrm{stat}[2] = \text{} number of iterations used to calculate final estimates of :math:\theta and :math:\sigma.

:math:\mathrm{stat}[3] = k, the rank of the weighted least squares equations.

.. _g02ha-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ldx} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ldx} \geq n.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > m.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry: :math:\mathrm{ipsi} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{ipsi} = 0, :math:1, :math:2, :math:3 or :math:4.

(errno :math:3)
On entry, :math:\mathrm{cucv} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{cucv} \geq m.

(errno :math:3)
On entry, :math:\mathrm{cucv} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{cucv}\geq \sqrt{m}.

(errno :math:3)
On entry: :math:\mathrm{h1}, :math:\mathrm{h2}, :math:\mathrm{h3} incorrectly set.

Constraint: :math:0.0\leq \mathrm{h1}\leq \mathrm{h2}\leq \mathrm{h3} and :math:\mathrm{h3} > 0.0.

(errno :math:3)
On entry: :math:\mathrm{cpsi} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{cpsi} > 0.0.

(errno :math:3)
On entry, :math:\mathrm{dchi} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{dchi} > 0.0.

(errno :math:3)
On entry, :math:\mathrm{sigma} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{sigma} > 0.0.

(errno :math:4)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} > 0.

(errno :math:4)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol} > 0.0.

(errno :math:5)
The number of iterations required to calculate the weights exceeds :math:\mathrm{maxit}. (Only if :math:\mathrm{indw}\neq 0.)

(errno :math:6)
The number of iterations required to calculate :math:\beta_1 exceeds :math:\mathrm{maxit}. (Only if :math:\mathrm{indw} < 0 and :math:\mathrm{isigma} < 0.)

(errno :math:7)
Iterations to calculate estimates of :math:\mathrm{theta} failed to converge in :math:\mathrm{maxit} iterations: :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:7)
The number of iterations required to calculate :math:\theta and :math:\sigma exceeds :math:\mathrm{maxit}.

(errno :math:12)
Error degrees of freedom :math:n-k\leq 0, where :math:n = \langle\mathit{\boldsymbol{value}}\rangle and the rank of :math:\mathrm{x}, :math:k = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:12)
Estimated value of :math:\mathrm{sigma} is zero.

**Warns**
**NagAlgorithmicWarning**
(errno :math:8)
Weighted least squares equations not of full rank: rank :math:\text{} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:9)
Failure to invert matrix while calculating covariance.

(errno :math:10)
Factor for covariance matrix :math:\text{} = 0, uncorrected :math:\left({X^\mathrm{T}X}\right)^{-1} given.

(errno :math:11)
Variance of an element of :math:\mathrm{theta}\leq 0.0, correlations set to :math:0.

.. _g02ha-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

For the linear regression model

.. math::
y = X\theta +\epsilon \text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+-------------------------------------------------------------------------------------------------------------------------------+
|where|:math:y is a vector of length :math:n of the dependent variable,                                                           |
+-----+-------------------------------------------------------------------------------------------------------------------------------+
|     |:math:X is an :math:n\times m matrix of independent variables of column rank :math:k,                                    |
+-----+-------------------------------------------------------------------------------------------------------------------------------+
|     |:math:\theta is a vector of length :math:m of unknown parameters,                                                          |
+-----+-------------------------------------------------------------------------------------------------------------------------------+
|and  |:math:\epsilon is a vector of length :math:n of unknown errors with :math:\mathrm{var}\left(\epsilon_i\right) = \sigma^2,|
+-----+-------------------------------------------------------------------------------------------------------------------------------+

robustm calculates the M-estimates given by the solution, :math:\hat{\theta }, to the equation

.. math::
\sum_{{i = 1}}^n\psi \left(r_i/\left(\sigma w_i\right)\right)w_ix_{{ij}} = 0\text{, }\quad j = 1,2,\ldots,m\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|where|:math:r_i is the :math:i\ th residual, i.e., the :math:i\ th element of :math:r = y-X\hat{\theta },                                                                                                              |
+-----+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     |:math:\psi is a suitable weight function,                                                                                                                                                                              |
+-----+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     |:math:w_i are suitable weights,                                                                                                                                                                                        |
+-----+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|and  |:math:\sigma may be estimated at each iteration by the median absolute deviation of the residuals  :math:\displaystyle \hat{\sigma } = \mathrm{med}_i\left({\left[\left\lvert r_i\right\rvert \right]/\beta_1}\right)|
+-----+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

or as the solution to

.. math::
\sum_{{i = 1}}^n\chi \left(r_i/\left(\hat{\sigma }w_i\right)\right)w_i^2 = \left(n-k\right)\beta_2

for suitable weight function :math:\chi, where :math:\beta_1 and :math:\beta_2 are constants, chosen so that the estimator of :math:\sigma is asymptotically unbiased if the errors, :math:\epsilon_i, have a Normal distribution.
Alternatively :math:\sigma may be held at a constant value.

The above describes the Schweppe type regression.
If the :math:w_i are assumed to equal :math:1 for all :math:i then Huber type regression is obtained.
A third type, due to Mallows, replaces (1) <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02haf.html#eqn1>__ by

.. math::
\sum_{{i = 1}}^n\psi \left(r_i/\sigma \right)w_ix_{{ij}} = 0\text{, }\quad j = 1,2,\ldots,m\text{.}

This may be obtained by use of the transformations

.. math::
\begin{array}{ll}w_i^*←\sqrt{w_i}&\\y_i^*←y_i \sqrt{w_i}&\\x_{{ij}}^*←x_{{ij}} \sqrt{w_i}\text{,}&j = 1,2,\ldots,m\end{array}

(see Section 3 of Marazzi (1987a)).

For Huber and Schweppe type regressions, :math:\beta_1 is the 75th percentile of the standard Normal distribution.
For Mallows type regression :math:\beta_1 is the solution to

.. math::
\frac{1}{n}\sum_{{i = 1}}^n\Phi \left(\beta_1/\sqrt{w_i}\right) = 0.75\text{,}

where :math:\Phi is the standard Normal cumulative distribution function (see :meth:specfun.cdf_normal <naginterfaces.library.specfun.cdf_normal>).

:math:\beta_2 is given by

.. math::
\begin{array}{ll} \beta_2 = \int_{{-\infty }}^{\infty } \chi \left(z\right) \phi \left(z\right) {dz} &\text{in the Huber case;}\\&\\&\\ \beta_2 = \frac{1}{n} \sum_{{i = 1}}^n w_i \int_{{-\infty }}^{\infty } \chi \left(z\right) \phi \left(z\right) {dz} &\text{in the Mallows case;}\\&\\&\\ \beta_2 = \frac{1}{n} \sum_{{i = 1}}^n w_i^2 \int_{{-\infty }}^{\infty } \chi \left(z/w_i\right) \phi \left(z\right) {dz} &\text{in the Schweppe case;}\end{array}

where :math:\phi is the standard Normal density, i.e., :math:\frac{1}{\sqrt{2\pi }}\mathrm{exp}\left(-\frac{1}{2}x^2\right)\text{.}

The calculation of the estimates of :math:\theta can be formulated as an iterative weighted least squares problem with a diagonal weight matrix :math:G given by

.. math::
G_{{ii}} = \left\{\begin{array}{ll} \frac{{\psi \left(r_i/\left(\sigma w_i\right)\right)}}{{\left(r_i/\left(\sigma w_i\right)\right)}} \text{,} &r_i\neq 0\\&\\ \psi^{\prime } \left(0\right) \text{,} &r_i = 0\end{array}\text{,}\right.

where :math:\psi^{\prime }\left(t\right) is the derivative of :math:\psi at the point :math:t.

The value of :math:\theta at each iteration is given by the weighted least squares regression of :math:y on :math:X.
This is carried out by first transforming the :math:y and :math:X by

.. math::
\begin{array}{ll}\tilde{y}_i = y_i\sqrt{G_{{ii}}}&\\\tilde{x}_{{ij}} = x_{{ij}}\sqrt{G_{{ii}}}\text{,}&j = 1,2,\ldots,m\end{array}

and then solving the associated least squares problem.
If :math:X is of full column rank then an orthogonal-triangular (:math:QR) decomposition is used; if not, a singular value decomposition is used.

The following functions are available for :math:\psi and :math:\chi in robustm.

(a) **Unit Weights**

.. math::
\psi \left(t\right) = t\text{, }\quad \chi \left(t\right) = \frac{t^2}{2}\text{.}

This gives least squares regression.

(#) **Huber's Function**

.. math::
\psi \left(t\right) = \mathrm{max}\left({-c}, \mathrm{min}\left(c, t\right)\right)\text{, }\quad \chi \left(t\right) = \left\{\begin{array}{ll} \frac{t^2}{2} \text{,} &\left\lvert t\right\rvert \leq d\\&\\ \frac{d^2}{2} \text{,} &\left\lvert t\right\rvert > d\end{array}\right.

(#) **Hampel's Piecewise Linear Function**

.. math::
\psi_{{h_1,h_2,h_3}}\left(t\right) = -\psi_{{h_1,h_2,h_3}}\left(-t\right) = \left\{\begin{array}{ll}t\text{,}& 0\leq t\leq h_1 \\&\\h_1\text{,}& h_1 \leq t\leq h_2 \\&\\ h_1 \left(h_3-t\right) / \left(h_3-h_2\right) \text{,} & h_2 \leq t\leq h_3 \\&\\0\text{,}&h_3 < t\end{array}\right.

.. math::
\chi \left(t\right) = \left\{\begin{array}{ll} \frac{t^2}{2} \text{,} &\left\lvert t\right\rvert \leq d\\&\\ \frac{d^2}{2} \text{,} &\left\lvert t\right\rvert > d\end{array}\right.

(#) **Andrew's Sine Wave Function**

.. math::
\psi \left(t\right) = \left\{\begin{array}{ll}\sin\left(t\right)\text{,}&-\pi \leq t\leq \pi \\&\\0\text{,}&\left\lvert t\right\rvert > \pi \end{array}\right. \quad \text{ }\quad \chi \left(t\right) = \left\{\begin{array}{ll} \frac{t^2}{2} \text{,} &\left\lvert t\right\rvert \leq d\\&\\ \frac{d^2}{2} \text{,} &\left\lvert t\right\rvert > d\end{array}\right.

(#) **Tukey's Bi-weight**

.. math::
\psi \left(t\right) = \left\{\begin{array}{ll} t \left(1-t^2\right)^2 \text{,} &\left\lvert t\right\rvert \leq 1\\&\\0\text{,}&\left\lvert t\right\rvert > 1\end{array}\right. \quad \text{ }\quad \chi \left(t\right) = \left\{\begin{array}{ll} \frac{t^2}{2} \text{,} &\left\lvert t\right\rvert \leq d\\&\\ \frac{d^2}{2} \text{,} &\left\lvert t\right\rvert > d\end{array}\right.

where :math:c, :math:h_1, :math:h_2, :math:h_3, and :math:d are given constants.

Several schemes for calculating weights have been proposed, see Hampel et al. (1986) and Marazzi (1987a).
As the different independent variables may be measured on different scales, one group of proposed weights aims to bound a standardized measure of influence.
To obtain such weights the matrix :math:A has to be found such that:

.. math::
\frac{1}{n}\sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)z_iz_i^\mathrm{T} = I

and

.. math::
z_i = Ax_i\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+------------------------------------------------------------------------------------------+
|where|:math:x_i is a vector of length :math:m containing the :math:i\ th row of :math:X,|
+-----+------------------------------------------------------------------------------------------+
|     |:math:A is an :math:m\times m lower triangular matrix,                                |
+-----+------------------------------------------------------------------------------------------+
|and  |:math:u is a suitable function.                                                         |
+-----+------------------------------------------------------------------------------------------+

The weights are then calculated as

.. math::
w_i = f\left(\left\lVert z_i\right\rVert_2\right)

for a suitable function :math:f.

robustm finds :math:A using the iterative procedure

.. math::
A_k = \left(S_k+I\right)A_{{k-1}}\text{,}

where :math:S_k = \left(s_{{jl}}\right),

.. math::
s_{{jl}} = \left\{\begin{array}{ll} {-\mathrm{min}\left[\mathrm{max}\left({h_{{jl}}/n}, {-BL}\right), {BL}\right]} \text{,} &j > \mathrm{l}\\&\\ {-\mathrm{min}\left[\mathrm{max}\left({\frac{1}{2}\left(h_{{jj}}/n-1\right)}, {-BD}\right), {BD}\right]} \text{,}&j = \mathrm{l}\end{array}\right.

and

.. math::
h_{{jl}} = \sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)z_{{ij}}z_{{il}}

and :math:BL and :math:BD are bounds set at :math:0.9.

Two weights are available in robustm:

.. math::
u\left(t\right) = g_1\left(\frac{c}{t}\right)\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+------------------------------------------------------------------------------------------------------------+
|where|:math:g_1\left(t\right) = t^2+\left(1-t^2\right)\left(2\Phi \left(t\right)-1\right)-2t\phi \left(t\right),|
+-----+------------------------------------------------------------------------------------------------------------+
|     |:math:\Phi \left(t\right) is the standard Normal cumulative distribution function,                        |
+-----+------------------------------------------------------------------------------------------------------------+
|     |:math:\phi \left(t\right) is the standard Normal probability density function,                            |
+-----+------------------------------------------------------------------------------------------------------------+
|and  |:math:f\left(t\right) = \frac{1}{t}.                                                                      |
+-----+------------------------------------------------------------------------------------------------------------+

These are for use with Schweppe type regression.

(#) **Maronna's Proposed Weights**

.. math::
\begin{array}{l} u\left(t\right) = \left\{\begin{array}{ll} \frac{c}{t^2} &\left\lvert t\right\rvert > c\\1&\left\lvert t\right\rvert \leq c\end{array}\right. \\ f\left(t\right) = \sqrt{u\left(t\right)} \text{.} \end{array}

These are for use with Mallows type regression.

Finally the asymptotic variance-covariance matrix, :math:C, of the estimates :math:\theta is calculated.

For Huber type regression

.. math::
C = f_H\left(X^\mathrm{T}X\right)^{-1}\hat{\sigma }^2\text{,}

where

.. math::
f_H = \frac{1}{{n-m}}\frac{{\sum_{{i = 1}}^n\psi^2\left(r_i/\hat{\sigma }\right)}}{\left(\frac{1}{n}\sum_{{i = 1}}^n\psi^{\prime }\left(\frac{r_i}{\hat{\sigma }}\right)\right)^2}\kappa^2

.. math::
\kappa^2 = 1+\frac{m}{n}\frac{{\frac{1}{n}\sum_{{i = 1}}^n\left(\psi^{\prime }\left(r_i/\hat{\sigma }\right)-\frac{1}{n}\sum_{{i = 1}}^n\psi^{\prime }\left(r_i/\hat{\sigma }\right)\right)^2}}{\left(\frac{1}{n}\sum_{{i = 1}}^n\psi^{\prime }\left(\frac{r_i}{\hat{\sigma }}\right)\right)^2}\text{.}

See Huber (1981) and Marazzi (1987b).

For Mallows and Schweppe type regressions :math:C is of the form

.. math::
\frac{\hat{\sigma }}{n}^2S_1^{-1}S_2S_1^{-1}\text{,}

where :math:S_1 = \frac{1}{n}X^\mathrm{T}DX and :math:S_2 = \frac{1}{n}X^\mathrm{T}PX.

:math:D is a diagonal matrix such that the :math:i\ th element approximates :math:E\left(\psi^{\prime }\left(r_i/\left(\sigma w_i\right)\right)\right) in the Schweppe case and :math:E\left(\psi^{\prime }\left(r_i/\sigma \right)w_i\right) in the Mallows case.

:math:P is a diagonal matrix such that the :math:i\ th element approximates :math:E\left(\psi^2\left(r_i/\left(\sigma w_i\right)\right)w_i^2\right) in the Schweppe case and :math:E\left(\psi^2\left(r_i/\sigma \right)w_i^2\right) in the Mallows case.

Two approximations are available in robustm:

(1) Average over the :math:r_i

.. math::
\begin{array}{cc}\text{Schweppe}&\text{Mallows}\\&\\ D_i = \left(\frac{1}{n}\sum_{{j = 1}}^n\psi^{\prime }\left(\frac{r_j}{{\hat{\sigma }w_i}}\right)\right) w_i \quad \text{ }\quad & D_i = \left(\frac{1}{n}\sum_{{j = 1}}^n\psi^{\prime }\left(\frac{r_j}{\hat{\sigma }}\right)\right) w_i \\&\\ P_i = \left(\frac{1}{n}\sum_{{j = 1}}^n\psi^2\left(\frac{r_j}{{\hat{\sigma }w_i}}\right)\right) w_i^2 \quad \text{ }\quad & P_i = \left(\frac{1}{n}\sum_{{j = 1}}^n\psi^2\left(\frac{r_j}{\hat{\sigma }}\right)\right) w_i^2 \end{array}

(#) Replace expected value by observed

.. math::
\begin{array}{cc}\text{Schweppe}&\text{Mallows}\\&\\ D_i = \psi^{\prime } \left(\frac{r_i}{{\hat{\sigma }w_i}}\right) w_i \quad \text{ }\quad & D_i = \psi^{\prime } \left(\frac{r_i}{\hat{\sigma }}\right) w_i \\&\\ P_i = \psi^2 \left(\frac{r_i}{{\hat{\sigma }w_i}}\right) w_i^2 \quad \text{ }\quad & P_i = \psi^2 \left(\frac{r_i}{\hat{\sigma }}\right) w_i^2 \end{array}\text{.}

See Hampel et al. (1986) and Marazzi (1987b).

**Note:** there is no explicit provision in the function for a constant term in the regression model.
However, the addition of a dummy variable whose value is :math:1.0 for all observations will produce a value of :math:\hat{\theta } corresponding to the usual constant term.

robustm is based on routines in ROBETH; see Marazzi (1987a).

.. _g02ha-py2-py-references:

**References**
Hampel, F R, Ronchetti, E M, Rousseeuw, P J and Stahel, W A, 1986, Robust Statistics. The Approach Based on Influence Functions, Wiley

Huber, P J, 1981, Robust Statistics, Wiley

Marazzi, A, 1987, Weights for bounded influence regression in ROBETH, Cah. Rech. Doc. IUMSP, No. 3 ROB 3, Institut Universitaire de Médecine Sociale et Préventive, Lausanne

Marazzi, A, 1987, Subroutines for robust and bounded influence regression in ROBETH, Cah. Rech. Doc. IUMSP, No. 3 ROB 2, Institut Universitaire de Médecine Sociale et Préventive, Lausanne
"""
raise NotImplementedError

[docs]def robustm_wts(ucv, x, a, bl=0.9, bd=0.9, tol=5e-5, maxit=50, nitmon=0, data=None, io_manager=None):
r"""
robustm_wts finds, for a real matrix :math:X of full column rank, a lower triangular matrix :math:A such that :math:\left(A^\mathrm{T}A\right)^{-1} is proportional to a robust estimate of the covariance of the variables. robustm_wts is intended for the calculation of weights of bounded influence regression using :meth:robustm_user.

.. _g02hb-py2-py-doc:

For full information please refer to the NAG Library document for g02hb

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hbf.html

.. _g02hb-py2-py-parameters:

**Parameters**
**ucv** : callable retval = ucv(t, data=None)
:math:\mathrm{ucv} must return the value of the function :math:u for a given value of its argument.

The value of :math:u must be non-negative.

**Parameters**
**t** : float
The argument for which :math:\mathrm{ucv} must be evaluated.

**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.

**Returns**
**retval** : float
The value of :math:u\left(t\right) evaluated at :math:\mathrm{t}.

**x** : float, array-like, shape :math:\left(n, m\right)
The real matrix :math:X, i.e., the independent variables. :math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\textit{j}\ th element of :math:\mathrm{x}, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**a** : float, array-like, shape :math:\left(m\times \left(m+1\right)/2\right)
An initial estimate of the lower triangular real matrix :math:A. Only the lower triangular elements must be given and these should be stored row-wise in the array.

The diagonal elements must be :math:\text{}\neq 0, although in practice will usually be :math:\text{} > 0.

If the magnitudes of the columns of :math:X are of the same order the identity matrix will often provide a suitable initial value for :math:A.

If the columns of :math:X are of different magnitudes, the diagonal elements of the initial value of :math:A should be approximately inversely proportional to the magnitude of the columns of :math:X.

**bl** : float, optional
The magnitude of the bound for the off-diagonal elements of :math:S_k.

**bd** : float, optional
The magnitude of the bound for the diagonal elements of :math:S_k.

**tol** : float, optional
The relative precision for the final value of :math:A. Iteration will stop when the maximum value of :math:\left\lvert s_{{jl}}\right\rvert is less than :math:\mathrm{tol}.

**maxit** : int, optional
The maximum number of iterations that will be used during the calculation of :math:A.

A value of :math:\mathrm{maxit} = 50 will often be adequate.

**nitmon** : int, optional
Determines the amount of information that is printed on each iteration.

:math:\mathrm{nitmon} > 0

The value of :math:A and the maximum value of :math:\left\lvert s_{{jl}}\right\rvert will be printed at the first and every :math:\mathrm{nitmon} iterations.

:math:\mathrm{nitmon}\leq 0

No iteration monitoring is printed.

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see :class:~naginterfaces.base.utils.FileObjManager).

**data** : arbitrary, optional
User-communication data for callback functions.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**a** : float, ndarray, shape :math:\left(m\times \left(m+1\right)/2\right)
The lower triangular elements of the matrix :math:A, stored row-wise.

**z** : float, ndarray, shape :math:\left(n\right)
The value :math:\left\lVert z_{\textit{i}}\right\rVert_2, for :math:\textit{i} = 1,2,\ldots,n.

**nit** : int
The number of iterations performed.

.. _g02hb-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ldx} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ldx} \geq n.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq m.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:2)
On entry, :math:\mathrm{bd} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{bd} > 0.0.

(errno :math:2)
On entry, :math:\mathrm{bl} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{bl} > 0.0.

(errno :math:2)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} > 0.

(errno :math:2)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol} > 0.0.

(errno :math:2)
On entry, :math:i = \langle\mathit{\boldsymbol{value}}\rangle and the :math:i\ th diagonal element of :math:A is :math:0.

Constraint: all diagonal elements of :math:A must be non-zero.

(errno :math:3)
Value returned by :math:\mathrm{ucv} function :math:\text{} < 0: :math:u\left(\langle\mathit{\boldsymbol{value}}\rangle\right) = \langle\mathit{\boldsymbol{value}}\rangle.

The value of :math:u must be non-negative.

(errno :math:4)
Iterations to calculate weights failed to converge in :math:\mathrm{maxit} iterations: :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

.. _g02hb-py2-py-notes:

**Notes**
In fitting the linear regression model

.. math::
y = X\theta +\epsilon \text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+---------------------------------------------------------------------+
|where|:math:y is a vector of length :math:n of the dependent variable, |
+-----+---------------------------------------------------------------------+
|     |:math:X is an :math:n\times m matrix of independent variables,   |
+-----+---------------------------------------------------------------------+
|     |:math:\theta is a vector of length :math:m of unknown parameters,|
+-----+---------------------------------------------------------------------+
|and  |:math:\epsilon is a vector of length :math:n of unknown errors,  |
+-----+---------------------------------------------------------------------+

it may be desirable to bound the influence of rows of the :math:X matrix.
This can be achieved by calculating a weight for each observation.
Several schemes for calculating weights have been proposed (see Hampel et al. (1986) and Marazzi (1987)).
As the different independent variables may be measured on different scales one group of proposed weights aims to bound a standardized measure of influence.
To obtain such weights the matrix :math:A has to be found such that

.. math::
\frac{1}{n}\sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)z_iz_i^\mathrm{T} = I\text{ }\quad \left(I\text{ is the identity matrix}\right)

and

.. math::
z_i = Ax_i\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+----------------------------------------------------------------------------------------------------------+
|where|:math:x_i is a vector of length :math:m containing the elements of the :math:i\ th row of :math:X,|
+-----+----------------------------------------------------------------------------------------------------------+
|     |:math:A is an :math:m\times m lower triangular matrix,                                                |
+-----+----------------------------------------------------------------------------------------------------------+
|     |:math:z_i is a vector of length :math:m,                                                              |
+-----+----------------------------------------------------------------------------------------------------------+
|and  |:math:u is a suitable function.                                                                         |
+-----+----------------------------------------------------------------------------------------------------------+

The weights for use with :meth:robustm_user may then be computed using

.. math::
w_i = f\left(\left\lVert z_i\right\rVert_2\right)

for a suitable user-supplied function :math:f.

robustm_wts finds :math:A using the iterative procedure

.. math::
A_k = \left(S_k+I\right)A_{{k-1}}\text{,}

where :math:S_k = \left(s_{{jl}}\right), for :math:\textit{l} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m, is a lower triangular matrix such that

:math:s_{{jl}} = \left\{\begin{array}{ll}-\mathrm{min}\left[\mathrm{max}\left(h_{{jl}}/n,-\textit{BL}\right),\textit{BL}\right]\text{,}&j > l\\&\\-\mathrm{min}\left[\mathrm{max}\left(\frac{1}{2}\left(h_{{jj}}/n-1\right),-\textit{BD}\right),\textit{BD}\right]\text{,}&j = l\end{array}\right.

:math:h_{{jl}} = \sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)z_{{ij}}z_{{il}}

and :math:\textit{BD} and :math:\textit{BL} are suitable bounds.

In addition the values of :math:\left\lVert z_i\right\rVert_2, for :math:i = 1,2,\ldots,n, are calculated.

robustm_wts is based on routines in ROBETH; see Marazzi (1987).

.. _g02hb-py2-py-references:

**References**
Hampel, F R, Ronchetti, E M, Rousseeuw, P J and Stahel, W A, 1986, Robust Statistics. The Approach Based on Influence Functions, Wiley

Huber, P J, 1981, Robust Statistics, Wiley

Marazzi, A, 1987, Weights for bounded influence regression in ROBETH, Cah. Rech. Doc. IUMSP, No. 3 ROB 3, Institut Universitaire de Médecine Sociale et Préventive, Lausanne
"""
raise NotImplementedError

[docs]def robustm_user(psi, psip0, beta, indw, isigma, x, y, wgt, theta, sigma, chi=None, tol=5e-5, eps=5e-6, maxit=50, nitmon=0, data=None, io_manager=None):
r"""
robustm_user performs bounded influence regression (:math:M-estimates) using an iterative weighted least squares algorithm.

.. _g02hd-py2-py-doc:

For full information please refer to the NAG Library document for g02hd

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hdf.html

.. _g02hd-py2-py-parameters:

**Parameters**
**psi** : callable retval = psi(t, data=None)
:math:\mathrm{psi} must return the value of the weight function :math:\psi for a given value of its argument.

**Parameters**
**t** : float
The argument for which :math:\mathrm{psi} must be evaluated.

**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.

**Returns**
**retval** : float
The value of the weight function :math:\psi evaluated at :math:\mathrm{t}.

**psip0** : float
The value of :math:\psi^{\prime }\left(0\right).

**beta** : float
If :math:\mathrm{isigma} < 0, :math:\mathrm{beta} must specify the value of :math:\beta_1.

For Huber and Schweppe type regressions, :math:\beta_1 is the :math:75\ th percentile of the standard Normal distribution (see :meth:stat.inv_cdf_normal <naginterfaces.library.stat.inv_cdf_normal>).

For Mallows type regression :math:\beta_1 is the solution to

.. math::
\frac{1}{n}\sum_{{i = 1}}^n\Phi \left(\beta_1/\sqrt{w_i}\right) = 0.75\text{,}

where :math:\Phi is the standard Normal cumulative distribution function (see :meth:specfun.cdf_normal <naginterfaces.library.specfun.cdf_normal>).

If :math:\mathrm{isigma} > 0, :math:\mathrm{beta} must specify the value of :math:\beta_2.

.. math::
\begin{array}{lll}\beta_2 = &\int_{{-\infty }}^{\infty }\chi \left(z\right)\phi \left(z\right){dz}\text{,}&\text{in the Huber case;}\\&&\\&&\\\beta_2 = &\frac{1}{n}\sum_{{i = 1}}^nw_i\int_{{-\infty }}^{\infty }\chi \left(z\right)\phi \left(z\right){dz}\text{,}&\text{in the Mallows case;}\\&&\\&&\\\beta_2 = &\frac{1}{n}\sum_{{i = 1}}^nw_i^2\int_{{-\infty }}^{\infty }\chi \left(z/w_i\right)\phi \left(z\right){dz}\text{,}&\text{in the Schweppe case;}\end{array}

where :math:\phi is the standard normal density, i.e., :math:\frac{1}{\sqrt{2\pi }}\mathrm{exp}\left(-\frac{1}{2}x^2\right).

If :math:\mathrm{isigma} = 0, :math:\mathrm{beta} is not referenced.

**indw** : int
Determines the type of regression to be performed.

:math:\mathrm{indw} = 0

Huber type regression.

:math:\mathrm{indw} < 0

Mallows type regression.

:math:\mathrm{indw} > 0

Schweppe type regression.

**isigma** : int
Determines how :math:\sigma is to be estimated.

:math:\mathrm{isigma} = 0

:math:\sigma is held constant at its initial value.

:math:\mathrm{isigma} < 0

:math:\sigma is estimated by median absolute deviation of residuals.

:math:\mathrm{isigma} > 0

:math:\sigma is estimated using the :math:\chi function.

**x** : float, array-like, shape :math:\left(n, m\right)
The values of the :math:X matrix, i.e., the independent variables. :math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\textit{j}\ th element of :math:\mathrm{x}, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

If :math:\mathrm{indw} < 0, during calculations the elements of :math:\mathrm{x} will be transformed as described in :ref:Notes <g02hd-py2-py-notes>.

Before exit the inverse transformation will be applied.

As a result there may be slight differences between the input :math:\mathrm{x} and the output :math:\mathrm{x}.

**y** : float, array-like, shape :math:\left(n\right)
The data values of the dependent variable.

:math:\mathrm{y}[\textit{i}-1] must contain the value of :math:y for the :math:\textit{i}\ th observation, for :math:\textit{i} = 1,2,\ldots,n.

If :math:\mathrm{indw} < 0, during calculations the elements of :math:\mathrm{y} will be transformed as described in :ref:Notes <g02hd-py2-py-notes>.

Before exit the inverse transformation will be applied.

As a result there may be slight differences between the input :math:\mathrm{y} and the output :math:\mathrm{y}.

**wgt** : float, array-like, shape :math:\left(n\right)
The weight for the :math:\textit{i}\ th observation, for :math:\textit{i} = 1,2,\ldots,n.

If :math:\mathrm{indw} < 0, during calculations elements of :math:\mathrm{wgt} will be transformed as described in :ref:Notes <g02hd-py2-py-notes>.

Before exit the inverse transformation will be applied.

As a result there may be slight differences between the input :math:\mathrm{wgt} and the output :math:\mathrm{wgt}.

If :math:\mathrm{wgt}[i-1]\leq 0, the :math:i\ th observation is not included in the analysis.

If :math:\mathrm{indw} = 0, :math:\mathrm{wgt} is not referenced.

**theta** : float, array-like, shape :math:\left(m\right)
Starting values of the parameter vector :math:\theta. These may be obtained from least squares regression. Alternatively if :math:\mathrm{isigma} < 0 and :math:\mathrm{sigma} = 1 or if :math:\mathrm{isigma} > 0 and :math:\mathrm{sigma} approximately equals the standard deviation of the dependent variable, :math:y, then :math:\mathrm{theta}[\textit{i}-1] = 0.0, for :math:\textit{i} = 1,2,\ldots,m may provide reasonable starting values.

**sigma** : float
A starting value for the estimation of :math:\sigma. :math:\mathrm{sigma} should be approximately the standard deviation of the residuals from the model evaluated at the value of :math:\theta given by :math:\mathrm{theta} on entry.

**chi** : None or callable retval = chi(t, data=None), optional
Note: if this argument is **None** then a NAG-supplied facility will be used.

If :math:\mathrm{isigma} > 0, :math:\mathrm{chi} must return the value of the weight function :math:\chi for a given value of its argument.

The value of :math:\chi must be non-negative.

**Parameters**
**t** : float
The argument for which :math:\mathrm{chi} must be evaluated.

**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.

**Returns**
**retval** : float
The value of the weight function :math:\chi evaluated at :math:\mathrm{t}.

**tol** : float, optional
The relative precision for the final estimates. Convergence is assumed when both the relative change in the value of :math:\mathrm{sigma} and the relative change in the value of each element of :math:\mathrm{theta} are less than :math:\mathrm{tol}.

It is advisable for :math:\mathrm{tol} to be greater than :math:100\times \text{machine precision}.

**eps** : float, optional
A relative tolerance to be used to determine the rank of :math:X. See :meth:linsys.real_gen_solve <naginterfaces.library.linsys.real_gen_solve> for further details.

If :math:\mathrm{eps} < \text{machine precision} or :math:\mathrm{eps} > 1.0, machine precision will be used in place of :math:\mathrm{tol}.

A reasonable value for :math:\mathrm{eps} is :math:5.0\times 10^{-6} where this value is possible.

**maxit** : int, optional
The maximum number of iterations that should be used during the estimation.

A value of :math:\mathrm{maxit} = 50 should be adequate for most uses.

**nitmon** : int, optional
Determines the amount of information that is printed on each iteration.

:math:\mathrm{nitmon}\leq 0

No information is printed.

:math:\mathrm{nitmon} > 0

On the first and every :math:\mathrm{nitmon} iterations the values of :math:\mathrm{sigma}, :math:\mathrm{theta} and the change in :math:\mathrm{theta} during the iteration are printed.

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see :class:~naginterfaces.base.utils.FileObjManager).

**data** : arbitrary, optional
User-communication data for callback functions.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**x** : float, ndarray, shape :math:\left(n, m\right)
Unchanged, except as described above.

**y** : float, ndarray, shape :math:\left(n\right)
Unchanged, except as described above.

**wgt** : float, ndarray, shape :math:\left(n\right)
Unchanged, except as described above.

**theta** : float, ndarray, shape :math:\left(m\right)
The M-estimate of :math:\theta_{\textit{i}}, for :math:\textit{i} = 1,2,\ldots,m.

**k** : int
The column rank of the matrix :math:X.

**sigma** : float
The final estimate of :math:\sigma if :math:\mathrm{isigma} \neq 0 or the value assigned on entry if :math:\mathrm{isigma} = 0.

**rs** : float, ndarray, shape :math:\left(n\right)
The residuals from the model evaluated at final value of :math:\mathrm{theta}, i.e., :math:\mathrm{rs} contains the vector :math:\left(y-X\hat{\theta }\right).

**nit** : int
The number of iterations that were used during the estimation.

.. _g02hd-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ldx} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ldx} \geq n.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > m.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\mathrm{beta} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{beta} > 0.0.

(errno :math:2)
On entry, :math:\mathrm{sigma} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{sigma} > 0.0.

(errno :math:3)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} > 0.

(errno :math:3)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol} > 0.0.

(errno :math:4)
Value given by :math:\mathrm{chi} function :math:\text{} < 0: :math:\mathrm{chi}\left(\langle\mathit{\boldsymbol{value}}\rangle\right) = \langle\mathit{\boldsymbol{value}}\rangle.

The value of :math:\mathrm{chi} must be non-negative.

(errno :math:5)
Estimated value of :math:\mathrm{sigma} is zero.

(errno :math:6)
Iterations to solve the weighted least squares equations failed to converge.

(errno :math:8)
The function has failed to converge in :math:\mathrm{maxit} iterations.

(errno :math:9)
Having removed cases with zero weight, the value of :math:n-\mathrm{k}\leq 0, i.e., no degree of freedom for error. This error will only occur if :math:\mathrm{isigma} > 0.

**Warns**
**NagAlgorithmicWarning**
(errno :math:7)
The weighted least squares equations are not of full rank. This may be due to the :math:X matrix not being of full rank, in which case the results will be valid. It may also occur if some of the :math:G_{{ii}} values become very small or zero, see Further Comments <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hdf.html#fcomments>__. The rank of the equations is given by :math:\mathrm{k}. If the matrix just fails the test for nonsingularity then the result :math:\mathrm{errno} = 7 and :math:\mathrm{k} = m is possible (see :meth:linsys.real_gen_solve <naginterfaces.library.linsys.real_gen_solve>).

.. _g02hd-py2-py-notes:

**Notes**
For the linear regression model

.. math::
y = X\theta +\epsilon \text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+-----------------------------------------------------------------------------------------------------------------------+
|where|:math:y is a vector of length :math:n of the dependent variable,                                                   |
+-----+-----------------------------------------------------------------------------------------------------------------------+
|     |:math:X is an :math:n\times m matrix of independent variables of column rank :math:k,                            |
+-----+-----------------------------------------------------------------------------------------------------------------------+
|     |:math:\theta is a vector of length :math:m of unknown parameters,                                                  |
+-----+-----------------------------------------------------------------------------------------------------------------------+
|and  |:math:\epsilon is a vector of length :math:n of unknown errors with var :math:\left(\epsilon_i\right) = \sigma^2,|
+-----+-----------------------------------------------------------------------------------------------------------------------+

robustm_user calculates the M-estimates given by the solution, :math:\hat{\theta }, to the equation

.. math::
\sum_{{i = 1}}^n\psi \left(r_i/\left(\sigma w_i\right)\right)w_ix_{{ij}} = 0\text{, }\quad j = 1,2,\ldots,m\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|where|:math:r_i is the :math:i\ th residual, i.e., the :math:i\ th element of the vector :math:r = y-X\hat{\theta },                                                                                    |
+-----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     |:math:\psi is a suitable weight function,                                                                                                                                                               |
+-----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     |:math:w_i are suitable weights such as those that can be calculated by using output from :meth:robustm_wts,                                                                                           |
+-----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|and  |:math:\sigma may be estimated at each iteration by the median absolute deviation of the residuals :math:\hat{\sigma } = \mathrm{med}_i\left({\left[\left\lvert r_i\right\rvert \right]/\beta_1}\right)|
+-----+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

or as the solution to

.. math::
\sum_{{i = 1}}^n\chi \left(r_i/\left(\hat{\sigma }w_i\right)\right)w_i^2 = \left(n-k\right)\beta_2

for a suitable weight function :math:\chi, where :math:\beta_1 and :math:\beta_2 are constants, chosen so that the estimator of :math:\sigma is asymptotically unbiased if the errors, :math:\epsilon_i, have a Normal distribution.
Alternatively :math:\sigma may be held at a constant value.

The above describes the Schweppe type regression.
If the :math:w_i are assumed to equal :math:1 for all :math:i, then Huber type regression is obtained.
A third type, due to Mallows, replaces (1) <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hdf.html#eqn1>__ by

.. math::
\sum_{{i = 1}}^n\psi \left(r_i/\sigma \right)w_ix_{{ij}} = 0\text{, }\quad j = 1,2,\ldots,m\text{.}

This may be obtained by use of the transformations

.. math::

(see Marazzi (1987)).

The calculation of the estimates of :math:\theta can be formulated as an iterative weighted least squares problem with a diagonal weight matrix :math:G given by

.. math::
G_{{ii}} = \left\{\begin{array}{cl}\frac{{\psi \left(r_i/\left(\sigma w_i\right)\right)}}{{\left(r_i/\left(\sigma w_i\right)\right)}}\text{,}&r_i\neq 0\\&\\\psi^{\prime }\left(0\right)\text{,}&r_i = 0\text{.}\end{array}\right. \text{.}

The value of :math:\theta at each iteration is given by the weighted least squares regression of :math:y on :math:X.
This is carried out by first transforming the :math:y and :math:X by

.. math::
\begin{array}{ll}\tilde{y}_i& = y_i\sqrt{G_{{ii}}}\\\tilde{x}_{{ij}}& = x_{{ij}}\sqrt{G_{{ii}}}\text{, }\quad j = 1,2,\ldots,m\end{array}

and then using :meth:linsys.real_gen_solve <naginterfaces.library.linsys.real_gen_solve>.
If :math:X is of full column rank then an orthogonal-triangular (:math:QR) decomposition is used; if not, a singular value decomposition is used.

Observations with zero or negative weights are not included in the solution.

**Note:** there is no explicit provision in the function for a constant term in the regression model.
However, the addition of a dummy variable whose value is :math:1.0 for all observations will produce a value of :math:\hat{\theta } corresponding to the usual constant term.

robustm_user is based on routines in ROBETH, see Marazzi (1987).

.. _g02hd-py2-py-references:

**References**
Hampel, F R, Ronchetti, E M, Rousseeuw, P J and Stahel, W A, 1986, Robust Statistics. The Approach Based on Influence Functions, Wiley

Huber, P J, 1981, Robust Statistics, Wiley

Marazzi, A, 1987, Subroutines for robust and bounded influence regression in ROBETH, Cah. Rech. Doc. IUMSP, No. 3 ROB 2, Institut Universitaire de Médecine Sociale et Préventive, Lausanne
"""
raise NotImplementedError

[docs]def robustm_user_varmat(psi, psp, indw, indc, sigma, x, rs, wgt, data=None):
r"""
robustm_user_varmat calculates an estimate of the asymptotic variance-covariance matrix for the bounded influence regression estimates (M-estimates).
It is intended for use with :meth:robustm_user.

.. _g02hf-py2-py-doc:

For full information please refer to the NAG Library document for g02hf

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hff.html

.. _g02hf-py2-py-parameters:

**Parameters**
**psi** : callable retval = psi(t, data=None)
:math:\mathrm{psi} must return the value of the :math:\psi function for a given value of its argument.

**Parameters**
**t** : float
The argument for which :math:\mathrm{psi} must be evaluated.

**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.

**Returns**
**retval** : float
The value of the function :math:\psi evaluated at :math:\mathrm{t}.

**psp** : callable retval = psp(t, data=None)
:math:\mathrm{psp} must return the value of :math:\psi^{\prime }\left(t\right) = \frac{d}{{dt}}\psi \left(t\right) for a given value of its argument.

**Parameters**
**t** : float
The argument for which :math:\mathrm{psp} must be evaluated.

**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.

**Returns**
**retval** : float
The value of :math:\psi^{\prime }\left(t\right) evaluated at :math:\mathrm{t}.

**indw** : int
The type of regression for which the asymptotic variance-covariance matrix is to be calculated.

:math:\mathrm{indw} = -1

Mallows type regression.

:math:\mathrm{indw} = 0

Huber type regression.

:math:\mathrm{indw} = 1

Schweppe type regression.

**indc** : int
If :math:\mathrm{indw} \neq 0, :math:\mathrm{indc} must specify the approximation to be used.

If :math:\mathrm{indc} = 1, averaging over residuals.

If :math:\mathrm{indc} = 0, replacing expected by observed.

If :math:\mathrm{indw} = 0, :math:\mathrm{indc} is not referenced.

**sigma** : float
The value of :math:\hat{\sigma }, as given by :meth:robustm_user.

**x** : float, array-like, shape :math:\left(n, m\right)
The values of the :math:X matrix, i.e., the independent variables. :math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\textit{j}\ th element of :math:X, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**rs** : float, array-like, shape :math:\left(n\right)
The residuals from the bounded influence regression. These are given by :meth:robustm_user.

**wgt** : float, array-like, shape :math:\left(n\right)
If :math:\mathrm{indw} \neq 0, :math:\mathrm{wgt} must contain the vector of weights used by the bounded influence regression. These should be used with :meth:robustm_user.

If :math:\mathrm{indw} = 0, :math:\mathrm{wgt} is not referenced.

**data** : arbitrary, optional
User-communication data for callback functions.

**Returns**
**c** : float, ndarray, shape :math:\left(m, m\right)
The estimate of the variance-covariance matrix.

**wk** : float, ndarray, shape :math:\left(m\times \left(n+m+1\right)+2\times n\right)
If :math:\mathrm{indw} \neq 0, :math:\mathrm{wk}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,n, will contain the diagonal elements of the matrix :math:D and :math:\mathrm{wk}[\textit{i}-1], for :math:\textit{i} = n+1,\ldots,2n, will contain the diagonal elements of matrix :math:P.

The rest of the array is used as workspace.

.. _g02hf-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ldc} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ldc} \geq m.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ldx} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ldx} \geq n.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n > m.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:2)
On entry, :math:\mathrm{sigma} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{sigma}\geq 0.0.

(errno :math:3)
:math:\textit{S}_1 matrix is singular or almost singular.

(errno :math:3)
:math:X^\mathrm{T}X matrix not positive definite.

(errno :math:4)
Correction factor :math:= 0 (Huber type regression).

.. _g02hf-py2-py-notes:

**Notes**
For a description of bounded influence regression see :meth:robustm_user.
Let :math:\theta be the regression parameters and let :math:C be the asymptotic variance-covariance matrix of :math:\hat{\theta }.
Then for Huber type regression

.. math::
C = f_H\left(X^\mathrm{T}X\right)^{-1}\hat{\sigma }^2\text{,}

where

.. math::
f_H = \frac{1}{{n-m}}\frac{{\sum_{{i = 1}}^n\psi^2\left(r_i/\hat{\sigma }\right)}}{\left(\frac{1}{n}\sum \psi^{\prime }\left(\frac{r_i}{\hat{\sigma }}\right)\right)^2}\kappa^2

.. math::
\kappa^2 = 1+\frac{m}{n}\frac{{\frac{1}{n}\sum_{{i = 1}}^n\left(\psi^{\prime }\left(r_i/\hat{\sigma }\right)-\frac{1}{n}\sum_{{i = 1}}^n\psi^{\prime }\left(r_i/\hat{\sigma }\right)\right)^2}}{\left(\frac{1}{n}\sum_{{i = 1}}^n\psi^{\prime }\left(\frac{r_i}{\hat{\sigma }}\right)\right)^2}\text{,}

see Huber (1981) and Marazzi (1987).

For Mallows and Schweppe type regressions, :math:C is of the form

.. math::
\frac{\hat{\sigma }}{n}^2S_1^{-1}S_2S_1^{-1}\text{,}

where :math:S_1 = \frac{1}{n}X^\mathrm{T}DX and :math:S_2 = \frac{1}{n}X^\mathrm{T}PX.

:math:D is a diagonal matrix such that the :math:i\ th element approximates :math:E\left(\psi^{\prime }\left(r_i/\left(\sigma w_i\right)\right)\right) in the Schweppe case and :math:E\left(\psi^{\prime }\left(r_i/\sigma \right)w_i\right) in the Mallows case.

:math:P is a diagonal matrix such that the :math:i\ th element approximates :math:E\left(\psi^2\left(r_i/\left(\sigma w_i\right)\right)w_i^2\right) in the Schweppe case and :math:E\left(\psi^2\left(r_i/\sigma \right)w_i^2\right) in the Mallows case.

Two approximations are available in robustm_user_varmat:

(1) Average over the :math:r_i

.. math::
\begin{array}{cc}\text{Schweppe}&\text{Mallows}\\&\\D_i = \left(\frac{1}{n}\sum_{{j = 1}}^n\psi^{\prime }\left(\frac{r_j}{{\hat{\sigma }w_i}}\right)\right) w_i\quad \text{ }\quad &D_i = \left(\frac{1}{n}\sum_{{j = 1}}^n\psi^{\prime }\left(\frac{r_j}{\hat{\sigma }}\right)\right) w_i\\&\\P_i = \left(\frac{1}{n}\sum_{{j = 1}}^n\psi^2\left(\frac{r_j}{{\hat{\sigma }w_i}}\right)\right) w_i^2\quad \text{ }\quad &P_i = \left(\frac{1}{n}\sum_{{j = 1}}^n\psi^2\left(\frac{r_j}{\hat{\sigma }}\right)\right) w_i^2\end{array}

(#) Replace expected value by observed

.. math::
\begin{array}{cc}\text{Schweppe}&\text{Mallows}\\&\\D_i = \psi^{\prime } \left(\frac{r_i}{{\hat{\sigma }w_i}}\right) w_i \quad \text{ }\quad &D_i = \psi^{\prime } \left(\frac{r_i}{\hat{\sigma }}\right) w_i\\&\\P_i = \psi^2 \left(\frac{r_i}{{\hat{\sigma }w_i}}\right) w_i^2 \quad \text{ }\quad &P_i = \psi^2 \left(\frac{r_i}{\hat{\sigma }}\right) w_i^2\end{array}

See Hampel et al. (1986) and Marazzi (1987).

In all cases :math:\hat{\sigma } is a robust estimate of :math:\sigma.

robustm_user_varmat is based on routines in ROBETH; see Marazzi (1987).

.. _g02hf-py2-py-references:

**References**
Hampel, F R, Ronchetti, E M, Rousseeuw, P J and Stahel, W A, 1986, Robust Statistics. The Approach Based on Influence Functions, Wiley

Huber, P J, 1981, Robust Statistics, Wiley

Marazzi, A, 1987, Subroutines for robust and bounded influence regression in ROBETH, Cah. Rech. Doc. IUMSP, No. 3 ROB 2, Institut Universitaire de Médecine Sociale et Préventive, Lausanne
"""
raise NotImplementedError

[docs]def robustm_corr_huber(x, eps, maxit=150, nitmon=0, tol=5e-5, io_manager=None):
r"""
robustm_corr_huber computes a robust estimate of the covariance matrix for an expected fraction of gross errors.

.. _g02hk-py2-py-doc:

For full information please refer to the NAG Library document for g02hk

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hkf.html

.. _g02hk-py2-py-parameters:

**Parameters**
**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation for the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**eps** : float
:math:\epsilon, the expected fraction of gross errors expected in the sample.

**maxit** : int, optional
The maximum number of iterations that will be used during the calculation of the covariance matrix.

**nitmon** : int, optional
Indicates the amount of information on the iteration that is printed.

:math:\mathrm{nitmon} > 0

The value of :math:A, :math:\theta and :math:\delta (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hkf.html#accuracy>__) will be printed at the first and every :math:\mathrm{nitmon} iterations.

:math:\mathrm{nitmon}\leq 0

No iteration monitoring is printed.

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see :class:~naginterfaces.base.utils.FileObjManager).

**tol** : float, optional
The relative precision for the final estimates of the covariance matrix.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**cov** : float, ndarray, shape :math:\left(m\times \left(m+1\right)/2\right)
A robust estimate of the covariance matrix, :math:C. The upper triangular part of the matrix :math:C is stored packed by columns. :math:C_{{ij}} is returned in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1], :math:i\leq j.

**theta** : float, ndarray, shape :math:\left(m\right)
The robust estimate of the location parameters :math:\theta_{\textit{j}}, for :math:\textit{j} = 1,2,\ldots,m.

**nit** : int
The number of iterations performed.

.. _g02hk-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} > 0.

(errno :math:1)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol} > 0.0.

(errno :math:1)
On entry, :math:\mathrm{eps} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0.0\leq \mathrm{eps}\leq 1.0.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq m.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:2)
On entry, a variable has a constant value, i.e., all elements in column :math:\langle\mathit{\boldsymbol{value}}\rangle of :math:\mathrm{x} are identical.

(errno :math:3)
The iterative procedure to find :math:C has failed to converge in :math:\mathrm{maxit} iterations.

(errno :math:4)
The iterative procedure to find :math:C has become unstable. This may happen if the value of :math:\mathrm{eps} is too large for the sample.

.. _g02hk-py2-py-notes:

**Notes**
In the NAG Library the traditional C interface for this routine uses a different algorithmic base. Please contact NAG if you have any questions about compatibility.

For a set of :math:n observations on :math:m variables in a matrix :math:X, a robust estimate of the covariance matrix, :math:C, and a robust estimate of location, :math:\theta, are given by

.. math::
C = \tau^2\left(A^\mathrm{T}A\right)^{-1}\text{,}

where :math:\tau^2 is a correction factor and :math:A is a lower triangular matrix found as the solution to the following equations:

.. math::
z_i = A\left(x_i-\theta \right)\text{,}

.. math::
\frac{1}{n}\sum_{{i = 1}}^nw\left(\left\lVert z_i\right\rVert_2\right)z_i = 0\text{,}

and

.. math::
\frac{1}{n}\sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)z_iz_i^\mathrm{T}-I = 0\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+-------------------------------------------------------------------------------------------------------------------+
|where|:math:x_i is a vector of length :math:m containing the elements of the :math:i\ th row of :math:\mathrm{x},|
+-----+-------------------------------------------------------------------------------------------------------------------+
|     |:math:z_i is a vector of length :math:m,                                                                       |
+-----+-------------------------------------------------------------------------------------------------------------------+
|     |:math:I is the identity matrix and :math:0 is the zero matrix,                                                 |
+-----+-------------------------------------------------------------------------------------------------------------------+
|and  |:math:w and :math:u are suitable functions.                                                                    |
+-----+-------------------------------------------------------------------------------------------------------------------+

robustm_corr_huber uses weight functions:

.. math::
\begin{array}{ll}u\left(t\right) = \frac{a_u}{t^2}\text{,}&\text{if }t < a_u^2\\&\\u\left(t\right) = 1\text{,}&\text{if }a_u^2\leq t\leq b_u^2\\&\\u\left(t\right) = \frac{{b_u}}{t^2}\text{,}&\text{if }t > b_u^2\end{array}

and

.. math::
\begin{array}{ll}w\left(t\right) = 1\text{,}&\text{if }t\leq c_w\\&\\w\left(t\right) = \frac{c_w}{t}\text{,}&\text{if }t > c_w\end{array}

for constants :math:a_u, :math:b_u and :math:c_w.

These functions solve a minimax problem considered by Huber (see Huber (1981)).
The values of :math:a_u, :math:b_u and :math:c_w are calculated from the expected fraction of gross errors, :math:\epsilon (see Huber (1981) and Marazzi (1987)).
The expected fraction of gross errors is the estimated proportion of outliers in the sample.

In order to make the estimate asymptotically unbiased under a Normal model a correction factor, :math:\tau^2, is calculated, (see Huber (1981) and Marazzi (1987)).

The matrix :math:C is calculated using :meth:robustm_corr_user_deriv.
Initial estimates of :math:\theta_j, for :math:j = 1,2,\ldots,m, are given by the median of the :math:j\ th column of :math:X and the initial value of :math:A is based on the median absolute deviation (see Marazzi (1987)). robustm_corr_huber is based on routines in ROBETH; see Marazzi (1987).

.. _g02hk-py2-py-references:

**References**
Huber, P J, 1981, Robust Statistics, Wiley

Marazzi, A, 1987, Weights for bounded influence regression in ROBETH, Cah. Rech. Doc. IUMSP, No. 3 ROB 3, Institut Universitaire de Médecine Sociale et Préventive, Lausanne
"""
raise NotImplementedError

[docs]def robustm_corr_user_deriv(ucv, indm, x, a, theta, bl=0.9, bd=0.9, maxit=150, nitmon=0, tol=5e-5, data=None, io_manager=None):
r"""
robustm_corr_user_deriv calculates a robust estimate of the covariance matrix for user-supplied weight functions and their derivatives.

.. _g02hl-py2-py-doc:

For full information please refer to the NAG Library document for g02hl

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hlf.html

.. _g02hl-py2-py-parameters:

**Parameters**
**ucv** : callable (u, ud, w, wd) = ucv(t, data=None)
:math:\mathrm{ucv} must return the values of the functions :math:u and :math:w and their derivatives for a given value of its argument.

**Parameters**
**t** : float
The argument for which the functions :math:u and :math:w must be evaluated.

**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.

**Returns**
**u** : float
The value of the :math:u function at the point :math:\mathrm{t}.

**ud** : float
The value of the derivative of the :math:u function at the point :math:\mathrm{t}.

**w** : float
The value of the :math:w function at the point :math:\mathrm{t}.

**wd** : float
The value of the derivative of the :math:w function at the point :math:\mathrm{t}.

**indm** : int
Indicates which form of the function :math:v will be used.

:math:\mathrm{indm} = 1

:math:v = 1.

:math:\mathrm{indm}\neq 1

:math:v = u.

**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**a** : float, array-like, shape :math:\left(m\times \left(m+1\right)/2\right)
An initial estimate of the lower triangular real matrix :math:A. Only the lower triangular elements must be given and these should be stored row-wise in the array.

The diagonal elements must be :math:\text{}\neq 0, and in practice will usually be :math:\text{} > 0.

If the magnitudes of the columns of :math:X are of the same order, the identity matrix will often provide a suitable initial value for :math:A.

If the columns of :math:X are of different magnitudes, the diagonal elements of the initial value of :math:A should be approximately inversely proportional to the magnitude of the columns of :math:X.

**theta** : float, array-like, shape :math:\left(m\right)
An initial estimate of the location parameter, :math:\theta_{\textit{j}}, for :math:\textit{j} = 1,2,\ldots,m.

In many cases an initial estimate of :math:\theta_{\textit{j}} = 0, for :math:\textit{j} = 1,2,\ldots,m, will be adequate.

Alternatively medians may be used as given by :meth:univar.robust_1var_median <naginterfaces.library.univar.robust_1var_median>.

**bl** : float, optional
The magnitude of the bound for the off-diagonal elements of :math:S_k, :math:BL.

**bd** : float, optional
The magnitude of the bound for the diagonal elements of :math:S_k, :math:BD.

**maxit** : int, optional
The maximum number of iterations that will be used during the calculation of :math:A.

**nitmon** : int, optional
Indicates the amount of information on the iteration that is printed.

:math:\mathrm{nitmon} > 0

The value of :math:A, :math:\theta and :math:\delta (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hlf.html#accuracy>__) will be printed at the first and every :math:\mathrm{nitmon} iterations.

:math:\mathrm{nitmon}\leq 0

No iteration monitoring is printed.

When printing occurs the output is directed to the file object associated with the advisory I/O unit (see :class:~naginterfaces.base.utils.FileObjManager).

**tol** : float, optional
The relative precision for the final estimates of the covariance matrix. Iteration will stop when maximum :math:\delta (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hlf.html#accuracy>__) is less than :math:\mathrm{tol}.

**data** : arbitrary, optional
User-communication data for callback functions.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**cov** : float, ndarray, shape :math:\left(m\times \left(m+1\right)/2\right)
Contains a robust estimate of the covariance matrix, :math:C. The upper triangular part of the matrix :math:C is stored packed by columns (lower triangular stored by rows), :math:C_{{ij}} is returned in :math:\mathrm{cov}[\left(j\times \left(j-1\right)/2+i\right)-1], :math:i\leq j.

**a** : float, ndarray, shape :math:\left(m\times \left(m+1\right)/2\right)
The lower triangular elements of the inverse of the matrix :math:A, stored row-wise.

**wt** : float, ndarray, shape :math:\left(n\right)
:math:\mathrm{wt}[\textit{i}-1] contains the weights, :math:\textit{wt}_{\textit{i}} = u\left(\left\lVert z_{\textit{i}}\right\rVert_2\right), for :math:\textit{i} = 1,2,\ldots,n.

**theta** : float, ndarray, shape :math:\left(m\right)
Contains the robust estimate of the location parameter, :math:\theta_{\textit{j}}, for :math:\textit{j} = 1,2,\ldots,m.

**nit** : int
The number of iterations performed.

.. _g02hl-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq m.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:2)
On entry, :math:\mathrm{bd} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{bd} > 0.0.

(errno :math:2)
On entry, :math:\mathrm{bl} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{bl} > 0.0.

(errno :math:2)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} > 0.

(errno :math:2)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol} > 0.0.

(errno :math:2)
On entry, :math:i = \langle\mathit{\boldsymbol{value}}\rangle and the :math:i\ th diagonal element of :math:A is :math:0.

Constraint: all diagonal elements of :math:A must be non-zero.

(errno :math:3)
On entry, a variable has a constant value, i.e., all elements in column :math:\langle\mathit{\boldsymbol{value}}\rangle of :math:\mathrm{x} are identical.

(errno :math:4)
:math:w value returned by :math:\mathrm{ucv} < 0.0: :math:w\left(\langle\mathit{\boldsymbol{value}}\rangle\right) = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{w}\geq 0.0.

(errno :math:4)
:math:\textit{u} value returned by :math:\mathrm{ucv} < 0.0: :math:u\left(\langle\mathit{\boldsymbol{value}}\rangle\right) = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{u}\geq 0.0.

(errno :math:5)
Iterations to calculate weights failed to converge.

**Warns**
**NagAlgorithmicWarning**
(errno :math:6)
The sum :math:\textit{D}_2 is zero. Try either a larger initial estimate of :math:A or make :math:u and :math:w less strict.

(errno :math:6)
The sum :math:\textit{D}_3 is zero. Try either a larger initial estimate of :math:A or make :math:u and :math:w less strict.

(errno :math:6)
The sum :math:\textit{D}_1 is zero. Try either a larger initial estimate of :math:A or make :math:u and :math:w less strict.

.. _g02hl-py2-py-notes:

**Notes**
For a set of :math:n observations on :math:m variables in a matrix :math:X, a robust estimate of the covariance matrix, :math:C, and a robust estimate of location, :math:\theta, are given by:

.. math::
C = \tau^2\left(A^\mathrm{T}A\right)^{-1}\text{,}

where :math:\tau^2 is a correction factor and :math:A is a lower triangular matrix found as the solution to the following equations.

.. math::
z_i = A\left(x_i-\theta \right)

.. math::
\frac{1}{n}\sum_{{i = 1}}^nw\left(\left\lVert z_i\right\rVert_2\right)z_i = 0

and

.. math::
\frac{1}{n}\sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)z_iz_i^\mathrm{T}-v\left(\left\lVert z_i\right\rVert_2\right)I = 0\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+----------------------------------------------------------------------------------------------------------+
|where|:math:x_i is a vector of length :math:m containing the elements of the :math:i\ th row of :math:X,|
+-----+----------------------------------------------------------------------------------------------------------+
|     |:math:z_i is a vector of length :math:m,                                                              |
+-----+----------------------------------------------------------------------------------------------------------+
|     |:math:I is the identity matrix and :math:0 is the zero matrix,                                        |
+-----+----------------------------------------------------------------------------------------------------------+
|and  |:math:w and :math:u are suitable functions.                                                           |
+-----+----------------------------------------------------------------------------------------------------------+

robustm_corr_user_deriv covers two situations:

(i) :math:v\left(t\right) = 1 for all :math:t,

(#) :math:v\left(t\right) = u\left(t\right).

The robust covariance matrix may be calculated from a weighted sum of squares and cross-products matrix about :math:\theta using weights :math:\textit{wt}_i = u\left(\left\lVert z_i\right\rVert \right).
In case \(1) a divisor of :math:n is used and in case \(2) a divisor of :math:\sum_{{i = 1}}^n\textit{wt}_i is used.
If :math:w\left(.\right) = \sqrt{u\left(.\right)}, then the robust covariance matrix can be calculated by scaling each row of :math:X by :math:\sqrt{\textit{wt}_i} and calculating an unweighted covariance matrix about :math:\theta.

In order to make the estimate asymptotically unbiased under a Normal model a correction factor, :math:\tau^2, is needed.
The value of the correction factor will depend on the functions employed (see Huber (1981) and Marazzi (1987)).

robustm_corr_user_deriv finds :math:A using the iterative procedure as given by Huber.

.. math::
A_k = \left(S_k+I\right)A_{{k-1}}

and

.. math::
\theta_{j_k} = \frac{{b_j}}{D_1}+\theta_{j_{{k-1}}}\text{,}

where :math:S_k = \left(s_{{jl}}\right), for :math:\textit{l} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m, is a lower triangular matrix such that:

.. math::
s_{{jl}} = \left\{\begin{array}{ll}-\mathrm{min}\left[\mathrm{max}\left(h_{{jl}}/D_3,-\textit{BL}\right),\textit{BL}\right]\text{,}&j > l\\&\\-\mathrm{min}\left[\mathrm{max}\left(\left(h_{{jj}}/\left(2D_3-D_4/D_2\right)\right),-\textit{BD}\right),\textit{BD}\right]\text{,}&j = l\end{array}\right. \text{,}

where

:math:D_1 = \sum_{{i = 1}}^n\left\{w\left(\left\lVert z_i\right\rVert_2\right)+\frac{1}{m}w^{\prime }\left(\left\lVert z_i\right\rVert_2\right)\left\lVert z_i\right\rVert_2\right\}

:math:D_2 = \sum_{{i = 1}}^n\left\{\frac{1}{p}\left(u^{\prime }\left(\left\lVert z_i\right\rVert_2\right)\left\lVert z_i\right\rVert_2+2u\left(\left\lVert z_i\right\rVert_2\right)\right)\left\lVert z_i\right\rVert_2-v^{\prime }\left(\left\lVert z_i\right\rVert_2\right)\right\}\left\lVert z_i\right\rVert_2

:math:D_3 = \frac{1}{{m+2}}\sum_{{i = 1}}^n\left\{\frac{1}{m}\left(u^{\prime }\left(\left\lVert z_i\right\rVert_2\right)\left\lVert z_i\right\rVert_2+2u\left(\left\lVert z_i\right\rVert_2\right)\right)+u\left(\left\lVert z_i\right\rVert_2\right)\right\}\left\lVert z_i\right\rVert_2^2

:math:D_4 = \sum_{{i = 1}}^n\left\{\frac{1}{m}u\left(\left\lVert z_i\right\rVert_2\right)\left\lVert z_i\right\rVert_2^2-v\left(\left\lVert z_i\right\rVert_2^2\right)\right\}

:math:h_{{jl}} = \sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)z_{{ij}}z_{{il}}, for :math:j > l

:math:h_{{jj}} = \sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)\left(z_{{ij}}^2-\left\lVert z_{{ij}}\right\rVert_2^2/m\right)

:math:b_j = \sum_{{i = 1}}^nw\left(\left\lVert z_i\right\rVert_2\right)\left(x_{{ij}}-b_j\right)

and :math:\textit{BD} and :math:\textit{BL} are suitable bounds.

robustm_corr_user_deriv is based on routines in ROBETH; see Marazzi (1987).

.. _g02hl-py2-py-references:

**References**
Huber, P J, 1981, Robust Statistics, Wiley

Marazzi, A, 1987, Weights for bounded influence regression in ROBETH, Cah. Rech. Doc. IUMSP, No. 3 ROB 3, Institut Universitaire de Médecine Sociale et Préventive, Lausanne
"""
raise NotImplementedError

[docs]def robustm_corr_user(ucv, indm, x, a, theta, bl=0.9, bd=0.9, maxit=150, nitmon=0, tol=5e-5, data=None, io_manager=None):
r"""
robustm_corr_user computes a robust estimate of the covariance matrix for user-supplied weight functions.
The derivatives of the weight functions are not required.

.. _g02hm-py2-py-doc:

For full information please refer to the NAG Library document for g02hm

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hmf.html

.. _g02hm-py2-py-parameters:

**Parameters**
**ucv** : callable (u, w) = ucv(t, data=None)
:math:\mathrm{ucv} must return the values of the functions :math:u and :math:w for a given value of its argument.

**Parameters**
**t** : float
The argument for which the functions :math:u and :math:w must be evaluated.

**data** : arbitrary, optional, modifiable in place
User-communication data for callback functions.

**Returns**
**u** : float
The value of the :math:u function at the point :math:\mathrm{t}.

**w** : float
The value of the :math:w function at the point :math:\mathrm{t}.

**indm** : int
Indicates which form of the function :math:v will be used.

:math:\mathrm{indm} = 1

:math:v = 1.

:math:\mathrm{indm}\neq 1

:math:v = u.

**x** : float, array-like, shape :math:\left(n, m\right)
:math:\mathrm{x}[\textit{i}-1,\textit{j}-1] must contain the :math:\textit{i}\ th observation on the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m, for :math:\textit{i} = 1,2,\ldots,n.

**a** : float, array-like, shape :math:\left(m\times \left(m+1\right)/2\right)
An initial estimate of the lower triangular real matrix :math:A. Only the lower triangular elements must be given and these should be stored row-wise in the array.

The diagonal elements must be :math:\text{}\neq 0, and in practice will usually be :math:\text{} > 0.

If the magnitudes of the columns of :math:X are of the same order, the identity matrix will often provide a suitable initial value for :math:A.

If the columns of :math:X are of different magnitudes, the diagonal elements of the initial value of :math:A should be approximately inversely proportional to the magnitude of the columns of :math:X.

**theta** : float, array-like, shape :math:\left(m\right)
An initial estimate of the location parameter, :math:\theta_{\textit{j}}, for :math:\textit{j} = 1,2,\ldots,m.

In many cases an initial estimate of :math:\theta_{\textit{j}} = 0, for :math:\textit{j} = 1,2,\ldots,m, will be adequate.

Alternatively medians may be used as given by :meth:univar.robust_1var_median <naginterfaces.library.univar.robust_1var_median>.

**bl** : float, optional
The magnitude of the bound for the off-diagonal elements of :math:S_k, :math:BL.

**bd** : float, optional
The magnitude of the bound for the diagonal elements of :math:S_k, :math:BD.

**maxit** : int, optional
The maximum number of iterations that will be used during the calculation of :math:A.

**nitmon** : int, optional
Indicates the amount of information on the iteration that is printed.

:math:\mathrm{nitmon} > 0

The value of :math:A, :math:\theta and :math:\delta (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hmf.html#accuracy>__) will be printed at the first and every :math:\mathrm{nitmon} iterations.

:math:\mathrm{nitmon}\leq 0

No iteration monitoring is printed.

**tol** : float, optional
The relative precision for the final estimate of the covariance matrix. Iteration will stop when maximum :math:\delta (see Accuracy <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02hmf.html#accuracy>__) is less than :math:\mathrm{tol}.

**data** : arbitrary, optional
User-communication data for callback functions.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**cov** : float, ndarray, shape :math:\left(m\times \left(m+1\right)/2\right)
A robust estimate of the covariance matrix, :math:C. The upper triangular part of the matrix :math:C is stored packed by columns (lower triangular stored by rows), that is :math:C_{{ij}} is returned in :math:\mathrm{cov}[j\times \left(j-1\right)/2+i-1], :math:i\leq j.

**a** : float, ndarray, shape :math:\left(m\times \left(m+1\right)/2\right)
The lower triangular elements of the inverse of the matrix :math:A, stored row-wise.

**wt** : float, ndarray, shape :math:\left(n\right)
:math:\mathrm{wt}[\textit{i}-1] contains the weights, :math:\textit{wt}_{\textit{i}} = u\left(\left\lVert z_{\textit{i}}\right\rVert_2\right), for :math:\textit{i} = 1,2,\ldots,n.

**theta** : float, ndarray, shape :math:\left(m\right)
Contains the robust estimate of the location parameter, :math:\theta_{\textit{j}}, for :math:\textit{j} = 1,2,\ldots,m.

**nit** : int
The number of iterations performed.

.. _g02hm-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle and :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq m.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 2.

(errno :math:1)
On entry, :math:m = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:m \geq 1.

(errno :math:2)
On entry, :math:\mathrm{bd} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{bd} > 0.0.

(errno :math:2)
On entry, :math:\mathrm{bl} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{bl} > 0.0.

(errno :math:2)
On entry, :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{maxit} > 0.

(errno :math:2)
On entry, :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{tol} > 0.0.

(errno :math:2)
On entry, :math:i = \langle\mathit{\boldsymbol{value}}\rangle and the :math:i\ th diagonal element of :math:A is :math:0.

Constraint: all diagonal elements of :math:A must be non-zero.

(errno :math:3)
On entry, a variable has a constant value, i.e., all elements in column :math:\langle\mathit{\boldsymbol{value}}\rangle of :math:\mathrm{x} are identical.

(errno :math:4)
:math:w value returned by :math:\mathrm{ucv} < 0.0: :math:w\left(\langle\mathit{\boldsymbol{value}}\rangle\right) = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{w}\geq 0.0.

(errno :math:4)
:math:\textit{u} value returned by :math:\mathrm{ucv} < 0.0: :math:u\left(\langle\mathit{\boldsymbol{value}}\rangle\right) = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{u}\geq 0.0.

(errno :math:5)
Iterations to calculate weights failed to converge.

**Warns**
**NagAlgorithmicWarning**
(errno :math:6)
The sum :math:D_2 is zero. Try either a larger initial estimate of :math:A or make :math:u and :math:w less strict.

(errno :math:6)
The sum :math:D_1 is zero. Try either a larger initial estimate of :math:A or make :math:u and :math:w less strict.

.. _g02hm-py2-py-notes:

**Notes**
For a set of :math:n observations on :math:m variables in a matrix :math:X, a robust estimate of the covariance matrix, :math:C, and a robust estimate of location, :math:\theta, are given by

.. math::
C = \tau^2\left(A^\mathrm{T}A\right)^{-1}\text{,}

where :math:\tau^2 is a correction factor and :math:A is a lower triangular matrix found as the solution to the following equations.

.. math::
z_i = A\left(x_i-\theta \right)

.. math::
\frac{1}{n}\sum_{{i = 1}}^nw\left(\left\lVert z_i\right\rVert_2\right)z_i = 0

and

.. math::
\frac{1}{n}\sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)z_iz_i^\mathrm{T}-v\left(\left\lVert z_i\right\rVert_2\right)I = 0\text{,}

.. rst-class:: nag-rules-none nag-align-left

+-----+----------------------------------------------------------------------------------------------------------+
|where|:math:x_i is a vector of length :math:m containing the elements of the :math:i\ th row of :math:X,|
+-----+----------------------------------------------------------------------------------------------------------+
|     |:math:z_i is a vector of length :math:m,                                                              |
+-----+----------------------------------------------------------------------------------------------------------+
|     |:math:I is the identity matrix and :math:0 is the zero matrix.                                        |
+-----+----------------------------------------------------------------------------------------------------------+
|and  |:math:w and :math:u are suitable functions.                                                           |
+-----+----------------------------------------------------------------------------------------------------------+

robustm_corr_user covers two situations:

(i) :math:v\left(t\right) = 1 for all :math:t,

(#) :math:v\left(t\right) = u\left(t\right).

The robust covariance matrix may be calculated from a weighted sum of squares and cross-products matrix about :math:\theta using weights :math:\textit{wt}_i = u\left(\left\lVert z_i\right\rVert \right).
In case \(1) a divisor of :math:n is used and in case \(2) a divisor of :math:\sum_{{i = 1}}^n\textit{wt}_i is used.
If :math:w\left(.\right) = \sqrt{u\left(.\right)}, then the robust covariance matrix can be calculated by scaling each row of :math:X by :math:\sqrt{\textit{wt}_i} and calculating an unweighted covariance matrix about :math:\theta.

In order to make the estimate asymptotically unbiased under a Normal model a correction factor, :math:\tau^2, is needed.
The value of the correction factor will depend on the functions employed (see Huber (1981) and Marazzi (1987)).

robustm_corr_user finds :math:A using the iterative procedure as given by Huber; see Huber (1981).

.. math::
A_k = \left(S_k+I\right)A_{{k-1}}

and

.. math::
\theta_{j_k} = \frac{{b_j}}{D_1}+\theta_{j_{{k-1}}}\text{,}

where :math:S_k = \left(s_{{jl}}\right), for :math:\textit{l} = 1,2,\ldots,m, for :math:\textit{j} = 1,2,\ldots,m is a lower triangular matrix such that

.. math::
s_{{jl}} = \left\{\begin{array}{ll}-\mathrm{min}\left[\mathrm{max}\left(h_{{jl}}/D_2,-\textit{BL}\right),\textit{BL}\right]\text{,}&j > l\\&\\-\mathrm{min}\left[\mathrm{max}\left(\frac{1}{2}\left(h_{{jj}}/D_2-1\right),-\textit{BD}\right),\textit{BD}\right]\text{,}&j = l\end{array}\right. \text{,}

where

:math:D_1 = \sum_{{i = 1}}^nw\left(\left\lVert z_i\right\rVert_2\right)

:math:D_2 = \sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)

:math:h_{{jl}} = \sum_{{i = 1}}^nu\left(\left\lVert z_i\right\rVert_2\right)z_{{ij}}z_{{il}}, for :math:j\geq l

:math:b_j = \sum_{{i = 1}}^nw\left(\left\lVert z_i\right\rVert_2\right)\left(x_{{ij}}-b_j\right)

and :math:\textit{BD} and :math:\textit{BL} are suitable bounds.

The value of :math:\tau may be chosen so that :math:C is unbiased if the observations are from a given distribution.

robustm_corr_user is based on routines in ROBETH; see Marazzi (1987).

.. _g02hm-py2-py-references:

**References**
Huber, P J, 1981, Robust Statistics, Wiley

Marazzi, A, 1987, Weights for bounded influence regression in ROBETH, Cah. Rech. Doc. IUMSP, No. 3 ROB 3, Institut Universitaire de Médecine Sociale et Préventive, Lausanne
"""
raise NotImplementedError

[docs]def mixeff_reml(dat, levels, yvid, cwid, fvid, fint, rvid, nvpr, vpr, rint, svid, gamma, lb, maxit=-1, tol=0.0, io_manager=None):
r"""
mixeff_reml fits a linear mixed effects regression model using restricted maximum likelihood (REML).

.. deprecated:: 27.0.0.0
mixeff_reml is deprecated.
Please use :meth:lmm_init followed by :meth:lmm_fit instead.
See also the :ref:Replacement Calls <replace> document.

.. _g02ja-py2-py-doc:

For full information please refer to the NAG Library document for g02ja

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jaf.html

.. _g02ja-py2-py-parameters:

**Parameters**
**dat** : float, array-like, shape :math:\left(n, \textit{ncol}\right)
Array containing all of the data. For the :math:i\ th observation:

:math:\mathrm{dat}[i-1,\mathrm{yvid}-1] holds the dependent variable, :math:y;

if :math:\mathrm{cwid}\neq 0, :math:\mathrm{dat}[i-1,\mathrm{cwid}-1] holds the case weights;

if :math:\mathrm{svid}\neq 0, :math:\mathrm{dat}[i-1,\mathrm{svid}-1] holds the subject variable.

The remaining columns hold the values of the independent variables.

**levels** : int, array-like, shape :math:\left(\textit{ncol}\right)
:math:\mathrm{levels}[i-1] contains the number of levels associated with the :math:i\ th variable of the data matrix :math:\mathrm{dat}. If this variable is continuous or binary (i.e., only takes the values zero or one) then :math:\mathrm{levels}[i-1] should be :math:1; if the variable is discrete then :math:\mathrm{levels}[i-1] is the number of levels associated with it and :math:\mathrm{dat}[\textit{j}-1,i-1] is assumed to take the values :math:1 to :math:\mathrm{levels}[i-1], for :math:\textit{j} = 1,2,\ldots,n.

**yvid** : int
The column of :math:\mathrm{dat} holding the dependent, :math:y, variable.

**cwid** : int
The column of :math:\mathrm{dat} holding the case weights.

If :math:\mathrm{cwid} = 0, no weights are used.

**fvid** : int, array-like, shape :math:\left(\textit{nfv}\right)
The columns of the data matrix :math:\mathrm{dat} holding the fixed independent variables with :math:\mathrm{fvid}[i-1] holding the column number corresponding to the :math:i\ th fixed variable.

**fint** : int
Flag indicating whether a fixed intercept is included (:math:\mathrm{fint} = 1).

**rvid** : int, array-like, shape :math:\left(\textit{nrv}\right)
The columns of the data matrix :math:\mathrm{dat} holding the random independent variables with :math:\mathrm{rvid}[i-1] holding the column number corresponding to the :math:i\ th random variable.

**nvpr** : int
If :math:\mathrm{rint} = 1 and :math:\mathrm{svid}\neq 0, :math:\mathrm{nvpr} is the number of variance components being :math:\text{estimated}-2, (:math:g-1), else :math:\mathrm{nvpr} = g.

If :math:\textit{nrv} = 0, :math:\mathrm{nvpr} is not referenced.

**vpr** : int, array-like, shape :math:\left(\textit{nrv}\right)
:math:\mathrm{vpr}[i-1] holds a flag indicating the variance of the :math:i\ th random variable. The variance of the :math:i\ th random variable is :math:\sigma_j^2, where :math:j = \mathrm{vpr}[i-1]+1 if :math:\mathrm{rint} = 1 and :math:\mathrm{svid} \neq 0 and :math:j = \mathrm{vpr}[i-1] otherwise. Random variables with the same value of :math:j are assumed to be taken from the same distribution.

**rint** : int
Flag indicating whether a random intercept is included (:math:\mathrm{rint} = 1).

If :math:\mathrm{svid} = 0, :math:\mathrm{rint} is not referenced.

**svid** : int
The column of :math:\mathrm{dat} holding the subject variable.

If :math:\mathrm{svid} = 0, no subject variable is used.

Specifying a subject variable is equivalent to specifying the interaction between that variable and all of the random-effects.

Letting the notation :math:Z_1\times Z_S denote the interaction between variables :math:Z_1 and :math:Z_S, fitting a model with :math:\mathrm{rint} = 0, random-effects :math:Z_1+Z_2 and subject variable :math:Z_S is equivalent to fitting a model with random-effects :math:Z_1\times Z_S+Z_2\times Z_S and no subject variable.

If :math:\mathrm{rint} = 1 the model is equivalent to fitting :math:Z_S+Z_1\times Z_S+Z_2\times Z_S and no subject variable.

**gamma** : float, array-like, shape :math:\left(\mathrm{nvpr}+2\right)
Holds the initial values of the variance components, :math:\gamma_0, with :math:\mathrm{gamma}[\textit{i}-1] the initial value for :math:\sigma_{\textit{i}}^2/\sigma_R^2, for :math:\textit{i} = 1,2,\ldots,g. If :math:\mathrm{rint} = 1 and :math:\mathrm{svid}\neq 0, :math:g = \mathrm{nvpr}+1, else :math:g = \mathrm{nvpr}.

If :math:\mathrm{gamma}[0] = {-1.0}, the remaining elements of :math:\mathrm{gamma} are ignored and the initial values for the variance components are estimated from the data using MIVQUE0.

**lb** : int
The size of the array :math:\mathrm{b}.

**maxit** : int, optional
The maximum number of iterations.

If :math:\mathrm{maxit} < 0, the default value of :math:100 is used.

If :math:\mathrm{maxit} = 0, the parameter estimates :math:\left(\beta, \nu \right) and corresponding standard errors are calculated based on the value of :math:\gamma_0 supplied in :math:\mathrm{gamma}.

**tol** : float, optional
The tolerance used to assess convergence.

If :math:\mathrm{tol}\leq 0.0, the default value of :math:\epsilon^{0.7} is used, where :math:\epsilon is the machine precision.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**gamma** : float, ndarray, shape :math:\left(\mathrm{nvpr}+2\right)
:math:\mathrm{gamma}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,g, holds the final estimate of :math:\sigma_{\textit{i}}^2 and :math:\mathrm{gamma}[g] holds the final estimate for :math:\sigma_R^2.

**nff** : int
The number of fixed effects estimated (i.e., the number of columns, :math:p, in the design matrix :math:X).

**nrf** : int
The number of random effects estimated (i.e., the number of columns, :math:q, in the design matrix :math:Z).

**df** : int
The degrees of freedom.

**reml** : float
:math:-2l_R\left(\hat{\gamma }\right) where :math:l_R is the log of the restricted maximum likelihood calculated at :math:\hat{\gamma }, the estimated variance components returned in :math:\mathrm{gamma}.

**b** : float, ndarray, shape :math:\left(\mathrm{lb}\right)
The parameter estimates, :math:\left(\beta, \nu \right), with the first :math:\mathrm{nff} elements of :math:\mathrm{b} containing the fixed effect parameter estimates, :math:\beta and the next :math:\mathrm{nrf} elements of :math:\mathrm{b} containing the random effect parameter estimates, :math:\nu.

**Fixed effects**

If :math:\mathrm{fint} = 1, :math:\mathrm{b}[0] contains the estimate of the fixed intercept.

Let :math:L_i denote the number of levels associated with the :math:i\ th fixed variable, that is :math:L_i = \mathrm{levels}[ \mathrm{fvid}[i-1] -1].

Define

if :math:\mathrm{fint} = 1, :math:F_1 = 2 else if :math:\mathrm{fint} = 0, :math:F_1 = 1;

:math:F_{{i+1}} = F_i+\mathrm{max}\left({L_i-1}, 1\right), :math:i\geq 1.

Then for :math:i = 1,2,\ldots,\textit{nfv}:

if :math:L_i > 1, :math:\mathrm{b}[ F_i+\textit{j}-2 -1] contains the parameter estimate for the :math:\textit{j}\ th level of the :math:i\ th fixed variable, for :math:\textit{j} = 2,3,\ldots,L_i;

if :math:L_i\leq 1, :math:\mathrm{b}[F_i-1] contains the parameter estimate for the :math:i\ th fixed variable.

**Random effects**

Redefining :math:L_i to denote the number of levels associated with the :math:i\ th random variable, that is :math:L_i = \mathrm{levels}[ \mathrm{rvid}[i-1] -1].

Define

if :math:\mathrm{rint} = 1, :math:R_1 = 2 else if :math:\mathrm{rint} = 0, :math:R_1 = 1;

:math:R_{{i+1}} = R_i+L_i, :math:i\geq 1.

Then for :math:i = 1,2,\ldots,\textit{nrv}:

if :math:\mathrm{svid} = 0,

if :math:L_i > 1, :math:\mathrm{b}[ \mathrm{nff} + R_i +\textit{j}-1 -1] contains the parameter estimate for the :math:\textit{j}\ th level of the :math:i\ th random variable, for :math:\textit{j} = 1,2,\ldots,L_i;

if :math:L_i\leq 1, :math:\mathrm{b}[ \mathrm{nff} + R_i -1] contains the parameter estimate for the :math:i\ th random variable;

if :math:\mathrm{svid}\neq 0,

let :math:L_S denote the number of levels associated with the subject variable, that is :math:L_S = \mathrm{levels}[ \mathrm{svid} -1];

if :math:L_i > 1, :math:\mathrm{b}[ \mathrm{nff} + \left(\textit{s}-1\right) L_S + R_i + \textit{j} - 1 -1] contains the parameter estimate for the interaction between the :math:\textit{s}\ th level of the subject variable and the :math:\textit{j}\ th level of the :math:i\ th random variable, for :math:\textit{j} = 1,2,\ldots,L_i, for :math:\textit{s} = 1,2,\ldots,L_S;

if :math:L_i\leq 1, :math:\mathrm{b}[ \mathrm{nff} + \left(\textit{s}-1\right) L_S + R_i -1] contains the parameter estimate for the interaction between the :math:\textit{s}\ th level of the subject variable and the :math:i\ th random variable, for :math:\textit{s} = 1,2,\ldots,L_S;

if :math:\mathrm{rint} = 1, :math:\mathrm{b}[\mathrm{nff}] contains the estimate of the random intercept.

**se** : float, ndarray, shape :math:\left(\mathrm{lb}\right)
The standard errors of the parameter estimates given in :math:\mathrm{b}.

**warn** : int
Is set to :math:1 if a variance component was estimated to be a negative value during the fitting process. Otherwise :math:\mathrm{warn} is set to :math:0.

If :math:\mathrm{warn} = 1, the negative estimate is set to zero and the estimation process allowed to continue.

.. _g02ja-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: number of observations with nonzero weights must be greater than one.

(errno :math:1)
On entry, :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ncol} \geq 1.

(errno :math:1)
On entry, :math:\textit{nfv} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \textit{nfv} < \textit{ncol}.

(errno :math:1)
On entry, :math:\textit{nrv} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \textit{nrv} < \textit{ncol} and :math:\textit{nrv}+\mathrm{rint} > 0.

(errno :math:1)
On entry, :math:\mathrm{nvpr} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{nrv} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \mathrm{nvpr}\leq \textit{nrv} and (:math:\textit{nrv} \neq 0 or :math:\mathrm{nvpr} \geq 1).

(errno :math:1)
On entry, :math:\mathrm{yvid} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{yvid}\leq \textit{ncol}.

(errno :math:1)
On entry, :math:\mathrm{svid} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \mathrm{svid}\leq \textit{ncol}.

(errno :math:1)
On entry, :math:\mathrm{cwid} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \mathrm{cwid}\leq \textit{ncol} and any supplied weights must be :math:\text{}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{fint} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{fint} = 0 or :math:1.

(errno :math:1)
On entry, :math:\mathrm{rint} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{rint} = 0 or :math:1.

(errno :math:1)
On entry, :math:\mathrm{lb} too small: :math:\mathrm{lb} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:2)
On entry, :math:\mathrm{levels}[\textit{i}] < 1, for at least one :math:\textit{i}.

(errno :math:2)
On entry, :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{fvid}[i]\leq \textit{ncol}, for all :math:i.

(errno :math:2)
On entry, :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{rvid}[i]\leq \textit{ncol}, for all :math:i.

(errno :math:2)
On entry, :math:\mathrm{nvpr} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{vpr}[i]\leq \mathrm{nvpr}, for all :math:i.

(errno :math:2)
On entry, invalid data: categorical variable with value greater than that specified in :math:\mathrm{levels}.

(errno :math:2)
On entry, :math:\mathrm{gamma}[i] < 0.0, for at least one :math:i.

(errno :math:3)
Degrees of freedom :math:< 1: :math:\mathrm{df} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:4)
Routine failed to converge to specified tolerance: :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:4)
Routine failed to converge in :math:\mathrm{maxit} iterations: :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

.. _g02ja-py2-py-notes:

**Notes**
mixeff_reml fits a model of the form:

.. math::
y = X\beta +Z\nu +\epsilon

where

:math:y is a vector of :math:n observations on the dependent variable,

:math:X is a known :math:n\times p design matrix for the fixed independent variables,

:math:\beta is a vector of length :math:p of unknown fixed effects,

:math:Z is a known :math:n\times q design matrix for the random independent variables,

:math:\nu is a vector of length :math:q of unknown random effects,

and

:math:\epsilon is a vector of length :math:n of unknown random errors.

Both :math:\nu and :math:\epsilon are assumed to have a Gaussian distribution with expectation zero and

.. math::
\mathrm{Var}\left[\begin{array}{c}\nu \\\epsilon \end{array}\right] = \left[\begin{array}{cc}G&0\\0&R\end{array}\right]

where :math:R = \sigma_R^2I, :math:I is the :math:n\times n identity matrix and :math:G is a diagonal matrix.
It is assumed that the random variables, :math:Z, can be subdivided into :math:g\leq q groups with each group being identically distributed with expectations zero and variance :math:\sigma_i^2.
The diagonal elements of matrix :math:G, therefore, take one of the values :math:\left\{\sigma_i^2:i = 1,2,\ldots,g\right\}, depending on which group the associated random variable belongs to.

The model, therefore, contains three sets of unknowns, the fixed effects, :math:\beta, the random effects :math:\nu and a vector of :math:g+1 variance components, :math:\gamma, where :math:\gamma = \left\{\sigma_1^2, \sigma_2^2, \ldots, \sigma_{{g-1}}^2, \sigma_g^2, \sigma_R^2\right\}.
Rather than working directly with :math:\gamma, mixeff_reml uses an iterative process to estimate :math:\gamma^* = \left\{{\sigma_1^2/\sigma_R^2}, {\sigma_2^2/\sigma_R^2}, \ldots, {\sigma_{{g-1}}^2/\sigma_R^2}, {\sigma_g^2/\sigma_R^2}, 1\right\}.
Due to the iterative nature of the estimation a set of initial values, :math:\gamma_0, for :math:\gamma^* is required. mixeff_reml allows these initial values either to be supplied by you or calculated from the data using the minimum variance quadratic unbiased estimators (MIVQUE0) suggested by Rao (1972).

mixeff_reml fits the model using a quasi-Newton algorithm to maximize the restricted log-likelihood function:

.. math::
-2l_R = \log\left(\left\lvert V\right\rvert \right)+\left(n-p\right)\log\left(r^{\prime }V^{-1}r\right)+\log\left(\left\lvert X^{\prime }V^{-1}X\right\rvert \right)+\left(n-p\right)\left(1+\log\left(2\pi /\left(n-p\right)\right)\right)

where

.. math::

Once the final estimates for :math:\gamma^* have been obtained, the value of :math:\sigma_R^2 is given by:

.. math::
\sigma_R^2 = \left(r^{\prime }V^{-1}r\right)/\left(n-p\right)\text{.}

Case weights, :math:W_c, can be incorporated into the model by replacing :math:X^{\prime }X and :math:Z^{\prime }Z with :math:X^{\prime }W_cX and :math:Z^{\prime }W_cZ respectively, for a diagonal weight matrix :math:W_c.

The log-likelihood, :math:l_R, is calculated using the sweep algorithm detailed in Wolfinger et al. (1994).

.. _g02ja-py2-py-references:

**References**
Goodnight, J H, 1979, A tutorial on the SWEEP operator, The American Statistician (33(3)), 149--158

Harville, D A, 1977, Maximum likelihood approaches to variance component estimation and to related problems, JASA (72), 320--340

Rao, C R, 1972, Estimation of variance and covariance components in a linear model, J. Am. Stat. Assoc. (67), 112--115

Stroup, W W, 1989, Predictable functions and prediction space in the mixed model procedure, Applications of Mixed Models in Agriculture and Related Disciplines (Southern Cooperative Series Bulletin No. 343), 39--48

Wolfinger, R, Tobias, R and Sall, J, 1994, Computing Gaussian likelihoods and their derivatives for general linear mixed models, SIAM Sci. Statist. Comput. (15), 1294--1310
"""
raise NotImplementedError

[docs]def mixeff_ml(dat, levels, yvid, cwid, fvid, fint, rvid, nvpr, vpr, rint, svid, gamma, lb, maxit=-1, tol=0.0, io_manager=None):
r"""
mixeff_ml fits a linear mixed effects regression model using maximum likelihood (ML).

.. deprecated:: 27.0.0.0
mixeff_ml is deprecated.
Please use :meth:lmm_init followed by :meth:lmm_fit instead.
See also the :ref:Replacement Calls <replace> document.

.. _g02jb-py2-py-doc:

For full information please refer to the NAG Library document for g02jb

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jbf.html

.. _g02jb-py2-py-parameters:

**Parameters**
**dat** : float, array-like, shape :math:\left(n, \textit{ncol}\right)
Array containing all of the data. For the :math:i\ th observation:

:math:\mathrm{dat}[i-1,\mathrm{yvid}-1] holds the dependent variable, :math:y;

if :math:\mathrm{cwid}\neq 0, :math:\mathrm{dat}[i-1,\mathrm{cwid}-1] holds the case weights;

if :math:\mathrm{svid}\neq 0, :math:\mathrm{dat}[i-1,\mathrm{svid}-1] holds the subject variable.

The remaining columns hold the values of the independent variables.

**levels** : int, array-like, shape :math:\left(\textit{ncol}\right)
:math:\mathrm{levels}[i-1] contains the number of levels associated with the :math:i\ th variable of the data matrix :math:\mathrm{dat}. If this variable is continuous or binary (i.e., only takes the values zero or one) then :math:\mathrm{levels}[i-1] should be :math:1; if the variable is discrete then :math:\mathrm{levels}[i-1] is the number of levels associated with it and :math:\mathrm{dat}[\textit{j}-1,i-1] is assumed to take the values :math:1 to :math:\mathrm{levels}[i-1], for :math:\textit{j} = 1,2,\ldots,n.

**yvid** : int
The column of :math:\mathrm{dat} holding the dependent, :math:y, variable.

**cwid** : int
The column of :math:\mathrm{dat} holding the case weights.

If :math:\mathrm{cwid} = 0, no weights are used.

**fvid** : int, array-like, shape :math:\left(\textit{nfv}\right)
The columns of the data matrix :math:\mathrm{dat} holding the fixed independent variables with :math:\mathrm{fvid}[i-1] holding the column number corresponding to the :math:i\ th fixed variable.

**fint** : int
Flag indicating whether a fixed intercept is included (:math:\mathrm{fint} = 1).

**rvid** : int, array-like, shape :math:\left(\textit{nrv}\right)
The columns of the data matrix :math:\mathrm{dat} holding the random independent variables with :math:\mathrm{rvid}[i-1] holding the column number corresponding to the :math:i\ th random variable.

**nvpr** : int
If :math:\mathrm{rint} = 1 and :math:\mathrm{svid}\neq 0, :math:\mathrm{nvpr} is the number of variance components being :math:\text{estimated}-2, (:math:g-1), else :math:\mathrm{nvpr} = g.

If :math:\textit{nrv} = 0, :math:\mathrm{nvpr} is not referenced.

**vpr** : int, array-like, shape :math:\left(\textit{nrv}\right)
:math:\mathrm{vpr}[i-1] holds a flag indicating the variance of the :math:i\ th random variable. The variance of the :math:i\ th random variable is :math:\sigma_j^2, where :math:j = \mathrm{vpr}[i-1]+1 if :math:\mathrm{rint} = 1 and :math:\mathrm{svid} \neq 0 and :math:j = \mathrm{vpr}[i-1] otherwise. Random variables with the same value of :math:j are assumed to be taken from the same distribution.

**rint** : int
Flag indicating whether a random intercept is included (:math:\mathrm{rint} = 1).

If :math:\mathrm{svid} = 0, :math:\mathrm{rint} is not referenced.

**svid** : int
The column of :math:\mathrm{dat} holding the subject variable.

If :math:\mathrm{svid} = 0, no subject variable is used.

Specifying a subject variable is equivalent to specifying the interaction between that variable and all of the random-effects.

Letting the notation :math:Z_1\times Z_S denote the interaction between variables :math:Z_1 and :math:Z_S, fitting a model with :math:\mathrm{rint} = 0, random-effects :math:Z_1+Z_2 and subject variable :math:Z_S is equivalent to fitting a model with random-effects :math:Z_1\times Z_S+Z_2\times Z_S and no subject variable.

If :math:\mathrm{rint} = 1 the model is equivalent to fitting :math:Z_S+Z_1\times Z_S+Z_2\times Z_S and no subject variable.

**gamma** : float, array-like, shape :math:\left(\mathrm{nvpr}+2\right)
Holds the initial values of the variance components, :math:\gamma_0, with :math:\mathrm{gamma}[\textit{i}-1] the initial value for :math:\sigma_{\textit{i}}^2/\sigma_R^2, for :math:\textit{i} = 1,2,\ldots,g. If :math:\mathrm{rint} = 1 and :math:\mathrm{svid}\neq 0, :math:g = \mathrm{nvpr}+1, else :math:g = \mathrm{nvpr}.

If :math:\mathrm{gamma}[0] = {-1.0}, the remaining elements of :math:\mathrm{gamma} are ignored and the initial values for the variance components are estimated from the data using MIVQUE0.

**lb** : int
The size of the array :math:\mathrm{b}.

**maxit** : int, optional
The maximum number of iterations.

If :math:\mathrm{maxit} < 0, the default value of :math:100 is used.

If :math:\mathrm{maxit} = 0, the parameter estimates :math:\left(\beta, \nu \right) and corresponding standard errors are calculated based on the value of :math:\gamma_0 supplied in :math:\mathrm{gamma}.

**tol** : float, optional
The tolerance used to assess convergence.

If :math:\mathrm{tol}\leq 0.0, the default value of :math:\epsilon^{0.7} is used, where :math:\epsilon is the machine precision.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**gamma** : float, ndarray, shape :math:\left(\mathrm{nvpr}+2\right)
:math:\mathrm{gamma}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,g, holds the final estimate of :math:\sigma_{\textit{i}}^2 and :math:\mathrm{gamma}[g] holds the final estimate for :math:\sigma_R^2.

**nff** : int
The number of fixed effects estimated (i.e., the number of columns, :math:p, in the design matrix :math:X).

**nrf** : int
The number of random effects estimated (i.e., the number of columns, :math:q, in the design matrix :math:Z).

**df** : int
The degrees of freedom.

**ml** : float
:math:-2l_R\left(\hat{\gamma }\right) where :math:l_R is the log of the maximum likelihood calculated at :math:\hat{\gamma }, the estimated variance components returned in :math:\mathrm{gamma}.

**b** : float, ndarray, shape :math:\left(\mathrm{lb}\right)
The parameter estimates, :math:\left(\beta, \nu \right), with the first :math:\mathrm{nff} elements of :math:\mathrm{b} containing the fixed effect parameter estimates, :math:\beta and the next :math:\mathrm{nrf} elements of :math:\mathrm{b} containing the random effect parameter estimates, :math:\nu.

**Fixed effects**

If :math:\mathrm{fint} = 1, :math:\mathrm{b}[0] contains the estimate of the fixed intercept.

Let :math:L_i denote the number of levels associated with the :math:i\ th fixed variable, that is :math:L_i = \mathrm{levels}[ \mathrm{fvid}[i-1] -1].

Define

if :math:\mathrm{fint} = 1, :math:F_1 = 2 else if :math:\mathrm{fint} = 0, :math:F_1 = 1;

:math:F_{{i+1}} = F_i+\mathrm{max}\left({L_i-1}, 1\right), :math:i\geq 1.

Then for :math:i = 1,2,\ldots,\textit{nfv}:

if :math:L_i > 1, :math:\mathrm{b}[ F_i+\textit{j}-2 -1] contains the parameter estimate for the :math:\textit{j}\ th level of the :math:i\ th fixed variable, for :math:\textit{j} = 2,3,\ldots,L_i;

if :math:L_i\leq 1, :math:\mathrm{b}[F_i-1] contains the parameter estimate for the :math:i\ th fixed variable.

**Random effects**

Redefining :math:L_i to denote the number of levels associated with the :math:i\ th random variable, that is :math:L_i = \mathrm{levels}[ \mathrm{rvid}[i-1] -1].

Define

if :math:\mathrm{rint} = 1, :math:R_1 = 2 else if :math:\mathrm{rint} = 0, :math:R_1 = 1;

:math:R_{{i+1}} = R_i+L_i, :math:i\geq 1.

Then for :math:i = 1,2,\ldots,\textit{nrv}:

if :math:\mathrm{svid} = 0,

if :math:L_i > 1, :math:\mathrm{b}[ \mathrm{nff} + R_i +\textit{j}-1 -1] contains the parameter estimate for the :math:\textit{j}\ th level of the :math:i\ th random variable, for :math:\textit{j} = 1,2,\ldots,L_i;

if :math:L_i\leq 1, :math:\mathrm{b}[ \mathrm{nff} + R_i -1] contains the parameter estimate for the :math:i\ th random variable;

if :math:\mathrm{svid}\neq 0,

let :math:L_S denote the number of levels associated with the subject variable, that is :math:L_S = \mathrm{levels}[ \mathrm{svid} -1];

if :math:L_i > 1, :math:\mathrm{b}[ \mathrm{nff} + \left(\textit{s}-1\right) L_S + R_i + \textit{j} - 1 -1] contains the parameter estimate for the interaction between the :math:\textit{s}\ th level of the subject variable and the :math:\textit{j}\ th level of the :math:i\ th random variable, for :math:\textit{j} = 1,2,\ldots,L_i, for :math:\textit{s} = 1,2,\ldots,L_S;

if :math:L_i\leq 1, :math:\mathrm{b}[ \mathrm{nff} + \left(\textit{s}-1\right) L_S + R_i -1] contains the parameter estimate for the interaction between the :math:\textit{s}\ th level of the subject variable and the :math:i\ th random variable, for :math:\textit{s} = 1,2,\ldots,L_S;

if :math:\mathrm{rint} = 1, :math:\mathrm{b}[\mathrm{nff}] contains the estimate of the random intercept.

**se** : float, ndarray, shape :math:\left(\mathrm{lb}\right)
The standard errors of the parameter estimates given in :math:\mathrm{b}.

**warn** : int
Is set to :math:1 if a variance component was estimated to be a negative value during the fitting process. Otherwise :math:\mathrm{warn} is set to :math:0.

If :math:\mathrm{warn} = 1, the negative estimate is set to zero and the estimation process allowed to continue.

.. _g02jb-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 1.

(errno :math:1)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: number of observations with nonzero weights must be greater than one.

(errno :math:1)
On entry, :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ncol} \geq 1.

(errno :math:1)
On entry, :math:\textit{nfv} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \textit{nfv} < \textit{ncol}.

(errno :math:1)
On entry, :math:\textit{nrv} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \textit{nrv} < \textit{ncol} and :math:\textit{nrv}+\mathrm{rint} > 0.

(errno :math:1)
On entry, :math:\mathrm{nvpr} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{nrv} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \mathrm{nvpr}\leq \textit{nrv} and (:math:\textit{nrv} \leq 0 or :math:\mathrm{nvpr} \geq 1).

(errno :math:1)
On entry, :math:\mathrm{yvid} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{yvid}\leq \textit{ncol}.

(errno :math:1)
On entry, :math:\mathrm{svid} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \mathrm{svid}\leq \textit{ncol}.

(errno :math:1)
On entry, :math:\mathrm{cwid} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:0\leq \mathrm{cwid}\leq \textit{ncol} and any supplied weights must be :math:\text{}\geq 0.0.

(errno :math:1)
On entry, :math:\mathrm{fint} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{fint} = 0 or :math:1.

(errno :math:1)
On entry, :math:\mathrm{rint} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{rint} = 0 or :math:1.

(errno :math:1)
On entry, :math:\mathrm{lb} too small: :math:\mathrm{lb} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:2)
On entry, :math:\mathrm{levels}[\textit{i}] < 1, for at least one :math:\textit{i}.

(errno :math:2)
On entry, :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{fvid}[i]\leq \textit{ncol}, for all :math:i.

(errno :math:2)
On entry, :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{rvid}[i]\leq \textit{ncol}, for all :math:i.

(errno :math:2)
On entry, :math:\mathrm{nvpr} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{vpr}[i]\leq \mathrm{nvpr}, for all :math:i.

(errno :math:2)
On entry, invalid data: categorical variable with value greater than that specified in :math:\mathrm{levels}.

(errno :math:2)
On entry, :math:\mathrm{gamma}[\textit{i}] < 0, for at least one :math:\textit{i}.

(errno :math:3)
Degrees of freedom :math:< 1: :math:\mathrm{df} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:4)
Routine failed to converge to specified tolerance: :math:\mathrm{tol} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:4)
Routine failed to converge in :math:\mathrm{maxit} iterations: :math:\mathrm{maxit} = \langle\mathit{\boldsymbol{value}}\rangle.

.. _g02jb-py2-py-notes:

**Notes**
mixeff_ml fits a model of the form:

.. math::
y = X\beta +Z\nu +\epsilon

where

:math:y is a vector of :math:n observations on the dependent variable,

:math:X is a known :math:n\times p design matrix for the fixed independent variables,

:math:\beta is a vector of length :math:p of unknown fixed effects,

:math:Z is a known :math:n\times q design matrix for the random independent variables,

:math:\nu is a vector of length :math:q of unknown random effects;

and

:math:\epsilon is a vector of length :math:n of unknown random errors.

Both :math:\nu and :math:\epsilon are assumed to have a Gaussian distribution with expectation zero and

.. math::
\mathrm{Var}\left[\begin{array}{c}\nu \\\epsilon \end{array}\right] = \left[\begin{array}{cc}G&0\\0&R\end{array}\right]

where :math:R = \sigma_R^2I, :math:I is the :math:n\times n identity matrix and :math:G is a diagonal matrix.
It is assumed that the random variables, :math:Z, can be subdivided into :math:g\leq q groups with each group being identically distributed with expectations zero and variance :math:\sigma_i^2.
The diagonal elements of matrix :math:G, therefore, take one of the values :math:\left\{\sigma_i^2:i = 1,2,\ldots,g\right\}, depending on which group the associated random variable belongs to.

The model, therefore, contains three sets of unknowns, the fixed effects, :math:\beta, the random effects :math:\nu and a vector of :math:g+1 variance components, :math:\gamma, where :math:\gamma = \left\{\sigma_1^2, \sigma_2^2, \ldots, \sigma_{{g-1}}^2, \sigma_g^2, \sigma_R^2\right\}.
Rather than working directly with :math:\gamma, mixeff_ml uses an iterative process to estimate :math:\gamma^* = \left\{{\sigma_1^2/\sigma_R^2}, {\sigma_2^2/\sigma_R^2}, \ldots, {\sigma_{{g-1}}^2/\sigma_R^2}, {\sigma_g^2/\sigma_R^2}, 1\right\}.
Due to the iterative nature of the estimation a set of initial values, :math:\gamma_0, for :math:\gamma^* is required. mixeff_ml allows these initial values either to be supplied by you or calculated from the data using the minimum variance quadratic unbiased estimators (MIVQUE0) suggested by Rao (1972).

mixeff_ml fits the model using a quasi-Newton algorithm to maximize the log-likelihood function:

.. math::
-2l_R = \log\left(\left\lvert V\right\rvert \right)+\left(n\right)\log\left(r^{\prime }V^{-1}r\right)+\log\left(2\pi /n\right)

where

.. math::

Once the final estimates for :math:\gamma^* have been obtained, the value of :math:\sigma_R^2 is given by:

.. math::
\sigma_R^2 = \left(r^{\prime }V^{-1}r\right)/\left(n-p\right)\text{.}

Case weights, :math:W_c, can be incorporated into the model by replacing :math:X^{\prime }X and :math:Z^{\prime }Z with :math:X^{\prime }W_cX and :math:Z^{\prime }W_cZ respectively, for a diagonal weight matrix :math:W_c.

The log-likelihood, :math:l_R, is calculated using the sweep algorithm detailed in Wolfinger et al. (1994).

.. _g02jb-py2-py-references:

**References**
Goodnight, J H, 1979, A tutorial on the SWEEP operator, The American Statistician (33(3)), 149--158

Harville, D A, 1977, Maximum likelihood approaches to variance component estimation and to related problems, JASA (72), 320--340

Rao, C R, 1972, Estimation of variance and covariance components in a linear model, J. Am. Stat. Assoc. (67), 112--115

Stroup, W W, 1989, Predictable functions and prediction space in the mixed model procedure, Applications of Mixed Models in Agriculture and Related Disciplines (Southern Cooperative Series Bulletin No. 343), 39--48

Wolfinger, R, Tobias, R and Sall, J, 1994, Computing Gaussian likelihoods and their derivatives for general linear mixed models, SIAM Sci. Statist. Comput. (15), 1294--1310
"""
raise NotImplementedError

[docs]def mixeff_hier_init(dat, levels, y, fixed, rndm, wt=None):
r"""
mixeff_hier_init preprocesses a dataset prior to fitting a linear mixed effects regression model of the following form via either :meth:mixeff_hier_reml or :meth:mixeff_hier_ml.

.. deprecated:: 27.0.0.0
mixeff_hier_init is deprecated.
Please use :meth:lmm_init instead.
See also the :ref:Replacement Calls <replace> document.

.. _g02jc-py2-py-doc:

For full information please refer to the NAG Library document for g02jc

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jcf.html

.. _g02jc-py2-py-parameters:

**Parameters**
**dat** : float, array-like, shape :math:\left(n, \textit{ncol}\right)
A matrix of data, with :math:\mathrm{dat}[i-1,j-1] holding the :math:i\ th observation on the :math:j\ th variable. The two design matrices :math:X and :math:Z are constructed from :math:\mathrm{dat} and the information given in :math:\mathrm{fixed} (for :math:X) and :math:\mathrm{rndm} (for :math:Z).

**levels** : int, array-like, shape :math:\left(\textit{ncol}\right)
:math:\mathrm{levels}[i-1] contains the number of levels associated with the :math:i\ th variable held in :math:\mathrm{dat}.

If the :math:i\ th variable is continuous or binary (i.e., only takes the values zero or one), then :math:\mathrm{levels}[i-1] must be set to :math:1.

Otherwise the :math:i\ th variable is assumed to take an integer value between :math:1 and :math:\mathrm{levels}[i-1], (i.e., the :math:i\ th variable is discrete with :math:\mathrm{levels}[i-1] levels).

**y** : float, array-like, shape :math:\left(n\right)
:math:y, the vector of observations on the dependent variable.

**fixed** : int, array-like, shape :math:\left(\textit{lfixed}\right)
Defines the structure of the fixed effects design matrix, :math:X.

:math:\mathrm{fixed}[0]

The number of variables, :math:N_F, to include as fixed effects (not including the intercept if present).

:math:\mathrm{fixed}[1]

The fixed intercept flag which must contain :math:1 if a fixed intercept is to be included and :math:0 otherwise.

:math:\mathrm{fixed}[2+i-1]

The column of :math:\mathrm{dat} holding the :math:\textit{i}\ th fixed variable, for :math:\textit{i} = 1,2,\ldots,\mathrm{fixed}[0].

See Construction of the fixed effects design matrix, X <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jcf.html#fc-constructionofx>__ for more details on the construction of :math:X.

**rndm** : int, array-like, shape :math:\left(:, \textit{nrndm}\right)
:math:\mathrm{rndm}[i-1,j-1] defines the structure of the random effects design matrix, :math:Z. The :math:b\ th column of :math:\mathrm{rndm} defines a block of columns in the design matrix :math:Z.

:math:\mathrm{rndm}[0,b-1]

The number of variables, :math:N_{R_b}, to include as random effects in the :math:b\ th block (not including the random intercept if present).

:math:\mathrm{rndm}[1,b-1]

The random intercept flag which must contain :math:1 if block :math:b includes a random intercept and :math:0 otherwise.

:math:\mathrm{rndm}[2+i-1,b-1]

The column of :math:\mathrm{dat} holding the :math:\textit{i}\ th random variable in the :math:b\ th block, for :math:\textit{i} = 1,2,\ldots,\mathrm{rndm}[0,b-1].

:math:\mathrm{rndm}[3+N_{R_b}-1,b-1]

The number of subject variables, :math:N_{S_b}, for the :math:b\ th block. The subject variables define the nesting structure for this block.

:math:\mathrm{rndm}[3+N_{R_b}+i-1,b-1]

The column of :math:\mathrm{dat} holding the :math:\textit{i}\ th subject variable in the :math:b\ th block, for :math:\textit{i} = 1,2,\ldots,\mathrm{rndm}[3+N_{R_b}-1,b-1].

See Construction of random effects design matrix, Z <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jcf.html#fc-constructionofz>__ for more details on the construction of :math:Z.

**wt** : None or float, array-like, shape :math:\left(:\right), optional
Note: the required length for this argument is determined as follows: if :math:\mathrm{wt}\text{ is not }\mathbf{None}: :math:n; otherwise: :math:0.

Optionally, the weights to be used in the weighted regression.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model, in which case the effective number of observations is the number of observations with nonzero weights.

If weights are not provided then :math:\mathrm{wt} must be set to **None** and the effective number of observations is :math:\textit{n}.

**Returns**
**nff** : int
:math:p, the number of fixed effects estimated, i.e., the number of columns in the design matrix :math:X.

**nlsv** : int
The number of levels for the overall subject variable (see Construction of random effects design matrix, Z <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jcf.html#fc-constructionofz>__ for a description of what this means). If there is no overall subject variable, :math:\mathrm{nlsv} = 1.

**nrf** : int
The number of random effects estimated in each of the overall subject blocks. The number of columns in the design matrix :math:Z is given by :math:q = \mathrm{nrf}\times \mathrm{nlsv}.

**comm** : dict, communication object
Communication structure.

.. _g02jc-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:2)
On entry, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq 1.

(errno :math:3)
On entry, :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ncol} \geq 0.

(errno :math:4)
On entry, variable :math:j of observation :math:i is less than :math:1 or greater than :math:\mathrm{levels}[j-1]: :math:i = \langle\mathit{\boldsymbol{value}}\rangle, :math:j = \langle\mathit{\boldsymbol{value}}\rangle, value :math:= \langle\mathit{\boldsymbol{value}}\rangle, :math:\mathrm{levels}[j-1] = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:6)
On entry, :math:\mathrm{levels}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{levels}[i-1] \geq 1.

(errno :math:8)
On entry, :math:\mathrm{wt}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{wt}[i-1] \geq 0.0.

(errno :math:9)
On entry, number of fixed parameters, :math:\langle\mathit{\boldsymbol{value}}\rangle is less than zero.

(errno :math:10)
On entry, :math:\textit{lfixed} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{lfixed} \geq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:11)
On entry, :math:\textit{nrndm} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{nrndm} > 0.

(errno :math:12)
On entry, number of random parameters for random statement :math:i is less than :math:0: :math:i = \langle\mathit{\boldsymbol{value}}\rangle, number of parameters :math:\text{} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:102)
On entry, more fixed factors than observations, :math:n = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:n \geq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:108)
On entry, no observations due to zero weights.

(errno :math:109)
On entry, invalid value for fixed intercept flag: value :math:\text{} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:112)
On entry, invalid value for random intercept flag for random statement :math:i: :math:i = \langle\mathit{\boldsymbol{value}}\rangle, value :math:\text{} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:209)
On entry, index of fixed variable :math:j is less than :math:1 or greater than :math:\textit{ncol}: :math:j = \langle\mathit{\boldsymbol{value}}\rangle, index :math:\text{} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:212)
On entry, must be at least one parameter, or an intercept in each random statement :math:i: :math:i = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:312)
On entry, index of random variable :math:j in random statement :math:i is less than :math:1 or greater than :math:\textit{ncol}: :math:i = \langle\mathit{\boldsymbol{value}}\rangle, :math:j = \langle\mathit{\boldsymbol{value}}\rangle, index :math:\text{} = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{ncol} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:412)
On entry, number of subject parameters for random statement :math:i is less than :math:0: :math:i = \langle\mathit{\boldsymbol{value}}\rangle, number of parameters :math:\text{} = \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:512)
On entry, nesting variable :math:j in random statement :math:i has one level: :math:j = \langle\mathit{\boldsymbol{value}}\rangle, :math:i = \langle\mathit{\boldsymbol{value}}\rangle.

.. _g02jc-py2-py-notes:

**Notes**
mixeff_hier_init must be called prior to fitting a linear mixed effects regression model with either :meth:mixeff_hier_reml or :meth:mixeff_hier_ml.

The model fitting functions :meth:mixeff_hier_reml and :meth:mixeff_hier_ml fit a model of the following form:

.. math::
y = X\beta +Z\nu +\epsilon

.. rst-class:: nag-rules-none nag-align-left

+-----+----------------------------------------------------------------------------------+
|where|:math:y is a vector of :math:n observations on the dependent variable,        |
+-----+----------------------------------------------------------------------------------+
|     |:math:X is an :math:n\times p design matrix of fixed independent variables, |
+-----+----------------------------------------------------------------------------------+
|     |:math:\beta is a vector of :math:p unknown fixed effects,                   |
+-----+----------------------------------------------------------------------------------+
|     |:math:Z is an :math:n\times q design matrix of random independent variables,|
+-----+----------------------------------------------------------------------------------+
|     |:math:\nu is a vector of length :math:q of unknown random effects,          |
+-----+----------------------------------------------------------------------------------+
|     |:math:\epsilon is a vector of length :math:n of unknown random errors,        |
+-----+----------------------------------------------------------------------------------+

and :math:\nu and :math:\epsilon are Normally distributed with expectation zero and variance/covariance matrix defined by

.. math::
\mathrm{Var}\left[\begin{array}{c}\nu \\\epsilon \end{array}\right] = \left[\begin{array}{cc}G&0\\0&R\end{array}\right]

where :math:R = \sigma_R^2I, :math:I is the :math:n\times n identity matrix and :math:G is a diagonal matrix.

Case weights can be incorporated into the model by replacing :math:X and :math:Z with :math:W_c^{{1/2}}X and :math:W_c^{{1/2}}Z respectively where :math:W_c is a diagonal weight matrix.
"""
raise NotImplementedError

[docs]def mixeff_hier_reml(vpr, gamma, comm, iopt=None, ropt=None, io_manager=None):
r"""
mixeff_hier_reml fits a multi-level linear mixed effects regression model using restricted maximum likelihood (REML).
Prior to calling mixeff_hier_reml the initialization function :meth:mixeff_hier_init must be called.

.. deprecated:: 27.0.0.0
mixeff_hier_reml is deprecated.
Please use :meth:lmm_fit instead.
See also the :ref:Replacement Calls <replace> document.

.. _g02jd-py2-py-doc:

For full information please refer to the NAG Library document for g02jd

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jdf.html

.. _g02jd-py2-py-parameters:

**Parameters**
**vpr** : int, array-like, shape :math:\left(\textit{lvpr}\right)
A vector of flags indicating the mapping between the random variables specified in :math:\textit{rndm} and the variance components, :math:\sigma_i^2. See Further Comments <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jdf.html#fcomments>__ for more details.

**gamma** : float, array-like, shape :math:\left(\textit{nvpr}+1\right)
Holds the initial values of the variance components, :math:\gamma_0, with :math:\mathrm{gamma}[\textit{i}-1] the initial value for :math:\sigma_{\textit{i}}^2/\sigma_R^2, for :math:\textit{i} = 1,2,\ldots,\textit{nvpr}.

If :math:\mathrm{gamma}[0] = {-1.0}, the remaining elements of :math:\mathrm{gamma} are ignored and the initial values for the variance components are estimated from the data using MIVQUE0.

**comm** : dict, communication object, modified in place
Communication structure.

This argument must have been initialized by a prior call to :meth:mixeff_hier_init.

**iopt** : None or int, array-like, shape :math:\left(\textit{liopt}\right), optional
Options passed to the optimization function.

By default mixeff_hier_reml fits the specified model using a modified Newton optimization algorithm as implemented in :meth:opt.bounds_mod_deriv2_comp <naginterfaces.library.opt.bounds_mod_deriv2_comp>. In some cases, where the calculation of the derivatives is computationally expensive it may be more efficient to use a sequential QP algorithm.

The sequential QP algorithm as implemented in :meth:opt.nlp1_solve <naginterfaces.library.opt.nlp1_solve> can be chosen by setting :math:\mathrm{iopt}[4] = 1.

If :math:\textit{liopt} < 5 or :math:\mathrm{iopt}[4]\neq 1, then the modified Newton algorithm will be used.

Different options are available depending on the optimization function used.

In all cases, using a value of :math:-1 will cause the default value to be used.

In addition only the first :math:\textit{liopt} values of :math:\mathrm{iopt} are used, so for example, if only the first element of :math:\mathrm{iopt} needs changing and default values for all other options are sufficient :math:\textit{liopt} can be set to :math:1.

The following table lists the association between elements of :math:\mathrm{iopt} and arguments in the optimizer when the modified Newton algorithm is being used.

.. rst-class:: nag-rules-none nag-align-left

+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:i|Description                                     |Equivalent argument    |Default Value                     |
+=========+================================================+=======================+==================================+
|:math:0|Number of iterations                            |:math:\textit{maxcal}|:math:1000                      |
+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:1|Unit number for monitoring information          |n/a                    |The advisory unit number          |
+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:2|Print options (:math:1 = print)               |n/a                    |:math:-1 (no printing performed)|
+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:3|Frequency that monitoring information is printed|:math:\textit{iprint}|:math:-1                        |
+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:4|Optimizer used                                  |n/a                    |n/a                               |
+---------+------------------------------------------------+-----------------------+----------------------------------+

If requested, monitoring information is displayed in a similar format to that given by the modified Newton optimizer.

The following table lists the association between elements of :math:\mathrm{iopt} and options in the optimizer when the sequential QP algorithm is being used.

.. rst-class:: nag-rules-none nag-align-left

+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:i|Description                                                  |Equivalent option      |Default Value                                            |
+=========+=============================================================+=======================+=========================================================+
|:math:0|Number of iterations                                         |'Major Iteration Limit'|:math:\mathrm{max}\left(50,3\times \textit{nvpr}\right)|
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:1|Unit number for monitoring information                       |n/a                    |The advisory unit number                                 |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:2|Print options (:math:1 = \text{} print, otherwise no print)|'List'/'Nolist'        |:math:-1 (no printing performed)                       |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:3|Frequency that monitoring information is printed             |'Major Print Level'    |:math:0                                                |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:4|Optimizer used                                               |n/a                    |n/a                                                      |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:5|Number of minor iterations                                   |'Minor Iteration Limit'|:math:\mathrm{max}\left(50,3\times \textit{nvpr}\right)|
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:6|Frequency that additional monitoring information is printed  |'Minor Print Level'    |:math:0                                                |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+

If :math:\textit{liopt}\leq 0, default values are used for all options and :math:\mathrm{iopt} is not referenced.

**ropt** : None or float, array-like, shape :math:\left(\textit{lropt}\right), optional
Options passed to the optimization function.

Different options are available depending on the optimization function used.

In all cases, using a value of :math:-1.0 will cause the default value to be used.

In addition only the first :math:\textit{lropt} values of :math:\mathrm{ropt} are used, so for example, if only the first element of :math:\mathrm{ropt} needs changing and default values for all other options are sufficient :math:\textit{lropt} can be set to :math:1.

The following table lists the association between elements of :math:\mathrm{ropt} and arguments in the optimizer when the modified Newton algorithm is being used.

.. rst-class:: nag-rules-none nag-align-left

+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:i|Description                           |Equivalent argument    |Default Value                                                                                                               |
+=========+======================================+=======================+============================================================================================================================+
|:math:0|Sweep tolerance                       |n/a                    |:math:\mathrm{max}\left(\sqrt{\mathrm{eps}},\sqrt{\mathrm{eps}}\times \mathrm{max}_i\left(\textit{zz}_{{ii}}\right)\right)|
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:1|Lower bound for :math:\gamma^*      |n/a                    |:math:\mathrm{eps}/100                                                                                                    |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:2|Upper bound for :math:\gamma^*      |n/a                    |:math:10^{20}                                                                                                             |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:3|Accuracy of linear minimizations      |:math:\textit{eta}   |:math:0.9                                                                                                                 |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:4|Accuracy to which solution is required|:math:\textit{xtol}  |:math:0.0                                                                                                                 |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:5|Initial distance from solution        |:math:\textit{stepmx}|:math:100000.0                                                                                                            |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+

The following table lists the association between elements of :math:\mathrm{ropt} and options in the optimizer when the sequential QP algorithm is being used.

.. rst-class:: nag-rules-none nag-align-left

+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:i|Description                     |Equivalent option      |Default Value                                                                                                               |
+=========+================================+=======================+============================================================================================================================+
|:math:0|Sweep tolerance                 |n/a                    |:math:\mathrm{max}\left(\sqrt{\mathrm{eps}},\sqrt{\mathrm{eps}}\times \mathrm{max}_i\left(\textit{zz}_{{ii}}\right)\right)|
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:1|Lower bound for :math:\gamma^*|n/a                    |:math:\mathrm{eps}/100                                                                                                    |
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:2|Upper bound for :math:\gamma^*|n/a                    |:math:10^{20}                                                                                                             |
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:3|Line search tolerance           |'Line Search Tolerance'|:math:0.9                                                                                                                 |
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:4|Optimality tolerance            |'Optimality Tolerance' |:math:\mathrm{eps}^{0.72}                                                                                                 |
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+

where :math:\mathrm{eps} is the machine precision returned by :meth:machine.precision <naginterfaces.library.machine.precision> and :math:\textit{zz}_{{ii}} denotes the :math:i diagonal element of :math:Z^\mathrm{T}Z.

If :math:\textit{lropt}\leq 0, then default values are used for all options and :math:\mathrm{ropt} may be set to **None**.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**gamma** : float, ndarray, shape :math:\left(\textit{nvpr}+1\right)
:math:\mathrm{gamma}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,\textit{nvpr}, holds the final estimate of :math:\sigma_{\textit{i}}^2 and :math:\mathrm{gamma}[\textit{nvpr}] holds the final estimate for :math:\sigma_R^2.

**effn** : int
Effective number of observations. If no weights were supplied to :meth:mixeff_hier_init or all supplied weights were nonzero, :math:\mathrm{effn} = {\textit{n}}.

**rnkx** : int
The rank of the design matrix, :math:X, for the fixed effects.

**ncov** : int
Number of variance components not estimated to be zero. If none of the variance components are estimated to be zero, then :math:\mathrm{ncov} = \textit{nvpr}.

**lnlike** : float
:math:-2l_R\left(\hat{\gamma }\right) where :math:l_R is the log of the restricted maximum likelihood calculated at :math:\hat{\gamma }, the estimated variance components returned in :math:\mathrm{gamma}.

**estid** : int, ndarray, shape :math:\left(:, \textit{lb}\right)
An array describing the parameter estimates returned in :math:\mathrm{b}. The first :math:{\textit{nlsv}}\times {\textit{nrf}} columns of :math:\mathrm{estid} describe the parameter estimates for the random effects and the last :math:\textit{nff} columns the parameter estimates for the fixed effects.

For fixed effects:

for :math:l = {\textit{nrf}}\times {\textit{nlsv}}+1,\ldots,{\textit{nrf}}\times {\textit{nlsv}}+{\textit{nff}}

if :math:\mathrm{b}[l-1] contains the parameter estimate for the intercept, then

.. math::
\mathrm{estid}[0,l-1] = \mathrm{estid}[1,l-1] = \mathrm{estid}[2,l-1] = 0\text{;}

if :math:\mathrm{b}[l-1] contains the parameter estimate for the :math:i\ th level of the :math:j\ th fixed variable, that is the vector of values held in the :math:k\ th column of :math:\textit{dat} when :math:{\textit{fixed}}[j+1] = k then

.. math::
\begin{array}{l} \mathrm{estid}[0,l-1] = 0\text{, }\quad \\ \mathrm{estid}[1,l-1] = j\text{, }\quad \\ \mathrm{estid}[2,l-1] = i\text{;} \end{array}

if the :math:j\ th variable is continuous or binary, that is :math:{\textit{levels}}[{\textit{fixed}}[j+1]-1] = 1, :math:\mathrm{estid}[2,l-1] = 0;

any remaining rows of the :math:l\ th column of :math:\mathrm{estid} are set to :math:{0}.

For random effects:

let

:math:N_{R_b} denote the number of random variables in the :math:b\ th random statement, that is :math:N_{R_b} = {\textit{rndm}}\left(1, b\right);

:math:R_{{jb}} denote the :math:j\ th random variable from the :math:b\ th random statement, that is the vector of values held in the :math:k\ th column of :math:\textit{dat} when :math:{\textit{rndm}}\left({2+j}, b\right) = k;

:math:N_{S_b} denote the number of subject variables in the :math:b\ th random statement, that is :math:N_{S_b} = {\textit{rndm}}\left({3+N_{R_b}}, b\right);

:math:S_{{jb}} denote the :math:j\ th subject variable from the :math:b\ th random statement, that is the vector of values held in the :math:k\ th column of :math:\textit{dat} when :math:{\textit{rndm}}\left({3+N_{R_b}+j}, b\right) = k;

:math:L\left(S_{{jb}}\right) denote the number of levels for :math:S_{{jb}}, that is :math:L\left(S_{{jb}}\right) = {\textit{levels}}[{\textit{rndm}}\left({3+N_{R_b}+j}, b\right)-1];

then

for :math:l = 1,2,\ldots {\textit{nrf}}\times {\textit{nlsv}}, if :math:\mathrm{b}[l-1] contains the parameter estimate for the :math:i\ th level of :math:R_{{jb}} when :math:S_{{\textit{k}b}} = s_{\textit{k}}, for :math:\textit{k} = 1,2,\ldots,N_{S_b} and :math:1\leq s_{\textit{k}}\leq L\left(S_{{jb}}\right), i.e., :math:s_k is a valid value for the :math:k\ th subject variable, then

.. math::
\begin{array}{l} \mathrm{estid}[0,l-1] = b\text{, }\quad \\ \mathrm{estid}[1,l-1] = j\text{, }\quad \\ \mathrm{estid}[2,l-1] = i\text{, }\quad \\ \mathrm{estid}[{3+k}-1,l-1] = s_k\text{, }k = 1,2,\ldots,N_{S_b}\text{;} \end{array}

if the parameter being estimated is for the intercept, then :math:\mathrm{estid}[1,l-1] = \mathrm{estid}[2,l-1] = 0;

if the :math:j\ th variable is continuous, or binary, that is :math:L\left(S_{{jb}}\right) = 1, then :math:\mathrm{estid}[2,l-1] = 0;

the remaining rows of the :math:l\ th column of :math:\mathrm{estid} are set to :math:{0}.

In some situations, certain combinations of variables are never observed.

In such circumstances all elements of the :math:l\ th row of :math:\mathrm{estid} are set to :math:-999.

**b** : float, ndarray, shape :math:\left(\textit{nff}+\textit{nrf}\times \textit{nlsv}\right)
The parameter estimates, with the first :math:{\textit{nrf}}\times {\textit{nlsv}} elements of :math:\mathrm{b} containing the parameter estimates for the random effects, :math:\nu, and the remaining :math:\textit{nff} elements containing the parameter estimates for the fixed effects, :math:\beta. The order of these estimates are described by the :math:\mathrm{estid} argument.

**se** : float, ndarray, shape :math:\left(\textit{nff}+\textit{nrf}\times \textit{nlsv}\right)
The standard errors of the parameter estimates given in :math:\mathrm{b}.

**czz** : float, ndarray, shape :math:\left(\textit{nrf}, :\right)
If :math:{\textit{nlsv}} = 1, then :math:\mathrm{czz} holds the lower triangular portion of the matrix :math:\left(1/\sigma^2\right)\left(Z^\mathrm{T} \hat{R}^{-1}Z + \hat{G}^{-1}\right), where :math:\hat{R} and :math:\hat{G} are the estimates of :math:R and :math:G respectively. If :math:{\textit{nlsv}} > 1, then :math:\mathrm{czz} holds this matrix in compressed form, with the first :math:\textit{nrf} columns holding the part of the matrix corresponding to the first level of the overall subject variable, the next :math:\textit{nrf} columns the part corresponding to the second level of the overall subject variable etc.

**cxx** : float, ndarray, shape :math:\left(\textit{nff}, :\right)
:math:\mathrm{cxx} holds the lower triangular portion of the matrix :math:\left(1/\sigma^2\right)X^\mathrm{T}\hat{V}^{-1}X, where :math:\hat{V} is the estimated value of :math:V.

**cxz** : float, ndarray, shape :math:\left(\textit{nff}, :\right)
If :math:{\textit{nlsv}} = 1, then :math:\mathrm{cxz} holds the matrix :math:\left(1/\sigma^2\right)\left(X^\mathrm{T}\hat{V}^{-1}Z\right)\hat{G}, where :math:\hat{V} and :math:\hat{G} are the estimates of :math:V and :math:G respectively. If :math:{\textit{nlsv}} > 1, then :math:\mathrm{cxz} holds this matrix in compressed form, with the first :math:\textit{nrf} columns holding the part of the matrix corresponding to the first level of the overall subject variable, the next :math:\textit{nrf} columns the part corresponding to the second level of the overall subject variable etc.

.. _g02jd-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{lvpr} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{lvpr} \geq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:2)
On entry, :math:\mathrm{vpr}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle and :math:\textit{nvpr} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{vpr}[i]\leq \textit{nvpr}.

(errno :math:3)
On entry, :math:\textit{nvpr} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \textit{nvpr}\leq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:4)
On entry, :math:\mathrm{gamma}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{gamma}[0] = {-1.0} or :math:\mathrm{gamma}[i-1] \geq 0.0.

(errno :math:9)
On entry, :math:\textit{lb} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{lb} \geq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:11)
On entry, :math:\textit{ldid} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ldid} \geq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:21)
On entry, :math:\mathrm{comm}\ ['iopts'] has not been initialized correctly.

(errno :math:32)
On entry, at least one value of i, for :math:\textit{i} = 1,2,\ldots,\textit{nvpr}, does not appear in :math:\mathrm{vpr}.

**Warns**
**NagAlgorithmicWarning**
(errno :math:101)
Optimal solution found, but requested accuracy not achieved.

(errno :math:102)
Too many major iterations.

(errno :math:103)
Current point cannot be improved upon.

(errno :math:104)
At least one negative estimate for :math:\mathrm{gamma} was obtained. All negative estimates have been set to zero.

.. _g02jd-py2-py-notes:

**Notes**
mixeff_hier_reml fits a model of the form:

.. math::
y = X\beta +Z\nu +\epsilon

.. rst-class:: nag-rules-none nag-align-left

+-----+--------------------------------------------------------------------------------------------+
|where|:math:y is a vector of :math:n observations on the dependent variable,                  |
+-----+--------------------------------------------------------------------------------------------+
|     |:math:X is a known :math:n\times p design matrix for the fixed independent variables, |
+-----+--------------------------------------------------------------------------------------------+
|     |:math:\beta is a vector of length :math:p of unknown fixed effects,                   |
+-----+--------------------------------------------------------------------------------------------+
|     |:math:Z is a known :math:n\times q design matrix for the random independent variables,|
+-----+--------------------------------------------------------------------------------------------+
|     |:math:\nu is a vector of length :math:q of unknown random effects,                    |
+-----+--------------------------------------------------------------------------------------------+
|and  |:math:\epsilon is a vector of length :math:n of unknown random errors.                  |
+-----+--------------------------------------------------------------------------------------------+

Both :math:\nu and :math:\epsilon are assumed to have a Gaussian distribution with expectation zero and variance/covariance matrix defined by

.. math::
\mathrm{Var}\left[\begin{array}{c}\nu \\\epsilon \end{array}\right] = \left[\begin{array}{cc}G&0\\0&R\end{array}\right]

where :math:R = \sigma_R^2I, :math:I is the :math:n\times n identity matrix and :math:G is a diagonal matrix.
It is assumed that the random variables, :math:Z, can be subdivided into :math:g\leq q groups with each group being identically distributed with expectation zero and variance :math:\sigma_i^2.
The diagonal elements of matrix :math:G, therefore, take one of the values :math:\left\{\sigma_i^2:i = 1,2,\ldots,g\right\}, depending on which group the associated random variable belongs to.

The model, therefore, contains three sets of unknowns: the fixed effects :math:\beta, the random effects :math:\nu and a vector of :math:g+1 variance components :math:\gamma, where :math:\gamma = \left\{\sigma_1^2, \sigma_2^2, \ldots, \sigma_{{g-1}}^2, \sigma_g^2, \sigma_R^2\right\}.
Rather than working directly with :math:\gamma, mixeff_hier_reml uses an iterative process to estimate :math:\gamma^* = \left\{{\sigma_1^2/\sigma_R^2}, {\sigma_2^2/\sigma_R^2}, \ldots, {\sigma_{{g-1}}^2/\sigma_R^2}, {\sigma_g^2/\sigma_R^2}, 1\right\}.
Due to the iterative nature of the estimation a set of initial values, :math:\gamma_0, for :math:\gamma^* is required. mixeff_hier_reml allows these initial values either to be supplied by you or calculated from the data using the minimum variance quadratic unbiased estimators (MIVQUE0) suggested by Rao (1972).

mixeff_hier_reml fits the model by maximizing the restricted log-likelihood function:

.. math::
-2l_R = \log\left(\left\lvert V\right\rvert \right)+\left(n-p\right)\log\left(r^\mathrm{T}V^{-1}r\right)+\log\left(\left\lvert X^\mathrm{T}V^{-1}X\right\rvert \right)+\left(n-p\right)\left(1+\log\left(2\pi /\left(n-p\right)\right)\right)

where

.. math::

Once the final estimates for :math:\gamma^* have been obtained, the value of :math:\sigma_R^2 is given by

.. math::
\sigma_R^2 = \left(r^\mathrm{T}V^{-1}r\right)/\left(n-p\right)\text{.}

Case weights, :math:W_c, can be incorporated into the model by replacing :math:X^\mathrm{T}X and :math:Z^\mathrm{T}Z with :math:X^\mathrm{T}W_cX and :math:Z^\mathrm{T}W_cZ respectively, for a diagonal weight matrix :math:W_c.

The log-likelihood, :math:l_R, is calculated using the sweep algorithm detailed in Wolfinger et al. (1994).

.. _g02jd-py2-py-references:

**References**
Goodnight, J H, 1979, A tutorial on the SWEEP operator, The American Statistician (33(3)), 149--158

Harville, D A, 1977, Maximum likelihood approaches to variance component estimation and to related problems, JASA (72), 320--340

Rao, C R, 1972, Estimation of variance and covariance components in a linear model, J. Am. Stat. Assoc. (67), 112--115

Stroup, W W, 1989, Predictable functions and prediction space in the mixed model procedure, Applications of Mixed Models in Agriculture and Related Disciplines (Southern Cooperative Series Bulletin No. 343), 39--48

Wolfinger, R, Tobias, R and Sall, J, 1994, Computing Gaussian likelihoods and their derivatives for general linear mixed models, SIAM Sci. Statist. Comput. (15), 1294--1310
"""
raise NotImplementedError

[docs]def mixeff_hier_ml(vpr, nvpr, gamma, comm, iopt=None, ropt=None, io_manager=None):
r"""
mixeff_hier_ml fits a multi-level linear mixed effects regression model using maximum likelihood (ML).
Prior to calling mixeff_hier_ml the initialization function :meth:mixeff_hier_init must be called.

.. deprecated:: 27.0.0.0
mixeff_hier_ml is deprecated.
Please use :meth:lmm_fit instead.
See also the :ref:Replacement Calls <replace> document.

.. _g02je-py2-py-doc:

For full information please refer to the NAG Library document for g02je

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jef.html

.. _g02je-py2-py-parameters:

**Parameters**
**vpr** : int, array-like, shape :math:\left(\textit{lvpr}\right)
A vector of flags indicating the mapping between the random variables specified in :math:\textit{rndm} and the variance components, :math:\sigma_i^2. See Further Comments <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jef.html#fcomments>__ for more details.

**nvpr** : int
:math:g, the number of variance components being estimated (excluding the overall variance, :math:\sigma_R^2).

**gamma** : float, array-like, shape :math:\left(\mathrm{nvpr}+1\right)
Holds the initial values of the variance components, :math:\gamma_0, with :math:\mathrm{gamma}[\textit{i}-1] the initial value for :math:\sigma_{\textit{i}}^2/\sigma_R^2, for :math:\textit{i} = 1,2,\ldots,\mathrm{nvpr}.

If :math:\mathrm{gamma}[0] = {-1.0}, the remaining elements of :math:\mathrm{gamma} are ignored and the initial values for the variance components are estimated from the data using MIVQUE0.

**comm** : dict, communication object, modified in place
Communication structure.

This argument must have been initialized by a prior call to :meth:mixeff_hier_init.

**iopt** : None or int, array-like, shape :math:\left(\textit{liopt}\right), optional
Options passed to the optimization function.

By default mixeff_hier_ml fits the specified model using a modified Newton optimization algorithm as implemented in :meth:opt.bounds_mod_deriv2_comp <naginterfaces.library.opt.bounds_mod_deriv2_comp>. In some cases, where the calculation of the derivatives is computationally expensive it may be more efficient to use a sequential QP algorithm.

The sequential QP algorithm as implemented in :meth:opt.nlp1_solve <naginterfaces.library.opt.nlp1_solve> can be chosen by setting :math:\mathrm{iopt}[4] = 1.

If :math:\textit{liopt} < 5 or :math:\mathrm{iopt}[4]\neq 1, then the modified Newton algorithm will be used.

Different options are available depending on the optimization function used.

In all cases, using a value of :math:-1 will cause the default value to be used.

In addition only the first :math:\textit{liopt} values of :math:\mathrm{iopt} are used, so for example, if only the first element of :math:\mathrm{iopt} needs changing and default values for all other options are sufficient :math:\textit{liopt} can be set to :math:1.

The following table lists the association between elements of :math:\mathrm{iopt} and arguments in the optimizer when the modified Newton algorithm is being used.

.. rst-class:: nag-rules-none nag-align-left

+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:i|Description                                     |Equivalent argument    |Default Value                     |
+=========+================================================+=======================+==================================+
|:math:0|Number of iterations                            |:math:\textit{maxcal}|:math:1000                      |
+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:1|Unit number for monitoring information          |n/a                    |The advisory unit number          |
+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:2|Print options (:math:1 = print)               |n/a                    |:math:-1 (no printing performed)|
+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:3|Frequency that monitoring information is printed|:math:\textit{iprint}|:math:-1                        |
+---------+------------------------------------------------+-----------------------+----------------------------------+
|:math:4|Optimizer used                                  |n/a                    |n/a                               |
+---------+------------------------------------------------+-----------------------+----------------------------------+

If requested, monitoring information is displayed in a similar format to that given by the modified Newton optimizer.

The following table lists the association between elements of :math:\mathrm{iopt} and options in the optimizer when the sequential QP algorithm is being used.

.. rst-class:: nag-rules-none nag-align-left

+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:i|Description                                                  |Equivalent option      |Default Value                                            |
+=========+=============================================================+=======================+=========================================================+
|:math:0|Number of iterations                                         |'Major Iteration Limit'|:math:\mathrm{max}\left(50,3\times \mathrm{nvpr}\right)|
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:1|Unit number for monitoring information                       |n/a                    |The advisory unit number                                 |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:2|Print options (:math:1 = \text{} print, otherwise no print)|'List'/'Nolist'        |:math:-1 (no printing performed)                       |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:3|Frequency that monitoring information is printed             |'Major Print Level'    |:math:0                                                |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:4|Optimizer used                                               |n/a                    |n/a                                                      |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:5|Number of minor iterations                                   |'Minor Iteration Limit'|:math:\mathrm{max}\left(50,3\times \mathrm{nvpr}\right)|
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+
|:math:6|Frequency that additional monitoring information is printed  |'Minor Print Level'    |:math:0                                                |
+---------+-------------------------------------------------------------+-----------------------+---------------------------------------------------------+

If :math:\textit{liopt}\leq 0, default values are used for all options and :math:\mathrm{iopt} is not referenced.

**ropt** : None or float, array-like, shape :math:\left(\textit{lropt}\right), optional
Options passed to the optimization function.

Different options are available depending on the optimization function used.

In all cases, using a value of :math:-1.0 will cause the default value to be used.

In addition only the first :math:\textit{lropt} values of :math:\mathrm{ropt} are used, so for example, if only the first element of :math:\mathrm{ropt} needs changing and default values for all other options are sufficient :math:\textit{lropt} can be set to :math:1.

The following table lists the association between elements of :math:\mathrm{ropt} and arguments in the optimizer when the modified Newton algorithm is being used.

.. rst-class:: nag-rules-none nag-align-left

+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:i|Description                           |Equivalent argument    |Default Value                                                                                                               |
+=========+======================================+=======================+============================================================================================================================+
|:math:0|Sweep tolerance                       |n/a                    |:math:\mathrm{max}\left(\sqrt{\mathrm{eps}},\sqrt{\mathrm{eps}}\times \mathrm{max}_i\left(\textit{zz}_{{ii}}\right)\right)|
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:1|Lower bound for :math:\gamma^*      |n/a                    |:math:\mathrm{eps}/100                                                                                                    |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:2|Upper bound for :math:\gamma^*      |n/a                    |:math:10^{20}                                                                                                             |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:3|Accuracy of linear minimizations      |:math:\textit{eta}   |:math:0.9                                                                                                                 |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:4|Accuracy to which solution is required|:math:\textit{xtol}  |:math:0.0                                                                                                                 |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:5|Initial distance from solution        |:math:\textit{stepmx}|:math:100000.0                                                                                                            |
+---------+--------------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+

The following table lists the association between elements of :math:\mathrm{ropt} and options in the optimizer when the sequential QP algorithm is being used.

.. rst-class:: nag-rules-none nag-align-left

+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:i|Description                     |Equivalent option      |Default Value                                                                                                               |
+=========+================================+=======================+============================================================================================================================+
|:math:0|Sweep tolerance                 |n/a                    |:math:\mathrm{max}\left(\sqrt{\mathrm{eps}},\sqrt{\mathrm{eps}}\times \mathrm{max}_i\left(\textit{zz}_{{ii}}\right)\right)|
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:1|Lower bound for :math:\gamma^*|n/a                    |:math:\mathrm{eps}/100                                                                                                    |
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:2|Upper bound for :math:\gamma^*|n/a                    |:math:10^{20}                                                                                                             |
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:3|Line search tolerance           |'Line Search Tolerance'|:math:0.9                                                                                                                 |
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+
|:math:4|Optimality tolerance            |'Optimality Tolerance' |:math:\mathrm{eps}^{0.72}                                                                                                 |
+---------+--------------------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------+

where :math:\mathrm{eps} is the machine precision returned by :meth:machine.precision <naginterfaces.library.machine.precision> and :math:\textit{zz}_{{ii}} denotes the :math:i diagonal element of :math:Z^\mathrm{T}Z.

If :math:\textit{lropt}\leq 0, then default values are used for all options and :math:\mathrm{ropt} may be set to **None**.

**io_manager** : FileObjManager, optional
Manager for I/O in this routine.

**Returns**
**gamma** : float, ndarray, shape :math:\left(\mathrm{nvpr}+1\right)
:math:\mathrm{gamma}[\textit{i}-1], for :math:\textit{i} = 1,2,\ldots,\mathrm{nvpr}, holds the final estimate of :math:\sigma_{\textit{i}}^2 and :math:\mathrm{gamma}[\mathrm{nvpr}] holds the final estimate for :math:\sigma_R^2.

**effn** : int
Effective number of observations. If no weights were supplied to :meth:mixeff_hier_init or all supplied weights were nonzero, :math:\mathrm{effn} = {\textit{n}}.

**rnkx** : int
The rank of the design matrix, :math:X, for the fixed effects.

**ncov** : int
Number of variance components not estimated to be zero. If none of the variance components are estimated to be zero, then :math:\mathrm{ncov} = \mathrm{nvpr}.

**lnlike** : float
:math:-2l_R\left(\hat{\gamma }\right) where :math:l_R is the log of the maximum likelihood calculated at :math:\hat{\gamma }, the estimated variance components returned in :math:\mathrm{gamma}.

**estid** : int, ndarray, shape :math:\left(:, \textit{lb}\right)
An array describing the parameter estimates returned in :math:\mathrm{b}. The first :math:{\textit{nlsv}}\times {\textit{nrf}} columns of :math:\mathrm{estid} describe the parameter estimates for the random effects and the last :math:\textit{nff} columns the parameter estimates for the fixed effects.

For fixed effects:

for :math:l = {\textit{nrf}}\times {\textit{nlsv}}+1,\ldots,{\textit{nrf}}\times {\textit{nlsv}}+{\textit{nff}}

if :math:\mathrm{b}[l-1] contains the parameter estimate for the intercept, then

.. math::
\mathrm{estid}[0,l-1] = \mathrm{estid}[1,l-1] = \mathrm{estid}[2,l-1] = 0\text{;}

if :math:\mathrm{b}[l-1] contains the parameter estimate for the :math:i\ th level of the :math:j\ th fixed variable, that is the vector of values held in the :math:k\ th column of :math:\textit{dat} when :math:{\textit{fixed}}[j+1] = k then

.. math::
\begin{array}{l} \mathrm{estid}[0,l-1] = 0\text{, }\quad \\ \mathrm{estid}[1,l-1] = j\text{, }\quad \\ \mathrm{estid}[2,l-1] = i\text{;} \end{array}

if the :math:j\ th variable is continuous or binary, that is :math:{\textit{levels}}[{\textit{fixed}}[j+1]-1] = 1, :math:\mathrm{estid}[2,l-1] = 0;

any remaining rows of the :math:l\ th column of :math:\mathrm{estid} are set to :math:{0}.

For random effects:

let

:math:N_{R_b} denote the number of random variables in the :math:b\ th random statement, that is :math:N_{R_b} = {\textit{rndm}}\left(1, b\right);

:math:R_{{jb}} denote the :math:j\ th random variable from the :math:b\ th random statement, that is the vector of values held in the :math:k\ th column of :math:\textit{dat} when :math:{\textit{rndm}}\left({2+j}, b\right) = k;

:math:N_{S_b} denote the number of subject variables in the :math:b\ th random statement, that is :math:N_{S_b} = {\textit{rndm}}\left({3+N_{R_b}}, b\right);

:math:S_{{jb}} denote the :math:j\ th subject variable from the :math:b\ th random statement, that is the vector of values held in the :math:k\ th column of :math:\textit{dat} when :math:{\textit{rndm}}\left({3+N_{R_b}+j}, b\right) = k;

:math:L\left(S_{{jb}}\right) denote the number of levels for :math:S_{{jb}}, that is :math:L\left(S_{{jb}}\right) = {\textit{levels}}[{\textit{rndm}}\left({3+N_{R_b}+j}, b\right)-1];

then

for :math:l = 1,2,\ldots {\textit{nrf}}\times {\textit{nlsv}}, if :math:\mathrm{b}[l-1] contains the parameter estimate for the :math:i\ th level of :math:R_{{jb}} when :math:S_{{\textit{k}b}} = s_{\textit{k}}, for :math:\textit{k} = 1,2,\ldots,N_{S_b} and :math:1\leq s_{\textit{k}}\leq L\left(S_{{jb}}\right), i.e., :math:s_k is a valid value for the :math:k\ th subject variable, then

.. math::
\begin{array}{l} \mathrm{estid}[0,l-1] = b\text{, }\quad \\ \mathrm{estid}[1,l-1] = j\text{, }\quad \\ \mathrm{estid}[2,l-1] = i\text{, }\quad \\ \mathrm{estid}[{3+k}-1,l-1] = s_k\text{, }k = 1,2,\ldots,N_{S_b}\text{;} \end{array}

if the parameter being estimated is for the intercept, then :math:\mathrm{estid}[1,l-1] = \mathrm{estid}[2,l-1] = 0;

if the :math:j\ th variable is continuous, or binary, that is :math:L\left(S_{{jb}}\right) = 1, then :math:\mathrm{estid}[2,l-1] = 0;

the remaining rows of the :math:l\ th column of :math:\mathrm{estid} are set to :math:{0}.

In some situations, certain combinations of variables are never observed.

In such circumstances all elements of the :math:l\ th row of :math:\mathrm{estid} are set to :math:-999.

**b** : float, ndarray, shape :math:\left(\textit{nff}+\textit{nrf}\times \textit{nlsv}\right)
The parameter estimates, with the first :math:{\textit{nrf}}\times {\textit{nlsv}} elements of :math:\mathrm{b} containing the parameter estimates for the random effects, :math:\nu, and the remaining :math:\textit{nff} elements containing the parameter estimates for the fixed effects, :math:\beta. The order of these estimates are described by the :math:\mathrm{estid} argument.

**se** : float, ndarray, shape :math:\left(\textit{nff}+\textit{nrf}\times \textit{nlsv}\right)
The standard errors of the parameter estimates given in :math:\mathrm{b}.

**czz** : float, ndarray, shape :math:\left(\textit{nrf}, :\right)
If :math:{\textit{nlsv}} = 1, then :math:\mathrm{czz} holds the lower triangular portion of the matrix :math:\left(1/\sigma^2\right)\left(Z^\mathrm{T} \hat{R}^{-1}Z + \hat{G}^{-1}\right), where :math:\hat{R} and :math:\hat{G} are the estimates of :math:R and :math:G respectively. If :math:{\textit{nlsv}} > 1, then :math:\mathrm{czz} holds this matrix in compressed form, with the first :math:\textit{nrf} columns holding the part of the matrix corresponding to the first level of the overall subject variable, the next :math:\textit{nrf} columns the part corresponding to the second level of the overall subject variable etc.

**cxx** : float, ndarray, shape :math:\left(\textit{nff}, :\right)
:math:\mathrm{cxx} holds the lower triangular portion of the matrix :math:\left(1/\sigma^2\right)X^\mathrm{T}\hat{V}^{-1}X, where :math:\hat{V} is the estimated value of :math:V.

**cxz** : float, ndarray, shape :math:\left(\textit{nff}, :\right)
If :math:{\textit{nlsv}} = 1, then :math:\mathrm{cxz} holds the matrix :math:\left(1/\sigma^2\right)\left(X^\mathrm{T}\hat{V}^{-1}Z\right)\hat{G}, where :math:\hat{V} and :math:\hat{G} are the estimates of :math:V and :math:G respectively. If :math:{\textit{nlsv}} > 1, then :math:\mathrm{cxz} holds this matrix in compressed form, with the first :math:\textit{nrf} columns holding the part of the matrix corresponding to the first level of the overall subject variable, the next :math:\textit{nrf} columns the part corresponding to the second level of the overall subject variable etc.

.. _g02je-py2-py-errors:

**Raises**
**NagValueError**
(errno :math:1)
On entry, :math:\textit{lvpr} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{lvpr} \geq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:2)
On entry, :math:\mathrm{vpr}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle and :math:\mathrm{nvpr} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{vpr}[i]\leq \mathrm{nvpr}.

(errno :math:3)
On entry, :math:\mathrm{nvpr} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:1\leq \mathrm{nvpr}\leq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:4)
On entry, :math:\mathrm{gamma}[\langle\mathit{\boldsymbol{value}}\rangle] = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\mathrm{gamma}[0] = {-1.0} or :math:\mathrm{gamma}[i-1] \geq 0.0.

(errno :math:9)
On entry, :math:\textit{lb} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{lb} \geq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:11)
On entry, :math:\textit{ldid} = \langle\mathit{\boldsymbol{value}}\rangle.

Constraint: :math:\textit{ldid} \geq \langle\mathit{\boldsymbol{value}}\rangle.

(errno :math:21)
On entry, :math:\mathrm{comm}\ ['iopts'] has not been initialized correctly.

(errno :math:32)
On entry, at least one value of i, for :math:\textit{i} = 1,2,\ldots,\mathrm{nvpr}, does not appear in :math:\mathrm{vpr}.

**Warns**
**NagAlgorithmicWarning**
(errno :math:101)
Optimal solution found, but requested accuracy not achieved.

(errno :math:102)
Too many major iterations.

(errno :math:103)
Current point cannot be improved upon.

(errno :math:104)
At least one negative estimate for :math:\mathrm{gamma} was obtained. All negative estimates have been set to zero.

.. _g02je-py2-py-notes:

**Notes**
mixeff_hier_ml fits a model of the form:

.. math::
y = X\beta +Z\nu +\epsilon

.. rst-class:: nag-rules-none nag-align-left

+-----+--------------------------------------------------------------------------------------------+
|where|:math:y is a vector of :math:n observations on the dependent variable,                  |
+-----+--------------------------------------------------------------------------------------------+
|     |:math:X is a known :math:n\times p design matrix for the fixed independent variables, |
+-----+--------------------------------------------------------------------------------------------+
|     |:math:\beta is a vector of length :math:p of unknown fixed effects,                   |
+-----+--------------------------------------------------------------------------------------------+
|     |:math:Z is a known :math:n\times q design matrix for the random independent variables,|
+-----+--------------------------------------------------------------------------------------------+
|     |:math:\nu is a vector of length :math:q of unknown random effects,                    |
+-----+--------------------------------------------------------------------------------------------+
|and  |:math:\epsilon is a vector of length :math:n of unknown random errors.                  |
+-----+--------------------------------------------------------------------------------------------+

Both :math:\nu and :math:\epsilon are assumed to have a Gaussian distribution with expectation zero and variance/covariance matrix defined by

.. math::
\mathrm{Var}\left[\begin{array}{c}\nu \\\epsilon \end{array}\right] = \left[\begin{array}{cc}G&0\\0&R\end{array}\right]

where :math:R = \sigma_R^2I, :math:I is the :math:n\times n identity matrix and :math:G is a diagonal matrix.
It is assumed that the random variables, :math:Z, can be subdivided into :math:g\leq q groups with each group being identically distributed with expectation zero and variance :math:\sigma_i^2.
The diagonal elements of matrix :math:G, therefore, take one of the values :math:\left\{\sigma_i^2:i = 1,2,\ldots,g\right\}, depending on which group the associated random variable belongs to.

The model, therefore, contains three sets of unknowns: the fixed effects :math:\beta, the random effects :math:\nu and a vector of :math:g+1 variance components :math:\gamma, where :math:\gamma = \left\{\sigma_1^2, \sigma_2^2, \ldots, \sigma_{{g-1}}^2, \sigma_g^2, \sigma_R^2\right\}.
Rather than working directly with :math:\gamma, mixeff_hier_ml uses an iterative process to estimate :math:\gamma^* = \left\{{\sigma_1^2/\sigma_R^2}, {\sigma_2^2/\sigma_R^2}, \ldots, {\sigma_{{g-1}}^2/\sigma_R^2}, {\sigma_g^2/\sigma_R^2}, 1\right\}.
Due to the iterative nature of the estimation a set of initial values, :math:\gamma_0, for :math:\gamma^* is required. mixeff_hier_ml allows these initial values either to be supplied by you or calculated from the data using the minimum variance quadratic unbiased estimators (MIVQUE0) suggested by Rao (1972).

mixeff_hier_ml fits the model by maximizing the log-likelihood function:

.. math::
-2l_R = \log\left(\left\lvert V\right\rvert \right)+n\log\left(r^\mathrm{T}V^{-1}r\right)+\log\left(2\pi /n\right)

where

.. math::

Once the final estimates for :math:\gamma^* have been obtained, the value of :math:\sigma_R^2 is given by

.. math::
\sigma_R^2 = \left(r^\mathrm{T}V^{-1}r\right)/\left(n-p\right)\text{.}

Case weights, :math:W_c, can be incorporated into the model by replacing :math:X^\mathrm{T}X and :math:Z^\mathrm{T}Z with :math:X^\mathrm{T}W_cX and :math:Z^\mathrm{T}W_cZ respectively, for a diagonal weight matrix :math:W_c.

The log-likelihood, :math:l_R, is calculated using the sweep algorithm detailed in Wolfinger et al. (1994).

.. _g02je-py2-py-references:

**References**
Goodnight, J H, 1979, A tutorial on the SWEEP operator, The American Statistician (33(3)), 149--158

Harville, D A, 1977, Maximum likelihood approaches to variance component estimation and to related problems, JASA (72), 320--340

Rao, C R, 1972, Estimation of variance and covariance components in a linear model, J. Am. Stat. Assoc. (67), 112--115

Stroup, W W, 1989, Predictable functions and prediction space in the mixed model procedure, Applications of Mixed Models in Agriculture and Related Disciplines (Southern Cooperative Series Bulletin No. 343), 39--48

Wolfinger, R, Tobias, R and Sall, J, 1994, Computing Gaussian likelihoods and their derivatives for general linear mixed models, SIAM Sci. Statist. Comput. (15), 1294--1310
"""
raise NotImplementedError

[docs]def lmm_init(hlmm, hddesc, hfixed, y, dat, hrndm=None, wt=None):
r"""
lmm_init preprocesses a dataset prior to fitting a linear mixed effects regression model via :meth:lmm_fit.

Note: this function uses optional algorithmic parameters, see also: :meth:blgm.optset <naginterfaces.library.blgm.optset>, :meth:blgm.optget <naginterfaces.library.blgm.optget>, :meth:lmm_fit.

.. _g02jf-py2-py-doc:

For full information please refer to the NAG Library document for g02jf

https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jff.html

.. _g02jf-py2-py-parameters:

**Parameters**
**hlmm** : Handle, modified in place
On entry: must be set to a null Handle or, alternatively, an existing G22 handle may be supplied in which case lmm_init will destroy the supplied G22 handle as if :meth:blgm.handle_free <naginterfaces.library.blgm.handle_free> had been called.

On exit: holds a G22 handle to the internal data structure containing a description of the model. You **must not** change the G22 handle other than through the functions in submodule correg or submodule :mod:~naginterfaces.library.blgm.

**hddesc** : Handle
A G22 handle to the internal data structure containing a description of the data matrix, :math:D, as returned in :math:\textit{hddesc} by :meth:blgm.lm_describe_data <naginterfaces.library.blgm.lm_describe_data>.

**hfixed** : Handle
A G22 handle to the internal data structure containing a description of the fixed part of the model :math:\mathcal{M}_f as returned in :math:\textit{hform} by :meth:blgm.lm_formula <naginterfaces.library.blgm.lm_formula>.

If :math:\mathrm{hfixed} is a null Handle then the model is assumed to not have a fixed part.

**y** : float, array-like, shape :math:\left(n\right)
:math:y, the vector of observations on the dependent variable.

**dat** : float, array-like, shape :math:\left(:, :\right)
The data matrix, :math:D. By default, :math:D_{{ij}}, the :math:\textit{i}\ th value for the :math:\textit{j}\ th variable, for :math:\textit{j} = 1,2,\ldots,m_d, for :math:\textit{i} = 1,2,\ldots,n, should be supplied in :math:\mathrm{dat}[i-1,j-1].

If the option 'Storage Order', described in :meth:blgm.lm_describe_data <naginterfaces.library.blgm.lm_describe_data>, is set to 'VAROBS', :math:D_{{ij}} should be supplied in :math:\mathrm{dat}[j-1,i-1].

If either :math:y_i, :math:w_i or :math:D_{{ij}}, for a variable :math:j used in the model, is NaN (Not A Number) then that value is treated as missing and the whole observation is excluded from the analysis.

**hrndm** : None or Handle, list, shape :math:\left(\textit{nrndm}\right), optional
A series of G22 handles to internal data structures containing a description of the random part of the model :math:\mathcal{M}_r as returned in :math:\textit{hform} by :meth:blgm.lm_formula <naginterfaces.library.blgm.lm_formula>.

**wt** : None or float, array-like, shape :math:\left(n\right), optional
Optionally, the diagonal elements of the weight matrix :math:W_c.

If :math:\mathrm{wt}[i-1] = 0.0, the :math:i\ th observation is not included in the model and the effective number of observations is the number of observations with nonzero weights.

If weights are not provided then :math:\mathrm{wt} must be set to **None**, and the effective number of observations is :math:n.

**Returns**
**fnlsv** : int
The number of levels for the overall subject variable in :math:\mathcal{M}_f. If there is no overall subject variable, :math:\mathrm{fnlsv} = 1.

**nff** : int
The number of fixed effects estimated in each of the :math:\mathrm{fnlsv} subject blocks. The number of columns, :math:p, in the design matrix :math:X is given by :math:p = \mathrm{nff}\times \mathrm{fnlsv}.

**rnlsv** : int
The number of levels for the overall subject variable in :math:\mathcal{M}_r. If there is no overall subject variable, :math:\mathrm{rnlsv} = 1.

**nrf** : int
The number of random effects estimated in each of the :math:\mathrm{rnlsv} subject blocks. The number of columns, :math:q, in the design matrix :math:Z is given by :math:q = \mathrm{nrf}\times \mathrm{rnlsv}.

**nvpr** : int
:math:g, the number of variance components being estimated (excluding the overall variance, :math:\sigma_R^2). This is defined by the number of terms in the random part of the model, :math:\mathcal{M}_r (see Algorithmic Details <https://www.nag.com/numeric/nl/nagdoc_28.7/flhtml/g02/g02jff.html#algdetails>__ for details).

**comm** : dict, communication object
Communication structure.

.. _g02jf-py2-py-other_params:

**Other Parameters**
**'Gamma Lower Bound'** : float
Default :math:\text{} = \sqrt{\text{machine precision}}/100

A lower bound for the elements of :math:\gamma^*, where :math:\gamma^* = \gamma /\sigma_R^2.

**'Gamma Upper Bound'** : float
Default :math:\text{} = 10^{20}

An upper bound for the elements of :math:\gamma^*, where :math:\gamma^* = \gamma /\sigma_R^2.

**'Initial Distance'** : float
Default :math:\text{} = 100000.0

The initial distance from the solution.

When :math:\text{‘Solver'} = \texttt{'E04LB'}, :meth:lmm_fit passes 'Initial Distance' to the solver as :math:\textit{stepmx}.

When :math:\text{‘Solver'} = \texttt{'E04UC'}, this option is ignored.

**'Initial Value Strategy'** : int
Default :math:\text{} = \text{special}

Controls how :meth:lmm_fit will choose the initial values for the variance components, :math:\gamma, if not supplied.

:math:\text{‘Initial Value Strategy'} = 0
The MIVQUE0 estimates of the variance components based on the likelihood specified by 'Likelihood' are used.

:math:\text{‘Initial Value Strategy'} = 1
The MIVQUE0 estimates based on the maximum likelihood are used, irrespective of the value of 'Likelihood'.

See Rao (1972) for a description of the minimum variance quadratic unbiased estimators (MIVQUE0).

By default, for small problems, :math:\text{‘Initial Value Strategy'} = 0 and for large problems :math:\text{‘Initial Value Strategy'} = 1.

**'Likelihood'** : str
Default :math:\text{} = \texttt{'REML'}

'Likelihood' defines whether :meth:lmm_fit will use the restricted maximum likelihood (REML) or the maximum likelihood (ML) when fitting the model.

**'Linear Minimization Accuracy'** : float
Default :math:\text{} = 0.9

The accuracy of the linear minimizations.

When :math:\text{‘Solver'} = \texttt{'E04LB'}, :meth:lmm_fit passes 'Linear Minimization Accuracy' to the solver as :math:\textit{eta}.

When :math:\text{‘Solver'} = \texttt{'E04UC'}, this option is ignored.

**'Line Search Tolerance'** : float
Default :math:\text{} = 0.9

The line search tolerance.

When :math:\text{‘Solver'} = \texttt{'E04LB'}, this option is ignored.

When :math:\text{‘Solver'} = \texttt{'E04UC'}, :meth:lmm_fit passes 'Line Search Tolerance' to the solver as 'Line Search Tolerance'.

**'List'** : valueless
Option 'List' enables printing of each option specification as it is supplied. 'NoList' suppresses this printing.

**'NoList'** : valueless
Default

Option 'List' enables printing of each option specification as it is supplied. 'NoList' suppresses this printing.

**'Major Iteration Limit'** : int
Default :math:\text{} = \text{special}

The number of major iterations.

When :math:\text{‘Solver'} = \texttt{'E04LB'}, :meth:lmm_fit passes 'Major Iteration Limit' to the solver as :math:\textit{maxcal}. In this case, the default value used is :math:1000.

When :math:\text{‘Solver'} = \texttt{'E04UC'}, :meth:lmm_fit passes 'Major Iteration Limit' to the solver as 'Major Iteration Limit'. In this case, the default value used is :math:\mathrm{max}\left(50,3\times g\right), where :math:g is the number of variance components being estimated (excluding the overall variance, :math:\sigma_R^2).

**'Major Print Level'** : int
Default :math:\text{} = \text{special}

The fr