naginterfaces.library.opt.handle_​solve_​nldf

naginterfaces.library.opt.handle_solve_nldf(handle, lsqfun, lsqgrd, x, nres, confun=None, congrd=None, monit=None, data=None, io_manager=None)[source]

handle_solve_nldf is a solver from the NAG optimization modelling suite for general nonlinear data-fitting problems with constraints. Various loss and regularization functions are supported.

Note: this function uses optional algorithmic parameters, see also: handle_opt_set(), handle_opt_get().

For full information please refer to the NAG Library document for e04gn

https://www.nag.com/numeric/nl/nagdoc_28.5/flhtml/e04/e04gnf.html

Parameters
handleHandle

The handle to the problem. It needs to be initialized (e.g., by handle_init()) and to hold a problem formulation compatible with handle_solve_nldf. It must not be changed between calls to the NAG optimization modelling suite.

lsqfuncallable (rx, inform) = lsqfun(x, nres, inform, data=None)

must evaluate the value of the nonlinear residuals, , at a specified point .

Parameters
xfloat, ndarray, shape

, the vector of variable values at which the residuals, , are to be evaluated.

nresint

, the current number of residuals in the model.

informint

A non-negative value.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

Returns
rxfloat, array-like, shape

The value of the residual vector, , evaluated at .

informint

May be used to indicate that some residuals could not be computed at the requested point. This can be done by setting to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with = 25.

lsqgrdcallable inform = lsqgrd(x, nres, rdx, inform, data=None)

evaluates the residual gradients, , at a specified point .

Parameters
xfloat, ndarray, shape

, the vector of variable values at which the residual gradients, , are to be evaluated.

nresint

, the current number of residuals in the model.

rdxfloat, ndarray, shape , to be modified in place

On entry: the elements should only be assigned and not referenced.

On exit: the vector containing the nonzero residual gradients evaluated at ,

where

The elements must be stored in the same order as the defined sparsity pattern provided in the call to handle_set_nlnls().

informint

A non-negative value.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

Returns
informint

May be used to indicate that the residual gradients could not be computed at the requested point. This can be done by setting to a negative value. The solver will attempt a rescue procedure and request an alternative point. If the rescue procedure fails, the solver will exit with = 25.

xfloat, array-like, shape

, the initial estimates of the variables, .

nresint

, the current number of residuals in the model.

confunNone or callable (gx, inform) = confun(x, ncnln, inform, data=None), optional

Note: if this argument is None then a NAG-supplied facility will be used.

must calculate the values of the -element vector of nonlinear constraint functions at a specified value of the -element vector of variables.

If there are no nonlinear constraints then will never be called by handle_solve_nldf and it may be None.

Parameters
xfloat, ndarray, shape

The vector of variable values at which the constraint functions are to be evaluated.

ncnlnint

, the number of nonlinear constraints, as specified in an earlier call to handle_set_nlnconstr().

informint

A non-negative value.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

Returns
gxfloat, array-like, shape

The values of the nonlinear constraint functions at .

informint

Must be set to a value describing the action to be taken by the solver on return from . Specifically, if the value is negative, then the value of will be discarded and the solver will either attempt to find a different trial point or terminate immediately with = 25; otherwise, the solver will proceed normally.

congrdNone or callable inform = congrd(x, gdx, inform, data=None), optional

Note: if this argument is None then a NAG-supplied facility will be used.

must calculate the nonzero values of the sparse Jacobian of the nonlinear constraint functions, , at a specified value of the -element vector of variables.

If there are no nonlinear constraints, will never be called by handle_solve_nldf and may be None.

Parameters
xfloat, ndarray, shape

The vector of variable values at which the Jacobian of the constraint functions is to be evaluated.

gdxfloat, ndarray, shape , to be modified in place

On entry: the elements should only be assigned and not referenced.

On exit: the nonzero values of the Jacobian of the nonlinear constraints, in the order specified by and in an earlier call to handle_set_nlnconstr(). will be the gradient , where and .

informint

A non-negative value.

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

Returns
informint

Must be set to a value describing the action to be taken by the solver on return from . Specifically, if the value is negative the solution of the current problem will terminate immediately with = 25; otherwise, computations will continue.

monitNone or callable monit(x, rinfo, stats, data=None), optional

Note: if this argument is None then a NAG-supplied facility will be used.

is provided to enable monitoring of the progress of the optimization and, if necessary, to halt the optimization process.

If no monitoring is required, may be None.

is called at the end of every th step where is controlled by the option ‘NLDF Monitor Frequency’ (if the value is , is not called).

Parameters
xfloat, ndarray, shape

The current best point.

rinfofloat, ndarray, shape

Best objective value computed and various indicators (the values are as described in the main argument ).

statsfloat, ndarray, shape

Solver statistics at monitoring steps or at the end of the current iteration (the values are as described in the main argument ).

dataarbitrary, optional, modifiable in place

User-communication data for callback functions.

dataarbitrary, optional

User-communication data for callback functions.

io_managerFileObjManager, optional

Manager for I/O in this routine.

Returns
xfloat, ndarray, shape

The final values of the variables, .

rxfloat, ndarray, shape

The values of the residuals at the final point given in .

rinfofloat, ndarray, shape

Objective value and various indicators at monitoring steps or at the end of the final iteration. The measures are:

Objective function value, in [equation].

Loss function value, in [equation].

Regularization term value, in [equation].

Solution optimality measure.

Reserved for future use.

statsfloat, ndarray, shape

Solver statistics at monitoring steps or at the end of the final iteration:

Number of iterations performed.

Total time in seconds spent in the solver. It includes time spent in user-supplied subroutines.

Reserved for future use.

Other Parameters
‘Defaults’valueless

This special keyword may be used to reset all options to their default values. Any value given with this keyword will be ignored.

‘NLDF Iteration Limit’int

Default

The maximum number of iterations to be performed by handle_solve_nldf. If this limit is reached, then the solver will terminate with = 22.

Constraint: .

‘NLDF Monitor Frequency’int

Default

If , the user-supplied function will be called at the end of every th step for monitoring purposes.

Constraint: .

‘NLDF Stop Tolerance’float

Default

This argument sets the value of which specifies the tolerance for the optimality measure.

When both loss function and regularization are differentiable, and with only simple bound constraints, the optimality measures are defined by [equation] and [equation] and ‘NLDF Stop Tolerance’ is passed to the solver handle_solve_bounds_foas() as ‘FOAS Stop Tolerance’.

When any of the loss function or regularization is non-differentiable, or there presents linear, quadratic or general nonlinear constraint, the optimality measure is defined by (5) and ‘NLDF Stop Tolerance’ is passed to the solver handle_solve_ipopt() as ‘Stop Tolerance 1’.

Constraint: .

‘NLDF Loss Function Type’str

Default ’L2’

This argument sets the loss function type used in the objective function.

Constraint: , , , , , , or .

‘Reg Term Type’str

Default ’OFF’

This argument sets the regularization function type used in the objective function.

Note: if there is no residual in the model, regularization will be turned off.

Constraint: , or .

‘NLDF Huber Function Width’float

Default

Sets the parameter defined in the Huber loss function [equation].

Constraint: .

‘NLDF Cauchy Function Sharpness’float

Default

Sets the parameter defined in the Cauchy loss function [equation].

Constraint: .

‘NLDF SmoothL1 Function Width’float

Default

Sets the parameter defined in the SmoothL1 loss function [equation].

Constraint: .

‘NLDF Quantile Parameter’float

Default

Sets the parameter defined in the Quantile loss function [equation].

Constraint: .

‘Reg Coefficient’float

Default

Sets the regularization coefficient in the definition of the objective function [equation]. Note: if set to , regularization will be turned off.

Constraint: .

‘Infinite Bound Size’float

Default

This defines the ‘infinite’ bound in the definition of the problem constraints. Any upper bound greater than or equal to will be regarded as (and similarly any lower bound less than or equal to will be regarded as ). Note that a modification of this option does not influence constraints which have already been defined; only the constraints formulated after the change will be affected.

Constraint: .

‘Monitoring File’int

Default

If , the unit number for the secondary (monitoring) output. If , no secondary output is provided. The information output to this unit is controlled by ‘Monitoring Level’.

Constraint: .

‘Monitoring Level’int

Default

This argument sets the amount of information detail that will be printed by the solver to the secondary output. The meaning of the levels is the same as for ‘Print Level’.

Constraint: .

‘Print File’int

Default

If , the unit number for the primary output of the solver. If , the primary output is completely turned off independently of other settings. The default value is the advisory message unit number at the time of the initialization of the options, e.g., at the initialization of the handle. The information output to this unit is controlled by ‘Print Level’.

Constraint: .

‘Print Level’int

Default

This argument defines how detailed information should be printed by the solver to the primary and secondary output.

Output

No output from the solver.

The Header and Summary.

, , ,

Additionally, the Iteration log.

Constraint: .

‘Print Options’str

Default

If , a listing of options will be printed to the primary output and is always printed to the secondary output.

Constraint: or .

‘Print Solution’str

Default

If or , the final values of the primal variables are printed on the primary and secondary outputs.

Constraint: , or .

‘Stats Time’str

Default

This argument turns on timing. This might be helpful for a choice of different solving approaches. It is possible to choose between CPU and wall clock time. Choice ‘YES’ is equivalent to ‘WALL CLOCK’.

Constraint: , , or .

‘Time Limit’float

Default

A limit to the number of seconds that the solver can use to solve one problem. If at the end of an iteration this limit is exceeded, the solver will terminate with = 23.

Constraint: .

Raises
NagValueError
(errno )

handle_solve_nldf is not available in this implementation.

(errno )

has not been initialized.

(errno )

does not belong to the NAG optimization modelling suite, has not been initialized properly or is corrupted.

(errno )

has not been initialized properly or is corrupted.

(errno )

This solver does not support the model defined in the handle.

(errno )

The problem is already being solved.

(errno )

On entry, , expected .

Constraint: must match the current number of variables of the model in the .

(errno )

On entry, , expected .

Constraint: must match the current number of residuals defined in the .

(errno )

Please provide a proper function.

(errno )

Please provide a proper function.

(errno )

The current starting point is unusable.

(errno )

The solver terminated with not enough degrees of freedom.

(errno )

The solver detected an infeasible problem.

Warns
NagAlgorithmicWarning
(errno )

Problem was solved to an acceptable level; full accuracy was not achieved.

NagAlgorithmicMajorWarning
(errno )

Maximum number of iterations reached.

(errno )

The solver terminated after the maximum time allowed was exhausted.

(errno )

The solver was terminated because no further progress could be achieved.

(errno )

Invalid number detected in user function.

(errno )

The solver terminated after failure during line search.

(errno )

The solver terminated after an error in the step computation.

NagCallbackTerminateWarning
(errno )

User requested termination during a monitoring step.

Notes

handle_solve_nldf solves a data-fitting problem of the form

where

is the number of decision variables,

is the number of the nonlinear constraints and , and are -dimensional vectors,

is the number of quadratic constraints,

is the number of the linear constraints and is a matrix, and are -dimensional vectors,

there are box constraints and and are -dimensional vectors.

Here,

is an -dimensional vector representing the model parameters,

is the loss function,

is the regularization function,

is the regularization coefficient,

is the number of residuals and is the th residual, which is defined as

where is the predicted value of the th data point, given . For the th data point, and are the observed values of the independent and dependant variables respectively.

The available loss and regularization function types are summarized in Table [label omitted], where is the function parameter and denotes an indicator function taking the value if the logical expression is true and otherwise. Loss function and regularization types can be specified by options ‘NLDF Loss Function Type’ and ‘Reg Term Type’, respectively. For example, set and to use -norm loss function with -norm (Ridge) regularization. See Algorithmic Details for more details on the loss functions.

Loss function

‘NLDF Loss Function Type’

-norm

‘L2’

-norm

‘L1’

-norm

‘LINF’

Huber (see [equation])

‘HUBER’

Cauchy (see [equation])

‘CAUCHY’

Atan

‘ATAN’

SmoothL1 (see [equation])

‘SMOOTHL1’

Quantile (see [equation])

‘QUANTILE’

Regularization

‘Reg Term Type’

Lasso (-norm)

‘L1’

Ridge (-norm)

‘L2’

handle_solve_nldf serves as a solver for problems stored as a . The points to an internal data structure which defines the problem and serves as a means of communication for functions in the NAG optimization modelling suite. After the has been initialized (e.g., handle_init() has been called), handle_set_nlnls() can be used to add a model and define its residual sparsity structure. handle_set_qconstr() and handle_set_qconstr_fac() may be used to set or modify quadratic constraints. Linear constraints , , are handled by handle_set_linconstr(). Variable box bounds and can be specified with handle_set_simplebounds(), and handle_set_nlnconstr() can set or modify nonlinear constraints. Once the problem is fully described, the may be passed to the solver handle_solve_nldf. When the is no longer needed, handle_free() should be called to destroy it and deallocate the memory held within. See the E04 Introduction for more details about the NAG optimization modelling suite.

Nonlinear Programming (NLP) solvers handle_solve_bounds_foas() and handle_solve_ipopt() are used as solver engines by handle_solve_nldf, which defines the selected loss function and regularization, then transforms the problem into standard form that the NLP solvers allow. For best performance, when the objective function is differentiable and without any constraint other than simple bound constraints, handle_solve_bounds_foas() is used. For non-differentiable objective functions or cases where constraints other than simple variable bounds are present, handle_solve_ipopt() is used. See Algorithmic Details and handle_solve_ipopt() for more details on algorithmic details.

The algorithm behaviour can be modified by various options (see Other Parameters) which can be set by handle_opt_set() and handle_opt_set_file() anytime between the initialization of the by e.g., handle_init() and a call to the solver. Once the solver has finished, options may be modified for the next solve. The solver may be called repeatedly with various starting points and/or options. Option getter handle_opt_get() can be called to retrieve the current value of any option.