The HMM Procedure

OPTIMIZE Statement

OPTIMIZE <options>;

The OPTIMIZE statement optimizes the parameter estimation.

You can specify the following options:

ALGORITHM=type <(suboptions)>

specifies the optimization algorithm to use. You can specify the following types:

ACTIVESET <(suboptions)>

uses the active set algorithm. You can specify one or more of the following suboptions in parentheses:

FEASTOL=number

defines the feasible tolerance. The optimizer exits if the constraint violation is less than number and the scaled optimality conditions are less than the value of the OPTTOL= suboption. By default, FEASTOL=.

MSRANGE=number

defines the range from which each variable can take values during the sampling process in multistart mode, where number must be a positive number. This option is ignored if MULTISTART=0. This option affects only the sampling process that determines starting points for the local solver. It does not affect the bounds of the original nonlinear optimization problem. By default, MSRANGE=2.

MULTISTART=0 | 1

specifies whether to enable multistart mode. You can specify the following values:

0: disables multistart mode.
1: solves the problem from multiple starting points, possibly finding a better local optimum as a result. The computing cost in multistart mode might be huge.

By default, MULTISTART=0.

OBJLIMIT=number

specifies an upper limit on the magnitude of the objective value, where number must be greater than or equal to . If OBJLIMIT=M, the algorithm terminates when the objective value becomes less than for a minimization problem or when the objective value exceeds M for a maximization problem. The termination of the algorithm implies that either the problem is unbounded or the algorithm diverges. If optimization were allowed to continue, numerical difficulty might be encountered. By default, OBJLIMIT=. If number is less than , it is reset to the default value of .

OPTTOL=

specifies the measure by which the current iterate is determined to be an acceptable approximation of a local optimum, where is a positive real number. The optimizer determines that the current iterate is a local optimum when the norm of the scaled vector of the optimality conditions is less than and the true constraint violation is less than the value of the FEASTOL= option. By default, OPTTOL=.

SOLTYPE=0 | 1

specifies the type of solution that the optimizer should return. You can specify the following values:

0: requests that the optimizer return a locally optimal solution, provided that it locates one.
1: requests that the optimizer return the best feasible solution that was found, provided that its objective value is better than that of the locally optimal solution that was found.

By default, SOLTYPE=1.

EM <(suboption)>

uses the expectation-maximization (EM) algorithm. This algorithm is supported only for the Gaussian HMM and the maximum likelihood method. You can specify the following suboption in parentheses:

MINITER=number: specifies the minimum number of iterations in the optimization process, where number must be a nonnegative integer. By default, MINITER=0.
SINGLE: executes the EM algorithm in a single thread on a single node. By default, the EM algorithm runs in parallel on multiple threads and nodes if you divide the data into sections and specify the SECTION= option in the ID statement.

INTERIORPOINT <(suboptions)>

uses the interior point algorithm. You can specify the same suboptions in parentheses that you can specify for ALGORITHM=ACTIVESET.

IPDIRECT <(suboptions)>

uses the interior point direct algorithm, in which the use of direct factorizations and other enhancements can reduce both the number of iterations and the CPU time for many problem types. You can specify the same suboptions in parentheses that you can specify for ALGORITHM=INTERIORPOINT.

SGD <(suboptions)>

uses the stochastic gradient descent algorithm. You can specify one or more of the following suboptions in parentheses:

ADAPTIVEDECAY=number

specifies the adaptive rate for the gradient descent algorithm when it is used, where the algorithm becomes ADAGRAD if ADAPTIVEDECAY=0 and becomes ADADELTA otherwise. By default, ADAPTIVEDECAY=0.95.

ANNEALINGRATE=number

specifies the annealing parameter, , where number must be a nonnegative double. Annealing is a way to automatically reduce the learning rate as SGD progresses, causing smaller steps as SGD approaches a solution. Effectively, this option replaces the learning rate parameter, , with

eta prime equals StartFraction eta Over 1 plus beta t EndFraction

where t is the number of iterations that SGD has performed. By default, ANNEALINGRATE=.

CHAIN=(options)

specifies the options for the sequence data (which constitute a chain). You can specify the following options within parentheses:

LENGTH=number: specifies the length of a chain in a minibatch in optimization process, where number must be a positive integer. By default, LENGTH=1000.
BUFFER=number: specifies the buffer length on the left side of a chain in a minibatch in optimization process, where number must be a nonnegative integer. By default, BUFFER=20.

LEARNINGRATE=number

specifies the learning rate parameter, , for SGD, where number must be a positive double. New iterates for SGD are found by using

theta Subscript k plus 1 Baseline equals theta Subscript k Baseline minus StartFraction eta Over double-vertical-bar upper I Subscript k Baseline double-vertical-bar EndFraction sigma-summation Underscript left-parenthesis x Subscript i Baseline comma y Subscript i Baseline right-parenthesis element-of upper I Subscript k Baseline Endscripts nabla upper L left-parenthesis theta Subscript k Baseline semicolon x Subscript i Baseline comma y Subscript i Baseline right-parenthesis

where is the current weight vector, is the new weight vector, is the minibatch that is used during iteration k, and is the negative log likelihood that is associated with the ith chain of observation. If you see a huge objective value with SGD (especially with a small data table), it is likely that too large a value was specified for this option. By default, LEARNINGRATE=10.

MINIBATCHSIZE=number

specifies the size of the minibatches used in SGD, where number must be a positive integer. By default, MINIBATCHSIZE=1.

MOMENTUM=number

specifies the value for momentum parameter , where number must be greater than or equal to 0 and less than 1. New iterates for SGD that uses the momentum method are found by using

StartLayout 1st Row 1st Column Blank 2nd Column theta Subscript k plus 1 Baseline equals theta Subscript k Baseline plus nabla theta Subscript k Baseline 2nd Row 1st Column Blank 2nd Column nabla theta Subscript k Baseline equals alpha nabla theta Subscript k Baseline minus StartFraction eta Over double-vertical-bar upper I Subscript k Baseline double-vertical-bar EndFraction sigma-summation Underscript left-parenthesis x Subscript i Baseline comma y Subscript i Baseline right-parenthesis element-of upper I Subscript k Baseline Endscripts nabla upper L left-parenthesis theta Subscript k Baseline semicolon x Subscript i Baseline comma y Subscript i Baseline right-parenthesis EndLayout

NONADUPDATE

suppresses the adaptive gradient descent algorithm.

USELOCKING

requests that computational threads share a common weight vector and that they update the weight vector without race conditions. By default, computational threads update a single weight vector simultaneously. This causes intentional race conditions and nondeterministic behavior, but it increases performance significantly.

By default, ALGORITHM=ACTIVESET.

COVOFPARM=YES | NO

specifies whether to calculate the covariance of parameters. You can specify the following values:

YES: calculates the covariance of parameters.
NO: suppresses the calculation of the covariance of parameters.

By default, COVOFPARM=YES. This option supports only the EM algorithm.

MAXITER=number

specifies the maximum number of iterations in the optimization process, where number must be a nonnegative integer. By default, MAXITER=128.

MAXTIME=t

specifies an upper limit of t seconds for the optimization process, including problem generation time and solution time, where t can be any positive number. The default value of t is the positive number that has the largest absolute value that your operating environment can represent. If you omit this option, the optimizer does not stop on the basis of the amount of time elapsed. This option is not supported for the EM algorithm.

PRINTITERFREQ=number

specifies how frequently to output the optimization process history, where number must be a nonnegative integer. If PRINTITERFREQ=0, no iterations are output. If number=n and n is positive, the optimization process history is output every n iterations. By default, PRINTITERFREQ=0.

PRINTLEVEL=0 | 1 | 2 | 3

specifies the print level. You can specify the following values:

0: suppresses output of the initial parameter values, the initial objective value, the algorithm information, the final parameter values, and the final objective value.
1: outputs the initial parameter values and the initial objective value.
2: outputs the initial parameter values, the initial objective value, and the algorithm information.
3: outputs the initial parameter values, the initial objective value, the algorithm information, the final parameter values, and the final objective value.

By default, PRINTLEVEL=0.

Last updated: May 01, 2023