HMM Procedure

Matrix Expression

The INITIAL and PRIOR statements operate on matrices. That is, you can specify the parameter matrices or constant matrices through the INITIAL and PRIOR statements’ built-in operators and functions. You can add elements of the matrices A and B by using the expression A+B, perform matrix multiplication by using the expression A*B, and perform elementwise multiplication by using the expression A#B. You can get the diagonal elements of the matrix A by using the function DIAG(A), and you can get the identity matrix by using the function I(n).

Each equation is written as a matrix expression that is composed of constants, operators, and functions.

Constants

Constants are either scalar constants (such as –1.2, 0.3, and so on) or matrix constants enclosed in braces (such as the matrix {1 2, 3 4} or the row vector {-0.2 5.3 12}).

The matrix constant cannot be the first item in the INITIAL statement unless you enclose it in parentheses. For example, you cannot specify the following statement:

initial {-0.1 -0.2, -0.3 -0.4} = MU;

You can specify the preceding example by enclosing the first matrix constant in parentheses, as follows:

initial ({-0.1 -0.2, -0.3 -0.4}) = MU;

Operators

Operators define the operations on operands. Table 2 lists all built-in operators that the INITIAL statement supports.

Table 2: Operators

Operator	Name	Description
+	Addition	Adds corresponding matrix elements
=	Comparison, equal	Compares matrix elements
<	Comparison, less than	Compares matrix elements
<=	Comparison, not greater than	Compares matrix elements
>	Comparison, greater than	Compares matrix elements
>=	Comparison, not less than	Compares matrix elements
\|\|	Concatenation, horizontal	Concatenates matrices horizontally
//	Concatenation, vertical	Concatenates matrices vertically
@	Direct product	Takes the direct product of two matrices
	Distribution	Links the distribution function to the matrix elements
:	Index creation	Creates an index vector
#	Multiplication, elementwise	Performs elementwise multiplication
	Multiplication, matrix	Performs matrix multiplication
	Sign reverse	Reverses the signs of elements
	Subscripts	Selects submatrices
	Subtraction	Subtracts corresponding matrix elements
`	Transpose	Transposes a matrix

For more information about each operator, see the section Details of Operators.

Table 3 shows the precedence of matrix operators.

Table 3: Precedence of Operators

Priority Group	Operators
I (highest)	(subscripts)	(transpose)
II	(sign reverse)
III	*	#	@
IV	(subtraction)	+
V	\|\|	//	:
VI (lowest)	=	<	<=	>	>=

Each equation can be a compound expression that involves several matrix operators and operands. The rules for evaluating compound expressions are as follows:

Evaluation follows the order of operator precedence as described in Table 3. Group I has the highest priority; that is, Group I operators are evaluated first. Group II operators are evaluated after Group I operators, and so on. For example, returns 7.
If neighboring operators in an expression have equal precedence, the expression is evaluated from left to right, except for the Group I operators. For example, returns .
All expressions in parentheses are evaluated first, following the two preceding rules. For example, returns 9.

Functions

Functions are divided into two types. One type of function refers to parameters to be estimated; examples are MU(K,I) and SIGMA(K,I,J). The other type does not; examples are I(n) and DIAG(A).

Functions that refer to parameters are listed in Table 4. The arguments of functions can be matrices. The simplest case, scalar arguments, is discussed first. For convenience, the scalar indices i and j refer to the position of the element in the coefficient matrix, the scalar l refers to the lag value, the scalar c refers to the component index, and the scalar k refers to the state value.

Table 4: Functions That Refer to Parameters

Function	Description
`AR(k,l,i,j)`	Autoregressive parameter of the lag l of the jth dependent variable, , to the ith dependent variable at time t, , for state k in regime-switching autoregression
`CONST(k,i)`	Intercept parameter of the ith time series, , for state k in regime-switching regression or regime-switching autoregression, or the alias of MU for Gaussian HMM or GM HMM
`COV(k,i,j)`	Covariance of innovations parameter between the ith and jth error processes for state k in regime-switching regression or regime-switching autoregression, or the alias of SIGMA for Gaussian HMM or GM HMM
`CPM(i,j)`	The (i, j)th element in the category probability matrix in the finite HMM
`LAMBDA(K)`	Mean parameter of the time series, , for state k in the Poisson HMM
`ISPV(i)`	The ith element in the initial state probability vector
`LTREND(k,i)`	Linear trend parameter of the ith time series, , for state k in regime-switching regression or regime-switching autoregression when the TREND= option is specified in the MODEL statement
`MCP(i,j)`	The (i, j)th element in the mixture component probabilities in the GM HMM
`MU(k,i)`	Mean parameter of the ith time series, , for state k in the Gaussian HMM, or alias for CONST for regime-switching regression or regime-switching autoregression
`MU(k,c,i)`	Mean parameter of the ith time series, , for the cth component at state k in the GM HMM
`QTREND(k,i)`	Quadratic trend parameter of the ith time series, , for state k in regime-switching regression or regime-switching autoregression when TREND=QUAD is specified in the MODEL statement
`SD(k,i,j)`	Alias of SDUMMY
`SDUMMY(k,i,j)`	The jth seasonal dummy of the ith time series at time t, , for state k in regime-switching regression or regime-switching autoregression, where nseason, where nseason is the value of the NSEASON= option in the MODEL statement
`SIGMA(k,i,j)`	Covariance parameter between the ith and jth time series, and , for state k in the Gaussian HMM, or the alias of COV for regime-switching regression or regime-switching autoregression
`SIGMA(k,c,i,j)`	Covariance parameter between the ith and jth time series, and , for cth component at state k in the GM HMM
`TPM(i,j)`	The (i, j)th element in the transition probability matrix
`XL(k,l,i,j)`	Exogenous parameter of the lag l of the jth exogenous variable, , to the ith dependent variable at time t, , for state k in regime-switching regression or regime-switching autoregression

The functions that refer to parameters, as shown in Table 4, are valid only when the parameters are included in the model. For example, the ISPV function is valid only when you specify the ESTISPV option in the MODEL statement; otherwise, the HMM procedure issues errors and then stops.

The functions that refer to parameters, as shown in Table 4, accept vector and matrix arguments and return the matrix that is constructed by the corresponding parameters. According to the number of arguments, the following list shows what matrix a function returns when the arguments are vectors:

A function, FUNC, that has one vector argument I, where , returns a vector , where FUNC(), . ISPV is a type of FUNC.
A function, FUNC, that has two vector arguments I and J, where and , returns a matrix,

where FUNC(), . CONST, LTREND, MU, QTREND, and TPM are types of FUNC.
A function, FUNC, that has N vector arguments and , where and N is odd and where , returns a block matrix,

where FUNC(), , and is constructed in the same way as FUNC().

For example, a function, FUNC, that has three vector arguments K, I, and J, where , , and , returns a matrix,

where FUNC(), , and is the integer such that and . COV and MU for GM HMM and SIGMA for Gaussian HMM are types of FUNC.
A function, FUNC, that has N vector arguments and , where and N is even and where , returns a block matrix,

where FUNC(), , , and is constructed in the same way as FUNC().

For example, a function, FUNC, that has four vector arguments K, L, I, and J, where , , , and , returns a matrix,

where FUNC(), , where is the integer such that and . AR and SIGMA for GM HMM and XL are types of FUNC.

The functions that refer to parameters can accept empty arguments or omit any number of last arguments. The empty or omitted arguments are replaced by all possible values for those arguments. For example, PROC HMM is used to fit a bivariate three-state Gaussian HMM as follows:

   model y1 y2 / type=gaussian nstate=3;

In order to initialize the second row of TPM to be , you can use the following statement:

initial tpm(2,{1 2 3}) =  {0.1 0.7 0.2};

Taking advantage of empty arguments, you can specify the preceding example as follows:

initial tpm(2,) =  {0.1 0.7 0.2};

To get all coefficients of the covariance matrix for state 2, you can use SIGMA(2, {1 2}, {1 2}), or SIGMA(2, , ), or SIGMA(2). To get all coefficients of exogenous variables on dependent variables, you can use SIGMA({1 2 3}, {1 2}, {1 2}), or SIGMA( , , ), or SIGMA(), or even just SIGMA.

Another type of function does not refer to parameters but generates useful matrices. Table 5 lists all built-in functions.

Table 5: Functions That Do Not Refer to Parameters

Function	Description
`DIAG(A)`	Creates a diagonal matrix from a vector or extracts the diagonal elements of a matrix
`I(n)`	Creates an identity matrix
`J(m,n,elem)`	Creates an matrix whose elements are all equal to `elem`
`SHAPE(A,m,n)`	Creates an matrix whose elements are from matrix `A`

For more information about each function in Table 5, see the section Details of Functions.

Details of Operators

This section describes all operators that are available in the HMM procedure. Each subsection describes the operator and shows how it is used.

Addition Operator: +

The addition operator (+) computes a new matrix whose elements are the sums of corresponding elements of the two operands. Following are examples of how the addition operator is used:

matrix1 + matrix2

adds the element in the ith row and jth column of the first matrix to the element in the ith row and jth column of the second matrix, for , where matrix1 and matrix2 are both matrices.

For example, {1 2 3, 4 5 6} + {7 8 9, 10 11 12} results in {8 10 12, 14 16 18}.

matrix + scalar

adds the scalar value to each element of the matrix. For example, you can obtain {2 3 4, 5 6 7} from {1 2 3, 4 5 6} + 1.

matrix + vector

adds the vector value to each row or column of the matrix:

If you add an column vector, each row of the vector is added to each row of the matrix. For example, you can obtain {2 3 4, 5 6 7} from {1 2 3, 4 5 6} + {1, 1}.
If you add a row vector, each column of the vector is added to each column of the matrix. For example, you can obtain {2 3 4, 5 6 7} from {1 2 3, 4 5 6} + {1 1 1}.

Comparison Operators: =, <, <=, >, >=

The comparison operators (=, <, <=, >, >=) compare two matrices element by element and return a list of equivalent restrictions on only scalar constants and parameters.

matrix1 = matrix2

matrix1 < matrix2

matrix1 <= matrix2

matrix1 > matrix2

matrix1 >= matrix2

For example, the INITIAL statement with matrix expressions

initial SIGMA(1,{1,2},{1,2}) = SIGMA(2,{3,4},{3,4});

is transformed into the following equivalent INITIAL statement with scalar parameters:

initial SIGMA(1,1,1) = SIGMA(2,3,3),
        SIGMA(1,1,2) = SIGMA(2,3,4),
        SIGMA(1,2,1) = SIGMA(2,4,3),
        SIGMA(1,2,2) = SIGMA(2,4,4);

You can also use the comparison operators to conveniently compare all elements of a matrix to a scalar. If either argument is a scalar, then the HMM procedure performs an elementwise comparison between each element of the matrix and the scalar.

You can also compare an matrix to a row or column vector:

If the comparison is to an column vector, the HMM procedure compares each row of the vector to each row of the matrix.
If the comparison is to a row vector, the HMM procedure compares each column of the vector to each column of the matrix.

For example, the following statements are equivalent:

initial SIGMA(1,4:5,1:3) = 0.2;

initial SIGMA(1,4:5,1:3) = {0.2, 0.2};

initial SIGMA(1,4:5,1:3) = {0.2 0.2 0.2};

Concatenation Operator, Horizontal: ||

The horizontal concatenation operator (||) produces a new matrix by horizontally joining matrix1 and matrix2.

matrix1 matrix2

The matrices must have the same number of rows, which is also the number of rows in the new matrix. The number of columns in the new matrix is the number of columns in matrix1 plus the number of columns in matrix2.

For example, {1 1 1, 7 7 7} || {0 0 0, 8 8 8} returns {1 1 1 0 0 0, 7 7 7 8 8 8}.

Concatenation Operator, Vertical: //

The vertical concatenation operator (//) produces a new matrix by vertically joining matrix1 and matrix2.

matrix1 // matrix2

The matrices must have the same number of columns, which is also the number of columns in the new matrix. The number of rows in the new matrix is the number of rows in matrix1 plus the number of rows in matrix2.

For example, {1 1 1} // {0 0 0, 8 8 8} returns {1 1 1, 0 0 0, 8 8 8}.

Direct Product Operator: @

The direct product operator (@) computes a new matrix that is the direct product (also called the Kronecker product) of matrix1 and matrix2.

matrix1 @ matrix2

For matrices and , the direct product is denoted by . The number of rows in the new matrix equals the product of the number of rows in matrix1 and the number of rows in matrix2; the number of columns in the new matrix equals the product of the number of columns in matrix1 and the number of columns in matrix2.

Specifically, if is an matrix and is an matrix, then the Kronecker product is the following block matrix:

bold upper A circled-times bold upper B equals Start 3 By 3 Matrix 1st Row 1st Column upper A 11 upper B 2nd Column midline-horizontal-ellipsis 3rd Column upper A Subscript 1 p Baseline upper B 2nd Row 1st Column vertical-ellipsis 2nd Column down-right-diagonal-ellipsis 3rd Column vertical-ellipsis 3rd Row 1st Column upper A Subscript n Baseline 1 Baseline upper B 2nd Column midline-horizontal-ellipsis 3rd Column upper A Subscript n p Baseline upper B EndMatrix

For example, {1 2, 3 4} @ {0 2} returns {0 2 0 4, 0 6 0 8}, and {0 2} @ {1 2, 3 4} returns {0 0 2 4, 0 0 6 8}. Note that the direct product of two matrices is not commutative.

Distribution Operator:

The distribution operator links the matrix to its distribution function.

matrix distribution-function

This operator is used in the PRIOR statement. For more information, see the section PRIOR Statement.

For example, ISPV DIR({1,1,1}) means that the initial state probability vector (ISPV) follows a Dirichlet distribution with an argument of a vector {1,1,1}.

Index Creation Operator: :

The index creation operator (:) creates a column vector whose first element is value1, second element is value1+1, and so on, until the last element, which is less than or equal to value2.

value1 : value2

For example, 3 : 6 returns {3 4 5 6}.

If value1 is greater than value2, a reverse-order index is created. For example, 6 : 3 returns {6 5 4 3}.

Neither value1 nor value2 is required to be an integer.

Multiplication Operator, Elementwise: #

The elementwise multiplication operator (#) computes a new matrix whose elements are the products of the operands. Following are examples of how the elementwise multiplication operator is used:

matrix1 # matrix2

computes a new matrix whose elements are the products of the corresponding elements of matrix1 and matrix2.

For example, {1 2, 3 4} # {4 8, 0 5} returns {4 16, 0 20}.

matrix # scalar

multiplies each element in matrix by the scalar value.

matrix # vector

multiplies each row or column of the matrix by a corresponding element of the vector:

If you multiply by an column vector, each row of the matrix is multiplied by the corresponding row of the vector.
If you multiply by a row vector, each column of the matrix is multiplied by the corresponding column of the vector.

For example, a matrix can be multiplied on either side by a , , , or scalar.

The product of elementwise multiplication is also known as the Schur or Hadamard product. Elementwise multiplication (which uses the # operator) should not be confused with matrix multiplication (which uses the * operator).

Multiplication Operator, Matrix: *

The matrix multiplication operator (*) computes a new matrix by performing matrix multiplication.

matrix1 * matrix2

The first matrix must have the same number of columns as the second matrix has rows. The new matrix has the same number of rows as the first matrix and the same number of columns as the second matrix. That is, if is an matrix and is a matrix, then the product is an matrix. The (i, j) element of the product is the sum .

For example, {1 2, 3 4} * {1, 2} returns {5, 11}.

Sign Reversal Operator: –

The sign reversal operator () computes a new matrix whose elements are formed by reversing the sign of each element in matrix. The sign reversal operator is also called the unary minus operator.

- matrix

For example, -{-1 7 6, 2 0 -8} returns {1 -7 -6, -2 0 8}.

Subscripts: [ ]

Subscripts are used with matrices to select submatrices, where rows, columns, and elements are expressions that evaluate to scalars or vectors. If these expressions are numeric, they must contain valid subscript values of rows and columns, or the indices, in the argument matrix.

matrix[rows, columns]

matrix[elements]

For example, {1 2 3, 4 5 6, 7 8 9}[2,3] returns 6; {1 2 3, 4 5 6, 7 8 9}[2,1:3] returns {4 5 6}; and {1 2 3, 4 5 6, 7 8 9}[,3] returns {3, 6, 9}. Because the HMM procedure stores matrices in row-major order, {11 22 33, 44 55 66, 77 88 99}[{3 5 9}] returns {33, 55, 99}.

Subtraction Operator: –

The subtraction operator () computes a new matrix whose elements are formed by subtracting the corresponding elements of the second operand from the first operand.

matrix1 - matrix2

computes a new matrix whose elements are formed by subtracting the corresponding elements of matrix2 from those of matrix1.

For example, {1 2 3, 4 5 6} - {1 1 1, 1 1 1} returns {0 1 2, 3 4 5}.

matrix - scalar

subtracts the scalar value from each element of the matrix.

For example, {1 2 3, 4 5 6} - 1 returns {0 1 2, 3 4 5}.

matrix - vector

subtracts the vector from each element of the matrix:

If you subtract an column vector, each row of the vector is subtracted from each row of the matrix. For example, {1 2 3, 4 5 6} - {1, 1} returns {0 1 2, 3 4 5}.
If you subtract a row vector, each column of the vector is subtracted from each column of the matrix. For example, {1 2 3, 4 5 6} - {1 1 1} returns {0 1 2, 3 4 5}.

Transpose Operator: `

The transpose operator, denoted by the backquote (`), exchanges the rows and columns of matrix, producing the transpose of matrix.

matrix`

If v is the value in the ith row and jth column of matrix, then the transpose of matrix contains v in the jth row and ith column. If matrix contains n rows and p columns, the transpose has p rows and n columns.

For example, {1 2, 3 4, 5 6}` returns {1 3 5, 2 4 6}.

Details of Functions

DIAG Function

The DIAG function creates a diagonal matrix from a vector or extracts the diagonal elements of a matrix.

DIAG(matrix)

The matrix argument can be either a square matrix or a vector:

If matrix is a square matrix, the DIAG function creates a vector from the diagonal elements of the matrix.

For example, DIAG({1 2 3, 4 5 6, 7 8 9}) returns {1, 5, 9}.
If matrix is a vector, the DIAG function creates a matrix whose diagonal elements are the values in the vector. All off-diagonal elements are zeros.

For example, DIAG({1 5 9}) or DIAG({1, 5, 9}) returns {1 0 0, 0 5 0, 0 0 9}.

You can call the DIAG function repeatedly. For example, DIAG(DIAG({1 2 3, 4 5 6, 7 8 9})) returns {1 0 0, 0 5 0, 0 0 9}.

I Function

The I function creates an identity matrix that contains dim rows and columns.

I(dim)

The diagonal elements of an identity matrix are ones; all other elements are zeros. The value of dim must be an integer greater than or equal to 1. Noninteger operands are rounded to the nearest integer.

For example, I(3) returns {1 0 0, 0 1 0, 0 0 1}.

J Function

The J function creates a matrix that contains nrow rows and ncol columns, in which all elements are equal to value.

J(nrow, ncol, value)

The arguments nrow and ncol are both integers; value can be any expression that returns a linear combination of scalar constants and parameters.

For example, J(2, 3, 1) returns {1 1 1, 1 1 1}.

SHAPE Function

The SHAPE function creates a new matrix from data in matrix.

SHAPE(matrix, nrow, ncol)

The values nrow and ncol specify the number of rows and columns, respectively, in the new matrix. The SHAPE function produces the new matrix by traversing the argument matrix in row-major order until it reaches the specified number of elements. If necessary, the SHAPE function reuses elements.

For example, SHAPE({1 2 3, 4 5 6}, 3, 2) returns {1 2, 3 4, 5 6}; SHAPE({1 2 3, 4 5 6}, 5, 2) returns {1 2, 3 4, 5 6, 1 2, 3 4}; and SHAPE({1 2 3, 4 5 6}, 1, 4) returns {1 2 3 4}.

Last updated: July 09, 2026