CNTSELECT Procedure

Conway-Maxwell-Poisson Regression

The Conway-Maxwell-Poisson (CMP) distribution is a generalization of the Poisson distribution that enables you to model both underdispersed and overdispersed data. It was originally proposed by Conway and Maxwell (1962), but its implementation to model under- and overdispersed count data is attributed to Shmueli et al. (2005).

Recall that , given the vector of covariates , is independently Poisson-distributed as

upper P left-parenthesis upper Y Subscript i Baseline equals y Subscript i Baseline vertical-bar bold x Subscript i Baseline right-parenthesis equals StartFraction e Superscript minus lamda Super Subscript i Superscript Baseline lamda Subscript i Superscript y Super Subscript i Superscript Baseline Over y Subscript i Baseline factorial EndFraction comma y Subscript i Baseline equals 0 comma 1 comma 2 comma ellipsis

The CMP distribution is defined as

upper P left-parenthesis upper Y Subscript i Baseline equals y Subscript i Baseline vertical-bar bold x Subscript i Baseline comma bold g Subscript i Baseline right-parenthesis equals StartFraction 1 Over upper Z left-parenthesis lamda Subscript i Baseline comma nu Subscript i Baseline right-parenthesis EndFraction StartFraction lamda Subscript i Superscript y Super Subscript i Superscript Baseline Over left-parenthesis y Subscript i Baseline factorial right-parenthesis Superscript nu Super Subscript i Superscript Baseline EndFraction comma y Subscript i Baseline equals 0 comma 1 comma 2 comma ellipsis

where the normalization factor is

upper Z left-parenthesis lamda Subscript i Baseline comma nu Subscript i Baseline right-parenthesis equals sigma-summation Underscript n equals 0 Overscript normal infinity Endscripts StartFraction lamda Subscript i Superscript n Baseline Over left-parenthesis n factorial right-parenthesis Superscript nu Super Subscript i Superscript Baseline EndFraction

and

lamda Subscript i Baseline equals exp left-parenthesis bold x prime Subscript i Baseline bold-italic beta right-parenthesis

nu Subscript i Baseline equals exp left-parenthesis minus bold g prime Subscript i Baseline bold-italic delta right-parenthesis

The vector is a parameter vector. (The intercept is , and the coefficients for the k regressors are .) The vector is an parameter vector. (The intercept is represented by , and the coefficients for the m regressors are .) The covariates are represented by the vectors and .

One of the restrictive properties of the Poisson model is that the conditional mean and variance must be equal:

upper E left-parenthesis y Subscript i Baseline vertical-bar bold x Subscript i Baseline right-parenthesis equals upper V left-parenthesis y Subscript i Baseline vertical-bar bold x Subscript i Baseline right-parenthesis equals lamda Subscript i Baseline equals exp left-parenthesis bold x prime Subscript i Baseline bold-italic beta right-parenthesis

The CMP distribution overcomes this restriction by defining an additional parameter, , which governs the rate of decay of successive ratios of probabilities such that

upper P left-parenthesis upper Y Subscript i Baseline equals y Subscript i Baseline minus 1 right-parenthesis slash upper P left-parenthesis upper Y Subscript i Baseline equals y Subscript i Baseline right-parenthesis equals StartFraction left-parenthesis y Subscript i Baseline right-parenthesis Superscript nu Super Subscript i Superscript Baseline Over lamda Subscript i Baseline EndFraction

The introduction of the additional parameter, , allows for flexibility in modeling the tail behavior of the distribution. If , the ratio is equal to the rate of decay of the Poisson distribution. If , the rate of decay decreases, enabling you to model processes that have longer tails than the Poisson distribution’s (overdispersed data). If , the rate of decay increases in a nonlinear fashion, thus shortening the tail of the distribution (underdispersed data).

The CMP distribution has several special cases. If and , the CMP distribution results in the Bernoulli distribution. In this case, the data can take only the values 0 and 1; this represents an extreme underdispersion. If , the Poisson distribution is recovered along with its equidispersion property. When and , the normalization factor is convergent and forms a geometric series,

upper Z left-parenthesis lamda Subscript i Baseline comma 0 right-parenthesis equals StartFraction 1 Over 1 minus lamda Subscript i Baseline EndFraction

and the probability density function becomes

upper P left-parenthesis upper Y equals y Subscript i Baseline semicolon lamda Subscript i Baseline comma nu Subscript i Baseline equals 0 right-parenthesis equals left-parenthesis 1 minus lamda Subscript i Baseline right-parenthesis lamda Subscript i Superscript y Super Subscript i

The geometric distribution represents a case of severe overdispersion.

Mean, Variance, and Dispersion for the Conway-Maxwell-Poisson Model

The mean and the variance of the Conway-Maxwell-Poisson distribution are defined as

upper E left-parenthesis upper Y right-parenthesis equals StartFraction partial-differential ln upper Z Over partial-differential ln lamda EndFraction

upper V left-parenthesis upper Y right-parenthesis equals StartFraction partial-differential squared ln upper Z Over partial-differential squared ln lamda EndFraction

The CMP distribution does not have closed-form expressions for its moments in terms of its parameters and . However, the moments can be approximated. Shmueli et al. (2005) use asymptotic expressions for Z to derive and as

upper E left-parenthesis upper Y right-parenthesis almost-equals lamda Superscript 1 slash nu Baseline plus StartFraction 1 Over 2 nu EndFraction minus one-half

upper V left-parenthesis upper Y right-parenthesis almost-equals StartFraction 1 Over nu EndFraction lamda Superscript 1 slash nu

Shmueli et al. (2005) noted that these approximations might not be accurate for underdispersed data () or for values of .

In the CMP model, the summation of infinite series is evaluated using a logarithmic expansion. The mean and variance are calculated as follows for the Shmueli et al. (2005) model:

upper E left-parenthesis upper Y right-parenthesis equals StartFraction 1 Over upper Z left-parenthesis lamda comma nu right-parenthesis EndFraction sigma-summation Underscript j equals 0 Overscript normal infinity Endscripts StartFraction j lamda Superscript j Baseline Over left-parenthesis j factorial right-parenthesis Superscript nu Baseline EndFraction

upper V left-parenthesis upper Y right-parenthesis equals StartFraction 1 Over upper Z left-parenthesis lamda comma nu right-parenthesis EndFraction sigma-summation Underscript j equals 0 Overscript normal infinity Endscripts StartFraction j squared lamda Superscript j Baseline Over left-parenthesis j factorial right-parenthesis Superscript nu Baseline EndFraction minus upper E left-parenthesis upper Y right-parenthesis squared

The dispersion is defined as

upper D left-parenthesis upper Y right-parenthesis equals StartFraction upper V left-parenthesis upper Y right-parenthesis Over upper E left-parenthesis upper Y right-parenthesis EndFraction

Log-Likelihood Function for the Conway-Maxwell-Poisson Model

The log-likelihood function for the Conway-Maxwell-Poisson regression model can be written as

The gradients can be written as

where is the derivative of with respect to evaluated at and , and is the derivative with respect to evaluated at and .

Conway-Maxwell-Poisson Regression: Guikema and Coffelt (2008) Reparameterization

Guikema and Coffelt (2008) propose a reparameterization of the Shmueli et al. (2005) Conway-Maxwell-Poisson model to provide a measure of central tendency that can be interpreted in the context of the generalized linear model. By substituting , you can write the Guikema and Coffelt (2008) formulation as

upper P left-parenthesis upper Y equals y Subscript i Baseline semicolon mu Subscript i Baseline comma nu Subscript i Baseline right-parenthesis equals StartFraction 1 Over upper S left-parenthesis mu Subscript i Baseline comma nu Subscript i Baseline right-parenthesis EndFraction left-parenthesis StartFraction mu Subscript i Superscript y Super Subscript i Superscript Baseline Over y Subscript i Baseline factorial EndFraction right-parenthesis Superscript nu Super Subscript i

where the new normalization factor is defined as

upper S left-parenthesis mu comma nu right-parenthesis equals sigma-summation Underscript n equals 0 Overscript normal infinity Endscripts left-parenthesis StartFraction mu Superscript n Baseline Over n factorial EndFraction right-parenthesis Superscript nu

and

mu Subscript i Baseline equals exp left-parenthesis bold x prime Subscript i Baseline bold-italic beta right-parenthesis

In terms of their new formulations, the mean and variance of Y are given as

upper E left-parenthesis upper Y right-parenthesis equals StartFraction 1 Over nu EndFraction StartFraction partial-differential ln upper S Over partial-differential ln mu EndFraction

upper V left-parenthesis upper Y right-parenthesis equals StartFraction 1 Over nu squared EndFraction StartFraction partial-differential squared ln upper S Over partial-differential squared ln mu EndFraction

They can be approximated as

upper E left-parenthesis upper Y right-parenthesis almost-equals mu plus StartFraction 1 Over 2 nu EndFraction minus one-half

upper V left-parenthesis upper Y right-parenthesis almost-equals StartFraction mu Over nu EndFraction

When you specify the PRED= option in the OUTPUT statement, the CNTSELECT procedure calculates the predicted value of the response variable by using the approximation for if ; otherwise the exact formula is used. In PROC CNTSELECT, the mean and variance are calculated according to the following formulas for the Guikema and Coffelt (2008) model:

In terms of the new parameter , the log-likelihood function is specified as

script upper L equals sigma-summation Underscript i equals 1 Overscript upper N Endscripts left-parenthesis nu Subscript i Baseline y Subscript i Baseline ln left-parenthesis mu Subscript i Baseline right-parenthesis minus nu Subscript i Baseline ln left-parenthesis y Subscript i Baseline factorial right-parenthesis minus ln left-bracket upper S left-parenthesis mu Subscript i Baseline comma nu Subscript i Baseline right-parenthesis right-bracket right-parenthesis

and the gradients are calculated as

StartLayout 1st Row 1st Column script upper L Subscript beta 2nd Column equals 3rd Column sigma-summation Underscript i equals 1 Overscript upper N Endscripts left-parenthesis nu Subscript i Baseline y Subscript i Baseline minus StartFraction upper S Subscript mu Baseline left-parenthesis mu Subscript i Baseline comma nu Subscript i Baseline right-parenthesis Over upper S left-parenthesis mu Subscript i Baseline comma nu Subscript i Baseline right-parenthesis EndFraction mu Subscript i Baseline right-parenthesis bold x Subscript i 2nd Row 1st Column script upper L Subscript delta 2nd Column equals 3rd Column sigma-summation Underscript i equals 1 Overscript upper N Endscripts left-parenthesis ln left-parenthesis y Subscript i Baseline factorial right-parenthesis plus StartFraction upper S Subscript nu Baseline left-parenthesis mu Subscript i Baseline comma nu Subscript i Baseline right-parenthesis Over upper S left-parenthesis mu Subscript i Baseline comma nu Subscript i Baseline right-parenthesis EndFraction minus y Subscript i Baseline log mu Subscript i Baseline right-parenthesis nu Subscript i Baseline bold g Subscript i EndLayout

where is the derivative of with respect to evaluated at and , and is the derivative with respect to evaluated at and . By default, PROC CNTSELECT uses the Guikema and Coffelt (2008) specification. You can estimate the Shmueli et al. (2005) model by specifying the PARAMETER=LAMBDA option in the MODEL statement. If you specify DISP=CMPOISSON in the MODEL statement and you omit the DISPMODEL statement, the model is estimated according to the Lord, Guikema, and Geedipally (2008) specification, where represents a single parameter that does not depend on any covariates. The Lord, Guikema, and Geedipally (2008) specification makes the model comparable to the negative binomial model because it has only one parameter.

The dispersion is defined as

Using the Guikema and Coffelt (2008) specification results in the integral part of representing the mode, which is a reasonable approximation of the mean. The dispersion can be written as

When < 1, the variance can be shown to be greater than the mean and the dispersion can be shown to be greater than 1. This is a result of overdispersed data. When = 1 and the mean and variance are equal, the dispersion is equal to 1 (Poisson model). When > 1, the variance is smaller than the mean and the dispersion is less than 1. This is a result of underdispersed data.

All Conway-Maxwell-Poisson models in PROC CNTSELECT are parameterized in terms of dispersion, where

minus ln left-parenthesis nu right-parenthesis equals delta 0 plus sigma-summation Underscript n equals 1 Overscript q Endscripts delta Subscript n Baseline g Subscript n

Negative values of indicate that the data are approximately overdispersed, and positive values of indicate that the data are approximately underdispersed.

Last updated: July 09, 2026