HPCOUNTREG Procedure

(View the complete code for this example.)

This example shows the use of the HPCOUNTREG procedure with an emphasis on large data set processing.

The following DATA step generates one million replicates from the zero-inflated Poisson (ZIP) model. The model contains seven variables and three variables that correspond to the zero-inflated process.

    data simulate;
       call streaminit(12345);
       array vars x1-x7;
       array zero_vars z1-z3;

       array parms{7}  (.3 .4 .2 .4 -.3 -.5 -.3);
       array zero_parms{3} (-.6 .3 .2);

       intercept=2;
       z_intercept=-1;
       theta=0.5;

       do i=1 to 1000000;
          sum_xb=0;
          sum_gz=0;
          do j=1 to 7;
             vars[j]=rand('NORMAL',0,1);
             sum_xb=sum_xb+parms[j]*vars[j];
          end;
          mu=exp(intercept+sum_xb);
          y_p=rand('POISSON', mu);

          do j=1 to 3;
             zero_vars[j]=rand('NORMAL',0,1);
             sum_gz = sum_gz+zero_parms[j]*zero_vars[j];
          end;
          z_gamma = z_intercept+sum_gz;
          pzero = cdf('LOGISTIC',z_gamma);
          cut=rand('UNIFORM');
          if cut<pzero then y_p=0;
          output;
       end;
    keep y_p x1-x7 z1-z3;
    run;

The following statements estimate a zero-inflated Poisson model:

 proc hpcountreg data=simulate dist=zip;
    performance nthreads=2 details;
    model y_p=x1-x7;
    zeromodel y_p ~ z1-z3;
 run;

The model is executed in single-machine mode with two threads. These settings are used to obtain a hypothetical environment that might resemble running the HPCOUNTREG procedure on a desktop workstation with a dual-core CPU. Output 18.1.1 shows the "Performance Information" table for this hypothetical scenario.

Output 18.1.2 shows the results for the zero-inflated Poisson model. The "Model Fit Summary" table shows detailed information about the model and indicates that all one million observations were used to fit the model. All parameter estimates in the "Parameter Estimates" table are highly significant and correspond to their theoretical values set during the data generating process.

In the following statements, the PERFORMANCE statement is modified to use a single-machine mode with twenty threads:

 proc hpcountreg data=simulate dist=zip;
    performance nthreads=20 details;
    model y_p=x1-x7;
    zeromodel y_p ~ z1-z3;
 run;

Because the two models being estimated are identical, it is reasonable to expect that Output 18.1.2 and Output 18.1.3 would show the same results. However, you can see a significant difference in performance between the two models.

In certain circumstances, you might observe slight numerical differences in the results, depending on the number of nodes and threads involved. This happens because the order in which partial results are accumulated can make a difference in the final result, owing to the limits of numerical precision and the propagation of error in numerical computations.

As this example suggests, increasing the number of threads improves performance considerably.