The POWER Procedure

Example 90.11 Logistic Regression Using the CUSTOM Statement

(View the complete code for this example.)

The framework in Lyles, Lin, and Williamson (2007) provides an effective strategy for using the CUSTOM statement to compute power or sample size for generalized linear models. The process involves the following steps:

  1. Specify the SAS code that you would use to perform the data analysis.

  2. Create an exemplary data set that resembles the nature of the data you expect to obtain, covering as many reasonable values of the response variable as possible.

  3. Run the analysis on the exemplary data set and extract the appropriate test statistic.

  4. If you are planning for a likelihood ratio test, run the analysis again, but this time with a reduced model (lacking the predictors being tested). Again extract the appropriate test statistic.

  5. Run PROC POWER with the CUSTOM statement, specifying the noncentral chi-square distribution with primary noncentrality equal to the appropriate transformation of the test statistics.

Although the power computation is conditional (assuming fixed predictors), you can approximate unconditional power by using a WEIGHT statement in the data analysis and constructing the exemplary data to be representative of the conjectured distribution of the predictors.

Suppose you want to compute the power of the Wald chi-square and likelihood ratio tests for the interaction term in a logistic regression model that has two binary covariates, as in Lyles, Lin, and Williamson (2007, section 3.2). You plan to use a sample size and significance level = 0.05. You have conjectured values for the regression coefficients as shown in Table 90.42.

Table 90.42: Conjectured Regression Coefficient Values

Source

Coefficient

Value

Intercept

0

X1

–1.5

X2

0.5

X1*X2

2


You expect the covariates to be distributed according to a multinomial distribution with probabilities as shown in Table 90.43.

Table 90.43: Conjectured Covariate Distribution

X1

X2

Probability

0

0

0.2

0

1

0.25

1

0

0.25

1

1

0.3


The strategy for the power computation is summarized as follows:

  1. Create an exemplary data set to represent both the regression coefficients and the covariate distribution.

  2. Use the LOGISTIC procedure to compute the primary noncentrality values and degrees of freedom you need for the power calculation.

  3. Fetch the primary noncentrality values and degrees of freedom from PROC LOGISTIC output and pass them into the CUSTOM statement in PROC POWER along with the other power analysis parameters.

The following steps describe this strategy in detail:

  1. First create a data set that has one row for each unique design profile and compute the response probability that is associated with each profile according to your conjectured regression coefficients values. The following DATA step computes these probabilities as values of a variable PY:

    * Compute P(Yj|Xi) values from conjectured betas;
    data Exemplary;
       _B0 = 0; _B1 = -1.5; _B2 = 0.5; _B3 = 2;
       do X1 = 0 to 1;
          do X2 = 0 to 1;
             _pi = logistic(_B0 + _B1*X1 + _B2*X2 + _B3*X1*X2);
             do Y = 0 to 1;
                PY = _pi**Y * (1-_pi)**(1-Y);
                output;
             end;
          end;
       end;
       keep X1 X2 Y PY;
    run;
    
    proc print data=Exemplary;
    run;
    

    The output data set is shown in Output 90.11.1.

    Output 90.11.1: Response Probabilities Produced from Regression Coefficients

    ObsX1X2YPY
    10000.50000
    20010.50000
    30100.37754
    40110.62246
    51000.81757
    61010.18243
    71100.26894
    81110.73106


    Next expand this data set into an exemplary data set that represents your assumed covariate distribution. The following DATA step replicates each design profile the appropriate number of times according to the multinomial probabilities in Table 90.43.

    * Expand according to relative allocations of design profiles;
    data Exemplary;
       set Exemplary;
       if      (X1 = 0 and X2 = 0) then _alloc=4;
       else if (X1 = 0 and X2 = 1) then _alloc=5;
       else if (X1 = 1 and X2 = 0) then _alloc=5;
       else                             _alloc=6;
       do _i = 1 to _alloc;
          output;
       end;
       keep X1 X2 Y PY;
    run;
    
    proc print data=Exemplary;
    run;
    

    Output 90.11.2 shows the resulting exemplary data set.

    Output 90.11.2: Exemplary Data Set

    ObsX1X2YPY
    10000.50000
    20000.50000
    30000.50000
    40000.50000
    50010.50000
    60010.50000
    70010.50000
    80010.50000
    90100.37754
    100100.37754
    110100.37754
    120100.37754
    130100.37754
    140110.62246
    150110.62246
    160110.62246
    170110.62246
    180110.62246
    191000.81757
    201000.81757
    211000.81757
    221000.81757
    231000.81757
    241010.18243
    251010.18243
    261010.18243
    271010.18243
    281010.18243
    291100.26894
    301100.26894
    311100.26894
    321100.26894
    331100.26894
    341100.26894
    351110.73106
    361110.73106
    371110.73106
    381110.73106
    391110.73106
    401110.73106


  2. Next use the exemplary data as the input data set in PROC LOGISTIC, assigning PY as the WEIGHT variable. First fit the full model and save both the parameter estimates and fit statistics output tables as data sets. (You will later use a parameter estimate for the Wald chi-square power analysis and a fit statistic for the likelihood ratio test.)

    * Run full model;
    proc logistic data=Exemplary;
       ods output parameterestimates=fitFull fitstatistics=statsFull;
       weight PY;
       model Y = X1 X2 X1*X2;
    run;
    

    Output 90.11.3 and Output 90.11.4 show these tables from the PROC LOGISTIC output.

    Output 90.11.3: Parameter Estimates for Full Model

    The LOGISTIC Procedure

    Analysis of Maximum Likelihood Estimates
    ParameterDFEstimateStandard
    Error
    Wald
    Chi-Square
    Pr > ChiSq
    Intercept12.6E-171.00000.00001.0000
    X111.49991.53000.96110.3269
    X21-0.50001.36050.13510.7132
    X1*X21-1.99992.00990.99010.3197


    Output 90.11.4: Fit Statistics for Full Model

    Model Fit Statistics
    CriterionIntercept OnlyIntercept and
    Covariates
    AIC29.69231.911
    SC31.38138.666
    -2 Log L27.69223.911


    The numbers you need for the power analysis for the Wald chi-square test of X1*X2 are the degrees of freedom (1) and the Wald chi-square test statistic (0.9901). The number you need for the likelihood ratio test is the –2LogL value (23.911). The following DATA step saves these values to data sets:

    * Fetch Wald chi-square statistic for test of X1*X2;
    data fitFull;
       set fitFull;
       where Variable = "X1*X2";
       keep DF WaldChiSq;
    run;
    
    * Fetch -2LogL for full model;
    data statsFull;
       set statsFull;
       where Criterion = "-2 Log L";
       Neg2LogLFull = InterceptAndCovariates;
       keep Neg2LogLFull;
    run;
    

    Next run the reduced model (dropping the interaction term, the one you want to test) and save the fit statistics table.

    * Run reduced model;
    proc logistic data=Exemplary;
       ods output fitstatistics=statsRed;
       weight PY;
       model Y = X1 X2;
    run;
    

    Output 90.11.5 shows the fit statistics table from the PROC LOGISTIC output.

    Output 90.11.5: Fit Statistics for Reduced Model

    The LOGISTIC Procedure

    Model Fit Statistics
    CriterionIntercept OnlyIntercept and
    Covariates
    AIC29.69230.939
    SC31.38136.005
    -2 Log L27.69224.939


    The number you need for the power analysis for the likelihood ratio test is the –2LogL value (24.939). The following DATA step saves these values to data sets:

    * Fetch -2LogL for reduced model;
    data statsRed;
       set statsRed;
       where Criterion = "-2 Log L";
       Neg2LogLRed = InterceptAndCovariates;
       keep Neg2LogLRed;
    run;
    

    Now you are ready to compute the primary noncentrality values. You need to divide both the Wald chi-square and –2LogL difference by the "effective sample size" of the exemplary data set—that is, the number of rows for each response value (20). The following statements compute the primary noncentrality and store all three numbers that you need for PROC POWER in macro variables:

    * Compute primary noncentralities for Wald and LR tests;
    data PrimNC;
       merge fitFull statsFull statsRed;
       WaldPrimNC = WaldChiSq / 20;
       LRPrimNC = -(Neg2LogLFull-Neg2LogLRed) / 20;
       keep DF WaldPrimNC LRPrimNC;
       call symput ("DF", DF);
       call symput ("WaldPrimNC", WaldPrimNC);
       call symput ("LRPrimNC", LRPrimNC);
    run;
    
    proc print data=PrimNC;
    run;
    

    The primary noncentrality values thus computed are shown in Output 90.11.6.

    Output 90.11.6: Primary Noncentrality Values Derived from PROC LOGISTIC

    ObsDFWaldPrimNCLRPrimNC
    110.0495060.051396


  3. Perform the power analysis by using the CUSTOM statement in PROC POWER as follows:

    proc power;
       custom
          dist   = chisquare
          primnc = &WaldPrimNC &LRPrimNC
          testdf = &DF
          ntotal = 100
          power  = .;
    run;
    

The computed powers for the two tests, shown in Output 90.11.7, match the results in Lyles, Lin, and Williamson (2007, section 3.2).

Output 90.11.7: Computed Power Values

The POWER Procedure
Custom Test

Fixed Scenario Elements
DistributionChi-square
MethodDefault
Test Degrees of Freedom1
Total Sample Size100
Alpha0.05
Primary Noncentrality Multiplier1
Critical Value Multiplier1

Computed Power
IndexPrimary NCPower
10.04950.605
20.05140.621


The following statements validate the PROC POWER results by using the QUANTILE and SDF functions in the DATA step:

data Power;
   set PrimNC;
   Crit = quantile("chisq", 1-0.05, DF);
   WaldPower = sdf("chisq", Crit, DF, 100 * WaldPrimNC);
   LRPower = sdf("chisq", Crit, DF, 100 * LRPrimNC);
   keep WaldPower LRPower;
run;
proc print data=Power;
run;

The computed powers for the two tests, shown in Output 90.11.8, match the results in Lyles, Lin, and Williamson (2007, section 3.2).

Output 90.11.8: Validation of Power Values

ObsWaldPowerLRPower
10.604520.62063