Bayesian Additive Regression Trees Action Set

This section contains Lua code that creates the same trainData table and performs the same analysis as shown in the CASL version of this example. For more information about coding in Lua, see Getting Started with SAS Viya for Lua and SAS Viya: System Programming Guide.

Because this example uses simulated data that are produced from DATA step code, you can submit the runCode action on your Lua client to create the simulated data in your CAS session. The following DATA step creates the trainData data table, which consists of 10,000 observations on a continuous response variable (y) and 40 continuous variables (x1–x40):

s:dataStep_runCode{single='no', code=[[
data trainData;
   drop i j pi w1-w40 u f1 f2 f3 f4 nTotalObs nObsPerThread nExtras;
   array x{40};
   array w{40};
   call streaminit(6524);
   nTotalObs=10000;
   pi=constant('pi');
   nObsPerThread = int(nTotalObs/_nthreads_);
   nExtras       = mod(nTotalObs,_nthreads_);
   if _threadid_ <= nExtras then nObsPerThread = nObsPerThread + 1;
   do i=1 to nObsPerThread;
      u = rand('uniform');
      do j=1 to dim(x);
         w{j} = rand('Uniform');
         x{j} = (w{j} + u)/2;
      end;
      f1 = sin(pi * x1 * x2 );
      f2 = (x3-0.5)**2;
      f3 = x4;
      f4 = x5;
      fb = 10*f1 +20*f2+10*f3+5*f4;
      y = fb +  rand('Normal');
      output;
   end;
run;
]] }

By default, the CAS server executes the preceding DATA step in parallel in every thread on every node in your session. For example, if your DATA step creates 10 observations and executes in parallel in a session that has three worker nodes, each of which uses 16 threads, then the resulting CAS table has 160 observations on each worker node, yielding a CAS table that has a total of 480 observations. The preceding DATA step uses this technique to create a table that has a specified number of observations in parallel on your CAS server by dividing the desired number of observations (nTotalObs) by the total number of threads (_nthreads_). If the number of observations cannot be exactly divided by the number of threads, the code adds one observation to each of a subset of the threads until all observations are accounted for.

The following statements use the bartGauss action in the bart action set to fit a Bayesian additive regression trees (BART) model to these data. The table parameter names the input data table to be analyzed. You can use one of two mutually exclusive modes to specify the variables in the model by using the bartGauss action: in one mode, you use the target, inputs, and nominals parameters; in the other mode, you use the class and model parameters. In this example, the first mode is used. The target parameter names the target (response) variable, and the inputs parameter lists the explanatory variables x1–x40. In this example, because the input data are not large, the trainInMem and obsLeafMapInMem parameters are used to store elements of the model in memory when the model is trained. The store parameter stores the fitted model in the modelFit data table.

s:loadactionset{actionset="bart"}

s:bart_bartGauss{
   table='trainData',
   target='y',
   inputs={'x1','x2','x3','x4','x5','x6','x7','x8','x9','x10',
           'x11','x12','x13','x14','x15','x16','x17','x18','x19','x20',
           'x21','x22','x23','x24','x25','x26','x27','x28','x29','x30',
           'x31','x32','x33','x34','x35','x36','x37','x38','x39','x40'
   },
   seed=9181,
   obsLeafMapInMem=true,
   trainInMem=true,
   store={name='modelFit', replace=true}
   }

The output from this analysis is displayed by default. It is presented in Output 4.1.8 through Output 4.1.14.

The "Model Information" table in Output 4.1.8 summarizes important information about the model that you fit. Information includes the input data source, response (target) variable, bin method, missing rule, simulation size, thinning rate, and random number seed.

Output 4.1.9 displays the "Number of Observations" table. All 10,000 observations in the data table are used in the analysis.

Output 4.1.10 displays the "Prior Information" table. The table summarizes the parameters that define the BART prior for the tree structure and the variance of the response variable. The BART prior parameters typically do not require hyperparameter tuning, because the default BART model tends to perform well in terms of model fit, and this performance does not change significantly if there are small changes in the prior parameters.

Output 4.1.11 displays the "Sample Information" table. This table summarizes basic information about the posterior samples of the sum-of-trees ensemble, such as the number of samples saved, the proportion of tree-modifying operations accepted, and the size of the trees.

When you run the bartGauss action on a cluster of machines, you can use a distributed mode that divides the MCMC samples and runs multiple chains in parallel. By default, each worker node in the cluster has a full copy of the training data and runs a separate chain. You can specify the number of separate chains by using the distributeChains parameter. In this example, the bartGauss action was run on a cluster of machines; Output 4.1.12 displays the "Chain Information" table. This table indicates the number of parallel chains that the procedure used and the number of posterior samples that were saved for prediction from each chain. Note that the fit of a BART model depends on the number of chains that are run.

Output 4.1.13 displays the fit statistics for the fitted model. In this case, there is only one: the average square error (ASE).

Finally, the bartGauss action displays the table shown in Output 4.1.14, which shows the amount of time (in seconds) that it took to perform the various tasks in the analysis.

This section contains Python code that creates the same trainData table and performs the same analysis as shown in the CASL version of this example. For more information about coding in Python, see Getting Started with SAS Viya for Python and SAS Viya: System Programming Guide.

Because this example uses simulated data that are produced from DATA step code, you can submit the runCode action on your Python client to create the simulated data in your CAS session. The following DATA step creates the trainData data table, which consists of 10,000 observations on a continuous response variable (y) and 40 continuous variables (x1–x40):

s.datastep.runcode(
    code='''
   data trainData;
      drop i j pi w1-w40 u f1 f2 f3 f4 nTotalObs nObsPerThread nExtras;
      array x{40};
      array w{40};
      call streaminit(6524);
      nTotalObs=10000;
      pi=constant('pi');
      nObsPerThread = int(nTotalObs/_nthreads_);
      nExtras       = mod(nTotalObs,_nthreads_);
      if _threadid_ <= nExtras then nObsPerThread = nObsPerThread + 1;
      do i=1 to nObsPerThread;
         u = rand('uniform');
         do j=1 to dim(x);
            w{j} = rand('Uniform');
            x{j} = (w{j} + u)/2;
         end;
         f1 = sin(pi * x1 * x2 );
         f2 = (x3-0.5)**2;
         f3 = x4;
         f4 = x5;
         fb = 10*f1 +20*f2+10*f3+5*f4;
         y = fb +  rand('Normal');
         output;
      end;
   run;
   ''',
    single='no')

By default, the CAS server executes the preceding DATA step in parallel in every thread on every node in your session. For example, if your DATA step creates 10 observations and executes in parallel in a session that has three worker nodes, each of which uses 16 threads, then the resulting CAS table has 160 observations on each worker node, yielding a CAS table that has a total of 480 observations. The preceding DATA step uses this technique to create a table that has a specified number of observations in parallel on your CAS server by dividing the desired number of observations (nTotalObs) by the total number of threads (_nthreads_). If the number of observations cannot be exactly divided by the number of threads, the code adds one observation to each of a subset of the threads until all observations are accounted for.

The following statements use the bartGauss action in the bart action set to fit a Bayesian additive regression trees (BART) model to these data. The table parameter names the input data table to be analyzed. You can use one of two mutually exclusive modes to specify the variables in the model by using the bartGauss action: in one mode, you use the target, inputs, and nominals parameters; in the other mode, you use the class and model parameters. In this example, the first mode is used. The target parameter names the target (response) variable, and the inputs parameter lists the explanatory variables x1–x40. In this example, because the input data are not large, the trainInMem and obsLeafMapInMem parameters are used to store elements of the model in memory when the model is trained. The store parameter stores the fitted model in the modelFit data table.

s.loadactionset(actionset='bart')

s.bart.bartgauss(
    inputs=['x1', 'x2', 'x3', 'x4', 'x5', 'x6', 'x7', 'x8', 'x9', 'x10', 'x11',
            'x12', 'x13', 'x14', 'x15', 'x16', 'x17', 'x18', 'x19', 'x20',
            'x21', 'x22', 'x23', 'x24', 'x25', 'x26', 'x27', 'x28', 'x29',
            'x30', 'x31', 'x32', 'x33', 'x34', 'x35', 'x36', 'x37', 'x38',
            'x39', 'x40'],
    obsLeafMapInMem='true',
    seed='9181',
    store=dict(name='modelFit', replace='true'),
    table='trainData',
    target='y',
    trainInMem='true')

The output from this analysis is displayed by default. It is presented in Output 4.1.15 through Output 4.1.21.

The "Model Information" table in Output 4.1.15 summarizes important information about the model that you fit. Information includes the input data source, response (target) variable, bin method, missing rule, simulation size, thinning rate, and random number seed.

Output 4.1.16 displays the "Number of Observations" table. All 10,000 observations in the data table are used in the analysis.

Output 4.1.17 displays the "Prior Information" table. The table summarizes the parameters that define the BART prior for the tree structure and the variance of the response variable. The BART prior parameters typically do not require hyperparameter tuning, because the default BART model tends to perform well in terms of model fit, and this performance does not change significantly if there are small changes in the prior parameters.

Output 4.1.18 displays the "Sample Information" table. This table summarizes basic information about the posterior samples of the sum-of-trees ensemble, such as the number of samples saved, the proportion of tree-modifying operations accepted, and the size of the trees.

When you run the bartGauss action on a cluster of machines, you can use a distributed mode that divides the MCMC samples and runs multiple chains in parallel. By default, each worker node in the cluster has a full copy of the training data and runs a separate chain. You can specify the number of separate chains by using the distributeChains parameter. In this example, the bartGauss action was run on a cluster of machines; Output 4.1.19 displays the "Chain Information" table. This table indicates the number of parallel chains that the procedure used and the number of posterior samples that were saved for prediction from each chain. Note that the fit of a BART model depends on the number of chains that are run.

Output 4.1.20 displays the fit statistics for the fitted model. In this case, there is only one: the average square error (ASE).

Finally, the bartGauss action displays the table shown in Output 4.1.21, which shows the amount of time (in seconds) that it took to perform the various tasks in the analysis.

This section contains R code that creates the same trainData table and performs the same analysis as shown in the CASL version of this example. For more information about coding in R, see Getting Started with SAS Viya for R and SAS Viya: System Programming Guide.

Because this example uses simulated data that are produced from DATA step code, you can submit the runCode action on your R client to create the simulated data in your CAS session. The following DATA step creates the trainData data table, which consists of 10,000 observations on a continuous response variable (y) and 40 continuous variables (x1–x40):

cas.dataStep.runCode(s,
    code='
   data trainData;
      drop i j pi w1-w40 u f1 f2 f3 f4 nTotalObs nObsPerThread nExtras;
      array x{40};
      array w{40};
      call streaminit(6524);
      nTotalObs=10000;
      pi=constant(\'pi\');
      nObsPerThread = int(nTotalObs/_nthreads_);
      nExtras       = mod(nTotalObs,_nthreads_);
      if _threadid_ <= nExtras then nObsPerThread = nObsPerThread + 1;
      do i=1 to nObsPerThread;
         u = rand(\'uniform\');
         do j=1 to dim(x);
            w{j} = rand(\'Uniform\');
            x{j} = (w{j} + u)/2;
         end;
         f1 = sin(pi * x1 * x2 );
         f2 = (x3-0.5)**2;
         f3 = x4;
         f4 = x5;
         fb = 10*f1 +20*f2+10*f3+5*f4;
         y = fb +  rand(\'Normal\');
         output;
      end;
   run;
   ',
    single='no')

By default, the CAS server executes the preceding DATA step in parallel in every thread on every node in your session. For example, if your DATA step creates 10 observations and executes in parallel in a session that has three worker nodes, each of which uses 16 threads, then the resulting CAS table has 160 observations on each worker node, yielding a CAS table that has a total of 480 observations. The preceding DATA step uses this technique to create a table that has a specified number of observations in parallel on your CAS server by dividing the desired number of observations (nTotalObs) by the total number of threads (_nthreads_). If the number of observations cannot be exactly divided by the number of threads, the code adds one observation to each of a subset of the threads until all observations are accounted for.

The following statements use the bartGauss action in the bart action set to fit a Bayesian additive regression trees (BART) model to these data. The table parameter names the input data table to be analyzed. You can use one of two mutually exclusive modes to specify the variables in the model by using the bartGauss action: in one mode, you use the target, inputs, and nominals parameters; in the other mode, you use the class and model parameters. In this example, the first mode is used. The target parameter names the target (response) variable, and the inputs parameter lists the explanatory variables x1–x40. In this example, because the input data are not large, the trainInMem and obsLeafMapInMem parameters are used to store elements of the model in memory when the model is trained. The store parameter stores the fitted model in the modelFit data table.

m <- cas.builtins.loadActionSet(s, actionset='bart')

cas.bart.bartGauss(s,
    inputs=c('x1', 'x2', 'x3', 'x4', 'x5', 'x6', 'x7', 'x8', 'x9', 'x10', 'x11',
             'x12', 'x13', 'x14', 'x15', 'x16', 'x17', 'x18', 'x19', 'x20',
             'x21', 'x22', 'x23', 'x24', 'x25', 'x26', 'x27', 'x28', 'x29',
             'x30', 'x31', 'x32', 'x33', 'x34', 'x35', 'x36', 'x37', 'x38',
             'x39', 'x40'),
    obsLeafMapInMem='true',
    seed='9181',
    store=list(name='modelFit', replace='true'),
    table='trainData',
    target='y',
    trainInMem='true')

The output from this analysis is displayed by default. It is presented in Output 4.1.22 through Output 4.1.28.

The "Model Information" table in Output 4.1.22 summarizes important information about the model that you fit. Information includes the input data source, response (target) variable, bin method, missing rule, simulation size, thinning rate, and random number seed.

Output 4.1.23 displays the "Number of Observations" table. All 10,000 observations in the data table are used in the analysis.

Output 4.1.24 displays the "Prior Information" table. The table summarizes the parameters that define the BART prior for the tree structure and the variance of the response variable. The BART prior parameters typically do not require hyperparameter tuning, because the default BART model tends to perform well in terms of model fit, and this performance does not change significantly if there are small changes in the prior parameters.

Output 4.1.25 displays the "Sample Information" table. This table summarizes basic information about the posterior samples of the sum-of-trees ensemble, such as the number of samples saved, the proportion of tree-modifying operations accepted, and the size of the trees.

When you run the bartGauss action on a cluster of machines, you can use a distributed mode that divides the MCMC samples and runs multiple chains in parallel. By default, each worker node in the cluster has a full copy of the training data and runs a separate chain. You can specify the number of separate chains by using the distributeChains parameter. In this example, the bartGauss action was run on a cluster of machines; Output 4.1.26 displays the "Chain Information" table. This table indicates the number of parallel chains that the procedure used and the number of posterior samples that were saved for prediction from each chain. Note that the fit of a BART model depends on the number of chains that are run.

Output 4.1.27 displays the fit statistics for the fitted model. In this case, there is only one: the average square error (ASE).

Finally, the bartGauss action displays the table shown in Output 4.1.28, which shows the amount of time (in seconds) that it took to perform the various tasks in the analysis.