SUPERLEARNER Procedure
Meta-learning Methods
Let 
denote the block of input data that contains observation i; let 
denote the number of observations in block k; and let 
denote the parameter estimates of base learner model b by training it on the input data, excluding block k. Let 
denote the out-of-sample predicted response that is made by base learner b, given the inputs 
.
The meta-learner model m is defined as a linear combination of the base learner models:
Continuous Response
For a continuous response, you estimate the super learner coefficients 
by minimizing the following cross-validated empirical risk that is based on the squared loss function:
Solving this minimization problem is equivalent to fitting a linear regression model (without the intercept) that regresses the observed responses on the base learner out-of-sample predictions. In addition, it is theoretically advantageous to restrict the search for the coefficients 
to the convex combination (Van der Laan, Polley, and Hubbard 2007); that is, 
and 
for all 
. Solving the cross-validated empirical risk minimization with this convex constraint is the default meta-learning method for a continuous response.
Binary Response
For a binary response 
, now 
denotes the out-of-sample predicted probability of 
that is made by base learner b, given the inputs . Three meta-learning approaches are available for a binary response.
The first approach is to estimate the super learner coefficients by minimizing the cross-validated empirical risk that is based on the squared loss function.
The second approach is to estimate by maximizing the following binomial log likelihood:
As in the continuous response case, the coefficients are restricted so that and for all .
The third approach is to maximize the area under the ROC curve (AUC). This approach is effective when the response variable is highly imbalanced or when classification performance is the primary objective (LeDell, van der Laan, and Petersen 2016). To maximize the AUC in the meta-learning step, you estimate the super learner coefficients by maximizing the following:
where 
is the indicator function. As in the continuous response case, the search for coefficients is restricted to the convex combination. The AUC optimization problem is solved by using the black-box solver. For more information about the black-box solver, see the discussion in the section "Black-Box Optimization Solver" in SAS Optimization: Mathematical Optimization Procedures. Because the objective function is nonconvex and piecewise constant, the black-box solver converges to a local optimum. The availability of AUC maximization is limited by the SAS software product that you are using. PROC SUPERLEARNER displays an error message and stops the analysis if METHOD=CCAUC is not available with your product license.
Discrete Super Learner
The discrete super learner model (also known as a cross-validated selector) is a variation of the super learner that selects a single base learner model that has the smallest cross-validated risk instead of searching for the optimal combination of all base learners. The base learner that has the smallest cross-validated risk receives a super learner coefficient of 1, and the remaining base learners receive coefficients of 0. You can train a discrete super learner by specifying the METHOD=CVSELECTOR option in the PROC SUPERLEARNER statement.