SUPERLEARNER Procedure

The SUPERLEARNER procedure implements the super learner model in SAS Viya (Van der Laan and Rose 2011; Phillips et al. 2023; Naimi and Balzer 2018). In the terminology of machine learning, super learner models are predictive models. The super learner model constructs a predictive model by combining predictions from other models. You can use PROC SUPERLEARNER to train and score predictive models for continuous and binary response variables and to compute predictive margins.

There is no single best machine learning model for all data sets and prediction problems. Given a specific prediction task, it is often difficult to choose a predictive model a priori that will perform best. Wolpert (1992) proposed an approach to increase prediction accuracy by combining multiple models into a single model. Later, this approach became known as "stacking." Breiman (1996) demonstrated additional improvements to predictive performance by using nonnegative least squares for stacking. Van der Laan and colleagues proved the theoretical optimality of stacking in large samples and coined an additional name, "super learner" (Van der Laan and Dudoit 2003; Dudoit and van der Laan 2005; Van der Vaart, Dudoit, and van der Laan 2006; Van der Laan, Polley, and Hubbard 2007). Van der Laan and Rose (2011) provided a detailed description of the super learner model and conducted simulation studies to evaluate its finite sample performance. Phillips et al. (2023) provided practical recommendations for specifying super learner models. The super learner model considers a collection of candidate models, or base learners, and systematically chooses the best combination of them. Optimality theory shows that in large samples, the super learner model performs at least as well as the best base learner model in the ensemble, where predictive performance is defined in terms of a bounded loss function (Van der Laan, Polley, and Hubbard 2007).

When you train a super learner model, you must specify a list of predictive models (also called base learners), a cross-validation scheme, and a meta-learner model. The base learners that make up the ensemble can range from parametric regressions to nonparametric machine learning algorithms. The super learner model uses cross-validation, a sample splitting technique, to assess each base learner’s performance. This is an iterative process in which each base learner is trained on a portion of the input data and predicted responses are generated for observations that were not used in the training process. The meta-learner model then computes the best weighted combination of base learners. The output of the training process is an ensemble predictive model that uses a vector of input variables (also called predictor variables) to predict the value of a response variable (also called a target variable). The purpose of a predictive model is to use a set of input variables to predict a target variable. You create a predictive model by using training data in which the target values are known. If the model is able to accurately predict the value of the target for observations that were not included in the training data, then the model is said to generalize well. Good generalization is an important property of a predictive model. Internally, the super learner model uses cross-validation to mitigate overfitting of the training data and obtain good generalization. The internal cross-validation estimates the out-of-sample performance of each base learner. Base learners that demonstrate better out-of-sample performance tend to receive larger coefficients and generally contribute more to the final predictions.

For more information about super learner models, see the section Details: SUPERLEARNER Procedure.