Data Science Pilot Action Set

Automated machine learning tools range in capability from fully automated, black-box tools to those that offer a series of workflows to guide inexperienced users in their quest for optimal models (in terms of predictive accuracy, interpretability, and so on). The dsAutoMl action performs policy-based, scalable, end-to-end automated machine learning—specifically, machine learning pipeline exploration, execution, and ranking. This is in contrast to the other actions in the dataSciencePilot action set, which can be used for automating subcomponents of an end-to-end machine learning workflow.

The dsAutoMl action can be used for both regression and classification problems. As an action for automated machine learning, it requires only minimal input from the user: the input data set; the target variable to predict; and optional policy parameters for data exploration, variable screening, feature selection, feature transformation and generation, and the models (algorithms) to consider in the pipeline exploration.

The policy parameters determine the scale of the pipeline exploration. The action applies these parameters in the following order:

The dsAutoMl action considers the models that are specified in the modelTypes parameter during its pipeline sampling. The action makes the following model types available: decision tree, generalized linear model (GLM), logistic regression, neural network, random forest, and gradient boosting. By default, only the decision tree, GLM, and logistic regression models are activated. You can specify any combination of these model types. Each pipeline is composed of a set of features and a model type. During the construction of a pipeline, features are selected for inclusion by using a feature-representation selection algorithm. The action groups together pipelines that differ only in their model type, and the hyperparameters of these pipelines are optimized using the tuneAll action or the modelComposer action. You can select the hyperparameter optimizer by using the hyperParameterOptimizer parameter of the dsAutoMl action. The hyperparameter tuning optimizes the objective parameter value by using cross validation or a separately specified validation table. By default, classification problems use the misclassification error rate (MCE) as their objective parameter value, and regression problems use the average square error (ASR). You can use the topk subparameter of the saveState parameter to save the top-k best-performing pipelines.

The dsAutoMl action generates CAS output tables that contain metadata about the feature transformation and generation pipelines; metadata about the generated features; metadata about the executed and ranked machine learning pipelines; the analytic store scoring object for feature generation; and the best-performing models. You should note that the decision tree and neural network models do not produce analytic store scoring objects. Instead these models produce model-specific scoring objects. You should use the dtreeScore action to score with the saved decision tree model and use the annScore action to score with the saved neural net model object.