Data Science Pilot Action Set: Syntax
Provides actions for automating data science workflows, including automatic machine learning pipeline exploration, execution and ranking.
featureMachine Action
Automated feature transformation and generation engine.
CASL Syntax
Summary: Input and Output Tables
If a row includes a subparameter, you can specify the name, caslib, and so on in the subparameter. Otherwise, you can specify the name, caslib, and so on in the parameter.
|
Parameter |
Subparameter |
Description |
|---|---|---|
|
required parametertable |
— |
specifies the table name, caslib, and other common parameters. |
|
Parameter |
Subparameter |
Description |
|---|---|---|
|
— |
specifies the CAS table to store the analysis results. | |
|
required parameterfeatureOut |
— |
specifies the CAS table to store the feature transformation and generation pipelines. |
|
— |
specifies the CAS table to store the feature transformation and generation model. | |
|
required parametertransformationOut |
— |
specifies the CAS table to store the feature transformation and generation pipelines. |
Parameter Descriptions
casout={casouttable}
specifies the CAS table to store the analysis results.
| Long form | casout={name="table-name"} |
|---|---|
| Shortcut form | casout="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=TRUE | FALSE
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | FALSE |
|---|
replace=TRUE | FALSE
when set to True, overwrites an existing table that has the same name.
| Default | FALSE |
|---|
copyVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the names of variables to be copied to the output table.
distinctCountLimit=integer
specifies the distinct count limit. If the limit is exceeded, and the misraGries parameter is set to True, the Misra-Gries frequency sketch algorithm is used to estimate the frequency distribution. Otherwise, the distinct count operation is aborted.
| Default | 10000 |
|---|---|
| Minimum value | 256 |
ecdfTolerance=double
specifies the tolerance value for the empirical cumulative distribution function. This value is used by the quantile sketch algorithm.
| Default | 0.001 |
|---|---|
| Range | 0–0.1 |
event="string"
specifies the target variable level that you want to model. Multilevel classification problems are cast into a one-versus-all binary classification problem, where the value of the event parameter denotes the level that you are modeling.
explorationPolicy={avaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) policy.
| Alias | avaptPolicy |
|---|
The avaptPolicy value can be one or more of the following:
cardinality={cardinalityAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) cardinality policy.
The cardinalityAvaptPolicy value can be one or more of the following:
lowMediumCutoff=double
specifies the cardinality threshold for the low-medium cutoff.
| Default | 32 |
|---|---|
| Range | 2–256 |
mediumHighCutoff=double
specifies the cardinality threshold for the medium-high cutoff.
| Default | 64 |
|---|---|
| Range | 2–1024 |
minNObsPerTargetLevel=double
specifies the minimum number of observations for each target level.
| Default | 10 |
|---|---|
| Range | 5–100 |
cv={cvAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) coefficient of variation policy.
| Alias | coefficientVariation |
|---|
The cvAvaptPolicy value can be one or more of the following:
lowMoment=double
specifies the low-high percentage threshold for the moment coefficient of variation (CV).
| Default | 1E-05 |
|---|---|
| Range | 0–100 |
lowRobust=double
specifies the low-high percentage threshold for the robust coefficient of variation (CV).
| Default | 0.001 |
|---|---|
| Range | 0–100 |
dateTimeVariables={"variable-name-1" <, "variable-name-2", ...>}
specifies the datetime variables.
| Alias | dateTime |
|---|
dateVariables={"variable-name-1" <, "variable-name-2", ...>}
specifies the date variables.
| Alias | date |
|---|
entropy={entropyAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) entropy policy.
The entropyAvaptPolicy value can be one or more of the following:
giniLowMediumCutoff=double
specifies the Gini entropy threshold for the low-medium cutoff.
| Default | 0.25 |
|---|---|
| Range | 0–1 |
giniMediumHighCutoff=double
specifies the Gini entropy threshold for the medium-high cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–1 |
shannonLowMediumCutoff=double
specifies the Shannon entropy threshold for the low-medium cutoff.
| Default | 0.25 |
|---|---|
| Range | 0–1 |
shannonMediumHighCutoff=double
specifies the Shannon entropy threshold for the medium-high cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–1 |
iqv={iqvAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) index of qualitative variation policy.
| Alias | qualitativeVariationIndex |
|---|
The iqvAvaptPolicy value can be one or more of the following:
highTopBottom=double
specifies the low-high cutoff frequency ratio threshold between the most frequent and least frequent levels of a nominal variable.
| Alias | highTop1Bottom1 |
|---|---|
| Default | 100 |
| Minimum value | 1 |
highTopTwo=double
specifies the low-high cutoff frequency ratio threshold between the most frequent and second most frequent levels of a nominal variable.
| Alias | highTop1Top2 |
|---|---|
| Default | 10 |
| Minimum value | 1 |
highVariationRatio=double
specifies the variation ratio threshold for the low-high cutoff.
| Alias | highModVr |
|---|---|
| Default | 0.5 |
| Range | (0–1] |
kurtosis={kurtosisAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) kurtosis policy.
The kurtosisAvaptPolicy value can be one or more of the following:
momentLowMediumCutoff=double
specifies the moment kurtosis threshold for the low-medium cutoff.
| Default | 5 |
|---|---|
| Range | 0–100 |
momentMediumHighCutoff=double
specifies the moment kurtosis threshold for the medium-high cutoff.
| Default | 10 |
|---|---|
| Range | 0–100 |
robustLowMediumCutoff=double
specifies the robust kurtosis threshold for the low-medium cutoff.
| Default | 2 |
|---|---|
| Range | 0–100 |
robustMediumHighCutoff=double
specifies the robust kurtosis threshold for the medium-high cutoff.
| Default | 3 |
|---|---|
| Range | 0–100 |
missing={missingAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) missing grouping policy.
The missingAvaptPolicy value can be one or more of the following:
lowMediumCutoff=double
specifies the missing percentage threshold for the low-medium cutoff.
| Default | 5 |
|---|---|
| Range | 0–100 |
mediumHighCutoff=double
specifies the missing percentage threshold for the medium-high cutoff.
| Default | 25 |
|---|---|
| Range | 0–100 |
nominal={nominalAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) nominal policy.
The nominalAvaptPolicy value can be one or more of the following:
cardinalityRatio=double
specifies the AVAPT nominal policy cardinality ratio threshold.
| Default | 0.25 |
|---|---|
| Range | (0–1] |
cardinalityThreshold=double
specifies the AVAPT nominal policy cardinality threshold.
| Default | 1024 |
|---|---|
| Minimum value | 32 |
includeNegative=TRUE | FALSE
when set to True, includes numeric variables with some negative values in the nominal analysis.
| Default | FALSE |
|---|
includeNonIntegral=TRUE | FALSE
when set to True, includes numeric variables with some nonintegral values in the nominal analysis.
| Default | FALSE |
|---|
intervals={"variable-name-1" <, "variable-name-2", ...>}
specifies variables to consider as intervals.
nominals={"variable-name-1" <, "variable-name-2", ...>}
specifies variables to consider as nominals.
outlier={outlierAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) outlier policy.
The outlierAvaptPolicy value can be one or more of the following:
momentLowMediumCutoff=double
specifies the z-score outlier percentage threshold for the low-medium cutoff.
| Default | 1 |
|---|---|
| Range | 0–100 |
momentMediumHighCutoff=double
specifies the z-score outlier percentage threshold for the medium-high cutoff.
| Default | 2.5 |
|---|---|
| Range | 0–100 |
robustLowMediumCutoff=double
specifies the modified interquartile range outlier percentage threshold for the low-medium cutoff.
| Default | 1 |
|---|---|
| Range | 0–100 |
robustMediumHighCutoff=double
specifies the modified interquartile range outlier percentage threshold for the medium-high cutoff.
| Default | 2.5 |
|---|---|
| Range | 0–100 |
skewness={skewnessAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) skewness policy.
The skewnessAvaptPolicy value can be one or more of the following:
momentLowMediumCutoff=double
specifies the moment skewness threshold for the low-medium cutoff.
| Default | 2 |
|---|---|
| Range | 0–100 |
momentMediumHighCutoff=double
specifies the moment skewness threshold for the medium-high cutoff.
| Default | 10 |
|---|---|
| Range | 0–100 |
robustLowMediumCutoff=double
specifies the robust skewness threshold for the low-medium cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–3 |
robustMediumHighCutoff=double
specifies the robust skewness threshold for the medium-high cutoff.
| Default | 2 |
|---|---|
| Range | 0–3 |
timeVariables={"variable-name-1" <, "variable-name-2", ...>}
specifies the time variables.
| Alias | time |
|---|
* featureOut={casouttable}
specifies the CAS table to store the feature transformation and generation pipelines.
| Long form | featureOut={name="table-name"} |
|---|---|
| Shortcut form | featureOut="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=TRUE | FALSE
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | FALSE |
|---|
replace=TRUE | FALSE
when set to True, overwrites an existing table that has the same name.
| Default | FALSE |
|---|
freq="variable-name"
specifies the frequency variable.
inputs={{casinvardesc-1} <, {casinvardesc-2}, ...>}
specifies the variables to use for the analysis. You can specify a subset of the variables from the input table.
For more information about specifying the inputs parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
| Alias | vars |
|---|
misraGries=TRUE | FALSE
when set to True, uses the Misra-Gries algorithm for the frequency distribution estimation, if the distinct count limit is exceeded.
| Default | TRUE |
|---|
rankPolicy={rankPolicy}
specifies the feature ranking policy.
| Alias | rank |
|---|
| Long form | rankPolicy={intervalStat="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"} |
|---|---|
| Shortcut form | rankPolicy="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU" |
The rankPolicy value can be one or more of the following:
intervalStat="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"
specifies the interval variable ranking statistic.
| Alias | interval |
|---|---|
| Default | SU |
nominalStat="CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU"
specifies the nominal variable ranking statistic.
| Alias | nominal |
|---|---|
| Default | SU |
rankMissingIndicators=TRUE | FALSE
when set to True, missing indicator features take part in the feature ranking. Otherwise, they are excluded from the ranking, and hence will always appear in the final feature set.
| Default | TRUE |
|---|
separateRank=TRUE | FALSE
when set to True, performs a separate feature ranking for interval and nominal features.
| Alias | separate |
|---|---|
| Default | TRUE |
topKInteractions=integer
specifies the number of top-ranked interaction features to generate and save.
| Alias | topKInteract |
|---|---|
| Minimum value | 1 |
topKSave=integer
specifies the number of features per variable to save.
| Alias | topK |
|---|---|
| Default | 1 |
| Minimum value | 1 |
saveState={casouttable}
specifies the CAS table to store the feature transformation and generation model.
| Alias | saveModel |
|---|
| Long form | saveState={name="table-name"} |
|---|---|
| Shortcut form | saveState="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=TRUE | FALSE
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | FALSE |
|---|
replace=TRUE | FALSE
when set to True, overwrites an existing table that has the same name.
| Default | FALSE |
|---|
screenPolicy={sweeperPolicy}
specifies the variable screening policy to use for recommending that variables be screened out, transformed, or copied.
| Alias | sweeperPolicy |
|---|
The sweeperPolicy value can be one or more of the following:
constant=TRUE | FALSE
when set to True, uses the variable screening policy to identify variables that have constant values.
| Alias | unique |
|---|---|
| Default | TRUE |
groupRareLevels=TRUE | FALSE
when set to True, uses the variable screening policy to identify nominal variables that have rare levels.
| Alias | groupRare |
|---|---|
| Default | TRUE |
leakagePercentThreshold=double
specifies the variable screening policy for variables that have a very high level of information about the target. Variables that have a greater target entropy percentage reduction than the specified threshold are flagged as leakage variables.
| Alias | leakagePercentageThreshold |
|---|---|
| Default | 90 |
| Range | (0–100] |
lowCv=TRUE | FALSE
when set to True, uses the variable screening policy to identify variables that have a low coefficient of variation (CV).
| Alias | lowCoefficientVariation |
|---|---|
| Default | TRUE |
lowMutualInformation=double
specifies the variable screening policy for variables that have a low level of information about the target.
| Alias | lowInformation |
|---|---|
| Default | 0.05 |
| Minimum value | 0 |
missingIndicatorPercent=double
specifies the variable screening policy for generating missing indicator variables.
| Alias | missingIndicatorPercentage |
|---|---|
| Default | 75 |
| Range | [10–100) |
missingPercentThreshold=double
specifies the variable screening policy for identifying variables that have a very high missing rate.
| Alias | missingPercentageThreshold |
|---|---|
| Default | 90 |
| Range | [10–100) |
redundant=double
specifies the symmetric uncertainty (SU) threshold for identifying redundant variables. If the SU for two variables exceeds the threshold, the variable that has less information about the target is flagged as redundant.
| Default | 1 |
|---|---|
| Range | (0–1] |
seed=integer
specifies a seed value for random number generation. This value is used for repeatable random number generation in some scenarios.
| Default | 0 |
|---|
* table={castable}
specifies the table name, caslib, and other common parameters.
| Long form | table={name="table-name"} |
|---|---|
| Shortcut form | table="table-name" |
The castable value can be one or more of the following:
caslib="string"
specifies the caslib for the input table that you want to use with the action. By default, the active caslib is used. Specify a value only if you need to access a table from a different caslib.
computedOnDemand=TRUE | FALSE
when set to True, creates the computed variables when the table is loaded instead of when the action begins.
| Alias | compOnDemand |
|---|---|
| Default | FALSE |
computedVars={{casinvardesc-1} <, {casinvardesc-2}, ...>}
specifies the names of the computed variables to create. Specify an expression for each variable in the computedVarsProgram parameter. If you do not specify this parameter, then all variables from computedVarsProgram are automatically included.
| Alias | compVars |
|---|
For more information about specifying the computedVars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
computedVarsProgram="string"
specifies an expression for each computed variable that you include in the computedVars parameter.
| Alias | compPgm |
|---|
dataSourceOptions={key-1=any-list-or-data-type-1 <, key-2=any-list-or-data-type-2, ...>}
specifies data source options.
| Alias | options, dataSource |
|---|
importOptions={fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}
specifies the settings for reading a table from a data source.
| Alias | import |
|---|
For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).
* name="table-name"
specifies the name of the input table.
singlePass=TRUE | FALSE
when set to True, does not create a transient table on the server. Setting this parameter to True can be efficient, but the data might not have stable ordering upon repeated runs.
| Default | FALSE |
|---|
vars={{casinvardesc-1} <, {casinvardesc-2}, ...>}
specifies the variables to use in the action.
For more information about specifying the vars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
where="where-expression"
specifies an expression for subsetting the input data.
whereTable={groupbytable}
specifies an input table that contains rows to use as a WHERE filter. If the vars parameter is not specified, then all the variable names that are common to the input table and the filtering table are used to find matching rows. If the where parameter for the input table and this parameter are specified, then this filtering table is applied first.
The groupbytable value can be one or more of the following:
casLib="string"
specifies the caslib for the filter table. By default, the active caslib is used.
dataSourceOptions={adls_noreq-parameters | bigquery-parameters | clouddex-parameters | db2-parameters | dnfs-parameters | esp-parameters | hadoop-parameters | hana-parameters | impala-parameters | jdbc-parameters | mongodb-parameters | mysql-parameters | odbc-parameters | oracle-parameters | path-parameters | postgres-parameters | redshift-parameters | s3-parameters | sforce-parameters | snowflake-parameters | spark-parameters | spde-parameters | sqlserver-parameters | teradata-parameters | vertica-parameters | yellowbrick-parameters}
specifies data source options.
| Alias | options, dataSource |
|---|
For more information about specifying the dataSourceOptions parameter, see the common dataSourceOptions parameter (Appendix A: Common Parameters).
importOptions={fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}
specifies the settings for reading a table from a data source.
| Alias | import |
|---|
For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).
* name="table-name"
specifies the name of the filter table.
vars={{casinvardesc-1} <, {casinvardesc-2}, ...>}
specifies the variable names to use from the filter table.
For more information about specifying the vars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
where="where-expression"
specifies an expression for subsetting the data from the filter table.
* target="variable-name"
specifies the target variable.
| Alias | evalVar |
|---|
* transformationOut={casouttable}
specifies the CAS table to store the feature transformation and generation pipelines.
| Long form | transformationOut={name="table-name"} |
|---|---|
| Shortcut form | transformationOut="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=TRUE | FALSE
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | FALSE |
|---|
replace=TRUE | FALSE
when set to True, overwrites an existing table that has the same name.
| Default | FALSE |
|---|
transformationPolicy={transformationSpace}
specifies the feature transformation and generation space in which the feature machine operates.
| Alias | ftgPolicy |
|---|
The transformationSpace value can be one or more of the following:
cardinality=TRUE | FALSE
when set to True, includes cardinality-reducing transformations.
| Default | TRUE |
|---|
entropy=TRUE | FALSE
when set to True, includes transformations for the treatment of low entropy.
| Default | FALSE |
|---|
interaction=TRUE | FALSE
when set to True, detects and generates interaction features.
| Default | FALSE |
|---|
iqv=TRUE | FALSE
when set to True, includes transformations for the treatment of low indices of qualitative variation (IQV).
| Default | FALSE |
|---|
kurtosis=TRUE | FALSE
when set to True, includes transformations for the treatment of high kurtosis.
| Default | FALSE |
|---|
missing=TRUE | FALSE
when set to True, includes transformations for the treatment of missing values.
| Default | TRUE |
|---|
outlier=TRUE | FALSE
when set to True, includes transformations for the treatment of outliers.
| Default | FALSE |
|---|
polynomial=TRUE | FALSE
when set to True, includes up to third-order polynomial transformations.
| Default | FALSE |
|---|
skewness=TRUE | FALSE
when set to True, includes transformations for the treatment of high skewness.
| Default | TRUE |
|---|
featureMachine Action
Automated feature transformation and generation engine.
Lua Syntax
Summary: Input and Output Tables
If a row includes a subparameter, you can specify the name, caslib, and so on in the subparameter. Otherwise, you can specify the name, caslib, and so on in the parameter.
|
Parameter |
Subparameter |
Description |
|---|---|---|
|
required parametertable |
— |
specifies the table name, caslib, and other common parameters. |
|
Parameter |
Subparameter |
Description |
|---|---|---|
|
— |
specifies the CAS table to store the analysis results. | |
|
required parameterfeatureOut |
— |
specifies the CAS table to store the feature transformation and generation pipelines. |
|
— |
specifies the CAS table to store the feature transformation and generation model. | |
|
required parametertransformationOut |
— |
specifies the CAS table to store the feature transformation and generation pipelines. |
Parameter Descriptions
casout={casouttable}
specifies the CAS table to store the analysis results.
| Long form | casout={name="table-name"} |
|---|---|
| Shortcut form | casout="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=true | false
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | false |
|---|
replace=true | false
when set to True, overwrites an existing table that has the same name.
| Default | false |
|---|
copyVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the names of variables to be copied to the output table.
distinctCountLimit=integer
specifies the distinct count limit. If the limit is exceeded, and the misraGries parameter is set to True, the Misra-Gries frequency sketch algorithm is used to estimate the frequency distribution. Otherwise, the distinct count operation is aborted.
| Default | 10000 |
|---|---|
| Minimum value | 256 |
ecdfTolerance=double
specifies the tolerance value for the empirical cumulative distribution function. This value is used by the quantile sketch algorithm.
| Default | 0.001 |
|---|---|
| Range | 0–0.1 |
event="string"
specifies the target variable level that you want to model. Multilevel classification problems are cast into a one-versus-all binary classification problem, where the value of the event parameter denotes the level that you are modeling.
explorationPolicy={avaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) policy.
| Alias | avaptPolicy |
|---|
The avaptPolicy value can be one or more of the following:
cardinality={cardinalityAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) cardinality policy.
The cardinalityAvaptPolicy value can be one or more of the following:
lowMediumCutoff=double
specifies the cardinality threshold for the low-medium cutoff.
| Default | 32 |
|---|---|
| Range | 2–256 |
mediumHighCutoff=double
specifies the cardinality threshold for the medium-high cutoff.
| Default | 64 |
|---|---|
| Range | 2–1024 |
minNObsPerTargetLevel=double
specifies the minimum number of observations for each target level.
| Default | 10 |
|---|---|
| Range | 5–100 |
cv={cvAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) coefficient of variation policy.
| Alias | coefficientVariation |
|---|
The cvAvaptPolicy value can be one or more of the following:
lowMoment=double
specifies the low-high percentage threshold for the moment coefficient of variation (CV).
| Default | 1E-05 |
|---|---|
| Range | 0–100 |
lowRobust=double
specifies the low-high percentage threshold for the robust coefficient of variation (CV).
| Default | 0.001 |
|---|---|
| Range | 0–100 |
dateTimeVariables={"variable-name-1" <, "variable-name-2", ...>}
specifies the datetime variables.
| Alias | dateTime |
|---|
dateVariables={"variable-name-1" <, "variable-name-2", ...>}
specifies the date variables.
| Alias | date |
|---|
entropy={entropyAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) entropy policy.
The entropyAvaptPolicy value can be one or more of the following:
giniLowMediumCutoff=double
specifies the Gini entropy threshold for the low-medium cutoff.
| Default | 0.25 |
|---|---|
| Range | 0–1 |
giniMediumHighCutoff=double
specifies the Gini entropy threshold for the medium-high cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–1 |
shannonLowMediumCutoff=double
specifies the Shannon entropy threshold for the low-medium cutoff.
| Default | 0.25 |
|---|---|
| Range | 0–1 |
shannonMediumHighCutoff=double
specifies the Shannon entropy threshold for the medium-high cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–1 |
iqv={iqvAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) index of qualitative variation policy.
| Alias | qualitativeVariationIndex |
|---|
The iqvAvaptPolicy value can be one or more of the following:
highTopBottom=double
specifies the low-high cutoff frequency ratio threshold between the most frequent and least frequent levels of a nominal variable.
| Alias | highTop1Bottom1 |
|---|---|
| Default | 100 |
| Minimum value | 1 |
highTopTwo=double
specifies the low-high cutoff frequency ratio threshold between the most frequent and second most frequent levels of a nominal variable.
| Alias | highTop1Top2 |
|---|---|
| Default | 10 |
| Minimum value | 1 |
highVariationRatio=double
specifies the variation ratio threshold for the low-high cutoff.
| Alias | highModVr |
|---|---|
| Default | 0.5 |
| Range | (0–1] |
kurtosis={kurtosisAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) kurtosis policy.
The kurtosisAvaptPolicy value can be one or more of the following:
momentLowMediumCutoff=double
specifies the moment kurtosis threshold for the low-medium cutoff.
| Default | 5 |
|---|---|
| Range | 0–100 |
momentMediumHighCutoff=double
specifies the moment kurtosis threshold for the medium-high cutoff.
| Default | 10 |
|---|---|
| Range | 0–100 |
robustLowMediumCutoff=double
specifies the robust kurtosis threshold for the low-medium cutoff.
| Default | 2 |
|---|---|
| Range | 0–100 |
robustMediumHighCutoff=double
specifies the robust kurtosis threshold for the medium-high cutoff.
| Default | 3 |
|---|---|
| Range | 0–100 |
missing={missingAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) missing grouping policy.
The missingAvaptPolicy value can be one or more of the following:
lowMediumCutoff=double
specifies the missing percentage threshold for the low-medium cutoff.
| Default | 5 |
|---|---|
| Range | 0–100 |
mediumHighCutoff=double
specifies the missing percentage threshold for the medium-high cutoff.
| Default | 25 |
|---|---|
| Range | 0–100 |
nominal={nominalAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) nominal policy.
The nominalAvaptPolicy value can be one or more of the following:
cardinalityRatio=double
specifies the AVAPT nominal policy cardinality ratio threshold.
| Default | 0.25 |
|---|---|
| Range | (0–1] |
cardinalityThreshold=double
specifies the AVAPT nominal policy cardinality threshold.
| Default | 1024 |
|---|---|
| Minimum value | 32 |
includeNegative=true | false
when set to True, includes numeric variables with some negative values in the nominal analysis.
| Default | false |
|---|
includeNonIntegral=true | false
when set to True, includes numeric variables with some nonintegral values in the nominal analysis.
| Default | false |
|---|
intervals={"variable-name-1" <, "variable-name-2", ...>}
specifies variables to consider as intervals.
nominals={"variable-name-1" <, "variable-name-2", ...>}
specifies variables to consider as nominals.
outlier={outlierAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) outlier policy.
The outlierAvaptPolicy value can be one or more of the following:
momentLowMediumCutoff=double
specifies the z-score outlier percentage threshold for the low-medium cutoff.
| Default | 1 |
|---|---|
| Range | 0–100 |
momentMediumHighCutoff=double
specifies the z-score outlier percentage threshold for the medium-high cutoff.
| Default | 2.5 |
|---|---|
| Range | 0–100 |
robustLowMediumCutoff=double
specifies the modified interquartile range outlier percentage threshold for the low-medium cutoff.
| Default | 1 |
|---|---|
| Range | 0–100 |
robustMediumHighCutoff=double
specifies the modified interquartile range outlier percentage threshold for the medium-high cutoff.
| Default | 2.5 |
|---|---|
| Range | 0–100 |
skewness={skewnessAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) skewness policy.
The skewnessAvaptPolicy value can be one or more of the following:
momentLowMediumCutoff=double
specifies the moment skewness threshold for the low-medium cutoff.
| Default | 2 |
|---|---|
| Range | 0–100 |
momentMediumHighCutoff=double
specifies the moment skewness threshold for the medium-high cutoff.
| Default | 10 |
|---|---|
| Range | 0–100 |
robustLowMediumCutoff=double
specifies the robust skewness threshold for the low-medium cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–3 |
robustMediumHighCutoff=double
specifies the robust skewness threshold for the medium-high cutoff.
| Default | 2 |
|---|---|
| Range | 0–3 |
timeVariables={"variable-name-1" <, "variable-name-2", ...>}
specifies the time variables.
| Alias | time |
|---|
* featureOut={casouttable}
specifies the CAS table to store the feature transformation and generation pipelines.
| Long form | featureOut={name="table-name"} |
|---|---|
| Shortcut form | featureOut="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=true | false
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | false |
|---|
replace=true | false
when set to True, overwrites an existing table that has the same name.
| Default | false |
|---|
freq="variable-name"
specifies the frequency variable.
inputs={{casinvardesc-1} <, {casinvardesc-2}, ...>}
specifies the variables to use for the analysis. You can specify a subset of the variables from the input table.
For more information about specifying the inputs parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
| Alias | vars |
|---|
misraGries=true | false
when set to True, uses the Misra-Gries algorithm for the frequency distribution estimation, if the distinct count limit is exceeded.
| Default | true |
|---|
rankPolicy={rankPolicy}
specifies the feature ranking policy.
| Alias | rank |
|---|
| Long form | rankPolicy={intervalStat="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"} |
|---|---|
| Shortcut form | rankPolicy="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU" |
The rankPolicy value can be one or more of the following:
intervalStat="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"
specifies the interval variable ranking statistic.
| Alias | interval |
|---|---|
| Default | SU |
nominalStat="CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU"
specifies the nominal variable ranking statistic.
| Alias | nominal |
|---|---|
| Default | SU |
rankMissingIndicators=true | false
when set to True, missing indicator features take part in the feature ranking. Otherwise, they are excluded from the ranking, and hence will always appear in the final feature set.
| Default | true |
|---|
separateRank=true | false
when set to True, performs a separate feature ranking for interval and nominal features.
| Alias | separate |
|---|---|
| Default | true |
topKInteractions=integer
specifies the number of top-ranked interaction features to generate and save.
| Alias | topKInteract |
|---|---|
| Minimum value | 1 |
topKSave=integer
specifies the number of features per variable to save.
| Alias | topK |
|---|---|
| Default | 1 |
| Minimum value | 1 |
saveState={casouttable}
specifies the CAS table to store the feature transformation and generation model.
| Alias | saveModel |
|---|
| Long form | saveState={name="table-name"} |
|---|---|
| Shortcut form | saveState="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=true | false
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | false |
|---|
replace=true | false
when set to True, overwrites an existing table that has the same name.
| Default | false |
|---|
screenPolicy={sweeperPolicy}
specifies the variable screening policy to use for recommending that variables be screened out, transformed, or copied.
| Alias | sweeperPolicy |
|---|
The sweeperPolicy value can be one or more of the following:
constant=true | false
when set to True, uses the variable screening policy to identify variables that have constant values.
| Alias | unique |
|---|---|
| Default | true |
groupRareLevels=true | false
when set to True, uses the variable screening policy to identify nominal variables that have rare levels.
| Alias | groupRare |
|---|---|
| Default | true |
leakagePercentThreshold=double
specifies the variable screening policy for variables that have a very high level of information about the target. Variables that have a greater target entropy percentage reduction than the specified threshold are flagged as leakage variables.
| Alias | leakagePercentageThreshold |
|---|---|
| Default | 90 |
| Range | (0–100] |
lowCv=true | false
when set to True, uses the variable screening policy to identify variables that have a low coefficient of variation (CV).
| Alias | lowCoefficientVariation |
|---|---|
| Default | true |
lowMutualInformation=double
specifies the variable screening policy for variables that have a low level of information about the target.
| Alias | lowInformation |
|---|---|
| Default | 0.05 |
| Minimum value | 0 |
missingIndicatorPercent=double
specifies the variable screening policy for generating missing indicator variables.
| Alias | missingIndicatorPercentage |
|---|---|
| Default | 75 |
| Range | [10–100) |
missingPercentThreshold=double
specifies the variable screening policy for identifying variables that have a very high missing rate.
| Alias | missingPercentageThreshold |
|---|---|
| Default | 90 |
| Range | [10–100) |
redundant=double
specifies the symmetric uncertainty (SU) threshold for identifying redundant variables. If the SU for two variables exceeds the threshold, the variable that has less information about the target is flagged as redundant.
| Default | 1 |
|---|---|
| Range | (0–1] |
seed=integer
specifies a seed value for random number generation. This value is used for repeatable random number generation in some scenarios.
| Default | 0 |
|---|
* table={castable}
specifies the table name, caslib, and other common parameters.
| Long form | table={name="table-name"} |
|---|---|
| Shortcut form | table="table-name" |
The castable value can be one or more of the following:
caslib="string"
specifies the caslib for the input table that you want to use with the action. By default, the active caslib is used. Specify a value only if you need to access a table from a different caslib.
computedOnDemand=true | false
when set to True, creates the computed variables when the table is loaded instead of when the action begins.
| Alias | compOnDemand |
|---|---|
| Default | false |
computedVars={{casinvardesc-1} <, {casinvardesc-2}, ...>}
specifies the names of the computed variables to create. Specify an expression for each variable in the computedVarsProgram parameter. If you do not specify this parameter, then all variables from computedVarsProgram are automatically included.
| Alias | compVars |
|---|
For more information about specifying the computedVars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
computedVarsProgram="string"
specifies an expression for each computed variable that you include in the computedVars parameter.
| Alias | compPgm |
|---|
dataSourceOptions={key-1=any-list-or-data-type-1 <, key-2=any-list-or-data-type-2, ...>}
specifies data source options.
| Alias | options, dataSource |
|---|
importOptions={fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}
specifies the settings for reading a table from a data source.
| Alias | import |
|---|
For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).
* name="table-name"
specifies the name of the input table.
singlePass=true | false
when set to True, does not create a transient table on the server. Setting this parameter to True can be efficient, but the data might not have stable ordering upon repeated runs.
| Default | false |
|---|
vars={{casinvardesc-1} <, {casinvardesc-2}, ...>}
specifies the variables to use in the action.
For more information about specifying the vars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
where="where-expression"
specifies an expression for subsetting the input data.
whereTable={groupbytable}
specifies an input table that contains rows to use as a WHERE filter. If the vars parameter is not specified, then all the variable names that are common to the input table and the filtering table are used to find matching rows. If the where parameter for the input table and this parameter are specified, then this filtering table is applied first.
The groupbytable value can be one or more of the following:
casLib="string"
specifies the caslib for the filter table. By default, the active caslib is used.
dataSourceOptions={adls_noreq-parameters | bigquery-parameters | clouddex-parameters | db2-parameters | dnfs-parameters | esp-parameters | hadoop-parameters | hana-parameters | impala-parameters | jdbc-parameters | mongodb-parameters | mysql-parameters | odbc-parameters | oracle-parameters | path-parameters | postgres-parameters | redshift-parameters | s3-parameters | sforce-parameters | snowflake-parameters | spark-parameters | spde-parameters | sqlserver-parameters | teradata-parameters | vertica-parameters | yellowbrick-parameters}
specifies data source options.
| Alias | options, dataSource |
|---|
For more information about specifying the dataSourceOptions parameter, see the common dataSourceOptions parameter (Appendix A: Common Parameters).
importOptions={fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}
specifies the settings for reading a table from a data source.
| Alias | import |
|---|
For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).
* name="table-name"
specifies the name of the filter table.
vars={{casinvardesc-1} <, {casinvardesc-2}, ...>}
specifies the variable names to use from the filter table.
For more information about specifying the vars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
where="where-expression"
specifies an expression for subsetting the data from the filter table.
* target="variable-name"
specifies the target variable.
| Alias | evalVar |
|---|
* transformationOut={casouttable}
specifies the CAS table to store the feature transformation and generation pipelines.
| Long form | transformationOut={name="table-name"} |
|---|---|
| Shortcut form | transformationOut="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars={"variable-name-1" <, "variable-name-2", ...>}
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=true | false
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | false |
|---|
replace=true | false
when set to True, overwrites an existing table that has the same name.
| Default | false |
|---|
transformationPolicy={transformationSpace}
specifies the feature transformation and generation space in which the feature machine operates.
| Alias | ftgPolicy |
|---|
The transformationSpace value can be one or more of the following:
cardinality=true | false
when set to True, includes cardinality-reducing transformations.
| Default | true |
|---|
entropy=true | false
when set to True, includes transformations for the treatment of low entropy.
| Default | false |
|---|
interaction=true | false
when set to True, detects and generates interaction features.
| Default | false |
|---|
iqv=true | false
when set to True, includes transformations for the treatment of low indices of qualitative variation (IQV).
| Default | false |
|---|
kurtosis=true | false
when set to True, includes transformations for the treatment of high kurtosis.
| Default | false |
|---|
missing=true | false
when set to True, includes transformations for the treatment of missing values.
| Default | true |
|---|
outlier=true | false
when set to True, includes transformations for the treatment of outliers.
| Default | false |
|---|
polynomial=true | false
when set to True, includes up to third-order polynomial transformations.
| Default | false |
|---|
skewness=true | false
when set to True, includes transformations for the treatment of high skewness.
| Default | true |
|---|
featureMachine Action
Automated feature transformation and generation engine.
Python Syntax
Summary: Input and Output Tables
If a row includes a subparameter, you can specify the name, caslib, and so on in the subparameter. Otherwise, you can specify the name, caslib, and so on in the parameter.
|
Parameter |
Subparameter |
Description |
|---|---|---|
|
required parametertable |
— |
specifies the table name, caslib, and other common parameters. |
|
Parameter |
Subparameter |
Description |
|---|---|---|
|
— |
specifies the CAS table to store the analysis results. | |
|
required parameterfeatureOut |
— |
specifies the CAS table to store the feature transformation and generation pipelines. |
|
— |
specifies the CAS table to store the feature transformation and generation model. | |
|
required parametertransformationOut |
— |
specifies the CAS table to store the feature transformation and generation pipelines. |
Parameter Descriptions
casout={casouttable}
specifies the CAS table to store the analysis results.
| Long form | casout={"name":"table-name"} |
|---|---|
| Shortcut form | casout="table-name" |
The casouttable value can be one or more of the following:
"caslib":"string"
specifies the name of the caslib for the output table.
"indexVars":["variable-name-1" <, "variable-name-2", ...>]
specifies the list of variables to create indexes for in the output data.
"lifetime":64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
"memoryFormat":"DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
"name":"table-name"
specifies the name for the output table.
"promote":True | False
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | False |
|---|
"replace":True | False
when set to True, overwrites an existing table that has the same name.
| Default | False |
|---|
copyVars=["variable-name-1" <, "variable-name-2", ...>]
specifies the names of variables to be copied to the output table.
distinctCountLimit=integer
specifies the distinct count limit. If the limit is exceeded, and the misraGries parameter is set to True, the Misra-Gries frequency sketch algorithm is used to estimate the frequency distribution. Otherwise, the distinct count operation is aborted.
| Default | 10000 |
|---|---|
| Minimum value | 256 |
ecdfTolerance=double
specifies the tolerance value for the empirical cumulative distribution function. This value is used by the quantile sketch algorithm.
| Default | 0.001 |
|---|---|
| Range | 0–0.1 |
event="string"
specifies the target variable level that you want to model. Multilevel classification problems are cast into a one-versus-all binary classification problem, where the value of the event parameter denotes the level that you are modeling.
explorationPolicy={avaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) policy.
| Alias | avaptPolicy |
|---|
The avaptPolicy value can be one or more of the following:
"cardinality":{cardinalityAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) cardinality policy.
The cardinalityAvaptPolicy value can be one or more of the following:
"lowMediumCutoff":double
specifies the cardinality threshold for the low-medium cutoff.
| Default | 32 |
|---|---|
| Range | 2–256 |
"mediumHighCutoff":double
specifies the cardinality threshold for the medium-high cutoff.
| Default | 64 |
|---|---|
| Range | 2–1024 |
"minNObsPerTargetLevel":double
specifies the minimum number of observations for each target level.
| Default | 10 |
|---|---|
| Range | 5–100 |
"cv":{cvAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) coefficient of variation policy.
| Alias | coefficientVariation |
|---|
The cvAvaptPolicy value can be one or more of the following:
"lowMoment":double
specifies the low-high percentage threshold for the moment coefficient of variation (CV).
| Default | 1E-05 |
|---|---|
| Range | 0–100 |
"lowRobust":double
specifies the low-high percentage threshold for the robust coefficient of variation (CV).
| Default | 0.001 |
|---|---|
| Range | 0–100 |
"dateTimeVariables":["variable-name-1" <, "variable-name-2", ...>]
specifies the datetime variables.
| Alias | dateTime |
|---|
"dateVariables":["variable-name-1" <, "variable-name-2", ...>]
specifies the date variables.
| Alias | date |
|---|
"entropy":{entropyAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) entropy policy.
The entropyAvaptPolicy value can be one or more of the following:
"giniLowMediumCutoff":double
specifies the Gini entropy threshold for the low-medium cutoff.
| Default | 0.25 |
|---|---|
| Range | 0–1 |
"giniMediumHighCutoff":double
specifies the Gini entropy threshold for the medium-high cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–1 |
"shannonLowMediumCutoff":double
specifies the Shannon entropy threshold for the low-medium cutoff.
| Default | 0.25 |
|---|---|
| Range | 0–1 |
"shannonMediumHighCutoff":double
specifies the Shannon entropy threshold for the medium-high cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–1 |
"iqv":{iqvAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) index of qualitative variation policy.
| Alias | qualitativeVariationIndex |
|---|
The iqvAvaptPolicy value can be one or more of the following:
"highTopBottom":double
specifies the low-high cutoff frequency ratio threshold between the most frequent and least frequent levels of a nominal variable.
| Alias | highTop1Bottom1 |
|---|---|
| Default | 100 |
| Minimum value | 1 |
"highTopTwo":double
specifies the low-high cutoff frequency ratio threshold between the most frequent and second most frequent levels of a nominal variable.
| Alias | highTop1Top2 |
|---|---|
| Default | 10 |
| Minimum value | 1 |
"highVariationRatio":double
specifies the variation ratio threshold for the low-high cutoff.
| Alias | highModVr |
|---|---|
| Default | 0.5 |
| Range | (0–1] |
"kurtosis":{kurtosisAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) kurtosis policy.
The kurtosisAvaptPolicy value can be one or more of the following:
"momentLowMediumCutoff":double
specifies the moment kurtosis threshold for the low-medium cutoff.
| Default | 5 |
|---|---|
| Range | 0–100 |
"momentMediumHighCutoff":double
specifies the moment kurtosis threshold for the medium-high cutoff.
| Default | 10 |
|---|---|
| Range | 0–100 |
"robustLowMediumCutoff":double
specifies the robust kurtosis threshold for the low-medium cutoff.
| Default | 2 |
|---|---|
| Range | 0–100 |
"robustMediumHighCutoff":double
specifies the robust kurtosis threshold for the medium-high cutoff.
| Default | 3 |
|---|---|
| Range | 0–100 |
"missing":{missingAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) missing grouping policy.
The missingAvaptPolicy value can be one or more of the following:
"lowMediumCutoff":double
specifies the missing percentage threshold for the low-medium cutoff.
| Default | 5 |
|---|---|
| Range | 0–100 |
"mediumHighCutoff":double
specifies the missing percentage threshold for the medium-high cutoff.
| Default | 25 |
|---|---|
| Range | 0–100 |
"nominal":{nominalAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) nominal policy.
The nominalAvaptPolicy value can be one or more of the following:
"cardinalityRatio":double
specifies the AVAPT nominal policy cardinality ratio threshold.
| Default | 0.25 |
|---|---|
| Range | (0–1] |
"cardinalityThreshold":double
specifies the AVAPT nominal policy cardinality threshold.
| Default | 1024 |
|---|---|
| Minimum value | 32 |
"includeNegative":True | False
when set to True, includes numeric variables with some negative values in the nominal analysis.
| Default | False |
|---|
"includeNonIntegral":True | False
when set to True, includes numeric variables with some nonintegral values in the nominal analysis.
| Default | False |
|---|
"intervals":["variable-name-1" <, "variable-name-2", ...>]
specifies variables to consider as intervals.
"nominals":["variable-name-1" <, "variable-name-2", ...>]
specifies variables to consider as nominals.
"outlier":{outlierAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) outlier policy.
The outlierAvaptPolicy value can be one or more of the following:
"momentLowMediumCutoff":double
specifies the z-score outlier percentage threshold for the low-medium cutoff.
| Default | 1 |
|---|---|
| Range | 0–100 |
"momentMediumHighCutoff":double
specifies the z-score outlier percentage threshold for the medium-high cutoff.
| Default | 2.5 |
|---|---|
| Range | 0–100 |
"robustLowMediumCutoff":double
specifies the modified interquartile range outlier percentage threshold for the low-medium cutoff.
| Default | 1 |
|---|---|
| Range | 0–100 |
"robustMediumHighCutoff":double
specifies the modified interquartile range outlier percentage threshold for the medium-high cutoff.
| Default | 2.5 |
|---|---|
| Range | 0–100 |
"skewness":{skewnessAvaptPolicy}
specifies the automatic variable analysis and grouping (AVAPT) skewness policy.
The skewnessAvaptPolicy value can be one or more of the following:
"momentLowMediumCutoff":double
specifies the moment skewness threshold for the low-medium cutoff.
| Default | 2 |
|---|---|
| Range | 0–100 |
"momentMediumHighCutoff":double
specifies the moment skewness threshold for the medium-high cutoff.
| Default | 10 |
|---|---|
| Range | 0–100 |
"robustLowMediumCutoff":double
specifies the robust skewness threshold for the low-medium cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–3 |
"robustMediumHighCutoff":double
specifies the robust skewness threshold for the medium-high cutoff.
| Default | 2 |
|---|---|
| Range | 0–3 |
"timeVariables":["variable-name-1" <, "variable-name-2", ...>]
specifies the time variables.
| Alias | time |
|---|
* featureOut={casouttable}
specifies the CAS table to store the feature transformation and generation pipelines.
| Long form | featureOut={"name":"table-name"} |
|---|---|
| Shortcut form | featureOut="table-name" |
The casouttable value can be one or more of the following:
"caslib":"string"
specifies the name of the caslib for the output table.
"indexVars":["variable-name-1" <, "variable-name-2", ...>]
specifies the list of variables to create indexes for in the output data.
"lifetime":64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
"memoryFormat":"DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
"name":"table-name"
specifies the name for the output table.
"promote":True | False
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | False |
|---|
"replace":True | False
when set to True, overwrites an existing table that has the same name.
| Default | False |
|---|
freq="variable-name"
specifies the frequency variable.
inputs=[{casinvardesc-1} <, {casinvardesc-2}, ...>]
specifies the variables to use for the analysis. You can specify a subset of the variables from the input table.
For more information about specifying the inputs parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
| Alias | vars |
|---|
misraGries=True | False
when set to True, uses the Misra-Gries algorithm for the frequency distribution estimation, if the distinct count limit is exceeded.
| Default | True |
|---|
rankPolicy={rankPolicy}
specifies the feature ranking policy.
| Alias | rank |
|---|
| Long form | rankPolicy={"intervalStat":"AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"} |
|---|---|
| Shortcut form | rankPolicy="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU" |
The rankPolicy value can be one or more of the following:
"intervalStat":"AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"
specifies the interval variable ranking statistic.
| Alias | interval |
|---|---|
| Default | SU |
"nominalStat":"CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU"
specifies the nominal variable ranking statistic.
| Alias | nominal |
|---|---|
| Default | SU |
"rankMissingIndicators":True | False
when set to True, missing indicator features take part in the feature ranking. Otherwise, they are excluded from the ranking, and hence will always appear in the final feature set.
| Default | True |
|---|
"separateRank":True | False
when set to True, performs a separate feature ranking for interval and nominal features.
| Alias | separate |
|---|---|
| Default | True |
"topKInteractions":integer
specifies the number of top-ranked interaction features to generate and save.
| Alias | topKInteract |
|---|---|
| Minimum value | 1 |
"topKSave":integer
specifies the number of features per variable to save.
| Alias | topK |
|---|---|
| Default | 1 |
| Minimum value | 1 |
saveState={casouttable}
specifies the CAS table to store the feature transformation and generation model.
| Alias | saveModel |
|---|
| Long form | saveState={"name":"table-name"} |
|---|---|
| Shortcut form | saveState="table-name" |
The casouttable value can be one or more of the following:
"caslib":"string"
specifies the name of the caslib for the output table.
"indexVars":["variable-name-1" <, "variable-name-2", ...>]
specifies the list of variables to create indexes for in the output data.
"lifetime":64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
"memoryFormat":"DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
"name":"table-name"
specifies the name for the output table.
"promote":True | False
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | False |
|---|
"replace":True | False
when set to True, overwrites an existing table that has the same name.
| Default | False |
|---|
screenPolicy={sweeperPolicy}
specifies the variable screening policy to use for recommending that variables be screened out, transformed, or copied.
| Alias | sweeperPolicy |
|---|
The sweeperPolicy value can be one or more of the following:
"constant":True | False
when set to True, uses the variable screening policy to identify variables that have constant values.
| Alias | unique |
|---|---|
| Default | True |
"groupRareLevels":True | False
when set to True, uses the variable screening policy to identify nominal variables that have rare levels.
| Alias | groupRare |
|---|---|
| Default | True |
"leakagePercentThreshold":double
specifies the variable screening policy for variables that have a very high level of information about the target. Variables that have a greater target entropy percentage reduction than the specified threshold are flagged as leakage variables.
| Alias | leakagePercentageThreshold |
|---|---|
| Default | 90 |
| Range | (0–100] |
"lowCv":True | False
when set to True, uses the variable screening policy to identify variables that have a low coefficient of variation (CV).
| Alias | lowCoefficientVariation |
|---|---|
| Default | True |
"lowMutualInformation":double
specifies the variable screening policy for variables that have a low level of information about the target.
| Alias | lowInformation |
|---|---|
| Default | 0.05 |
| Minimum value | 0 |
"missingIndicatorPercent":double
specifies the variable screening policy for generating missing indicator variables.
| Alias | missingIndicatorPercentage |
|---|---|
| Default | 75 |
| Range | [10–100) |
"missingPercentThreshold":double
specifies the variable screening policy for identifying variables that have a very high missing rate.
| Alias | missingPercentageThreshold |
|---|---|
| Default | 90 |
| Range | [10–100) |
"redundant":double
specifies the symmetric uncertainty (SU) threshold for identifying redundant variables. If the SU for two variables exceeds the threshold, the variable that has less information about the target is flagged as redundant.
| Default | 1 |
|---|---|
| Range | (0–1] |
seed=integer
specifies a seed value for random number generation. This value is used for repeatable random number generation in some scenarios.
| Default | 0 |
|---|
* table={castable}
specifies the table name, caslib, and other common parameters.
| Long form | table={"name":"table-name"} |
|---|---|
| Shortcut form | table="table-name" |
The castable value can be one or more of the following:
"caslib":"string"
specifies the caslib for the input table that you want to use with the action. By default, the active caslib is used. Specify a value only if you need to access a table from a different caslib.
"computedOnDemand":True | False
when set to True, creates the computed variables when the table is loaded instead of when the action begins.
| Alias | compOnDemand |
|---|---|
| Default | False |
"computedVars":[{casinvardesc-1} <, {casinvardesc-2}, ...>]
specifies the names of the computed variables to create. Specify an expression for each variable in the computedVarsProgram parameter. If you do not specify this parameter, then all variables from computedVarsProgram are automatically included.
| Alias | compVars |
|---|
For more information about specifying the computedVars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
"computedVarsProgram":"string"
specifies an expression for each computed variable that you include in the computedVars parameter.
| Alias | compPgm |
|---|
"dataSourceOptions":{"key-1":{any-list-or-data-type-1} <, "key-2":{any-list-or-data-type-2}, ...>}
specifies data source options.
| Alias | options, dataSource |
|---|
"importOptions":{"fileType":"ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}
specifies the settings for reading a table from a data source.
| Alias | import_ |
|---|
For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).
* "name":"table-name"
specifies the name of the input table.
"singlePass":True | False
when set to True, does not create a transient table on the server. Setting this parameter to True can be efficient, but the data might not have stable ordering upon repeated runs.
| Default | False |
|---|
"vars":[{casinvardesc-1} <, {casinvardesc-2}, ...>]
specifies the variables to use in the action.
For more information about specifying the vars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
"where":"where-expression"
specifies an expression for subsetting the input data.
"whereTable":{groupbytable}
specifies an input table that contains rows to use as a WHERE filter. If the vars parameter is not specified, then all the variable names that are common to the input table and the filtering table are used to find matching rows. If the where parameter for the input table and this parameter are specified, then this filtering table is applied first.
The groupbytable value can be one or more of the following:
"casLib":"string"
specifies the caslib for the filter table. By default, the active caslib is used.
"dataSourceOptions":{adls_noreq-parameters | bigquery-parameters | clouddex-parameters | db2-parameters | dnfs-parameters | esp-parameters | hadoop-parameters | hana-parameters | impala-parameters | jdbc-parameters | mongodb-parameters | mysql-parameters | odbc-parameters | oracle-parameters | path-parameters | postgres-parameters | redshift-parameters | s3-parameters | sforce-parameters | snowflake-parameters | spark-parameters | spde-parameters | sqlserver-parameters | teradata-parameters | vertica-parameters | yellowbrick-parameters}
specifies data source options.
| Alias | options, dataSource |
|---|
For more information about specifying the dataSourceOptions parameter, see the common dataSourceOptions parameter (Appendix A: Common Parameters).
"importOptions":{"fileType":"ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}
specifies the settings for reading a table from a data source.
| Alias | import_ |
|---|
For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).
* "name":"table-name"
specifies the name of the filter table.
"vars":[{casinvardesc-1} <, {casinvardesc-2}, ...>]
specifies the variable names to use from the filter table.
For more information about specifying the vars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
"where":"where-expression"
specifies an expression for subsetting the data from the filter table.
* target="variable-name"
specifies the target variable.
| Alias | evalVar |
|---|
* transformationOut={casouttable}
specifies the CAS table to store the feature transformation and generation pipelines.
| Long form | transformationOut={"name":"table-name"} |
|---|---|
| Shortcut form | transformationOut="table-name" |
The casouttable value can be one or more of the following:
"caslib":"string"
specifies the name of the caslib for the output table.
"indexVars":["variable-name-1" <, "variable-name-2", ...>]
specifies the list of variables to create indexes for in the output data.
"lifetime":64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
"memoryFormat":"DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
"name":"table-name"
specifies the name for the output table.
"promote":True | False
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | False |
|---|
"replace":True | False
when set to True, overwrites an existing table that has the same name.
| Default | False |
|---|
transformationPolicy={transformationSpace}
specifies the feature transformation and generation space in which the feature machine operates.
| Alias | ftgPolicy |
|---|
The transformationSpace value can be one or more of the following:
"cardinality":True | False
when set to True, includes cardinality-reducing transformations.
| Default | True |
|---|
"entropy":True | False
when set to True, includes transformations for the treatment of low entropy.
| Default | False |
|---|
"interaction":True | False
when set to True, detects and generates interaction features.
| Default | False |
|---|
"iqv":True | False
when set to True, includes transformations for the treatment of low indices of qualitative variation (IQV).
| Default | False |
|---|
"kurtosis":True | False
when set to True, includes transformations for the treatment of high kurtosis.
| Default | False |
|---|
"missing":True | False
when set to True, includes transformations for the treatment of missing values.
| Default | True |
|---|
"outlier":True | False
when set to True, includes transformations for the treatment of outliers.
| Default | False |
|---|
"polynomial":True | False
when set to True, includes up to third-order polynomial transformations.
| Default | False |
|---|
"skewness":True | False
when set to True, includes transformations for the treatment of high skewness.
| Default | True |
|---|
featureMachine Action
Automated feature transformation and generation engine.
R Syntax
Summary: Input and Output Tables
If a row includes a subparameter, you can specify the name, caslib, and so on in the subparameter. Otherwise, you can specify the name, caslib, and so on in the parameter.
|
Parameter |
Subparameter |
Description |
|---|---|---|
|
required parametertable |
— |
specifies the table name, caslib, and other common parameters. |
|
Parameter |
Subparameter |
Description |
|---|---|---|
|
— |
specifies the CAS table to store the analysis results. | |
|
required parameterfeatureOut |
— |
specifies the CAS table to store the feature transformation and generation pipelines. |
|
— |
specifies the CAS table to store the feature transformation and generation model. | |
|
required parametertransformationOut |
— |
specifies the CAS table to store the feature transformation and generation pipelines. |
Parameter Descriptions
casout=list(casouttable)
specifies the CAS table to store the analysis results.
| Long form | casout=list(name="table-name") |
|---|---|
| Shortcut form | casout="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars=list("variable-name-1" <, "variable-name-2", ...>)
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=TRUE | FALSE
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | FALSE |
|---|
replace=TRUE | FALSE
when set to True, overwrites an existing table that has the same name.
| Default | FALSE |
|---|
copyVars=list("variable-name-1" <, "variable-name-2", ...>)
specifies the names of variables to be copied to the output table.
distinctCountLimit=integer
specifies the distinct count limit. If the limit is exceeded, and the misraGries parameter is set to True, the Misra-Gries frequency sketch algorithm is used to estimate the frequency distribution. Otherwise, the distinct count operation is aborted.
| Default | 10000 |
|---|---|
| Minimum value | 256 |
ecdfTolerance=double
specifies the tolerance value for the empirical cumulative distribution function. This value is used by the quantile sketch algorithm.
| Default | 0.001 |
|---|---|
| Range | 0–0.1 |
event="string"
specifies the target variable level that you want to model. Multilevel classification problems are cast into a one-versus-all binary classification problem, where the value of the event parameter denotes the level that you are modeling.
explorationPolicy=list(avaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) policy.
| Alias | avaptPolicy |
|---|
The avaptPolicy value can be one or more of the following:
cardinality=list(cardinalityAvaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) cardinality policy.
The cardinalityAvaptPolicy value can be one or more of the following:
lowMediumCutoff=double
specifies the cardinality threshold for the low-medium cutoff.
| Default | 32 |
|---|---|
| Range | 2–256 |
mediumHighCutoff=double
specifies the cardinality threshold for the medium-high cutoff.
| Default | 64 |
|---|---|
| Range | 2–1024 |
minNObsPerTargetLevel=double
specifies the minimum number of observations for each target level.
| Default | 10 |
|---|---|
| Range | 5–100 |
cv=list(cvAvaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) coefficient of variation policy.
| Alias | coefficientVariation |
|---|
The cvAvaptPolicy value can be one or more of the following:
lowMoment=double
specifies the low-high percentage threshold for the moment coefficient of variation (CV).
| Default | 1E-05 |
|---|---|
| Range | 0–100 |
lowRobust=double
specifies the low-high percentage threshold for the robust coefficient of variation (CV).
| Default | 0.001 |
|---|---|
| Range | 0–100 |
dateTimeVariables=list("variable-name-1" <, "variable-name-2", ...>)
specifies the datetime variables.
| Alias | dateTime |
|---|
dateVariables=list("variable-name-1" <, "variable-name-2", ...>)
specifies the date variables.
| Alias | date |
|---|
entropy=list(entropyAvaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) entropy policy.
The entropyAvaptPolicy value can be one or more of the following:
giniLowMediumCutoff=double
specifies the Gini entropy threshold for the low-medium cutoff.
| Default | 0.25 |
|---|---|
| Range | 0–1 |
giniMediumHighCutoff=double
specifies the Gini entropy threshold for the medium-high cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–1 |
shannonLowMediumCutoff=double
specifies the Shannon entropy threshold for the low-medium cutoff.
| Default | 0.25 |
|---|---|
| Range | 0–1 |
shannonMediumHighCutoff=double
specifies the Shannon entropy threshold for the medium-high cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–1 |
iqv=list(iqvAvaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) index of qualitative variation policy.
| Alias | qualitativeVariationIndex |
|---|
The iqvAvaptPolicy value can be one or more of the following:
highTopBottom=double
specifies the low-high cutoff frequency ratio threshold between the most frequent and least frequent levels of a nominal variable.
| Alias | highTop1Bottom1 |
|---|---|
| Default | 100 |
| Minimum value | 1 |
highTopTwo=double
specifies the low-high cutoff frequency ratio threshold between the most frequent and second most frequent levels of a nominal variable.
| Alias | highTop1Top2 |
|---|---|
| Default | 10 |
| Minimum value | 1 |
highVariationRatio=double
specifies the variation ratio threshold for the low-high cutoff.
| Alias | highModVr |
|---|---|
| Default | 0.5 |
| Range | (0–1] |
kurtosis=list(kurtosisAvaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) kurtosis policy.
The kurtosisAvaptPolicy value can be one or more of the following:
momentLowMediumCutoff=double
specifies the moment kurtosis threshold for the low-medium cutoff.
| Default | 5 |
|---|---|
| Range | 0–100 |
momentMediumHighCutoff=double
specifies the moment kurtosis threshold for the medium-high cutoff.
| Default | 10 |
|---|---|
| Range | 0–100 |
robustLowMediumCutoff=double
specifies the robust kurtosis threshold for the low-medium cutoff.
| Default | 2 |
|---|---|
| Range | 0–100 |
robustMediumHighCutoff=double
specifies the robust kurtosis threshold for the medium-high cutoff.
| Default | 3 |
|---|---|
| Range | 0–100 |
missing=list(missingAvaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) missing grouping policy.
The missingAvaptPolicy value can be one or more of the following:
lowMediumCutoff=double
specifies the missing percentage threshold for the low-medium cutoff.
| Default | 5 |
|---|---|
| Range | 0–100 |
mediumHighCutoff=double
specifies the missing percentage threshold for the medium-high cutoff.
| Default | 25 |
|---|---|
| Range | 0–100 |
nominal=list(nominalAvaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) nominal policy.
The nominalAvaptPolicy value can be one or more of the following:
cardinalityRatio=double
specifies the AVAPT nominal policy cardinality ratio threshold.
| Default | 0.25 |
|---|---|
| Range | (0–1] |
cardinalityThreshold=double
specifies the AVAPT nominal policy cardinality threshold.
| Default | 1024 |
|---|---|
| Minimum value | 32 |
includeNegative=TRUE | FALSE
when set to True, includes numeric variables with some negative values in the nominal analysis.
| Default | FALSE |
|---|
includeNonIntegral=TRUE | FALSE
when set to True, includes numeric variables with some nonintegral values in the nominal analysis.
| Default | FALSE |
|---|
intervals=list("variable-name-1" <, "variable-name-2", ...>)
specifies variables to consider as intervals.
nominals=list("variable-name-1" <, "variable-name-2", ...>)
specifies variables to consider as nominals.
outlier=list(outlierAvaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) outlier policy.
The outlierAvaptPolicy value can be one or more of the following:
momentLowMediumCutoff=double
specifies the z-score outlier percentage threshold for the low-medium cutoff.
| Default | 1 |
|---|---|
| Range | 0–100 |
momentMediumHighCutoff=double
specifies the z-score outlier percentage threshold for the medium-high cutoff.
| Default | 2.5 |
|---|---|
| Range | 0–100 |
robustLowMediumCutoff=double
specifies the modified interquartile range outlier percentage threshold for the low-medium cutoff.
| Default | 1 |
|---|---|
| Range | 0–100 |
robustMediumHighCutoff=double
specifies the modified interquartile range outlier percentage threshold for the medium-high cutoff.
| Default | 2.5 |
|---|---|
| Range | 0–100 |
skewness=list(skewnessAvaptPolicy)
specifies the automatic variable analysis and grouping (AVAPT) skewness policy.
The skewnessAvaptPolicy value can be one or more of the following:
momentLowMediumCutoff=double
specifies the moment skewness threshold for the low-medium cutoff.
| Default | 2 |
|---|---|
| Range | 0–100 |
momentMediumHighCutoff=double
specifies the moment skewness threshold for the medium-high cutoff.
| Default | 10 |
|---|---|
| Range | 0–100 |
robustLowMediumCutoff=double
specifies the robust skewness threshold for the low-medium cutoff.
| Default | 0.75 |
|---|---|
| Range | 0–3 |
robustMediumHighCutoff=double
specifies the robust skewness threshold for the medium-high cutoff.
| Default | 2 |
|---|---|
| Range | 0–3 |
timeVariables=list("variable-name-1" <, "variable-name-2", ...>)
specifies the time variables.
| Alias | time |
|---|
* featureOut=list(casouttable)
specifies the CAS table to store the feature transformation and generation pipelines.
| Long form | featureOut=list(name="table-name") |
|---|---|
| Shortcut form | featureOut="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars=list("variable-name-1" <, "variable-name-2", ...>)
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=TRUE | FALSE
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | FALSE |
|---|
replace=TRUE | FALSE
when set to True, overwrites an existing table that has the same name.
| Default | FALSE |
|---|
freq="variable-name"
specifies the frequency variable.
inputs=list( list(casinvardesc-1) <, list(casinvardesc-2), ...>)
specifies the variables to use for the analysis. You can specify a subset of the variables from the input table.
For more information about specifying the inputs parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
| Alias | vars |
|---|
misraGries=TRUE | FALSE
when set to True, uses the Misra-Gries algorithm for the frequency distribution estimation, if the distinct count limit is exceeded.
| Default | TRUE |
|---|
rankPolicy=list(rankPolicy)
specifies the feature ranking policy.
| Alias | rank |
|---|
| Long form | rankPolicy=list(intervalStat="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU") |
|---|---|
| Shortcut form | rankPolicy="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU" |
The rankPolicy value can be one or more of the following:
intervalStat="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"
specifies the interval variable ranking statistic.
| Alias | interval |
|---|---|
| Default | SU |
nominalStat="CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU"
specifies the nominal variable ranking statistic.
| Alias | nominal |
|---|---|
| Default | SU |
rankMissingIndicators=TRUE | FALSE
when set to True, missing indicator features take part in the feature ranking. Otherwise, they are excluded from the ranking, and hence will always appear in the final feature set.
| Default | TRUE |
|---|
separateRank=TRUE | FALSE
when set to True, performs a separate feature ranking for interval and nominal features.
| Alias | separate |
|---|---|
| Default | TRUE |
topKInteractions=integer
specifies the number of top-ranked interaction features to generate and save.
| Alias | topKInteract |
|---|---|
| Minimum value | 1 |
topKSave=integer
specifies the number of features per variable to save.
| Alias | topK |
|---|---|
| Default | 1 |
| Minimum value | 1 |
saveState=list(casouttable)
specifies the CAS table to store the feature transformation and generation model.
| Alias | saveModel |
|---|
| Long form | saveState=list(name="table-name") |
|---|---|
| Shortcut form | saveState="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars=list("variable-name-1" <, "variable-name-2", ...>)
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=TRUE | FALSE
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | FALSE |
|---|
replace=TRUE | FALSE
when set to True, overwrites an existing table that has the same name.
| Default | FALSE |
|---|
screenPolicy=list(sweeperPolicy)
specifies the variable screening policy to use for recommending that variables be screened out, transformed, or copied.
| Alias | sweeperPolicy |
|---|
The sweeperPolicy value can be one or more of the following:
constant=TRUE | FALSE
when set to True, uses the variable screening policy to identify variables that have constant values.
| Alias | unique |
|---|---|
| Default | TRUE |
groupRareLevels=TRUE | FALSE
when set to True, uses the variable screening policy to identify nominal variables that have rare levels.
| Alias | groupRare |
|---|---|
| Default | TRUE |
leakagePercentThreshold=double
specifies the variable screening policy for variables that have a very high level of information about the target. Variables that have a greater target entropy percentage reduction than the specified threshold are flagged as leakage variables.
| Alias | leakagePercentageThreshold |
|---|---|
| Default | 90 |
| Range | (0–100] |
lowCv=TRUE | FALSE
when set to True, uses the variable screening policy to identify variables that have a low coefficient of variation (CV).
| Alias | lowCoefficientVariation |
|---|---|
| Default | TRUE |
lowMutualInformation=double
specifies the variable screening policy for variables that have a low level of information about the target.
| Alias | lowInformation |
|---|---|
| Default | 0.05 |
| Minimum value | 0 |
missingIndicatorPercent=double
specifies the variable screening policy for generating missing indicator variables.
| Alias | missingIndicatorPercentage |
|---|---|
| Default | 75 |
| Range | [10–100) |
missingPercentThreshold=double
specifies the variable screening policy for identifying variables that have a very high missing rate.
| Alias | missingPercentageThreshold |
|---|---|
| Default | 90 |
| Range | [10–100) |
redundant=double
specifies the symmetric uncertainty (SU) threshold for identifying redundant variables. If the SU for two variables exceeds the threshold, the variable that has less information about the target is flagged as redundant.
| Default | 1 |
|---|---|
| Range | (0–1] |
seed=integer
specifies a seed value for random number generation. This value is used for repeatable random number generation in some scenarios.
| Default | 0 |
|---|
* table=list(castable)
specifies the table name, caslib, and other common parameters.
| Long form | table=list(name="table-name") |
|---|---|
| Shortcut form | table="table-name" |
The castable value can be one or more of the following:
caslib="string"
specifies the caslib for the input table that you want to use with the action. By default, the active caslib is used. Specify a value only if you need to access a table from a different caslib.
computedOnDemand=TRUE | FALSE
when set to True, creates the computed variables when the table is loaded instead of when the action begins.
| Alias | compOnDemand |
|---|---|
| Default | FALSE |
computedVars=list( list(casinvardesc-1) <, list(casinvardesc-2), ...>)
specifies the names of the computed variables to create. Specify an expression for each variable in the computedVarsProgram parameter. If you do not specify this parameter, then all variables from computedVarsProgram are automatically included.
| Alias | compVars |
|---|
For more information about specifying the computedVars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
computedVarsProgram="string"
specifies an expression for each computed variable that you include in the computedVars parameter.
| Alias | compPgm |
|---|
dataSourceOptions=list(key-1=list(any-list-or-data-type-1) <, key-2=list(any-list-or-data-type-2), ...>)
specifies data source options.
| Alias | options, dataSource |
|---|
importOptions=list(fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters)
specifies the settings for reading a table from a data source.
| Alias | import |
|---|
For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).
* name="table-name"
specifies the name of the input table.
singlePass=TRUE | FALSE
when set to True, does not create a transient table on the server. Setting this parameter to True can be efficient, but the data might not have stable ordering upon repeated runs.
| Default | FALSE |
|---|
vars=list( list(casinvardesc-1) <, list(casinvardesc-2), ...>)
specifies the variables to use in the action.
For more information about specifying the vars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
where="where-expression"
specifies an expression for subsetting the input data.
whereTable=list(groupbytable)
specifies an input table that contains rows to use as a WHERE filter. If the vars parameter is not specified, then all the variable names that are common to the input table and the filtering table are used to find matching rows. If the where parameter for the input table and this parameter are specified, then this filtering table is applied first.
The groupbytable value can be one or more of the following:
casLib="string"
specifies the caslib for the filter table. By default, the active caslib is used.
dataSourceOptions=list(adls_noreq-parameters | bigquery-parameters | clouddex-parameters | db2-parameters | dnfs-parameters | esp-parameters | hadoop-parameters | hana-parameters | impala-parameters | jdbc-parameters | mongodb-parameters | mysql-parameters | odbc-parameters | oracle-parameters | path-parameters | postgres-parameters | redshift-parameters | s3-parameters | sforce-parameters | snowflake-parameters | spark-parameters | spde-parameters | sqlserver-parameters | teradata-parameters | vertica-parameters | yellowbrick-parameters)
specifies data source options.
| Alias | options, dataSource |
|---|
For more information about specifying the dataSourceOptions parameter, see the common dataSourceOptions parameter (Appendix A: Common Parameters).
importOptions=list(fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters)
specifies the settings for reading a table from a data source.
| Alias | import |
|---|
For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).
* name="table-name"
specifies the name of the filter table.
vars=list( list(casinvardesc-1) <, list(casinvardesc-2), ...>)
specifies the variable names to use from the filter table.
For more information about specifying the vars parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).
where="where-expression"
specifies an expression for subsetting the data from the filter table.
* target="variable-name"
specifies the target variable.
| Alias | evalVar |
|---|
* transformationOut=list(casouttable)
specifies the CAS table to store the feature transformation and generation pipelines.
| Long form | transformationOut=list(name="table-name") |
|---|---|
| Shortcut form | transformationOut="table-name" |
The casouttable value can be one or more of the following:
caslib="string"
specifies the name of the caslib for the output table.
indexVars=list("variable-name-1" <, "variable-name-2", ...>)
specifies the list of variables to create indexes for in the output data.
lifetime=64-bit-integer
specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.
| Default | 0 |
|---|---|
| Minimum value | 0 |
memoryFormat="DVR" | "INHERIT" | "STANDARD"
specifies the memory format for the output table.
| Default | INHERIT |
|---|
DVR
use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.
name="table-name"
specifies the name for the output table.
promote=TRUE | FALSE
when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.
| Default | FALSE |
|---|
replace=TRUE | FALSE
when set to True, overwrites an existing table that has the same name.
| Default | FALSE |
|---|
transformationPolicy=list(transformationSpace)
specifies the feature transformation and generation space in which the feature machine operates.
| Alias | ftgPolicy |
|---|
The transformationSpace value can be one or more of the following:
cardinality=TRUE | FALSE
when set to True, includes cardinality-reducing transformations.
| Default | TRUE |
|---|
entropy=TRUE | FALSE
when set to True, includes transformations for the treatment of low entropy.
| Default | FALSE |
|---|
interaction=TRUE | FALSE
when set to True, detects and generates interaction features.
| Default | FALSE |
|---|
iqv=TRUE | FALSE
when set to True, includes transformations for the treatment of low indices of qualitative variation (IQV).
| Default | FALSE |
|---|
kurtosis=TRUE | FALSE
when set to True, includes transformations for the treatment of high kurtosis.
| Default | FALSE |
|---|
missing=TRUE | FALSE
when set to True, includes transformations for the treatment of missing values.
| Default | TRUE |
|---|
outlier=TRUE | FALSE
when set to True, includes transformations for the treatment of outliers.
| Default | FALSE |
|---|
polynomial=TRUE | FALSE
when set to True, includes up to third-order polynomial transformations.
| Default | FALSE |
|---|
skewness=TRUE | FALSE
when set to True, includes transformations for the treatment of high skewness.
| Default | TRUE |
|---|