Data Science Pilot Action Set

name="table-name",

promote=TRUE | FALSE,

replace=TRUE | FALSE,

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

transformationPolicy={

cardinality=TRUE | FALSE,

entropy=TRUE | FALSE,

interaction=TRUE | FALSE,

iqv=TRUE | FALSE,

kurtosis=TRUE | FALSE,

missing=TRUE | FALSE,

outlier=TRUE | FALSE,

polynomial=TRUE | FALSE,

skewness=TRUE | FALSE

weight="variable-name"

;

indicates a required parameter

Summary: Input and Output Tables

If a row includes a subparameter, you can specify the name, caslib, and so on in the subparameter. Otherwise, you can specify the name, caslib, and so on in the parameter.

Parameters for Reading Input Tables
Parameter	Subparameter	Description
required parametertable	—	specifies the table name, caslib, and other common parameters.

Parameters for Creating Output Tables
Parameter	Subparameter	Description
casout	—	specifies the CAS table to store the analysis results.
required parameterfeatureOut	—	specifies the CAS table to store the feature transformation and generation pipelines.
saveState	—	specifies the CAS table to store the feature transformation and generation model.
required parametertransformationOut	—	specifies the CAS table to store the feature transformation and generation pipelines.

Parameter Descriptions

casout={casouttable}

specifies the CAS table to store the analysis results.

Long form	casout={name="table-name"}
Shortcut form	casout="table-name"

The casouttable value can be one or more of the following:

caslib="string"

specifies the name of the caslib for the output table.

indexVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the list of variables to create indexes for in the output data.

lifetime=64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

memoryFormat="DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

use the default memory format that is set for the server. By default, the server uses the standard memory format. If an administrator sets the CAS_DEFAULT_MEMORY_FORMAT environment variable to DVR, then the DVR memory format is set as the default for the server.

STANDARD

use the standard memory format.

name="table-name"

specifies the name for the output table.

promote=TRUE | FALSE

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	FALSE

replace=TRUE | FALSE

when set to True, overwrites an existing table that has the same name.

Default	FALSE

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

copyVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the names of variables to be copied to the output table.

distinctCountLimit=integer

specifies the distinct count limit. If the limit is exceeded, and the misraGries parameter is set to True, the Misra-Gries frequency sketch algorithm is used to estimate the frequency distribution. Otherwise, the distinct count operation is aborted.

Default	10000
Minimum value	256

ecdfTolerance=double

specifies the tolerance value for the empirical cumulative distribution function. This value is used by the quantile sketch algorithm.

Default	0.001
Range	1E-06–0.1

event="string"

specifies the target variable level that you want to model. Multilevel classification problems are cast into a one-versus-all binary classification problem, where the value of the event parameter denotes the level that you are modeling.

explorationPolicy={avaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) policy.

Alias	avaptPolicy

The avaptPolicy value can be one or more of the following:

cardinality={cardinalityAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) cardinality policy.

The cardinalityAvaptPolicy value can be one or more of the following:

lowMediumCutoff=double

specifies the cardinality threshold for the low-medium cutoff.

Default	32
Range	2–256

mediumHighCutoff=double

specifies the cardinality threshold for the medium-high cutoff.

Default	64
Range	2–1024

minNObsPerTargetLevel=double

specifies the minimum number of observations for each target level.

Default	10
Range	5–100

cv={cvAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) coefficient of variation policy.

Alias	coefficientVariation

The cvAvaptPolicy value can be one or more of the following:

lowMoment=double

specifies the absolute value of the low-high percentage threshold for the moment coefficient of variation (CV).

Default	1
Minimum value	0

lowRobust=double

specifies the absolute value of the low-high percentage threshold for the robust coefficient of variation (CV).

Default	1
Minimum value	0

dateTimeVariables={"variable-name-1" <, "variable-name-2", ...>}

specifies the datetime variables.

Alias	dateTime

dateVariables={"variable-name-1" <, "variable-name-2", ...>}

specifies the date variables.

Alias	date

entropy={entropyAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) entropy policy.

The entropyAvaptPolicy value can be one or more of the following:

giniLowMediumCutoff=double

specifies the Gini entropy threshold for the low-medium cutoff.

Default	0.25
Range	0–1

giniMediumHighCutoff=double

specifies the Gini entropy threshold for the medium-high cutoff.

Default	0.75
Range	0–1

shannonLowMediumCutoff=double

specifies the Shannon entropy threshold for the low-medium cutoff.

Default	0.25
Range	0–1

shannonMediumHighCutoff=double

specifies the Shannon entropy threshold for the medium-high cutoff.

Default	0.75
Range	0–1

iqv={iqvAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) index of qualitative variation policy.

Alias	qualitativeVariationIndex

The iqvAvaptPolicy value can be one or more of the following:

highTopBottom=double

specifies the low-high cutoff frequency ratio threshold between the most frequent and least frequent levels of a nominal variable.

Alias	highTop1Bottom1
Default	100
Minimum value	1

highTopTwo=double

specifies the low-high cutoff frequency ratio threshold between the most frequent and second most frequent levels of a nominal variable.

Alias	highTop1Top2
Default	10
Minimum value	1

highVariationRatio=double

specifies the variation ratio threshold for the low-high cutoff.

Alias	highModVr
Default	0.5
Range	(0–1]

kurtosis={kurtosisAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) kurtosis policy.

The kurtosisAvaptPolicy value can be one or more of the following:

momentLowMediumCutoff=double

specifies the absolute value of the moment kurtosis threshold for the low-medium cutoff.

Default	5
Minimum value	0

momentMediumHighCutoff=double

specifies the absolute value of the moment kurtosis threshold for the medium-high cutoff.

Default	10
Minimum value	0

robustLowMediumCutoff=double

specifies the absolute value of the robust kurtosis threshold for the low-medium cutoff.

Default	2
Minimum value	0

robustMediumHighCutoff=double

specifies the absolute value of the robust kurtosis threshold for the medium-high cutoff.

Default	3
Minimum value	0

missing={missingAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) missing grouping policy.

The missingAvaptPolicy value can be one or more of the following:

lowMediumCutoff=double

specifies the missing percentage threshold for the low-medium cutoff.

Default	5
Range	0–100

mediumHighCutoff=double

specifies the missing percentage threshold for the medium-high cutoff.

Default	25
Range	0–100

nominal={nominalAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) nominal policy.

The nominalAvaptPolicy value can be one or more of the following:

cardinalityRatio=double

specifies the AVAPT nominal policy cardinality ratio threshold.

Default	0.25
Range	(0–1]

cardinalityThreshold=double

specifies the AVAPT nominal policy cardinality threshold.

Default	1024
Minimum value	32

includeNegative=TRUE | FALSE

when set to True, includes numeric variables with some negative values in the nominal analysis.

Default	FALSE

includeNonIntegral=TRUE | FALSE

when set to True, includes numeric variables with some nonintegral values in the nominal analysis.

Default	FALSE

intervals={"variable-name-1" <, "variable-name-2", ...>}

specifies variables to consider as intervals.

nominals={"variable-name-1" <, "variable-name-2", ...>}

specifies variables to consider as nominals.

outlier={outlierAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) outlier policy.

The outlierAvaptPolicy value can be one or more of the following:

momentLowMediumCutoff=double

specifies the z-score outlier percentage threshold for the low-medium cutoff.

Default	1
Range	0–100

momentMediumHighCutoff=double

specifies the z-score outlier percentage threshold for the medium-high cutoff.

Default	2.5
Range	0–100

robustLowMediumCutoff=double

specifies the modified interquartile range outlier percentage threshold for the low-medium cutoff.

Default	1
Range	0–100

robustMediumHighCutoff=double

specifies the modified interquartile range outlier percentage threshold for the medium-high cutoff.

Default	2.5
Range	0–100

skewness={skewnessAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) skewness policy.

The skewnessAvaptPolicy value can be one or more of the following:

momentLowMediumCutoff=double

specifies the moment skewness threshold for the low-medium cutoff.

Default	2
Range	0–100

momentMediumHighCutoff=double

specifies the moment skewness threshold for the medium-high cutoff.

Default	10
Range	0–100

robustLowMediumCutoff=double

specifies the robust skewness threshold for the low-medium cutoff.

Default	0.75
Range	0–3

robustMediumHighCutoff=double

specifies the robust skewness threshold for the medium-high cutoff.

Default	2
Range	0–3

timeVariables={"variable-name-1" <, "variable-name-2", ...>}

specifies the time variables.

Alias	time

* featureOut={casouttable}

specifies the CAS table to store the feature transformation and generation pipelines.

Long form	featureOut={name="table-name"}
Shortcut form	featureOut="table-name"

The casouttable value can be one or more of the following:

caslib="string"

specifies the name of the caslib for the output table.

indexVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the list of variables to create indexes for in the output data.

lifetime=64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

memoryFormat="DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

name="table-name"

specifies the name for the output table.

promote=TRUE | FALSE

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	FALSE

replace=TRUE | FALSE

when set to True, overwrites an existing table that has the same name.

Default	FALSE

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

freq="variable-name"

specifies the frequency variable.

inputs={{casinvardesc-1} <, {casinvardesc-2}, ...>}

specifies the variables to use for the analysis. You can specify a subset of the variables from the input table.

For more information about specifying the inputs parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).

Alias	vars

misraGries=TRUE | FALSE

when set to True, uses the Misra-Gries algorithm for the frequency distribution estimation, if the distinct count limit is exceeded.

Default	TRUE

rankPolicy={rankPolicy}

specifies the feature ranking policy.

Alias	rank

Long form	rankPolicy={intervalStat="AVGQUANKURT" \| "AVGQUANSKEW" \| "CLASSICALKURT" \| "CLASSICALSKEW" \| "ENTROPY" \| "MI" \| "NORMMI" \| "PEARSON" \| "SU"}
Shortcut form	rankPolicy="AVGQUANKURT" \| "AVGQUANSKEW" \| "CLASSICALKURT" \| "CLASSICALSKEW" \| "ENTROPY" \| "MI" \| "NORMMI" \| "PEARSON" \| "SU"

The rankPolicy value can be one or more of the following:

intervalStat="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"

specifies the interval variable ranking statistic.

Alias	interval
Default	SU

AVGQUANKURT

uses the average quantile (robust) kurtosis.

AVGQUANSKEW

uses the average quantile (robust) skewness.

CLASSICALKURT

uses the classical (moment) kurtosis.

CLASSICALSKEW

uses the classical (moment) skewness.

ENTROPY

uses the entropy percentage decrease.

MI

uses the mutual information statistic.

NORMMI

uses the normalized mutual information statistic.

PEARSON

uses the Pearson correlation.

SU

uses the symmetric uncertainty statistic.

nominalStat="CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU"

specifies the nominal variable ranking statistic.

Alias	nominal
Default	SU

CHISQ

uses the chi-square statistic.

CRAMERSV

uses Cramer's V.

ENTROPY

uses the entropy percentage decrease.

FTEST

uses the F test.

G2

uses the G2 statistic.

IV

uses the information value statistic.

MI

uses the mutual information statistic.

NORMMI

uses the normalized mutual information statistic.

SU

uses the symmetric uncertainty statistic.

rankMissingIndicators=TRUE | FALSE

when set to True, missing indicator features take part in the feature ranking. Otherwise, they are excluded from the ranking, and hence will always appear in the final feature set.

Default	TRUE

separateRank=TRUE | FALSE

when set to True, performs a separate feature ranking for interval and nominal features.

Alias	separate
Default	TRUE

topKInteractions=integer

specifies the number of top-ranked interaction features to generate and save.

Alias	topKInteract
Minimum value	1

topKSave=integer

specifies the number of features per variable to save.

Alias	topK
Default	1
Minimum value	1

saveState={casouttable}

specifies the CAS table to store the feature transformation and generation model.

Alias	saveModel

Long form	saveState={name="table-name"}
Shortcut form	saveState="table-name"

The casouttable value can be one or more of the following:

caslib="string"

specifies the name of the caslib for the output table.

indexVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the list of variables to create indexes for in the output data.

lifetime=64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

memoryFormat="DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

name="table-name"

specifies the name for the output table.

promote=TRUE | FALSE

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	FALSE

replace=TRUE | FALSE

when set to True, overwrites an existing table that has the same name.

Default	FALSE

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

screenPolicy={sweeperPolicy}

specifies the variable screening policy to use for recommending that variables be screened out, transformed, or copied.

Alias	sweeperPolicy

The sweeperPolicy value can be one or more of the following:

constant=TRUE | FALSE

when set to True, uses the variable screening policy to identify variables that have constant values.

Alias	unique
Default	TRUE

groupRareLevels=TRUE | FALSE

when set to True, uses the variable screening policy to identify nominal variables that have rare levels.

Alias	groupRare
Default	TRUE

leakagePercentThreshold=double

specifies the variable screening policy for variables that have a very high level of information about the target. Variables that have a greater target entropy percentage reduction than the specified threshold are flagged as leakage variables.

Alias	leakagePercentageThreshold
Default	90
Range	(0–100]

lowCv=TRUE | FALSE

when set to True, uses the variable screening policy to identify variables that have a low coefficient of variation (CV).

Alias	lowCoefficientVariation
Default	TRUE

lowMutualInformation=double

specifies the variable screening policy for variables that have a low level of information about the target.

Alias	lowInformation
Default	0.05
Minimum value	0

missingIndicatorPercent=double

specifies the variable screening policy for generating missing indicator variables.

Alias	missingIndicatorPercentage
Default	75
Range	[10–100)

missingPercentThreshold=double

specifies the variable screening policy for identifying variables that have a very high missing rate.

Alias	missingPercentageThreshold
Default	90
Range	[10–100)

redundant=double

specifies the symmetric uncertainty (SU) threshold for identifying redundant variables. If the SU for two variables exceeds the threshold, the variable that has less information about the target is flagged as redundant.

Default	1
Range	(0–1]

seed=integer

specifies a seed value for random number generation. This value is used for repeatable random number generation in some scenarios.

Default	0

* table={castable}

specifies the table name, caslib, and other common parameters.

Long form	table={name="table-name"}
Shortcut form	table="table-name"

The castable value can be one or more of the following:

caslib="string"

specifies the caslib for the input table that you want to use with the action. By default, the active caslib is used. Specify a value only if you need to access a table from a different caslib.

computedOnDemand=TRUE | FALSE

when set to True, creates the computed variables when the table is loaded instead of when the action begins.

Alias	compOnDemand
Default	FALSE

computedVars={{casinvardesc-1} <, {casinvardesc-2}, ...>}

specifies the names of the computed variables to create. Specify an expression for each variable in the computedVarsProgram parameter. If you do not specify this parameter, then all variables from computedVarsProgram are automatically included.

Alias	compVars

The casinvardesc value can be one or more of the following:

format="string"

specifies the format to apply to the variable.

formattedLength=integer

specifies the length of the format field plus the length of the format precision.

label="string"

specifies the descriptive label for the variable.

* name="variable-name"

specifies the name for the variable.

nfd=integer

specifies the length of the format precision.

nfl=integer

specifies the length of the format field.

computedVarsProgram="string"

specifies an expression for each computed variable that you include in the computedVars parameter.

Alias	compPgm

dataSourceOptions={key-1=any-list-or-data-type-1 <, key-2=any-list-or-data-type-2, ...>}

specifies data source options.

Aliases	options
Aliases	dataSource

importOptions={fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}

specifies the settings for reading a table from a data source.

Alias	import

For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).

* name="table-name"

specifies the name of the input table.

singlePass=TRUE | FALSE

when set to True, does not create a transient table on the server. Setting this parameter to True can be efficient, but the data might not have stable ordering upon repeated runs.

Default	FALSE

vars={{casinvardesc-1} <, {casinvardesc-2}, ...>}

specifies the variables to use in the action.

The casinvardesc value can be one or more of the following:

format="string"

specifies the format to apply to the variable.

formattedLength=integer

specifies the length of the format field plus the length of the format precision.

label="string"

specifies the descriptive label for the variable.

* name="variable-name"

specifies the name for the variable.

nfd=integer

specifies the length of the format precision.

nfl=integer

specifies the length of the format field.

where="where-expression"

specifies an expression for subsetting the input data.

whereTable={groupbytable}

specifies an input table that contains rows to use as a WHERE filter. If the vars parameter is not specified, then all the variable names that are common to the input table and the filtering table are used to find matching rows. If the where parameter for the input table and this parameter are specified, then this filtering table is applied first.

The groupbytable value can be one or more of the following:

casLib="string"

specifies the caslib for the filter table. By default, the active caslib is used.

dataSourceOptions={adls_noreq-parameters | bigquery-parameters | cas_noreq-parameters | clouddex-parameters | db2-parameters | dnfs-parameters | esp-parameters | fedsvr-parameters | gcs_noreq-parameters | hadoop-parameters | hana-parameters | impala-parameters | informix-parameters | jdbc-parameters | mongodb-parameters | mysql-parameters | odbc-parameters | oracle-parameters | path-parameters | postgres-parameters | redshift-parameters | s3-parameters | sapiq-parameters | sforce-parameters | singlestore_standard-parameters | snowflake-parameters | spark-parameters | spde-parameters | sqlserver-parameters | ss_noreq-parameters | teradata-parameters | vertica-parameters | yellowbrick-parameters}

specifies data source options.

Aliases	options
Aliases	dataSource

For more information about specifying the dataSourceOptions parameter, see the common dataSourceOptions parameter (Appendix A: Common Parameters).

importOptions={fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}

specifies the settings for reading a table from a data source.

Alias	import

For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).

* name="table-name"

specifies the name of the filter table.

vars={{casinvardesc-1} <, {casinvardesc-2}, ...>}

specifies the variable names to use from the filter table.

The casinvardesc value can be one or more of the following:

format="string"

specifies the format to apply to the variable.

formattedLength=integer

specifies the length of the format field plus the length of the format precision.

label="string"

specifies the descriptive label for the variable.

* name="variable-name"

specifies the name for the variable.

nfd=integer

specifies the length of the format precision.

nfl=integer

specifies the length of the format field.

where="where-expression"

specifies an expression for subsetting the data from the filter table.

* target="variable-name"

specifies the target variable.

Alias	evalVar

* transformationOut={casouttable}

specifies the CAS table to store the feature transformation and generation pipelines.

Long form	transformationOut={name="table-name"}
Shortcut form	transformationOut="table-name"

The casouttable value can be one or more of the following:

caslib="string"

specifies the name of the caslib for the output table.

indexVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the list of variables to create indexes for in the output data.

lifetime=64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

memoryFormat="DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

name="table-name"

specifies the name for the output table.

promote=TRUE | FALSE

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	FALSE

replace=TRUE | FALSE

when set to True, overwrites an existing table that has the same name.

Default	FALSE

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

transformationPolicy={transformationSpace}

specifies the feature transformation and generation space in which the feature machine operates.

Alias	ftgPolicy

The transformationSpace value can be one or more of the following:

cardinality=TRUE | FALSE

when set to True, includes cardinality-reducing transformations.

Default	TRUE

entropy=TRUE | FALSE

when set to True, includes transformations for the treatment of low entropy.

Default	FALSE

interaction=TRUE | FALSE

when set to True, detects and generates interaction features.

Default	FALSE

iqv=TRUE | FALSE

when set to True, includes transformations for the treatment of low indices of qualitative variation (IQV).

Default	FALSE

kurtosis=TRUE | FALSE

when set to True, includes transformations for the treatment of high kurtosis.

Default	FALSE

missing=TRUE | FALSE

when set to True, includes transformations for the treatment of missing values.

Default	TRUE

outlier=TRUE | FALSE

when set to True, includes transformations for the treatment of outliers.

Default	FALSE

polynomial=TRUE | FALSE

when set to True, includes up to third-order polynomial transformations.

Default	FALSE

skewness=TRUE | FALSE

when set to True, includes transformations for the treatment of high skewness.

Default	TRUE

weight="variable-name"

specifies the weight variable.

featureMachine Action

Automated feature transformation and generation engine.

Lua Syntax
Summary: Input and Output Tables
Parameter Descriptions

Lua Syntax

results, info = s:dataSciencePilot_featureMachine{

casout={

caslib="string",

indexVars={"variable-name-1" <, "variable-name-2", ...>},

lifetime=64-bit-integer,

name="table-name",

promote=true | false,

replace=true | false,

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

copyVars={"variable-name-1" <, "variable-name-2", ...>},

distinctCountLimit=integer,

ecdfTolerance=double,

event="string",

explorationPolicy={

cardinality={

lowMediumCutoff=double

mediumHighCutoff=double

minNObsPerTargetLevel=double

cv={

lowMoment=double

lowRobust=double

dateTimeVariables={"variable-name-1" <, "variable-name-2", ...>},

dateVariables={"variable-name-1" <, "variable-name-2", ...>},

entropy={

giniLowMediumCutoff=double

giniMediumHighCutoff=double

shannonLowMediumCutoff=double

shannonMediumHighCutoff=double

iqv={

highTopBottom=double

highTopTwo=double

highVariationRatio=double

kurtosis={

missing={

lowMediumCutoff=double

mediumHighCutoff=double

nominal={

cardinalityRatio=double

cardinalityThreshold=double

includeNegative=true | false

includeNonIntegral=true | false

intervals={"variable-name-1" <, "variable-name-2", ...>}

nominals={"variable-name-1" <, "variable-name-2", ...>}

outlier={

skewness={

timeVariables={"variable-name-1" <, "variable-name-2", ...>}

featureOut={

caslib="string",

indexVars={"variable-name-1" <, "variable-name-2", ...>},

lifetime=64-bit-integer,

name="table-name",

promote=true | false,

replace=true | false,

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

freq="variable-name",

inputs={{

format="string",

formattedLength=integer,

label="string",

name="variable-name",

nfd=integer,

nfl=integer

}, {...}},

misraGries=true | false,

rankPolicy={

nominalStat="CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU",

rankMissingIndicators=true | false,

separateRank=true | false,

topKInteractions=integer,

topKSave=integer

saveState={

caslib="string",

indexVars={"variable-name-1" <, "variable-name-2", ...>},

lifetime=64-bit-integer,

name="table-name",

promote=true | false,

replace=true | false,

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

screenPolicy={

constant=true | false,

groupRareLevels=true | false,

leakagePercentThreshold=double,

lowCv=true | false,

lowMutualInformation=double,

missingIndicatorPercent=double,

missingPercentThreshold=double,

redundant=double

seed=integer,

table={

caslib="string",

computedOnDemand=true | false,

computedVars={{

format="string",

formattedLength=integer,

label="string",

name="variable-name",

nfd=integer,

nfl=integer

}, {...}},

computedVarsProgram="string",

dataSourceOptions={key-1=any-list-or-data-type-1 <, key-2=any-list-or-data-type-2, ...>},

name="table-name",

singlePass=true | false,

vars={{

format="string",

formattedLength=integer,

label="string",

name="variable-name",

nfd=integer,

nfl=integer

}, {...}},

where="where-expression",

whereTable={

casLib="string"

name="table-name"

vars={{

format="string",

formattedLength=integer,

label="string",

name="variable-name",

nfd=integer,

nfl=integer

}, {...}}

where="where-expression"

}

target="variable-name",

transformationOut={

caslib="string",

indexVars={"variable-name-1" <, "variable-name-2", ...>},

lifetime=64-bit-integer,

name="table-name",

promote=true | false,

replace=true | false,

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

transformationPolicy={

cardinality=true | false,

entropy=true | false,

interaction=true | false,

iqv=true | false,

kurtosis=true | false,

missing=true | false,

outlier=true | false,

polynomial=true | false,

skewness=true | false

weight="variable-name"

}

indicates a required parameter

Summary: Input and Output Tables

If a row includes a subparameter, you can specify the name, caslib, and so on in the subparameter. Otherwise, you can specify the name, caslib, and so on in the parameter.

Parameters for Reading Input Tables
Parameter	Subparameter	Description
required parametertable	—	specifies the table name, caslib, and other common parameters.

Parameters for Creating Output Tables
Parameter	Subparameter	Description
casout	—	specifies the CAS table to store the analysis results.
required parameterfeatureOut	—	specifies the CAS table to store the feature transformation and generation pipelines.
saveState	—	specifies the CAS table to store the feature transformation and generation model.
required parametertransformationOut	—	specifies the CAS table to store the feature transformation and generation pipelines.

Parameter Descriptions

casout={casouttable}

specifies the CAS table to store the analysis results.

Long form	casout={name="table-name"}
Shortcut form	casout="table-name"

The casouttable value can be one or more of the following:

caslib="string"

specifies the name of the caslib for the output table.

indexVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the list of variables to create indexes for in the output data.

lifetime=64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

memoryFormat="DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

name="table-name"

specifies the name for the output table.

promote=true | false

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	false

replace=true | false

when set to True, overwrites an existing table that has the same name.

Default	false

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

copyVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the names of variables to be copied to the output table.

distinctCountLimit=integer

Default	10000
Minimum value	256

ecdfTolerance=double

specifies the tolerance value for the empirical cumulative distribution function. This value is used by the quantile sketch algorithm.

Default	0.001
Range	1E-06–0.1

event="string"

explorationPolicy={avaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) policy.

Alias	avaptPolicy

The avaptPolicy value can be one or more of the following:

cardinality={cardinalityAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) cardinality policy.

The cardinalityAvaptPolicy value can be one or more of the following:

lowMediumCutoff=double

specifies the cardinality threshold for the low-medium cutoff.

Default	32
Range	2–256

mediumHighCutoff=double

specifies the cardinality threshold for the medium-high cutoff.

Default	64
Range	2–1024

minNObsPerTargetLevel=double

specifies the minimum number of observations for each target level.

Default	10
Range	5–100

cv={cvAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) coefficient of variation policy.

Alias	coefficientVariation

The cvAvaptPolicy value can be one or more of the following:

lowMoment=double

specifies the absolute value of the low-high percentage threshold for the moment coefficient of variation (CV).

Default	1
Minimum value	0

lowRobust=double

specifies the absolute value of the low-high percentage threshold for the robust coefficient of variation (CV).

Default	1
Minimum value	0

dateTimeVariables={"variable-name-1" <, "variable-name-2", ...>}

specifies the datetime variables.

Alias	dateTime

dateVariables={"variable-name-1" <, "variable-name-2", ...>}

specifies the date variables.

Alias	date

entropy={entropyAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) entropy policy.

The entropyAvaptPolicy value can be one or more of the following:

giniLowMediumCutoff=double

specifies the Gini entropy threshold for the low-medium cutoff.

Default	0.25
Range	0–1

giniMediumHighCutoff=double

specifies the Gini entropy threshold for the medium-high cutoff.

Default	0.75
Range	0–1

shannonLowMediumCutoff=double

specifies the Shannon entropy threshold for the low-medium cutoff.

Default	0.25
Range	0–1

shannonMediumHighCutoff=double

specifies the Shannon entropy threshold for the medium-high cutoff.

Default	0.75
Range	0–1

iqv={iqvAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) index of qualitative variation policy.

Alias	qualitativeVariationIndex

The iqvAvaptPolicy value can be one or more of the following:

highTopBottom=double

specifies the low-high cutoff frequency ratio threshold between the most frequent and least frequent levels of a nominal variable.

Alias	highTop1Bottom1
Default	100
Minimum value	1

highTopTwo=double

specifies the low-high cutoff frequency ratio threshold between the most frequent and second most frequent levels of a nominal variable.

Alias	highTop1Top2
Default	10
Minimum value	1

highVariationRatio=double

specifies the variation ratio threshold for the low-high cutoff.

Alias	highModVr
Default	0.5
Range	(0–1]

kurtosis={kurtosisAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) kurtosis policy.

The kurtosisAvaptPolicy value can be one or more of the following:

momentLowMediumCutoff=double

specifies the absolute value of the moment kurtosis threshold for the low-medium cutoff.

Default	5
Minimum value	0

momentMediumHighCutoff=double

specifies the absolute value of the moment kurtosis threshold for the medium-high cutoff.

Default	10
Minimum value	0

robustLowMediumCutoff=double

specifies the absolute value of the robust kurtosis threshold for the low-medium cutoff.

Default	2
Minimum value	0

robustMediumHighCutoff=double

specifies the absolute value of the robust kurtosis threshold for the medium-high cutoff.

Default	3
Minimum value	0

missing={missingAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) missing grouping policy.

The missingAvaptPolicy value can be one or more of the following:

lowMediumCutoff=double

specifies the missing percentage threshold for the low-medium cutoff.

Default	5
Range	0–100

mediumHighCutoff=double

specifies the missing percentage threshold for the medium-high cutoff.

Default	25
Range	0–100

nominal={nominalAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) nominal policy.

The nominalAvaptPolicy value can be one or more of the following:

cardinalityRatio=double

specifies the AVAPT nominal policy cardinality ratio threshold.

Default	0.25
Range	(0–1]

cardinalityThreshold=double

specifies the AVAPT nominal policy cardinality threshold.

Default	1024
Minimum value	32

includeNegative=true | false

when set to True, includes numeric variables with some negative values in the nominal analysis.

Default	false

includeNonIntegral=true | false

when set to True, includes numeric variables with some nonintegral values in the nominal analysis.

Default	false

intervals={"variable-name-1" <, "variable-name-2", ...>}

specifies variables to consider as intervals.

nominals={"variable-name-1" <, "variable-name-2", ...>}

specifies variables to consider as nominals.

outlier={outlierAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) outlier policy.

The outlierAvaptPolicy value can be one or more of the following:

momentLowMediumCutoff=double

specifies the z-score outlier percentage threshold for the low-medium cutoff.

Default	1
Range	0–100

momentMediumHighCutoff=double

specifies the z-score outlier percentage threshold for the medium-high cutoff.

Default	2.5
Range	0–100

robustLowMediumCutoff=double

specifies the modified interquartile range outlier percentage threshold for the low-medium cutoff.

Default	1
Range	0–100

robustMediumHighCutoff=double

specifies the modified interquartile range outlier percentage threshold for the medium-high cutoff.

Default	2.5
Range	0–100

skewness={skewnessAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) skewness policy.

The skewnessAvaptPolicy value can be one or more of the following:

momentLowMediumCutoff=double

specifies the moment skewness threshold for the low-medium cutoff.

Default	2
Range	0–100

momentMediumHighCutoff=double

specifies the moment skewness threshold for the medium-high cutoff.

Default	10
Range	0–100

robustLowMediumCutoff=double

specifies the robust skewness threshold for the low-medium cutoff.

Default	0.75
Range	0–3

robustMediumHighCutoff=double

specifies the robust skewness threshold for the medium-high cutoff.

Default	2
Range	0–3

timeVariables={"variable-name-1" <, "variable-name-2", ...>}

specifies the time variables.

Alias	time

* featureOut={casouttable}

specifies the CAS table to store the feature transformation and generation pipelines.

Long form	featureOut={name="table-name"}
Shortcut form	featureOut="table-name"

The casouttable value can be one or more of the following:

caslib="string"

specifies the name of the caslib for the output table.

indexVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the list of variables to create indexes for in the output data.

lifetime=64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

memoryFormat="DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

name="table-name"

specifies the name for the output table.

promote=true | false

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	false

replace=true | false

when set to True, overwrites an existing table that has the same name.

Default	false

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

freq="variable-name"

specifies the frequency variable.

inputs={{casinvardesc-1} <, {casinvardesc-2}, ...>}

specifies the variables to use for the analysis. You can specify a subset of the variables from the input table.

For more information about specifying the inputs parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).

Alias	vars

misraGries=true | false

when set to True, uses the Misra-Gries algorithm for the frequency distribution estimation, if the distinct count limit is exceeded.

Default	true

rankPolicy={rankPolicy}

specifies the feature ranking policy.

Alias	rank

Long form	rankPolicy={intervalStat="AVGQUANKURT" \| "AVGQUANSKEW" \| "CLASSICALKURT" \| "CLASSICALSKEW" \| "ENTROPY" \| "MI" \| "NORMMI" \| "PEARSON" \| "SU"}
Shortcut form	rankPolicy="AVGQUANKURT" \| "AVGQUANSKEW" \| "CLASSICALKURT" \| "CLASSICALSKEW" \| "ENTROPY" \| "MI" \| "NORMMI" \| "PEARSON" \| "SU"

The rankPolicy value can be one or more of the following:

intervalStat="AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"

specifies the interval variable ranking statistic.

Alias	interval
Default	SU

AVGQUANKURT

uses the average quantile (robust) kurtosis.

AVGQUANSKEW

uses the average quantile (robust) skewness.

CLASSICALKURT

uses the classical (moment) kurtosis.

CLASSICALSKEW

uses the classical (moment) skewness.

ENTROPY

uses the entropy percentage decrease.

MI

uses the mutual information statistic.

NORMMI

uses the normalized mutual information statistic.

PEARSON

uses the Pearson correlation.

SU

uses the symmetric uncertainty statistic.

nominalStat="CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU"

specifies the nominal variable ranking statistic.

Alias	nominal
Default	SU

CHISQ

uses the chi-square statistic.

CRAMERSV

uses Cramer's V.

ENTROPY

uses the entropy percentage decrease.

FTEST

uses the F test.

G2

uses the G2 statistic.

IV

uses the information value statistic.

MI

uses the mutual information statistic.

NORMMI

uses the normalized mutual information statistic.

SU

uses the symmetric uncertainty statistic.

rankMissingIndicators=true | false

when set to True, missing indicator features take part in the feature ranking. Otherwise, they are excluded from the ranking, and hence will always appear in the final feature set.

Default	true

separateRank=true | false

when set to True, performs a separate feature ranking for interval and nominal features.

Alias	separate
Default	true

topKInteractions=integer

specifies the number of top-ranked interaction features to generate and save.

Alias	topKInteract
Minimum value	1

topKSave=integer

specifies the number of features per variable to save.

Alias	topK
Default	1
Minimum value	1

saveState={casouttable}

specifies the CAS table to store the feature transformation and generation model.

Alias	saveModel

Long form	saveState={name="table-name"}
Shortcut form	saveState="table-name"

The casouttable value can be one or more of the following:

caslib="string"

specifies the name of the caslib for the output table.

indexVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the list of variables to create indexes for in the output data.

lifetime=64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

memoryFormat="DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

name="table-name"

specifies the name for the output table.

promote=true | false

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	false

replace=true | false

when set to True, overwrites an existing table that has the same name.

Default	false

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

screenPolicy={sweeperPolicy}

specifies the variable screening policy to use for recommending that variables be screened out, transformed, or copied.

Alias	sweeperPolicy

The sweeperPolicy value can be one or more of the following:

constant=true | false

when set to True, uses the variable screening policy to identify variables that have constant values.

Alias	unique
Default	true

groupRareLevels=true | false

when set to True, uses the variable screening policy to identify nominal variables that have rare levels.

Alias	groupRare
Default	true

leakagePercentThreshold=double

Alias	leakagePercentageThreshold
Default	90
Range	(0–100]

lowCv=true | false

when set to True, uses the variable screening policy to identify variables that have a low coefficient of variation (CV).

Alias	lowCoefficientVariation
Default	true

lowMutualInformation=double

specifies the variable screening policy for variables that have a low level of information about the target.

Alias	lowInformation
Default	0.05
Minimum value	0

missingIndicatorPercent=double

specifies the variable screening policy for generating missing indicator variables.

Alias	missingIndicatorPercentage
Default	75
Range	[10–100)

missingPercentThreshold=double

specifies the variable screening policy for identifying variables that have a very high missing rate.

Alias	missingPercentageThreshold
Default	90
Range	[10–100)

redundant=double

Default	1
Range	(0–1]

seed=integer

specifies a seed value for random number generation. This value is used for repeatable random number generation in some scenarios.

Default	0

* table={castable}

specifies the table name, caslib, and other common parameters.

Long form	table={name="table-name"}
Shortcut form	table="table-name"

The castable value can be one or more of the following:

caslib="string"

specifies the caslib for the input table that you want to use with the action. By default, the active caslib is used. Specify a value only if you need to access a table from a different caslib.

computedOnDemand=true | false

when set to True, creates the computed variables when the table is loaded instead of when the action begins.

Alias	compOnDemand
Default	false

computedVars={{casinvardesc-1} <, {casinvardesc-2}, ...>}

Alias	compVars

The casinvardesc value can be one or more of the following:

format="string"

specifies the format to apply to the variable.

formattedLength=integer

specifies the length of the format field plus the length of the format precision.

label="string"

specifies the descriptive label for the variable.

* name="variable-name"

specifies the name for the variable.

nfd=integer

specifies the length of the format precision.

nfl=integer

specifies the length of the format field.

computedVarsProgram="string"

specifies an expression for each computed variable that you include in the computedVars parameter.

Alias	compPgm

dataSourceOptions={key-1=any-list-or-data-type-1 <, key-2=any-list-or-data-type-2, ...>}

specifies data source options.

Aliases	options
Aliases	dataSource

importOptions={fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}

specifies the settings for reading a table from a data source.

Alias	import

For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).

* name="table-name"

specifies the name of the input table.

singlePass=true | false

when set to True, does not create a transient table on the server. Setting this parameter to True can be efficient, but the data might not have stable ordering upon repeated runs.

Default	false

vars={{casinvardesc-1} <, {casinvardesc-2}, ...>}

specifies the variables to use in the action.

The casinvardesc value can be one or more of the following:

format="string"

specifies the format to apply to the variable.

formattedLength=integer

specifies the length of the format field plus the length of the format precision.

label="string"

specifies the descriptive label for the variable.

* name="variable-name"

specifies the name for the variable.

nfd=integer

specifies the length of the format precision.

nfl=integer

specifies the length of the format field.

where="where-expression"

specifies an expression for subsetting the input data.

whereTable={groupbytable}

The groupbytable value can be one or more of the following:

casLib="string"

specifies the caslib for the filter table. By default, the active caslib is used.

dataSourceOptions={adls_noreq-parameters | bigquery-parameters | cas_noreq-parameters | clouddex-parameters | db2-parameters | dnfs-parameters | esp-parameters | fedsvr-parameters | gcs_noreq-parameters | hadoop-parameters | hana-parameters | impala-parameters | informix-parameters | jdbc-parameters | mongodb-parameters | mysql-parameters | odbc-parameters | oracle-parameters | path-parameters | postgres-parameters | redshift-parameters | s3-parameters | sapiq-parameters | sforce-parameters | singlestore_standard-parameters | snowflake-parameters | spark-parameters | spde-parameters | sqlserver-parameters | ss_noreq-parameters | teradata-parameters | vertica-parameters | yellowbrick-parameters}

specifies data source options.

Aliases	options
Aliases	dataSource

For more information about specifying the dataSourceOptions parameter, see the common dataSourceOptions parameter (Appendix A: Common Parameters).

importOptions={fileType="ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}

specifies the settings for reading a table from a data source.

Alias	import

For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).

* name="table-name"

specifies the name of the filter table.

vars={{casinvardesc-1} <, {casinvardesc-2}, ...>}

specifies the variable names to use from the filter table.

The casinvardesc value can be one or more of the following:

format="string"

specifies the format to apply to the variable.

formattedLength=integer

specifies the length of the format field plus the length of the format precision.

label="string"

specifies the descriptive label for the variable.

* name="variable-name"

specifies the name for the variable.

nfd=integer

specifies the length of the format precision.

nfl=integer

specifies the length of the format field.

where="where-expression"

specifies an expression for subsetting the data from the filter table.

* target="variable-name"

specifies the target variable.

Alias	evalVar

* transformationOut={casouttable}

specifies the CAS table to store the feature transformation and generation pipelines.

Long form	transformationOut={name="table-name"}
Shortcut form	transformationOut="table-name"

The casouttable value can be one or more of the following:

caslib="string"

specifies the name of the caslib for the output table.

indexVars={"variable-name-1" <, "variable-name-2", ...>}

specifies the list of variables to create indexes for in the output data.

lifetime=64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

memoryFormat="DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

name="table-name"

specifies the name for the output table.

promote=true | false

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	false

replace=true | false

when set to True, overwrites an existing table that has the same name.

Default	false

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

transformationPolicy={transformationSpace}

specifies the feature transformation and generation space in which the feature machine operates.

Alias	ftgPolicy

The transformationSpace value can be one or more of the following:

cardinality=true | false

when set to True, includes cardinality-reducing transformations.

Default	true

entropy=true | false

when set to True, includes transformations for the treatment of low entropy.

Default	false

interaction=true | false

when set to True, detects and generates interaction features.

Default	false

iqv=true | false

when set to True, includes transformations for the treatment of low indices of qualitative variation (IQV).

Default	false

kurtosis=true | false

when set to True, includes transformations for the treatment of high kurtosis.

Default	false

missing=true | false

when set to True, includes transformations for the treatment of missing values.

Default	true

outlier=true | false

when set to True, includes transformations for the treatment of outliers.

Default	false

polynomial=true | false

when set to True, includes up to third-order polynomial transformations.

Default	false

skewness=true | false

when set to True, includes transformations for the treatment of high skewness.

Default	true

weight="variable-name"

specifies the weight variable.

featureMachine Action

Automated feature transformation and generation engine.

Python Syntax
Summary: Input and Output Tables
Parameter Descriptions

Python Syntax

results=s.dataSciencePilot.featureMachine(

casout={

"caslib":"string",

"indexVars":["variable-name-1" <, "variable-name-2", ...>],

"lifetime":64-bit-integer,

"name":"table-name",

"promote":True | False,

"replace":True | False,

"tableRedistUpPolicy":"DEFER" | "NOREDIST" | "REBALANCE"

copyVars=["variable-name-1" <, "variable-name-2", ...>],

distinctCountLimit=integer,

ecdfTolerance=double,

event="string",

explorationPolicy={

"cardinality":{

"lowMediumCutoff":double

"mediumHighCutoff":double

"minNObsPerTargetLevel":double

"cv":{

"lowMoment":double

"lowRobust":double

"dateTimeVariables":["variable-name-1" <, "variable-name-2", ...>],

"dateVariables":["variable-name-1" <, "variable-name-2", ...>],

"entropy":{

"giniLowMediumCutoff":double

"giniMediumHighCutoff":double

"shannonLowMediumCutoff":double

"shannonMediumHighCutoff":double

"iqv":{

"highTopBottom":double

"highTopTwo":double

"highVariationRatio":double

"kurtosis":{

"momentLowMediumCutoff":double

"momentMediumHighCutoff":double

"robustLowMediumCutoff":double

"robustMediumHighCutoff":double

"missing":{

"lowMediumCutoff":double

"mediumHighCutoff":double

"nominal":{

"cardinalityRatio":double

"cardinalityThreshold":double

"includeNegative":True | False

"includeNonIntegral":True | False

"intervals":["variable-name-1" <, "variable-name-2", ...>]

"nominals":["variable-name-1" <, "variable-name-2", ...>]

"outlier":{

"momentLowMediumCutoff":double

"momentMediumHighCutoff":double

"robustLowMediumCutoff":double

"robustMediumHighCutoff":double

"skewness":{

"momentLowMediumCutoff":double

"momentMediumHighCutoff":double

"robustLowMediumCutoff":double

"robustMediumHighCutoff":double

"timeVariables":["variable-name-1" <, "variable-name-2", ...>]

featureOut={

"caslib":"string",

"indexVars":["variable-name-1" <, "variable-name-2", ...>],

"lifetime":64-bit-integer,

"name":"table-name",

"promote":True | False,

"replace":True | False,

"tableRedistUpPolicy":"DEFER" | "NOREDIST" | "REBALANCE"

freq="variable-name",

inputs=[{

"format":"string",

"formattedLength":integer,

"label":"string",

"name":"variable-name",

"nfd":integer,

"nfl":integer

}<, {...}>],

misraGries=True | False,

rankPolicy={

"nominalStat":"CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU",

"rankMissingIndicators":True | False,

"separateRank":True | False,

"topKInteractions":integer,

"topKSave":integer

saveState={

"caslib":"string",

"indexVars":["variable-name-1" <, "variable-name-2", ...>],

"lifetime":64-bit-integer,

"name":"table-name",

"promote":True | False,

"replace":True | False,

"tableRedistUpPolicy":"DEFER" | "NOREDIST" | "REBALANCE"

screenPolicy={

"constant":True | False,

"groupRareLevels":True | False,

"leakagePercentThreshold":double,

"lowCv":True | False,

"lowMutualInformation":double,

"missingIndicatorPercent":double,

"missingPercentThreshold":double,

"redundant":double

seed=integer,

table={

"caslib":"string",

"computedOnDemand":True | False,

"computedVars":[{

"format":"string",

"formattedLength":integer,

"label":"string",

"name":"variable-name",

"nfd":integer,

"nfl":integer

}<, {...}>],

"computedVarsProgram":"string",

"dataSourceOptions":{"key-1":{any-list-or-data-type-1} <, "key-2":{any-list-or-data-type-2}, ...>},

"importOptions":{"fileType":"ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DELIMITED" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SOUND" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters},

"name":"table-name",

"singlePass":True | False,

"vars":[{

"format":"string",

"formattedLength":integer,

"label":"string",

"name":"variable-name",

"nfd":integer,

"nfl":integer

}<, {...}>],

"where":"where-expression",

"whereTable":{

"casLib":"string"

"dataSourceOptions":{adls_noreq-parameters | bigquery-parameters | cas_noreq-parameters | clouddex-parameters | db2-parameters | dnfs-parameters | esp-parameters | fedsvr-parameters | gcs_noreq-parameters | hadoop-parameters | hana-parameters | impala-parameters | informix-parameters | jdbc-parameters | mongodb-parameters | mysql-parameters | odbc-parameters | oracle-parameters | path-parameters | postgres-parameters | redshift-parameters | s3-parameters | sapiq-parameters | sforce-parameters | singlestore_standard-parameters | snowflake-parameters | spark-parameters | spde-parameters | sqlserver-parameters | ss_noreq-parameters | teradata-parameters | vertica-parameters | yellowbrick-parameters}

"name":"table-name"

"vars":[{

"format":"string",

"formattedLength":integer,

"label":"string",

"name":"variable-name",

"nfd":integer,

"nfl":integer

}<, {...}>]

"where":"where-expression"

}

target="variable-name",

transformationOut={

"caslib":"string",

"indexVars":["variable-name-1" <, "variable-name-2", ...>],

"lifetime":64-bit-integer,

"name":"table-name",

"promote":True | False,

"replace":True | False,

"tableRedistUpPolicy":"DEFER" | "NOREDIST" | "REBALANCE"

transformationPolicy={

"cardinality":True | False,

"entropy":True | False,

"interaction":True | False,

"iqv":True | False,

"kurtosis":True | False,

"missing":True | False,

"outlier":True | False,

"polynomial":True | False,

"skewness":True | False

weight="variable-name"

)

indicates a required parameter

Summary: Input and Output Tables

If a row includes a subparameter, you can specify the name, caslib, and so on in the subparameter. Otherwise, you can specify the name, caslib, and so on in the parameter.

Parameters for Reading Input Tables
Parameter	Subparameter	Description
required parametertable	—	specifies the table name, caslib, and other common parameters.

Parameters for Creating Output Tables
Parameter	Subparameter	Description
casout	—	specifies the CAS table to store the analysis results.
required parameterfeatureOut	—	specifies the CAS table to store the feature transformation and generation pipelines.
saveState	—	specifies the CAS table to store the feature transformation and generation model.
required parametertransformationOut	—	specifies the CAS table to store the feature transformation and generation pipelines.

Parameter Descriptions

casout={casouttable}

specifies the CAS table to store the analysis results.

Long form	casout={"name":"table-name"}
Shortcut form	casout="table-name"

The casouttable value can be one or more of the following:

"caslib":"string"

specifies the name of the caslib for the output table.

"indexVars":["variable-name-1" <, "variable-name-2", ...>]

specifies the list of variables to create indexes for in the output data.

"lifetime":64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

"memoryFormat":"DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

"name":"table-name"

specifies the name for the output table.

"promote":True | False

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	False

"replace":True | False

when set to True, overwrites an existing table that has the same name.

Default	False

"tableRedistUpPolicy":"DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

copyVars=["variable-name-1" <, "variable-name-2", ...>]

specifies the names of variables to be copied to the output table.

distinctCountLimit=integer

Default	10000
Minimum value	256

ecdfTolerance=double

specifies the tolerance value for the empirical cumulative distribution function. This value is used by the quantile sketch algorithm.

Default	0.001
Range	1E-06–0.1

event="string"

explorationPolicy={avaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) policy.

Alias	avaptPolicy

The avaptPolicy value can be one or more of the following:

"cardinality":{cardinalityAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) cardinality policy.

The cardinalityAvaptPolicy value can be one or more of the following:

"lowMediumCutoff":double

specifies the cardinality threshold for the low-medium cutoff.

Default	32
Range	2–256

"mediumHighCutoff":double

specifies the cardinality threshold for the medium-high cutoff.

Default	64
Range	2–1024

"minNObsPerTargetLevel":double

specifies the minimum number of observations for each target level.

Default	10
Range	5–100

"cv":{cvAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) coefficient of variation policy.

Alias	coefficientVariation

The cvAvaptPolicy value can be one or more of the following:

"lowMoment":double

specifies the absolute value of the low-high percentage threshold for the moment coefficient of variation (CV).

Default	1
Minimum value	0

"lowRobust":double

specifies the absolute value of the low-high percentage threshold for the robust coefficient of variation (CV).

Default	1
Minimum value	0

"dateTimeVariables":["variable-name-1" <, "variable-name-2", ...>]

specifies the datetime variables.

Alias	dateTime

"dateVariables":["variable-name-1" <, "variable-name-2", ...>]

specifies the date variables.

Alias	date

"entropy":{entropyAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) entropy policy.

The entropyAvaptPolicy value can be one or more of the following:

"giniLowMediumCutoff":double

specifies the Gini entropy threshold for the low-medium cutoff.

Default	0.25
Range	0–1

"giniMediumHighCutoff":double

specifies the Gini entropy threshold for the medium-high cutoff.

Default	0.75
Range	0–1

"shannonLowMediumCutoff":double

specifies the Shannon entropy threshold for the low-medium cutoff.

Default	0.25
Range	0–1

"shannonMediumHighCutoff":double

specifies the Shannon entropy threshold for the medium-high cutoff.

Default	0.75
Range	0–1

"iqv":{iqvAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) index of qualitative variation policy.

Alias	qualitativeVariationIndex

The iqvAvaptPolicy value can be one or more of the following:

"highTopBottom":double

specifies the low-high cutoff frequency ratio threshold between the most frequent and least frequent levels of a nominal variable.

Alias	highTop1Bottom1
Default	100
Minimum value	1

"highTopTwo":double

specifies the low-high cutoff frequency ratio threshold between the most frequent and second most frequent levels of a nominal variable.

Alias	highTop1Top2
Default	10
Minimum value	1

"highVariationRatio":double

specifies the variation ratio threshold for the low-high cutoff.

Alias	highModVr
Default	0.5
Range	(0–1]

"kurtosis":{kurtosisAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) kurtosis policy.

The kurtosisAvaptPolicy value can be one or more of the following:

"momentLowMediumCutoff":double

specifies the absolute value of the moment kurtosis threshold for the low-medium cutoff.

Default	5
Minimum value	0

"momentMediumHighCutoff":double

specifies the absolute value of the moment kurtosis threshold for the medium-high cutoff.

Default	10
Minimum value	0

"robustLowMediumCutoff":double

specifies the absolute value of the robust kurtosis threshold for the low-medium cutoff.

Default	2
Minimum value	0

"robustMediumHighCutoff":double

specifies the absolute value of the robust kurtosis threshold for the medium-high cutoff.

Default	3
Minimum value	0

"missing":{missingAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) missing grouping policy.

The missingAvaptPolicy value can be one or more of the following:

"lowMediumCutoff":double

specifies the missing percentage threshold for the low-medium cutoff.

Default	5
Range	0–100

"mediumHighCutoff":double

specifies the missing percentage threshold for the medium-high cutoff.

Default	25
Range	0–100

"nominal":{nominalAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) nominal policy.

The nominalAvaptPolicy value can be one or more of the following:

"cardinalityRatio":double

specifies the AVAPT nominal policy cardinality ratio threshold.

Default	0.25
Range	(0–1]

"cardinalityThreshold":double

specifies the AVAPT nominal policy cardinality threshold.

Default	1024
Minimum value	32

"includeNegative":True | False

when set to True, includes numeric variables with some negative values in the nominal analysis.

Default	False

"includeNonIntegral":True | False

when set to True, includes numeric variables with some nonintegral values in the nominal analysis.

Default	False

"intervals":["variable-name-1" <, "variable-name-2", ...>]

specifies variables to consider as intervals.

"nominals":["variable-name-1" <, "variable-name-2", ...>]

specifies variables to consider as nominals.

"outlier":{outlierAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) outlier policy.

The outlierAvaptPolicy value can be one or more of the following:

"momentLowMediumCutoff":double

specifies the z-score outlier percentage threshold for the low-medium cutoff.

Default	1
Range	0–100

"momentMediumHighCutoff":double

specifies the z-score outlier percentage threshold for the medium-high cutoff.

Default	2.5
Range	0–100

"robustLowMediumCutoff":double

specifies the modified interquartile range outlier percentage threshold for the low-medium cutoff.

Default	1
Range	0–100

"robustMediumHighCutoff":double

specifies the modified interquartile range outlier percentage threshold for the medium-high cutoff.

Default	2.5
Range	0–100

"skewness":{skewnessAvaptPolicy}

specifies the automatic variable analysis and grouping (AVAPT) skewness policy.

The skewnessAvaptPolicy value can be one or more of the following:

"momentLowMediumCutoff":double

specifies the moment skewness threshold for the low-medium cutoff.

Default	2
Range	0–100

"momentMediumHighCutoff":double

specifies the moment skewness threshold for the medium-high cutoff.

Default	10
Range	0–100

"robustLowMediumCutoff":double

specifies the robust skewness threshold for the low-medium cutoff.

Default	0.75
Range	0–3

"robustMediumHighCutoff":double

specifies the robust skewness threshold for the medium-high cutoff.

Default	2
Range	0–3

"timeVariables":["variable-name-1" <, "variable-name-2", ...>]

specifies the time variables.

Alias	time

* featureOut={casouttable}

specifies the CAS table to store the feature transformation and generation pipelines.

Long form	featureOut={"name":"table-name"}
Shortcut form	featureOut="table-name"

The casouttable value can be one or more of the following:

"caslib":"string"

specifies the name of the caslib for the output table.

"indexVars":["variable-name-1" <, "variable-name-2", ...>]

specifies the list of variables to create indexes for in the output data.

"lifetime":64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

"memoryFormat":"DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

"name":"table-name"

specifies the name for the output table.

"promote":True | False

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	False

"replace":True | False

when set to True, overwrites an existing table that has the same name.

Default	False

"tableRedistUpPolicy":"DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

freq="variable-name"

specifies the frequency variable.

inputs=[{casinvardesc-1} <, {casinvardesc-2}, ...>]

specifies the variables to use for the analysis. You can specify a subset of the variables from the input table.

For more information about specifying the inputs parameter, see the common casinvardesc parameter (Appendix A: Common Parameters).

Alias	vars

misraGries=True | False

when set to True, uses the Misra-Gries algorithm for the frequency distribution estimation, if the distinct count limit is exceeded.

Default	True

rankPolicy={rankPolicy}

specifies the feature ranking policy.

Alias	rank

Long form	rankPolicy={"intervalStat":"AVGQUANKURT" \| "AVGQUANSKEW" \| "CLASSICALKURT" \| "CLASSICALSKEW" \| "ENTROPY" \| "MI" \| "NORMMI" \| "PEARSON" \| "SU"}
Shortcut form	rankPolicy="AVGQUANKURT" \| "AVGQUANSKEW" \| "CLASSICALKURT" \| "CLASSICALSKEW" \| "ENTROPY" \| "MI" \| "NORMMI" \| "PEARSON" \| "SU"

The rankPolicy value can be one or more of the following:

"intervalStat":"AVGQUANKURT" | "AVGQUANSKEW" | "CLASSICALKURT" | "CLASSICALSKEW" | "ENTROPY" | "MI" | "NORMMI" | "PEARSON" | "SU"

specifies the interval variable ranking statistic.

Alias	interval
Default	SU

AVGQUANKURT

uses the average quantile (robust) kurtosis.

AVGQUANSKEW

uses the average quantile (robust) skewness.

CLASSICALKURT

uses the classical (moment) kurtosis.

CLASSICALSKEW

uses the classical (moment) skewness.

ENTROPY

uses the entropy percentage decrease.

MI

uses the mutual information statistic.

NORMMI

uses the normalized mutual information statistic.

PEARSON

uses the Pearson correlation.

SU

uses the symmetric uncertainty statistic.

"nominalStat":"CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU"

specifies the nominal variable ranking statistic.

Alias	nominal
Default	SU

CHISQ

uses the chi-square statistic.

CRAMERSV

uses Cramer's V.

ENTROPY

uses the entropy percentage decrease.

FTEST

uses the F test.

G2

uses the G2 statistic.

IV

uses the information value statistic.

MI

uses the mutual information statistic.

NORMMI

uses the normalized mutual information statistic.

SU

uses the symmetric uncertainty statistic.

"rankMissingIndicators":True | False

when set to True, missing indicator features take part in the feature ranking. Otherwise, they are excluded from the ranking, and hence will always appear in the final feature set.

Default	True

"separateRank":True | False

when set to True, performs a separate feature ranking for interval and nominal features.

Alias	separate
Default	True

"topKInteractions":integer

specifies the number of top-ranked interaction features to generate and save.

Alias	topKInteract
Minimum value	1

"topKSave":integer

specifies the number of features per variable to save.

Alias	topK
Default	1
Minimum value	1

saveState={casouttable}

specifies the CAS table to store the feature transformation and generation model.

Alias	saveModel

Long form	saveState={"name":"table-name"}
Shortcut form	saveState="table-name"

The casouttable value can be one or more of the following:

"caslib":"string"

specifies the name of the caslib for the output table.

"indexVars":["variable-name-1" <, "variable-name-2", ...>]

specifies the list of variables to create indexes for in the output data.

"lifetime":64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

"memoryFormat":"DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

"name":"table-name"

specifies the name for the output table.

"promote":True | False

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	False

"replace":True | False

when set to True, overwrites an existing table that has the same name.

Default	False

"tableRedistUpPolicy":"DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

screenPolicy={sweeperPolicy}

specifies the variable screening policy to use for recommending that variables be screened out, transformed, or copied.

Alias	sweeperPolicy

The sweeperPolicy value can be one or more of the following:

"constant":True | False

when set to True, uses the variable screening policy to identify variables that have constant values.

Alias	unique
Default	True

"groupRareLevels":True | False

when set to True, uses the variable screening policy to identify nominal variables that have rare levels.

Alias	groupRare
Default	True

"leakagePercentThreshold":double

Alias	leakagePercentageThreshold
Default	90
Range	(0–100]

"lowCv":True | False

when set to True, uses the variable screening policy to identify variables that have a low coefficient of variation (CV).

Alias	lowCoefficientVariation
Default	True

"lowMutualInformation":double

specifies the variable screening policy for variables that have a low level of information about the target.

Alias	lowInformation
Default	0.05
Minimum value	0

"missingIndicatorPercent":double

specifies the variable screening policy for generating missing indicator variables.

Alias	missingIndicatorPercentage
Default	75
Range	[10–100)

"missingPercentThreshold":double

specifies the variable screening policy for identifying variables that have a very high missing rate.

Alias	missingPercentageThreshold
Default	90
Range	[10–100)

"redundant":double

Default	1
Range	(0–1]

seed=integer

specifies a seed value for random number generation. This value is used for repeatable random number generation in some scenarios.

Default	0

* table={castable}

specifies the table name, caslib, and other common parameters.

Long form	table={"name":"table-name"}
Shortcut form	table="table-name"

The castable value can be one or more of the following:

"caslib":"string"

specifies the caslib for the input table that you want to use with the action. By default, the active caslib is used. Specify a value only if you need to access a table from a different caslib.

"computedOnDemand":True | False

when set to True, creates the computed variables when the table is loaded instead of when the action begins.

Alias	compOnDemand
Default	False

"computedVars":[{casinvardesc-1} <, {casinvardesc-2}, ...>]

Alias	compVars

The casinvardesc value can be one or more of the following:

"format":"string"

specifies the format to apply to the variable.

"formattedLength":integer

specifies the length of the format field plus the length of the format precision.

"label":"string"

specifies the descriptive label for the variable.

* "name":"variable-name"

specifies the name for the variable.

"nfd":integer

specifies the length of the format precision.

"nfl":integer

specifies the length of the format field.

"computedVarsProgram":"string"

specifies an expression for each computed variable that you include in the computedVars parameter.

Alias	compPgm

"dataSourceOptions":{"key-1":{any-list-or-data-type-1} <, "key-2":{any-list-or-data-type-2}, ...>}

specifies data source options.

Aliases	options
Aliases	dataSource

"importOptions":{"fileType":"ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}

specifies the settings for reading a table from a data source.

Alias	import_

For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).

* "name":"table-name"

specifies the name of the input table.

"singlePass":True | False

when set to True, does not create a transient table on the server. Setting this parameter to True can be efficient, but the data might not have stable ordering upon repeated runs.

Default	False

"vars":[{casinvardesc-1} <, {casinvardesc-2}, ...>]

specifies the variables to use in the action.

The casinvardesc value can be one or more of the following:

"format":"string"

specifies the format to apply to the variable.

"formattedLength":integer

specifies the length of the format field plus the length of the format precision.

"label":"string"

specifies the descriptive label for the variable.

* "name":"variable-name"

specifies the name for the variable.

"nfd":integer

specifies the length of the format precision.

"nfl":integer

specifies the length of the format field.

"where":"where-expression"

specifies an expression for subsetting the input data.

"whereTable":{groupbytable}

The groupbytable value can be one or more of the following:

"casLib":"string"

specifies the caslib for the filter table. By default, the active caslib is used.

"dataSourceOptions":{adls_noreq-parameters | bigquery-parameters | cas_noreq-parameters | clouddex-parameters | db2-parameters | dnfs-parameters | esp-parameters | fedsvr-parameters | gcs_noreq-parameters | hadoop-parameters | hana-parameters | impala-parameters | informix-parameters | jdbc-parameters | mongodb-parameters | mysql-parameters | odbc-parameters | oracle-parameters | path-parameters | postgres-parameters | redshift-parameters | s3-parameters | sapiq-parameters | sforce-parameters | singlestore_standard-parameters | snowflake-parameters | spark-parameters | spde-parameters | sqlserver-parameters | ss_noreq-parameters | teradata-parameters | vertica-parameters | yellowbrick-parameters}

specifies data source options.

Aliases	options
Aliases	dataSource

For more information about specifying the dataSourceOptions parameter, see the common dataSourceOptions parameter (Appendix A: Common Parameters).

"importOptions":{"fileType":"ANY" | "AUDIO" | "AUTO" | "BASESAS" | "CSV" | "DOCUMENT" | "DTA" | "ESP" | "EXCEL" | "FMT" | "HDAT" | "IMAGE" | "JMP" | "LASR" | "PARQUET" | "SPSS" | "VIDEO" | "XLS", fileType-specific-parameters}

specifies the settings for reading a table from a data source.

Alias	import_

For more information about specifying the importOptions parameter, see the common importOptions parameter (Appendix A: Common Parameters).

* "name":"table-name"

specifies the name of the filter table.

"vars":[{casinvardesc-1} <, {casinvardesc-2}, ...>]

specifies the variable names to use from the filter table.

The casinvardesc value can be one or more of the following:

"format":"string"

specifies the format to apply to the variable.

"formattedLength":integer

specifies the length of the format field plus the length of the format precision.

"label":"string"

specifies the descriptive label for the variable.

* "name":"variable-name"

specifies the name for the variable.

"nfd":integer

specifies the length of the format precision.

"nfl":integer

specifies the length of the format field.

"where":"where-expression"

specifies an expression for subsetting the data from the filter table.

* target="variable-name"

specifies the target variable.

Alias	evalVar

* transformationOut={casouttable}

specifies the CAS table to store the feature transformation and generation pipelines.

Long form	transformationOut={"name":"table-name"}
Shortcut form	transformationOut="table-name"

The casouttable value can be one or more of the following:

"caslib":"string"

specifies the name of the caslib for the output table.

"indexVars":["variable-name-1" <, "variable-name-2", ...>]

specifies the list of variables to create indexes for in the output data.

"lifetime":64-bit-integer

specifies the number of seconds to keep the table in memory after it is last accessed. The table is dropped if it is not accessed for the specified number of seconds.

Default	0
Minimum value	0

"memoryFormat":"DVR" | "INHERIT" | "STANDARD"

specifies the memory format for the output table.

Default	INHERIT

DVR

use the duplicate value reduction memory format. This memory format can reduce the memory consumption and file size when the input data contains duplicate values.

INHERIT

STANDARD

use the standard memory format.

"name":"table-name"

specifies the name for the output table.

"promote":True | False

when set to True, adds the output table with a global scope. This enables other sessions to access the table, subject to access controls. The target caslib must also have a global scope.

Default	False

"replace":True | False

when set to True, overwrites an existing table that has the same name.

Default	False

"tableRedistUpPolicy":"DEFER" | "NOREDIST" | "REBALANCE"

Specifies the Table Redistribution Policy when the number of worker pods increases on a running CAS server.

DEFER

Defer redistribution policy selection to higher-level entity.

NOREDIST

Do not redistribute table data when the number of worker pods changes on a running CAS server.

REBALANCE

Rebalance table data when the number of worker pods changes on a running CAS server.

transformationPolicy={transformationSpace}

specifies the feature transformation and generation space in which the feature machine operates.

Alias	ftgPolicy

The transformationSpace value can be one or more of the following:

"cardinality":True | False

when set to True, includes cardinality-reducing transformations.

Default	True

"entropy":True | False

when set to True, includes transformations for the treatment of low entropy.

Default	False

"interaction":True | False

when set to True, detects and generates interaction features.

Default	False

"iqv":True | False

when set to True, includes transformations for the treatment of low indices of qualitative variation (IQV).

Default	False

"kurtosis":True | False

when set to True, includes transformations for the treatment of high kurtosis.

Default	False

"missing":True | False

when set to True, includes transformations for the treatment of missing values.

Default	True

"outlier":True | False

when set to True, includes transformations for the treatment of outliers.

Default	False

"polynomial":True | False

when set to True, includes up to third-order polynomial transformations.

Default	False

"skewness":True | False

when set to True, includes transformations for the treatment of high skewness.

Default	True

weight="variable-name"

specifies the weight variable.

featureMachine Action

Automated feature transformation and generation engine.

R Syntax
Summary: Input and Output Tables
Parameter Descriptions

R Syntax

results <– cas.dataSciencePilot.featureMachine(s,

casout=list(

caslib="string",

indexVars=list("variable-name-1" <, "variable-name-2", ...>),

lifetime=64-bit-integer,

name="table-name",

promote=TRUE | FALSE,

replace=TRUE | FALSE,

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

copyVars=list("variable-name-1" <, "variable-name-2", ...>),

distinctCountLimit=integer,

ecdfTolerance=double,

event="string",

explorationPolicy=list(

cardinality=list(

lowMediumCutoff=double

mediumHighCutoff=double

minNObsPerTargetLevel=double

cv=list(

lowMoment=double

lowRobust=double

dateTimeVariables=list("variable-name-1" <, "variable-name-2", ...>),

dateVariables=list("variable-name-1" <, "variable-name-2", ...>),

entropy=list(

giniLowMediumCutoff=double

giniMediumHighCutoff=double

shannonLowMediumCutoff=double

shannonMediumHighCutoff=double

iqv=list(

highTopBottom=double

highTopTwo=double

highVariationRatio=double

kurtosis=list(

missing=list(

lowMediumCutoff=double

mediumHighCutoff=double

nominal=list(

cardinalityRatio=double

cardinalityThreshold=double

includeNegative=TRUE | FALSE

includeNonIntegral=TRUE | FALSE

intervals=list("variable-name-1" <, "variable-name-2", ...>)

nominals=list("variable-name-1" <, "variable-name-2", ...>)

outlier=list(

skewness=list(

timeVariables=list("variable-name-1" <, "variable-name-2", ...>)

featureOut=list(

caslib="string",

indexVars=list("variable-name-1" <, "variable-name-2", ...>),

lifetime=64-bit-integer,

name="table-name",

promote=TRUE | FALSE,

replace=TRUE | FALSE,

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

freq="variable-name",

inputs=list( list(

format="string",

formattedLength=integer,

label="string",

name="variable-name",

nfd=integer,

nfl=integer

) <, list(...)>),

misraGries=TRUE | FALSE,

rankPolicy=list(

nominalStat="CHISQ" | "CRAMERSV" | "ENTROPY" | "FTEST" | "G2" | "IV" | "MI" | "NORMMI" | "SU",

rankMissingIndicators=TRUE | FALSE,

separateRank=TRUE | FALSE,

topKInteractions=integer,

topKSave=integer

saveState=list(

caslib="string",

indexVars=list("variable-name-1" <, "variable-name-2", ...>),

lifetime=64-bit-integer,

name="table-name",

promote=TRUE | FALSE,

replace=TRUE | FALSE,

tableRedistUpPolicy="DEFER" | "NOREDIST" | "REBALANCE"

screenPolicy=list(

constant=TRUE | FALSE,

groupRareLevels=TRUE | FALSE,

leakagePercentThreshold=double,

lowCv=TRUE | FALSE,

lowMutualInformation=double,

missingIndicatorPercent=double,

missingPercentThreshold=double,

redundant=double

seed=integer,

table=list(

caslib="string",

computedOnDemand=TRUE | FALSE,

computedVars=list( list(

format="string",

formattedLength=integer,

label="string",

name="variable-name",

nfd=integer,

nfl=integer

) <, list(...)>),

computedVarsProgram="string",

dataSourceOptions=list(key-1=list(any-list-or-data-type-1) <, key-2=list(any-list-or-data-type-2), ...>),

name="table-name",

singlePass=TRUE | FALSE,

vars=list( list(

format="string",

formattedLength=integer,

label="string",

name="variable-name",

nfd=integer,

nfl=integer

) <, list(...)>),

where="where-expression",

whereTable=list(

casLib="string"

name="table-name"

vars=list( list(

format="string",

formattedLength=integer,

label="string",

name="variable-name",

nfd=integer,

nfl=integer

) <, list(...)>)

where="where-expression"

)

target="variable-name",

transformationOut=list(

caslib="string",

indexVars=list("variable-name-1" <, "variable-name-2", ...>),

lifetime=64-bit-integer,