DEEPCAUSAL Procedure

DNN Statement

DNN <options>;

The DNN statement specifies the parameters for the deep neural network (DNN). It includes options that are related to the DNNs for the propensity score model and outcome models. You can specify the following options:

NODES=(numeric-list)

specifies the number of nodes for each hidden layer in the DNN, where each value in the numeric-list must be a positive number and values are separated by commas or spaces. The length of the list specifies the number of layers, and each value specifies the number of nodes for the corresponding layer. If the length of the list is denoted by n, the ith () value denotes the number of nodes for the ith layer. By default, NODES=(32 32 32 32), which yields four hidden layers, with 32 nodes in each layer.

TRAIN=(train-options)

specifies the train options for the DNN. You can specify one or more of the following train-options:

GPU=(<(gpu-options)>)

specifies that the DNN perform calculations by using graphics processing unit (GPU) hardware. You can specify the following gpu-options:

DEVICE=(numeric-list) DEVICES=(numeric-list)

specifies a list of GPU devices to use.

PRECISION=FP16 | FP32

specifies the computational precision in forward-backward computations. You can specify the following values:

FP16: uses half-precision (16-bit floating-point) computations to train and score the DNN. The lower computational precision in forward-backward computations can encourage tensor core engagements.
FP32: uses single-precision (32-bit floating-point) computations to train and score the DNN.

By default, PRECISION=FP32.

USEEXCLUSIVE

specifies that only GPU devices be used.

USETENSORRT

enables the use of the TensorRT software development kit for fast inference.

MISSING=MAX | MEAN | MIN | NONE

specifies the policy to use for replacing missing input covariate values with imputed values. You can specify one of the following values:

MAX: replaces missing values with the maximum of the corresponding covariate.
MEAN: replaces missing values with the mean of the corresponding covariate.
MIN: replaces missing values with the minimum of the corresponding covariate.
NONE: ignores observations that have missing values.

By default, MISSING=MEAN. The MISSING= option is for DNN training only. In the later inference, observations that have missing covariates are removed.

NTHREADS=number

specifies the number of threads to use for the DNN.

OPTIMIZATION=(optimizer-options) OPTIMIZE=(optimizer-options) OPTIMIZER=(optimizer-options)

specifies the settings for the optimization algorithm, optimization mode, and other settings, such as a seed, the maximum number of epochs, and so on. You can specify the following optimizer-options:

ALGORITHM=algorithm-keyword<(common-algorithm-options)>

specifies the optimization algorithm and the corresponding options. First, you can specify some common options of all the optimization algorithms, common-algorithm-options, in the parentheses that follow the algorithm-keyword. They are as follows:

CLIPGRADMAX=number

specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value. The number should be greater than or equal to 0.

CLIPGRADMIN=number

specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value. The number should be less than or equal to 0.

FCMPEARLYSTOPPING=symbol-name FCMPES=symbol-name

specifies the FCMP early stopping function.

FCMPLEARNINGRATE=symbol-name FCMPLR=symbol-name

specifies the FCMP learning rate function.

GAMMA=number

specifies the gamma value for the learning rate policy. The number must be between 0 and 100. By default, GAMMA=0.1.

LEARNINGRATE=number

specifies the learning rate (a positive value) for stochastic gradient descent (SGD). By default, LEARNINGRATE=0.001.

specifies the learning rate policy. You can specify one of the following values:

FIXED: specifies a fixed learning rate.
INV: sets the learning rate option value after each epoch, according to the initial learning rate, the value of the GAMMA= option, and the value of the POWER= option. The rate is calculated as .
MULTISTEP: sets the learning rate after each of the epochs that you specify in the STEPS= option. The updated learning rate is the product of the previous learning rate value and the GAMMA= option value.
POLY: sets the learning rate option value after each epoch, according to the initial learning rate, the maximum number of epochs, and the value of the POWER= option. The rate is calculated as .
STEP: sets the learning rate option by multiplying the current learning rate option value by the GAMMA= option value. The number of steps is specified in the STEPSIZE= option. The configured learning rate is recalculated for each group of epochs, according to the step size.

By default, LEARNINGRATEPOLICY=FIXED.

NOUSELOCKING

specifies that gradients be computed asynchronously with multiple threads.

POWER=number

specifies the power for the learning rate policy. The number must be nonnegative. By default, POWER=0.75.

STEPS=(numeric-list)

specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the GAMMA= option. For example, if you specify STEPS=(5 9 13), then the learning rate is multiplied by gamma after the 5th, 9th, and 13th epochs.

STEPSIZE=number

specifies the step size (a value greater than or equal to 1) when the learning rate policy is set to STEP. By default, STEPSIZE=10.

For the ALGORITHM= option, you can specify one of the following algorithm-keywords and its associated options:

ADAM<(adam-options)> ADAMSGD<(adam-options)>

specifies the ADAM optimization algorithm. For the adam-options, you can specify one or more of the common-algorithm-options. You can also specify the following adam-options:

BETA1=number: specifies the exponential decay rate for the first moment in an ADAM learning algorithm. The number must be in the range . By default, BETA1=0.9.
BETA2=number: specifies the exponential decay rate for the second moment in an ADAM learning algorithm. The number must be in the range . By default, BETA2=0.999.

LARS<(lars-options)>

specifies the LARS optimization algorithm. For the lars-options, you can specify one or more of the common-algorithm-options. You can also specify one or more of the following lars-options:

LAYERRATES=(numeric-list) LOCALRATES=(numeric-list): produces an array of values that specifies the learning rate for each model layer. You cannot specify this option in the DNN statement; you can specify it only in the PSMODEL or MODEL statement. In the PSMODEL statement, the number of local learning rates that you specify in the numeric-list must equal , where m is the number of hidden layers of the DNN for the propensity score model. In the MODEL statement, the number of local learning rates that you specify in the numeric-list must equal , where n is the number of hidden layers of the DNN for the outcome model.
MOMENTUM=number: specifies the momentum parameter value for stochastic gradient descent (SGD). The number must be in the range . By default, MOMENTUM=0.9.
SCALEFACTOR=number: specifies the scaling coefficient that is used to adjust the base learning rate. The number must be in the range . By default, SCALEFACTOR=0.001.
WARMUP=number: specifies the number of warmup epochs to complete before applying layerwise learning rates in the LARS optimization algorithm. The number must be greater than or equal to –1. By default, WARMUP=–1, which means that there are no warmup epochs.

LBFGS<(lbfgs-options)>

specifies the L-BFGS optimization algorithm. In the lbfgs-options, you can specify one or more of the common-algorithm-options. The GAMMA=, LEARNINGRATE=, LEARNINGRATEPOLICY=, POWER=, and STEPSIZE= options do not support the L-BFGS optimization algorithm. You can also specify one or more of the following lbfgs-options:

BACKTRACKRATIO=number: specifies the backtrack ratio of line search iterations for the L-BFGS optimization algorithm. The number must be in the range . By default, BACKTRACKRATIO=0.5.
M=number NUMCORRECTIONS=number: specifies the number of corrections to use in the L-BFGS update. The number must be greater than or equal to 0. By default, M=6.
MAXLINESEARCHITERS=number: specifies the maximum number of line search iterations for L-BFGS optimization algorithm. The number must be an integer greater than or equal to 1. By default, MAXLINESEARCHITERS=60.

MOMENTUMALG<(momentumalg-options)> MOMENTUMSGD<(momentumalg-options)>

specifies the MOMENTUM optimization algorithm. In the momentumalg-options, you can specify one or more of the common-algorithm-options. You can also specify the following momentumalg-option:

MOMENTUM=number: specifies the momentum parameter value for stochastic gradient descent (SGD). The number must be between 0 and 1. By default, MOMENTUM=0.9.

TRISH<(trish-options)> TRISHSGD<(trish-options)>

specifies the trust-region-ish (TRish) optimization algorithm. In the trish-options, you can specify one or more of the common-algorithm-options. You can also specify one or more of the following trish-options:

GAMMA1=number: specifies the Gamma1 scaling for the gradient. The number must be in the range . By default, GAMMA1=2.
GAMMA2=number: specifies the Gamma2 scaling for the gradient. The number must be in the range . By default, GAMMA2=2.
MOMENTUM=number: specifies the momentum parameter value for stochastic gradient descent (SGD). The number must be in the range . By default, MOMENTUM=0.9.

VANILLA<(common-algorithm-options)> VANILLASGD<(common-algorithm-options)>

specifies the vanilla optimization algorithm.

By default, ALGORITHM=VANILLA.

COMPRESSION=(compression-options)

sparsifies the communication between workers for faster training. You can specify one or more of the following compression-options:

LOCALMOMENTUM=number: determines the strength of aggregation on the masked gradient elements when compression is used. The number must be in the range . By default, LOCALMOMENTUM=0.95.
SAMPLERATIO=number: determines the number of samples for gradients in each layer. The number must be in the range . By default, SAMPLERATIO=0.05.
SPARSITYRATIO=number: determines the number of samples for gradients in each layer. The number must be in the range . By default, SPARSITYRATIO=0.75.

DROPOUT=number

specifies the probability that the output of a neuron in a fully connected layer will be set to 0 during training. The specified probability is recalculated each time an observation is processed. The number must be in the range . By default, DROPOUT=0.

DROPOUTINPUT=number

specifies the probability that an input variable will be set to 0 during training. The specified probability is recalculated each time an observation is processed. The number must be in the range . By default, DROPOUTINPUT=0.

DROPOUTTYPE=INVERTED | STANDARD

specifies what type of dropout to use. You can specify one of the following types:

INVERTED: specifies the inverted dropout, in which activations of some neurons are set to 0 and the remaining ones are scaled.
STANDARD: specifies the standard dropout, in which activations of some neurons are set to 0 and the remaining ones are not scaled.

By default, DROPOUTTYPE=STANDARD.

FCONV=number

specifies the relative function convergence criterion. If the relative loss error, , does not result in a change in the objective function, the optimization stops. The number must be greater than or equal to 0. By default, FCONV=0, and the relative function convergence is not checked.

FREEZEBATCHNORMSTATS FREEZEBNSTATS

freezes statistics of all batch normalization layers.

IGNORETRAININGERROR

specifies that the model training continue without interruption by ignoring the following issue: there are training data observations that contain invalid or missing variable data. If you omit this option, training stops, and PROC DEEPCAUSAL terminates if bad input data are found.

MAXEPOCHS=number

specifies the maximum number of epochs. The number must be greater than or equal to 1. By default, MAXEPOCHS=1. For SGD with a single-machine server, or for a session that uses one worker on a distributed server, one epoch is reached when the optimizer passes through the data one time. For a session that uses more than one worker, one epoch is reached when all the workers exchange the weights with the controller one time. The SYNCFREQ= option specifies the number of times that each worker passes through the data before exchanging weights with the controller. For the L-BFGS optimization algorithm with a full batch, each iteration might process more than one epoch, and the final number of epochs might exceed the parameter value for the maximum number of epochs.

MINIBATCHBUFSIZE=number

specifies the number of observations to buffer in memory (including input data and intermediate calculations) before processing the remaining records in the minibatch. The number must be greater than or equal to 1 and must be smaller than the number that you specify for the MINIBATCHSIZE= option. The MINIBATCHBUFSIZE= option setting is ignored on GPUs.

MINIBATCHSIZE=number

specifies the number of observations per thread in a minibatch. The parameter controls the number of observations used

per worker in each thread to compute the gradient, prior to updating the weights. Larger values use more memory. The number is greater than or equal to 1. By default, MINIBATCHSIZE=1. When you use synchronous SGD (the default), the total minibatch size is calculated as miniBatchSize * number of threads * number of workers. When you use asynchronous SGD by specifying the ELASTICSYNCFREQ= option, each worker trains its own local model. In this case, the total minibatch size for each worker is calculated as miniBatchSize * number of threads.

MODE=DOWNPOUR | ELASTIC | SYNCHRONOUS

specifies the optimization mode and the corresponding options. You can specify one of the following values:

DOWNPOUR

specifies the downpour optimization mode.

ELASTIC<(elastic-options)>

specifies the elastic optimization mode. You can specify the following elastic-options:

ALPHA=number ELASTICALPHA=number: specifies the significance level (a value between 0 and 1) that is used for elastic SGD. By default, ALPHA=0.
ELASTICSYNCFREQ=number SYNCFREQ=number: specifies the frequency (a nonnegative value) for communication between the workers and controller for exchanging weights. You can exchange the weights more often than once each epoch by specifying a number that is less than the number of batches in an epoch. If the number is greater than the number of batches in an epoch, then the weights are exchanged once for each epoch. By default, ELASTICSYNCFREQ=0.

SYNCHRONOUS<(SYNCFREQ=number)>

specifies the synchronous optimization mode. The SYNCFREQ= option, which is not required, specifies the synchronization frequency for SGD in terms of epochs. By default, SYNCFREQ=1.

By default, MODE=SYNCHRONOUS.

NOBNSRCLAYERWARNINGS

suppresses the warning, if the batch normalization source layer has an atypical type, activation, or includeBias setting.

REGL1=number

specifies the weight for the L1 regularization term. The number must be greater than or equal to 0. By default, REGL1=0. At the default setting, PROC DEEPCAUSAL does not perform L1 regularization. Setting the L1 regularization weight to 0 disables the regularization. Initial L1 weight values should be small (such as 1E–3). You can combine L1 regularization with L2 regularization.

REGL2=number

specifies the weight for the L2 regularization term. The number must be greater than or equal to 0. By default, REGL2=0. At the default setting, PROC DEEPCAUSAL does not perform L2 regularization. Setting the L2 regularization weight to 0 disables the regularization. Initial L2 weight values should be small (such as 1E–3). You can combine L2 regularization with L1 regularization.

SEED=number

specifies the random number seed value for the random number generator in the L-BFGS optimization algorithm. Seed values less than or equal to 0 generate random number streams by using the time of day from the computer’s clock. Seed values greater than 0 generate reproducible random number sequences. By default, SEED=0.

SNAPSHOTFREQ=number

specifies the frequency for generating snapshots of the neural weights and storing the weights in a weight table during the training process. When asynchronous SGD is specified, PROC DEEPCAUSAL synchronizes all the weights before writing out the weights. The number must be greater than or equal to 0. By default, SNAPSHOTFREQ=0.

STAGNATION=number

specifies the number of iterations to complete without improvement before stopping the optimization early. The number must be greater than or equal to 0. By default, STAGNATION=0; this turns off the stagnation monitoring.

THRESHOLD=number

specifies the threshold that is used to determine whether the loss error or validation scores are improving or stagnating across iterations. The iteration does not improve when the magnitude of the score change between successive iterations is less than or equal to the absolute value of current_score * threshold value. When an iteration does not improve, the stagnation counter increments. Otherwise, the stagnation counter is set to 0. The number must be greater than or equal to 0. By default, THRESHOLD=.

TOTALMINIBATCHSIZE=(totalminibatchsize-options)

specifies the number of observations in a minibatch. You can specify one or more of the following totalminibatchsize-options:

NONOTIFY: suppresses the message that is printed if the total minibatch size is rounded up.
ROUND: rounds up the total minibatch size.
SIZE=number N=number: specifies the number of observations in a minibatch. The computation for the total minibatch size is nWorkers * miniBatchSize * nThreads. For example, a two-worker scenario with 32 threads per worker and a minibatch size of 10 has a total minibatch size of 2*32*10 = 640 records.

You can use these options to control the number of observations that the DNN uses to compute the gradient prior to updating the weights. Larger values use more memory. If the specified size cannot be evenly divided by the number of threads (if you are using asynchronous SGD) or by the number of threads * number of workers (if you are using synchronous SGD), then PROC DEEPCAUSAL terminates with an error unless you specify the ROUND option. In that case, the total minibatch size is rounded up so that it is evenly divided.

RECORDSEED=number

specifies the random number seed for random record selection within a worker. The default (RECORDSEED=0) disables random record selection. Records are read in the order in which they are written in memory. Seed values that are less than or equal to 0 generate random number streams by using the time of day from the computer’s clock. Seed values greater than 0 generate reproducible random number sequences.

SEED=number

specifies the random number seed value for the random number generator in stochastic gradient descent (SGD). Seed values that are less than or equal to 0 generate random number streams by using the time of day from the computer’s clock. Seed values greater than 0 generate reproducible random number sequences. By default, SEED=0.

TARGETMISSING=MAX | MEAN | MIN | NONE

specifies the policy to use for replacing missing outcome variables with imputed values. You can specify one of the following values:

MAX: replaces missing values with the maximum.
MEAN: replaces missing values with the mean value.
MIN: replaces missing values with the minimum.
NONE: ignores observations that have missing target variables.

By default, TARGETMISSING=MEAN. The TARGETMISSING= option is for DNN training only. In the later inference, observations that have missing outcome variables have missing values for residuals and for any estimates that need residuals.

Last updated: November 24, 2025