Common Parameter: algorithm
Parameters for method="ADAM" | "ADAMSGD"
beta1=double
specifies the exponential decay rate for the first moment in an ADAM learning algorithm.
| Default | 0.9 |
|---|---|
| Range | [0–1) |
beta2=double
specifies the exponential decay rate for the second moment in an ADAM learning algorithm.
| Default | 0.999 |
|---|---|
| Range | [0–1) |
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="LARS"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
layerRates={double-1 <, double-2, ...>}
an array of values that specifies the learning rate for each model layer.
| Alias | localRates |
|---|
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
momentum=double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
scaleFactor=double
the scaling coefficient that is used to adjust the base learning rate.
| Default | 0.001 |
|---|---|
| Range | 0–1 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
warmup=64-bit-integer
specifies the number of epochs to complete before applying layer-wise learning rates.
| Default | -1 |
|---|---|
| Minimum value | -1 |
Parameters for method="LBFGS"
backtrackRatio=double
specifies the backtrack ratio of line search iterations for the L-BFGS solver.
| Range | [0–1) |
|---|
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
m=64-bit-integer
specifies the number of corrections used in the L-BFGS update.
| Minimum value | 0 |
|---|
maxLineSearchIters=64-bit-integer
Specifies the maximum number of line search iterations for L-BFGS solver.
| Minimum value | 1 |
|---|
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="MOMENTUM" | "MOMENTUMSGD"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
momentum=double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="NATGRAD" | "RIEGRAD"
approximateType=64-bit-integer
specifies the block-diagonal approximation of the Fisher information matrix.
| Default | 1 |
|---|---|
| Range | 1–1 |
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="TRISH" | "TRISHSGD"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
gamma1=double
Gamma1 scaling for the gradient
| Default | 2 |
|---|---|
| Range | [0–10000) |
gamma2=double
Gamma2 scaling for the gradient
| Default | 0.5 |
|---|---|
| Range | [0–10000) |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
momentum=double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="VANILLA" | "VANILLASGD"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="ADAM" | "ADAMSGD"
beta1=double
specifies the exponential decay rate for the first moment in an ADAM learning algorithm.
| Default | 0.9 |
|---|---|
| Range | [0–1) |
beta2=double
specifies the exponential decay rate for the second moment in an ADAM learning algorithm.
| Default | 0.999 |
|---|---|
| Range | [0–1) |
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=true | false
when set to False, computes gradients asynchronously with multiple threads.
| Default | true |
|---|
Parameters for method="LARS"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
layerRates={double-1 <, double-2, ...>}
an array of values that specifies the learning rate for each model layer.
| Alias | localRates |
|---|
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
momentum=double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
scaleFactor=double
the scaling coefficient that is used to adjust the base learning rate.
| Default | 0.001 |
|---|---|
| Range | 0–1 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=true | false
when set to False, computes gradients asynchronously with multiple threads.
| Default | true |
|---|
warmup=64-bit-integer
specifies the number of epochs to complete before applying layer-wise learning rates.
| Default | -1 |
|---|---|
| Minimum value | -1 |
Parameters for method="LBFGS"
backtrackRatio=double
specifies the backtrack ratio of line search iterations for the L-BFGS solver.
| Range | [0–1) |
|---|
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
m=64-bit-integer
specifies the number of corrections used in the L-BFGS update.
| Minimum value | 0 |
|---|
maxLineSearchIters=64-bit-integer
Specifies the maximum number of line search iterations for L-BFGS solver.
| Minimum value | 1 |
|---|
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=true | false
when set to False, computes gradients asynchronously with multiple threads.
| Default | true |
|---|
Parameters for method="MOMENTUM" | "MOMENTUMSGD"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
momentum=double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=true | false
when set to False, computes gradients asynchronously with multiple threads.
| Default | true |
|---|
Parameters for method="NATGRAD" | "RIEGRAD"
approximateType=64-bit-integer
specifies the block-diagonal approximation of the Fisher information matrix.
| Default | 1 |
|---|---|
| Range | 1–1 |
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=true | false
when set to False, computes gradients asynchronously with multiple threads.
| Default | true |
|---|
Parameters for method="TRISH" | "TRISHSGD"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
gamma1=double
Gamma1 scaling for the gradient
| Default | 2 |
|---|---|
| Range | [0–10000) |
gamma2=double
Gamma2 scaling for the gradient
| Default | 0.5 |
|---|---|
| Range | [0–10000) |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
momentum=double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=true | false
when set to False, computes gradients asynchronously with multiple threads.
| Default | true |
|---|
Parameters for method="VANILLA" | "VANILLASGD"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps={64-bit-integer-1 <, 64-bit-integer-2, ...>}
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=true | false
when set to False, computes gradients asynchronously with multiple threads.
| Default | true |
|---|
Parameters for method="ADAM" | "ADAMSGD"
"beta1":double
specifies the exponential decay rate for the first moment in an ADAM learning algorithm.
| Default | 0.9 |
|---|---|
| Range | [0–1) |
"beta2":double
specifies the exponential decay rate for the second moment in an ADAM learning algorithm.
| Default | 0.999 |
|---|---|
| Range | [0–1) |
"clipGradMax":double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
"clipGradMin":double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
"fcmpEarlyStopping":"string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
"fcmpLearningRate":"string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
"gamma":double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
"learningRate":double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
"learningRatePolicy":"FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
"power":double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
"steps":[64-bit-integer-1 <, 64-bit-integer-2, ...>]
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
"stepSize":64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
"useLocking":True | False
when set to False, computes gradients asynchronously with multiple threads.
| Default | True |
|---|
Parameters for method="LARS"
"clipGradMax":double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
"clipGradMin":double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
"fcmpEarlyStopping":"string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
"fcmpLearningRate":"string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
"gamma":double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
"layerRates":[double-1 <, double-2, ...>]
an array of values that specifies the learning rate for each model layer.
| Alias | localRates |
|---|
"learningRate":double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
"learningRatePolicy":"FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
"momentum":double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
"power":double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
"scaleFactor":double
the scaling coefficient that is used to adjust the base learning rate.
| Default | 0.001 |
|---|---|
| Range | 0–1 |
"steps":[64-bit-integer-1 <, 64-bit-integer-2, ...>]
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
"stepSize":64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
"useLocking":True | False
when set to False, computes gradients asynchronously with multiple threads.
| Default | True |
|---|
"warmup":64-bit-integer
specifies the number of epochs to complete before applying layer-wise learning rates.
| Default | -1 |
|---|---|
| Minimum value | -1 |
Parameters for method="LBFGS"
"backtrackRatio":double
specifies the backtrack ratio of line search iterations for the L-BFGS solver.
| Range | [0–1) |
|---|
"clipGradMax":double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
"clipGradMin":double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
"fcmpEarlyStopping":"string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
"fcmpLearningRate":"string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
"gamma":double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
"learningRate":double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
"learningRatePolicy":"FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
"m":64-bit-integer
specifies the number of corrections used in the L-BFGS update.
| Minimum value | 0 |
|---|
"maxLineSearchIters":64-bit-integer
Specifies the maximum number of line search iterations for L-BFGS solver.
| Minimum value | 1 |
|---|
"power":double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
"steps":[64-bit-integer-1 <, 64-bit-integer-2, ...>]
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
"stepSize":64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
"useLocking":True | False
when set to False, computes gradients asynchronously with multiple threads.
| Default | True |
|---|
Parameters for method="MOMENTUM" | "MOMENTUMSGD"
"clipGradMax":double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
"clipGradMin":double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
"fcmpEarlyStopping":"string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
"fcmpLearningRate":"string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
"gamma":double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
"learningRate":double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
"learningRatePolicy":"FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
"momentum":double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
"power":double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
"steps":[64-bit-integer-1 <, 64-bit-integer-2, ...>]
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
"stepSize":64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
"useLocking":True | False
when set to False, computes gradients asynchronously with multiple threads.
| Default | True |
|---|
Parameters for method="NATGRAD" | "RIEGRAD"
"approximateType":64-bit-integer
specifies the block-diagonal approximation of the Fisher information matrix.
| Default | 1 |
|---|---|
| Range | 1–1 |
"clipGradMax":double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
"clipGradMin":double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
"fcmpEarlyStopping":"string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
"fcmpLearningRate":"string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
"gamma":double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
"learningRate":double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
"learningRatePolicy":"FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
"power":double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
"steps":[64-bit-integer-1 <, 64-bit-integer-2, ...>]
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
"stepSize":64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
"useLocking":True | False
when set to False, computes gradients asynchronously with multiple threads.
| Default | True |
|---|
Parameters for method="TRISH" | "TRISHSGD"
"clipGradMax":double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
"clipGradMin":double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
"fcmpEarlyStopping":"string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
"fcmpLearningRate":"string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
"gamma":double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
"gamma1":double
Gamma1 scaling for the gradient
| Default | 2 |
|---|---|
| Range | [0–10000) |
"gamma2":double
Gamma2 scaling for the gradient
| Default | 0.5 |
|---|---|
| Range | [0–10000) |
"learningRate":double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
"learningRatePolicy":"FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
"momentum":double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
"power":double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
"steps":[64-bit-integer-1 <, 64-bit-integer-2, ...>]
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
"stepSize":64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
"useLocking":True | False
when set to False, computes gradients asynchronously with multiple threads.
| Default | True |
|---|
Parameters for method="VANILLA" | "VANILLASGD"
"clipGradMax":double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
"clipGradMin":double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
"fcmpEarlyStopping":"string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
"fcmpLearningRate":"string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
"gamma":double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
"learningRate":double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
"learningRatePolicy":"FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
"power":double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
"steps":[64-bit-integer-1 <, 64-bit-integer-2, ...>]
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
"stepSize":64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
"useLocking":True | False
when set to False, computes gradients asynchronously with multiple threads.
| Default | True |
|---|
Parameters for method="ADAM" | "ADAMSGD"
beta1=double
specifies the exponential decay rate for the first moment in an ADAM learning algorithm.
| Default | 0.9 |
|---|---|
| Range | [0–1) |
beta2=double
specifies the exponential decay rate for the second moment in an ADAM learning algorithm.
| Default | 0.999 |
|---|---|
| Range | [0–1) |
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps=list(64-bit-integer-1 <, 64-bit-integer-2, ...>)
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="LARS"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
layerRates=list(double-1 <, double-2, ...>)
an array of values that specifies the learning rate for each model layer.
| Alias | localRates |
|---|
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
momentum=double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
scaleFactor=double
the scaling coefficient that is used to adjust the base learning rate.
| Default | 0.001 |
|---|---|
| Range | 0–1 |
steps=list(64-bit-integer-1 <, 64-bit-integer-2, ...>)
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
warmup=64-bit-integer
specifies the number of epochs to complete before applying layer-wise learning rates.
| Default | -1 |
|---|---|
| Minimum value | -1 |
Parameters for method="LBFGS"
backtrackRatio=double
specifies the backtrack ratio of line search iterations for the L-BFGS solver.
| Range | [0–1) |
|---|
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
m=64-bit-integer
specifies the number of corrections used in the L-BFGS update.
| Minimum value | 0 |
|---|
maxLineSearchIters=64-bit-integer
Specifies the maximum number of line search iterations for L-BFGS solver.
| Minimum value | 1 |
|---|
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps=list(64-bit-integer-1 <, 64-bit-integer-2, ...>)
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="MOMENTUM" | "MOMENTUMSGD"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
momentum=double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps=list(64-bit-integer-1 <, 64-bit-integer-2, ...>)
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="NATGRAD" | "RIEGRAD"
approximateType=64-bit-integer
specifies the block-diagonal approximation of the Fisher information matrix.
| Default | 1 |
|---|---|
| Range | 1–1 |
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps=list(64-bit-integer-1 <, 64-bit-integer-2, ...>)
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="TRISH" | "TRISHSGD"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
gamma1=double
Gamma1 scaling for the gradient
| Default | 2 |
|---|---|
| Range | [0–10000) |
gamma2=double
Gamma2 scaling for the gradient
| Default | 0.5 |
|---|---|
| Range | [0–10000) |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
momentum=double
specifies the momentum parameter value for Stochastic Gradient Descent (SGD).
| Default | 0.9 |
|---|---|
| Range | 0–1 |
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps=list(64-bit-integer-1 <, 64-bit-integer-2, ...>)
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|
Parameters for method="VANILLA" | "VANILLASGD"
clipGradMax=double
specifies the maximum gradient value. All gradient values that exceed the specified maximum value are set to the specified maximum value.
| Minimum value | 0 |
|---|
clipGradMin=double
specifies the minimum gradient value. All gradient values that are less than the specified minimum value are set to the specified minimum value.
| Maximum value | 0 |
|---|
fcmpEarlyStopping="string"
specifies FCMP early stopping function.
| Alias | fcmpES |
|---|
fcmpLearningRate="string"
specifies the FCMP learning rate function.
| Alias | fcmpLR |
|---|
gamma=double
specifies the gamma value for the learning rate policy.
| Default | 0.1 |
|---|---|
| Range | 0–100 |
learningRate=double
specifies the learning rate for Stochastic Gradient Descent (SGD).
| Default | 0.001 |
|---|---|
| Range | 0–MACBIG |
learningRatePolicy="FIXED" | "INV" | "MULTISTEP" | "POLY" | "STEP"
specifies the learning rate policy.
| Default | FIXED |
|---|
INV
sets the learning rate parameter value after each epoch, according to the initial learning rate, the value of the gamma parameter, and the value of the power parameter. The rate is calculated as the learningRate*(1+gamma*current epoch)^(-power).
MULTISTEP
sets the learning rate after each of the epochs specified in the steps parameter. The updated learning rate is the product of the previous learning rate value and the gamma parameter value.
power=double
specifies the power for the learning rate policy.
| Default | 0.75 |
|---|---|
| Minimum value | 0 |
steps=list(64-bit-integer-1 <, 64-bit-integer-2, ...>)
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
stepSize=64-bit-integer
specifies the step size when the learning rate policy is set to STEP.
| Default | 10 |
|---|---|
| Minimum value | 1 |
useLocking=TRUE | FALSE
when set to False, computes gradients asynchronously with multiple threads.
| Default | TRUE |
|---|