DYNAMICLINEAR Procedure
Precision of GPU Computation in PROC DYNAMICLINEAR
By default, the GPU workflow in PROC DYNAMICLINEAR uses single-precision floating-point arithmetic. This can lead to differences in the results compared to the CPU workflow, which uses double-precision arithmetic. The precision of GPU computation can affect the accuracy of the predictions and the convergence of the model, especially in cases where the data has a large range or the model is complex. It is important to understand these differences and consider them when you interpret the results from PROC DYNAMICLINEAR.
The GPU workflow supports three different precisions: TF32, FP32, and FP64.
TF32 uses 19 bits for the significand and 8 bits for the exponent, allowing it to represent a wide range of values while maintaining a reasonable level of precision.
FP32, or single-precision floating-point format, uses 23 bits for the significand and 8 bits for the exponent, providing higher precision than TF32 but with increased computational requirements.
FP64, or double-precision floating-point format, uses 52 bits for the significand and 11 bits for the exponent, offering even higher precision but with significantly increased computational requirements.
The default precision is FP32, which provides a good balance between performance and accuracy for many applications. However, you can use FP64 for higher precision at the cost of slower computations. The choice of precision can depend on the specific requirements of the analysis and the characteristics of the data that you are analyzing. In general, it is recommended that you start with the default FP32 precision and evaluate the results before considering other precision options.
TF32 is not a C++ data type; it is purely a compute mode for NVIDIA Tensor Cores on Ampere+ GPUs. Within cuBLAS GEMM (general matrix multiplication) operations, the Tensor Core internally truncates the mantissa from 23 to 10 bits, performs multiplication in that reduced precision, and accumulates the result in FP32. The output is then written back as a regular single-precision float. Because GPU architectures are optimized around GEMM operations, TF32 can deliver substantial throughput gains (measured in tera floating-point operations per second, or TFLOPS) over FP32 and FP64, as illustrated by the following A100 Tensor Core peak throughput figures:
| Precision | GEMM TFLOPS |
|---|---|
| FP64 | 19.5 |
| FP32 | 19.5 |
| TF32 | 156 |
| FP16 | 312 |
Because PROC DYNAMICLINEAR is currently more GEMV (general matrix-vector multiplication)-intensive than GEMM-intensive, TF32 provides less benefit than it would for a workload that is dominated by GEMM operations. However, as the implementation of PROC DYNAMICLINEAR continues to evolve, future versions might use more GEMM operations, which could make TF32 a more attractive option for users who want to maximize performance on compatible GPUs.