DYNAMICLINEAR Procedure

When you work with SGDLMs, it is crucial to understand and plan for memory requirements, because these requirements scale with the size of the problem. For example, consider modeling a system that contains 400 variables. The adjacency matrix and the precision matrix , which are pivotal for importance sampling and prediction generation, would each have dimensions of , totaling 160,000 elements per matrix. Storing each element in double precision, which is standard practice and requires 8 bytes per element, means that a single matrix consumes approximately 1.28MB of memory, calculated by 160,000 bytes.

In Monte Carlo simulations that encompass 10,000 iterations, storing all adjacency and precision matrices would require approximately 25.6GB of memory, which is twice the product of 1.28MB per matrix and 10,000 iterations, or 12.8GB multiplied by 2. This figure does not account for the memory consumption during other steps. Adequate memory resources are paramount. If you have insufficient memory, PROC DYNAMICLINEAR cannot run.

If memory constraints present an issue, you might need to reduce the number of variables being modeled, shorten the length of time series, or decrease the number of simulations. Each of these adjustments comes with trade-offs: reducing the number of variables and shortening series length might oversimplify the model, whereas fewer simulations could compromise the statistical robustness of the forecast results. Therefore, it is essential to find an optimal point that meets the memory limitations without significantly compromising the model’s integrity and results.

Generating detailed output from the filtering and forecasting process demands significant memory resources, because these data must be retained in memory prior to being recorded in tables upon completion of the program’s execution. Exercise caution when enabling such output options. For more information, see the document of the FILTER statement and the FORECAST statement.

Currently, an SGDLM that contains approximately 350 variables is estimated to require around 50GB of memory for processing. The PROC DYNAMICLINEAR development team is actively working on memory optimization strategies. These include harnessing sparsity within the matrices, taking advantage of distributed computing capabilities, and using more memory-efficient data structures. These improvements will aim to alleviate the memory burden in large-scale applications.