NETWORK Procedure
Execution Modes and Data Movement
When you run PROC NETWORK, the execution mode and the underlying data movement that must be implemented to support that execution mode depend on which algorithm you select.
The data movement and execution modes for each algorithm are listed in Table 6. The following abbreviations are used in Table 6 for processing mode:
SM: single machine
MM: multiple machines
MT: multithreaded execution
For a single-machine CAS server, there is no data movement. The algorithm runs on the same machine where the data reside. For a multiple-machine CAS server, the assumption is that the data reside in parts on one or more of the machines in the server. The following phrases are used in Table 6 to describe data movements for a multiple-machine CAS server:
Moved to SM: Some algorithms run only in single-machine mode. In such cases, one particular machine (chosen randomly) is assigned the role of the processing machine, and the data from all the other machines are moved over to this processing machine.
Repeated on MM: Some algorithms use multiple machines, but each machine requires a global view of the input data. In such cases, each data part is repeated on all machines. Each machine processes a portion of the work across the entire graph. The resulting output tables are distributed tables.
Shuffled across MM: Some algorithms use multiple machines and require only a portion of the data (distributed graphs). However, because the original data are usually randomly distributed, the first step is to shuffle data between machines such that the data are appropriately aligned for the particular algorithm’s needs. When the data are aligned correctly, each machine processes a part of the data and then iteratively merges results across the grid to obtain the final result. Again, the resulting output tables are distributed tables.
None: Some algorithms require no data movement and process the original data (randomly distributed) directly.
Table 6: Execution Modes and Data Movement
| Statement and Options | Data Movement | Processing Mode |
|---|---|---|
| PROC NETWORK | ||
| DISTRIBUTED=FALSE | Moved to SM | SM (MT) |
| DISTRIBUTED=TRUE | None | MM (MT) |
| BICONNECTEDCOMPONENTS | Moved to SM | SM |
| BY | Shuffled across MM | MM (MT) |
| CENTRALITY | ||
| AUTH= | Moved to SM | SM (MT) |
| BETWEEN= | Repeated on MM | MM (MT) |
| CLOSE= | Repeated on MM | MM (MT) |
| DEGREE= | Moved to SM | SM |
| DEGREE= (with DISTRIBUTED=TRUE) | None | MM |
| INFLUENCE= | Moved to SM | SM |
| Otherwise | Moved to SM | SM (MT) |
| CLIQUE | Moved to SM | SM (MT) |
| COMMUNITY | ||
| ALGORITHM=LABELPROPAGATION † | ||
| with LABELUPDATEMODE=ASYNCHRONOUS | Moved to SM | SM |
| with LABELUPDATEMODE=SYNCHRONOUS | Shuffled across MM | MM (MT) |
| ALGORITHM=LOUVAIN | Moved to SM | SM |
| CONNECTEDCOMPONENTS | ||
| ALGORITHM=DFS | Moved to SM | SM |
| ALGORITHM=UNIONFIND (with DISTRIBUTED=TRUE) | None | MM |
| Otherwise | Moved to SM | SM (MT) |
| CORE | Moved to SM | SM (MT) |
| CYCLE | ||
| ALGORITHM=BACKTRACK | Moved to SM | SM |
| ALGORITHM=BUILD | Moved to SM | SM (MT) |
| NODESIMILARITY | Repeated on MM | MM (MT) |
| PATH | Repeated on MM | MM (MT) |
| PATTERNMATCH | ||
| With QUERYKEY= | Repeated on MM | MM (MT) |
| Without QUERYKEY= | Moved to SM | SM (MT) |
| PROJECTION | Repeated on MM | MM (MT) |
| REACH | Repeated on MM | MM (MT) |
| SHORTESTPATH | Repeated on MM | MM (MT) |
| SUMMARY | Moved to SM | SM (MT) |
| TOPOLOGICALSORT | Moved to SM | SM |
| TRANSITIVECLOSURE | Moved to SM | SM (MT) |
† To specify a distributed graph algorithm, you can also specify DISTRIBUTED=TRUE in the PROC NETWORK statement.
In addition, on each machine, some algorithms (in addition to the input phase) take advantage of multicore chip technology by executing on multiple threads simultaneously. You can use the NTHREADS=number option in the PROC NETWORK statement to specify the number of threads to use. The default is the number of cores on the machine that executes the process or the number of cores permissible based on your installation (whichever is less). Specifying a number greater than the number of available cores might hurt performance. Specifying a high number does not guarantee shorter solution time; the actual change in solution time depends on the computing hardware and the scalability of the underlying algorithms. In some circumstances, the NETWORK procedure might use fewer threads than the specified number because the procedure’s internal algorithms have determined that a smaller number is preferable.
In the case of BY-group processing, the data must first be partitioned such that each observation within a BY group resides on the same machine. If the data are not already partitioned, PROC NETWORK shuffles the data appropriately as a first step. When the data are partitioned, the selected algorithm runs against the groups (on each machine) by using multiple threads (one group per thread). If the algorithm itself is a multithreaded algorithm, then it uses multiple threads (on each group) if and only if the value of the NTHREADS= option exceeds the number of groups that are assigned to the processing machine. In this case, the resulting output tables are distributed tables (partitioned by group). You can partition your input data in advance by using the PARTITION= option in a DATA step. Partitioning in advance avoids the need for PROC NETWORK to shuffle the data.
Because of communication costs, increasing the number of machines does not guarantee faster execution, especially when you are dealing with small graphs. For all the documentation examples, the CAS session is configured for 16 worker nodes, each with 80 cores, unless otherwise noted. For general information about CAS sessions, see SAS Cloud Analytic Services: Fundamentals.