The NETWORK Procedure

Example 2.13 Shortest Paths of the New York Road Network

This example looks at the road networks in the state of New York (NY). The distance graph raw data are maintained at the DIMACS challenge website (Demetrescu 2010). The NY road network includes 264,346 intersections (nodes) and 733,846 roads (links). Although the input data table is not large, the computing power that you need to find all-pairs shortest paths is enormous. In addition, the storage space that you need to handle the results data can easily overwhelm the capacity of a single machine. In this example, a session of 100 machines (each with 32 cores) was configured to process this graph.

The following statements construct the links data table mycas.RoadNY from a local copy of the raw distance graph data:

filename in 'USA-road-d.NY.gr';
data mycas.RoadNY (drop=a);
   infile in firstobs=8;
   input a $ from $ to $ weight;
run;

The following statements find the all-pairs shortest paths of the NY road network (that have a total path weight of less than 20,000) by using a distributed algorithm. This algorithm takes advantage of all the machines and cores in your configured session.

proc network
   logFreqTime      = 10
   logLevel         = aggressive
   direction        = directed
   links            = mycas.RoadNY;
   shortestPath
      maxPathWeight = 20000
      outWeights    = mycas.shortPathSummary
      outPaths      = mycas.shortPathPaths;
run;
%put &_NETWORK_;

The progress of the procedure is shown in Output 2.13.1.

Output 2.13.1: PROC NETWORK Log: Shortest Paths of the NY Road Network

NOTE: ------------------------------------------------------------------------------------------
NOTE: ------------------------------------------------------------------------------------------
NOTE: Running NETWORK.                                                                          
NOTE: ------------------------------------------------------------------------------------------
NOTE: ------------------------------------------------------------------------------------------
NOTE: Reading the links data.                                                                   
NOTE: Data input used 0.09 (cpu: 18.19) seconds.                                                
NOTE: Building the input graph storage used 0.39 (cpu: 62.64) seconds.                          
NOTE: The number of nodes in the input graph is 264346.                                         
NOTE: The number of links in the input graph is 733846.                                         
NOTE: Processing shortest paths problem using 3200 threads across 100 machines.                 
NOTE: Processing the shortest paths problem between 264346 source nodes and 264346 sink nodes.  
                                                    Real                                        
      Algorithm              Sources   Complete     Time                                        
      shortestPath             32320        12%     9.04                                        
      shortestPath             90240        34%    19.10                                        
      shortestPath            148160        56%    29.04                                        
      shortestPath            202880        76%    39.04                                        
      shortestPath            245466        92%    49.03                                        
      shortestPath            264346       100%    52.78                                        
NOTE: Processing the shortest paths problem used 52.78 (cpu: 125379.52) seconds.                
NOTE: The Cloud Analytic Services server processed the request in 54.430456 seconds.            
NOTE: The data set MYCAS.SHORTPATHSUMMARY has 104263396 observations and 3 variables.           
NOTE: The data set MYCAS.SHORTPATHPATHS has 1419295876 observations and 6 variables.            
STATUS=OK  PROBLEM_TYPE=SHORTESTPATH  SOLUTION_STATUS=OK  NUM_PATHS=104263396                   
CPU_TIME=125889.30  REAL_TIME=54.43                                                             


Notice that the resulting output data tables, mycas.shortPathSummary and mycas.shortPathPaths, are large distributed data tables.

Last updated: November 30, 2020