The NETWORK Procedure
Example 2.13 Shortest Paths of the New York Road Network
This example looks at the road networks in the state of New York (NY). The distance graph raw data are maintained at the DIMACS challenge website (Demetrescu 2010). The NY road network includes 264,346 intersections (nodes) and 733,846 roads (links). Although the input data table is not large, the computing power that you need to find all-pairs shortest paths is enormous. In addition, the storage space that you need to handle the results data can easily overwhelm the capacity of a single machine. In this example, a session of 100 machines (each with 32 cores) was configured to process this graph.
The following statements construct the links data table mycas.RoadNY from a local copy of the raw distance graph data:
filename in 'USA-road-d.NY.gr';
data mycas.RoadNY (drop=a);
infile in firstobs=8;
input a $ from $ to $ weight;
run;
The following statements find the all-pairs shortest paths of the NY road network (that have a total path weight of less than 20,000) by using a distributed algorithm. This algorithm takes advantage of all the machines and cores in your configured session.
proc network
logFreqTime = 10
logLevel = aggressive
direction = directed
links = mycas.RoadNY;
shortestPath
maxPathWeight = 20000
outWeights = mycas.shortPathSummary
outPaths = mycas.shortPathPaths;
run;
%put &_NETWORK_;
The progress of the procedure is shown in Output 2.13.1.
Output 2.13.1: PROC NETWORK Log: Shortest Paths of the NY Road Network
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: Running NETWORK. |
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: ------------------------------------------------------------------------------------------ |
| NOTE: Reading the links data. |
| NOTE: Data input used 0.09 (cpu: 18.19) seconds. |
| NOTE: Building the input graph storage used 0.39 (cpu: 62.64) seconds. |
| NOTE: The number of nodes in the input graph is 264346. |
| NOTE: The number of links in the input graph is 733846. |
| NOTE: Processing shortest paths problem using 3200 threads across 100 machines. |
| NOTE: Processing the shortest paths problem between 264346 source nodes and 264346 sink nodes. |
| Real |
| Algorithm Sources Complete Time |
| shortestPath 32320 12% 9.04 |
| shortestPath 90240 34% 19.10 |
| shortestPath 148160 56% 29.04 |
| shortestPath 202880 76% 39.04 |
| shortestPath 245466 92% 49.03 |
| shortestPath 264346 100% 52.78 |
| NOTE: Processing the shortest paths problem used 52.78 (cpu: 125379.52) seconds. |
| NOTE: The Cloud Analytic Services server processed the request in 54.430456 seconds. |
| NOTE: The data set MYCAS.SHORTPATHSUMMARY has 104263396 observations and 3 variables. |
| NOTE: The data set MYCAS.SHORTPATHPATHS has 1419295876 observations and 6 variables. |
| STATUS=OK PROBLEM_TYPE=SHORTESTPATH SOLUTION_STATUS=OK NUM_PATHS=104263396 |
| CPU_TIME=125889.30 REAL_TIME=54.43 |
Notice that the resulting output data tables, mycas.shortPathSummary and mycas.shortPathPaths, are large distributed data tables.