HADOOP Procedure

Example 2: Submitting HDFS Commands with Wildcard Characters

Features:

SAS_HADOOP_CONFIG_PATH environment variable

SAS_HADOOP_JAR_PATH environment variable

SAS_HADOOP_RESTFUL environment variable

PROC HADOOP statement

HDFS statement

Wildcard characters

OPTIONS statement

SET system option

Details

This PROC HADOOP example submits HDFS commands to a Hadoop server. The statements display the contents of the specified files, change the permissions for one HDFS file, and list the files in a specified HDFS pathname.

Program

options set=SAS_HADOOP_CONFIG_PATH="configuration-files-pathname";
options set=SAS_HADOOP_JAR_PATH="JAR-files-pathname";
options set=SAS_HADOOP_RESTFUL 1;
proc hadoop username='sasabc' password='sasabc' verbose;
   hdfs cat='/user/sasabc/*';
   hdfs chmod='/user/sasabc/' permission=rwxr-xr-x;
   hdfs ls='/user/sasabc/*';
run;

options set=SAS_HADOOP_CONFIG_PATH="configuration-files-pathname";
options set=SAS_HADOOP_JAR_PATH="JAR-files-pathname";
options set=SAS_HADOOP_RESTFUL 1;
proc hadoop username='sasabc' password='sasabc' verbose;
   hdfs cat='/user/sasabc/*';
   hdfs chmod='/user/sasabc/' permission=rwxr-xr-x;
   hdfs ls='/user/sasabc/*';
run;

Program Description

Define the SAS_HADOOP_CONFIG_PATH environment variable, the SAS_HADOOP_JAR_PATH environment variable, and the SAS_HADOOP_RESTFUL environment variable. The OPTIONS statements include the SET system option to define the environment variables. The first two environment variables set the location of the Hadoop cluster configuration files and the Hadoop JAR files so that the required files are available to the SAS session. The SAS_HADOOP_RESTFUL environment variable specifies to connect to the Hadoop server by using the WebHDFS REST API.

options set=SAS_HADOOP_CONFIG_PATH="configuration-files-pathname";
options set=SAS_HADOOP_JAR_PATH="JAR-files-pathname";
options set=SAS_HADOOP_RESTFUL 1;

Execute the PROC HADOOP statement.The PROC HADOOP statement controls access to the Hadoop server by identifying the user ID and password on the Hadoop server. The statement specifies the VERBOSE option, which enables additional messages to be written to the SAS log.

proc hadoop username='sasabc' password='sasabc' verbose;

Display the contents of HDFS files.The first HDFS statement specifies the CAT= option to display the contents of HDFS files. The wildcard character * specifies to match one or more characters. All files that are contained in the directory /user/sasabc/ are displayed in the SAS log.

   hdfs cat='/user/sasabc/*';

Change the file access permissions.The second HDFS statement specifies the CHMOD= option to change the file access permissions for the specified HDFS pathname. The file access permissions provide the owner with Read, Write, and Execute permission, group members with Read and Execute permission, and users with Read and Execute permission.

   hdfs chmod='/user/sasabc/' permission=rwxr-xr-x;

List the files in an HDFS pathname.The third HDFS statement specifies the LS= option to list the files in the specified HDFS pathname to the SAS log. The wildcard character * specifies to match one or more characters. All files that are contained in the directory /user/sasabc/ are displayed in the SAS log. The output for each file consists of its permissions, User ID, User ID group, file size, creation date, creation time, and the filename.

   hdfs ls='/user/sasabc/*';
run;

Last updated: September 15, 2026