What’s New in SAS Grid Manager 9.4

Overview

SAS Grid Manager has the following new features and enhancements:

• In SAS 9.4M9, Grid Management Services (GMS) and the Grid Manager plug-in for SAS Management Console are no longer available. Instead, use SAS Environment Manager.
• When a release of SAS 9 with SAS Grid Manager for Platform is installed on Red Hat Enterprise Linux version 8 or later, you must install the libnsl library. See Updating Red Hat Enterprise Linux.
• In the SAS 9.4M7 security update, Log4J version 1 is replaced by Reload4j.
• In SAS 9.4M8, Log4J2.default release 2.17.1 is used.
• In SAS 9.4M8, the nmap command is used instead of the telnet command.
• In SAS 9.4M8, new fields are available when specifying general settings for a SAS Workload Orchestrator configuration.
• In SAS 9.4M6, a new grid provider has been added. It consists of SAS Workload Orchestrator and SAS Job Flow Scheduler. The product name is SAS Grid Manager.
• In SAS 9.4M6, the product name for a grid that uses Platform Suite for SAS has been changed to SAS Grid Manager for Platform.
• Support for grid options sets has been added.
• Support for grid-launched workspace servers has been added.
• New options have been added to the SAS Grid Manager Client Utility, including support for grid options sets, support for interactive SAS sessions and command processing, support for waiting for results from grid jobs, and support for submitting a SAS program located on a grid.
• New job options for Platform Suite for SAS have been added.
• SAS Grid Manager for Hadoop adds support for SAS grid processing co-located on a Hadoop cluster.
• In SAS 9.4M3, a SAS Grid Manager agent plug-in for SAS Environment Manager was added to support SAS Grid Manager for Platform. In SAS 9.4M5, metric data for grid queues was added to the SAS Grid Manager agent plug-in for SAS Environment Manager. In SAS 9.4M6, SAS Grid Manager agent plug-ins for SAS Environment Manager were added to support SAS Grid Manager and SAS Grid Manager for Platform.
• In SAS 9.4M3, a SAS Grid Manager module was added to SAS Environment Manager for SAS Grid Manager for Platform. In SAS 9.4M5, functions were added to specify dependencies and affinity requirements for high availability (HA) configurations, to update the LSF license, and to terminate jobs, requeue jobs, change the position of jobs within a queue, and switch an uncompleted job to a different queue. In 9.4M6, the module’s interface has been completely redesigned, although the functions provided remained the same.
• The GRDSVC_HOSTLIST and GRDSVC_OPTSETS functions have been added.
• In SAS 9.4M5, shortened option names are supported for the SAS Grid Manager Client Utility.
• In SAS 9.4M7, options have been added to the SAS Grid Manager Client Utility to support digital certificates if you are using a SAS Workload Orchestrator grid or a Hadoop cluster and the grid or cluster has TLS/SSL enabled.

New Grid Provider

In SAS 9.4M6, a new grid provider, provided by SAS, has been added. SAS Grid Manager consists of SAS Workload Orchestrator and SAS Job Flow Scheduler. SAS Grid Manager uses queues, host parameters, and user-specified limits to distribute jobs among hosts on the grid. High availability is supported through failover capability for the master host and defined services. The grid is configured and managed through the SAS Workload Orchestrator web interface.

Support for Grid Options Sets

A grid options set is a collection of grid options, SAS options, and required resources that are associated with a particular SAS client application. The purpose of a grid options set is to enable a SAS grid administrator to define a collection of options in SAS metadata. These options map to one or more SAS client applications and are automatically applied to workloads submitted to the grid based on the identity of the user accessing the client application. Some of the client applications that support grid options sets include the following:

• SAS Data Integration Studio
• SAS Grid Manager Client Utility
• SAS sessions that use the grdsvc_enable statement to submit jobs to the grid
• Grid-launched servers

Support for Grid-Launched Servers

With grid-launched servers, when an application requests a SAS workspace server from the object spawner, the spawner passes the request on to SAS Grid Manager. SAS Grid Manager checks the machines on the grid, using the grid’s policy management to determine which host is the least busy and starts the server.

Because load-balanced servers are started by the grid provider, jobs on these servers appear as grid jobs.

In SAS 9.4M1, stored process servers and pooled workspace servers can also be grid-launched.

New Options for the SAS Grid Manager Client Utility

The following new options were added to the SAS Grid Manager Client Utility (SASGSUB):

-GRIDOPTSET

specifies the name of a grid options set.

-GRIDWAITTIMEOUT

specifies that the SAS Grid Manager Client Utility waits until either the job has completed running or a specified time-out value is reached.

-GRIDWAITRESULTS

specifies that the SAS Grid Manager Client Utility waits until the job has completed running (either successfully or with an error) and then returns the results of the processing.

_PROMPT_

when used as a value for the METAUSER or GRIDUSER parameters, specifies that the SAS Grid Manager Client Utility prompts the user for a user ID.

In SAS 9.4M1, support was added to the SAS Grid Manager Client Utility to support interactive mode. These options enable you to start an interactive SAS session on the grid, using either SAS Display Manager mode or UNIX line mode, or to run an arbitrary command. It also enables you to monitor the output from SAS or a command running in batch mode.

-GRIDRUNSASLM

specifies that the SASGSUB command runs a SAS session in line mode and that standard output and standard error are displayed by SASGSUB. SASGSUB also redirects standard input to the SAS session running on the grid. The grid must be on UNIX.

-GRIDRUNSASDMS

specifies that the SASGSUB command runs a SAS session in SAS Display Manager mode. Standard output and standard error are displayed by SASGSUB, and the X Windows session is directed to the host and port specified on the -GRIDRUNSASDMS argument. The grid must be on UNIX.

-GRIDRUNCMDINT

specifies that an arbitrary command is run on the grid in interactive mode. Standard output and standard error are displayed by SASGSUB. Standard input is directed to the program running on the grid.

-GRIDWATCHOUTPUT

when SASGSUB is used for batch processing, specifies that the log output and the list output of the batch job are displayed on the user’s machine.

In SAS 9.4M3, options were added to the SAS Grid Manager Client Utility to control whether the job directory, job log, and output listing are retained or deleted after processing.

-GRIDWAITLOGLIST

specifies that, after a job has completed running, and after the job log and output listing are obtained, the job directory is deleted.

-GRIDWAITNORESULTS

specifies that, after a job has completed running and the results of the processing are returned, the job log, output listing, and job directory are deleted.

-GRIDWAITNORESULTSNOLOG

specifies that, after a job has completed running and the results of the processing are returned, the job log, output listing, and job directory are deleted, and the log and list files are not copied to the current directory.

In SAS 9.4M5, an option was added to the SAS Grid Manager Client Utility to submit a SAS program that exists on a shared file system and that therefore is shared across all grid nodes, rather than on an internal file system and accessible only by one host. Because the program is already on a shared file system, it does not have to be copied to a grid share when it runs.

-GRIDRUNPGM

specifies the path and file name of a program that you want to run on the grid. The program is stored on a grid node machine, and is not accessible to users on a client machine.

In SAS 9.4M7, options were added to the SAS Grid Manager Client Utility to support digital certificates if you are using either:

• a SAS Workload Orchestrator grid with TLS/SSL enabled
• a Hadoop cluster with TLS/SSL enabled

-SSLCALISTLOC

specifies the location of a single file that contains the public certificates for all of the trusted certificate authorities (CA) in the trust chain.

-SSLCERTISS

specifies the name of the issuer of the digital certificate that should be used by TLS.

-SSLCERTLOC

specifies the location of a file that contains a digital certificate for the machine's public key.

-SSLCERTSERIAL

specifies the serial number of the digital certificate that should be used by TLS.

-SSLCERTSUBJ

specifies the subject name of the digital certificate that should be used by TLS.

-SSLCLIENTAUTH

specifies that the server should perform client authentication.

-SSLCRLCHECK

specifies that certificate revocation lists (CRLs) are checked when digital certificates are validated.

-SSLCRLLOC

specifies the location of a file that contains a CRL.

-SSLPKCS12LOC

specifies the location of the PKCS #12 DER encoding package file that contains the certificate and the private key.

-SSLPKCS12PASS

specifies the password that TLS requires in order to decrypt the PKCS #12 DER encoding package file.

-SSLPVTKEYLOC

specifies the location of the file that contains the private key that corresponds to the digital certificate that was specified by using the SSLCERTLOC= option.

-SSLPVTKEYPASS

specifies the password that TLS requires in order to decrypt the private key.

New Job Options

The following job options that are supported by Platform Suite for SAS were added:

app

specifies the LSF application name to use.

maxpendingtime

specifies how long a job waits in the pending state before being terminated.

jobname

specifies a job name to be used for all jobs.

SAS Grid Manager for Hadoop

In SAS 9.4M3, SAS Grid Manager for Hadoop was added. SAS Grid Manager for Hadoop provides workload management, accelerated processing, and scheduling of SAS analytics co-located on a Hadoop cluster. SAS Grid Manager for Hadoop leverages YARN to manage resources and distribute SAS analytics to a Hadoop cluster running multiple applications. It integrates with Oozie, which provides scheduling capability for SAS workflows. All of the existing SAS Grid syntax, submission modes, and integration with other SAS products and solutions is supported by SAS Grid Manager for Hadoop.

SAS Grid Manager for Platform Agent Plug-in for SAS Environment Manager

In SAS 9.4M3, a SAS Grid Manager for Platform agent plug-in was added to SAS Environment Manager. The plug-in monitors your SAS Grid Manager for Platform grid resources over time and generates events and alerts based on the monitored information.

The SAS Grid Manager for Platform agent plug-in for SAS Environment Manager provides metric data for the grid cluster and for individual grid hosts, and uses that data to perform these functions:

• display the current state of grid resources
• graph the data over time, providing a historical view and enabling you to see how the data changes
• create alerts that notify you whenever a selected measurement reaches a selected state

In SAS 9.4M5, the agent plug-in also provides metric data for grid queues.

In SAS 9.4M6, the name of the plug-in has been changed from “SAS Grid Manager agent plug-in” to “SAS Grid Manager for Platform agent plug-in”.

SAS Grid Manager Agent Plug-in for SAS Environment Manager

In SAS 9.4M6, a SAS Grid Manager agent plug-in for SAS Environment Manager has been added. It provides the same functions as the SAS Grid Manager for Platform agent plug-in, but for a SAS Grid Manager environment. SAS Grid Manager consists of SAS Workload Orchestrator and SAS Job Flow Scheduler.

SAS Grid Manager Module for SAS Environment Manager

In SAS 9.4M3, a SAS Grid Manager module was added to SAS Environment Manager.

The SAS Grid Manager module enables you to configure and perform actions on grid resources and high availability applications, and provides a view of your grid at the current moment in time. The functions provided by the module are divided into these areas:

Monitoring

• perform actions on grid resources (such as opening and closing hosts and activating and deactivating queues)
• view information for the cluster, grid hosts, queues, jobs, high-availability applications, and audit logs
• manage high availability applications

In 9.4M5, these additional functions were added to manage jobs:

• terminate and requeue jobs
• switch an uncompleted job to a different queue
• change the position of jobs in the queue

LSF Configuration Manager

modify the configuration information for the LSF cluster, including definitions for hosts, queues, users, administrators, resource limits, and cluster parameters

HA Configuration Manager

modify the configuration information for high availability applications

In SAS 9.4M5, functions were added to specify dependencies and affinity requirements. A dependent application is one that either is started or continues to run according to the state of the HA application or must be started before the HA application starts. An affinity requirement specifies where a selected application runs in relation to other applications in the HA configuration.

Utilities, added in 9.4M5 (available for LSF administrator only)

update the LSF license

New Functions

In SAS 9.4M3, these functions were added:

GRDSVC_HOSTLIST

This function returns a list of hosts that are available to run grid jobs.

GRDSVC_OPTSETS

Returns a list of valid options sets for a specified combination of SAS Application Server, SAS client application, and user.

Shortened SAS Grid Manager Client Utility Option Names

In SAS 9.4M5, alternate shortened forms of most SAS Grid Manager Client Utility options were added. Both the long and short versions of the option name are valid. In addition, several options have multiple valid forms. Here are the shortened option names: