DOCUMENTATION
Sign In

Get access to My SAS, trials, communities and more.

DS2 Programmer’s Guide
 

What’s New in SAS DS2 Programmer’s Guide

Overview

DS2 is a SAS proprietary programming language that is appropriate for advanced data manipulation. DS2 is included with Base SAS and SAS Viya and intersects with the SAS DATA step. It also includes additional data types, ANSI SQL types, programming structure elements, and user-defined methods and packages.

Changes and enhancements for SAS 9.4 and SAS Viya releases are detailed in the following sections.

DS2 Loggers

Beginning in February 2021, a new DS2 runtime logger is available on SAS Viya 3.5. The logger, App.TableServices.DS2.RunTime.TraceVariables, enables you to trace variable activity in DATA, PACKAGE, and THREAD program blocks. The logger is used with the TRACEVARIABLES option, which identifies the program blocks that are to be examined. For more information, see Run-Time Loggers and Example: Logging Trace Variables.

SCOREACCEL Procedure

In SAS 9.4M6, a new procedure, SCOREACCEL, provides an interface to the CAS server for DATA step and DS2 model publishing and scoring. Models can be published and run in CAS or in Hadoop or Teradata.

SAS Viya 3.4 has the following changes and enhancements:

  • Arguments to delete models from an external data source.
  • An option in the RunModel statement to run your scoring model with either MapReduce or Spark.
  • Arguments to provide an analytic store without a DS2 model program as input to the publishModel action. The DS2 model program is then generated in CAS by the publishModel action.

In SAS Viya 3.3, a new procedure, SCOREACCEL, provides an interface to the CAS server for DATA step and DS2 model publishing and scoring. Models can be published and run in CAS or in Hadoop or Teradata.

DSTODS2 Procedure

In SAS 9.4M5, a new procedure, DSTODS2, enables you to translate a subset of your SAS DATA step code into DS2 code. Then, if necessary, you can revise your program to take advantage of DS2 features and submit your program using PROC DS2.

Predefined DS2 Packages

The SAS Viya 3.5 release has the following new features and enhancements:

  • The documentation about the scope of package instances has been updated. The THIS and package-instance arguments of the _NEW_ operator have been deprecated and removed from the _NEW_ operator for packages. The lifetime of a package instance now depends on whether a DS2 package variable references it. The lifetime of a package instance can extend beyond the scope in which the instance itself was created.

In SAS 9.4M5, two new predefined packages are available, PCRXFIND and PCRXREPLACE, for regular expression matching and substitution. These packages are based on the PCRE 2 open-source regular expression library. The DS2 PRX functions have been deprecated because PCRXFIND and PCRXREPLACE provide superior performance to the existing PRX functions.

General Enhancements

In the SAS 9.4M9 release, SAS In-Database Code Accelerator always runs as a Spark application, for UNIX and Windows. You do not have to specify HADOOPPLATFORM=SPARK, because it is the default. MapReduce is no longer supported. Changes might be required in configuration and usage. For more information, see Overview of the SAS In-Database Code Accelerator.

In the SAS 9.4M7 release, GROUP, INDSNUM, OF, ORDER, and PARTITION have been added to the list of DS2 reserved words.

The SAS Viya 3.5 release has the following new features and enhancements:

  • Because some DS2 error messages that are written to the SAS log now contain an internal error code, an appendix that provides more information about the codes has been added to the documentation. If you contact SAS Technical Support, these internal error codes can help SAS more quickly determine the source of the problem.
  • A DS2 program that runs in CAS can create a column with a VARBINARY data type, read a VARBINARY column from a CAS table, and write a VARBINARY column to a CAS table. In addition, a DS2 program that runs in CAS can create a column with a BINARY data type. However, it cannot read a BINARY data column from a CAS table or write a BINARY column to a CAS table.
  • If you use in-database processing with the SAS Embedded Process on Teradata, note that the SASTransform table function has been deprecated. The SASTblOp table operator should be used with Teradata 14.10 or later.
  • The ability to read and write to MongoDB and Salesforce nonrelational databases with DS2 is added when appropriate SAS/ACCESS software is installed. The functionality is available when accessing the databases through a SAS library and a CAS library. For information about data type support in a SAS library, see “Data Types for MongoDB” and also “Data Types for Reading from Salesforce” and “Data Types for Writing to Salesforce” in Data Type Reference in SAS DS2 Language Reference. For information about data type support in a caslib, see Data Types for CAS in SAS DS2 Language Reference. For general information about working with the databases and details about connecting with a SAS library, see information about the databases in SAS/ACCESS for Nonrelational Databases: Reference. For information about assigning caslibs, see Data Connectors in SAS Cloud Analytic Services: User’s Guide.
  • Support for the VARBINARY data type when reading and writing to CAS tables.
  • There are improvements to data type support for various third-party relational databases when appropriate SAS/ACCESS software is installed. These features are available when the databases are accessed through a SAS library. This functionality is available in SAS Viya and in SAS 9.4M6.
    • DS2 now creates VARCHAR columns containing more than 65,535 characters as type STRING in Hive.
    • DS2 now reads and writes Teradata NUMBER columns.
    • DS2 now reads JSON columns in MySQL.

In the April 2019 release of SAS 9.4M6, the MongoDB and Salesforce non-relational databases are supported as data sources. Access is Read-only, and must be made through a SAS library. Appropriate SAS/ACCESS software must be installed.

SAS 9.4M6 has the following new features and enhancements:

  • The SAS In-Database Code Accelerator can be executed as either a MapReduce job or as a Spark application. A new system option, HADOOPPLATFORM, determines which execution platform is used. However, in SAS 9.4M6, the HADOOPPLATFORM=SPARK option is not supported on the Windows operating system for the SAS In-Database Code Accelerator.
  • User-written DS2 methods that return a DOUBLE value now return the specified missing value. Previously, the regular SAS missing value (a period) was returned.
  • CEDA processing of SPD Engine input files is supported by the SAS In-Database Code Accelerator for Hadoop.
  • SQL queries using a WHERE IN clause are now supported by the SAS In-Database Code Accelerator for Hadoop.
  • New methods are available for the HTTP package that enable you to specify a URL or proxy URL and a user name and password for those URLs. You can either specify an Open Authorization (OAuth) token or search for one in the SAS environment.
  • A RETAIN option has been added to the MERGE statement that produces a Cartesian product on a many-to-many match merge that is similar to a DATA step merge.
  • The DS2 language supports these new data sources: Spark as well as databases (such as PostgreSQL) that are compliant with JDBC.
  • The SAS In-Database Code Accelerator for Hadoop supports the SCRATCH_DB option for a Hive database that is used when a temporary table is created.

SAS Viya 3.4 has the following new features and enhancements:

  • New methods are available for the HTTP package that enable you to specify a URL or proxy URL and a user name and password for those URLs. You can either specify an Open Authorization (OAuth) token or search for one in the SAS environment.
  • A RETAIN option has been added to the MERGE statement that produces a Cartesian product on a many-to-many match merge that is similar to a DATA step merge.
  • A DS2 action enables you to delete models from an external data source.
  • An option in the runModelExternal action enables you to run your scoring model with either MapReduce or Spark.
  • DS2 action options can provide an analytic store without a DS2 model program as input to the publishModel action. The DS2 model program is generated in CAS by the publishModel action.
  • The DS2 language supports these new data sources: Spark as well as databases (such as PostgreSQL) that are compliant with JDBC.

SAS Viya 3.3 has the following changes and enhancements:

  • You can use new DS2 actions to publish and run DATA step and DS2 models in CAS or in Hadoop or Teradata. Alternatively, you can use the new SCOREACCEL procedure from the SAS client.
  • The SAS In-Database Code Accelerator for Hadoop supports the SCRATCH_DB option for a Hive database that is used when a temporary table is created.
  • DS2 supports BIGINT (INT64) and INTEGER (INT32) as well as CHAR, DOUBLE, and VARCHAR data types in the CAS server. Columns that are defined as SMALLINT and TINYINT data types in CAS are now created as INTEGER instead of DOUBLE.

SAS 9.4M5 has the following changes and enhancements:

  • When a variable is used but not declared, a warning is sent to the SAS log. The warning now indicates the data type, length, and, in some cases, precision, that is assigned to the undeclared variable.
  • In logical operations, a missing value in any expression, such as an IF expression, evaluates to False in SAS mode. A null value evaluates to neither true nor false in ANSI mode.
  • Methods with in_out parameters can have return values.
  • Methods in thread programs now allow packages to be passed in as parameters.
  • When the length of the target field of the $QUOTE format is not large enough to contain the string and its quotation marks, SAS returns as much of the string that fits into the field starting from the right most character taking into account two characters for the double quotation marks instead of blanks.

Documentation Enhancements

For SAS 9.4M8, the documentation is updated to clarify DS2 handling of hexadecimal values. For more information, see Binary Constants.

In August 2021, the documentation was enhanced to describe how DS2 loggers might be used to resolve problems. For more information, see Resolving DS2 Errors.

SAS 9.4M6 has the following changes and enhancements:

  • Documentation was added for using brackets ([[ ]]) for long string constants (NCHAR, NVARCHAR, and VARCHAR data types) during assignment. This is similar to the way that the LUA programming language delimits strings.
  • Documentation for IBM BigInsights and Pivotal has been removed. IBM BigInsights and Pivotal are no longer supported as Hadoop vendors. IBM and Pivotal asked their customers to deploy Hortonworks.

In SAS 9.4M5, the DS2 language concepts have been moved from the SAS DS2 Language Reference into this document. In addition, to provide a more comprehensive user experience, information for using DS2 with the CAS server has been incorporated into this document.

Last updated: November 17, 2025