Download Identity Tables, Metadata Tables, and SAS 360 Plan Tables
The nonPartitionedData endpoint of the Download API enables you to download identity tables, metadata tables, and plan tables from the system:
- Identity tables contain
the identity data that SAS Customer Intelligence
360 collects using customers’ presence. For
more information, see About Identity Database Tables in SAS Customer Intelligence 360: Data Dictionary.
Note: The IDENTITY_MAP table contains only one year of historical data. When you download this table, do not replace the existing table in your on-premises system. Instead, you can use the processed_dttm column to determine which newer records can be added to your existing data.
- Metadata tables are non-partitioned detail tables that contain information about published items (active or complete). This information includes metadata for tables in the Unified Data Model (UDM) and Common Data Model (CDM).
- SAS 360 Plan tables contain information about planning items, assets, and workflows. For example, these tables can include information about budgets, commitments, invoices, and associations between planning items and tasks, creatives, or messages. For more information about these tables, see Plan and Workflow Detail Tables in SAS Customer Intelligence 360: Data Dictionary.
- The downloaded data might contain
more columns than are documented in the Help Center or in the schema files.
These columns might have been introduced for internal consumption already
and will be added to a future version of the schema.
The same scenario can happen when you are not using the most recent schema. The data that you download always uses the columns from the latest schema, but you might have requested a schema version that has fewer columns.
- A schema might add columns to
tables, but the columns are not populated with data until a later release of
SAS Customer Intelligence
360. For example, a table in the current schema might have nine
columns, but data records populate only seven columns. When columns are
added but not yet populated with data, you can ignore these columns when you
process the data.
When SAS Customer Intelligence 360 is updated to populate these columns, you might need to wait 2–3 days before the data is populated in the downloaded tables.
To mitigate any issues, configure your ETL processes to allow for the physical files to have more columns than what is described in the JSON schema and the documentation.
To download these tables:
- Set up authorization for the REST
API. For REST API calls, in most situations the access point type is going
to be a General Access Point.
When you create the access point, make note of the value in the External gateway host field. Use this value when you see the variable <external gateway host> in the documentation.
- Send a GET call similar to this
example:
GET https://<external gateway host>/marketingGateway/discoverService/dataDownload/ eventData/detail/nonPartitionedData?schemaVersion=23Note: The files that are provided by the download links are delimited files that represent the structure of the tables from the schema that existed when those files were created. The download links are only to the tables of the schema that is requested, but the structure of those files conforms to the most recent schema version. For more information, see Schemas for Data Tables.This GET call can use these query parameters in the URL:
Parameters for Snapshot Data Name
Required or Optional
Data Type
Description
start
Optional
Integer
The start index of the items to return in the current response. The default value is
0.limit
Optional
Integer
The maximum number of items to return in the current response.
For the SAS Customer Intelligence 360 Download API, the limit parameter is restricted to a maximum value of
100. If you specify a value that is higher than the maximum, the limit is reset to the default value of20.When the count parameter (which is the total number of items that exist) is larger than the limit, the response returns only the first page of items in the response (or a specific page based on the start parameter). When results are paginated, the links in the response contain entries for the next, previous, and last pages of items that are available.
Tip Use the start and limit parameters to control the response based on your use case. If you want all of the items in a single response, set the limit parameter to a number that is equal to or above the total count.
However, if items in the response have a short-lived URL, you might want to set the limit to a smaller value. Then, you can process the smaller set of items before moving to the next page.
category
Optional
String
When you download tables with schema 4 or later, the category corresponds to the set of tables to download. Categories correspond to license offerings.
Note: Identity tables are returned for any category value, but metadata tables are category-specific.
These are the possible values:
ALL. This category value enables you to download all of the tables that you have access to.ENGAGEMETADATA. This category requires a license for SAS 360 Foundations, SAS 360 Direct, SAS 360 Audiences Premium, or SAS 360 Email. To view the list of tables for this category, see Tables by Category - ENGAGEMETADATA.METADATA_ALL. This category requires a license for SAS 360 Foundations, SAS 360 Direct, SAS 360 Audiences Premium, or SAS 360 Email. To view the list of tables for this category, see Tables by Category - METADATA_ALL.PLAN. This category requires a license for SAS 360 Plan. To view the list of tables for this category, see Tables by Category - PLAN.USAGE. This category is available for all licenses. To view the list of tables for this category, see Tables by Category - USAGE.
Set multiple category values to download only the tables that you want. You can separate multiple values with commas. For example:
category=ENGAGEMETADATA,USAGEFor more information, see Categories for Data Tables.
schemaVersion
Optional
Integer
The schema that the tables use. Use an integer that is
1or in the range3–23.The default value is
1, which is the schema that corresponds to the original data tables from SAS 360 Discover.Note: Schema 4 or later is required to download metadata tables. Identity tables are unchanged between schemas, so any schema returns the same set of tables.
Downloads with schema 4 and later are restricted by the values that are set for the category parameter. For more information, see Schemas for Data Tables.
Note: When you download tables with the nonPartitionedData endpoint, the query parametersdataRangeStartTimeStampanddataRangeEndTimeStampare not supported. - Download the tables from the JSON response.
These are the most relevant parts of the response:
- the top-level of the response includes the
start, count, and limit parameters. However, for this set of tables (which are
non-partitioned tables), the count value is always
1and does not represent the number of tables in the response. You cannot paginate the response by using the start or limit parameters. - the download URLs for each table are nested
in the top-level items object. See the following example for the structure of the
JSON
return and the URL location:
{ ... "items": [ { "schemaUrl": "<schema URL>", "schemaVersion": "23.0", "entities": [ { "entityName": "<table>", "dataUrlDetails": [ { "url": "<temporary URL>", "lastModifiedTimestamp": "<last modified timestamp>" }, { "url": "<temporary URL 2>", "lastModifiedTimestamp": "<last modified timestamp>" } ] }, ... More tables are listed here ... ] } ] ... }If you plan to process the response programmatically, be aware of these considerations:
- The response structure is different from the response that is used for detail tables and DBTs. Identity and metadata tables are not partitioned, so any download file contains the entire table’s data the corresponds to that timestamp.
- For identity and metadata tables, each table’s entry includes a lastModifiedTimestamp value in UTC format. If you keep a record of the lastModifiedTimestamp values, these can be used to determine when to download new tables.
- If you want to download the schema, you need to download the schema only once. That is, you get the same schema file regardless of which schemaUrl value you process.
For working script examples, see Best Practices for Using the Download API.
- the links list is a top-level object in the JSON response that enables you to navigate to different pages of table returns. For more information, see Control the REST Response with Query Parameters.
- the top-level of the response includes the
start, count, and limit parameters. However, for this set of tables (which are
non-partitioned tables), the count value is always
- If you downloaded the CDM tables, you can use a program that is provided by SAS to load this data into a third-party database. For more information, see SAS Customer Intelligence 360 CDM Loader (external) on GitHub.