What Is SAS Event Stream Processing?
- Product Overview
- Planning Your Event Stream Processing Application
- What is an Event Stream Processing Model?
Product Overview
SAS Event Stream Processing enables you to quickly process and analyze a large number of continuously flowing events. Events are delivered through high throughput, low latency called . Those event streams are published in applications that use the following:
- classes or executables
- the C, JAVA, or Python
- SAS Event Stream Processing Streamviewer
- SAS Event Stream Processing Studio
You can embed with dedicated within new or existing applications that you write in XML. You can use the ESP client to feed event stream processing engine definitions (called ) into an . In Jupyter Lab, you can use ESPPy, an open-source package for Python, to program event stream processing models and connect to an ESP server.
Event stream processing applications can perform real-time analytics on streams of events. Typical use cases for event stream processing include but are not limited to the following:
- sensor data monitoring and management
- operational systems condition monitoring and management
- object detection and classification
- cyber security analytics
- capital markets trading systems
- fraud detection and prevention
- personalized marketing
There are three variants of SAS Event Stream Processing.
|
Product |
Description |
|---|---|
|
SAS Event Stream Processing |
This is the core product. In addition to the ESP server and ESP client , it provides the following components:
|
|
SAS Event Stream Processing for Edge Computing |
This product provides a configurable disk footprint for simplified deployment of components to smaller edge systems. Use SAS Mirror Manager, a command-line utility for synchronizing a collection of SAS software repositories, to install it. Using this utility, you select packages to install in order to control the size of the deployment on the edge device. You must install the contents of the /basic directory. The following directories provide optional additions to functionality:
For more information about SAS Event Stream Processing for Edge Computing, see SAS Event Stream Processing for Edge Computing: Deployment Guide. |
|
SAS Event Stream Processing for CAS |
This version works alongside SAS Cloud Analytic Services (CAS) and provides the ability to query event stream projects and to run models in a cluster. It uses the loadStreams action set, which is automatically installed on all systems with SAS Viya: This action set provides actions to query event stream processing projects and get snapshots of streaming data. Single–pass actions can directly consume event stream processing data and perform analyses without first putting the streaming data into a table For more information, see Using SAS Event Stream Processing with SAS Cloud Analytic Services Actions in SAS Event Stream Processing: Using SAS Event Stream Processing with Other Applications. |
SAS Event Stream Processing is shipped with SAS Event Stream Manager, a web-based client that enables you to manage your SAS Event Stream Processing environment. For more information, see SAS Event Stream Manager: Using SAS Event Stream Manager.
You can deploy SAS Event Stream Processing in a test environment with a “rapid deployment,” or deploy it with SAS Viya. You can also obtain pre-built Docker images to deploy in a Kubernetes cluster. For more information, see SAS Event Stream Processing on Linux: Deployment Guide
Planning Your Event Stream Processing Application
Conceptually, an event is something that happens at a determinable time that can be recorded as a collection of fields. As you plan your event stream processing application, answer the following questions:
- What specific event streams are published into an application, and with what protocol and format?
- What happens to the data? That is, how are event streams transformed and analyzed?
- What are the resulting event streams of interest? What applications subscribe to these event streams, and in what format and protocol?
Your answers to these questions enable you to plan the structure of your event stream processing model.
What is an Event Stream Processing Model?
An event stream processing model specifies how input event streams from publishers are transformed and analyzed into meaningful resulting event streams consumed by subscribers. The following figure depicts the model hierarchy.
The Event Stream Processing Model Hierarchy
-
At the top of the model hierarchy is the engine. Each model contains only one engine instance with a unique name. The ESP server is an engine instance.
-
The engine contains one or more projects, each uniquely named. Projects run in a dedicated thread pool whose size is defined as a project attribute. You can specify a port so that projects can be spread across network interfaces for throughput scalability. Using a pool of threads in a project enables the event stream processing engine to use multiple processor cores for more efficient parallel processing.
-
A project contains one or more . A continuous query is represented by a . This graph is a set of connected nodes that follow a direction down one or more parallel paths. Continuous queries are data flows, which are data transformations and analysis of incoming event streams.
-
Each query has a unique name and begins with one or more .
-
Source windows are typically connected to one or more . Derived windows can detect patterns in the data, transform the data, aggregate the data, analyze the data, or perform computations based on the data. They can be connected to other derived windows.
-
Windows are connected by edges, which have an associated direction. In this context, edges are a program element that specifies connectivity between two or more windows.
-
Connectors publish or subscribe event streams to and from an engine. Connectors are in-process to the engine.
-
The publish/subscribe API can be used to subscribe to an event stream window either from the same machine or from another machine on the network. Similarly, the publish/subscribe API can be used to publish event streams into a running event stream processor project Source window.
-
Adapters are stand-alone executable programs that can be networked. Adapters use the publish/subscribe API to publish event streams to do the following:
- publish event streams to Source windows
- subscribe to event streams from any window
Several objects in the modeling layers measure time intervals in microseconds. The following intervals are measured in milliseconds:
- time-out period for patterns
- retention period in time-based retention
- pulse intervals for periodic window output
Most non-real-time operating systems have an interrupt granularity of approximately 10 milliseconds. Thus, specifying time intervals smaller than 10 milliseconds can lead to unpredictable results.