Guaranteed Delivery

Overview to Guaranteed Delivery

The C, Java, and Python Publish/Subscribe APIs support guaranteed delivery between a single publisher and multiple subscribers. Guaranteed delivery assumes a model where each event block that is published into a source window generates exactly one event block in a subscribed window. This one block in, one block out principle must hold for all published event blocks. The guaranteed delivery acknowledgment mechanism is not aware of the event processing performed by the model.

When a publish or subscribe connection is started, a client is established to perform various publish/subscribe activities. When a publish connection is started, the number of guaranteed subscribers required to acknowledge delivery of its event blocks is specified. The time-out value used to generate negative acknowledgments upon non-receipt from all expected subscribers is also specified. Every event block injected by the publisher contains a unique 64-bit ID set by the publisher. This ID is passed back to the publisher from the publish client with every acknowledgment or negative acknowledgment in a publisher user-defined callback function. The function is registered when the publish client is started.

When a subscribe connection is started, the subscribe client is passed a set of guaranteed delivery publishers as a list of host and port entries. The client then establishes a TCP connection to each publisher on the list. This connection is then used only to transport acknowledgments specific to this publisher/subsciber pair. The subscriber calls a new Publish/Subscribe API function to trigger an acknowledgment.

Event blocks contain new host, port, and ID fields. All event blocks are uniquely identified by the combination of these fields. This enables subscribers to identify duplicate (that is, resent) event blocks.

Guaranteed Delivery Data Flow Diagram

Complicated diagram showing the guaranteed delivery data flow

Please note the following:

Publishers and subscribers that do not use the guaranteed-delivery-enabled API functions are implicitly guaranteed delivery disabled.
Guaranteed delivery subscribers can be mixed with non-guaranteed delivery subscribers.
A guaranteed delivery-enabled publisher might wait to begin publishing until a READY callback has been received. This indicates that its configured number of subscribers have established their acknowledgment connections back to the publisher.
Event blocks received by a guaranteed-delivery-enabled subscriber as a result of a snapshot generated by the engine are not acknowledged.
Under certain conditions, subscribers receive duplicate event blocks. These conditions include the following:
- A publisher begins publishing before all related subscribers have started. Any started subscriber can receive duplicate event blocks until the number of started subscribers reaches the number of required acknowledgments passed by the publisher.
- A guaranteed delivery-enabled subscriber disconnects while the publisher is publishing. This triggers the same scenario described previously.
- A slow subscriber causes event blocks to time-out, which triggers a negative acknowledgment to the publisher. In this case all subscribers related to the publisher receives any resent event blocks, including those that have already called C_dfESPGDsubscriberAck() for those blocks.
If a guaranteed delivery-enabled subscriber fails to establish its acknowledgment connection, it retries at a configurable rate up to a configurable maximum number of retries.
Suppose that a guaranteed delivery-enabled publisher injects an event block that contains an ID, and that the ID is present in the publish client’s not acknowledged-ID list. In that case, the inject call is rejected by the publish client. The ID is cleared from the list when the publish client passes it to the ACK/NACK callback function of the new publisher.

Guaranteed Delivery Success Scenario

In the context of guaranteed delivery, the publisher and subscriber are customer applications that are the endpoints in the data flow. The subscribe and publish clients are event stream processing code that implements the publish/subscribe API calls made by the publisher and subscriber.

The flow of a guaranteed delivery success scenario is as follows:

The publisher passes an event block to the publish client, where the ID field in the event block has been set by the publisher. The publish client fills in the host-port field, adds the ID to its unacknowledged ID list, and injects it to the engine.
The event block is processed by the engine and the resulting Inserts, Updates, or Deletes on subscribe windows are forwarded to all subscribe clients.
A guaranteed delivery-enabled subscribe client receives an event block and passes it to the subscriber by using the standard subscriber callback.
Upon completion of all processing, the subscribers call a new API function with the event block pointer to trigger an acknowledgment.
The subscribe client sends the event block ID on the guaranteed delivery acknowledgment connection that matches the host or port in the event block, completely bypassing the engine.
Upon receipt of the acknowledgment, the publish client increments the number of acknowledgments received for this event block. If that number has reached the threshold passed to the publish client at start-up, the publish client invokes the new guaranteed delivery callback with parameters acknowledged and ID. It removes the ID from the list of unacknowledged IDs.

Guaranteed Delivery Failure Scenarios

There are three failure scenarios for guaranteed delivery flows:

Scenario	Description
Event Block Time-out	An event block-specific timer expires on a guaranteed-delivery-enabled publish client, and the number of acknowledgments received for this event block is below the required threshold. The publish client invokes the new guaranteed delivery callback with parameters `NACK` and `ID`. No further retransmission or other attempted recovery by the publish client or subscribe client is undertaken for this event block. The publisher most likely backs out this event block and resends. The publish client removes the ID from the list of unacknowledged IDs.
Invalid Guaranteed Delivery Acknowledged Connect Attempt	A guaranteed-delivery-enabled publish client receives a connect attempt on its guaranteed delivery acknowledged server but the number of required client connections has already been met. The publish client refuses the connection and logs an error message. For any subsequent event blocks received by the guaranteed delivery-enabled subscribe client, an error message is logged.
Invalid Event Block ID	A guaranteed-delivery-enabled publisher injects an event block that contains an ID already present in the publish client’s unacknowledged ID list. The inject call is rejected by the publish client and an error message is logged.

Additions to the Publish/Subscribe API for Guaranteed Delivery

The publish/subscribe API provides the following methods to implement guaranteed delivery sessions:

C_dfESPGDpublisherStart()
C_dfESPGDsubscriberStart()
C_dfESPGDsubscriberAck()
C_dfESPGDpublisherCB_func()
C_dfESPGDpublisherGetID()

For more information, see . For publish/subscribe operations without a guaranteed delivery version of the function, call the standard publish/subscribe API function.

Configuration File Contents

The publish client and subscribe client reads a configuration file at start-up to get customer-specific configuration information for guaranteed delivery. The format of both of these files is as follows.

Guaranteed Delivery-enabled Publisher Configuration File Contents
Local port number for guaranteed delivery acknowledgment connection server.
Time-out value for generating negative acknowledgments, in seconds.
Number of received acknowledgments required within time-out period to generate positive instead of negative acknowledgments.
File format:	`GDpub_port=<port>` `GDpub_timeout=<timeout>` `GDpub_numSubs=<number of subscribers generating acknowledged>`

Guaranteed Delivery-enabled Subscriber Configuration File Contents
List of guaranteed delivery-enabled publisher host or port entries. Each entry contains a host:port pair corresponding to a guaranteed delivery-enabled publisher from which the subscriber wishes to receive guaranteed delivery event blocks.
Acknowledgment connection retry interval, in seconds.
Acknowledgment connection maximum number of retry attempts.
File Format:	`GDsub_pub=<host:port>` `GDsub_retryInt=<interval>` `GDsub_maxRetries=<max>`