Skip to content

API Docs - v4.0.23

Reorder

akslack (Stream Processor)

This stream processor extension performs reordering of an out-of-order event stream.
 It implements the AQ-K-Slack based out-of-order handling algorithm (originally described in
http://dl.acm.org/citation.cfm?doid=2675743.2771828)

Syntax

reorder:akslack(<LONG> timestamp, <DOUBLE> correlation.field, <LONG> batch.size, <LONG> timer.timeout, <LONG> max.k, <BOOL> discard.flag, <DOUBLE> error.threshold, <DOUBLE> confidence.level)

QUERY PARAMETERS

Name Description Default Value Possible Data Types Optional Dynamic
timestamp Attribute used for ordering the events LONG No No
correlation.field Corresponds to the data field of which the accuracy directly gets affected by the adaptive operation of the Alpha K-Slack extension. This field is used by the Alpha K-Slack to calculate the runtime window coverage threshold which is an upper limit set for the unsuccessfully handled late arrivals DOUBLE No No
batch.size The parameter batch.size denotes the number of events that should be considered in the calculation of an alpha value. batch.size should be a value which should be greater than or equals to 15 10,000 LONG Yes No
timer.timeout Corresponds to a fixed time-out value in milliseconds, which is set at the beginning of the process. Once the time-out value expires, the extension drains all the events that are buffered within the reorder extension to outside. The time out has been implemented internally using a timer. The events buffered within the extension are released each time the timer ticks. -1 (timeout is infinite) LONG Yes No
max.k The maximum threshold value for K parameter in the Alpha K-Slack algorithm 9,223,372,036,854,775,807 (The maximum Long value) LONG Yes No
discard.flag Indicates whether the out-of-order events which appear after the expiration of the Alpha K-slack window should get discarded or not. When this value is set to true, the events would get discarded false BOOL Yes No
error.threshold Error threshold to be applied in Alpha K-Slack algorithm. This parameter must be defined simultaneously with confidenceLevel 0.03 (3%) DOUBLE Yes No
confidence.level Confidence level to be applied in Alpha K-Slack algorithm. This parameter must be defined simultaneously with errorThreshold 0.95 (95%) DOUBLE Yes No
Extra Return Attributes
Name Description Possible Types
beta0 Timestamp based on which the reordering is performed LONG
beta1 An upper limit value assigned for the unsuccessfully handled late arrivals DOUBLE
beta2 The number of events that should be considered in the calculation of an alpha value LONG
beta3 Fixed time-out value (in milliseconds) assigned for flushing all the events buffered inside the extension. LONG
beta4 Maximum threshold value assigned for K parameter. LONG
beta5 Flag set to indicate whether out-of-order events which arrive after buffer eviction to be discarded or not BOOL
beta6 Error threshold value set for Alpha K-Slack algorithm DOUBLE
beta7 Confidence level set for the Alpha K-Slack algorithm DOUBLE

Examples EXAMPLE 1

define stream inputStream (eventtt long,data double);
@info(name = 'query1')
from inputStream#reorder:akslack(eventtt, data, 20)
select eventtt, data
insert into outputStream;

This query performs reordering based on the 'eventtt' attribute values. In this example, 20 represents the batch size

kslack (Stream Processor)

This stream processor extension performs reordering of an out-of-order event stream.
 It implements the K-Slack based out-of-order handling algorithm (originally described in
https://www2.informatik.uni-erlangen.de/publication/download/IPDPS2013.pdf)

Syntax

reorder:kslack(<LONG> timestamp, <LONG> timer.timeout, <LONG> max.k, <BOOL> discard.flag)

QUERY PARAMETERS

Name Description Default Value Possible Data Types Optional Dynamic
timestamp Attribute used for ordering the events LONG No No
timer.timeout Corresponds to a fixed time-out value in milliseconds, which is set at the beginning of the process. Once the time-out value expires, the extension drains all the events that are buffered within the reorder extension to outside. The time out has been implemented internally using a timer. The events buffered within the extension are released each time the timer ticks. -1 (timeout is infinite) LONG Yes No
max.k The maximum threshold value for K parameter in the K-Slack algorithm 9,223,372,036,854,775,807 (The maximum Long value) LONG Yes No
discard.flag Indicates whether the out-of-order events which appear after the expiration of the K-slack window should get discarded or not. When this value is set to true, the events would get discarded false BOOL Yes No

Examples EXAMPLE 1

define stream inputStream (eventtt long, price long, volume long);
@info(name = 'query1')
from inputStream#reorder:kslack(eventtt, 1000)
select eventtt, price, volume
insert into outputStream;

This query performs reordering based on the 'eventtt' attribute values. In this example, the timeout value is set to 1000 milliseconds