Striim 3.9.4 / 3.9.5 documentation

CREATE WINDOW

To aggregate, join, or perform calculations on the data, you must create a bounded data set. The usual way to do this is with a window, which bounds the stream by a specified number of events, a period of time, or both. As discussed in the Concepts Guide (see Window), this may be a sliding window, which always contains the most recent set of events, or a jumping window, which breaks the stream up into successive chunks.

A third alternative is a session window, which breaks the stream up into chunks when there are gaps in the flow of events, that is, when no new event has been received for a specified period of time (the "idle timeout").

The syntax for sliding and jumping windows is:

CREATE [ JUMPING ] WINDOW <name> 
OVER <stream name> 
KEEP {
 <int> ROWS |
 WITHIN <int> <time unit> |
 <int> ROWS WITHIN <int> <time unit>}
[ PARTITION BY <field name> ];

The syntax for session windows is:

CREATE SESSION WINDOW <name>
OVER <stream name>
IDLE TIMEOUT <int> <time unit>
[ PARTITION BY <field name> ];

If JUMPING or SESSION is not specified, the window uses the "sliding" mode (see the Concepts Guide for an explanation of the difference).

If PARTITION BY is specified, the criteria will be applied separately each value of the specified field name.

  • With a count-based window, PARTITION BY will keep the specified number of events for each field value. For example, a window with KEEP 100 ROWS PARTITION BY merchantID would contain 100 events for each merchant.

  • With a time-based window, PARTITION BY will start the timer separately for each field value. This could be used, for example, to raise an alert when a device has multiple errors in a certain period of time.

  • With a session window, PARTITION BY will apply the idle timeout separately for each value of the specified field, which might be a user ID, session ID, or IP address.

This example from MultiLogApp creates a one-hour jumping window for each company's API usage stream:

TQL:

CREATE JUMPING WINDOW CompanyWindow 
OVER CompanyApiUsageStream 
KEEP WITHIN 1 HOUR ON logTime 
PARTITION BY company;

UI:

  • Mode: Jumping

  • Size of Window: Time

    • Time: 1 hour

    • Event Time

    • On: logTime

The hour will be timed separately for each company, starting when the first event for that company is received.

The following is a detailed guide to the syntax for various types of windows. For more detailed discussion of windows in the context of applications, see Bounding data with windows.

Bounding data in batches (jumping) by system time
CREATE JUMPING WINDOW <name> OVER <stream name>
KEEP WITHIN <int> { SECOND | MINUTE | HOUR | DAY }
[ PARTITION BY <field name> ];

With this syntax, the window will output a batch of data each time the specified time interval has elapsed. For example, this window will emit a set of data every five minutes:

TQL:

CREATE JUMPING WINDOW P2J_ST
OVER PosSource_TransformedStream
KEEP WITHIN 5 MINUTE;

UI:

  • Mode: Jumping

  • Size of Window: Time

    • Time: 5 minute

    • System Time

Bounding data in batches (jumping) by event time
CREATE JUMPING WINDOW <name> OVER <stream name>
KEEP WITHIN <int> { SECOND | MINUTE | HOUR | DAY } ON <timestamp field name>
[ PARTITION BY <field name> ];

With this syntax, the window will output a batch of data each time it receives an event in which the specified timestamp field's value exceeds oldest event in the window by the specified amount of time. For example, assuming data is received continuously, this window will emit a set of data every five minutes:

TQL:

CREATE JUMPING WINDOW P5J_ET
OVER PosSource_TransformedStream
KEEP WITHIN 5 MINUTE 
ON dateTime;

UI:

  • Mode: Jumping

  • Size of Window: Time

    • Time: 5 minute

    • Event Time

    • On: dateTime

Bounding data in batches (jumping) by event count
CREATE JUMPING WINDOW <name>
OVER <stream name>
KEEP <int> ROWS
WITHIN <int>
[ PARTITION BY <field name> ];

With this syntax, the window will output a batch of data each time it contains the specified number of events. For example, the following will break the stream up into batches of 100 events:

TQL:

CREATE JUMPING WINDOW P11J_ROWS
OVER PosSource_TransformedStream
KEEP 100 ROWS;

UI:

  • Mode: Jumping

  • Size of Window: Count

    • Events: 100

Bounding data continuously (sliding) by time
CREATE WINDOW <name> OVER <stream name>
KEEP WITHIN <int> { SECOND | MINUTE | HOUR | DAY }
[ ON <timestamp field name> ] ]
[ SLIDE <int> { SECOND | MINUTE | HOUR | DAY } ]
[ PARTITION BY <field name> ];

With this syntax, the window emits data every time an event is added to the window, first removing any events that exceed the specified time interval from the window. For example, the following will emit the events received in the past five minutes each time it receives a new event:

TQL:

CREATE WINDOW P1S_ST
OVER PosSource_TransformedStream
KEEP WITHIN 5 MINUTE;

UI:

  • Mode: Sliding

  • Size of Window: Time

    • Time: 5 minute

    • System Time

The following is similar but uses event time:

TQL:

CREATE WINDOW P4S_ET 
OVER PosSource_TransformedStream 
KEEP WITHIN 5 MINUTE 
ON dateTime;

UI:

  • Mode: Sliding

  • Size of Window: Time

    • Time: 5 minute

    • Event Time

    • On: dateTime

If you wish to get events less often, use the Output interval (UI) / SLIDE (TQL) option. For example, the following will emit the past five minutes of events once a minute:

TQL:

CREATE WINDOW P3S_ST_OI
OVER PosSource_TransformedStream
KEEP WITHIN 5 MINUTE
SLIDE 1 MINUTE;

UI:

  • Mode: Sliding

  • Size of Window: Time

    • Time: 5 miinute

    • System Time

    • Output interval: 1 minute

CREATE WINDOW P6S_ET_OI
OVER PosSource_TransformedStream
KEEP WITHIN 5 MINUTE
ON dateTime
SLIDE 1 MINUTE;

UI:

  • Mode: Sliding

  • Size of Window: Time

    • Time: 5 miinute

    • Event Time

    • On: dateTime

    • Output interval: 1 minute

Bounding data continuously (sliding) by event count
CREATE WINDOW <name> OVER <stream name>
KEEP <int> ROWS 
[ SLIDE <int> ]
[ PARTITION BY <field name> ];

With this syntax, when the window has received the specified number of events, it will emit its contents. From then on, every time it receives a new event, it will remove the event that has been in the window the longest, then emit the remaining events contents. For example, the following will send the first 100 events it receives, and from then on send the most recent 100 events every time it receives a new one:

TQL:

CREATE WINDOW P10S_ROWS
OVER PosSource_TransformedStream
KEEP 100 ROWS;

UI:

  • Mode: Sliding

  • Size of Window: Count

    • Events: 100

If you wish to get events less often, use the Output interval (UI) / SLIDE (TQL) option. For example, the following will emit the most recent 100 events every tenth event:

TQL:

CREATE WINDOW P12S_ROWS_OI 
OVER PosSource_TransformedStream 
KEEP 100 ROWS 
SLIDE 10; 

UI:

  • Mode: Sliding

  • Size of Window: Count

    • Events: 100

    • Output interval: 10