diff --git a/docs/assets/session.png b/docs/assets/session.png new file mode 100644 index 0000000000..26d20a0259 Binary files /dev/null and b/docs/assets/session.png differ diff --git a/docs/user-guide/user-defined-functions/reduce/windowing/session.md b/docs/user-guide/user-defined-functions/reduce/windowing/session.md new file mode 100644 index 0000000000..babab57a8a --- /dev/null +++ b/docs/user-guide/user-defined-functions/reduce/windowing/session.md @@ -0,0 +1,74 @@ +# Session + +## Overview + +Session window is a type of Unaligned window where the window’s end time keeps moving until there is no data for a +given time duration. Unlike fixed and sliding windows, session windows do not overlap, nor do they have a set start +and end time. They can be used to group data based on activity. + +![plot](../../../../assets/session.png) + +```yaml +vertices: + - name: my-udf + udf: + groupBy: + window: + session: + timeout: duration +``` + +NOTE: A duration string is a possibly signed sequence of decimal numbers, each with optional fraction +and a unit suffix, such as "300ms", "1.5h" or "2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". + +### timeout + +The `timeout` is the duration of inactivity (no data flowing in for the particular key) after which the session is +considered to be closed. + +## Example + +To create a session window of timeout 1 minute, we can use the following snippet. + +```yaml +vertices: + - name: my-udf + udf: + groupBy: + window: + session: + timeout: 60s +``` + +The yaml snippet above contains an example spec of a _reduce_ vertex that uses session window aggregation. As we can see, +the timeout of the window is 60s. This means we no data arrives for a particular key for 60 seconds, we will mark +it as closed. + +Let's say, `time.now()` in the pipeline is `2031-09-29T18:46:30Z` as the current time, and we have a session gap of 30s. +If we receive events in this pattern: + +```text +Event-1 at 2031-09-29T18:45:40Z +Event-2 at 2031-09-29T18:45:55Z # Notice the 15 sec interval from Event-1, still within session gap +Event-3 at 2031-09-29T18:46:20Z # Notice the 25 sec interval from Event-2, still within session gap +Event-4 at 2031-09-29T18:46:55Z # Notice the 35 sec interval from Event-3, beyond the session gap +Event-5 at 2031-09-29T18:47:10Z # Notice the 15 sec interval from Event-4, within the new session gap +``` + +This would lead to two session windows as follows: + +```text +[2031-09-29T18:45:40Z, 2031-09-29T18:46:20Z) # includes Event-1, Event-2 and Event-3 +[2031-09-29T18:46:55Z, 2031-09-29T18:47:10Z) # includes Event-4 and Event-5 +``` + +In this example, the start time is inclusive and the end time is exclusive. `Event-1`, `Event-2`, and `Event-3` fall within +the first window, and this window closes 30 seconds after `Event-3` at `2031-09-29T18:46:50Z`. `Event-4` arrives 5 seconds +later, meaning it's beyond the session gap of the previous window, initiating a new window. The second window includes +`Event-4` and `Event-5`, and it closes 30 seconds after `Event-5` at `2031-09-29T18:47:40Z`, if no further events arrive +for the key until the timeout. + + + + + diff --git a/docs/user-guide/user-defined-functions/reduce/windowing/windowing.md b/docs/user-guide/user-defined-functions/reduce/windowing/windowing.md index 9262a45ae9..236fcdb485 100644 --- a/docs/user-guide/user-defined-functions/reduce/windowing/windowing.md +++ b/docs/user-guide/user-defined-functions/reduce/windowing/windowing.md @@ -32,6 +32,7 @@ Numaflow supports the following types of windows - [Fixed](fixed.md) - [Sliding](sliding.md) +- [Session](session.md) ## Non-Keyed v/s Keyed Windows diff --git a/mkdocs.yml b/mkdocs.yml index 621c12bce2..13fc173526 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -80,6 +80,7 @@ nav: - Overview: "user-guide/user-defined-functions/reduce/windowing/windowing.md" - Fixed: "user-guide/user-defined-functions/reduce/windowing/fixed.md" - Sliding: "user-guide/user-defined-functions/reduce/windowing/sliding.md" + - Session: "user-guide/user-defined-functions/reduce/windowing/session.md" - Examples: "user-guide/user-defined-functions/reduce/examples.md" - Reference: - user-guide/reference/pipeline-tuning.md