Event time ≠ processing time; the watermark is a trade-off, not a technical detail

Event time and processing time always diverge: network, retries, offline clients, replay. A watermark is an estimate of event-time progress: the system’s statement “we no longer expect events older than T”, from which it follows when a window can be considered complete enough. How much of it is an estimate and how much a guarantee depends on the source: for a partition with monotonic time it is nearly a guarantee; for mobile clients that go offline it is a heuristic with a tail. Choosing T is a trade-off between completeness and latency, and the business makes it, not the framework. Late data is the normal mode, not an error: every window needs a deliberately chosen policy.

From there the watermark splits into four distinct decisions that look like a single setting in the API: how long to wait (the watermark itself), how long to keep accepting data after the window closes (allowed lateness), what to do with whatever arrives even later (route it to a side output or lose it), and what to do with a result already emitted (re-firing, retraction, nothing). A separate item is the idle source: a partition nobody writes to anymore freezes the overall watermark, because it is computed as the minimum across all partitions, and the pipeline stalls while the data is perfectly fine.

// every constant here is a business decision, not a “default” config
WatermarkStrategy.<Event>forBoundedOutOfOrderness(Duration.ofSeconds(30))
    .withIdleness(Duration.ofMinutes(1));

stream.keyBy(Event::customerId)
    .window(TumblingEventTimeWindows.of(Duration.ofMinutes(5)))

    .allowedLateness(Duration.ofMinutes(10))
    // Window state lives 10 more minutes past the watermark. An event arriving
    // within that interval DOES enter the window and triggers a re-firing, i.e.
    // an updated version of the aggregate. This, not the side output, is what
    // downstream must be able to accept.

    .sideOutputLateData(lateTag)
    // This receives whatever arrives after window end + allowedLateness:
    // it can no longer enter the window and would otherwise just be dropped.

    .aggregate(new Revenue());

// A growing share of events in the side output is reality shifting, not “noise”:
// fix the source or revisit the 30s.

Honest assessment. “Exactly-once event-time aggregation” does not make the problem go away: a window closed by the watermark is closed according to the declared trade-off, and a late event is either lost or produces a correction. A downstream that cannot accept re-firing (an append-only report) silently records a lie — the late-data policy is end-to-end, all the way to the last consumer.