Event Time and Processing Time: Which Clock Defines the Result?

The order in which a system receives events is not always the order in which the real world produced them.

A phone can remain offline, a queue can retry, and one network route can be slower than another.

Event time describes when activity occurred; processing time describes when a system handled it.

Choosing the wrong clock can put facts into the wrong report or miss a time-sensitive pattern.

A concrete example: an offline purchase

A device records a purchase at 14:00 while offline.

It reconnects at 17:00 and uploads the event.

For daily sales by shopping time, the event belongs at 14:00. For measuring ingestion load, it belongs at 17:00.

Both timestamps are useful for different questions.

Event time

Event time comes from the activity:

• transaction timestamp,
• sensor measurement time,
• user action time,
• source database commit time.

It should include a clear time zone or universal instant.

The source clock can still be wrong.

Processing time

Processing time is the clock of the processor when it handles the event.

It is simple and immediately known. Results depend on network and system delay, making replay produce different window assignments.

Processing-time windows suit operational questions about current arrival.

Ingestion time

Some systems record an intermediate ingestion time when the platform first accepts the event.

It helps separate:

• source delay,
• broker delay,
• processing delay.

One pipeline can therefore carry event, ingestion, and processing timestamps.

Why events arrive late

Causes include:

• offline device,
• network retry,
• queue backlog,
• source batch upload,
• clock issue,
• processor failure,
• and regional replication.

Late data is normal behavior, not always corruption.

Out-of-order arrival

Event B can arrive before earlier event A.

Partitioned streams may preserve order only for one key. Events from several sources have no one reliable arrival order.

Sequence numbers and causal identifiers help where order matters.

Windows

Event-time windows group by occurrence:

14:00 <= event_time < 14:05

The processor cannot know immediately that every event for that interval has arrived.

It needs a progress estimate.

Watermarks

A watermark estimates:

Events earlier than this time are unlikely to arrive now.

When the watermark passes a window end, the system can emit an initial result.

A watermark is not proof that no earlier event will ever appear.

Watermark generation

Watermarks can use:

• maximum observed event time minus delay,
• source partition progress,
• device-specific lateness,
• or custom domain logic.

One idle partition can hold back a global watermark unless the system detects idleness.

Allowed lateness

The system can accept late events for a defined period after initial window closure.

Longer allowance improves completeness but keeps state longer and delays finality.

Business requirements should determine the interval.

Late-event policy

When an event arrives beyond allowed lateness, the system may:

• drop with metric,
• send to side output,
• correct a historical table,
• trigger reconciliation,
• or accept indefinitely.

Silent loss makes reports impossible to trust.

Result updates

A window can emit:

• preliminary result,
• revised result,
• final result.

Downstream consumers need update semantics:

• replace by window key,
• retract old value,
• or apply difference.

Appending every revision as a new total double-counts.

Triggers

Triggers decide when to emit:

• every event,
• processing-time interval,
• count threshold,
• watermark,
• or final lateness.

Early results improve responsiveness while later updates improve completeness.

State retention

The processor retains window state until it no longer accepts updates.

High-cardinality keys, long windows, and long lateness can consume large storage.

Expiration policy must match correction needs.

Joins

Joining two streams requires holding events while waiting for counterparts.

For example, payment and order events join within one hour of event time. Late or missing matches need timeout and side output.

Join windows express business expectations.

Clock skew

Source clocks can be fast, slow, or misconfigured.

Validate plausible ranges, synchronize devices where possible, and record ingestion time. A timestamp far in the future can push a naive watermark and close windows early.

Untrusted clients should not control critical ordering alone.

Timestamp authority

Choose the clock according to the event.

A bank authorization may use the server's committed timestamp, while a fitness sensor reading needs device measurement time plus ingestion time and clock-quality metadata. When sources are untrusted, bound acceptable skew and preserve the original value for investigation rather than silently rewriting history.

Time zones and daylight saving

Store event instants in UTC plus relevant local zone when business rules need it.

"Daily sales" may follow store-local calendar days, including 23- or 25-hour daylight-saving days.

Do not treat a local date as a universal instant.

Replay

Event-time processing produces the same logical window assignment during replay if timestamps and rules remain stable.

Processing-time windows assign according to replay speed and therefore differ.

This is one reason event time is valuable for business analytics.

Observability

Measure:

• event-to-ingestion delay,
• ingestion-to-processing delay,
• watermark lag,
• late-event rate,
• dropped events,
• state size,
• and revisions.

Segment by source and partition to find one delayed producer.

Knowledge check

1. How do event, ingestion, and processing time differ?
2. Why can a watermark not prove completeness?
3. What tradeoff does allowed lateness create?
4. How should downstream systems handle revised window results?
5. Why do processing-time results change during replay?

The one idea to remember

Event time preserves when the real-world activity occurred, while ingestion and processing times reveal system delay. Windows, watermarks, allowed lateness, revision semantics, clock validation, and observability make out-of-order data manageable.