Event-Driven AI: Real-Time Intelligence at Scale
Why real-time matters
Batch processing works until it doesn't. If your fraud detection model runs every hour, you're approving fraudulent transactions for 59 minutes. If your recommendation engine updates daily, you're showing stale suggestions to users whose interests shifted this morning.
Event-driven AI architectures process data as it arrives, running inference in real-time or near-real-time. The tradeoff is complexity -- streaming systems are harder to build, test, and debug than batch pipelines. But for use cases where latency matters, there's no substitute.
The core architecture
The pattern is consistent across most real-time AI systems:
Event source → Message broker → Feature extraction → Model serving → Action
Events come in from user activity, sensor data, transaction logs, or system metrics. A message broker (Kafka, Pulsar, Redpanda) durably queues them. A stream processor (Flink, Spark Streaming, or a custom consumer) extracts features. A model serving layer runs inference. The result triggers an action -- block a transaction, update a ranking, fire an alert.
Each component is independently scalable. You can add Kafka partitions, spin up more Flink workers, or add model replicas without changing the rest of the pipeline.
Windowing strategies
Raw events are rarely useful for model input directly. You need aggregated features -- "number of transactions in the last 5 minutes," "average session duration over the last hour," "distinct IP addresses in the last 30 seconds."
This is where windowing comes in. There are three main types.
Tumbling windows divide the stream into fixed, non-overlapping intervals. Every 5 minutes, you compute the aggregate and emit it. Simple, but you get stale features at the beginning of each window and miss patterns that span window boundaries.
Sliding windows overlap. A 5-minute window that slides every 30 seconds gives you a fresh aggregate twice a minute. Better for latency-sensitive features, but more expensive to compute.
Session windows group events by activity. A session closes after a gap of inactivity (say, 30 minutes of no events from a user). Useful for behavioral features where the natural unit isn't time-based but activity-based.
The choice depends on your use case. Fraud detection usually needs sliding windows with short intervals. Recommendation engines often work fine with tumbling windows. User behavior analysis benefits from session windows.
The feature consistency problem
This is the hardest part of real-time AI, and the one teams underestimate most.
Your model was trained on features computed in batch -- aggregations over historical data, calculated in a data warehouse, with the luxury of seeing the complete dataset. In production, you're computing those same features on a live stream, with incomplete data, under latency constraints.
If the features don't match, the model's predictions drift. It was trained on "average transaction amount over 30 days computed in Spark." In production, it's getting "average transaction amount over 30 days maintained as a running aggregate in Flink." Subtle differences in how nulls are handled, how timestamps are rounded, or how late-arriving events are treated can produce different numbers.
The solution is a feature store that serves both training and inference. The training pipeline reads historical features from the offline store. The serving pipeline reads real-time features from the online store. The feature store guarantees that the computation logic is the same in both paths. Feast, Tecton, and Hopsworks all do this, with varying levels of maturity.
Real-time vs near-real-time
Not everything needs sub-second latency. It's worth being precise about what "real-time" means for your use case.
True real-time (milliseconds) is necessary for fraud detection, ad bidding, and trading systems. The event happens, you need a decision immediately. This requires pre-computed features, models loaded in memory, and optimized serving infrastructure. You're paying for this with infrastructure cost and complexity.
Near-real-time (seconds to minutes) works for most recommendation systems, anomaly detection, and monitoring. You can afford to batch events into micro-batches, compute features in small windows, and tolerate some staleness. This is dramatically simpler to build and operate.
The mistake I see teams make: assuming they need true real-time when near-real-time would work fine. Ask "what happens if this decision is 30 seconds late?" If the answer is "nothing much," you probably don't need sub-second latency.
Use cases in detail
Fraud detection is the classic real-time AI use case. A transaction event hits Kafka. A stream processor enriches it with features -- user's typical spending pattern, merchant risk score, geographic anomalies, velocity checks (how many transactions in the last N minutes). The fraud model scores it in single-digit milliseconds. High-risk transactions get blocked or flagged before the merchant sees an authorization.
The challenge is false positives. Block too aggressively and you frustrate legitimate customers. The model needs both speed and precision, which usually means an ensemble -- a fast rules engine for obvious fraud, a lightweight model for scoring, and a heavier model for borderline cases that can tolerate slightly more latency.
Recommendation engines update in near-real-time based on user behavior. A user watches three cooking videos in a row -- the recommendation model should pick up on this within minutes, not wait for tomorrow's batch job. The stream processor maintains a rolling profile of recent activity, and the serving layer blends this with the user's long-term preferences.
IoT anomaly detection monitors sensor data for patterns that indicate failures. Temperature readings, vibration data, pressure levels -- each feeding into a stream processor that maintains rolling statistics. When a reading deviates from the expected range, or when a combination of sensors shows a pattern the model associates with imminent failure, an alert fires.
Backpressure handling
What happens when events arrive faster than your pipeline can process them? In a batch system, things just take longer. In a streaming system, you have a backpressure problem.
The naive approach -- dropping events -- is sometimes acceptable for analytics but usually not for AI inference where every event might matter.
Better strategies: buffer events in the broker (Kafka is good at this), dynamically scale consumers, or implement priority-based processing where high-value events jump the queue. Some systems use a tiered approach -- all events get a lightweight scoring, but only flagged events get the expensive model.
Graceful degradation
Models go down. They get slow. New deployments have bugs. Your architecture needs to handle this without falling over.
The pattern that works: always have a fallback. If the real-time model is unavailable, fall back to a rules engine or a cached prediction. If the feature store is slow, use stale features with a confidence penalty. If the whole inference pipeline is down, queue events and process them when it recovers.
Design for partial failures explicitly. In a pipeline with five stages, any stage can fail independently. Each stage should have its own timeout, retry policy, and fallback behavior.
Circuit breakers are essential. If your model serving layer starts timing out, stop sending requests to it and switch to the fallback immediately. Don't let one slow component bring down the entire pipeline by exhausting connection pools or filling up queues.
Getting started
If you're new to this, start with near-real-time. Kafka as the broker, a simple consumer group for feature extraction, and a model served behind a REST endpoint. Get the pipeline working end-to-end with basic features. Then optimize -- move to streaming frameworks for complex features, switch to gRPC for model serving, add a feature store for consistency.
The hardest problems aren't in the infrastructure. They're in feature engineering, feature consistency, and model monitoring. Get those right and the rest is plumbing.
