Publish-Subscribe & Event Fan-Out
Broadcast domain events to multiple independent subscribers without coupling publishers to consumer implementations or availability.
Overview
The Problem Statement
When a core business event occurs (e.g. `OrderPlaced`), multiple downstream systems must react: Inventory must reserve stock, Billing must charge the customer, Shipping must create a label, Analytics must record the funnel, and Notification must send an email. If the Order service calls all these systems synchronously, it becomes tightly coupled to their network endpoints, latency accumulates linearly, and any downstream outage aborts the order.
- Broadcasting 1-to-many domain events across organizational boundaries.
- Event-driven microservices architecture.
- Decoupling event producers from the knowledge of who consumes their events.
- Real-time analytics and telemetry aggregation.
- Strict point-to-point worker task distribution (use Message Queues instead).
- Synchronous command execution where the caller needs an immediate return value.
- Ultra-low latency inter-thread messaging within a single process (use channels or event emitters).
Why It Exists
A banking platform processes account transfers. Instead of asynchronous pub/sub, the transfer service calls 12 downstream compliance, analytics, and marketing microservices in sequence. During peak month-end traffic, one marketing microservice deploys buggy code that opens 5,000 DB connections and hangs. The transfer service exhausts its thread pool waiting on the marketing service, shutting down real-world bank transfers across the entire country.
- Severe runtime coupling between critical and non-critical services.
- Linear accumulation of network latencies across multiple downstream calls.
- Cascading outages when secondary consumer services experience downtime.
- Inability to onboard new microservices without modifying upstream producer code.
How It Works
In Pub/Sub, a Publisher emits an event to a named Topic. The publisher does not know and does not care what services are listening.
The message broker maintains individual Subscriptions (or Consumer Groups) attached to the topic. When an event is published, the broker fans out the message: it duplicates or references the event into the private queue/offset of each registered subscription.
Each subscription operates independently: Subscriber A (Billing) can process immediately, Subscriber B (Warehouse) can process in batches, and Subscriber C (Analytics) can lag by 10 minutes without impacting Subscriber A or the Publisher.
In distributed event logs (like Apache Kafka or AWS Kinesis), messages are appended to immutable partitioned logs. Consumers maintain their own cursor/offset, enabling independent replay of historical events.
In-memory Pub/Sub (e.g. Node EventEmitter or Go channels) broadcasts to in-process memory pointers with no persistence; if a subscriber is slow or crashes, messages are lost. Distributed Pub/Sub (e.g. Google Cloud Pub/Sub, SNS/SQS, Kafka) persists events to replicated disks and guarantees delivery even if subscribers are completely offline.
Fan-Out Broker Replication
Broker clones incoming message to separate FIFO queues for every active subscription.
Partitioned Log with Offset Cursors
Single immutable append-only disk log; subscribers maintain private offset pointers (Kafka style).
Topic Filtering / Attribute Matching
Subscribers register SQL-like filter expressions (e.g. `event_type = 'fraud'`). Broker only routes matching events.
Prove It: Predict the System Behavior
Topic 'orders.v1' has 3 consumer groups: Billing, Inventory, and Analytics. Inventory service experiences a total outage.
What happens to the Billing and Analytics consumer groups while Inventory is down?
Billing and Analytics are blocked because all subscribers must proceed in strict lockstep.
Billing and Analytics continue processing events in real-time with zero disruption; only Inventory accumulates consumer lag.
The publisher crashes because one of the subscribers failed to acknowledge the message.
Interactive Visualizer
Broadcast domain events across isolated subscriber groups. Simulate subscriber outages and watch consumer lag accumulate without affecting neighboring services.
Topic Broadcast & Subscriber Fan-Out Simulator
Topic: orders.v1 | 1 Publisher → 3 Independent Consumer Groups
Build It Step-by-Step
The engineering blueprint. Expand each sequential milestone to examine requirements, architectural impacts, common pitfalls, and blueprint pseudocode.
Model Topics & Subscriptions
A Topic represents a logical stream of events. A Subscription represents an independent consumer stream with its own queue and configuration.
struct Subscription {
id: string
queue: Queue<Message>
filterAttributes: Map<string, string>
}
struct Topic {
name: string
subscriptions: List<Subscription>
}Implement Publisher Ingress
Assign message ID (UUID), timestamp, and publish to topic.
Execute Fan-Out Routing Logic
When an event is published, loop through all subscriptions registered to that topic. Check optional attribute filters, and enqueue a copy into the subscription's buffer.
function publish(topicName, message):
topic = getTopic(topicName)
for sub in topic.subscriptions:
if matchesFilter(sub, message):
sub.queue.enqueue(clone(message))Enable Consumer Group Load Balancing
When a service scales to 5 pods, they shouldn't all process every event. They form a single Consumer Group, and the subscription distributes events across them in round-robin fashion.
Track Independent Subscriber ACKs & Offsets
If Subscription A ACKs message #100, message #100 is completed for A. Subscription B can still be processing message #95.
Handle Subscriber Deadlines & Backpressure
If a subscriber is processing slowly, bound its inbound buffer. When buffer is full, pause network delivery or drop according to policy.
Support Idempotent Consumption
Because distributed brokers guarantee at-least-once delivery, subscribers must combine message processing with an idempotency mechanism.
Code Implementation
Production-ready, idiomatic reference implementations across Go, TypeScript, Python, and Java with sticky language switching, copy-to-clipboard, and source download.
Core Architectural Design Decisions
Invariants & MechanicsSnapshot list of handlers inside lock to execute handlers outside the lock.
Catches exceptions per handler to isolate subscriber failures.
import threading
import uuid
import time
from typing import Callable, Dict, Any, List
class Event:
def __init__(self, topic: str, payload: Any):
self.id = str(uuid.uuid4())
self.topic = topic
self.payload = payload
self.timestamp = time.time()
class PubSubBroker:
def __init__(self):
self._lock = threading.Lock()
self._topics: Dict[str, Dict[str, Callable[[Event], None]]] = {}
def subscribe(self, topic: str, sub_id: str, handler: Callable[[Event], None]):
with self._lock:
if topic not in self._topics:
self._topics[topic] = {}
self._topics[topic][sub_id] = handler
def publish(self, topic: str, payload: Any):
with self._lock:
handlers = list(self._topics.get(topic, {}).values())
event = Event(topic, payload)
for handler in handlers:
try:
handler(event)
except Exception as e:
print(f"Error in subscriber handler for topic {topic}: {e}")
def unsubscribe(self, topic: str, sub_id: str):
with self._lock:
if topic in self._topics:
self._topics[topic].pop(sub_id, None)Edge Cases & Failure Modes
Investigate real-world production incidents. Expand each failure mode to analyze symptoms, root causes, and defensive engineering mitigations.
Production Considerations
Production checklist covering observability metrics, horizontal sharding, reliability fail-open patterns, and client identity security.
- Partition topics across multiple broker nodes (e.g. 32 partitions per topic) to allow parallel consumer scaling.
- Batch events on publisher side before network transit (e.g. transmit 500 events per TCP frame).
Trade-offs
Select an algorithm below to dynamically compare memory footprints, burst tolerance, boundary precision, and distributed suitability.
| Approach | Advantages | Disadvantages | Use When |
|---|---|---|---|
| Broker-Duplicated Fan-Out (RabbitMQ / SQS) | Each subscriber has an isolated queue; messages can be ACKed or deleted individually. | Storage and network multiply linearly with subscriber count. | Independent worker pools that need granular task acknowledgement and retries. |
| Partitioned Distributed Log (Kafka / Kinesis) | Single disk write for all subscribers; massive throughput (millions/sec); historical replay. | Cannot delete single messages; head-of-line blocking if a partition stalls. | High-throughput event streaming, metrics, clickstreams, and CQRS architectures. |
Further Reading
Authoritative industry references, IETF RFC standards, and systems engineering papers.
Kafka: A Distributed Messaging System for Log Processing
The seminal paper establishing the partitioned append-only log architecture.