Messenger Abstraction
The interfaces that make deployment-mode independence possible — LocalMessenger, TestMessenger, Kafka messengers, and the factory pattern.
Internals
In-depth explanations of how each architectural subsystem works internally and why it was designed that way.
These pages go beyond the component reference to explain the internal mechanics of each subsystem. They are useful for developers who need to understand why the system behaves the way it does, debug unexpected behavior, or evaluate whether Lucille’s design fits their use case.
Each page is self-contained — you can read them in any order based on what you need to understand.
| Page | What It Explains |
|---|---|
| Messenger Abstraction | The interfaces that make deployment-mode independence possible |
| Message Ordering | How Kafka keys preserve operation order across distributed components |
| Error Handling | The error philosophy, every failure scenario, and fault tolerance |
| Kafka Integration | Topics, serialization, consumer groups, offset strategies |
| Metrics and Observability | Codahale metrics, the watcher thread, heartbeats, MDC |
Deep dives that have been merged into their component pages:
- Document Model — now under Components > Document
- Pipeline Internals — now under Components > Pipeline
- Publisher Accounting — now under Components > Publisher
- Runner Orchestration — now under Components > Runner
Message Ordering
How Kafka keys preserve operation order across distributed components, and why WorkerIndexer pairs 1:1.
Error Handling and Fault Tolerance
The error handling philosophy, every failure scenario catalogued, and how fault tolerance is inherited from Kafka.
Kafka Integration
Topic naming, KafkaDocument metadata, serialization, consumer groups, offset strategies, and configuration.
Metrics and Observability
Codahale metrics, the WorkerPool watcher thread, heartbeats, stuck-worker detection, and MDC usage.