Andrew Cairns
@acairns.co.uk
Using metaphors and analogies to explain Software Engineering in fun ways: https://youtube.com/@metaphoricallyspeaking
Staff Software Engineer. Passionate about DDD, CQRS, Event Sourcing and Distributed Systems.
Kayaking, too.
Staff Software Engineer. Passionate about DDD, CQRS, Event Sourcing and Distributed Systems.
Kayaking, too.
"We're doing Event Sourcing with Kafka!"
Maybe. Depends how you're using it.
Kafka can be an event store, but I know several teams using it just for streaming, not sourcing.
Looks similar, but it's not the same thing!
#EventSourcing #Kafka
Maybe. Depends how you're using it.
Kafka can be an event store, but I know several teams using it just for streaming, not sourcing.
Looks similar, but it's not the same thing!
#EventSourcing #Kafka
November 3, 2025 at 9:24 AM
"We're doing Event Sourcing with Kafka!"
Maybe. Depends how you're using it.
Kafka can be an event store, but I know several teams using it just for streaming, not sourcing.
Looks similar, but it's not the same thing!
#EventSourcing #Kafka
Maybe. Depends how you're using it.
Kafka can be an event store, but I know several teams using it just for streaming, not sourcing.
Looks similar, but it's not the same thing!
#EventSourcing #Kafka
Your projection is broken.
This WILL happen. Plan for it!
Common strategy:
1. Build new projection (ProjectionV2)
2. Fill it from events
3. Switch traffic once caught up
4. Delete old projection
Projections are disposable.
They’re just views of events.
This WILL happen. Plan for it!
Common strategy:
1. Build new projection (ProjectionV2)
2. Fill it from events
3. Switch traffic once caught up
4. Delete old projection
Projections are disposable.
They’re just views of events.
October 27, 2025 at 9:24 AM
Your projection is broken.
This WILL happen. Plan for it!
Common strategy:
1. Build new projection (ProjectionV2)
2. Fill it from events
3. Switch traffic once caught up
4. Delete old projection
Projections are disposable.
They’re just views of events.
This WILL happen. Plan for it!
Common strategy:
1. Build new projection (ProjectionV2)
2. Fill it from events
3. Switch traffic once caught up
4. Delete old projection
Projections are disposable.
They’re just views of events.
Your Process Manager (thing coordinating multiple services) can ITSELF be Event Sourced!
Events IN: OrderPlaced, PaymentReceived
Events OUT: ReservationRequested, ShipmentScheduled
Orchestration with full history and testable like any aggregate
It's events all the way down
Events IN: OrderPlaced, PaymentReceived
Events OUT: ReservationRequested, ShipmentScheduled
Orchestration with full history and testable like any aggregate
It's events all the way down
October 24, 2025 at 8:24 AM
Your Process Manager (thing coordinating multiple services) can ITSELF be Event Sourced!
Events IN: OrderPlaced, PaymentReceived
Events OUT: ReservationRequested, ShipmentScheduled
Orchestration with full history and testable like any aggregate
It's events all the way down
Events IN: OrderPlaced, PaymentReceived
Events OUT: ReservationRequested, ShipmentScheduled
Orchestration with full history and testable like any aggregate
It's events all the way down
Year 1: OrderPlaced {orderId, customerId, items}
Year 2: Needs "shippingMethod"
But, old events?
1. Upcasting: transform old events on read
2. Weak schema: new fields optional
3. Multiple event types: OrderPlacedV2
Each works:
- Weak schema = simple
- Upcasting = control
Year 2: Needs "shippingMethod"
But, old events?
1. Upcasting: transform old events on read
2. Weak schema: new fields optional
3. Multiple event types: OrderPlacedV2
Each works:
- Weak schema = simple
- Upcasting = control
October 22, 2025 at 8:24 AM
Year 1: OrderPlaced {orderId, customerId, items}
Year 2: Needs "shippingMethod"
But, old events?
1. Upcasting: transform old events on read
2. Weak schema: new fields optional
3. Multiple event types: OrderPlacedV2
Each works:
- Weak schema = simple
- Upcasting = control
Year 2: Needs "shippingMethod"
But, old events?
1. Upcasting: transform old events on read
2. Weak schema: new fields optional
3. Multiple event types: OrderPlacedV2
Each works:
- Weak schema = simple
- Upcasting = control
The checkpoint problem:
Update projection → SUCCESS
Save checkpoint → CRASH
Checkpoint is now behind the data.
Restart = reprocessing.
Fix: Store checkpoint with projection data, update both in one transaction.
Either both succeed or both fail.
No inconsistency.
Update projection → SUCCESS
Save checkpoint → CRASH
Checkpoint is now behind the data.
Restart = reprocessing.
Fix: Store checkpoint with projection data, update both in one transaction.
Either both succeed or both fail.
No inconsistency.
October 21, 2025 at 7:40 AM
The checkpoint problem:
Update projection → SUCCESS
Save checkpoint → CRASH
Checkpoint is now behind the data.
Restart = reprocessing.
Fix: Store checkpoint with projection data, update both in one transaction.
Either both succeed or both fail.
No inconsistency.
Update projection → SUCCESS
Save checkpoint → CRASH
Checkpoint is now behind the data.
Restart = reprocessing.
Fix: Store checkpoint with projection data, update both in one transaction.
Either both succeed or both fail.
No inconsistency.
Make projections idempotent
Your projection processes event 458,295 and crashes before saving checkpoint.
On restart it processes 458,295 again.
If that duplicates data, you have a problem.
Already processed? Skip it or make replaying give same result.
Design for safe replays
Your projection processes event 458,295 and crashes before saving checkpoint.
On restart it processes 458,295 again.
If that duplicates data, you have a problem.
Already processed? Skip it or make replaying give same result.
Design for safe replays
October 17, 2025 at 8:24 AM
Make projections idempotent
Your projection processes event 458,295 and crashes before saving checkpoint.
On restart it processes 458,295 again.
If that duplicates data, you have a problem.
Already processed? Skip it or make replaying give same result.
Design for safe replays
Your projection processes event 458,295 and crashes before saving checkpoint.
On restart it processes 458,295 again.
If that duplicates data, you have a problem.
Already processed? Skip it or make replaying give same result.
Design for safe replays
Your projection crashes halfway through 1M events.
When it restarts, where does it begin?
Checkpoints solve this: periodically save "processed up to event 458,295"
Restart resumes from the checkpoint.
No checkpoint? You replay from the beginning.
Trade-off: checkpoint frequency vs replay cost.
When it restarts, where does it begin?
Checkpoints solve this: periodically save "processed up to event 458,295"
Restart resumes from the checkpoint.
No checkpoint? You replay from the beginning.
Trade-off: checkpoint frequency vs replay cost.
October 16, 2025 at 8:24 AM
Your projection crashes halfway through 1M events.
When it restarts, where does it begin?
Checkpoints solve this: periodically save "processed up to event 458,295"
Restart resumes from the checkpoint.
No checkpoint? You replay from the beginning.
Trade-off: checkpoint frequency vs replay cost.
When it restarts, where does it begin?
Checkpoints solve this: periodically save "processed up to event 458,295"
Restart resumes from the checkpoint.
No checkpoint? You replay from the beginning.
Trade-off: checkpoint frequency vs replay cost.
Medical records are append-only.
Doctors don't UPDATE past entries, they add corrections.
March 15: "Patient reports knee pain"
March 16: "Diagnosis: sprain"
March 20: "Corrected diagnosis: torn meniscus"
All three facts are true. All three stay in the record.
Just like Event Sourcing.
Doctors don't UPDATE past entries, they add corrections.
March 15: "Patient reports knee pain"
March 16: "Diagnosis: sprain"
March 20: "Corrected diagnosis: torn meniscus"
All three facts are true. All three stay in the record.
Just like Event Sourcing.
October 12, 2025 at 8:24 AM
Medical records are append-only.
Doctors don't UPDATE past entries, they add corrections.
March 15: "Patient reports knee pain"
March 16: "Diagnosis: sprain"
March 20: "Corrected diagnosis: torn meniscus"
All three facts are true. All three stay in the record.
Just like Event Sourcing.
Doctors don't UPDATE past entries, they add corrections.
March 15: "Patient reports knee pain"
March 16: "Diagnosis: sprain"
March 20: "Corrected diagnosis: torn meniscus"
All three facts are true. All three stay in the record.
Just like Event Sourcing.
Internal Events:
- CustomerAgeCalculated
- InventoryReserved
Private domain implementation. Can change anytime
External Events:
- OrderPlaced
- PaymentReceived
Published API. Version instead.
Exposing internals creates distributed coupling.
Keep them separate.
- CustomerAgeCalculated
- InventoryReserved
Private domain implementation. Can change anytime
External Events:
- OrderPlaced
- PaymentReceived
Published API. Version instead.
Exposing internals creates distributed coupling.
Keep them separate.
October 11, 2025 at 8:24 AM
Internal Events:
- CustomerAgeCalculated
- InventoryReserved
Private domain implementation. Can change anytime
External Events:
- OrderPlaced
- PaymentReceived
Published API. Version instead.
Exposing internals creates distributed coupling.
Keep them separate.
- CustomerAgeCalculated
- InventoryReserved
Private domain implementation. Can change anytime
External Events:
- OrderPlaced
- PaymentReceived
Published API. Version instead.
Exposing internals creates distributed coupling.
Keep them separate.
Reposted by Andrew Cairns
It is on YouTube but it was in the time of COVID so is an on-line teams type presentation thing...
youtu.be/eD7ls4KynRo
youtu.be/eD7ls4KynRo
What do you mean by “is”? - Duncan Jones - DDD Europe 2021
YouTube video by Domain-Driven Design Europe
youtu.be
October 10, 2025 at 9:27 AM
It is on YouTube but it was in the time of COVID so is an on-line teams type presentation thing...
youtu.be/eD7ls4KynRo
youtu.be/eD7ls4KynRo
Common Event Sourcing mistake: multiple aggregates in one stream.
One stream = One aggregate
Customer[123] stream → CustomerRegistered, EmailChanged, AddressUpdated
Order[456] stream → OrderPlaced, ItemAdded, OrderShipped
Different streams. Different things.
One stream = One aggregate
Customer[123] stream → CustomerRegistered, EmailChanged, AddressUpdated
Order[456] stream → OrderPlaced, ItemAdded, OrderShipped
Different streams. Different things.
October 10, 2025 at 8:24 AM
Common Event Sourcing mistake: multiple aggregates in one stream.
One stream = One aggregate
Customer[123] stream → CustomerRegistered, EmailChanged, AddressUpdated
Order[456] stream → OrderPlaced, ItemAdded, OrderShipped
Different streams. Different things.
One stream = One aggregate
Customer[123] stream → CustomerRegistered, EmailChanged, AddressUpdated
Order[456] stream → OrderPlaced, ItemAdded, OrderShipped
Different streams. Different things.
Reposted by Andrew Cairns
You can have the event chain organised as a workflow (based on an event stream) identified by an unique identifier and pass that identifier to the individual processes as a correlation id...and have them write back their doings to that event stream.
Thus your workflow is its own log.
Thus your workflow is its own log.
October 9, 2025 at 8:32 AM
You can have the event chain organised as a workflow (based on an event stream) identified by an unique identifier and pass that identifier to the individual processes as a correlation id...and have them write back their doings to that event stream.
Thus your workflow is its own log.
Thus your workflow is its own log.
I'm an advocate for giving every event a Correlation ID copied from the command that caused it.
OrderAccepted → OrderPaid → OrderShipped
All share CorrelationId: "ABC123"
Now you can... 👇
OrderAccepted → OrderPaid → OrderShipped
All share CorrelationId: "ABC123"
Now you can... 👇
October 9, 2025 at 8:24 AM
I'm an advocate for giving every event a Correlation ID copied from the command that caused it.
OrderAccepted → OrderPaid → OrderShipped
All share CorrelationId: "ABC123"
Now you can... 👇
OrderAccepted → OrderPaid → OrderShipped
All share CorrelationId: "ABC123"
Now you can... 👇
Your data team wants to build reports and dashboards but your events live in an append-only log...
They want a relational schema.
Solution: project events into a dedicated reporting database!
Why? 👇
They want a relational schema.
Solution: project events into a dedicated reporting database!
Why? 👇
October 8, 2025 at 8:24 AM
Your data team wants to build reports and dashboards but your events live in an append-only log...
They want a relational schema.
Solution: project events into a dedicated reporting database!
Why? 👇
They want a relational schema.
Solution: project events into a dedicated reporting database!
Why? 👇
How do you handle two processes acting on the same thing in Event Sourcing?
Optimistic concurrency with expected version:
1. Load events from stream (version 67)
2. Process command, generate new events
3. Write with expectedVersion=67
And then... 👇
Optimistic concurrency with expected version:
1. Load events from stream (version 67)
2. Process command, generate new events
3. Write with expectedVersion=67
And then... 👇
October 7, 2025 at 8:24 AM
How do you handle two processes acting on the same thing in Event Sourcing?
Optimistic concurrency with expected version:
1. Load events from stream (version 67)
2. Process command, generate new events
3. Write with expectedVersion=67
And then... 👇
Optimistic concurrency with expected version:
1. Load events from stream (version 67)
2. Process command, generate new events
3. Write with expectedVersion=67
And then... 👇
Event Sourcing requires a mental model shift.
"But I need to DELETE something!".
Here's the reality: you don't delete events.
You record a NEW event that something was deleted/cancelled/revoked.
OrderPlaced → OrderCancelled
Both facts are true. History doesn't have an undo button.
"But I need to DELETE something!".
Here's the reality: you don't delete events.
You record a NEW event that something was deleted/cancelled/revoked.
OrderPlaced → OrderCancelled
Both facts are true. History doesn't have an undo button.
October 6, 2025 at 8:24 AM
Event Sourcing requires a mental model shift.
"But I need to DELETE something!".
Here's the reality: you don't delete events.
You record a NEW event that something was deleted/cancelled/revoked.
OrderPlaced → OrderCancelled
Both facts are true. History doesn't have an undo button.
"But I need to DELETE something!".
Here's the reality: you don't delete events.
You record a NEW event that something was deleted/cancelled/revoked.
OrderPlaced → OrderCancelled
Both facts are true. History doesn't have an undo button.
Event Sourcing tests double as documentation.
Given: [OrderPlaced, ItemAdded]
When: RemoveItem
Then: [ItemRemoved]
This reads like a spec. It IS a spec.
Your tests become the living documentation of your business rules.
Given: [OrderPlaced, ItemAdded]
When: RemoveItem
Then: [ItemRemoved]
This reads like a spec. It IS a spec.
Your tests become the living documentation of your business rules.
October 5, 2025 at 8:24 AM
Event Sourcing tests double as documentation.
Given: [OrderPlaced, ItemAdded]
When: RemoveItem
Then: [ItemRemoved]
This reads like a spec. It IS a spec.
Your tests become the living documentation of your business rules.
Given: [OrderPlaced, ItemAdded]
When: RemoveItem
Then: [ItemRemoved]
This reads like a spec. It IS a spec.
Your tests become the living documentation of your business rules.
A Projection is something that converts your event stream into state for reading.
Kinda like a database view, but better:
- You can have dozens of them
- Optimised for specific queries
- Rebuild anytime from events
- Different databases even
Kinda like a database view, but better:
- You can have dozens of them
- Optimised for specific queries
- Rebuild anytime from events
- Different databases even
October 4, 2025 at 8:24 AM
A Projection is something that converts your event stream into state for reading.
Kinda like a database view, but better:
- You can have dozens of them
- Optimised for specific queries
- Rebuild anytime from events
- Different databases even
Kinda like a database view, but better:
- You can have dozens of them
- Optimised for specific queries
- Rebuild anytime from events
- Different databases even
Reposted by Andrew Cairns
Just gave a talk about this at #dev2next and it’s incredibly powerful yet not very complicated once you get in the right mindset.
ted.dev/talks
#Java #EventSourcing #cqrs
ted.dev/talks
#Java #EventSourcing #cqrs
October 3, 2025 at 2:32 PM
Just gave a talk about this at #dev2next and it’s incredibly powerful yet not very complicated once you get in the right mindset.
ted.dev/talks
#Java #EventSourcing #cqrs
ted.dev/talks
#Java #EventSourcing #cqrs
"How do you query an Event Sourced system?"
You don't. Not directly.
This is where CQRS (Command Query Responsibility Segregation) enters.
Commands write events → void return
Queries read → no mutations
Separate concerns. Separate models. Separate scaling.
You don't. Not directly.
This is where CQRS (Command Query Responsibility Segregation) enters.
Commands write events → void return
Queries read → no mutations
Separate concerns. Separate models. Separate scaling.
October 3, 2025 at 8:24 AM
"How do you query an Event Sourced system?"
You don't. Not directly.
This is where CQRS (Command Query Responsibility Segregation) enters.
Commands write events → void return
Queries read → no mutations
Separate concerns. Separate models. Separate scaling.
You don't. Not directly.
This is where CQRS (Command Query Responsibility Segregation) enters.
Commands write events → void return
Queries read → no mutations
Separate concerns. Separate models. Separate scaling.
Traditional approach: Save the structure, lose the story
Event Sourcing: Save the story, derive the structure
Your domain object's structure becomes transient.
Storage format ≠ Application format
Event Sourcing: Save the story, derive the structure
Your domain object's structure becomes transient.
Storage format ≠ Application format
October 2, 2025 at 8:24 AM
Traditional approach: Save the structure, lose the story
Event Sourcing: Save the story, derive the structure
Your domain object's structure becomes transient.
Storage format ≠ Application format
Event Sourcing: Save the story, derive the structure
Your domain object's structure becomes transient.
Storage format ≠ Application format
Want to understand Event Sourcing? Try:
1. Write every FACT that happens in your system on sticky notes
2. Put them on a wall
3. Group related ones together
You'll have hundreds. That's your event model.
(this works great even if you DON'T use Event Sourcing)
1. Write every FACT that happens in your system on sticky notes
2. Put them on a wall
3. Group related ones together
You'll have hundreds. That's your event model.
(this works great even if you DON'T use Event Sourcing)
October 1, 2025 at 8:24 AM
Want to understand Event Sourcing? Try:
1. Write every FACT that happens in your system on sticky notes
2. Put them on a wall
3. Group related ones together
You'll have hundreds. That's your event model.
(this works great even if you DON'T use Event Sourcing)
1. Write every FACT that happens in your system on sticky notes
2. Put them on a wall
3. Group related ones together
You'll have hundreds. That's your event model.
(this works great even if you DON'T use Event Sourcing)
An event is a fact something happened in your domain.
Named with past tense verbs: `CustomerRegistered` or `OrderPlaced`.
Events are immutable, which makes sense when you think about it because you can't change the past.
This principle is critical to Event Sourcing
Named with past tense verbs: `CustomerRegistered` or `OrderPlaced`.
Events are immutable, which makes sense when you think about it because you can't change the past.
This principle is critical to Event Sourcing
September 30, 2025 at 8:24 AM
An event is a fact something happened in your domain.
Named with past tense verbs: `CustomerRegistered` or `OrderPlaced`.
Events are immutable, which makes sense when you think about it because you can't change the past.
This principle is critical to Event Sourcing
Named with past tense verbs: `CustomerRegistered` or `OrderPlaced`.
Events are immutable, which makes sense when you think about it because you can't change the past.
This principle is critical to Event Sourcing
Storing your current state is like taking a photograph. Event Sourcing however is like having the full video.
This is why it can be hard to know WHY data changed when you only store WHAT has changed.
This is why it can be hard to know WHY data changed when you only store WHAT has changed.
September 29, 2025 at 7:33 PM
Storing your current state is like taking a photograph. Event Sourcing however is like having the full video.
This is why it can be hard to know WHY data changed when you only store WHAT has changed.
This is why it can be hard to know WHY data changed when you only store WHAT has changed.