Posts Tagged ‘Tombstones’
[DevoxxPL2019] Practical Event Sourcing: Avoiding Common Errors in Implementation
Lecturer
David Schmitz functions as a key architect at Senacor Technologies, with deep roots in rebuilding financial platforms using event-driven designs. His work spans multiple large-scale projects in banking and insurance, emphasizing sustainable architectures.
Abstract
This review critiques simplistic event sourcing adoptions, advocating refined tactics derived from extensive enterprise deployments. It defines events as unalterable facts in streams, integrates CQRS for separated concerns, and tackles hurdles in transactions, compensations, and privacy compliance. Via financial illustrations, it inspects techniques for consistency, deletion via tombstones, and polyglot persistence, reflecting on outcomes for durability and adaptability in regulated domains.
Understanding Event-Driven Foundations: Events as Core Artifacts
Event sourcing posits systems as event appendages, reconstructing states from chronological logs. David analogizes to routine transactions, like cafe orders, where each step—ordering, paying—yields events forming the narrative.
In enterprises, particularly finance, this supplants CRUD with append-only logs, yielding audit trails. Streams, via Kafka, order events, partitions grouping related ones (e.g., per account) for sequential processing.
Context: monolithic databases yield to distributed microservices, but naive per-service databases fragment truth. Event sourcing unifies via shared streams, though over-partitioning bloats management.
Analytically, events as facts enable replays for debugging or migrations, but demand schema versioning. Implications: bolsters traceability, yet escalates storage, mitigated by compaction removing intermediates.
CQRS Integration: Decoupling Commands and Queries
CQRS bifurcates modifications (commands appending events) from retrievals (queries on projections). David exemplifies transfers: commands emit events to sender/receiver streams; processors project balances.
Methodologically, Kafka Streams aggregate, ensuring eventual consistency sans distributed transactions. For guarantees, single-writer patterns serialize per-entity events.
Challenges: optimistic concurrency via versions prevents overwrites; failures invoke compensations—explicit reversals preserving history.
In insurance, initial oversights led to redesigns; polyglot views (e.g., Elasticsearch for searches) enhance flexibility.
Analytically, this scales sides independently, but lags demand monitoring. Consequences: agile evolutions, though compensations complicate logic, necessitating clear business rules.
Compliance Handling: Deletion and Rectification in Immutable Logs
GDPR mandates erasure/rectification, clashing with immutability. David proposes tombstones—events signaling deletions, prompting anonymization downstream.
For users, tombstones propagate, purging identifiers while retaining structures for audits. Rectifications append compensations, adjusting views idempotently.
Methodologically, upcasters during replays adapt old events to new schemas, ensuring compatibility.
Analytically, this reconciles permanence with privacy, but requires cross-domain coordination. Implications: legal alignment, though tombstones inflate streams, offset by compaction.
Maintainability and Tradeoffs: Insights from Deployments
Event sourcing decouples, but excessive streams hinder navigation. David advises aligning with bounded contexts, mixing with relational stores for queries.
From banking, optimistic conflicts resolved via retries; security via encryption guards sensitive events.
Methodologically, tools like KSQL query streams declaratively. Implications: audit-ready systems, but complexity in debugging distributed flows.
In reflection, event sourcing thrives with deliberate design, yielding robust, evolvable platforms in stringent sectors.