Reference Architecture
IllustrativeHigh-Performance Transaction Platform
A microservices platform for financial transactions built on Java, Spring Boot, Kafka, Redis and a relational ledger, designed for correctness first and throughput second.
The problem
Consider a platform that accepts payment or transfer instructions from web, mobile and partner channels and must record each one exactly once. Volume is uneven, with sharp peaks around salary dates, promotions and settlement cut-offs. Downstream networks and partner systems are slower and less reliable than the platform is expected to be.
The difficulty is not raw speed. It is preserving correctness while clients retry, messages arrive twice, nodes fail in the middle of a transaction and external parties time out without saying whether they acted. A design that performs well in a load test but cannot explain a missing unit of currency is not fit for purpose.
Requirements the design must meet
- A transaction must be applied exactly once from the customer's point of view, even when the client or an internal component retries.
- The ledger must balance at all times, with every movement recorded as an immutable double-entry posting.
- The platform must absorb peak load by scaling horizontally, without a redesign and without degrading the consistency of balances.
- A failure of any single service, node or availability zone must not lose an accepted transaction.
- Every state change must be auditable, and internal records must be reconciled against external settlement files.
- Operators must be able to see latency, error rate, queue depth and business anomalies in near real time.
The design
Components
Channels
- Web
- Mobile
- Partner APIs
- Batch files
API Gateway
- Authentication
- Rate limiting
- Idempotency keys
- Request signing
Transaction Services
Java and Spring Boot, stateless, horizontally scaled
- Payment initiation
- Authorisation
- Ledger posting
- Limits and risk
Kafka Event Backbone
- Transaction events
- Outbox relay
- Partitioned topics
- Dead-letter topics
Ledger DB & Redis
The ledger is authoritative, the cache is not
- Double-entry ledger
- Read replicas
- Balance cache
- Distributed locks
Reconciliation & Audit
- Settlement matching
- Break detection
- Immutable audit log
- Reporting exports
Monitoring
- Latency and error SLOs
- Consumer lag
- Tracing
- Business alerts
API gateway
The gateway terminates TLS, authenticates callers, applies rate limits per client and validates request signatures for partner traffic. It requires an idempotency key on every state-changing request and passes it downstream. Nothing behind the gateway is reachable from the public network.
Transaction services
Stateless Spring Boot services handle initiation, authorisation, limits and posting, each within its own bounded context. A transaction moves through an explicit state machine, so its status is always one of a small set of known values. Services scale horizontally behind the gateway and hold no state that would be lost on restart.
Ledger database
A relational database holds the double-entry ledger as append-only postings, with balances derived from them. ACID transactions and constraints enforce that debits equal credits at commit time. Read replicas serve statements and reporting so that analytical queries do not compete with the write path.
Kafka event backbone
Each service writes its state change and an outbox record in the same database transaction, and a relay publishes the outbox to Kafka. Topics are partitioned by account or transaction key, which preserves ordering where it matters. Consumers are idempotent, and events that repeatedly fail are routed to a dead-letter topic for inspection.
Redis cache and coordination
Redis holds idempotency records, rate-limit counters and short-lived cached reads such as limits and reference data. It is treated as an accelerator and never as the system of record. If Redis is unavailable the platform slows down, but balances remain correct because they are read from the ledger.
Reconciliation and audit
A reconciliation service matches internal postings against settlement and partner files, and raises a break for anything unmatched or mismatched. Breaks are worked through an operations queue with a full history. An immutable audit log records who and what changed each transaction and when.
Cloud infrastructure
Services run as containers on Kubernetes across several availability zones, with the database deployed in a primary and synchronous standby configuration. Infrastructure is defined in Terraform, and deployments are progressive so that a faulty release reaches a small share of traffic first. Backups are restored on a schedule to prove that recovery works.
Monitoring and alerting
Metrics, logs and distributed traces are correlated by transaction identifier from the gateway to the ledger. Service level objectives cover latency and error rate, and Kafka consumer lag is watched as an early sign of trouble. Business alerts, such as an unusual rate of declines or a growing reconciliation backlog, sit alongside the technical ones.
Trade-offs
Design decisions
| Decision | Why |
|---|---|
| A relational database as the ledger of record | Strong consistency and transactional constraints are worth more here than the write scalability of an eventually consistent store. The cost is that the ledger becomes the scaling constraint, which is addressed by keeping transactions short, partitioning by account range when necessary and moving reads to replicas. |
| Transactional outbox instead of dual writes | Writing to the database and to Kafka separately leaves a window in which one succeeds and the other does not. The outbox adds a relay and a small publication delay, and in return guarantees that every committed change is eventually published and nothing uncommitted ever is. |
| Idempotency enforced at the gateway and again at the ledger | Clients and networks retry, and at-least-once messaging delivers duplicates. Checking an idempotency key at the edge handles the common case cheaply, while a unique constraint in the ledger is the final guard that holds even if the cache has been lost. |
| Synchronous authorisation, asynchronous everything else | The customer needs an immediate and definite answer on whether a transaction was accepted, so that path is synchronous and kept short. Notifications, reporting, risk scoring updates and settlement run from events, which removes them from the latency budget and lets them fail and recover independently. |
| Services split by business capability, not by technical layer | Microservices bring operational cost: more deployments, more network calls and harder debugging. The split is justified only where capabilities scale or change at different rates, so the design starts with a small number of services and resists further division without a concrete reason. |
Intended outcomes
- A ledger that balances by construction and can explain every movement
- Retries and duplicate messages that do not produce duplicate transactions
- Capacity that grows by adding instances and partitions, not by redesign
- Discrepancies with external parties that are detected by reconciliation, not by customers
What the design is built to achieve. Illustrative, not measured results from a client project.
Representative technologies
- Java
- Spring Boot
- Kafka
- Redis
- PostgreSQL
- Kubernetes
- Docker
- Terraform
- AWS
- OpenTelemetry
- Prometheus
- Grafana
Have a system that needs to work in production?
Tell us what you are building. An engineer will read it and reply within two working days.