Offline-First Field Operations Platform
Delivery drivers work for hours with no connectivity, then sync. The hard problem is not the sync, it is what happens when the field and the office disagree.
Problem
A driver loses signal at 13:40. At 14:02 he hands a parcel to a customer and taps delivered on a phone with no connectivity. At 14:30 the office, seeing no update for an hour and reasonably assuming he is stuck, reassigns that job to another driver. At 17:00 he reaches signal and the delivery syncs, three hours late, against a row that changed underneath him. Every naive answer is wrong. Last write wins hands it to the dispatcher, and the system then claims a parcel sitting in a customer kitchen is still out for delivery, so it dispatches a second van to collect it. Server-authoritative is the same outcome wearing a nicer hat: the server state is not more true than the field, only more recent, and recency is not truth. Trusting device clocks builds consistency on a number any rooted phone can change by hand. Prompting a human produces forty conflicts on a bad day, and by the fifth nobody reads them, so the two that genuinely needed a person get buried in the thirty-eight that did not.
Approach
The rule is that physical truth outranks administrative intent. A delivery happened, and it is irreversible. A reassignment is an intention about what should happen. When a plan collides with an event, the event wins and the plan is told it was overtaken by reality. That is encoded as a rank: physical states outrank administrative ones, which outrank in-progress ones. A cancellation therefore cannot un-deliver a parcel. Most apparent conflicts are not conflicts at all, so field ownership is split explicitly and disjoint edits merge silently, which is what keeps the conflict inbox readable. Equal rank with overlapping fields escalates to a human with both versions attached, and an escalation writes nothing, because a system that escalates and also acts has escalated nothing.
Outcome
Built end to end and run end to end, which is the part that mattered: executing every documented step surfaced seven real defects, including one where the API reported a write as applied while the row never changed. All 115 tests pass across six layers with none skipped. Load tested to failure, diagnosed, fixed, and re-measured with the numbers published rather than summarised. Deployed and publicly inspectable, with the observability stack left open read-only so the internals are not merely claimed.
How it's built
One Postgres transaction per mutation covering six writes across five tables: lock the row, optimistic version check, guarded stock decrement, cash entry, ledger, outbox. Events are written to an outbox table inside that same transaction and relayed to Kafka by separate workers, so a broker outage becomes a delay rather than a data-loss event. Sockets are a view, never a write path: if every socket dropped simultaneously, not one delivery would be lost.
Physical truth outranks administrative intent, and ties escalate
Conflict resolution is a business decision, not a technical one. The technology can detect the conflict; only the business can say who is right.
The rule is domain-specific. It is correct for parcels and shop visits and would be wrong for a trading desk where a cancellation carries legal weight. Pretending it is universal would be the actual error. A rising escalation rate is the system being honest that reality contains a case the rules do not cover; the failure mode to fear is an escalation rate of zero, achieved by guessing.
Optimistic concurrency on a row version, not locks
You cannot hold a database lock across nine hours of a driver being offline. The update carries the version the device last saw, and a zero-row result is the entire detection mechanism. One integer comparison, working identically whether the device was gone four seconds or four days.
Conflicts are detected at sync time rather than prevented at write time, which is what makes arbitration necessary and is precisely the point. A stale write is not wrong, it is uninformed.
Order by device sequence, never by timestamp
A field phone clock drifts, gets set by hand, and jumps on a timezone change. Within a device its local counter is the truth about order, and it beats the timestamp even when the timestamp disagrees.
Cross-device ordering falls back to the client timestamp with a deterministic tiebreak, because replay has to be reproducible. Applied in arrival order instead, a shuffled batch would attempt delivered before picked up, be rejected as an illegal transition, and lose a real delivery to a networking artefact.
The outbox pattern rather than publishing to the broker inline
A database write and a broker publish cannot be made atomic. Writing the event to a table in the same transaction makes them inseparable, and a separate worker relays it afterwards. If the API published inline, a Kafka outage would stop drivers recording deliveries, taking down the most critical path in the system with a dependency it does not need in order to be correct.
At-least-once delivery, so consumers must be idempotent, deduplicated on event id and consumer group so fan-out is preserved. Deduplicating on event id alone would silently stop the second consumer group ever seeing an event the first consumed. Exactly-once delivery is not a real thing; at-least-once plus idempotent consumers gives exactly-once effects.
One transaction per mutation, not per batch
A 200-mutation batch from a device dark all afternoon contains some that succeed and some that conflict. One transaction per batch means a single stale item rolls back a whole day of legitimate field work.
Partial success has to be a first-class response shape, with a per-mutation verdict rather than one status for the request. The device is told item by item exactly what landed.
Authorization evaluated against the version the device saw
A driver reassigned away while legitimately offline is still authorized for what he did before the reassignment. Checking ownership against current state rejects the exact case the system exists to handle.
Requires recording the previous assignee atomically with the reassignment rather than inferring it. The naive fix of skipping the ownership check during a conflict would let anyone claim any job by manufacturing one.
Every conflict carries a human-readable reason
A system that silently overrides a human decision is a system that human will fight, work around, and eventually disable. An automated decision nobody understands is worse than no automated decision, because now there is both a wrong answer and nobody watching for it.
Every resolution path has to produce prose, which is more work than returning an enum. The reason string is the product; the resolution is just the mechanism.
Adjacent systems
Freelance Marketplace with Escrow
Two-sided marketplace for architecture and design work: the platform holds the money until delivery is confirmed, with bidding happening in real-time chat.
Offline-First Pharmacy Platform
A pharmacy management system that keeps selling when the internet drops, then reconciles every till operation exactly once when it returns.
Vehicle Rental Platform
Rent a scooter or e-bike from an app: scan the QR, the physical lock opens over cellular, ride, park in a permitted zone, pay by the minute from a prepaid wallet.