TigerBeetle is a financial transactions database built for correctness. Yet, building a correct system from correct components remains a challenge:
Composing systems, each correct in isolation, does not necessarily yield a correct system. In this post, we’ll explore how to maintain consistency in the absence of transactions, how to reason about correctness when intermediate states are externalized, and how to recover from partial failures.
TigerBeetle is a financial transactions database that offers two primitives for double-entry bookkeeping: accounts and transfers. A separate data store, such as Postgres, stores master data, such as name and address of the account holder or terms and conditions of the account.
This separation enables transfers to scale independently of general purpose master data (for example dealing with Black Friday events) and solves different security, compliance, or retention requirements of the independent data sets (for example enforce immutability of transfers).
Just as a bank may have need for both a filing cabinet and a bank vault, Postgres specializes in strings and describing entities (master data), while TigerBeetle specializes in integers and moving integers between these entities.
A transaction is a sequence of operations ending in a commit, where all operations take effect, or an abort, where no operation takes effect. The completion is instant, intermediate state is not externalized and is not observable. Disruptions (such as process failure or network failure) are mitigated transparently.
However, the sequential composition of two transactions is not itself a transaction. The completion of the entire sequence is (at best) eventual, intermediate state is externalized and observable. Disruptions are not mitigated transparently.
Since Postgres and TigerBeetle do not share a transaction boundary, the application must ensure consistency through repeated attempts at completion and coordination, not transactions.
To reason about such coordination, we need to understand the guarantees we expect our system to uphold.
A system is characterized by a set of safety and liveness properties. A safety property states that nothing bad ever happens, while a liveness property states that something good eventually happens.
... continue reading