Transaction Log

The transaction log is a crucial part of the master node. It is responsible for keeping track of the transactions and their current state in the execution pipeline from a master point of view.

Each master keeps a separate copy of the transaction log, which is synchronized through the master-to-master protocol. It also helps the master keep track of the current state of each worker - its current transaction, what it is currently executing, etc.

Location and directory structure

The transaction log is kept in the master’s storage directory in a subdirectory called txlog. There, you can find:

  • the current transaction log stream - a file named log-<timestamp>.dat;

  • transaction bodies under the transactions subdirectory. For example, transactions/00a/00ac16d7-0eae-4492-b3ee-a773a59bd236.xml.gz; the UUID here is the transaction id. It can be useful when tracking an error in the logs - you will be able to see the body of the transaction.

  • (optional) bak-<timestamp>.dat files - old transactions that are not part of the current transaction log, for example bak-1024281454853796621.dat. You can safely delete them.

Transaction states in the log

Every transaction in the transaction log goes through several stages:

state INCOMING state ACCEPTED state ENQUEUED state TESTING state OK state FAILED INCOMING -> ACCEPTED ACCEPTED -> ENQUEUED ENQUEUED -> TESTING TESTING -> OK TESTING -> FAILED

Incoming state

This state, brief in duration, is the starting point of every update. The transaction data is passed to other masters so they can receive the body. No synchronization between the masters happens for that particular transaction at this stage.

Accepted state

Here, the transaction is optionally synchronized with all other masters (peers). They must agree on the transaction’s order of execution before it can be moved to the next state.

Enqueued state

This is the state where workers come into play. Enqueued transactions are ready for execution by the workers.

Testing state

Each transaction is first given to a testing worker - see testing workers. The testing state means that a worker is currently testing that transaction and there is still no outcome; see OK state and FAILED state below for different outcomes.

OK state

The transaction has been executed successfully.

FAILED state

The query execution failed. This happens when there is an error during the execution of the transaction or it is marked as a killer transaction.