Transaction log

The transaction log is a crucial part of the master node. It is responsible for keeping track of the transactions and their current state in the execution pipeline from master’s point of view.

Each master keeps a separate copy of the transaction log, which is synchronised through the master-to-master protocol. It also helps the master to keep track of the current state of each worker - its current transaction, what it is currently executing, etc.

Location and directory structure

The transaction log is kept in the master’s storage directory in a subdirectory called txlog. There you can find:

  • the current transaction log stream - a file named log-<timestamp>.dat;
  • transaction bodies under the transactions subdirectory. For example, transactions/00a/00ac16d7-0eae-4492-b3ee-a773a59bd236.xml.gz; the UUID here is the transaction id. It can be useful when tracking an error in the logs - you will be able to see the body of the transaction.
  • (optionally) bak-<timestamp>.dat files, which are old transactions that are not part of the current transaction log, for example bak-1024281454853796621.dat. You can safely delete them.

Transaction states in the log

Every transaction in the transaction log moves through different stage:

state INCOMING state ACCEPTED state ENQUEUED state TESTING state OK state FAILED INCOMING -> ACCEPTED ACCEPTED -> ENQUEUED ENQUEUED -> TESTING TESTING -> OK TESTING -> FAILED

Incoming state

Every update starts from this state. It doesn’t last for a long time - the transaction’s data is passed to other masters so they can receive the body. No synchronisation between the masters happens for that particular transaction at this stage.

Accepted state

Here, the transaction is optionally synchronised with all other masters (peers). They must agree on the transaction’s order of execution before it can be moved to the next state.

Enqueued state

This is the state where workers come into play. Enqueued transactions are ready for execution by the workers.

Testing state

Each transaction is first given to a testing worker - see testing workers. The testing state means that a worker is currently testing that transaction and there is still no outcome; see OK state and FAILED state for different outcomes

OK state

The transaction has been executed successfully.

FAILED state

The query execution failed. This happens when there is an error during the execution of the transaction or it is marked as a killer transaction.