This article explains how to build a State Backend in Flink from scratch.

The StateBackend Interface

1
public interface StateBackend extends java.io.Serializable {
2

3
  CompletedCheckpointStorageLocation resolveCheckpoint(String externalPointer) throws IOException;
4

5
  CheckpointStorage createCheckpointStorage(JobID jobId) throws IOException;
6

7
  <K> AbstractKeyedStateBackend<K> createKeyedStateBackend（...);
8

9
  OperatorStateBackend createOperatorStateBackend(...);
10
}

Besides resolveCheckpoint (which resolves the Checkpoint location from externalPointer), a StateBackend needs the following capabilities:

• Initialize the CheckpointStorage associated with the State
• Create a Keyed StateBackend and an Operator StateBackend

Implementation Overview

Flink currently provides 4 StateBackend implementations:

• DefaultOperatorStateBackend (Operator StateBackend)
• MemoryStateBackend (Keyed StateBackend)
• FsStateBackend (Keyed StateBackend)
• RocksDBStateBackend (Keyed StateBackend)

Currently under development:

• DiskSpillingStateBackend

DefaultOperatorStateBackend

MemoryStateBackend

FsStateBackend

RocksDBStateBackend

Custom StateBackend