Storage and management of chain state, backed by Jellyfish Merkle Trees and RocksDB.
This crate provides a versioned, verifiable key-value store that also
supports lightweight, copy-on-write snapshots and transactional semantics.
Storage type is a handle for an instance of a backing store,
implemented using RocksDB. The storage records a sequence of versioned
Snapshot type is a lightweight snapshot of a particular
version of the chain state.
Snapshot instance can also be used as the basis for a copy-on-write
fork to build up changes before committing them to persistent storage. The
StateDelta type collects a group of writes, which can then be applied to
StateDelta overlay. Finally, the changes accumulated in the
StateDelta instance can be committed to the persistent
Reads are performed with the
StateRead trait, implemented by both
StateDelta, and reflect any currently cached writes.
Writes are performed with the
StateWrite trait, which is only
The storage system provides three data stores:
A verifiable key-value store, with UTF-8 keys and byte values, backed by the Jellyfish Merkle Tree. The JMT is a sparse merkle tree that records hashed keys, so we also record an index of the keys themselves to allow range queries on keys rather than key hashes. This index, however, is not part of the verifiable consensus state.
A secondary, non-verifiable key-value store with byte keys and byte values, backed directly by RocksDB. This is intended for use building application-specific indexes of the verifiable consensus state.
A tertiary, in-memory object store. This is intended for use implementing accumulators, like lists of data to be batch-processed at the end of the block. The object store clones on read to prevent violations of transactional semantics, so it should be used with immutable data structures like those in the
imcrate that implement copy-on-write behavior internally.
The storage system also supports prefixed “substores”, somewhat similar to the Cosmos SDK’s multistore design. Each substore has a separate JMT, whose root hash is written into the base store under the prefix. This allows use cases like storing IBC data in a subtree. The substore’s non-verifiable store is also stored in a separate RocksDB column family, allowing storage optimizations.
Remember that the chain state is a public API. Mapping from raw byte values
to typed data should be accomplished by means of extension traits. For
penumbra_proto crate provides an extension trait to
automatically (de)serialize into proto or domain types, allowing its use as
an object store.
rpc feature enabled, this crate also provides a GRPC interface to
the key-value store using Tonic.
- Concrete futures types used by the storage crate.
- A cache of changes to the state of the blockchain.
- A wrapper type for a byte slice that implements
Debugby escaping non-printable bytes.
- A root of a
- A snapshot of the underlying storage at a specific state version, suitable for read-only access by multiple threads, e.g., RPC calls.
- An arbitrarily-deeply nested stack of delta updates to an underlying state.
- A handle for a storage instance, backed by RocksDB.
- Extension trait providing
- Read access to chain state.
- Write access to chain state.
metricsRegisters all metrics used by this crate.