Crate cnidarium

Storage and management of chain state, backed by Jellyfish Merkle Trees and RocksDB.

This crate provides a versioned, verifiable key-value store that also supports lightweight, copy-on-write snapshots and transactional semantics. The Storage type is a handle for an instance of a backing store, implemented using RocksDB. The storage records a sequence of versioned Snapshots. The Snapshot type is a lightweight snapshot of a particular version of the chain state.

Each Snapshot instance can also be used as the basis for a copy-on-write fork to build up changes before committing them to persistent storage. The StateDelta type collects a group of writes, which can then be applied to the (in-memory) StateDelta overlay. Finally, the changes accumulated in the StateDelta instance can be committed to the persistent Storage.

Reads are performed with the StateRead trait, implemented by both Snapshot and StateDelta, and reflect any currently cached writes. Writes are performed with the StateWrite trait, which is only implemented for StateDelta.

The storage system provides three data stores:

• A verifiable key-value store, with UTF-8 keys and byte values, backed by the Jellyfish Merkle Tree. The JMT is a sparse merkle tree that records hashed keys, so we also record an index of the keys themselves to allow range queries on keys rather than key hashes. This index, however, is not part of the verifiable consensus state.

• A secondary, non-verifiable key-value store with byte keys and byte values, backed directly by RocksDB. This is intended for use building application-specific indexes of the verifiable consensus state.

• A tertiary, in-memory object store. This is intended for use implementing accumulators, like lists of data to be batch-processed at the end of the block. The object store clones on read to prevent violations of transactional semantics, so it should be used with immutable data structures like those in the im crate that implement copy-on-write behavior internally.

The storage system also supports prefixed “substores”, somewhat similar to the Cosmos SDK’s multistore design. Each substore has a separate JMT, whose root hash is written into the base store under the prefix. This allows use cases like storing IBC data in a subtree. The substore’s non-verifiable store is also stored in a separate RocksDB column family, allowing storage optimizations.

Remember that the chain state is a public API. Mapping from raw byte values to typed data should be accomplished by means of extension traits. For instance, the penumbra_proto crate provides an extension trait to automatically (de)serialize into proto or domain types, allowing its use as an object store.

With the rpc feature enabled, this crate also provides a GRPC interface to the key-value store using Tonic.

Modules

• future
  Concrete futures types used by the storage crate.
• rpcrpc

Structs

• Cache
  A cache of changes to the state of the blockchain.
• EscapedByteSlice
  A wrapper type for a byte slice that implements Debug by escaping non-printable bytes.
• RootHash
  A root of a JellyfishMerkleTree.
• Snapshot
  A snapshot of the underlying storage at a specific state version, suitable for read-only access by multiple threads, e.g., RPC calls.
• StagedWriteBatch
  A staged write batch that can be committed to RocksDB.
• StateDelta
  An arbitrarily-deeply nested stack of delta updates to an underlying state.
• Storage
  A handle for a storage instance, backed by RocksDB.
• TempStorage
  A Storage instance backed by a tempfile::TempDir for testing.

Traits

• ArcStateDeltaExt
  Extension trait providing try_begin_transaction() on Arc<StateDelta<S>>.
• StateRead
  Read access to chain state.
• StateWrite
  Write access to chain state.

Functions

• ics23_spec
• register_metricsmetrics
  Registers all metrics used by this crate.