Module store 

Object store WAL construction, recovery, writes and region reads.

Structs

Actor 🔒

The actor that owns the open batch and the sealed batches until they are durable.

DurableWaiter 🔒

A caller waiting until a region is durable through an entry id.

FetchedObject 🔒

An object as recovery fetched it: its key, header and footer.

ObjectStoreLogStore

A log store over the immutable WAL objects under one prefix.

PendingAppend 🔒

An append waiting for the object that holds its entries.

Recovered 🔒

What recovery rebuilt from the objects under a prefix.

SealedBatch 🔒

A sealed and encoded batch that holds its object sequence and waits to be created, indexed and acknowledged.

Enums

Command 🔒

CreateState 🔒

Where the conditional create of a sealed batch stands.

Constants

APPEND_BUFFER 🔒

Appends queued for the actor; a caller beyond them waits to send.

COMMAND_BUFFER 🔒

CREATE_RETRY_DELAY 🔒

Delay before a create that failed transiently is attempted again in the enqueued acknowledgement mode, where no caller is left to retry it.

MAX_IN_FLIGHT_CREATES 🔒

MAX_SEALED_BATCHES 🔒

Number of sealed batches that may wait to be created or indexed. One admission can seal two batches, the open batch before an append that would exhaust its positions and the append’s own batch, so the actor takes an append from its channel only while two more fit: a stalled object store holds callers back instead of growing the backlog.

MIN_FLUSH_INTERVAL 🔒

RECOVERY_CONCURRENCY 🔒

Number of objects whose footers recovery fetches at a time.

RECOVERY_TAIL_WINDOW 🔒

Bytes recovery reads from the end of an object in one request. The window holds the trailer and the footer of an object with up to 1364 regions of 48 bytes each, so a second request for the footer is rare.

Traits

WalObjectIo 🔒

Object access of the store, so tests can inject failures.

Functions

durable_entry_ids 🔒

encode_batch 🔒

epoch_of 🔒

fetch_footer 🔒

Reads the header, trailer and footer of object and verifies them: the header must carry the sequence of the key, the trailer must be well formed, the footer must match the checksum the trailer holds and its segments must tile the object body.

fetch_footers 🔒

Fetches and verifies the headers and footers of objects, up to concurrency objects at a time, and returns them ordered by object sequence whatever the order the fetches complete in. The first failure abandons the remaining fetches.

finish_recovery 🔒

Indexes the chain of objects, which are ordered by sequence. Objects off the chain are orphans: they are not indexed, but no later object takes their sequences.

is_transient 🔒

Returns true for a storage error that a later attempt may not meet: one the object store reports as temporary, or as persistent, which is how its retry layer reports a temporary error that outlasted its retries.

locate_footer 🔒

Verifies the header and trailer of the object object_seq of object_len bytes from its first bytes head and its last bytes tail, and returns the header and the trailer with the range the footer occupies in the object.

positive_bytes 🔒

record_obsolete 🔒

Records the obsolete watermark of region_id, which never moves down.

recover 🔒

Rebuilds the catalog from object headers and footers, so recovery costs a few small reads per object however large the objects are. Segments are not read; a segment checksum is verified by the read that decodes it. Footers are fetched for up to RECOVERY_CONCURRENCY objects at a time and indexed in sequence order, so the catalog checks the entry ranges of every object against its predecessors like a sequential replay. Only the objects on the chain select_chain picks are indexed.

select_chain 🔒

Returns, in sequence order, the objects on the chain that ends at the tip: the complete object with the largest epoch, then the largest sequence.

set_terminal 🔒

Records error as the terminal error unless one is already recorded, and returns the recorded one.

shared 🔒

Wraps an error that several callers receive.

stale_conflict 🔒

Resolves a create of this store’s epoch that met a different object: an object of an earlier epoch can never be on the chain, so the create fails like a transient error; an object of the same or a later epoch belongs to another writer and conflict is returned.

start_epoch 🔒

Writes the empty object that starts the epoch of this instance at object_seq, linked to the recovered tip, and returns it as the tip later objects extend. The epoch is one above the sequence the create claims, so no two instances share one however their opens interleave, and it is above the epoch of every object recovery listed.

terminal 🔒

Type Aliases

AppendResponse 🔒

CreateOutcome 🔒

ObsoleteEntryIds 🔒

QueuedAppend 🔒

TerminalError 🔒