Struct MergeReader

pub struct MergeReader {
    hot: BinaryHeap<Node>,
    cold: BinaryHeap<Node>,
    output_batch: Option<Batch>,
    metrics: Metrics,
}

Reader to merge sorted batches.

The merge reader merges Batches from multiple sources that yield sorted batches.

1. Batch is ordered by primary key, time index, sequence desc, op type desc (we can ignore op type as sequence is already unique).
2. Batches from sources must not be empty.

The reader won’t concatenate batches. Each batch returned by the reader also doesn’t contain duplicate rows. But the last (primary key, timestamp) of a batch may be the same as the first one in the next batch.

Fields

hot: BinaryHeap<Node>

Holds Nodes whose key range of current batch is overlapped with the merge window. Each node yields batches from a source.

Node in this heap must not be empty. A merge window is the (primary key, timestamp) range of the root node in the hot heap.

cold: BinaryHeap<Node>

Holds Node whose key range of current batch isn’t overlapped with the merge window.

Node in this heap must not be empty.

output_batch: Option<Batch>

Batch to output.

metrics: Metrics

Local metrics.

Implementations

impl MergeReader

pub async fn new(sources: Vec<Source>) -> Result<MergeReader>

Creates and initializes a new MergeReader.

fn refill_hot(&mut self)

Moves nodes in cold heap, whose key range is overlapped with current merge window to hot heap.

async fn fetch_batch_from_hottest(&mut self) -> Result<()>

Fetches one batch from the hottest node.

async fn fetch_rows_from_hottest(&mut self) -> Result<()>

Fetches non-duplicated rows from the hottest node.

fn reheap(&mut self, node: Node) -> Result<()>

Push the node popped from hot back to a proper heap.

fn maybe_output_batch( batch: Batch, output_batch: &mut Option<Batch>, ) -> Result<()>

If filter_deleted is set to true, removes deleted entries and sets the batch to the output_batch.

Ignores the batch if it is empty.

Trait Implementations

impl BatchReader for MergeReader

fn next_batch<'life0, 'async_trait>( &'life0 mut self, ) -> Pin<Box<dyn Future<Output = Result<Option<Batch>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Fetch next Batch.

impl Drop for MergeReader

fn drop(&mut self)

Executes the destructor for this type.