Struct FlatMergeReader 

pub struct FlatMergeReader {
    algo: MergeAlgo<GenericNode<BoxedRecordBatchStream>>,
    in_progress: BatchBuilder,
    output_batch: Option<RecordBatch>,
    batch_size: usize,
    metrics: MergeMetrics,
    metrics_reporter: Option<Arc<dyn MergeMetricsReport>>,
}

Iterator to merge multiple sorted iterators into a single sorted iterator.

All iterators must be sorted by primary key, time index, sequence desc.

Input batches must be in the flat format: the last four columns are time index, __primary_key, __sequence and __op_type. Row comparison decodes these internal columns and returns an error on batches that don’t match the flat format; the name-based gate in BatchBuilder only makes output assembly degrade gracefully on generic schemas, not sorting.

Fields

algo: MergeAlgo<GenericNode<BoxedRecordBatchStream>>

The merge algorithm to maintain heaps.

in_progress: BatchBuilder

Current buffered rows to output.

output_batch: Option<RecordBatch>

Non-empty batch to output.

batch_size: usize

Batch size to merge rows. This is not a hard limit, the iterator may return smaller batches to avoid concatenating rows.

metrics: MergeMetrics

Local metrics.

metrics_reporter: Option<Arc<dyn MergeMetricsReport>>

Optional metrics reporter.

Implementations

impl FlatMergeReader

pub async fn new( schema: SchemaRef, iters: Vec<BoxedRecordBatchStream>, batch_size: usize, metrics_reporter: Option<Arc<dyn MergeMetricsReport>>, ) -> Result<Self>

Creates a new iterator to merge sorted iters.

pub async fn next_batch(&mut self) -> Result<Option<RecordBatch>>

Fetches next sorted batch.

pub fn into_stream(self) -> impl Stream<Item = Result<RecordBatch>>

Converts the reader into a stream.

async fn fetch_batch_from_hottest(&mut self) -> Result<()>

Fetches a batch from the hottest node.

async fn fetch_row_from_hottest(&mut self) -> Result<()>

Fetches a row from the hottest node.

fn maybe_output_batch( batch: RecordBatch, output_batch: &mut Option<RecordBatch>, )

Adds the batch to the output batch if it is not empty.

Trait Implementations

impl Drop for FlatMergeReader

fn drop(&mut self)

Executes the destructor for this type.