Skip to main content

merge_and_dedup_with_batch_size

Function merge_and_dedup_with_batch_size 

Source
pub fn merge_and_dedup_with_batch_size(
    schema: &SchemaRef,
    append_mode: bool,
    merge_mode: MergeMode,
    field_column_start: usize,
    input_iters: Vec<BoxedRecordBatchIterator>,
    batch_size: usize,
) -> Result<BoxedRecordBatchIterator>
Expand description

Merges and optionally deduplicates record batch iterators with an explicit output batch size.

batch_size controls the target number of rows in batches assembled by the merge iterator and is clamped to at least one. The other arguments have the same meaning as in merge_and_dedup.