mito2::read::dedup

Trait DedupStrategy

Source 
pub(crate) trait DedupStrategy: Send {
    // Required methods
    fn push_batch(
        &mut self,
        batch: Batch,
        metrics: &mut DedupMetrics,
    ) -> Result<Option<Batch>>;
    fn finish(&mut self, metrics: &mut DedupMetrics) -> Result<Option<Batch>>;
}
Expand description

Strategy to remove duplicate rows from sorted batches.

Required Methods§

Source

fn push_batch( &mut self, batch: Batch, metrics: &mut DedupMetrics, ) -> Result<Option<Batch>>

Pushes a batch to the dedup strategy. Returns the deduplicated batch.

Source

fn finish(&mut self, metrics: &mut DedupMetrics) -> Result<Option<Batch>>

Finishes the deduplication and resets the strategy.

Users must ensure that push_batch is called for all batches before calling this method.

Implementors§