Module scan_util 

Utilities for scanners.

Structs

CompareCostReverse 🔒

Wrapper for file metrics that compares by total cost in reverse order. This allows using BinaryHeap as a min-heap for efficient top-K selection.

FileScanMetrics

Per-file scan metrics.

PartitionMetrics

Metrics while reading a partition.

PartitionMetricsInner 🔒

PartitionMetricsList 🔒

List of PartitionMetrics.

ScanMetricsSet 🔒

Verbose scan metrics for a partition.

SeriesDistributorMetrics 🔒

Metrics for the series distributor.

SplitRecordBatchStream 🔒

A stream wrapper that splits record batches from an inner stream.

Constants

BATCH_SIZE_THRESHOLD 🔒

Minimum batch size after splitting. The batch size is less than 60 because a series may only have 60 samples per hour.

NUM_SERIES_THRESHOLD 🔒

Number of series threshold for splitting batches.

SPLIT_ROW_THRESHOLD 🔒

Files with row count greater than this threshold can contribute to the estimation.

Functions

build_file_range_scan_stream

Build the stream of scanning the input FileRanges.

build_flat_file_range_scan_stream

Build the stream of scanning the input FileRanges using flat reader that returns RecordBatch.

can_split_series 🔒

maybe_scan_flat_other_ranges 🔒

maybe_scan_other_ranges 🔒

new_filter_metrics 🔒

Creates a new ReaderFilterMetrics with optional apply metrics initialized based on the explain_verbose flag.

scan_file_ranges 🔒

Scans file ranges at index.

scan_flat_file_ranges 🔒

Scans file ranges at index using flat reader that returns RecordBatch.

scan_flat_mem_ranges 🔒

Scans memtable ranges at index using flat format that returns RecordBatch.

scan_mem_ranges 🔒

Scans memtable ranges at index.

should_split_flat_batches_for_merge 🔒

Returns true if splitting flat record batches may improve merge performance.

split_record_batch 🔒

Splits the batch by timestamps.