Skip to main content

Module pruner

Module pruner 

Source
Expand description

Pruner for parallel file pruning across scanner partitions.

Structsยง

FileBuilderEntry ๐Ÿ”’
Per-file state tracking.
PartitionPruner
Local pruner in a partition that supports prefetching files to prune.
PruneRequest ๐Ÿ”’
Request to prune a file.
Pruner
A pruner that prunes files for all partitions of a scanner.
PrunerInner ๐Ÿ”’
PrunerOptions
Options to create a Pruner.

Constantsยง

PREFETCH_COUNT ๐Ÿ”’
Number of files to pre-fetch ahead of the current position.

Functionsยง

cache_builder_if_needed ๐Ÿ”’
Caches a freshly pruned builder if it is still referenced, and records the corresponding builder memory/count deltas for verbose metrics.
should_cache_builder ๐Ÿ”’
Returns true if a freshly pruned builder should be cached for this file.