Struct FulltextIndexApplier 

pub struct FulltextIndexApplier {
    requests: Arc<BTreeMap<ColumnId, FulltextRequest>>,
    index_source: IndexSource,
    bloom_filter_index_cache: Option<Arc<IndexCache<(FileId, ColumnId, Tag), BloomFilterMeta>>>,
    predicate_key: PredicateKey,
}

FulltextIndexApplier is responsible for applying fulltext index to the provided SST files

Fields

requests: Arc<BTreeMap<ColumnId, FulltextRequest>>

Requests to be applied.

index_source: IndexSource

The source of the index.

bloom_filter_index_cache: Option<Arc<IndexCache<(FileId, ColumnId, Tag), BloomFilterMeta>>>

Cache for bloom filter index.

predicate_key: PredicateKey

Predicate key. Used to identify the predicate and fetch result from cache.

Implementations

impl FulltextIndexApplier

pub fn new( table_dir: String, path_type: PathType, store: ObjectStore, requests: BTreeMap<ColumnId, FulltextRequest>, puffin_manager_factory: PuffinManagerFactory, ) -> Self

Creates a new FulltextIndexApplier.

pub fn with_file_cache(self, file_cache: Option<Arc<FileCache>>) -> Self

Sets the file cache.

pub fn with_puffin_metadata_cache( self, puffin_metadata_cache: Option<PuffinMetadataCacheRef>, ) -> Self

Sets the puffin metadata cache.

pub fn with_bloom_filter_cache( self, bloom_filter_index_cache: Option<Arc<IndexCache<(FileId, ColumnId, Tag), BloomFilterMeta>>>, ) -> Self

Sets the bloom filter cache.

pub fn predicate_key(&self) -> &PredicateKey

Returns the predicate key.

impl FulltextIndexApplier

pub async fn apply_fine( &self, file_id: RegionFileId, file_size_hint: Option<u64>, ) -> Result<Option<BTreeSet<RowId>>>

Applies fine-grained fulltext index to the specified SST file. Returns the row ids that match the queries.

async fn apply_fine_one_column( &self, file_size_hint: Option<u64>, file_id: RegionFileId, column_id: ColumnId, request: &FulltextRequest, ) -> Result<Option<BTreeSet<RowId>>>

impl FulltextIndexApplier

pub async fn apply_coarse( &self, file_id: RegionFileId, file_size_hint: Option<u64>, row_groups: impl Iterator<Item = (usize, bool)>, ) -> Result<Option<Vec<(usize, Vec<Range<usize>>)>>>

Applies coarse-grained fulltext index to the specified SST file. Returns (row group id -> ranges) that match the queries.

Row group id existing in the returned result means that the row group is searched. Empty ranges means that the row group is searched but no rows are found.

async fn apply_coarse_one_column( &self, file_id: RegionFileId, file_size_hint: Option<u64>, column_id: ColumnId, terms: &[FulltextTerm], output: &mut [(usize, Vec<Range<usize>>)], ) -> Result<bool>

fn init_coarse_output( row_groups: impl Iterator<Item = (usize, bool)>, ) -> (Vec<(usize, Range<usize>)>, Vec<(usize, Vec<Range<usize>>)>)

Initializes the coarse output. Must call adjust_coarse_output after applying bloom filters.

row_groups is a list of (row group length, whether to search).

Returns (input, output):

• input is a list of (row group index to search, row group range based on start of the file).
• output is a list of (row group index to search, row group ranges based on start of the file).

fn adjust_coarse_output( input: Vec<(usize, Range<usize>)>, output: &mut [(usize, Vec<Range<usize>>)], )

Adjusts the coarse output. Makes the output ranges based on row group start.

fn terms_to_predicates( terms: &[FulltextTerm], config: &Config, ) -> Vec<InListPredicate>

Converts terms to predicates.

Split terms by non-alphanumeric characters and convert them to lowercase if case-insensitive. Multiple terms are combined with AND semantics.

fn term_to_probes<'a>( term: &'a str, config: &'a Config, ) -> impl Iterator<Item = Vec<u8>> + 'a