Struct FlatProjectionMapper 

pub struct FlatProjectionMapper {
    metadata: RegionMetadataRef,
    output_schema: SchemaRef,
    column_ids: Vec<ColumnId>,
    batch_schema: Vec<(ColumnId, ConcreteDataType)>,
    is_empty_projection: bool,
    batch_indices: Vec<usize>,
    input_arrow_schema: SchemaRef,
}

Handles projection and converts batches in flat format with correct schema.

This mapper support duplicate and unsorted projection indices. The output schema is determined by the projection indices.

Fields

metadata: RegionMetadataRef

Metadata of the region.

output_schema: SchemaRef

Schema for converted [RecordBatch] to return.

column_ids: Vec<ColumnId>

Ids of columns to project. It keeps ids in the same order as the projection indices to build the mapper. The mapper won’t deduplicate the column ids.

Note that this doesn’t contain the __table_id and __tsid.

batch_schema: Vec<(ColumnId, ConcreteDataType)>

Ids and DataTypes of columns of the expected batch. We can use this to check if the batch is compatible with the expected schema.

It doesn’t contain internal columns but always contains the time index column.

is_empty_projection: bool

true If the original projection is empty.

batch_indices: Vec<usize>

The index in flat format [RecordBatch] for each column in the output [RecordBatch].

input_arrow_schema: SchemaRef

Precomputed Arrow schema for input batches.

Implementations

impl FlatProjectionMapper

pub fn new( metadata: &RegionMetadataRef, projection: impl Iterator<Item = usize>, ) -> Result<Self>

Returns a new mapper with projection. If projection is empty, it outputs [RecordBatch] without any column but only a row count. SELECT COUNT(*) FROM table is an example that uses an empty projection. DataFusion accepts empty RecordBatch and only use its row count in this query.

pub fn all(metadata: &RegionMetadataRef) -> Result<Self>

Returns a new mapper without projection.

pub(crate) fn metadata(&self) -> &RegionMetadataRef

Returns the metadata that created the mapper.

pub(crate) fn column_ids(&self) -> &[ColumnId]

Returns ids of projected columns that we need to read from memtables and SSTs.

pub(crate) fn field_column_start(&self) -> usize

Returns the field column start index in output batch.

pub(crate) fn batch_schema(&self) -> &[(ColumnId, ConcreteDataType)]

Returns ids of columns of the batch that the mapper expects to convert.

pub(crate) fn input_arrow_schema(&self, compaction: bool) -> SchemaRef

Returns the input arrow schema from sources.

The merge reader can use this schema.

pub(crate) fn output_schema(&self) -> SchemaRef

Returns the schema of converted [RecordBatch]. This is the schema that the stream will output. This schema may contain less columns than FlatProjectionMapper::column_ids().

pub(crate) fn empty_record_batch(&self) -> RecordBatch

Returns an empty [RecordBatch].

pub(crate) fn convert(&self, batch: &RecordBatch) -> Result<RecordBatch>

Converts a flat format [RecordBatch] to a normal [RecordBatch].

The batch must match the projection using to build the mapper.