Struct SstRangeIndexWriter 

pub struct SstRangeIndexWriter {
    codec: SparsePrimaryKeyCodec,
    schema: SchemaRef,
    writer: ParquetIndexWriter,
    current_row_group_id: Option<u32>,
    current_row_group_offset: i64,
    last_primary_key: Option<Vec<u8>>,
    current_row: Option<RangeIndexRow>,
    buffered_rows: Vec<RangeIndexRow>,
    metrics: SstRangeIndexWriterMetrics,
    failed: bool,
}

Incrementally writes the series row ranges of one SST to a Parquet file.

Each non-empty input batch is tagged with its source SST row-group ID. A row group may be supplied in multiple batches, but IDs must start at zero and be contiguous. Input primary keys must preserve the SST’s global sort order.

Fields

codec: SparsePrimaryKeyCodec

schema: SchemaRef

writer: ParquetIndexWriter

current_row_group_id: Option<u32>

current_row_group_offset: i64

last_primary_key: Option<Vec<u8>>

current_row: Option<RangeIndexRow>

buffered_rows: Vec<RangeIndexRow>

metrics: SstRangeIndexWriterMetrics

failed: bool

Implementations

impl SstRangeIndexWriter

pub async fn try_new( metadata: RegionMetadataRef, object_store: ObjectStore, path: &str, options: SstRangeIndexWriterOptions, ) -> Result<Self>

Creates a writer for path in object_store.

The file name in path must be unique in the object store so aborting the writer only removes temporary files that belong to this writer.

pub fn metrics(&self) -> &SstRangeIndexWriterMetrics

Returns the metrics collected so far.

pub async fn write( &mut self, row_group_id: u32, batch: &RecordBatch, ) -> Result<()>

Adds one batch from row_group_id of the source SST.

The batch may be a primary-key-only projection or a full flat batch. Its schema must contain a named __primary_key column. After this method returns an error, callers must call Self::abort.

pub async fn finish(self) -> Result<SstRangeIndexWriterMetrics>

Finishes and commits the index file.

pub async fn abort(self) -> Result<SstRangeIndexWriterMetrics>

Aborts the writer and removes incomplete output files.

async fn write_inner( &mut self, row_group_id: u32, batch: &RecordBatch, ) -> Result<()>

async fn enter_row_group(&mut self, row_group_id: u32) -> Result<()>

async fn write_binary_primary_keys( &mut self, primary_keys: &BinaryArray, ) -> Result<()>

async fn write_dictionary_primary_keys( &mut self, primary_keys: &DictionaryArray<UInt32Type>, ) -> Result<()>

async fn update_primary_key( &mut self, primary_key: &[u8], run_len: usize, ) -> Result<()>

async fn finish_current_row(&mut self) -> Result<()>

async fn flush_rows(&mut self) -> Result<()>

async fn finish_inner(&mut self) -> Result<()>

async fn cleanup(&mut self)