Module flat_format 

Format to store in parquet.

It can store both encoded primary key and raw key columns.

We store two additional internal columns at last:

• __primary_key, the encoded primary key of the row (tags). Type: dictionary(uint32, binary)
• __sequence, the sequence number of a row. Type: uint64
• __op_type, the op type of the row. Type: uint8

The format is

primary key columns, field columns, time index, encoded primary key, __sequence, __op_type.

It stores field columns in the same order as [RegionMetadata::field_columns()](store_api::metadata::RegionMetadata::field_columns())
and stores primary key columns in the same order as [RegionMetadata::primary_key].

Structs

FlatConvertFormat 🔒

Converts a batch that doesn’t have decoded primary key columns into a batch that has decoded primary key columns in flat format.

FlatReadFormat

Helper for reading the flat SST format with projection.

FlatWriteFormat 🔒

Helper for writing the SST format.

ParquetFlat 🔒

Helper to reads the parquet in flat format directly.

ParquetPrimaryKeyToFlat 🔒

Helper to reads the parquet from primary key format into the flat format.

Enums

ParquetAdapter 🔒

Wraps the parquet helper for different formats.

Functions

op_type_column_index 🔒

Returns the position of the op type key column.

primary_key_column_index 🔒

Returns the position of the primary key column.

sequence_column_index 🔒

Returns the position of the sequence column.

sst_column_id_indices 🔒

Returns a map that the key is the column id and the value is the column position in the SST. It only supports SSTs with raw primary key columns.

time_index_column_index 🔒

Returns the position of the time index column.