Struct Batch

pub struct Batch {
    primary_key: Vec<u8>,
    pk_values: Option<CompositeValues>,
    timestamps: VectorRef,
    sequences: Arc<UInt64Vector>,
    op_types: Arc<UInt8Vector>,
    fields: Vec<BatchColumn>,
    fields_idx: Option<HashMap<ColumnId, usize>>,
}

Storage internal representation of a batch of rows for a primary key (time series).

Rows are sorted by primary key, timestamp, sequence desc, op_type desc. Fields always keep the same relative order as fields in RegionMetadata.

Fields

primary_key: Vec<u8>

Primary key encoded in a comparable form.

pk_values: Option<CompositeValues>

Possibly decoded primary_key values. Some places would decode it in advance.

timestamps: VectorRef

Timestamps of rows, should be sorted and not null.

sequences: Arc<UInt64Vector>

Sequences of rows

UInt64 type, not null.

op_types: Arc<UInt8Vector>

Op types of rows

UInt8 type, not null.

fields: Vec<BatchColumn>

Fields organized in columnar format.

fields_idx: Option<HashMap<ColumnId, usize>>

Cache for field index lookup.

Implementations

impl Batch

pub fn new( primary_key: Vec<u8>, timestamps: VectorRef, sequences: Arc<UInt64Vector>, op_types: Arc<UInt8Vector>, fields: Vec<BatchColumn>, ) -> Result<Batch>

Creates a new batch.

pub fn with_fields(self, fields: Vec<BatchColumn>) -> Result<Batch>

Tries to set fields for the batch.

pub fn primary_key(&self) -> &[u8]

Returns primary key of the batch.

pub fn pk_values(&self) -> Option<&CompositeValues>

Returns possibly decoded primary-key values.

pub fn set_pk_values(&mut self, pk_values: CompositeValues)

Sets possibly decoded primary-key values.

pub fn fields(&self) -> &[BatchColumn]

Returns fields in the batch.

pub fn timestamps(&self) -> &VectorRef

Returns timestamps of the batch.

pub fn sequences(&self) -> &Arc<UInt64Vector>

Returns sequences of the batch.

pub fn op_types(&self) -> &Arc<UInt8Vector>

Returns op types of the batch.

pub fn num_rows(&self) -> usize

Returns the number of rows in the batch.

pub fn is_empty(&self) -> bool

Returns true if the number of rows in the batch is 0.

pub fn first_timestamp(&self) -> Option<Timestamp>

Returns the first timestamp in the batch or None if the batch is empty.

pub fn last_timestamp(&self) -> Option<Timestamp>

Returns the last timestamp in the batch or None if the batch is empty.

pub fn first_sequence(&self) -> Option<SequenceNumber>

Returns the first sequence in the batch or None if the batch is empty.

pub fn last_sequence(&self) -> Option<SequenceNumber>

Returns the last sequence in the batch or None if the batch is empty.

pub fn set_primary_key(&mut self, primary_key: Vec<u8>)

Replaces the primary key of the batch.

Notice that this Batch also contains a maybe-exist pk_values. Be sure to update that field as well.

pub fn slice(&self, offset: usize, length: usize) -> Batch

Slice the batch, returning a new batch.

Panics

Panics if offset + length > self.num_rows().

pub fn concat(batches: Vec<Batch>) -> Result<Batch>

Takes batches and concat them into one batch.

All batches must have the same primary key.

pub fn filter_deleted(&mut self) -> Result<()>

Removes rows whose op type is delete.

pub fn filter(&mut self, predicate: &BooleanVector) -> Result<()>

pub fn filter_by_sequence( &mut self, sequence: Option<SequenceNumber>, ) -> Result<()>

Filters rows by the given sequence. Only preserves rows with sequence less than or equal to sequence.

pub fn sort(&mut self, dedup: bool) -> Result<()>

Sorts rows in the batch. If dedup is true, it also removes duplicated rows according to primary keys.

It orders rows by timestamp, sequence desc and only keep the latest row for the same timestamp. It doesn’t consider op type as sequence should already provide uniqueness for a row.

pub fn memory_size(&self) -> usize

Returns the estimated memory size of the batch.

pub(crate) fn projected_fields( metadata: &RegionMetadata, projection: &[ColumnId], ) -> Vec<(ColumnId, ConcreteDataType)>

Returns ids and datatypes of fields in the Batch after applying the projection.

pub(crate) fn timestamps_native(&self) -> Option<&[i64]>

Returns timestamps in a native slice or None if the batch is empty.

fn take_in_place(&mut self, indices: &UInt32Vector) -> Result<()>

Takes the batch in place.

fn get_timestamp(&self, index: usize) -> Timestamp

Gets a timestamp at given index.

Panics

Panics if index is out-of-bound or the timestamp vector returns null.

pub(crate) fn get_sequence(&self, index: usize) -> SequenceNumber

Gets a sequence at given index.

Panics

Panics if index is out-of-bound or the sequence vector returns null.

pub(crate) fn check_monotonic(&self) -> Result<(), String>

Checks the batch is monotonic by timestamps.

pub(crate) fn check_next_batch(&self, other: &Batch) -> Result<(), String>

Returns Ok if the given batch is behind the current batch.

pub fn pk_col_value( &mut self, codec: &dyn PrimaryKeyCodec, col_idx_in_pk: usize, column_id: ColumnId, ) -> Result<Option<&Value>>

Returns the value of the column in the primary key.

Lazily decodes the primary key and caches the result.

pub fn field_col_value(&mut self, column_id: ColumnId) -> Option<&BatchColumn>

Returns values of the field in the batch.

Lazily caches the field index.

Trait Implementations

impl Clone for Batch

fn clone(&self) -> Batch

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Batch

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<Batch> for BatchBuilder

fn from(batch: Batch) -> Self

Converts to this type from the input type.

impl PartialEq for Batch

fn eq(&self, other: &Batch) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Batch