Struct RangeArray

pub struct RangeArray {
    array: DictionaryArray<Int64Type>,
}

An compound logical “array” type. Represent serval ranges (slices) of one array. It’s useful to use case like compute sliding window, or range selector from promql.

It’s build on top of Arrow’s [DictionaryArray]. [DictionaryArray] contains two sub-arrays, one for dictionary key and another for dictionary value. Both of them can be arbitrary types, but here the key array is fixed to u32 type.

            │ ┌─────┬─────┬─────┬─────┐
     Two    │ │ i64 │ i64 │ i64 │ i64 │      Keys
   Arrays   │ └─────┴─────┴─────┴─────┘     (Fixed to i64)
     In     │
  Arrow's   │ ┌────────────────────────────┐
Dictionary  │ │                            │ Values
   Array    │ └────────────────────────────┘(Any Type)

Because the i64 key array is reinterpreted into two u32 for offset and length in RangeArray to represent a “range”:

63            32│31             0
┌───────────────┼───────────────┐
│  offset (u32) │  length (u32) │
└───────────────┼───────────────┘

Then the [DictionaryArray] can be expanded to several ranges like this:

Keys
┌───────┬───────┬───────┐      ┌───────┐
│ (0,2) │ (1,2) │ (2,2) │ ─┐   │[A,B,C]│ values.slice(0,2)
└───────┴───────┴───────┘  │   ├───────┤
 Values                    ├─► │[B,C,D]│ values.slice(1,2)
┌───┬───┬───┬───┬───┐      │   ├───────┤
│ A │ B │ C │ D │ E │     ─┘   │[C,D,E]│ values.slice(2,2)
└───┴───┴───┴───┴───┘          └───────┘

Fields

array: DictionaryArray<Int64Type>

Implementations

impl RangeArray

pub const fn key_type() -> DataType

pub fn value_type(&self) -> DataType

pub fn try_new(dict: DictionaryArray<Int64Type>) -> Result<Self>

pub fn from_ranges<R>(values: ArrayRef, ranges: R) -> Result<Self>

where R: IntoIterator<Item = RangeTuple> + Clone,

pub unsafe fn from_ranges_unchecked<R>(values: ArrayRef, ranges: R) -> Self

where R: IntoIterator<Item = RangeTuple>,

Construct RangeArray from given range without checking its validity.

Safety

Caller should ensure the given range are valid. Otherwise use from_ranges instead.

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn get(&self, index: usize) -> Option<ArrayRef>

pub fn into_dict(self) -> DictionaryArray<Int64Type>

Return the underlying Arrow’s [DictionaryArray]. Notes the dictionary array might be invalid from Arrow’s definition. Be care if try to access this dictionary through Arrow’s API.

fn check_ranges<R>(value_len: usize, ranges: R) -> Result<()>

where R: IntoIterator<Item = RangeTuple>,

pub fn convert_field(field: &Field) -> Field

Change the field’s datatype to the type after processed by RangeArray. Like Utf8 will become Dictionary<Int64, Utf8>.

pub fn convert_data_type(value_type: DataType) -> DataType

Build datatype of wrapped RangeArray on given value type.

pub fn values(&self) -> &ArrayRef

pub fn ranges(&self) -> impl Iterator<Item = Option<RangeTuple>> + '_

Trait Implementations

impl Array for RangeArray

fn as_any(&self) -> &dyn Any

Returns the array as Any so that it can be downcasted to a specific implementation.

fn into_data(self) -> ArrayData

Returns the underlying data of this array

fn to_data(&self) -> ArrayData

Returns the underlying data of this array

fn slice(&self, offset: usize, length: usize) -> ArrayRef

Returns a zero-copy slice of this array with the indicated offset and length.

fn nulls(&self) -> Option<&NullBuffer>

Returns the null buffer of this array if any.

fn data_type(&self) -> &DataType

Returns a reference to the [DataType] of this array.

fn len(&self) -> usize

Returns the length (i.e., number of elements) of this array.

fn is_empty(&self) -> bool

Returns whether this array is empty.

fn offset(&self) -> usize

Returns the offset into the underlying data used by this array(-slice). Note that the underlying data can be shared by many arrays. This defaults to 0.

fn get_buffer_memory_size(&self) -> usize

Returns the total number of bytes of memory pointed to by this array. The buffers store bytes in the Arrow memory format, and include the data as well as the validity map. Note that this does not always correspond to the exact memory usage of an array, since multiple arrays can share the same buffers or slices thereof.

fn get_array_memory_size(&self) -> usize

Returns the total number of bytes of memory occupied physically by this array. This value will always be greater than returned by get_buffer_memory_size() and includes the overhead of the data structures that contain the pointers to the various buffers.

fn shrink_to_fit(&mut self)

Shrinks the capacity of any exclusively owned buffer as much as possible

fn logical_nulls(&self) -> Option<NullBuffer>

Returns a potentially computed [NullBuffer] that represents the logical null values of this array, if any.

fn is_null(&self, index: usize) -> bool

Returns whether the element at index is null according to [Array::nulls]

fn is_valid(&self, index: usize) -> bool

Returns whether the element at index is not null, the opposite of [Self::is_null].

fn null_count(&self) -> usize

Returns the total number of physical null values in this array.

fn logical_null_count(&self) -> usize

Returns the total number of logical null values in this array.

fn is_nullable(&self) -> bool

Returns false if the array is guaranteed to not contain any logical nulls

impl Debug for RangeArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.