Module compute

Computation kernels on Arrow Arrays

Modules

kernels

Computation kernels on Arrow Arrays

Structs

CastOptions

CastOptions provides a way to override the default cast behaviors

FilterBuilder

A builder to construct FilterPredicate

FilterPredicate

A filtering predicate that can be applied to an [Array]

LexicographicalComparator

A lexicographical comparator that wraps given array data (columns) and can lexicographically compare data at given two indices. The lifetime is the same at the data wrapped.

Partitions

A computed set of partitions, see partition

SlicesIterator

An iterator of (usize, usize) each representing an interval [start, end) whose slots of a bitmap [Buffer] are true.

SortColumn

One column to be used in lexicographical sort

SortOptions

Options that define the sort order of a given column

TakeOptions

Options that define how take should behave

Enums

DatePart

Valid parts to extract from date/time/timestamp arrays.

Traits

StringArrayType

A trait for Arrow String Arrays, currently three types are supported:

Functions

and

Performs AND operation on two arrays. If either left or right value is null then the result is also null.

and_kleene

Logical ‘and’ boolean values with Kleene logic

and_not

Performs AND_NOT operation on two arrays. If either left or right value is null then the result is also null.

binary

Allies a binary infallable function to two [PrimitiveArray]s, producing a new [PrimitiveArray]

binary_mut

Applies a binary and infallible function to values in two arrays, replacing the values in the first array in place.

bit_and

Returns the bitwise and of all non-null input values.

bit_or

Returns the bitwise or of all non-null input values.

bit_xor

Returns the bitwise xor of all non-null input values.

bool_and

Returns true if all non-null input values are true, otherwise false.

bool_or

Returns true if any non-null input value is true, otherwise false.

build_filterDeprecated

Returns a prepared function optimized to filter multiple arrays.

can_cast_types

Return true if a value of type from_type can be cast into a value of to_type.

cast

Cast array to the provided data type and return a new Array with type to_type, if possible.

cast_with_options

Try to cast array to to_type if possible.

concat

Concatenate multiple [Array] of the same type into a single [ArrayRef].

concat_batches

Concatenates batches together into a single [RecordBatch].

contains

Perform SQL CONTAINS(left, right)

date_part

Given an array, return a new array with the extracted DatePart as signed 32-bit integer values.

dayDeprecated

Extracts the day of a given temporal primitive array as an array of integers

day_dynDeprecated

Extracts the day of a given temporal array as an array of integers.

doyDeprecated

Extracts the day of year of a given temporal primitive array as an array of integers.

doy_dynDeprecated

Extracts the day of year of a given temporal array as an array of integers.

ends_with

Perform SQL ENDSWITH(left, right)

filter

Returns a filtered values [Array] where the corresponding elements of predicate are true.

filter_record_batch

Returns a filtered [RecordBatch] where the corresponding elements of predicate are true.

hourDeprecated

Extracts the hours of a given temporal primitive array as an array of integers within the range of [0, 23].

hour_dynDeprecated

Extracts the hours of a given array as an array of integers within the range of [0, 23]. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

ilike

Perform SQL left ILIKE right

in_list

Checks if a [GenericListArray] contains a value in the [PrimitiveArray]

in_list_utf8

Checks if a [GenericListArray] contains a value in the [GenericStringArray]

interleave

Takes elements by index from a list of [Array], creating a new [Array] from those values.

interleave_record_batch

Interleave rows by index from multiple [RecordBatch] instances and return a new [RecordBatch].

is_not_null

Returns a non-null [BooleanArray] with whether each value of the array is not null.

is_null

Returns a non-null [BooleanArray] with whether each value of the array is null.

lexsort

Sort a list of ArrayRef using SortOptions provided for each array.

lexsort_to_indices

Sort elements lexicographically from a list of ArrayRef into an unsigned integer (UInt32Array) of indices.

like

Perform SQL left LIKE right

max

Returns the maximum value in the array, according to the natural order. For floating point arrays any NaN values are considered to be greater than any other non-null value

max_array

Returns the max of values in the array of ArrowNumericType type, or dictionary array with value of ArrowNumericType type.

max_binary

Returns the maximum value in the binary array, according to the natural order.

max_binary_view

Returns the maximum value in the binary view array, according to the natural order.

max_boolean

Returns the maximum value in the boolean array

max_string

Returns the maximum value in the string array, according to the natural order.

max_string_view

Returns the maximum value in the string view array, according to the natural order.

microsecondDeprecated

Extracts the microseconds of a given temporal primitive array as an array of integers

microsecond_dynDeprecated

Extracts the microseconds of a given temporal primitive array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

millisecondDeprecated

Extracts the milliseconds of a given temporal primitive array as an array of integers

millisecond_dynDeprecated

Extracts the milliseconds of a given temporal primitive array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

min

Returns the minimum value in the array, according to the natural order. For floating point arrays any NaN values are considered to be greater than any other non-null value

min_array

Returns the min of values in the array of ArrowNumericType type, or dictionary array with value of ArrowNumericType type.

min_binary

Returns the minimum value in the binary array, according to the natural order.

min_binary_view

Returns the minimum value in the binary view array, according to the natural order.

min_boolean

Returns the minimum value in the boolean array.

min_string

Returns the minimum value in the string array, according to the natural order.

min_string_view

Returns the minimum value in the string view array, according to the natural order.

minuteDeprecated

Extracts the minutes of a given temporal primitive array as an array of integers

minute_dynDeprecated

Extracts the minutes of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

monthDeprecated

Extracts the month of a given temporal primitive array as an array of integers within the range of [1, 12].

month_dynDeprecated

Extracts the month of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

multiply_fixed_point

Perform left * right operation on two decimal arrays. If either left or right value is null then the result is also null.

multiply_fixed_point_checked

Perform left * right operation on two decimal arrays. If either left or right value is null then the result is also null.

multiply_fixed_point_dyn

Perform left * right operation on two decimal arrays. If either left or right value is null then the result is also null.

nanosecondDeprecated

Extracts the nanoseconds of a given temporal primitive array as an array of integers

nanosecond_dynDeprecated

Extracts the nanoseconds of a given temporal primitive array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

nilike

Perform SQL left NOT ILIKE right

nlike

Perform SQL left NOT LIKE right

not

Performs unary NOT operation on an arrays. If value is null then the result is also null.

nullif

Returns a new array with the same values and the validity bit to false where the corresponding element ofright is true.

num_days_from_mondayDeprecated

Extracts the day of week of a given temporal primitive array as an array of integers.

num_days_from_monday_dynDeprecated

Extracts the day of week of a given temporal array as an array of integers.

num_days_from_sundayDeprecated

Extracts the day of week of a given temporal primitive array as an array of integers, starting at Sunday.

num_days_from_sunday_dynDeprecated

Extracts the day of week of a given temporal array as an array of integers, starting at Sunday.

or

Performs OR operation on two arrays. If either left or right value is null then the result is also null.

or_kleene

Logical ‘or’ boolean values with Kleene logic

partial_sort

It’s unstable_sort, may not preserve the order of equal elements

partition

Given a list of lexicographically sorted columns, computes the Partitions, where a partition consists of the set of consecutive rows with equal values

prep_null_mask_filter

Remove null values by do a bitmask AND operation with null bits and the boolean bits.

quarterDeprecated

Extracts the quarter of a given temporal primitive array as an array of integers within the range of [1, 4].

quarter_dynDeprecated

Extracts the quarter of a given temporal array as an array of integersa within the range of [1, 4]. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

rank

Assigns a rank to each value in array based on its position in the sorted order

regexp_is_match

Return BooleanArray indicating which strings in an array match an array of regular expressions.

regexp_is_match_scalar

Return BooleanArray indicating which strings in an array match a single regular expression.

regexp_is_match_utf8Deprecated

Perform SQL array ~ regex_array operation on [StringArray] / [LargeStringArray]. If regex_array element has an empty value, the corresponding result value is always true.

regexp_is_match_utf8_scalarDeprecated

Perform SQL array ~ regex_array operation on [StringArray] / [LargeStringArray] and a scalar.

regexp_match

Extract all groups matched by a regular expression for a given String array.

secondDeprecated

Extracts the seconds of a given temporal primitive array as an array of integers

second_dynDeprecated

Extracts the seconds of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

shift

Shifts array by defined number of items (to left or right) A positive value for offset shifts the array to the right a negative value shifts the array to the left.

sort

Sort the ArrayRef using SortOptions.

sort_limit

Sort the ArrayRef partially.

sort_to_indices

Sort elements from ArrayRef into an unsigned integer (UInt32Array) of indices. Floats are sorted using IEEE 754 totalOrder. limit is an option for partial_sort.

starts_with

Perform SQL STARTSWITH(left, right)

sum

Returns the sum of values in the primitive array.

sum_array

Returns the sum of values in the array.

sum_array_checked

Returns the sum of values in the array.

sum_checked

Returns the sum of values in the primitive array.

take

Take elements by index from [Array], creating a new [Array] from those indexes.

take_arrays

For each [ArrayRef] in the Vec<ArrayRef>, take elements by index and create a new Vec<ArrayRef> from those indices.

take_record_batch

Take rows by index from [RecordBatch] and returns a new [RecordBatch] from those indexes.

try_binary

Applies the provided fallible binary operation across a and b.

try_binary_mut

Applies the provided fallible binary operation across a and b by mutating the mutable [PrimitiveArray] a with the results.

try_unary

See [PrimitiveArray::try_unary]

try_unary_mut

See [PrimitiveArray::try_unary_mut]

unary

See [PrimitiveArray::unary]

unary_mut

See [PrimitiveArray::unary_mut]

union_extract

Returns the value of the target field when selected, or NULL otherwise.

weekDeprecated

Extracts the week of a given temporal primitive array as an array of integers

week_dynDeprecated

Extracts the week of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

yearDeprecated

Extracts the years of a given temporal primitive array as an array of integers

year_dynDeprecated

Extracts the years of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

Type Aliases

FilterDeprecated

Function that can filter arbitrary arrays