Skip to main content

check_interleave_bytes_overflow

Function check_interleave_bytes_overflow 

Source
fn check_interleave_bytes_overflow<T: ByteArrayType>(
    batches: &[(usize, RecordBatch)],
    col_idx: usize,
    indices: &[(usize, usize)],
) -> Result<(), ArrowError>
Expand description

Checks whether interleaving the selected rows from byte columns would overflow i32 offsets. Similar to arrow-rs interleave_bytes(), accumulates offsets and returns an error if the capacity exceeds i32::MAX.

TODO(yingwen): Remove this after upgrading to arrow >= 58.1.0, which handles offset overflow in interleave_bytes() natively.

See: https://github.com/apache/arrow-rs/blob/65ad652f2410fc51ad77da1805e85c0a76d9a7ea/arrow-select/src/interleave.rs#L208-L225