Skip to main content

Module writer

Module writer 

Source
Expand description

Parquet writer.

Structsยง

ObjectStoreWriterFactory
ParquetWriter
Parquet SST writer.
SizeAwareWriter ๐Ÿ”’
Workaround for [AsyncArrowWriter] does not provide a method to get total bytes written after close.
SourceStats ๐Ÿ”’
SstWriteContext ๐Ÿ”’
Protection and resource limits travel together for table-compaction output.
SstWriteLimits ๐Ÿ”’
Optional limits used by multi-output compaction. A batch may temporarily exceed the working threshold before its check; callers reserve that workspace.
WriteMemory ๐Ÿ”’
Tracks retained footer memory across rolled files as well as the current file.

Enumsยง

FlatBatchConverter ๐Ÿ”’
Converts a flat RecordBatch for writing to parquet.
SeriesBoundarySplit ๐Ÿ”’
Result of splitting a batch at the next series boundary.
SstArrowWriter ๐Ÿ”’
Arrow writer of the current SST file.

Traitsยง

SstWriteGuard
Registers durable ownership before either an SST or its index is published to storage.
WriterFactory

Functionsยง

encoded_primary_keys ๐Ÿ”’
Returns the dictionary keys and values of the encoded __primary_key column.
split_at_next_series ๐Ÿ”’
Splits batch at the first row whose encoded primary key differs from current_primary_key, the last primary key written to the current file.
timestamp_range_from_array ๐Ÿ”’
Gets min and max timestamp from an timestamp array.