Struct MitoConfig 

pub struct MitoConfig {

    pub num_workers: usize,
    pub worker_channel_size: usize,
    pub worker_request_batch_size: usize,
    pub manifest_checkpoint_distance: u64,
    pub experimental_manifest_keep_removed_file_count: usize,
    pub experimental_manifest_keep_removed_file_ttl: Duration,
    pub compress_manifest: bool,
    pub max_background_index_builds: usize,
    pub max_background_flushes: usize,
    pub max_background_compactions: usize,
    pub max_background_purges: usize,
    pub auto_flush_interval: Duration,
    pub global_write_buffer_size: ReadableSize,
    pub global_write_buffer_reject_size: ReadableSize,
    pub sst_meta_cache_size: ReadableSize,
    pub vector_cache_size: ReadableSize,
    pub page_cache_size: ReadableSize,
    pub selector_result_cache_size: ReadableSize,
    pub enable_write_cache: bool,
    pub write_cache_path: String,
    pub write_cache_size: ReadableSize,
    pub write_cache_ttl: Option<Duration>,
    pub preload_index_cache: bool,
    pub index_cache_percent: u8,
    pub sst_write_buffer_size: ReadableSize,
    pub parallel_scan_channel_size: usize,
    pub max_concurrent_scan_files: usize,
    pub allow_stale_entries: bool,
    pub scan_memory_limit: MemoryLimit,
    pub index: IndexConfig,
    pub inverted_index: InvertedIndexConfig,
    pub fulltext_index: FulltextIndexConfig,
    pub bloom_filter_index: BloomFilterConfig,
    pub memtable: MemtableConfig,
    pub min_compaction_interval: Duration,
    pub default_experimental_flat_format: bool,
    pub gc: GcConfig,
}

Configuration for MitoEngine. Before using the config, make sure to call MitoConfig::validate() to check if the config is valid.

Fields

num_workers: usize

Number of region workers (default: 1/2 of cpu cores). Sets to 0 to use the default value.

worker_channel_size: usize

Request channel size of each worker (default 128).

worker_request_batch_size: usize

Max batch size for a worker to handle requests (default 64).

manifest_checkpoint_distance: u64

Number of meta action updated to trigger a new checkpoint for the manifest (default 10).

experimental_manifest_keep_removed_file_count: usize

Number of removed files to keep in manifest’s removed_files field before also remove them from removed_files. Mostly for debugging purpose. If set to 0, it will only use keep_removed_file_ttl to decide when to remove files from removed_files field.

experimental_manifest_keep_removed_file_ttl: Duration

How long to keep removed files in the removed_files field of manifest after they are removed from manifest. files will only be removed from removed_files field if both keep_removed_file_count and keep_removed_file_ttl is reached.

compress_manifest: bool

Whether to compress manifest and checkpoint file by gzip (default false).

max_background_index_builds: usize

Max number of running background index build jobs (default: 1/8 of cpu cores).

max_background_flushes: usize

Max number of running background flush jobs (default: 1/2 of cpu cores).

max_background_compactions: usize

Max number of running background compaction jobs (default: 1/4 of cpu cores).

max_background_purges: usize

Max number of running background purge jobs (default: number of cpu cores).

auto_flush_interval: Duration

Interval to auto flush a region if it has not flushed yet (default 30 min).

global_write_buffer_size: ReadableSize

Global write buffer size threshold to trigger flush.

global_write_buffer_reject_size: ReadableSize

Global write buffer size threshold to reject write requests.

sst_meta_cache_size: ReadableSize

Cache size for SST metadata. Setting it to 0 to disable the cache.

vector_cache_size: ReadableSize

Cache size for vectors and arrow arrays. Setting it to 0 to disable the cache.

page_cache_size: ReadableSize

Cache size for pages of SST row groups. Setting it to 0 to disable the cache.

selector_result_cache_size: ReadableSize

Cache size for time series selector (e.g. last_value()). Setting it to 0 to disable the cache.

enable_write_cache: bool

Whether to enable the write cache.

write_cache_path: String

File system path for write cache dir’s root, defaults to {data_home}.

write_cache_size: ReadableSize

Capacity for write cache.

write_cache_ttl: Option<Duration>

TTL for write cache.

preload_index_cache: bool

Preload index (puffin) files into cache on region open (default: true).

index_cache_percent: u8

Percentage of write cache capacity allocated for index (puffin) files (default: 20). The remaining capacity is used for data (parquet) files. Must be between 0 and 100 (exclusive).

sst_write_buffer_size: ReadableSize

Buffer size for SST writing.

parallel_scan_channel_size: usize

Capacity of the channel to send data from parallel scan tasks to the main task (default 32).

max_concurrent_scan_files: usize

Maximum number of SST files to scan concurrently (default 384).

allow_stale_entries: bool

Whether to allow stale entries read during replay.

scan_memory_limit: MemoryLimit

Memory limit for table scans across all queries. Setting it to 0 disables the limit. Supports absolute size (e.g., “2GB”) or percentage (e.g., “50%”).

index: IndexConfig

Index configs.

inverted_index: InvertedIndexConfig

Inverted index configs.

fulltext_index: FulltextIndexConfig

Full-text index configs.

bloom_filter_index: BloomFilterConfig

Bloom filter index configs.

memtable: MemtableConfig

Memtable config

min_compaction_interval: Duration

Minimum time interval between two compactions. To align with the old behavior, the default value is 0 (no restrictions).

default_experimental_flat_format: bool

Whether to enable experimental flat format as the default format. When enabled, forces using BulkMemtable and BulkMemtableBuilder.

gc: GcConfig

Implementations

impl MitoConfig

pub fn sanitize(&mut self, data_home: &str) -> Result<()>

Sanitize incorrect configurations.

Returns an error if there is a configuration that unable to sanitize.

fn adjust_buffer_and_cache_size(&mut self, sys_memory: ReadableSize)

Trait Implementations

impl Clone for MitoConfig

fn clone(&self) -> MitoConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MitoConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for MitoConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for MitoConfig

where MitoConfig: Default,

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl From<MitoConfig> for ManifestOpenConfig

fn from(mito_config: MitoConfig) -> Self

Converts to this type from the input type.

impl PartialEq for MitoConfig

fn eq(&self, other: &MitoConfig) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for MitoConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for MitoConfig

impl StructuralPartialEq for MitoConfig