Struct CompactionScheduler 

pub(crate) struct CompactionScheduler {
    scheduler: SchedulerRef,
    region_status: HashMap<RegionId, CompactionStatus>,
    request_sender: Sender<WorkerRequestWithTime>,
    cache_manager: CacheManagerRef,
    engine_config: Arc<MitoConfig>,
    memory_manager: Arc<CompactionMemoryManager>,
    memory_policy: OnExhaustedPolicy,
    listener: WorkerListener,
    plugins: Plugins,
    next_plan_id: u64,
}

Compaction scheduler tracks and manages compaction tasks.

Fields

scheduler: SchedulerRef

region_status: HashMap<RegionId, CompactionStatus>

Compacting regions.

request_sender: Sender<WorkerRequestWithTime>

Request sender of the worker that this scheduler belongs to.

cache_manager: CacheManagerRef

engine_config: Arc<MitoConfig>

memory_manager: Arc<CompactionMemoryManager>

memory_policy: OnExhaustedPolicy

listener: WorkerListener

plugins: Plugins

Plugins for the compaction scheduler.

next_plan_id: u64

Scheduler-wide generation counter for compaction plans and executions. It outlives region statuses so close/reopen cannot reuse an old identity.

Implementations

impl CompactionScheduler

pub(super) fn dispatch_compaction_planning( &self, plan_id: u64, request: CompactionRequest, options: Options, time_range: Option<TimestampRange>, )

pub(super) async fn notify_planning_result( region_id: RegionId, plan_id: u64, request_sender: Sender<WorkerRequestWithTime>, planning: impl Future<Output = CompactionPlanningResult> + Send, )

Runs the planning future and always sends the planning result back to the worker, even if the planning panics.

The worker only leaves the picking phase after it receives the CompactionPickFinished notification. If a panicked planning task swallowed the notification, the region would be stuck in the picking phase forever, blocking all future compactions and pending DDLs (e.g. entering staging) of the region.

async fn prepare_compaction( request: CompactionRequest, options: Options, plugins: Plugins, max_background_compactions: usize, time_range: Option<TimestampRange>, ) -> CompactionPlanningResult

pub(super) async fn handle_compaction_pick_finished_inner( &mut self, finished: CompactionPickFinished, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, ) -> CompactionTransition

Applies a background planning result to the current compaction lifecycle.

Returns

Reports an automatic follow-up dispatched after Picking, or DDL requests released when Picking terminates without creating an execution. The owning worker must dispatch returned DDLs immediately because no execution callback will follow.

async fn finish_compaction_planning( &mut self, region_id: RegionId, err: Option<Arc<Error>>, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, ) -> CompactionTransition

async fn submit_prepared_compaction( &mut self, prepared: PreparedCompaction, files: CompactingFiles, waiters: Vec<OutputTx>, plan_id: u64, ) -> Result<Option<CompactionPhase>>

fn submit_compaction_task( &mut self, task: Box<CompactionTaskImpl>, region_id: RegionId, ) -> Result<(), (Error, Option<Box<CompactionTaskImpl>>)>

fn exceeds_compaction_memory_limit(&self, estimated_bytes: u64) -> Option<u64>

impl CompactionScheduler

pub(crate) fn new( scheduler: SchedulerRef, request_sender: Sender<WorkerRequestWithTime>, cache_manager: CacheManagerRef, engine_config: Arc<MitoConfig>, listener: WorkerListener, plugins: Plugins, memory_manager: Arc<CompactionMemoryManager>, memory_policy: OnExhaustedPolicy, ) -> Self

Creates an empty scheduler bound to one region worker.

Effects

Stores the worker request sender and shared scheduling resources. No compaction is dispatched until a scheduling method is called.

Constraints

All lifecycle methods on the returned scheduler must be driven by the same worker so region state transitions remain serialized.

pub(crate) fn schedule_automatic_compaction( &mut self, compact_options: Options, version_control: &Arc<VersionControl>, access_layer: &AccessLayerRef, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, ) -> Result<bool>

Accepts an automatic compaction trigger for a region.

Effects

Starts planning when no scheduler status exists for the region. Otherwise, an active status coalesces the trigger into one unrestricted follow-up cycle. Returns true only when this call dispatches planning.

Constraints

The owning region worker must call this method serially. Automatic triggers have no waiter and may be ignored while a DDL fence is active. The region identity comes from version_control; access_layer and manifest_ctx must belong to the same region.

pub(crate) fn schedule_manual_compaction( &mut self, compact_options: Options, version_control: &Arc<VersionControl>, access_layer: &AccessLayerRef, waiter: OptionOutputTx, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, max_parallelism: usize, time_range: Option<TimestampRange>, ) -> Result<bool>

Accepts a manual compaction request for a region.

Effects

Starts planning when no scheduler status exists for the region. During an automatic compaction, queues the request and replaces any older pending manual request; during a manual compaction, rejects the new request through waiter. Returns true only when this call dispatches planning.

Constraints

The owning region worker must call this method serially. The supplied waiter is completed exactly once by cycle completion, replacement, cancellation, failure, or region teardown. The region identity comes from version_control; access_layer and manifest_ctx must belong to the same region.

pub(crate) async fn handle_compaction_pick_finished( &mut self, finished: CompactionPickFinished, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, ) -> CompactionTransition

Applies a background planning result to the current region state.

Effects

Rejects stale plans, submits a prepared local or remote execution, or completes the cycle when planning produced no executable plan. The returned transition reports a dispatched automatic follow-up or DDL requests released by the resulting terminal transition.

Constraints

The owning worker must call this method for planning notifications and must execute every returned DDL request. The notification may be stale; stale notifications intentionally have no effect.

pub(crate) fn is_current_execution( &self, region_id: RegionId, execution: &CompactionExecution, ) -> bool

Returns whether a terminal notification belongs to the installed execution.

Effects

Performs a read-only identity check against the region’s current plan.

Constraints

Callers must check this before applying a compaction edit. Matching only the region id is insufficient because close/reopen can replace a status.

pub(crate) async fn on_execution_finished( &mut self, region_id: RegionId, execution: &CompactionExecution, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, ) -> CompactionTransition

Completes the installed execution after its edit has been applied.

Effects

Notifies waiters, schedules a pending manual or explicit automatic follow-up, or removes the region status. The returned transition reports a dispatched automatic follow-up or DDLs that are now safe to execute.

Constraints

The owning worker must call this only after accepting and applying the execution’s result, and must execute every returned DDL request. Stale executions are ignored.

pub(crate) async fn on_execution_cancelled( &mut self, region_id: RegionId, execution: &CompactionExecution, ) -> Vec<SenderDdlRequest>

Completes a cooperatively canceled execution.

Effects

Removes the matching region status, notifies compaction waiters, and returns DDLs that were waiting for cancellation.

Constraints

The owning worker must execute every returned DDL request. A stale cancellation is ignored.

pub(crate) fn on_execution_failed( &mut self, region_id: RegionId, execution: &CompactionExecution, err: Arc<Error>, )

Records failure of the installed execution.

Effects

Removes the matching region status and fails all compaction waiters and dependent DDL requests with the supplied error.

Constraints

A stale execution failure is ignored so it cannot tear down replacement state.

pub(crate) fn on_region_dropped(&mut self, region_id: RegionId)

Removes compaction state because the region was dropped.

Effects

Fails all compaction and DDL waiters owned by the status.

Constraints

The owning worker must invoke this as part of serialized region teardown.

pub(crate) fn on_region_closed(&mut self, region_id: RegionId)

Removes compaction state because the region was closed.

Effects

Fails all compaction and DDL waiters owned by the status.

Constraints

The owning worker must invoke this as part of serialized region teardown.

pub(crate) fn on_region_truncated(&mut self, region_id: RegionId)

Removes compaction state because the region was truncated.

Effects

Fails all compaction and DDL waiters owned by the status.

Constraints

The owning worker must invoke this after truncate completes.

pub(crate) fn try_cancel_and_add_ddl<T>( &mut self, region_id: RegionId, sender: OptionOutputTx, request: T, into_ddl_request: impl FnOnce(T) -> DdlRequest, ) -> Result<(), (OptionOutputTx, T)>

Cancels a running compaction and queues its dependent DDL atomically.

Effects

Requests cancellation when the active phase can still stop, then queues the DDL. The queued DDL forms a scheduling fence for subsequent compaction triggers. If no compaction is running, returns the sender and typed request unchanged.

Constraints

Production callers use this only for DdlRequest::Truncate and DdlRequest::EnterStaging. Ok(()) means the caller must wait for a terminal callback to receive and execute the queued DDL; Err means the caller must execute the returned request directly.

impl CompactionScheduler

fn next_plan_id(counter: &mut u64) -> u64

Returns the current plan id and advances the counter.

Takes the counter instead of &mut self so callers can bump it while holding a mutable borrow of a region status.

fn schedule_compaction( &mut self, trigger: CompactionTrigger, compact_options: Options, version_control: &Arc<VersionControl>, access_layer: &AccessLayerRef, waiter: OptionOutputTx, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, max_parallelism: usize, time_range: Option<TimestampRange>, ) -> Result<bool>

fn handle_pending_compaction_request( &mut self, region_id: RegionId, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, ) -> bool

async fn on_compaction_finished( &mut self, region_id: RegionId, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, ) -> CompactionTransition

Notifies the scheduler that the compaction job is finished successfully.

fn schedule_automatic_followup( &mut self, region_id: RegionId, manifest_ctx: &Arc<ManifestContext>, schema_metadata_manager: SchemaMetadataManagerRef, ) -> bool

async fn on_compaction_cancelled( &mut self, region_id: RegionId, ) -> Vec<SenderDdlRequest>

Notifies the scheduler that the compaction job is cancelled cooperatively.

fn on_compaction_failed(&mut self, region_id: RegionId, err: Arc<Error>)

Notifies the scheduler that the compaction job is failed.

fn remove_region_on_failure(&mut self, region_id: RegionId, err: Arc<Error>)

fn remove_region_on_cancel( &mut self, region_id: RegionId, ) -> Vec<SenderDdlRequest>

Trait Implementations

impl Drop for CompactionScheduler

fn drop(&mut self)

Executes the destructor for this type.