Module region_hook 

Region hook extension point for observing SST writes and manifest mutations.

Design

The RegionHook trait observes region activity through two categories of callbacks — manifest/file observation and region lifecycle:

• on_sst_files_written: Fires when mito2 physically writes SST data files. Provides per-file SstInfo + FileMeta; metadata richness varies by path (see SstFileInfo and the coverage footnote).

• on_manifest_updated: Fires after a manifest write is committed to the live (normal) manifest directory. Writes to the staging directory (enter staging, operations during staging, the intermediate apply-staging edit) are suppressed — their effects are accumulated and delivered in a single notification when the staged actions are promoted to the live manifest. Receives the full RegionMetaActionList so consumers can inspect what changed (file additions/ removals, schema changes, truncation, partition expression changes, etc.).

• on_region_opened / on_region_closed / on_region_dropped / on_region_files_removed: Region lifecycle callbacks for open, close, logical drop, and physical file removal. See Region lifecycle below.

Hook implementations are registered via the Plugins system:

plugins.insert(Arc::new(MyHook) as RegionHookRef);

Coverage

Only manifest writes to the normal (live) manifest directory trigger on_manifest_updated. Writes to the staging manifest directory (operations that happen while the region is in staging mode) are intentionally suppressed: their effects are accumulated and delivered in a single notification when the staged actions are promoted to the live manifest via exit_staging_on_success.

Scenario	on_sst_files_written	on_manifest_updated
Flush (memtable → SST)	✅ Yes	✅ Yes
Local compaction	✅ Yes	✅ Yes
Remote compaction	✅ (compactor node) ¹	✅ (compactor node) ¹
RegionEdit / bulk ingestion	❌ (files pre-written)	✅ Yes
Copy region	❌ (object-store copy)	✅ Yes
Apply staging (promote)	❌ (delegates to edit)	✅ Yes ²
Alter (schema change)	❌ (no SST files)	✅ Yes
Truncate	❌ (removes files)	✅ Yes
Enter staging	❌ (no SST files)	❌ (staging dir)
Operations during staging	N/A	❌ (staging dir)
Async index build	❌ (index files only)	✅ Yes

¹ Remote compaction runs on a dedicated compactor node via open_compaction_region(); pass plugins via OpenCompactionRegionRequest to enable hooks there. sst_infos is #[serde(skip)] over the wire, so each SstInfo is rebuilt from FileMeta with empty footer/index — see SstFileInfo for field-level detail. ² Apply staging fires on_manifest_updated once when exit_staging_on_success promotes all staged manifest actions (including the SST file additions) into the live manifest. The intermediate staging RegionEdit is written to the staging directory and does not fire the hook — its file list is included in the promote notification.

The following paths do not trigger any hook:

• Follower region sync / catchup (manifest read-only; followers don’t author changes)
• GC / checkpoint / remap (internal bookkeeping, not logical state changes)

An explicit region drop does fire lifecycle hooks — see Region lifecycle.

Region lifecycle

Beyond manifest/SST observation, the hook observes the high-level lifecycle of an active region:

Event	Method	When
Open	on_region_opened	A create or open request registers the region as active (the counterpart to close/drop). Does not fire for the compactor’s transient regions or the catch-up reopen.
Close	on_region_closed	A close request (or a close-after-flush) removes the region from the active set. Data files, manifest and WAL state are preserved; the region may be reopened.
Logical drop	on_region_dropped	A drop request has been handled: the region leaves the active set and its WAL entries are marked obsolete. Data files are not yet deleted.
Physical file removal	on_region_files_removed	The drop GC worker has deleted the region directory. Terminal file-lifecycle event.
Global GC pass	on_region_gc	The datanode’s global GC worker finished a GC pass for a region — both periodic GC for live regions and the global reclamation of dropped/repartitioned regions (is_region_dropped). Also fired by the offline cleanup path (handle_offline_cleanup_request, i.e. soft-drop PURGE) once the region directory is removed, with is_region_dropped = true and full_file_listing = true.

Notes:

• on_region_closed / on_region_dropped run inline in the region worker loop, so implementations must be fast (same contract as on_manifest_updated).
• on_region_opened runs inline in the worker loop on the create path, but on the open path it fires inside the spawned open task (common_runtime::spawn_global), i.e. concurrently with the worker loop — after WAL replay, before the region is registered and its open request is acknowledged. Implementations must still be fast and must not assume worker-loop-thread affinity or strict ordering against concurrent requests to other regions.
• on_region_files_removed runs on the background drop GC task, outside the worker loop.
• When global GC is enabled and a normal table region is dropped with partial_drop, its directory is left for global reclamation and on_region_files_removed is not fired by the drop worker (observe it via the global GC path instead).
• The offline cleanup path (handle_offline_cleanup_request, reached by a soft-drop PURGE) force-removes the region directory but has no RegionMetadataRef for the offline region, so it cannot fire on_region_files_removed. It fires on_region_gc instead — with is_region_dropped = true and full_file_listing = true — so extensions can reclaim sidecar state for the now-removed region. A hook Err is propagated so the caller retries the (idempotent) cleanup.
• Logical file removal (compaction, region edit, truncate) is already observable via on_manifest_updated (Edit.files_to_remove / Truncate action); only the drop worker’s physical directory deletion needs a dedicated file hook.

Invocation points

on_sst_files_written is invoked at the SST write site (flush task or compaction task), immediately after SST files are written but before the manifest is committed.

on_manifest_updated is funneled through ManifestContext::update_locked, the sole caller of the low-level RegionManifestManager::update, which packages each successful write into a PendingManifestHook. The caller owns the write lock, drops it, and then fires the receipt — the hook must never run under the lock. ManifestContext::update_manifest is the common case: it acquires the lock, delegates to update_locked, and fires the receipt in one go. Multi-step sequences (staging-exit, role-state backfill) call update_locked directly under their own held guard.

Non-logical writes (GC, staging bookkeeping) call the manager’s own methods directly and intentionally do not fire the hook.

Future work

on_region_files_removed currently covers only the drop GC worker’s physical directory removal. The global-GC reclamation path and the offline-cleanup path (soft-drop PURGE) are covered by on_region_gc instead. A broader per-file on_files_removed hook covering compaction removal and truncate is not yet implemented (though logical file removal is already observable via on_manifest_updated, and the global GC reclamation path is covered by on_region_gc). Role/leadership transitions (on_region_role_changed) are also not hooked.

Structs

PendingManifestHook 🔒

A deferred RegionHook::on_manifest_updated notification produced by a logical manifest write via ManifestContext::update_locked.

RegionGcInfo

What mito2’s GC pass deleted for a region, handed to RegionHook::on_region_gc.

SstFileInfo

Information about a single SST data file written during flush or compaction.

Traits

RegionHook

Hook for observing region mutations in mito2.

Type Aliases

RegionHookRef