Struct WalPruneManager

pub(crate) struct WalPruneManager {
    table_metadata_manager: TableMetadataManagerRef,
    receiver: Receiver<Event>,
    procedure_manager: ProcedureManagerRef,
    tracker: WalPruneProcedureTracker,
    semaphore: Arc<Semaphore>,
    wal_prune_context: Context,
    trigger_flush_threshold: u64,
}

WalPruneManager manages all remote WAL related tasks in metasrv.

WalPruneManager is responsible for:

1. Registering WalPruneProcedure loader in the procedure manager.
2. Periodically receive Event::Tick to submit WalPruneProcedure to prune remote WAL.
3. Use a semaphore to limit the number of concurrent WalPruneProcedures.

Fields

table_metadata_manager: TableMetadataManagerRef

Table metadata manager to restore topics from kvbackend.

receiver: Receiver<Event>

Receives Events.

procedure_manager: ProcedureManagerRef

Procedure manager.

tracker: WalPruneProcedureTracker

Tracker for running WalPruneProcedures.

semaphore: Arc<Semaphore>

Semaphore to limit the number of concurrent WalPruneProcedures.

wal_prune_context: Context

Context for WalPruneProcedure.

trigger_flush_threshold: u64

Trigger flush threshold for WalPruneProcedure. If None, never send flush requests.

Implementations

impl WalPruneManager

pub fn new( table_metadata_manager: TableMetadataManagerRef, limit: usize, receiver: Receiver<Event>, procedure_manager: ProcedureManagerRef, wal_prune_context: WalPruneContext, trigger_flush_threshold: u64, ) -> Self

Returns a new empty WalPruneManager.

pub async fn try_start(self) -> Result<()>

Start the WalPruneManager. It will register WalPruneProcedure loader in the procedure manager.

pub(crate) fn channel() -> (Sender<Event>, Receiver<Event>)

Returns a mpsc channel with a buffer capacity of 1024 for sending and receiving Event messages.

pub(crate) async fn run(&mut self)

Runs the main loop. Performs actions on received events.

• Tick: Submit limit WalPruneProcedures to prune remote WAL.

pub async fn submit_procedure(&self, topic_name: &str) -> Result<ProcedureId>

Submits a WalPruneProcedure for the given topic name.

async fn handle_tick_request(&self) -> Result<()>

async fn retrieve_sorted_topics(&self) -> Result<Vec<String>>

Retrieve topics from the table metadata manager. Since WalPruneManager submits procedures depending on the order of the topics, we should sort the topics. TODO(CookiePie): Can register topics in memory instead of retrieving from the table metadata manager every time.