Skip to main content

RECOVERY_CONCURRENCY

Constant RECOVERY_CONCURRENCY 

Source
const RECOVERY_CONCURRENCY: usize = 64;
Expand description

Number of objects whose footers recovery fetches at a time. On S3 an object costs two reads, its header and its tail, tens of milliseconds per lane however many vCPUs the node has, and recovery time scales with objects divided by concurrency up to 128 lanes without throttling: at 64 about 10,000 objects recover in about 10 s instead of about 100 s at 8, while the reads use about a third of S3’s request rate per prefix, leaving room for the data file reads of start-up. Each lane holds one RECOVERY_TAIL_WINDOW, 4 MiB in all.