Struct MetadataRegion 

pub struct MetadataRegion {
    pub(crate) mito: MitoEngine,
    cache: Cache<RegionId, RegionMetadataCacheEntry>,
    logical_region_lock: RwLock<HashMap<RegionId, Arc<RwLock<()>>>>,
}

The other two fields key and value will be used as a k-v storage. It contains two group of key:

• __region_<LOGICAL_REGION_ID> is used for marking table existence. It doesn’t have value.
• __column_<LOGICAL_REGION_ID>_<COLUMN_NAME> is used for marking column existence, the value is column’s semantic type. To avoid the key conflict, this column key will be encoded by base64([STANDARD_NO_PAD]).

This is a generic handler like MetricEngine. It will handle all the metadata related operations across physical tables. Thus every operation should be associated to a [RegionId], which is the physical table id + region sequence. This handler will transform the region group by itself.

Fields

mito: MitoEngine

cache: Cache<RegionId, RegionMetadataCacheEntry>

The cache for contents(key-value pairs) of region metadata.

The cache should be invalidated when any new values are put into the metadata region or any values are deleted from the metadata region.

logical_region_lock: RwLock<HashMap<RegionId, Arc<RwLock<()>>>>

Logical lock for operations that need to be serialized. Like update & read region columns.

Region entry will be registered on creating and opening logical region, and deregistered on removing logical region.

Implementations

impl MetadataRegion

pub fn new(mito: MitoEngine) -> Self

pub async fn open_logical_region(&self, logical_region_id: RegionId) -> bool

Open a logical region.

Returns true if the logical region is opened for the first time.

pub async fn read_lock_logical_region( &self, logical_region_id: RegionId, ) -> Result<OwnedRwLockReadGuard<()>>

Retrieve a read lock guard of given logical region id.

pub async fn write_lock_logical_region( &self, logical_region_id: RegionId, ) -> Result<OwnedRwLockWriteGuard<()>>

Retrieve a write lock guard of given logical region id.

pub async fn remove_logical_region( &self, physical_region_id: RegionId, logical_region_id: RegionId, ) -> Result<()>

Remove a registered logical region from metadata.

This method doesn’t check if the previous key exists.

pub async fn logical_columns( &self, physical_region_id: RegionId, logical_region_id: RegionId, ) -> Result<Vec<(String, ColumnMetadata)>>

Get all the columns of a given logical region. Return a list of (column_name, column_metadata).

pub async fn logical_regions( &self, physical_region_id: RegionId, ) -> Result<Vec<RegionId>>

Return all logical regions associated with the physical region.

impl MetadataRegion

pub fn concat_region_key(region_id: RegionId) -> String

pub fn concat_column_key(region_id: RegionId, column_name: &str) -> String

Column name will be encoded by base64([STANDARD_NO_PAD])

pub fn concat_column_key_prefix(region_id: RegionId) -> String

Concat a column key prefix without column name

pub fn parse_region_key(key: &str) -> Option<&str>

pub fn parse_column_key(key: &str) -> Result<Option<(RegionId, String)>>

Parse column key to (logical_region_id, column_name)

pub fn serialize_column_metadata(column_metadata: &ColumnMetadata) -> String

pub fn deserialize_column_metadata( column_metadata: &str, ) -> Result<ColumnMetadata>

impl MetadataRegion

fn build_prefix_read_request(prefix: &str, key_only: bool) -> ScanRequest

fn build_read_request() -> ScanRequest

async fn load_all( &self, metadata_region_id: RegionId, ) -> Result<RegionMetadataCacheEntry>

async fn get_all_with_prefix( &self, metadata_region_id: RegionId, prefix: &str, ) -> Result<HashMap<String, String>>

pub async fn get_all_key_with_prefix( &self, region_id: RegionId, prefix: &str, ) -> Result<Vec<String>>

async fn delete( &self, metadata_region_id: RegionId, keys: &[String], ) -> Result<()>

Delete the given keys. For performance consideration, this method doesn’t check if those keys exist or not.

pub(crate) fn build_put_request_from_iter( kv: impl Iterator<Item = (String, String)>, ) -> RegionPutRequest

fn build_delete_request(keys: &[String]) -> RegionDeleteRequest

pub async fn add_logical_regions( &self, physical_region_id: RegionId, write_region_id: bool, logical_regions: impl Iterator<Item = (RegionId, HashMap<&str, &ColumnMetadata>)>, ) -> Result<()>

Add logical regions to the metadata region.

pub async fn transform_logical_region_metadata( &self, physical_region_id: RegionId, source_region_id: RegionId, ) -> Result<()>

Updates logical region metadata so that any entries previously referencing source_region_id are modified to reference the data region of physical_region_id.

This method should be called after copying files from source_region_id into the target region. It scans the metadata for the target physical region, finds logical regions with the same region number as the source, and reinserts region and column entries updated to use the target’s region number.