Crate metric_engine

Metric Engine is a region engine to store timeseries data in metric monitoring scenario. It is something like a multiplexer over the Mito engine, which is for a more generic use case. By leveraging a synthetic wide physical table (region) that offers storage for multiple logical tables, Metric Engine is able to provide a more efficient storage solution that is able to handle a tremendous number of small tables in scenarios like Prometheus metrics.

For more details about implementation, please refer to MetricEngine.

This new engine doesn’t re-implement low level components like file R/W etc. It warps the existing mito engine, with extra storage and metadata multiplexing logic. I.e., it expose multiple logical regions based on two physical mito engine regions like this:

┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Metric Engine │ │ Metric Engine │ │ Metric Engine │
│   Region 1    │ │   Region 2    │ │   Region 3    │
└───────────────┘ └───────────────┘ └───────────────┘
        ▲               ▲                   ▲
        │               │                   │
        └───────────────┼───────────────────┘
                        │
              ┌─────────┴────────┐
              │ Metric Region    │
              │   Engine         │
              │    ┌─────────────┤
              │    │ Mito Region │
              │    │   Engine    │
              └────▲─────────────┘
                   │
                   │
             ┌─────┴───────────────┐
             │                     │
             │ Mito Engine Regions │
             │                     │
             └─────────────────────┘

Modules

config

data_region 🔒

engine

error

metadata_region 🔒

metrics 🔒

Internal metrics for observability.

repeated_task 🔒

row_modifier

utils 🔒