Module loki

Structs

JsonStreamItem 🔒

LOKI_INIT_SCHEMAS 🔒

LokiJsonParser 🔒

LokiMiddleItem

This is the holder of the loki lines parsed from json or protobuf. The generic here is either serde_json::Value or Vec. Depending on the target destination, this can be converted to LokiRawItem or LokiPipeline.

LokiPbParser

LokiPipeline

This is the line item prepared for the pipeline engine.

LokiRawItem

This is the line item for the Loki raw ingestion. We’ll persist the line in its whole, set labels into tags, and structured metadata into a big JSON.

PbStreamItem

Constants

LABEL_KEY 🔒

LINES_KEY 🔒

LOKI_LINE_COLUMN 🔒

LOKI_LINE_COLUMN_NAME 🔒

LOKI_PIPELINE_LABEL_PREFIX 🔒

LOKI_PIPELINE_METADATA_PREFIX 🔒

LOKI_STRUCTURED_METADATA_COLUMN 🔒

LOKI_TABLE_NAME 🔒

STREAMS_KEY 🔒

Functions

extract_item 🔒

This is the flow of the Loki ingestion. +––––+ | bytes | +––––+ | +–––––––––––+–––––––––––+ | | | | JSON content type | PB content type | +–––––––––––+–––––––––––+ | | | | JsonStreamItem | PbStreamItem | | stream: serde_json | stream: adapter | +–––––––––––+–––––––––––+ | | | | MiddleItem<serde_json> | MiddleItem | +–––––––––––+–––––––––––+ \ / \ / \ / +–––––––––––+ | MiddleItem | +–––––––––––+ | +––––––––+––––––––+ | | +——————+ +———————+ | LokiRawItem | | LokiPipelineItem | +——————+ +———————+ | | +——————+ +———————+ | Loki ingest | | run_pipeline | +——————+ +———————+

init_row 🔒

loki_ingest

parse_loki_labels

since we’re hand-parsing the labels, if any error is encountered, we’ll just skip the label note: pub here for bench usage ref:

process_labels 🔒

prost_ts_to_nano 🔒