Crate pipeline

Modules§

setup_pipeline: This macro is test only, do not use it in production. The schema_info cannot be used in auto-transform ts-infer mode for lacking the ts schema.
unwrap_or_continue_if_err
unwrap_or_warn_continue
yaml_map_get_str: Macro to extract a string value from a YAML map

Array
ContextOpt: ContextOpt is a collection of options(including table options and pipeline options) that should be extracted during the pipeline execution.
ContextReq: ContextReq is a collection of row insert requests with different options. The default option is all empty. Because options are set in query context, we have to split them into sequential calls The key is a ContextOpt struct for strong type. e.g: { “skip_wal=true,ttl=1d”: [RowInsertRequest], “ttl=1d”: [RowInsertRequest], }
DispatchedTo: Where the pipeline executed is dispatched to, with context information
GreptimePipelineParams: Parameters that can be used to configure the greptime pipelines.
GreptimeTransformer: fields not in the columns will be discarded to prevent automatic column creation in GreptimeDB
Map
Pipeline
PipelineContext
SchemaInfo: This is used to record the current state schema information and a sequential cache of field names. As you traverse the user input JSON, this will change. It will record a superset of all user input schemas.
SelectInfo: SelectInfo is used to store the selected keys from OpenTelemetry record attrs The key is used to uplift value from the attributes and serve as column name in the table
TransformedOutput

Content
IdentityTimeIndex
PipelineDefinition: Enum for holding information of a pipeline, which is either pipeline itself, or information that be used to retrieve a pipeline from PipelineHandler
PipelineExecOutput: The result of pipeline execution
PipelineWay
TransformerMode
Value: Value can be used as type acts as value: the enclosed value is the actual value acts as type: the enclosed value is the default value

identity_pipeline: Identity pipeline for Greptime This pipeline will convert the input JSON array to Greptime Rows params table is used to set the semantic type of the row key column to Tag
json_array_to_map
json_to_map
parse
simd_json_array_to_map
simd_json_to_map

PipelineInfo: Pipeline info. A tuple of timestamp and pipeline reference.
PipelineRef
PipelineTableRef
PipelineVersion: Pipeline version. An optional timestamp with nanosecond precision.