Crate flow

This crate manage dataflow in Greptime, including adapter, expr, plan, repr and utils. It can transform substrait plan into it’s own plan and execute it. It also contains definition of expression, adapter and plan, and internal state management.

Re-exports

• pub use error::Error;
• pub use error::Result;

Modules

• adapter 🔒
  for getting data from source and sending results to sink and communicating with other parts of the database
• compute 🔒
  Build and Compute the dataflow
• df_optimizer 🔒
  Datafusion optimizer for flow plan
• error
  Error definition for flow module
• expr 🔒
  for declare Expression in dataflow, including map, reduce, id and join(TODO!) etc.
• heartbeat
  Send heartbeat from flownode to metasrv
• metrics 🔒
  Some of the metrics used in the flow module.
• plan 🔒
  This module contain basic definition for dataflow’s plan that can be translate to hydro dataflow
• recording_rules 🔒
  Run flow as recording rule which is time-window-aware normal query triggered when new data arrives
• repr 🔒
  basically a wrapper around the datatype crate for basic Data Representation
• server 🔒
  Implementation of grpc service for flow node
• transform 🔒
  Transform Substrait into execution plan
• utils 🔒
  utilities for managing state of dataflow execution

Structs

• FlowConfig
  Flow config that exists both in standalone&distributed mode
• FlowWorkerManager
  FlowNodeManager manages the state of all tasks in the flow node, which should be run on the same thread
• FlownodeBuilder
  FlownodeInstance Builder
• FlownodeInstance
  The flownode server instance.
• FlownodeOptions
  Options for flow node
• FlownodeServer
• FrontendInvoker

Type Aliases

• FlowWorkerManagerRef
  Arc-ed FlowNodeManager, cheaper to clone