Struct UnionDistinctOn

pub struct UnionDistinctOn {
    left: LogicalPlan,
    right: LogicalPlan,
    compare_keys: Vec<String>,
    ts_col: String,
    output_schema: DFSchemaRef,
}

A special kind of UNION(OR in PromQL) operator, for PromQL specific use case.

This operator is similar to UNION from SQL, but it only accepts two inputs. The most different part is that it treat left child and right child differently:

• All columns from left child will be outputted.
• Only check collisions (when not distinct) on the columns specified by compare_keys.
• When there is a collision:
  • If the collision is from right child itself, only the first observed row will be preserved. All others are discarded.
  • If the collision is from left child, the row in right child will be discarded.
• The output order is not maintained. This plan will output left child first, then right child.
• The output schema contains all columns from left or right child plans.

From the implementation perspective, this operator is similar to HashJoin, but the probe side is the right child, and the build side is the left child. Another difference is that the probe is opting-out.

This plan will exhaust the right child first to build probe hash table, then streaming on left side, and use the left side to “mask” the hash table.

Fields

left: LogicalPlan

right: LogicalPlan

compare_keys: Vec<String>

The columns to compare for equality. TIME INDEX is included.

ts_col: String

output_schema: DFSchemaRef

Implementations

impl UnionDistinctOn

pub fn name() -> &'static str

pub fn new( left: LogicalPlan, right: LogicalPlan, compare_keys: Vec<String>, ts_col: String, output_schema: DFSchemaRef, ) -> Self

pub fn to_execution_plan( &self, left_exec: Arc<dyn ExecutionPlan>, right_exec: Arc<dyn ExecutionPlan>, ) -> Arc<dyn ExecutionPlan>

Trait Implementations

impl Debug for UnionDistinctOn

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for UnionDistinctOn

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for UnionDistinctOn

fn eq(&self, other: &UnionDistinctOn) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd for UnionDistinctOn

fn partial_cmp(&self, other: &Self) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator.

impl UserDefinedLogicalNodeCore for UnionDistinctOn

fn name(&self) -> &str

Return the plan’s name.

fn inputs(&self) -> Vec<&LogicalPlan>

Return the logical plan’s inputs.

fn schema(&self) -> &DFSchemaRef

Return the output schema of this logical plan node.

fn expressions(&self) -> Vec<Expr>

Returns all expressions in the current logical plan node. This should not include expressions of any inputs (aka non-recursively). These expressions are used for optimizer passes and rewrites.

fn fmt_for_explain(&self, f: &mut Formatter<'_>) -> Result

Write a single line, human readable string to f for use in explain plan.

fn with_exprs_and_inputs( &self, _exprs: Vec<Expr>, inputs: Vec<LogicalPlan>, ) -> DataFusionResult<Self>

Create a new UserDefinedLogicalNode with the specified children and expressions. This function is used during optimization when the plan is being rewritten and a new instance of the UserDefinedLogicalNode must be created.

fn prevent_predicate_push_down_columns(&self) -> HashSet<String>

A list of output columns (e.g. the names of columns in self.schema()) for which predicates can not be pushed below this node without changing the output.

fn from_template(&self, exprs: &[Expr], inputs: &[LogicalPlan]) -> Self

👎Deprecated since 39.0.0: use with_exprs_and_inputs instead

fn necessary_children_exprs( &self, _output_columns: &[usize], ) -> Option<Vec<Vec<usize>>>

Returns the necessary input columns for this node required to compute the columns in the output schema

fn supports_limit_pushdown(&self) -> bool

Returns true if a limit can be safely pushed down through this UserDefinedLogicalNode node.

impl Eq for UnionDistinctOn

impl StructuralPartialEq for UnionDistinctOn