Struct PlanRewriter

struct PlanRewriter {
    level: usize,
    stack: Vec<(LogicalPlan, usize)>,
    stage: Vec<LogicalPlan>,
    status: RewriterStatus,
    partition_cols: Option<Vec<String>>,
    alias_tracker: Option<AliasTracker>,
    column_requirements: Vec<(HashSet<Column>, usize)>,
    expand_on_next_call: bool,
    expand_on_next_part_cond_trans_commutative: bool,
    new_child_plan: Option<LogicalPlan>,
}

Fields

level: usize

Current level in the tree

stack: Vec<(LogicalPlan, usize)>

Simulated stack for the rewrite recursion

stage: Vec<LogicalPlan>

Stages to be expanded, will be added as parent node of merge scan one by one

status: RewriterStatus

partition_cols: Option<Vec<String>>

Partition columns of the table in current pass

alias_tracker: Option<AliasTracker>

column_requirements: Vec<(HashSet<Column>, usize)>

use stack count as scope to determine column requirements is needed or not i.e for a logical plan like:

1: Projection: t.number
2: Sort: t.pk1+t.pk2
3. Projection: t.number, t.pk1, t.pk2

Sort will make a column requirement for t.pk1 at level 2. Which making Projection at level 1 need to add a ref to t.pk1 as well. So that the expanded plan will be

Projection: t.number
  MergeSort: t.pk1
    MergeScan: remote_input=
Projection: t.number, "t.pk1+t.pk2" <--- the original `Projection` at level 1 get added with `t.pk1+t.pk2`
 Sort: t.pk1+t.pk2
   Projection: t.number, t.pk1, t.pk2

Making MergeSort can have t.pk1 as input. Meanwhile Projection at level 3 doesn’t need to add any new column because 3 > 2 and col requirements at level 2 is not applicable for level 3.

see more details in test expand_proj_step_aggr and expand_proj_sort_proj

TODO(discord9): a simpler solution to track column requirements for merge scan

expand_on_next_call: bool

Whether to expand on next call This is used to handle the case where a plan is transformed, but need to be expanded from it’s parent node. For example a Aggregate plan is split into two parts in frontend and datanode, and need to be expanded from the parent node of the Aggregate plan.

expand_on_next_part_cond_trans_commutative: bool

Expanding on next partial/conditional/transformed commutative plan This is used to handle the case where a plan is transformed, but still need to push down as many node as possible before next partial/conditional/transformed commutative plan. I.e.

Limit:
    Sort:

where Limit is partial commutative, and Sort is conditional commutative. In this case, we need to expand the Limit plan, so that we can push down the Sort plan as much as possible.

new_child_plan: Option<LogicalPlan>

Implementations

impl PlanRewriter

fn get_parent(&self) -> Option<&LogicalPlan>

fn should_expand(&mut self, plan: &LogicalPlan) -> bool

Return true if should stop and expand. The input plan is the parent node of current node

fn update_column_requirements(&mut self, plan: &LogicalPlan, plan_level: usize)

Update the column requirements for the current plan, plan_level is the level of the plan in the stack, which is used to determine if the column requirements are applicable for other plans in the stack.

fn is_expanded(&self) -> bool

fn set_expanded(&mut self)

fn set_unexpanded(&mut self)

fn maybe_update_alias(&mut self, node: &LogicalPlan)

Maybe update alias for original table columns in the plan

fn get_aliased_partition_columns( &self, ) -> Option<HashMap<String, HashSet<Column>>>

fn maybe_set_partitions(&mut self, plan: &LogicalPlan)

fn pop_stack(&mut self)

pop one stack item and reduce the level by 1

fn expand(&mut self, on_node: LogicalPlan) -> DfResult<LogicalPlan>

Trait Implementations

impl Debug for PlanRewriter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for PlanRewriter

fn default() -> PlanRewriter

Returns the “default value” for a type.

impl TreeNodeRewriter for PlanRewriter

fn f_down<'a>(&mut self, node: Self::Node) -> DfResult<Transformed<Self::Node>>

descend

fn f_up(&mut self, node: Self::Node) -> DfResult<Transformed<Self::Node>>

ascend

Besure to call pop_stack before returning

type Node = LogicalPlan

The node type which is rewritable.