Struct MapFilterProject

pub struct MapFilterProject {
    pub expressions: Vec<ScalarExpr>,
    pub predicates: Vec<(usize, ScalarExpr)>,
    pub projection: Vec<usize>,
    pub input_arity: usize,
}

A compound operator that can be applied row-by-row.

In practice, this operator is a sequence of map, filter, and project in arbitrary order, which can and is stored by reordering the sequence’s apply order to a map first, filter second and project third order.

input is a row(a sequence of values), which is also being used for store intermediate results, like map operator can append new columns to the row according to it’s expressions, filter operator decide whether this entire row can even be output by decide whether the row satisfy the predicates, project operator decide which columns of the row should be output.

This operator integrates the map, filter, and project operators. It applies a sequences of map expressions, which are allowed to refer to previous expressions, interleaved with predicates which must be satisfied for an output to be produced. If all predicates evaluate to Value::Boolean(True) the data at the identified columns are collected and produced as output in a packed Row.

This operator is a “builder” and its contents may contain expressions that are not yet executable. For example, it may contain temporal expressions in self.expressions, even though this is not something we can directly evaluate. The plan creation methods will defensively ensure that the right thing happens.

Fields

expressions: Vec<ScalarExpr>

A sequence of expressions that should be appended to the row.

Many of these expressions may not be produced in the output, and may only be present as common subexpressions.

predicates: Vec<(usize, ScalarExpr)>

Expressions that must evaluate to Datum::True for the output row to be produced.

Each entry is prepended with a column identifier indicating the column before which the predicate should first be applied. Most commonly this would be one plus the largest column identifier in the predicate’s referred columns, but it could be larger to implement guarded evaluation of predicates. Put it in another word, the first element of the tuple means the predicates can’t be evaluated until that number of columns is formed.

This list should be sorted by the first field.

projection: Vec<usize>

A sequence of column identifiers whose data form the output row.

input_arity: usize

The expected number of input columns.

This is needed to ensure correct identification of newly formed columns in the output.

Implementations

impl MapFilterProject

pub fn new(input_arity: usize) -> Self

Create a no-op operator for an input of a supplied arity.

pub fn get_nth_expr(&self, n: usize) -> Option<ScalarExpr>

pub fn output_arity(&self) -> usize

The number of columns expected in the output row.

pub fn compose(before: Self, after: Self) -> Result<Self, Error>

Given two mfps, return an mfp that applies one followed by the other. Note that the arguments are in the opposite order from how function composition is usually written in mathematics.

pub fn is_identity(&self) -> bool

True if the operator describes the identity transformation.

pub fn project<I>(self, columns: I) -> Result<Self, Error>

where I: IntoIterator<Item = usize> + Debug,

Retain only the indicated columns in the presented order.

i.e. before: self.projection = [1, 2, 0], columns = [1, 0]

flowchart TD
col-0
col-1
col-2
projection --> |0|col-1
projection --> |1|col-2
projection --> |2|col-0

after: self.projection = [2, 1]

flowchart TD
col-0
col-1
col-2
project("project:[1,2,0]")
project
project -->|0| col-1
project -->|1| col-2
project -->|2| col-0
new_project("apply new project:[1,0]")
new_project -->|0| col-2
new_project -->|1| col-1

pub fn filter<I>(self, predicates: I) -> Result<Self, Error>

where I: IntoIterator<Item = ScalarExpr>,

Retain only rows satisfying these predicates.

This method introduces predicates as eagerly as they can be evaluated, which may not be desired for predicates that may cause exceptions. If fine manipulation is required, the predicates can be added manually.

simply added to the end of the predicates list

while paying attention to column references maintained by self.projection

so self.projection = [1, 2, 0], filter = [0]+[1]>0: becomes:

flowchart TD
col-0
col-1
col-2
project("first project:[1,2,0]")
project
project -->|0| col-1
project -->|1| col-2
project -->|2| col-0
filter("then filter:[0]+[1]>0")
filter -->|0| col-1
filter --> |1| col-2

pub fn map<I>(self, expressions: I) -> Result<Self, Error>

where I: IntoIterator<Item = ScalarExpr>,

Append the result of evaluating expressions to each row.

simply append expressions to self.expressions

while paying attention to column references maintained by self.projection

hence, before apply map with a previously non-trivial projection would be like: before:

flowchart TD
col-0
col-1
col-2
projection --> |0|col-1
projection --> |1|col-2
projection --> |2|col-0

after apply map:

flowchart TD
col-0
col-1
col-2
project("project:[1,2,0]")
project
project -->|0| col-1
project -->|1| col-2
project -->|2| col-0
map("map:[0]/[1]/[2]")
map -->|0|col-1
map -->|1|col-2
map -->|2|col-0

pub fn into_map_filter_project( self, ) -> (Vec<ScalarExpr>, Vec<ScalarExpr>, Vec<usize>)

Like MapFilterProject::as_map_filter_project, but consumes self rather than cloning.

pub fn as_map_filter_project( &self, ) -> (Vec<ScalarExpr>, Vec<ScalarExpr>, Vec<usize>)

As the arguments to Map, Filter, and Project operators.

In principle, this operator can be implemented as a sequence of more elemental operators, likely less efficiently.

impl MapFilterProject

pub fn into_safe(self) -> SafeMfpPlan

Convert the MapFilterProject into a safe evaluation plan. Marking it safe to evaluate.

pub fn optimize(&mut self)

Optimize the MapFilterProject in place.

pub fn get_old_to_new_mapping(&self) -> BTreeMap<usize, usize>

get the mapping of old columns to new columns after the mfp

pub fn demand(&self) -> BTreeSet<usize>

Lists input columns whose values are used in outputs.

It is entirely appropriate to determine the demand of an instance and then both apply a projection to the subject of the instance and self.permute this instance.

pub fn permute( &mut self, shuffle: BTreeMap<usize, usize>, new_input_arity: usize, ) -> Result<(), Error>

Update input column references, due to an input projection or permutation.

The shuffle argument remaps expected column identifiers to new locations, with the expectation that shuffle describes all input columns, and so the intermediate results will be able to start at position shuffle.len().

The supplied shuffle may not list columns that are not “demanded” by the instance, and so we should ensure that self is optimized to not reference columns that are not demanded.

Trait Implementations

impl Clone for MapFilterProject

fn clone(&self) -> MapFilterProject

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MapFilterProject

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Ord for MapFilterProject

fn cmp(&self, other: &MapFilterProject) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

where Self: Sized,

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

where Self: Sized,

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

where Self: Sized,

Restrict a value to a certain interval.

impl PartialEq for MapFilterProject

fn eq(&self, other: &MapFilterProject) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd for MapFilterProject

fn partial_cmp(&self, other: &MapFilterProject) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator.

impl Eq for MapFilterProject

impl StructuralPartialEq for MapFilterProject