Struct JsonSchemaConcretizeRule 

pub(crate) struct JsonSchemaConcretizeRule;

Keeps JSON2 schemas consistent across distributed query boundaries.

An unresolved JSON2 column is represented as an empty Struct, while a remote stage always emits a concrete Arrow array. DataFusion expects the schema declared by a logical node, the schema used to build its physical plan, and the schema of its record batches to agree. This rule gives each MergeScanLogicalPlan the concrete schema emitted by its remote stage and propagates that schema through the local plan.

For example:

• SELECT j FROM t transfers the complete JSON2 value as Binary (Variant). A projection or window above the MergeScan must therefore also describe j as Binary, not an empty Struct.
• SELECT j.a FROM t may transfer the complete j as Binary and extract a locally, or transfer only the extracted scalar when the expression runs remotely. The boundary schema must describe the remote output rather than the local expression that consumes it.
• SELECT l.j, r.j FROM l JOIN r ON l.k = r.k has two independent boundaries. Each input and the join schema must agree on the concrete type of its JSON2 column.

Correcting only the physical MergeScan schema can make simple queries work because many operators access columns by position, but it leaves the logical plan describing a different type. Keeping the schemas consistent lets optimizers, physical planners, validators, and future type-aware operators rely on the normal DataFusion contract. It also keeps the generic physical MergeScan implementation independent of JSON2.

The boundary schema and the storage read layout answer different questions. For SELECT j.a::BIGINT FROM t, the storage scan may use a structured {a: Int64} layout, while the distributed boundary can still emit either the complete j as Binary or only a as Int64. Therefore this rule cannot replace the separate JSON2 scan-type inference.

Trait Implementations

impl AnalyzerRule for JsonSchemaConcretizeRule

fn analyze( &self, plan: LogicalPlan, _config: &ConfigOptions, ) -> Result<LogicalPlan>

Rewrite plan

fn name(&self) -> &str

A human readable name for this analyzer rule

impl Debug for JsonSchemaConcretizeRule

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.