pub(crate) fn rewrite_insert_assignments(
plan: LogicalPlan,
query_ctx: &QueryContextRef,
config: &ConfigOptions,
) -> Result<LogicalPlan>Expand description
Interprets strings assigned to timestamp columns at an INSERT boundary
using the session timezone. plan is the assignment projection under a
WriteOp::Insert.
DataFusion plans INSERT as a projection casting each source column to its
target column type, and that cast reads a naive string as UTC. Arrow does
apply a timezone when the cast target carries one, so the assignment is
routed through Timestamp(unit, Some(tz)) and back. Stripping the timezone
afterwards is value-preserving — arrow only shifts values in the opposite
direction.
The source query is left untouched. Reinterpreting a value where it is
produced would change what the source query means: pushing the conversion
below a UNION’s DISTINCT, for instance, moves the dedup key from the raw
strings to parsed instants and silently drops rows.
§Why TypeCoercion runs here
The rewrite reads source types, and those are only settled once a UNION’s
branch types have been reconciled: before coercion a union carries its loose
schema (the first branch’s types), so a mixed
SELECT 'string' UNION ALL SELECT CAST(.. AS TIMESTAMP) still looks like a
string. Retargeting that cast would leave Timestamp(None) -> Timestamp(Some(tz)) behind once coercion retypes the union — the one
direction in which arrow shifts the value instead of relabelling it.
Coercing here rather than deferring to the analyzer is forced by where an
INSERT is still identifiable: exec_dml_statement strips the Dml node and
executes its input, so by the time the analyzer runs, an assignment
projection is indistinguishable from any other projection.
Explicit casts stay out of this: the SQL layer turns a user’s
CAST(x AS TIMESTAMP) into an arrow_cast call, which only becomes an
Expr::Cast in the optimizer’s SimplifyExpressions. Assignment casts are
therefore the only Expr::Cast reaching a timestamp column here.
§Reach
The emitted cast only carries its timezone where the expression is evaluated
on this node. Substrait drops the timezone name when a plan is pushed down —
it encodes any zoned timestamp as PrecisionTimestampTz and decodes it back
as UTC — so a source reading from a table falls back to UTC, the behaviour it
had before this rule existed. Sources that never leave this node (literals,
VALUES, and UNIONs of them) keep the session timezone, and those are what
an INSERT’s timestamp assignment is in practice.