Docs Operators

Operator reference.

A pipeline is built from operators on a canvas. Each operator takes rows in, does one thing, and passes rows on, with the schema and the row count shown at every step. The 150+ operators fall into six families.

FamilyOperators (examples)What they do
CleansingTrim and standardise · Null handling · Deduplicate (exact and fuzzy) · Type conversion with error handlingMake values consistent before anything else looks at them.
EnrichmentLookup (cached) · Derived column · Aggregate · Pivot and unpivot · Sequence · RankAdd what the target needs from other tables and from expressions.
ValidationFormat check · Range check · Referential check · Business rule · Grain sum checkDecide whether a row is valid; a failed row becomes an exception record with an owner.
RestructuringJoin · Union · Split (conditional) · Sort · Window functions · Normalise nested structuresChange the shape of the data to the target's model.
Planning shapesWays of supply to sources · Hierarchy validation · Bucket alignment · Zero-fill history · Referential orderThe shaping steps a planning model needs, packaged.
AdvancedPython script · SQL transform · REST call · ML scoring · Pushdown to the warehouseCustom logic where a packaged operator is the wrong fit.

How operators behave

Linked tables. Each operator writes to an intermediate table that the next one reads, so a step can be inspected and re-run on its own. Debug mode. A pipeline runs interactively on a sample while it is built, with rows shown at every step. Exception outputs. Every validation operator has a valid output and an exception output; nothing is dropped. Lineage. Every column in a target traces back through the operators to its source column.

Python and SQL

The Python operator runs a function you write against the rows of a step, with the platform's libraries available; it is how scripts from other tools are carried over. The SQL operator runs a statement against the intermediate tables or, in ELT patterns, pushes the statement down to the warehouse.

The full reference with parameters per operator is in the product documentation behind sign-in.