The Transform Window

What it is for

The Transform window is the Data Hub's own wrangling tool, in the spirit of Power Query. It handles plain table work: renaming, retyping, filtering, sorting, deduplicating, grouping, joining, reshaping and checking data. Machine-learning-specific work (encoders, scalers, imbalance handling, PCA, embeddings, imputation and custom code) lives in PrepFlow.

Open it from a dataset with the Transform button. It is available for formats DuckDB can read, so a live database preview must first be imported as a snapshot.

The layout

• A ribbon of commands across the top.
• Applied steps on the side: your recipe, in order. Click a step to see the data as of that step, edit its settings in the inspector, switch it on or off, or remove it. A small marker shows when a step runs as SQL in the engine.
• A preview grid showing the current result.
• A code drawer with the generated Python (pandas) and SQL.

The ribbon

Commands are grouped on four tabs:

• Home: Choose columns, Reorder columns, Rename, Data type; Filter rows, Remove duplicates, Remove missing, Remove empty; Sort rows, Group by; Join and Append.
• Transform: Trim, Change case, Format, Split column, Merge columns; Replace values, Extract (regex), Data type; Pivot, Unpivot, First row as header; Clean names.
• Add column: Custom column from a formula; or build a column from existing ones with Merge columns, Split column, Extract (regex) or Hash.
• Data quality: assertions that a column has no nulls, stays within a range or is unique; Drop constant columns, Remove empty, Remove duplicates.

Some commands appear on more than one tab. Pick a command, set its options in the inspector, and the preview updates.

Joining and appending

Join combines the current table with another dataset in your library on one or more key pairs. Append stacks another dataset under it, mapping columns by name; a mapping can also be supplied as JSON. Steps that bring in another dataset are marked in the applied list.

Code views

• SQL tab: shows steps folded into one SQL query, and tells you how far the fold went ("folded through step N · the rest runs locally"). You can edit and run a query as a scratchpad, which never changes your steps. A plain SELECT (projection, WHERE, ORDER BY, CAST, simple aggregates and a single-condition JOIN) can be converted back into steps with → steps or used to replace the whole recipe. Anything outside that grammar is rejected and your steps stay as they were.
• Python tab: the equivalent pandas code, for reading or copying. It is display-only.

Parameters and templates

A recipe can declare parameters (text, number, true/false, choice or column) and use them in step settings as {{name}}. The values used for a particular run are stored with the resulting version, and a recipe is hashed with its references intact, so the same template stays the same however it is applied. Choose Save as template to keep a recipe in the project; Templates in this project lists them for reuse on other datasets.

Saving

Save writes the result as a new dataset version; the original version is kept and the new one records which recipe produced it. Open Versions to compare, promote, roll back or pin. Recipes appear in the lineage graph, and a pipeline can apply a saved recipe with Apply Recipe.

Limits

• Some steps work only on the bounded preview; SQL-folded steps and native formats handle far larger tables.
• Datasets parsed outside DuckDB (non-native formats) are bounded to 16 MB or 200,000 rows. Convert large inputs to Parquet.
• A step that cannot fold to SQL runs locally on the loaded rows and says so in the code drawer.