Dataprep on GCP: Visual Data Cleaning and Where It Fits Today

Before you can analyze data, you usually have to fix it. Inconsistent date formats, phone numbers with three different punctuation styles, whitespace-padded strings that break joins silently, columns that are supposed to be numeric but have “N/A” mixed in — this is the unglamorous work that eats a disproportionate share of every data project’s timeline. Dataprep was Google’s answer: a visual data preparation tool, built in partnership with Trifacta, that let you profile a dataset, spot these problems visually, and build a repeatable cleaning “recipe” without writing pandas or Spark code by hand.

If you’re evaluating it today, there’s an important piece of context worth knowing upfront: Google has increasingly steered new projects toward Cloud Data Fusion’s built-in Wrangler feature rather than the standalone Dataprep product, and it’s worth understanding both the original tool’s concepts (which are genuinely well-designed and still show up conceptually in its successor) and the practical migration path if you’re inheriting an existing Dataprep-based pipeline.

What Dataprep Was Built to Do

Dataprep’s core workflow had three stages, and understanding them is useful regardless of which specific tool you end up using, because the same conceptual pipeline shows up across every visual data-cleaning tool in this space.

┌─────────────┐     ┌──────────────┐     ┌─────────────────┐
│   Profile    │ ──► │    Recipe     │ ──► │      Run         │
│ (understand  │     │  (build       │     │  (execute as a   │
│  the data)   │     │   cleaning    │     │   Dataflow job)  │
│              │     │   steps)      │     │                  │
└─────────────┘     └──────────────┘     └─────────────────┘

Profiling happens automatically the moment you load a sample of your data. Dataprep computes column-level statistics — null percentages, unique value counts, min/max, a value distribution histogram — and visually flags anomalies like mismatched data types or outliers, without you writing a single query.

Recipe building is where you clean the data by direct manipulation: click a column, and Dataprep suggests transformations based on what it sees (split on delimiter, standardize date format, remove outliers). Every action becomes a step in a human-readable recipe, similar in spirit to a Jupyter notebook’s cell history but expressed as declarative transformation steps rather than code.

Execution compiles the recipe into an actual Cloud Dataflow (Apache Beam) job and runs it against the full dataset — the interactive preview only ever touches a sample, but the recipe scales to the entire dataset when you run it for real.

A Representative Recipe

Even without the UI in front of you, recipe steps are readable enough to reason about directly:

1. splitCol col: 'full_name' on: ' ' limit: 1
2. rename col: 'full_name_1' to: 'first_name'
3. rename col: 'full_name_2' to: 'last_name'
4. deleteCol col: 'full_name'
5. reformatDate col: 'signup_date' format: 'yyyy-MM-dd'
6. filter col: 'email' condition: isMismatched('email')
7. derive value: IF(ISMISSING([country]), 'Unknown', [country]) as: 'country_clean'

Reading this: split a full name into first and last name columns, drop the original, normalize a date format, filter out rows with malformed email addresses, and backfill missing country values with a default. This is exactly the kind of transformation that’s tedious and error-prone to write by hand in pandas across dozens of columns, and genuinely faster to build by clicking through real data samples and seeing the effect immediately.

Why the Landscape Has Shifted

Google positioned Cloud Data Fusion’s Wrangler feature — which shares real conceptual and technical DNA with Dataprep, since both descend from the same underlying data-wrangling approach — as the forward-looking path for visual data preparation on GCP. If you’re starting a new project today, building your cleaning logic inside a Data Fusion pipeline (where Wrangler directives are one step in a broader pipeline that also handles sourcing, joining, and loading) is the more future-proof choice, because it keeps preparation and pipeline orchestration in a single, actively developed product rather than two separate tools.

For teams that already have production Dataprep flows, this doesn’t mean an emergency migration — but it does mean new investment (new recipes, new team training) is better placed in the successor tooling, and existing flows are worth documenting well since long-term platform support commitment favors the newer product.

Migrating a Dataprep Flow to Data Fusion Wrangler

The practical migration path isn’t a lift-and-shift — recipe syntax and directive syntax differ — but the conceptual mapping is close enough that migration is more translation than redesign:

Export the Dataprep recipe as a readable script (Dataprep supports exporting recipes in a documented format).
Recreate the pipeline shell in Data Fusion: add the same source connection and sink as your original Dataprep flow’s input/output.
Translate each recipe step to a Wrangler directive. Most operations have a near-1:1 equivalent — splitCol maps to parse-as-csv or split-to-columns, deleteCol maps to drop, filter maps to filter-row-if-matched or filter-row-if-true.
Validate against the same sample data used in the original Dataprep profile, comparing output row-by-row on a small batch before pointing the new pipeline at production data.
Run both pipelines in parallel for one or two cycles before decommissioning the original, to catch any subtle transformation differences before they affect downstream consumers.

How Profiling Actually Catches Problems

The reason profiling matters more than it sounds is that data quality issues are rarely uniform — they cluster. A column that’s 99.5% clean numeric data with a handful of “N/A” strings mixed in will pass a casual glance at the first twenty rows and then throw a type error the first time you try to aggregate it in production.

This is the actual value of visual profiling over a blind script: the histogram surfaces exactly how many rows fall into each anomaly bucket before you write a single transformation, so the recipe you build is informed by the real shape of the data rather than assumptions about what it should look like. A negative order total, for instance, is a decision point — is it bad data, or a legitimate refund record that needs different handling rather than deletion? That’s a judgment call a human should make deliberately, and profiling is what surfaces the question in the first place instead of letting it get silently mishandled by a generic cleaning rule.

Real-World Use Case: Marketing Data Cleanup

A marketing analytics team pulling campaign data from six different ad platforms, each with its own date format, currency notation, and column naming convention, is a classic use case for this class of tool. Rather than writing and maintaining six separate parsing scripts, a data preparation recipe per source standardizes each feed into a common schema before it lands in BigQuery for cross-platform reporting. The visual profiling step is what actually saves the most time here — spotting that one platform’s “spend” column is sometimes a string with a currency symbol and sometimes a raw float, for instance, is something you notice immediately in a profiled histogram and might miss entirely in a blind script.

Best Practices (Whichever Tool You Use)

Always validate the recipe against edge cases, not just the happy path. Profiling on a random sample can miss a rare-but-real data quality issue that only shows up in 0.1% of rows — deliberately test against known bad records.
Keep recipes documented outside the tool itself. A recipe that lives only inside a UI, understood by one person, is a maintenance risk. Export and version-control the recipe definition alongside your other pipeline code.
Separate profiling from production execution. Interactive profiling samples a subset of data for speed; always validate that a recipe’s assumptions (a column is always numeric, a date is always in one format) hold across the full dataset before trusting it in production.
Re-profile periodically. Source data drifts. A recipe built against last year’s data assumptions can silently produce wrong output when an upstream system changes its export format.

Common Mistakes

Trusting the sample too much. A recipe built entirely against a small profiled sample can fail or silently mishandle data outside that sample’s range — always test-run against a larger or full-scale batch before scheduling a recipe for production.

No error routing. Rows that don’t match expected cleaning assumptions should be routed to an error or quarantine output rather than silently dropped or, worse, silently passed through malformed. This is a discipline that’s easy to skip when a tool makes cleaning feel effortless.

Treating visual cleaning as a substitute for actual data contracts. A cleaning recipe patches symptoms; it doesn’t fix a source system that produces inconsistent data. For pipelines that matter, push back on upstream teams to fix data quality at the source rather than accumulating an ever-larger recipe of workarounds.

Frequently Asked Questions

Is Dataprep still usable on GCP? Existing Dataprep flows continue to function, but Google’s investment and forward guidance point toward Data Fusion’s Wrangler for new development — treat new Dataprep adoption cautiously and evaluate Data Fusion first for greenfield work.

Does data preparation replace the need for a data engineer? No — it lowers the bar for building cleaning logic and speeds up iteration, but designing robust error handling, understanding data lineage, and integrating cleaning into a broader pipeline still benefits from engineering involvement.

What engine actually runs a Dataprep recipe? Dataprep compiles recipes into Cloud Dataflow (Apache Beam) jobs — the interactive UI is a design surface, but execution at scale runs on the same distributed processing engine used for hand-written Beam pipelines.

How is this different from just writing a BigQuery SQL script to clean data? For teams comfortable in SQL, cleaning directly in BigQuery with SQL transformations is a completely valid alternative, and often simpler for smaller teams. Visual data-prep tools earn their value primarily when the data has structural, not just value-level, problems (inconsistent schemas across sources, genuinely messy semi-structured data) where interactive profiling meaningfully speeds up figuring out what “clean” should even look like.

A Word on Cost and Ownership

Dataprep billed based on the underlying Dataflow compute consumed at execution time, plus a per-row Dataprep service fee on top — which meant a recipe that looked cheap to design in the interactive UI could turn out expensive to run repeatedly at full data volume. This is a useful lesson that outlasts the specific tool: any visual data tool that abstracts away the execution engine is still billing you for that engine underneath, and it’s worth checking the actual job cost after the first few production runs rather than assuming the friendly UI means friendly pricing. The same caution applies directly to Data Fusion, which has its own version of this gap between “designing feels free” and “running at scale has a real bill.”

Summary

Dataprep’s core idea — profile the data visually, build a cleaning recipe by direct manipulation, and run it at scale as a real distributed job — was a genuinely useful pattern, and it’s one of the reasons the concept lives on inside Data Fusion’s Wrangler rather than disappearing. If you’re inheriting existing Dataprep pipelines, they’ll keep working, but plan new development around Data Fusion instead. If you’re starting fresh and the core need is cleaning messy, inconsistent source data before it lands in a warehouse, evaluate Wrangler first — you get the same interactive, click-to-transform workflow inside a tool Google is actively continuing to invest in.