Medallion architecture: bronze, silver, gold

Why medallion shows up in every DE interview

If you have a Data Engineer onsite at Databricks, Snowflake, Netflix, Stripe, or any company running a lakehouse, you will get a medallion question. It might be phrased as "walk me through your layered architecture" or as the hostile version: "why do we even need a bronze layer if the source system already exists?" Either way, the interviewer wants to hear that you understand append-only ingestion, idempotent transforms, and where business logic lives.

The reason it is a default screening topic is that medallion is the cheapest way to test whether a candidate has shipped a real pipeline. Anyone can name-drop bronze, silver, gold. Only people who have actually owned production data know what goes wrong at each boundary, why schema evolution breaks silver before it breaks bronze, and how to write a tests suite that catches the issue before the dashboard does. Senior loops dig into time-travel, CDC into bronze, and contract enforcement at the silver-to-gold edge.

Bronze layer

Bronze is the raw landing zone. The job of this layer is to look as much like the source as possible while still being queryable. You parse JSON or Avro into columns, cast strings to dates, and stop. No joins, no enrichment, no filtering by is_test = false. The minute you start curating, you have killed the replay property.

What you store at bronze:

• Near-exact copy of the source, including columns you think are garbage today.
• Append-only writes — never overwrite, never delete.
• Ingestion metadata: load_timestamp, source_file, ingest_id, and ideally a hash of the raw payload for dedup.

CREATE TABLE bronze.events (
  raw_data VARIANT,             -- raw JSON, kept verbatim
  user_id BIGINT,
  event_at TIMESTAMP,
  ingested_at TIMESTAMP,
  source_file STRING,
  ingest_id STRING
);

The single biggest reason bronze exists: when the product team redefines what counts as an "active user" in Q3, you do not have to beg Kafka for a six-month replay. You re-run silver from bronze and the dashboards update by Monday. This is the moment juniors who skipped bronze learn why senior engineers insist on it.

Silver layer

Silver is the clean, conformed, joined view of the world. This is where business identity gets resolved (one user_id across web, mobile, server events), where timestamps move to UTC, where currencies normalize to USD, and where SCD2 dimensions track slowly changing attributes like country or plan tier.

What belongs in silver:

• Deduplication on the natural key.
• Type normalization (UTC timestamps, normalized currency, lowercase emails).
• Joins to lookup and dimension tables.
• SCD2 for changing attributes — plan changes, country moves, role transitions.
• Strict idempotency — re-running the job twice produces the same output.

What does not belong in silver:

• Business aggregations like DAU or weekly revenue.
• Denormalization tuned for a specific dashboard.

CREATE TABLE silver.events (
  event_id STRING,
  user_id BIGINT,
  user_country STRING,        -- joined FROM dim_user
  event_type STRING,
  event_at TIMESTAMP,         -- UTC
  amount_usd DECIMAL(18, 4),  -- normalized
  device_type STRING,
  is_test BOOLEAN
);

Silver is the source of truth for analytics and ML. The data scientist building a churn model reads from silver, not bronze, not gold. If silver is wrong, every downstream consumer is wrong. This is also why silver gets the strictest test coverage of any layer.

Gold layer

Gold is shaped for consumption. Star schema, denormalized marts, pre-aggregated rollups — whatever the BI tool or ML feature store needs. A gold table usually corresponds to one specific consumer: a Tableau dashboard, a Looker explore, a feature view in the ML platform.

CREATE TABLE gold.daily_revenue (
  DATE DATE,
  country STRING,
  product_category STRING,
  revenue_usd DECIMAL(18, 2),
  orders_count BIGINT,
  unique_users BIGINT
);

You typically end up with far fewer gold tables than silver tables, but each gold table is more opinionated. Performance matters here — pre-aggregations, clustering keys, materialized views, and column pruning are all fair game because the goal is sub-second query latency for the dashboard.

Tests at every layer

Layer-appropriate testing is what separates a candidate who has run pipelines from one who has only read about them. The interviewer is watching to see if you can map test types to the layer where they make sense, and the table below is the structure most senior DEs draw on the whiteboard.

At bronze, you mostly check that the data showed up at all — empty file, missing partition, new column you did not expect. You usually alert but do not block, because bronze is supposed to absorb whatever the source threw at you.

At silver, the tests get strict. Uniqueness on the primary key is non-negotiable. NOT NULL constraints on required columns. Referential integrity to dimensions. Accepted-values checks on enums like status and event_type. A failure here should halt the downstream gold build, because shipping silver with duplicates corrupts every aggregate.

At gold, you move into business-rule tests and reconciliation. Revenue must be non-negative. The sum across country dimensions must equal the fact total. DAU must be less than total registered users. You also add trend-based sanity checks — if today's DAU is 40% below the rolling 28-day median, page someone.

Data contracts

Modern teams formalize the boundary between layers as an explicit data contract. A contract is just a machine-readable promise: the producer of this table guarantees X about its shape, freshness, and quality, and the consumer can rely on it.

A contract typically describes:

• Schema — columns, types, nullability.
• Constraints — primary key, foreign keys, uniqueness.
• Freshness SLA — for example, "silver.orders is no more than 6 hours behind production".
• Quality SLA — for example, "uniqueness ≥ 99.9% on order_id".
• Versioning — semver, with breaking changes triggering a major bump and a migration window.

Common tools: Soda Contracts, Great Expectations, dbt model contracts, Apache Avro Schema Registry. The exact tool matters less than the cultural shift — a contract makes it explicit who is on the hook when a downstream dashboard breaks. Without contracts, every silver-to-gold incident turns into a finger-pointing exercise between the platform team and the analytics team.

Contracts matter most at two boundaries. The first is bronze ↔ source, because the upstream provider is usually outside your team's control, so you need an automated trip-wire when they ship a schema change. The second is silver ↔ gold, because that boundary has the most downstream consumers — a breaking change to a silver table ripples through dozens of dashboards.

Common pitfalls

Skipping bronze and writing straight to silver. The team saves a day of work, then loses a week when the marketing org redefines "qualified lead" and the source system has only kept 30 days of raw events. Always land bronze first, even if the bronze table is ugly. The append-only history is the entire reason the layer exists.

Letting business logic creep into bronze. Filtering out test users, joining to dimensions, or applying revenue recognition rules at the bronze level all break replay. When the rule changes — and it always changes — you cannot reconstruct history without re-ingesting from the source. Bronze stays dumb; logic moves to silver.

Blurring the silver-gold boundary. If silver already contains daily aggregations, your gold layer becomes a thin pass-through. The boundary collapses, ownership gets murky, and you end up with five tables that all do almost the same thing. Keep silver row-grain (one row per event or per entity-state) and push aggregations to gold.

Testing only the gold layer. By the time a test fails on gold, the bad data has already been written to silver and possibly to bronze partitions. The fix is to shift tests left — uniqueness and not-null on silver, freshness and schema match on bronze. Most production incidents announce themselves at silver if you are paying attention.

Not tracking lineage. With three layers and dozens of tables per layer, you cannot debug a wrong dashboard number without column-level lineage. Use dbt docs, OpenLineage, or DataHub to keep the dependency graph queryable. Lineage is also what makes incident response fast — when a column is wrong in gold, you trace it back to the silver model and then to the bronze partition in minutes, not hours.

One-shot full rebuilds at every layer. Fine for a side project, lethal for production. Bronze should be incremental on partition, silver should be incremental on a merge key, gold should be incremental or pre-aggregated. A nightly full rebuild burns compute, blows past SLAs, and stops being viable past a few hundred GB.

Ignoring time-travel. Iceberg and Delta give you table versioning for free. Use it for audits ("what did the table look like on Tuesday?") and for fast rollback after a bad deploy. Senior interviewers will ask about time-travel as a litmus test for whether you have run a modern lakehouse.

Related reading

• ETL vs ELT for Data Engineers
• Lakehouse: Iceberg vs Delta
• dbt incremental models
• Data contracts deep dive
• DWH layers explained
• Iceberg time-travel

If you want to drill DE interview questions like this every day, NAILDD is launching with 500+ data engineering problems across pipelines, modeling, and lakehouse architecture.

FAQ

Is bronze the same as a data lake?

Often, yes. Bronze typically lives in object storage — S3, GCS, ADLS — as Parquet, Iceberg, or Delta files. Silver and gold can be in the same lakehouse, or they can be pushed into a warehouse like Snowflake, BigQuery, or Databricks SQL Warehouse depending on how much BI traffic hits them. The boundary between "lake" and "warehouse" has blurred enough that the medallion model is layer-defined, not storage-defined.

How many layers are optimal?

Three is the standard — bronze, silver, gold. Some teams add a thin raw layer in front of bronze (literally the dropped file before any parsing) or an archive layer behind gold for compliance. Past five layers you are almost certainly over-engineering, and the marginal layer adds cost and confusion without paying for itself in clarity or replay.

Can I skip silver for a small pipeline?

For very small pipelines — one or two sources, a stable schema, and a single consumer — you can collapse silver into gold and run an ETL-style direct ingest. The risk is that "small" pipelines grow, and the day you onboard a second consumer is the day you regret not having a clean conformed layer. As a rule of thumb, once you have three or more downstream consumers of the same data, build silver explicitly.

How does CDC fit into medallion?

CDC streams from a source database land in bronze as an append-only log of row changes. Silver applies the changes to produce a current-state table (or a SCD2 history table). Gold reads from silver and computes business metrics. Debezium, Fivetran HVR, and AWS DMS are the common CDC tools; the medallion pattern stays the same regardless of which one feeds bronze.

What is the role of dbt in medallion?

dbt typically owns silver and gold — sources are declared on top of bronze, models build silver as conformed tables, and downstream models build gold as marts. Bronze is usually written by an ingestion tool (Airbyte, Fivetran, custom Spark jobs) outside of dbt. The handshake between the ingestion tool and dbt happens through dbt sources and source freshness checks.

Is medallion official Databricks-only terminology?

The name was popularized by Databricks, but the layered pattern predates the term. The same idea shows up as staging / integration / mart in Kimball-style warehouses, as landing / refined / curated in Azure docs, and as raw / clean / business in plenty of internal docs. The vocabulary is interchangeable; what interviewers care about is whether you understand the boundaries and the test strategy at each one.