API contract testing for SA interviews

Why contract testing matters

Picture the scene most systems analysts walk into during onsite loops at Stripe, Uber, or DoorDash: a microservice graph with 40+ services, a payments team that ships twice a day, and an integration suite that takes 22 minutes to go green. The interviewer asks: "How would you stop the payments service from breaking checkout when the team renames a field?" You can answer "more E2E tests" — and you will get a polite nod and a rejection email. Or you can answer contract testing, and the conversation becomes a real design discussion.

The pain that contract testing solves is specific. End-to-end tests are slow, flaky, and shared across teams, so nobody owns them. Unit tests on the provider side never catch the case where a consumer relies on a field the provider considers private. The result is a silent contract — assumptions baked into client code that nobody documented. When the provider changes that field, production breaks and the postmortem blames "communication."

Contract testing makes the contract explicit and machine-verifiable. The point is not to replace E2E tests — it is to push 80% of integration confidence into fast, isolated checks that run in seconds, so E2E only catches the truly weird stuff. In an SA interview, you are expected to know the difference between provider-driven and consumer-driven flavors, name the tooling, and call out where each one breaks down.

Provider-driven contracts

In a provider-driven model, the team that owns the API publishes a contract — almost always OpenAPI 3.1 these days — and consumers are expected to conform. The contract lives in the provider's repo, gets versioned alongside the code, and is the single source of truth. Tools like Spectral lint the spec, Schemathesis generates property-based tests from it, and Prism spins up a mock server consumers can develop against.

The mental model is simple:

1. Provider authors OpenAPI 3.1 spec and publishes it (Swagger Hub, GitHub, internal registry).
2. Each consumer integrates against that spec.
3. Consumer CI runs schema validation against responses from the provider's staging environment.
4. Provider CI runs Schemathesis on every PR to verify the implementation still matches the spec.

A minimal OpenAPI snippet the interviewer might draw on the whiteboard:

openapi: 3.1.0
info:
  title: Payments API
  version: 2.4.0
paths:
  /charges:
    post:
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ChargeRequest'
      responses:
        '201':
          description: Charge created
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Charge'
components:
  schemas:
    Charge:
      type: object
      required: [id, amount, currency, status]
      properties:
        id: { type: string }
        amount: { type: integer, minimum: 1 }
        currency: { type: string, enum: [USD, EUR, GBP] }
        status: { type: string, enum: [pending, succeeded, failed] }

Strengths. One spec, one source of truth, low coordination cost when there are few consumers. Works well when the provider has clear authority — public APIs (Stripe, Twilio), platform teams inside Big Tech.

Weaknesses. Provider has no visibility into which fields consumers actually use. Renaming an "unused" optional field at 2 PM Thursday is exactly how production incidents happen. Provider-driven contracts solve the spec problem; they do not solve the awareness problem.

Consumer-driven contracts with Pact

Consumer-driven contracts flip the direction. Each consumer team writes integration tests against a mock of the provider, and those tests generate a contract file as a side effect. The provider then has to verify it satisfies every consumer's contract before shipping. Pact is the dominant tool — open source, has a JVM, Node, Python, Go, .NET, and Rust implementation, and ships with a hosted broker (Pactflow) or a self-hosted one.

1. Consumer writes integration tests using a Pact mock server.
2. Tests pass → Pact emits a JSON contract describing every interaction.
3. Contract uploaded to the Pact Broker, tagged with consumer version + branch.
4. Provider CI pulls all contracts for "production" consumers and runs Pact verification.
5. If verification fails, the provider's PR is blocked.

A toy consumer-side test in JavaScript:

const { PactV3, MatchersV3 } = require('@pact-foundation/pact');
const { like, integer } = MatchersV3;

const provider = new PactV3({ consumer: 'checkout', provider: 'payments' });

provider
  .uponReceiving('a request for a charge')
  .withRequest({ method: 'POST', path: '/charges', body: { amount: 2500, currency: 'USD' } })
  .willRespondWith({
    status: 201,
    body: like({ id: 'ch_123', amount: integer(2500), currency: 'USD', status: 'succeeded' }),
  });

The matcher like() is the load-bearing trick. Pact does not assert exact equality on the response — it asserts type compatibility. If the provider returns { id: 'ch_xyz', amount: 4200, ... }, the test still passes, because the shape matches.

Strengths. Consumers are protected by construction. Provider sees a real list of consumers and the exact fields they read. Naming a field "internal" stops being a guess.

Weaknesses. Setup cost is real — every team needs a broker, CI integration, and a version-tagging strategy. Pact does not handle asynchronous messaging well out of the box (you need pact-message or a separate AsyncAPI flow). And it adds friction for fan-out APIs with 50+ consumers — the provider's CI runs slower as contracts accumulate.

OpenAPI testing tools

Even if the team is fully consumer-driven, OpenAPI specs still appear in interviews because they double as documentation, mock servers, and client SDK generators. The tools you should be able to name on the spot:

Schemathesis generates property-based tests directly from an OpenAPI spec. Hand it a URL and a spec, and it will hammer the endpoint with thousands of randomized but spec-valid payloads, then check the response matches the declared schema. Great at catching nullability bugs and type drift.

Dredd verifies an HTTP API against OpenAPI or API Blueprint. Less popular today, but still common in older Python and Ruby stacks.

Spectral lints the OpenAPI spec itself — catches missing descriptions, inconsistent naming, version drift. Run it in CI so the spec quality does not decay.

Prism runs a mock server from the spec, so frontend teams can develop against a fake before the backend ships.

In an interview, the right answer to "how do you keep the spec honest?" is: Spectral lints the spec on every PR, Schemathesis runs nightly against staging, and the build fails if either complains.

Schema validation at runtime

Static contracts catch design-time bugs; runtime validation catches the bugs that escape static checks. The pattern is to validate every request and every response at the service boundary, even in production, with a sampling rate when the payload is heavy.

Server-side validation happens when a request arrives. The framework (FastAPI with Pydantic, Express with Zod, Spring with Bean Validation) rejects malformed payloads with a 400 before the handler ever sees them. This is table stakes.

Client-side validation is the part teams skip. When the consumer receives a response, it validates the shape against the schema it expects. If the provider drifts, the consumer logs a warning, increments a metric, and ideally falls back to a safe default. This is also how you catch the "the provider added a new required field and forgot to tell us" failure mode that contract tests miss in async messaging.

Common runtime validators:

In a contract testing context, runtime validation is the safety net that catches the 5% of drift that static contracts miss — usually in environments where the staging spec and the production spec disagree, or in third-party APIs you cannot run Pact against.

Common pitfalls

The most common mistake interview candidates make is conflating contract testing with integration testing. Contract testing is fast and isolated — the provider runs verification against a contract file, not a live consumer. Integration testing spins up multiple real services. If your answer involves "spinning up the consumer in Docker," you are describing integration testing, not contract testing, and a senior interviewer will press on the difference.

A second trap is over-specifying matchers in Pact. New users write tests with exact-value assertions, then complain that contracts break every time the provider returns a different generated ID. The fix is to use type matchers like like() and integer() everywhere except for fields where the exact value carries semantics — enum values, status codes, error codes. The rule of thumb: if a field's value affects branching logic in the consumer, assert it exactly; otherwise assert the type.

The third pitfall is contract testing async flows with synchronous tools. Pact has pact-message for one-shot message verification, but Kafka topics with schema evolution need a different approach — typically AsyncAPI plus Schema Registry (Confluent or Apicurio). Candidates who claim "Pact covers Kafka" without qualifications are wrong, and good interviewers will catch it.

A fourth, subtler pitfall is treating the contract as a freeze. Contracts are versioned artifacts — providers can add optional fields without breaking consumers (this is the Tolerant Reader pattern), can deprecate fields with a grace window, and can rev the major version when breaking changes are unavoidable. Teams that treat any spec change as forbidden end up working around the system with undocumented headers and query params, which defeats the purpose.

Finally, the most political pitfall: contract testing without a broker. You can technically file-system-share Pact contracts in a monorepo, but the moment you split teams across repos, the broker becomes mandatory. Skipping it because "we'll add it later" is how you end up with 127 stale contract files in a shared S3 bucket and zero CI enforcement.

Related reading

• API Gateway vs BFF for systems analyst interviews
• Authentication vs authorization for SA interviews
• Kafka for systems analyst interviews
• Case interview for systems analyst
• Data contracts in data engineering interviews

If you want to drill systems analyst questions like this every day, NAILDD ships with 1,500+ interview problems across exactly this pattern — API design, contracts, distributed systems, and the trade-off questions hiring managers actually ask.

FAQ

Is contract testing the same as API testing?

No. API testing is a broad category that includes functional tests, load tests, security tests, and contract tests. Contract testing is specifically about verifying that a provider and consumer agree on the interface shape and behavior, without spinning up the real consumer. A request that returns 200 with the wrong response shape passes API functional testing but fails contract testing.

When should I use provider-driven vs consumer-driven?

Use provider-driven for public APIs, platform services with many unknown consumers, or when the provider has clear authority and stable consumers. Use consumer-driven when you have a known set of internal consumers, when the provider has been surprised by breaking changes in the past, or when the political dynamics favor protecting consumers. Many mature orgs run both — OpenAPI for the spec, Pact for the consumer-driven guarantees on top.

Does contract testing replace E2E tests?

No, but it should shrink them. A healthy split is roughly 70% contract, 20% integration, 10% E2E for service-graph confidence. E2E tests still catch user-flow bugs, third-party integrations, and infrastructure issues that contracts cannot see. The goal is to push the fast feedback into contract tests so E2E stays small, focused, and reliable.

How do contracts handle versioning?

The standard pattern is semantic versioning on the contract itself — additive changes (new optional fields) are minor, removing or renaming fields is major. Pact contracts get tagged with consumer version and branch (e.g., checkout@1.4.2-main), and the broker tracks which versions are deployed in production. Providers verify against the production-tagged contracts on every PR.

How does contract testing fit with Schema Registry for Kafka?

For async flows, the analog of OpenAPI is AsyncAPI, and the analog of runtime validation is the Schema Registry (Confluent, Apicurio, or AWS Glue). Producers register schemas; consumers fetch and validate. You can layer Pact's pact-message on top for explicit consumer-driven verification, but the Schema Registry handles the bulk of compatibility checking — typically with backward-compatible evolution as the default.

What is a Pact Broker and do I really need one?

The Pact Broker is a central service that stores contracts and tracks which consumer versions are compatible with which provider versions. Without it, you are emailing JSON files around. With it, you get the can-i-deploy check — a CI gate that blocks deploys if the version you are about to ship has not been verified against all production consumers. For anything beyond a two-team toy project, the broker is mandatory.