Prompt
How do I validate an OpenAPI spec against real API behavior?
Latest observation
Nov 15, 2026
OpenAI APIWeb search: off
To validate an OpenAPI spec against real API behavior, you want to check two things:
- Does the spec match what the API actually does?
- Does the API conform to the spec?
A good validation workflow usually combines both contract testing and spec linting.
1) Compare the spec to observed traffic
Capture real requests/responses from production, staging, or a test environment, then compare them to your OpenAPI document.
What to check
- Paths and methods exist in the spec
- Request parameters match documented names and types
- Request body schemas match actual payloads
- Response status codes are documented
- Response schemas match actual responses
- Required/optional fields align
- Error responses are documented too
Tools
- OpenAPI validators/lints:
spectral,swagger-cli,openapi-spec-validator - Traffic capture / schema inference:
- Optic
- Dredd
- Postman
- Schemathesis (also does property-based testing)
2) Run conformance tests against the live API
Generate tests from the OpenAPI spec and run them against your API.
What this checks
- The API returns responses that conform to schema
- The API rejects invalid inputs as expected
- Edge cases are handled properly
Useful tools
- Dredd: compares API implementation to OpenAPI spec
- Schemathesis: generates tests from OpenAPI and fuzzes inputs
- Prism: can mock from OpenAPI, useful for validating expected behavior vs implementation
- Postman/Newman: collection-based tests derived from the spec
3) Validate the spec itself
Before testing the API, ensure the spec is structurally valid.
Recommended checks
- OpenAPI syntax/format validity
- Schema correctness
- Reference resolution (
$ref) - No circular or unresolved references
- Consistent use of components, parameters, and responses
Tools
swagger-cli validateopenapi-spec-validatorspectral lint
4) Use a contract-testing loop
A practical workflow:
- Lint spec in CI
- Run conformance tests against staging
- Compare observed traffic to spec regularly
- Fail builds when API behavior diverges from the spec
- Update the spec or implementation to resolve drift
5) Example approaches by goal
If you want to check spec drift
Use Optic or Dredd with recorded traffic.
If you want to test implementation behavior
Use Schemathesis or Dredd against a running API.
If you want to validate only the spec file
Use Spectral and openapi-spec-validator.
6) Common pitfalls
- Spec documents only the “happy path”
- Missing error responses
- Response examples don’t match actual payloads
- Loose schemas hide bugs
- Auth behavior not covered
- Pagination, filtering, and sorting are under-specified
A simple recommended stack
If you want a practical setup:
- Spectral for spec linting
- Schemathesis for automated conformance/fuzz testing
- Optic or traffic comparison for drift detection in CI
If you want, I can also give you:
- a step-by-step CI pipeline example
- a tool comparison table
- or sample commands for Dredd/Schemathesis/Spectral