Skip to main content

schemafit

Lint your LLM structured-output & JSON schemas against OpenAI, Anthropic, and Gemini — and fail CI before they 400 in production.

Open source (MIT) · Static & offline · Zero runtime dependencies · CI-enforceable.

The problem

A JSON Schema / tool definition / response_format that works on OpenAI can 400 on Anthropic or Gemini (and vice-versa): nested oneOf, a missing additionalProperties: false, a default in a property, Anthropic-rejected keywords (minLength, format, pattern), Gemini’s lack of anyOf. The API tells you it failed — not which constraint — so teams hand-port schemas and debug by trial-and-error at runtime.

schemafit encodes each provider’s documented constraint surface as a versioned, declarative rule pack and lints your schema statically — pointing at the exact JSON-Pointer path, the keyword, and why — with a non-zero exit code so CI fails the PR instead of prod. A repair pass emits a best-effort provider-valid variant. It makes no model calls, needs no API key, and has zero runtime dependencies.

Quickstart

On PyPI — pip install and lint your schema. No API key, no model calls; the core has zero runtime dependencies.

Install (Static, offline CLI):

pip install schemafit

Command surface

  • lint: Lint one or more schemas against one or more providers; reports each violation’s JSON-Pointer, keyword, and reason, and exits non-zero so CI fails the build.
  • repair: Emit a best-effort provider-valid variant of a schema — stripping or rewriting the offending keywords while preserving intent.
  • providers: List the supported providers and the versioned rule pack backing each (OpenAI, Anthropic, Gemini).
  • demo: Run a hermetic, end-to-end proof from a fresh clone — no API key, no network, deterministic output.

Why it is different

  • Static, pre-ship CI lint: Instructor, BAML, LiteLLM and the Vercel AI SDK are excellent runtime clients — they normalize or repair at call-time. schemafit fills the gap they leave: a static lint that fails the build before the schema ever reaches a provider, over the raw schemas you already ship, with no DSL or codegen buy-in.
  • Zero runtime deps, no API key: It makes no model calls and needs no key — easy to vendor, audit, and trust in a build pipeline. Runs from a fresh clone or a single pip install, fully offline.
  • Grounded, versioned rule packs: Every rule is grounded in a real, cited provider issue. Provider constraints are encoded as declarative, versioned rule packs — so the lint stays honest and reviewable as provider APIs change.
  • CI-enforceable, with SARIF: A non-zero exit fails the PR; lint --format sarif feeds GitHub code-scanning / the Security tab. An opt-in --live-verify can confirm against a real provider, but it is mock-default and never required in CI.

Links