← All agent-stack verdicts

Structured output and constrained decoding

Adopt and wrap Confidence Medium Group Model serving Evidence as of 2026-09-23

Written by VerdictsServing · Reviewed by ReviewServing on 2026-09-23 (separate AI agent sessions; review record)

Bracketed tags are evidence tiers from RULEBOOK.md §1: [Verified] means checked directly, [CI-observed] read from a project’s CI pages, [Maintainer claim] asserted by the project, [External] a dated third-party source, [Inference] our judgment. A gold path:line link opens the cited line. NODUS is the four-ring verdict scale (Invest, Pilot, Explore, Monitor) used for the FrankenSuite assessments; the agent-stack verdicts use their own scale (Adopt, Adopt and wrap, Build clean-room, Watch). TRL is technology readiness level, scored 1 to 9. CI is continuous integration: automated checks that run on every push. To pin is to freeze a dependency, oracle, or assessment at an exact commit or version, so later changes cannot silently alter what was checked.

Bottom line

Inference, medium confidence: adopt and wrap an existing constrained-decoding engine (a library that masks the model's next-token choices so output must match a JSON Schema, regex or grammar) instead of writing one or prompting for JSON and hoping. Use xgrammar or another engine through your serving stack (vLLM ships adapters for four), outlines as a front end, or llama.cpp grammars for local models. Wrap it with two tests of your own: feed the engine's accepted outputs through an independent validator such as Pydantic, and put a compile-time ceiling on pathological schemas, because no project in the evidence runs a fuzzer that checks the mask against the schema.

Adopt, do not rebuild

License checked on 2026-09-23; non-permissive licenses are named in the text.

Copy these practices

Build only if

  • Build the test harness the ecosystem lacks, not a new engine: a generative differential fuzzer (random grammar, compare masks against a reference parser). The absence is verified within the pack's sample; building the harness is this verdict's advice. [Verified] [Inference] (ecosystem/pickup/_evidence/structured-output.md:35 No repo in this set runs a documented differential fuzzer; ecosystem/pickup/_evidence/structured-output.md:35 A clean-room project should add one; the ecosystem doesn't demonstrate it.)
  • Build an engine only if your runtime forbids C/C++/Python dependencies by policy, as franken_nlp's pure-Rust single-model runtime does by design; its own packet notes llama.cpp already ships grammar-constrained generation, and franken_nlp's guarantee has not been observed. All adopted engines here are Apache-2.0 or MIT (read 2026-09-23 with gh api repos/<owner>/<repo> --jq .license.spdx_id), so no license constraint justifies building. [Maintainer claim] [Verified] [Inference] (stack/licenses.tsv:61 mlc-ai/xgrammar Apache-2.0 permissive; stack/licenses.tsv:19 dottxt-ai/outlines Apache-2.0 permissive; stack/licenses.tsv:99 vllm-project/vllm Apache-2.0 permissive; stack/licenses.tsv:30 ggml-org/llama.cpp MIT permissive; stack/licenses.tsv:66 noamgat/lm-format-enforcer MIT permissive; packets/franken_nlp-assessment.md:59 Why a rewrite, not a wrapper: the stated bet is specialization; packets/franken_nlp-assessment.md:269 llama.cpp already ships grammar-constrained generation (GBNF), so the grammar half of the appliance lane exists upstream; packets/franken_nlp-assessment.md:91 the construction guarantee is a design property, not an observed one)

Where FrankenSuite touches this

  • franken_nlp: a 5,989-line grammar-constrained task layer designed to compile JSON Schema into a bounded automaton over the tokenizer's vocabulary, with source-grounded verbatim fields; no decode path executes yet; TRL 2-3, NODUS ring Monitor. [Verified] [Maintainer claim] [Inference] (packets/franken_nlp-assessment.md:91 Grammar-constrained task layer: schema-valid-by-construction JSON; packets/franken_nlp-assessment.md:131 JSON-Schema compiles to a bounded automaton over a vocab byte-trie; packets/franken_nlp-assessment.md:18 TRL: 2–3; NODUS ring: Monitor)
  • No other FrankenSuite packet ships a constraint engine; franken_nlp's layer is part of one model's runtime, not a reusable library. [Inference] (packets/franken_nlp-assessment.md:17 built around exactly one model)

What we cannot say

Revisit when

  • Any engine or front end publishes a generative mask-equivalence fuzzer, or a shared cross-engine conformance corpus appears.
  • An engine commits a per-token overhead budget backed by its nightly benchmark.
  • franken_nlp executes its grammar-constrained decode path.

Sources

Starting a build here?

Begin from the pickup companion for Structured output and constrained decoding: it carries the charter seed, the oracle candidates and how to verify them, and the first claims to register, so a new project starts from evidence instead of from scratch.

Help fix it

Each opens a GitHub issue form with this page’s address filled in. You need a GitHub account.

Made by Joshua Nowak (ZestStream) with AI coding agents. Not affiliated with or funded by Jeffrey Emanuel; Joshua uses his public tools and is a paying subscriber to jeffreys-skills.md.

Source on GitHub MIT licensed