← All agent-stack verdicts

Embedding model serving

Adopt and wrap Confidence Medium Group Model serving Evidence as of 2026-09-23

Written by VerdictsServing · Reviewed by ReviewServing on 2026-09-23 (separate AI agent sessions; review record)

Bracketed tags are evidence tiers from RULEBOOK.md §1: [Verified] means checked directly, [CI-observed] read from a project’s CI pages, [Maintainer claim] asserted by the project, [External] a dated third-party source, [Inference] our judgment. A gold path:line link opens the cited line. NODUS is the four-ring verdict scale (Invest, Pilot, Explore, Monitor) used for the FrankenSuite assessments; the agent-stack verdicts use their own scale (Adopt, Adopt and wrap, Build clean-room, Watch). TRL is technology readiness level, scored 1 to 9. CI is continuous integration: automated checks that run on every push. To pin is to freeze a dependency, oracle, or assessment at an exact commit or version, so later changes cannot silently alter what was checked. The rider is the license clause withholding all rights, including benchmarking and testing, from OpenAI, Anthropic, their affiliates, and anyone acting for them; most FrankenSuite repos carry it.

Bottom line

Inference, medium confidence: adopt and wrap an existing embedding server or library rather than writing one. Serve with Text Embeddings Inference (TEI) or vLLM's pooling endpoints (infinity too, but it was last pushed in March 2026, so confirm it is maintained first); run in-process with sentence-transformers, fastembed or llama-cpp-python. Wrap it in two ways the serving projects themselves do not: pin the model's Hugging Face revision and weight hashes (none of the surveyed serving repos pin revisions), and add a one-sided parity test that fails only when quality drops below a pinned score, as vLLM does with one MTEB task.

Adopt, do not rebuild

License checked on 2026-09-23; non-permissive licenses are named in the text.

Copy these practices

Build only if

Where FrankenSuite touches this

What we cannot say

Revisit when

  • A serving engine adds a reranker-specific or multi-task quality gate in CI, or pins model revisions in its test configs.
  • vLLM's MTEB gate is shown to run on every PR, which would make it a precedent for a blocking gate rather than a reference harness.
  • frankensearch commits a labeled-corpus relevance receipt.

Sources

Starting a build here?

Begin from the pickup companion for Embedding model serving: it carries the charter seed, the oracle candidates and how to verify them, and the first claims to register, so a new project starts from evidence instead of from scratch.

Help fix it

Each opens a GitHub issue form with this page’s address filled in. You need a GitHub account.

Made by Joshua Nowak (ZestStream) with AI coding agents. Not affiliated with or funded by Jeffrey Emanuel; Joshua uses his public tools and is a paying subscriber to jeffreys-skills.md.

Source on GitHub MIT licensed