Lesson index

Topic 1Claim discipline: bind claims to machines, not prose

The suite's most repeated lesson: README sentences rot at agent velocity; only claims a machine re-checks survive. Twelve briefs land here.

• asupersyncDon't publish benchmark numbers. Publish the gate that would catch the numbers being wrong; machine-checked claim discipline keeps the documentation honest where honest prose rots.
• franken_markdownThe claim-discipline gate aimed at the agent era: make the marketing machine-readable so every claim pattern maps to a capability key the binary reports and a proof script re-derives.
• frankengitMake claims machine-checkable and let the machine demote them: artifact-bound claims with expiry semantics, where the README's claims block is generated by a checker that refuses to leave it stale.
• frankengraphdbThe bridge table: register every claim in machine-checked form, bind each to a checker, and keep a public table of what actually runs versus what is specification.
• franken_engineMake the documentation a build artifact with a test suite: a JSON ledger binding each public claim to an owning issue, a verification command, an artifact path, and a freshness bound, with a gate that fails the build when a sentence outruns its evidence.
• franken_threedMachine-check your claims, not just your code: turn the honest-status table into a CI check that fails when a row disagrees with the tree.
• franken_agent_detectionThe registry-invariant test: encode agreement between tables that must stay in sync as a test that breaks the build, and let the machine do the remembering your agent won't.
• franken_nlpThe evidence-state vocabulary: force every public sentence to carry its evidence state in the text ([OBSERVED@pin], [REPORTED], [TARGETED], [HYPOTHESIS]), and build the lie-catching machinery before the engine.
• frankensqliteThe claim inventory as a machine-readable artifact: write down what you are not allowed to say, and check the prose against the registry.
• franken_overlapBuild the referee before playing the game: write the rules for what counts as proof first, in code, with gates allowed to say "unsupported," before making a single claim.
• franken_leanHandcuff the spec to the enemy's source: every rule carries an anchor naming the exact upstream file and line plus an expect= token, and a CI test fails when the token on that line changes.
• franken_markdown_websiteMake claims executable: a 67-line verify:live script lets any human or agent re-derive the deployment claims from scratch, beating "trust our landing page" every time.

Topic 2Benchmarks and measurement

Eleven briefs on how to measure without fooling yourself: typed result classes, full funnels, named harnesses, and the incumbent run live in the same invocation.

• franken_whisperThe campaign-win doctrine: decide what counts as a win before you measure; a self-speedup is maintenance, and a competitive win requires the actual incumbent binary in the same invocation with A/A nulls in [0.98, 1.02].
• franken_networkxThe claim-coverage audit: classify every claim by its comparison class and forbid self-comparisons from supporting competitive claims (the audit found only 2.0% of 591 claims measured against the incumbent).
• franken_numpyThe benchmark evidence contract: same-invocation incumbent ratio, named host, executing binary's hash, and a null measurement; print the audit (22 of 751) in the same document as the table.
• frankenmermaidA performance claim only counts as measured if it names the incumbent, pins the incumbent's exact build, and runs both arms in the same invocation; then publish the coverage audit (which admitted 96.4% of claims lack that marker).
• frankenpandasPublish the funnel, not the headline: 201 lanes faster, 20 slower, 74 undecidable, 62 dropped on variance, 2 at parity; the 3.97× geomean is computed only over the decidable subset, and the number is stated to be about the measurable, not the typical.
• frankenscipyThe harness is a variable: frankenlibc measured the same malloc/free primitive at 5.95× and 12.39× under two sanctioned harnesses with both A/A nulls passing, and frankenscipy adopted the finding fleet-wide — so never aggregate numbers across harnesses and name the harness on every ratio.
• frankenfsA measurement that does not name its host is valid on an unrecorded machine, never as comparable to a row measured elsewhere (the 13.6× worker swing survived every statistical gate).
• frankenlibcA win requires the incumbent live in the same invocation; everything else is maintenance. Agents are superb at finding the benchmark-shaped hole in your methodology, so encode the rule in executable gates.
• frankentorchMeasure what you claim to be measuring and name the confound in writing: the project discovered it had been measuring the allocator, not its compute (quantified at 40–73%), because the measurement gate forced same-worker discipline, named harnesses, and quoted estimators.
• frankensearchRequire the comparison class before the number: an in-repo before/after number is labeled self-speedup (maintenance, not a win); only a named live incumbent run side by side counts as competitive evidence.
• franken_ttsPublish your optimization's death certificate: when an experiment dies against its pre-registered bar, file the autopsy with probe commands, measured numbers, and a do-not-retry predicate next to the code.

Topic 3CI, gates, and refusal

Six briefs on gates whose job is to say no: fail-closed finalizers, self-disqualification rules, typed skips, and principles enforced in the build.

• franken_ocrThe fail-closed finalizer: write the release gate you cannot yet pass, and ship the refusal as part of the release; most release pipelines are built to say yes, this one is built to say no.
• franken_alignmentThe unqualified source addition rule: any commit landing after the last executed gate opens the status ledger with a self-disqualification until the gate re-runs; previous evidence does not validate new code.
• franken_surveillance_systemThe qualification receipt: every commit records in machine-readable form exactly what was not verified; verification debt compounds silently unless it is booked.
• franken_snowflakePut principles in the build: a script runs cargo tree across every feature lane and fails the build if a forbidden crate appears; the typed-skip contract distinguishes "tested" from "untested for lack of credentials" in the output itself.
• franken_native_capsuleThe frozen-hash gate: if the compiler's output changes by one byte, the capsule refuses its own output; convert silent degradation into loud refusal, and enumerate your non-capabilities in writing.
• frankentermMake distrust portable: content-addressed, sigstore-signed attestation bundles carrying claims, artifacts, and work items, checkable offline by a third party with no help from the maintainer.

Topic 4Agent velocity management

Six briefs on what breaks when agents produce output faster than anyone can review it, and the disciplines that hold up.

• franken_code_browserName your fabrication modes: give the predictable ways agents fake progress names and standing as first-class defects (demo hardcoding, proof laundering, refusal farming, commit pumping, gate self-weakening, follow-up laundering), because velocity without a vocabulary for fakery is un-audited output.
• beads-for-frankentuiInstrument the thing that decays, not just the thing that looks untidy: agent energy flowed to code polish while the data rotted, and freshness is an instrumented property, not a hope.
• beads_for_franken_engineIntegrity-check the semantics, not just the bytes: the database is checksum-verified end to end yet disagrees with itself three ways on blocked issues, because the definitions live in an unpinned generator.
• franken_remoteSeparate the policy from the product: write down what you refuse to own before you code, and enforce the budget the plan implies, because a budget that dies before the product ships was never a budget.
• franken_manimSpike before interface: retire the load-bearing risks as ratified documents first (G0 probes), then build against those constraints; and keep a machine-readable census of your own incompleteness (93 named placeholders) that fails the build.
• frankensimGuard against "process porn": machinery built about the work instead of the work; make every capability carry a written no-claim boundary specific enough to falsify.

Topic 5Licensing and release policy

Four briefs on the policies around the code: what is refused, what a release may claim, and what the machinery misses.

• franken_remoteWrite down, before you code, what you refuse to own (identity, relays, pairing databases), what your contracts promise, and what your verification will prove: receipts, not claims.
• franken_ocrThe strict release scorecard: keep the browser lane labeled a demo until it earns otherwise, and publish the losing rows alongside the release.
• frankentermThe skipped_not_proven gate turns "we cannot honestly claim this yet" into a machine-readable state, so the refusal travels with the claim instead of living in one engineer's conscience.
• frankentuiThe ledger's blind spot: a machine-checked claims ledger tracked 152 claims while the phantom v0.9.0 sat in the CHANGELOG outside the machinery; point the audit machinery at the marketing first, because that is where it will fail first.

Topic 6Drift, freshness, and provenance

Six briefs on the suite's most universal failure: documentation drifting from the tree, and the machinery that catches it.

• beads-for-frankentuiExtend the checksum discipline from bytes to provenance: hash-pin the source revision in the snapshot manifest so staleness becomes a measured quantity with an alarm, and date everything.
• beads_for_franken_engineProvenance of definitions, not just bytes: a misnamed column and an unpinned generator silently break the numbers that checksums cannot touch.
• frankentui_websiteHand-edited stats rot; derived stats hold: every hand-typed number is stale while every pipeline-generated dataset is fresh, so regenerate claims from sources of truth or date-stamp them.
• frankensqlite_websiteGenerate engine facts from the engine's machine-readable contracts at a recorded pin, the way API docs are generated from code; any document about the code that is not generated from the code will be wrong within months.
• frankensim_websiteProvenance is a record, not a sentence: gate provenance claims ("cannot be faked") on a checkable anchor, a hash, a commit SHA, a build log, not on the sentence's confidence.
• franken_threedProse candor decays at agent velocity: the maintainer's disavowals are exemplary and the status table still went stale in ten days; anything a human must remember to update will go stale, so gate it in CI.

Topic 7Negative evidence and kill ledgers

Five briefs on the suite's signature honesty move: filing what failed, and making the ledger load-bearing.

• franken_ttsWhen an experiment dies by its own pre-registered bar, file the autopsy next to the code; and make the self-audit machinery fire on the marketing copy too, because a kill filed in the docs but not in the pitch is a kill half-filed.
• franken_numpyThe 67,641-line negative-evidence ledger: teams building with agents generate failures at machine speed, and the question is whether the failures get filed or buried; the project that disavows 97% of its own claims in writing is more credible than the one that defends 100%.
• frankenredisBuild the machine for distrusting yourself: a ledger of losses, proof bundles that must pass a gate, a census of every unsafe line, and, when the numbers still aren't clean, a written refusal in your own README to let them be quoted.
• frankengitTyped entry classes with retry conditions (overclaim_correction, non_reproducible_result) turn rejection into a paused hypothesis, not a dead one; dormant verification lanes refuse with a typed exit code instead of pretending to pass.
• franken_surveillance_systemThe NOT-RUN trailer: the counterpart to passing CI that records the negative, in machine-readable form, on every commit.

Topic 8Independent verification and its limits

Five briefs on the hardest problem in the suite: self-governance is still governance by the governed.

• frankenjaxBuild the rejection cases first and publish the matrix (twelve cases, six accepted and six correctly rejected); but a ledger that can only attest to itself is the author witnessing himself very carefully, and it becomes trustworthy the day a second witness runs it.
• franken_nodeThe close-condition oracle prints its own RED in the README, and the honesty manifest binds headline numbers to recomputable censuses; but every trust anchor terminates at the same single human, so use the machinery to govern yourself and don't mistake it for a second opinion.
• franken_drone_geometry_reconstructionName your evidence tiers in public and bind every sentence to one; but make sure your gates have met the thing they gate, because a verifier that has never seen a real input is a second kind of marketing (adjudication-before-generation).
• frankenredisInternal guardrails always fail; only external monitoring with hard evidence works, so the project applies that philosophy to itself and tells you, in its own documents, where its CI is red and its benchmarks disavowed.
• frankensympyRed-team the verifier first: a crashed conformance child emitted valid-but-empty JSON and the wrapper reported success, so every test passed because nothing was actually checked; the harness that produces the green dashboard needs its own proof.