FrankenSuite research program · A shareable brief

FrankenSymPy A Computer Algebra System That Refuses to Claim Anything

A shareable brief from the FrankenSuite research program. Every claim below traces to the assessed repository; sources are linked at the end. Assessment pinned September 22, 2026.

Assessment pinned September 22, 2026 (the assessed commit)·~1,301 words·5 sections·Every claim traceable to a source
Six definitions. NODUS is the program’s four-ring verdict scale (Invest, Pilot, Explore, Monitor) assigned per assessment packet. TRL is technology readiness level, scored 1 to 9 per packet. CI is continuous integration: the project’s automated test runs. The pin is the exact commit the assessment froze on; every claim is evaluated at the pin, not at HEAD. The rider is the license clause withholding all rights, including benchmarking and analysis, from OpenAI, Anthropic, their affiliates, and anyone acting for them. Bus factor is how many people can leave before the project stalls; it is 1 for every repo here.

01 · What it is

What SymPy is, and why it matters

SymPy is an open-source Python library for symbolic mathematics. Where NumPy crunches numbers approximately, SymPy does algebra the way you would on paper: it simplifies expressions, factors polynomials, differentiates, integrates, and solves equations exactly, keeping symbols as symbols. It can render its answers as LaTeX for papers and notebooks.

Written entirely in Python, BSD-licensed, and in development since 2005, it is the standard open-source computer algebra system in scientific Python. It is the symbolic layer people reach for when the math has to be exact, not estimated.

Its exactness is also its cost. Symbolic computation means manipulating expression trees and big integers at the Python level, on hot paths, and that is slow. Its object model is deeply dynamic, which makes formal reasoning about results nearly impossible. And its answers carry no machine-checkable evidence: a factorization is a claim, not a certificate. If you want to trust a SymPy result inside an automated pipeline, with no human in the loop, you are trusting the whole interpreter. That is the gap FrankenSymPy aims at.

02 · Why build it

Why rewrite it in Rust?

The maintainer states the case through the architecture itself rather than a single manifesto, in three parts. First, performance: a native Rust engine avoids Python-level bignum and tree manipulation on hot paths. Second, memory safety without a foreign-function interface to C. Third, and this is the differentiator, proof-carrying algorithms: factorization, integration, and solving race multiple strategies internally, but only an independent verifier can promote a candidate result to an accepted one. A factorization would arrive with a certificate, not just an answer.

The design is dual-lane: a Python compatibility shell preserves SymPy's object model so existing habits transfer, while eligible expression regions lower into a deterministic native kernel on a content-addressed term DAG. Only a clean room allows re-deriving every representation instead of inheriting SymPy's object model, which the design treats as a compatibility surface rather than a foundation.

The counter-arguments. The "fast SymPy" lane has a precedent: SymEngine, a standalone fast C++ symbolic library with thin Python wrappers, has existed for years and never displaced SymPy. The mechanism matters. SymPy's moat is ecosystem lock-in (notebooks, downstream packages, two decades of accumulated quirks and answers), not speed, so a faster engine aimed at the wrong target still loses. And the demand side is unproven: whether "agents need certified symbolic results" is real demand or maintainer narrative is asserted nowhere in the tree. The technical case is written down; the human case is absent, and this brief will not invent it.

03 · What was built

What the project actually built

This is the youngest repository in the program: created August 20, 2026 and 33 days old at assessment, and the velocity is startling: 97,062 lines of Rust across 29 crates. The substance is real in the places that are hardest to fake. There is a full Zassenhaus factorization pipeline: Cantor–Zassenhaus splitting, Hensel lifting, recombination, multivariate support. There is an independent proof kernel with verifier entry points, and the dependency direction is constitutional: generator crates may depend on the kernel, never the reverse. There is exact integer, rational, and modular arithmetic, a semantic term DAG with BLAKE3 content-addressed interning, bounded integration and limits that refuse cleanly outside their scope, an asupersync-based portfolio runtime with cancellation and replay, a PyO3 bridge, and a 78-file Python compatibility shell. The unsafe count for the entire tree is zero, enforced at the crate level.

More interesting than the code is the self-audit machinery, which is unusually developed for the project's age:

  • A machine-readable claims registry. Twenty-seven claims, each with a status and a gate. Fifteen are marked "implemented but uncertified," eleven "planned," one "documented." Zero are validated. Zero are certified. The registry's defaults forbid present-tense claims until gates pass, and the repo's agent instructions bind contributing agents to it.
  • An adversarial self-audit. The September 8 revised audit reads like a red-team report against its own author. Its P0 finding: the conformance harness could fail open. A crashed child process emitting valid-but-empty JSON made the wrapper exit 0, reporting success on nothing. The hole was fixed within two weeks.
  • A paired benchmark with the error case left in. It diffs the engine against a pinned SymPy 1.14.0 oracle, and its most revealing line is the one error case: both engines hit a resource bound and said so, in opposite ways, and the report kept the case in the outcome mix instead of scoring around it.

Evidence · CI status

What the project’s own CI said at the pin

the pin = the commit the assessment was pinned to.

  • The build-and-test CI (continuous integration: automated checks that run on every change) gate has failed 786 consecutive runs; the last green was August 23.
  • At the pinned commit the pipeline fails at the cargo fmt formatting gate, so the 1,127 counted unit tests never execute.

The uncomfortable findings

Uncomfortable finding

Meanwhile the README claims "green workspace tests," a line that has been stale for a month. source source The much-cited 230/230 fixture admission against the SymPy oracle is construction-only: the project's own exact-surface diagnostic found 229 of 230 fixtures differing on method resolution order and pickling. The registry's "materially outperforms" performance claim sits above a paired report whose geometric mean favors SymPy. A fresh clone does not build at all. It needs three sibling repositories checked out at compatible pins, and only one pin is recorded. Four import sympy delegation sites in the development shell quietly hand calculus back to the incumbent the project aims to replace. There are no tags and no releases; version = "0.1.0" in a manifest with publishing disabled is a version string, not a release. And the license rider bars OpenAI and Anthropic, the two organizations best equipped to evaluate the work, from even benchmarking or analyzing it.

04 · Where it stands

Where it stands

In our program's terms: TRL 3–4, NODUS ring Explore. In plain English: components demonstrated in a lab and self-audit setting, nothing certified, nothing released, nothing running in an operational environment, and the front gate of its own CI has been red for a month. The codebase is admirably honest about all of that, and very far from shippable.

Technology readiness

TRL 3–4 on a 1-to-9 scale.

NODUS ring

MonitorExplorePilotInvest

The program’s adoption ring for this project.

Verdict key

NODUS ring
Explore: The default for substantive-but-unproven work.
TRL
Technology Readiness Level, a 1-to-9 scale of how proven a technology is. This brief rates the project at TRL 3–4.
CI
Continuous integration: the project’s automated checks, run on every change.
The pin
The exact commit the assessment was pinned to. “At the pin” means as of that commit.

Evidence tiers

  1. Tier 1 · Verified — Confirmed by direct inspection of a fresh clone, an API response, or a live page read by the analyst.
  2. Tier 2 · CI-observed — Observed executing on live CI pages. Attests the suite runs, not that it is green, unless pass/fail is legible.
  3. Tier 3 · Maintainer claim — Asserted in README or docs by the maintainer; not independently executed or reproduced.
  4. Tier 4 · External — Independent sources: APIs, papers, press, third-party benchmarks. Absence of coverage is reported as a finding.
  5. Tier 5 · Inference — The analyst’s judgment. Always labeled; never presented as fact.

What this verdict rests on. Tier labels are the assessment packet’s own annotations.

  • Tier 1 · Verified: Counts from the clone.
  • Tier 2 · CI-observed: Run #833 at the pin failed at cargo fmt --check with unit tests skipped; a 786-run red streak at the conclusion level.
  • Tier 3 · Maintainer claim: Nothing was compiled or executed; the SymPy oracle lane was never run.
  • Tier 4 · External: No independent coverage.
  • Tier 5 · Inference: The Explore ring at TRL 3–4.

Should you use it?

No.

There is no release artifact, no certified compatibility profile, and a fresh clone does not build.

Should you learn from it?

Yes.

Unusually, you can learn from it today, before the algebra system ever ships. The claims-registry pattern, the paired-benchmark admission discipline, and the adversarial self-audit format are all exportable now. They are the project's most durable output.

What would change the verdict, in order

  1. a green CI run (running the formatter is the cheapest ring-moving action available), any claim moving from "implemented but uncertified" to validated, a committed regression test for the harness fix, a tagged release, a narrowed license rider
  2. Until then it stays exactly where its own ledger puts it: substantive, uncertified, Explore
source

05 · What it teaches

What this teaches about building with agents

The transferable lesson is not about the algebra. It is about where the silent failures live.

The most dangerous bug found anywhere in this project was not in the mathematics. It was in the test harness: a crashed conformance child emitted valid-but-empty JSON, and the wrapper reported success. Every test passed because nothing was actually checked. The code being verified could have been anything; the verifier was the thing that was broken. The project caught it only because it ran an adversarial audit against its own evidence machinery. Then it fixed the hole, wrote it up, and published the write-up.

Agent-built systems arrive with green dashboards: tests passing, benchmarks run, claims neatly formatted. The instinct is to audit the output. The FrankenSymPy lesson is to red-team the verifier first: the wrapper, the oracle harness, the script that decides what counts as a pass. source A green dashboard is itself a claim, and the harness that produces it needs its own proof. The teams that internalize this will spend an uncomfortable fraction of their effort testing their tests. It is the most important testing they can do, because a broken verifier fails silently and a broken verifier that reports success is worse than no verifier at all.

Evidence

This page cites selected evidence. The full claim-by-claim audit is in the assessment packet.

Open a panel to see what each source evidences. Full claim-by-claim audit.

Full claim-by-claim audit with evidence tiers lives in the program's assessment packet.

The repositorygithub.com/Dicklesworthstone/frankensympy
README

its README

registries/claims.toml

LICENSE, claims registry (registries/claims.toml), adversarial self-audit (artifacts/audit/reality-check-2026-09-07.md), paired benchmark report, and CI run data at the assessed commit

READMEWikipedia

SymPy's history and license via Wikipedia and the SymPy project

SymEngine via its public repository.

SymEngine via its public repository.