Three Repositories, One Framework: What Recent Agent Projects Add to TRACE

Theory goes stale when it is checked only against one’s own papers. This week I examined the documentation and code of three open repositories—from Robert “Uncle Bob” Martin, HKUDS, and DeepSeek—through the lens of TRACE and Runtime Assurance Contracts (RAC). Each team has addressed an important part of agent engineering: authorization at intake, strategy evaluation, or output verification. Each also shows where an individual mechanism stops being sufficient.

Why I looked at other people’s code

When I posted the TRACE preprint six months ago, followed by a paper on runtime assurance contracts (RAC), the central claim was direct: a high aggregate score does not authorize an agent to act. If a required check returns “no” or $\bot$ (UNDETERMINED, meaning that the check could not reach a sufficient result), a model’s high score cannot make up for that gap.

TRACE organizes this idea into four layers:

• L1: a deterministic core, such as parsers, compilers, physical limits, and environment hooks;
• L2a / L2b: specialized machine-learning models and isolated LLM validators;
• L3: stateful orchestration, planning, and routing;
• L4: a human operator whose workload and oversight are measured.

A useful test of this decomposition is to examine systems built by other engineering teams for related problems. As of early November 2026, three open repositories had attracted more than 36,000 GitHub stars combined:

1. unclebob/ub-mode (~7.3k stars), an agent skill and set of command-line tools by Robert Martin.
2. HKUDS/AgentPilot (~8k stars), an iterative program-improvement loop that updates external policy state without changing model weights.
3. deepseek-ai/dsh-superagent-prototype (~21k stars), a research prototype for a background agent coordinator from the DeepSeek Harness team.

I reviewed their documentation and key code modules. The sections below describe the mechanisms, their limits in high-risk settings, and what the review has contributed to the TRACE ecosystem. I distinguish what is already implemented in the tracegate runtime gate from what has been specified or proposed for future work.

1. Uncle Bob turns off the “swarm”: unclebob/ub-mode

In spring 2026, Robert Martin experimented with swarm-forge, a heavyweight multi-agent orchestration system for Claude Code. By autumn, he had reconsidered whether most tasks need tightly managed swarms. The compact ub-mode repository instead combines a SKILL.md instruction file with three Clojure command-line utilities: crapper, dryer, and mutator.

Martin’s logic is straightforward: a dozen agents arguing in a chat consume tokens and blur accountability, while one agent working within a set of fast, deterministic checks can be cleaner and cheaper. In TRACE terms, this is a move toward Model Parsimony: where a deterministic L1 analyzer is sufficient, a multi-agent L3 system adds overhead.

The three tools work as follows:

• crapper calculates the CRAP (Change Risk Anti-Patterns) score for each function, combining cyclomatic complexity $C(m)$ with test coverage $\text{cov}(m) \in [0,1]$: $$\text{CRAP}(m) = C(m)^2 \cdot (1 - \text{cov}(m))^3 + C(m)$$ The ub-mode threshold is $\text{CRAP}(m) < 10$. A function with complexity $C=8$ and no coverage has a CRAP score of 72. At 70% coverage, its score is about 9.73, below the threshold. At complexity $C=10$, even 100% coverage yields a score of 10; tests alone cannot bring it below the strict threshold. The function must first be simplified.
• dryer detects structural code duplication and requires duplicates with similarity of at least 0.8 to be removed.
• mutator runs mutation testing and requires all surviving mutants to be killed.

Where does this approach reach its limits?

The first limit concerns separation of authority. ub-mode itself is an instruction in a system prompt (SKILL.md). The agent decides when to run the tools, interprets their output, and reports in its own Action Log that the checks passed. In RAC terms, this raises concerns under Invariant I1 (the governed policy cannot authorize itself) and the self-reporting vulnerability C8. If the agent skips a tool or cuts a corner when its context is full, the skill itself contains no external controller that will block the action.

Grady Booch identified another limit in a public discussion with Martin: code can have full test coverage, a CRAP score below the threshold, and no surviving mutants, yet still fail to meet a domain requirement.

This also exposes a limit of RAC. The three-valued logic ${1,0,\bot}$ cannot automatically detect a requirement missing from the contract. The value $\bot$ (UNDETERMINED) applies when a check specified by the contract cannot run or cannot produce a sufficient result—for example, when a coverage report is missing, an AST cannot be parsed, or an input falls outside an invariant’s defined scope. In that case the gate withholds automated authorization and routes the case to a human. But a requirement omitted from the specification remains invisible to crapper and to an automated RAC gate. Three-valued logic prevents a failed or incomplete specified measurement from being mistaken for success; an engineer at L4 still has to address specification completeness.

Status in TRACE and tracegate:

• Already implemented: validators run outside the agent prompt; INV-COVERAGE is enforced in tracegate/envelope/invariants/coverage.py; and mutation discrimination checks test sensitivity for changed symbols in tracegate/validation/mutation.py.
• Specified after this review: a configurable CRAP invariant has been specified for integration into the external tracegate gate. It can use the existing changed-AST-symbol parsing and .tracegate/coverage.json report without adding dependencies. If coverage data for a changed file is missing, the proposed check returns UNDETERMINED. This remains a specification, not an implemented feature.

2. What actually learns in HKUDS/AgentPilot

The second repository is HKUDS/AgentPilot, which includes the RSI Arena evaluation protocol. Its README leads with the claim “9B Strategy Model Beats 284B” on AIME and Terminal-Bench 2.1.

A static review of the evaluation documentation and code reveals an important distinction between the headline and the implementation.

First, the model weights remain frozen: there is no gradient update, LoRA, or RL optimizer. What the authors call learning takes place at L3. Between batches, the system updates external policy state, either through KL-UCB bandit statistics (policy/ucb/parent.py) or through textual policy memory that the language model adds to, revises, and retires.

Second, the formal-run code does not match the experiment documentation. arena/docs/evaluation.md names Qwen 3.5 9B as proposer and DeepSeek V4 Flash as worker. The research_setup.py and formal.py files instead specify qwen/qwen3.8-27b (reasoning = xhigh) as proposer and z-ai/glm-5.3-flash as worker. The repository does not establish which configuration produced the published numbers.

The reported gains—+8/36 on AIME and +12/49 on Terminal-Bench 2.1—come from one run and one final evaluation. Per-task paired outcomes and repetitions across runs are not reported. With a shared test split, the result of a paired test depends on how many tasks the final version won and lost. A net gain of eight is compatible with an 8–0 split ($p=0.008$) or a 12–4 split ($p=0.077$) under a two-sided exact sign test. The evidence therefore does not establish that the gains are stable. The doubled action budget (20 versus 10), the lower starting score on TB2.1 (22/49 versus 27/49), and the absence of an ablation separating KL-UCB from proposer size also make attribution difficult.

The repository nevertheless contains a carefully designed L3 decision layer. Four mechanisms stand out:

1. Cutoff-safe state (cutoff_safe_facts). SearchStateBuilder (search/state.py) builds state only from facts available at the decision point. Updates occur at batch barriers, and each choice records a DecisionTrace with deterministic tie-breaking (seeded_argmax).
2. Paired comparison with a shrunken lower bound. In evaluation/paired.py and evaluation/admission.py, each child is compared with its parent on the same validation tasks using: $$L = \hat{\Delta},\frac{n_{\mathrm{eff}}}{n_{\mathrm{eff}} + \kappa} - k \cdot \text{SE}(\hat{\Delta}), \qquad \kappa = 2,; k = 1$$
3. Separate admission and bandit reward. evaluation/admission.py uses a less stringent non-inferiority rule ($L \ge -\text{margin}$) to admit a neutral candidate to the pool for further search. evaluation/reward.py requires superiority ($L > \text{margin}$) before counting a branch as a KL-UCB success. Passing a sealed test split (EvidenceTier.SEALED_C) to the reward classifier raises an exception.
4. Skip semantics for infrastructure failures. classify_effectiveness in evaluation/reward.py distinguishes a weak idea from a worker failure, timeout, or boundary violation. Such a run is skipped and does not increment the Bernoulli observation count.

This last mechanism is structurally analogous to RAC’s three-valued gate logic, but the two are not equivalent. Both distinguish a negative result from the absence of a reliable result, yet they have different consequences. In AgentPilot, a skip controls how the search estimate is updated: an infrastructure failure does not count as a zero reward. Skips can still bias evaluation if their rate depends on the strategy—for example, if a more demanding algorithm times out more often. In RAC, $\bot$ (UNDETERMINED) controls authorization: an unreliable check result blocks permission to act.

The textual policy memory also needs a careful interpretation. The model adds, revises, and retires lessons that guide the selection of modification directions (D2) and actions (D3) in later iterations. Each lesson has a source_action_id and applicability conditions. In an offline benchmark, changing this search heuristic does not by itself change an authorization or safety rule. The concern in a high-risk system arises if unreviewed policy-memory changes alter permitted action boundaries or the conditions attached to an existing approval. The available review does not establish that this happens in AgentPilot.

A further open question is evaluator isolation. Worker writes are restricted to permitted roots (runtime/research_worker.py) and codex exec --sandbox workspace-write, but that mode does not block filesystem reads. If reference tests or verifiers are readable on the same host, protection against read-side leakage requires separate assessment.

Status in TRACE:

• Proposed for the TRACE × Prime Agent line: paired candidate promotion (non-inferiority for admission, superiority for promotion), barrier-based cutoff_safe_facts, and a type-level SEALED_C block. These are proposals in the research plan, not deployed features.
• Added to the L1 audit checklist: distinguish write isolation from read isolation in agent-worker environments.

3. When authorization is tied to a policy version: DeepSeek’s dsh-superagent-prototype

The third repository, deepseek-ai/dsh-superagent-prototype, is a TypeScript research prototype released by the DeepSeek Harness team alongside its essay “Exploring Super-agent Design: Towards a More Autonomous Agent System.” Its operating principle appears in docs/000-philosophy.md:

“Take as little human attention as possible, and give as much value as possible.”

This connects directly to operator attention at TRACE layer L4. How does the DeepSeek team try to conserve it?

First, incoming GitHub and Feishu events pass a deterministic authorization check in src/policy.ts before the router model runs. Irrelevant or unauthorized events are filtered without an LLM call. Second, proactive actions have three modes: off → observe → auto (docs/002-authorization.md and admitsInferred in src/policy.ts). In the default observe mode, the system records in SQLite what it would accept, but does not launch workers or edit code while the team reviews the rule’s behavior.

The most relevant mechanism is the authorization model in docs/001-domain.md, docs/002-authorization.md, src/policy.ts, and src/core/engine.ts. When an event passes the gate, the transactional engine creates a Ticket with an immutable Mandate that records:

• the originating identity;
• the permitted scope;
• the authorization strength;
• the active policy version.

When a deferred or suspended task resumes, revalidate in src/policy.ts checks its mandate against the current configuration (settingsPolicy). If the team has changed or weakened a rule, the old mandate is invalidated and the task cannot continue without new authorization. The agent cannot change its own mandate through prompt text or comments; the host engine enforces this boundary.

This design resembles the rule_version field in the RAC evidence record $e_t$ (Equation 3) and the version-bound authorization proposal in the Approval Must Expire policy-letter outline for ACM AI Letters. The DeepSeek team arrived at a related design while building coordination for background agents working on repositories. This is a useful independent example for comparison; it does not establish priority or prove that one framework is superior.

What are the prototype’s limits in a high-risk setting?

The project states a goal of conserving human attention, but does not quantify it. Its code does not calculate RBI (Review Burden Index, a TRACE L4 metric that combines handoffs per unit time with mean review time and compares reviewer load with oversight capacity), human override rate (OvR), panel signal-to-noise ratio (SNR), or router calibration error (CE). The transition from observe to auto is manual and has no statistical threshold over accumulated decisions. In addition, although the prototype has a clear intake control (Mandate), it does not place an output verification gate on worker-produced code. Worker sessions execute commands with the host OS user’s privileges (docs/003-execution.md).

Status in TRACE: the version-bound mandate is used as a comparative example in the Approval Must Expire policy-letter outline for ACM AI Letters and in the internal RAC mapping. It is evidence of a related engineering design, not a claim that TRACE originated the idea or that the prototype implements the full RAC contract.

Putting the three mechanisms along a task’s path

Placed in execution order, the repositories cover different parts of a task workflow and leave gaps between them.

The comparison shows that structural similarity between individual components does not itself provide safety guarantees. Even if these mechanisms were combined with an append-only evidence log backed by a hash chain, two limits would remain:

• A log does not enforce a decision. A hash chain can check the internal consistency of records. Detecting a complete rewrite with recomputed hashes requires an independently stored checkpoint or another storage protection. The log itself does not block an action; an external runtime gate and an environment hook must enforce that decision.
• Formal checks do not ensure a complete specification. Three-valued logic ${1,0,\bot}$ can stop the system when a specified checker cannot produce a reliable result. It cannot infer a domain requirement that an engineer left out of the contract. High-risk systems therefore still need L4: people with measured workload (RBI), override authority (OvR), and responsibility for the content of the rules.

The three detailed reviews, with links to their code and documentation, are available in the open TRACE library of related approaches. The working mappings remain in the project repository.

References and materials

1. TRACE framework: S. Zabolotnii, TRACE: A Metrologically-Grounded Engineering Framework for Trustworthy Agentic AI Systems in Operationally Critical Domains, arXiv:2605.03838.
2. Runtime Assurance Contracts: S. Zabolotnii, Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents, arXiv:2609.39717; code and mechanism corpus on GitHub and Zenodo.
3. TRACE library: Related engineering approaches.
4. Repositories reviewed: Robert C. Martin, ub-mode; HKUDS, AgentPilot; DeepSeek AI, dsh-superagent-prototype.