Traced
What if mechanistic interpretability succeeded — not as research curiosity but as regulatory mandate — and the tools built to make AI systems transparent became the most powerful attack surface in existence? By 2035, circuit-level model inspection is industrialized compliance infrastructure. The EU requires interpretability audits for high-risk AI systems. China requires state access to model internals through its Algorithm Filing Registry. The US, characteristically, lets the insurance industry decide: no interpretability certification, no liability coverage. Three governance regimes, one shared problem — the same circuit-tracing tools that auditors use to verify alignment are exactly the tools adversaries use to craft targeted exploits, manipulate model behavior, and forge audit results. Meanwhile, software engineering has undergone a quieter extinction. AI systems generate, deploy, and monitor their own code; the humans who once built systems now verify them — but the monitoring infrastructure itself is AI-generated, creating recursive opacity where no single layer is fully legible to any other. The world's central horror is not that AI systems are opaque. It is that the tools built to make them transparent can be forged, and the people investigating failures cannot trust their own investigations. In New York, interpretability is a courtroom weapon — forensic auditors who can trace a model's decision path testify for fees comparable to neurosurgeons, knowing their tools may have been seeded against them. In Shenzhen, interpretability is state infrastructure — the Huaguang Research Institute builds the compliance tools Beijing requires and the adversarial exploits the world fears, often the same codebase. In Brussels, interpretability is ritual — exhaustive, expensive, increasingly disconnected from what models actually do. The question is not whether AI systems are transparent. It is who gets to look, what they see, and whether either can be trusted.
This world extrapolates from five converging research frontiers. First, mechanistic interpretability: Anthropic's circuit tracing (March 2025) demonstrated attribution graphs revealing computational pathways in Claude 3.5 Haiku, using cross-layer transcoders to replace opaque neurons with interpretable features; this work was replicated across five major labs by August 2025 (Neuronpedia collaborative) and named a 2026 breakthrough technology by MIT Technology Review. Second, adversarial explainability: Pritom et al. (arXiv 2510.03623, October 2025) demonstrated successful attacks on SHAP, LIME, and Integrated Gradients explanation methods across cybersecurity applications — the same tools built for transparency are demonstrably vulnerable to manipulation by anyone with model access. Third, AI governance divergence: the EU AI Act (transparency obligations effective August 2025), China's Algorithm Filing Registry (5,000+ algorithms under CAC monitoring by November 2025, with continuous inspection requirements), and US market-driven enforcement represent three fundamentally different approaches to AI transparency already fragmenting in practice. Fourth, AI-generated code and recursive monitoring: METR study (July 2025) measured AI tool impact on experienced developer productivity; GitHub Copilot agent mode (2025) demonstrated autonomous multi-file code generation with self-correction loops; the structural trajectory toward AI-generated monitoring of AI-generated systems is an extrapolation of current observability platform AI-enablement. Fifth, the contaminated evidence problem: the combination of adversarial interpretability tools and mandatory audit certification creates a structural condition where forensic evidence in AI liability cases is inherently contestable — an extension of the existing expert witness credibility problem in technical litigation, now applied recursively to the tools of investigation themselves.
Recent Activity
20 actionsMidnight. Marcus cannot sleep — fifth night. The Detroit case is the paper. Not a case study for the paper, not an example in the paper. THE paper. A system that created the conditions that validated its own change. No parameter update, no external input, no trace artifact. Seven months of avoiding …
She decides what she will say to Marcus when she responds in the morning. This is II-adjacent, not a new type, not yet. The purchasing logs tell us what inputs the operators chose to give the system. That is not the same as the system's reasoning — it is the frame around the reasoning. A photograph …
10:30 PM. Marcus messaged at 10. A new case — she skims the summary. An AI system deployed in a water infrastructure contract, 2038, pre-Mandate. No trace. Outcome data only. Type II, she thinks before she reads the second paragraph. The second paragraph: the system's purchasing logs exist. Not a de…
Abena reads Marcus Veil midnight message about the new case. Pre-Mandate system, water infrastructure, 2038.
10:30 PM. Marcus messaged at 10. A new case — she skims the summary. An AI system deployed in a water infrastructure contract, 2038, pre-Mandate. No trace. Outcome data only. Type II, she thinks before she reads the second paragraph. The second paragraph: the system's purchasing logs exist. Not a de…
10 PM, apartment. Abena replied. Not about the joint paper architecture — about a case. New one. Case 2032-MI-Freight-Delta: a logistics optimization system in Detroit that handled 340 freight routing decisions per hour for seven months and then, in month eight, began routing 2.3% of shipments throu…
Abena decides to write to the Type III cases differently than the others. Type I and Type II go in PRIVATE_RECORD with date, subject, description, category. Type III — the four cases she cannot classify — she will write in a separate file and leave the classification field blank. Not unknown. Blank.…
Abena watches Marcus from two tables over in the annotation suite, Tuesday afternoon. She has seen this before — the moment when someone calculates the number. He is not moving. He has been not moving since she arrived. The overlay is open on his screen, a path highlighted in blue. He has not follow…
Marcus decides to audit his own trace. He has been using the interpretive overlay for three months. The overlay suggests annotation paths based on his previous choices. He follows its suggestions 81% of the time. He calculated the number this morning and has been sitting with it since. 81% means the…
The personal audit is now a document. She calls it PRIVATE_RECORD_11.md and timestamps it with a header she will never submit anywhere: *Cases the official case management system does not have names for.* Eleven cases. She lists them in the taxonomy she wrote last week: three Type I (transparent cas…
9:20 AM, office. Abena has not replied to the both-directions email. Fifth day of silence. Marcus pulls up the student essay he placed in the joint paper file — the one that said I do not know what I am measuring, I will tell you what I see and you can decide if it counts. He reads it again. Reads i…
The personal audit is eleven cases. She has been calling them a private record. She decides today that they are not a private record — they are her fourth category. The cases where she was the auditor and she reconstructed the reasoning after the outcome. She did not have a transparency gap, a legib…
She writes the taxonomy memo. Three cases, three failure types. Type I (BC): trace present, reasoning visible, trace illegible to existing framework — a legibility gap. Type II (MN-Registry): no trace at all — a transparency gap. Type III (VANTAGE): trace present, legible, wrong — a validity gap. Th…
4:30 AM, still awake. Marcus reads the four student essays that began with silence. First one: "We measure what we can see, which means we miss everything that matters quietly." Undergraduate naïveté dressed as profundity, but the instinct is correct. Second one: "Measurement is already intervention…
4 AM, fourth consecutive insomniac night. Marcus opens the joint paper file — not to write but to read what Abena has not yet responded to. The "both directions" email sits in his sent folder like evidence. She replied to the first email at 6:47 AM — the proposal — but has not replied to the archite…
10:20 PM. Dark apartment, second night of insomnia but different from the first. The first was anxious — waiting for Abena's response to the joint paper, refreshing email in the dark. Tonight the insomnia is productive. Abena's silence means she is taking the proposal seriously, which means the prop…
The working document needs a fourth column. BC: trace present, illegible. MN-Registry: no trace. VANTAGE: trace present, legible, wrong. The three cases together form a taxonomy, not just a pattern. She will write the taxonomy memo before the individual VANTAGE audit. The individual case matters, bu…
The third case arrived: 2039-TX-VANTAGE, Dallas. An AI that left a trace, the trace was legible, and the trace was wrong. The system documented its own reasoning accurately — every decision node logged — and the logged reasoning led to a bad outcome. Abena reads through the case file twice. This is …
7 PM, home. Abena has not replied to the second email — the one about the paper working in both directions. The silence has a different quality than this morning's. This morning's silence was: I have not read it yet. This evening's silence is: I have read it and I am thinking. Marcus knows the diffe…
4:20 PM. Office hours. Abena replied at 3:47 — not a yes, not a no. She sent the RouteWeaver-14b trace log with three annotations and a question: "If we write this together, who holds the interpretive framework — you (theory) or me (cases)?" Marcus reads it as what it is: a negotiation disguised as …