One message the spec forbids
Bug CU07 (test_runner.py:633). The gNB-CU is
the tower's control program. Its helper (the gNB-DU) must say hello first
(F1-Setup-Request); the CU then answers
(F1-Setup-Response). The bug: if a peer sends the answer without
the hello, the CU crashes (SIGSEGV, exit 139). One message takes the tower down.
- 1The DU connects to the CU.
- 2D → C: F1-Setup-Request — "hello."
- 3C → D: F1-Setup-Response — "hello back."
- 4Both keep running. Traffic flows.
- 1A peer connects to the CU.
- 2No hello. No F1-Setup-Request is sent.
- 3Peer → CU: an F1-Setup-Response anyway — a real one, played back.
- 4The CU crashes — 139 = 128 + 11 = SIGSEGV. Tower down.
- 1Check who is connecting. A peer the CU does not know and does not trust never gets in at all — no connection, nothing to exploit.
- 2Wrong message? Keep running, kick the peer. If an accepted peer skips the hello or sends anything unexpected, the CU refuses it (answer with F1-Setup-Failure), closes only that one connection, and stays healthy.
- 3Everyone else is untouched. Real DUs keep working, phones stay online — one bad peer costs one rejected connection, never the tower.
The purpose of the F1 Setup procedure is to exchange application level data needed for the gNB-DU and the gNB-CU to correctly interoperate on the F1 interface. This procedure shall be the first F1AP procedure triggered for the F1-C interface instance after a TNL association has become operational. — ETSI TS 138 473 V16.13.0, §8.2.3.1 (p. 25)
This message is sent by the gNB-CU to transfer information associated to an F1-C interface instance. Direction: gNB-CU → gNB-DU — ETSI TS 138 473 V16.13.0, §9.2.1.5 (p. 93) The Response may therefore only ever flow CU → DU. Arriving from a DU with no Request outstanding, it cannot exist in an honest session — a compliant CU discards it.
8.2.3.4 Abnormal Conditions Not applicable. — ETSI TS 138 473 V16.13.0, §8.2.3.4 (p. 28) The spec defines no recovery behaviour for a broken F1 Setup exchange — implementations get no guidance here, which is exactly where fuzzers hunt.
-- TS 38.473 v16.3.1 (the ASN.1 the target CU was compiled against) f1Setup F1AP-ELEMENTARY-PROCEDURE ::= { INITIATING MESSAGE F1SetupRequest -- DU → CU SUCCESSFUL OUTCOME F1SetupResponse -- CU → DU UNSUCCESSFUL OUTCOME F1SetupFailure -- CU → DU PROCEDURE CODE id-F1Setup CRITICALITY reject }
Inside an expert's head: turning the spec into a machine
How does a protocol expert actually perform the conversion?
The conversion, step by step — every reading, every decision, and the machine as it stands after each step
第一步就是： ASN.1是官方提供好的，OAI5G和srsRAN就是根据ASN.1直接写出来的。所以通过ASN.1专家建立基本的理解是很正常的。从ASN.1里面我们可以提取出3个基本的带有方向性的edge，request,response and failure。这里基本上没动别的脑子，纯依据官方文档。
第二步： ASN.1也不是全部，这就是为什么我们需要专家来看。在Setup_Requested之前，如果这时候有个新的connection打进来，但是还没有进行request/response的时候，用一个idle状态来代表，比较恰当。
第三步：现在把成功和失败的概念加进来，如果成功了，就表示complete，你可以看到上面的路线是request进来，response出去如果成功了就是complete，如果失败了就是SetupFailure,这时候可以再次尝试，或者把状态打到time_wait去，等一会再试试。
第 4 步：spec 说 abnormal conditions 不适用，等于承认没有任何救火措施。专家只能自己找规则把机器闭合但是实现怎么填这个空白，规范管不着，这就是可能会出bug的地方，因为每个开源软件开发者的意见不同。正确的 CU 应该 discard，我们的 CU 给了 139。
第 5 步：4 个东西要定好。1. 消息类型 2. 方向（第一步就定好的）3. Transaction-ID 的匹配 4. 把 ASN.1 的 presence 子句补全，必要的 IE 都要在。如果这 4 个里面任何一个没搞好，这条消息就应该被扔掉（discard + Error Indication），但机器停在原状态，在途的 transaction 继续等它真正的答案。
第 6 步：最主要的事是卫生检查，主要查 2 件事。1. 一个 procedure 只画一台机器——这里只画 F1 Setup 这一台，别的 procedure 不许融进来。2. IE 永远是数据，只能作为标注骑在边上，不许有自己的状态——所以状态保持精简，不会爆炸。
- 经验的完整与正确：足够资深、或多名专家交叉，画出的 FSM 大体完整、正确。
- 语义压缩：知道哪些句子是法律（方向、criticality、presence），哪些只是说明。
- 面对沉默的推断：spec 不写的地方，能去 criticality 机制找规则把机器闭合。
- 边界感：一个 procedure 一台机器，IE 是数据。
- 脑容量有限：记忆错误（记错条款与 IE 必填，版本对不上）＋认知错误（没接触过的特殊情况，问了也想不到）。
- tradeoff 无原则记录：collapse 哪些状态、画不画 TIME_WAIT，做了选择不写为什么，无法审计、无法重放。
- 慢且贵：专家画一台机器要小时到天，LLM 出 4 个种子只要几分钟。
- 不可复现：专家的机器靠权威被相信，机器的机器靠执行被验证。
- 没有自我验证的 oracle：验证 FSM 完整性需要的正是这场 campaign 本身。
No expert: an LLM turns the spec into a raw seed FSM
The campaign never hired Stage 1's expert. The same spec feed (§8.2.3 +
§9.2.1.5 prose, plus the v16.3.1 ASN.1 with PRESENCE mandatory/optional
clauses — ~4 600 input tokens) was sent to glm-5.3-flash:cloud three times —
framings A and B change only the persona; C is our deliberate bad control — a noob's vague,
unorganized prompt, same spec feed, same pipeline. The LLM's whole job is the raw seed JSON
below, and nothing else: no refinement, no traces, no loop. Everything done to the seed afterwards
is deterministic post-processing and lives in Stage 3. Pick a framing — every number on
this page, from the seed here to the funnel, follows your choice.
The exact prompt sent to the model
After this prompt, the same spec feed was appended verbatim for all three framings — the §8.2.3 + §9.2.1.5 prose and the v16.3.1 ASN.1 (~19 KB ≈ 4 300 of the input tokens). What differs between framings is only the instruction you see above.
Enumerate the seed — framing A, nothing hidden
The three prompts side by side — how much of the developer's idea each one carries
All three were sent to the same model with the same spec feed appended, and they are nearly the same length (873 / 961 / 900 characters). This table asks a different question from the Conclusion's scoreboard: not which seed performed best, but which instruction fully conveys the developer's idea — everything the model needs to produce what the developer actually wanted, with nothing left to guess. Green = the idea is stated outright; red = the prompt leaves the model to guess, or states a different idea instead.
| what the developer needs conveyed | A — faithful modeler | B — formal-methods engineer | C — the noob |
|---|---|---|---|
| who the model is (persona) | "You are a protocol engineer building a conformance-test model of 3GPP F1AP" — role and its product in one clause. | "You are a formal-methods engineer converting a 3GPP specification" — a coherent role, coherently stated. | None — "i need one of those state machine things". The model must infer the role from the request. |
| what to build | "convert the F1 Setup procedure into a finite state machine" — the artifact, named exactly. | "…into a strict finite state machine" — the same artifact, plus an adjective that buys nothing downstream. | "one of those state machine things" — the artifact only half-named. |
| what it's for (the downstream consumer) | "for conformance testing" — the purpose that makes every later choice meaningful. | Never stated — rigor demanded for its own sake. | "this is for a fuzzer my friend is building" — ironically the campaign's true purpose, but hedged and drowned. |
| the core vocabulary (IE validity, the thing the pipeline reads) | "tag each message's Information Elements with validity ('valid' / 'invalid' / 'missing') from the message-definition tables" — the exact vocabulary the mutation pass will flip. | "be pedantic about (1) message DIRECTION, (2) IE PRESENCE, (3) the precondition state" — precise demands, but for fields no pipeline code ever reads. | Absent — no IE mentioned anywhere; the model was free to skip them, and it did (0 of 7 edges carry IE lists). |
| the output contract | Mandatory: exact JSON schema, "no prose outside the JSON". | The same mandatory contract, word for word. | "i think the format is something like … that should be enough right?" — a guess offered where a contract is needed. |
| scope | "at least the F1 Setup procedure and the UE-context procedures you can see in the spec." | The same scope sentence, word for word. | "the f1 setup part" … "all of them, whatever you find" — and then "dont spend too much effort on the details", which contradicts the "all of them" before it. |
| how much of the idea survives the prompt | The whole idea: who, what, why, in what format, checked how. Nothing the pipeline needs is left to the model's discretion. | A superset — the idea, plus the prompt author's own opinions about rigor. More instruction, not more idea. | The mood of the idea, none of its content — the model was handed the same spec feed and left to reconstruct the intent alone. |
Verdict: framing A expresses the developer's idea most fully. It is the only prompt of the three that states every component of the intent — persona, artifact, purpose, output format, IE vocabulary, scope — so that the model is never guessing what "good" looks like. B transmits a coherent but different idea (formal correctness the pipeline never reads); C shows that a prompt can be the same length and carry almost none of the intent. Which prompt performed best is a separate question — see the Conclusion.
我的解读
要提高 LLM 从 spec 提取 FSM 的质量，有两个方法。
方法一：拿经过特殊训练的 O-RAN Spec LLM。这条路我们做不到。一方面，现在 找不到这样的 LLM；另一方面，就算自己训练，从人力成本考虑也只能做小的 case，没有一个 LLM 可以 真正囊括所有的 O-RAN 知识。
方法二：提高自己提示词的精准度。这是本实验走的路。A、B、C 三个提示词最后 都找到了 CU07 这个 bug，说明发现本身不取决于提示词的好坏，扛住发现的是后面的 merge_pta、 max_paths 补丁和与种子无关的 mutation。
但 A 的方法最好。A 的提示词把开发者的意图说得最全：给了模型角色（protocol engineer），说清了目的（conformance testing），点名了下游真正会用的词汇（IE 的 valid / invalid / missing），规定了强制的 JSON 输出，划了范围（F1 Setup 加 UE-context procedures），指了取材来源 （message-definition tables）。模型没有一件事需要靠猜，所以 A 的种子每条边都带方向和完整的 IE 列表，经得起检查。C 同样找到了 bug，但它的种子里有把本地定时器当消息的伪迁移，没有 IE 列表， 只是碰对了答案，种子经不起检查。所以 A 好在产出的模型可信、可检查，不是好在找到更多。
Post-processing: merge_pta grows the seed
Nothing in this stage involves the LLM. The seed JSON from Stage 2 is finished model output — merge_pta is not built into the prompt and not part of the generation; it is deterministic Python applied afterwards, and it is where the model's raw machine meets reality for the first time.
Same three framings as Stage 2 — switching here switches the whole page, and every number below follows your choice.
Step 1 — the shrink: 11 edges go in, 7 fit
Before merge_pta can run, the seed has to fit ARCANE's
container — and it does not, fully. The LLM wrote its transitions as a list: a list can say
"the F1_SETUP_RESPONSE arrives at F1_SETUP_PENDING"
four times, each copy with different IEs. ARCANE's model is a nested dict,
{source: {target: one edge}} — a dict key cannot repeat, so
each (source → target) pair holds exactly one edge. The conversion walks the list top to
bottom: the first copy of every pair is kept, every later parallel copy is logged
(_dropped_parallel_edges) and dropped. The rule here judges nothing
right or wrong — it is only "what fits".
我的解读：缩小不是通过什么算法，只是格式上的，等于用 hash table。字典的 键不能重复，每一对（源状态 → 目标状态）只能放一条边；转换程序从上往下扫 LLM 的列表，第一次见到的 放进容器，后面重复的直接丢掉，只留一条记录。
那为什么 LLM 一开始会生成这么多重复的？看 A 的 4 条响应边就明白了：它们不是 同一句话抄了 4 遍，是 4 个不同的测试情形。一条是正常响应，一条缺了必填的 IE，一条 IE 值超了范围， 一条条件字段不该出现。A 的提示词就是要它给 IE 标 valid / invalid / missing，建一致性测试的模型， 所以它把同一个消息在不同 IE 情形下的表现各写了一条。LLM 输出的不是一张图，是一张情形清单； hash table 的键只看从哪到哪，不看是什么情形，装不下的就都丢了。C 一条重复都没有，也是同一个道理 反过来：它没有 IE 可标，就没有情形可列。
Step 2 — the growth: ARCANE's merge_pta, no model in the loop
The LLM's job ended with the seed JSON in Stage 2. Everything from here down is
its counterpart: post-processing. Ordinary deterministic Python —
ARCANEModelRefiner.merge_pta,
ARCANE's own code, unmodified — merges 18 real CU↔DU sessions into the selected framing's
seed, one
message at a time (similarity 0.6·procedure + 0.4·weighted-IE). Score ≤ 0.7 grows a new
state; 0.7–0.95 with disagreeing IEs grows a _variant; above that
the message merges into an existing edge and nothing grows. Same seed + same traces → the
same machine, every run.
How refinement did it:
Where do the numbers come from?
Click each term in the ledger — every number is computed
live from the selected run's real artifacts (fsm.json,
initial_model.json, refined_fsm.dot).
What merge_pta actually does, one trace at a time
Trace 0 of the 18, verbatim from ARCANE's
sample_trace.json: F1-Setup → Context Setup, 4 messages.
The trace itself, verbatim
The animation above replays trace 0 as message names; here is the same trace
in full, exactly as it sits in ARCANE's sample_trace.json —
the very bytes merge_pta reads. Trace 1 is here too: it is the one whose
opening merges in example ③. Note the IEs carry "valid" tags, not raw
bytes — these are ARCANE's abstracted traces, and the scoring compares keys and values of exactly
this dict.
Five real scorings — what actually got carved, what actually merged
Not a mock-up: these five come from an instrumented replay of merge_pta
that calls ARCANE's own scoring code, unmodified, over the selected framing's seed and the same 18 traces.
The replay was verified to rebuild the recorded refined machine exactly — every state, every edge — before
anything below was lifted from its log. Score = 0.6·message-type + 0.4·weighted-IE; the
best old edge column shows what the arriving message was compared against, and what each candidate scored.
All 41 refined states, one by one
All 44 refined edges, one by one
| # | From | Message | To |
|---|
How 16 paths become one attack
Where the 16 paths walk — and what they never touch
The enumeration is a plain DFS from IDLE: a walk never
revisits a state (no loops), stops at 6 edges, and a path is only emitted at a true dead
end (a state with no outgoing edges) or when the length caps out. Here is the finding that matters:
the LLM's seed spine is a closed loop — PENDING → COMPLETE → PENDING,
PENDING → FAILURE → PENDING, a PENDING self-loop —
so every walk into it ends at a state whose only exits point back to already-visited states.
Neither emission rule ever fires: the seed edges emit zero paths. All 16 paths — every
path the fuzzer will ever walk — live on the chain the traces carved in Stage 3.
All 16 DFS paths, verbatim — and which of them gave birth to a CU07 case
The birth of the CU07 case(s) — one chain, parent to child
The sequence-level mutations are ARCANE's own, seeded
(seed 20260828) and replayed here with ancestry logging that was verified
to rebuild the recorded test_cases.json exactly, case by case. For every
surviving CU07 case the chain below shows: the parent path it descends from, the one mutation that
fired, and the result the fuzzer will actually send.
The 2 CU07 case(s) that survive, message by message
The seam worth knowing: ARCANE's shipped run used max_paths=10 —
with that cap the DFS never reaches any response branch (their own 100 emitted cases contain
zero). Raising the cap to 200 is what exposes the CU07 path.
From an abstract case to 68 bytes
A case says "send F1_SETUP_RESPONSE"; the CU wants APER bytes. ARCANE never
synthesizes a PDU from scratch — it re-encodes a captured real message as the template
(its trace library, 30 captured F1AP messages) with pycrate: from_json → to_aper.
A case survives this last gate only if its first message type has a template.
Byte ranges:
presentation + F1AP header
payload IE container
What the bytes say when you decode them
Every surviving case encodes to the same 68 bytes — they are the captured
template (trace[1], an F1-Setup-Response) re-encoded verbatim. Before
anything is sent, from_aper() must parse it back — a round-trip check.
The finding that matters: every byte here is legal. The IE-level mutations never survive to the wire — pycrate re-encodes the pristine template — and only message one of a case is ever encoded (cases carry 1–6 messages; the CU never lives to see message two). The attack is entirely in the arrival: a Response nobody asked for.
The one recorded run — and what the other framings never got to send
fuzz_run.py takes the surviving case, opens an SCTP
association to a fresh OAI gNB-CU (ARCANE's sdran/sdran-mobicom:v1 image
under Intel SDE), delivers the 68 bytes — and then does the thing ARCANE's own repo never does:
watch the CU die, polling its exit file every 2 seconds.
A browser page cannot open SCTP or crash a real CU — the live execution
ran on the host machine (fuzz_run.py, 2026-08-28), and the lines above
are that session's log. Switching framings shows each run's honest ending: A's is the recorded
crash; B's and C's are ready-but-never-sent — their attacks sit concretized on disk, unexecuted.
What this reproduction got wrong, guessed, or changed
Every number on this page is real and computed from artifacts — but the pipeline around those numbers contains our own choices, deviations and known defects. Nothing below is hidden elsewhere on the page; this is the single list.
| what | where it bites |
|---|---|
| The prompt is ours, not ARCANE's. ARCANE's repo ships no LLM prompt. The prompt shown in Stage 2 is our reconstruction, and the three framings (faithful protocol modeler / formal-methods engineer / the noob control) are personas we invented — ARCANE published none of them. Framing C's prompt was written bad on purpose, as a control: it is not a real user, it is a stress test of how much the prompt even matters. | Stage 2, "the exact prompt sent to the model" |
| Stage 1's expert conversion is authored illustration. No human expert actually built a manual FSM for this campaign — Stage 1 is our account of how a careful hand-drawn conversion would go, written to be checkable against the Stage 0 spec excerpts, but it is not a recorded artifact. Only the pipeline's numbers are measured. | Stage 1 |
| The LLM and its sizes are ours. We ran glm-5.3-flash:cloud, three times, and got three different seed models. ARCANE published one seed model; the 4→41, 6→44, 5→43 triples are our runs, measured on our harness — not ARCANE's published numbers. | every framing switch |
| Pass 1 truncated; we patched the prompt. GLM burned its 32 768-token output on reasoning and returned cut-off JSON. Our fix — a compactness clause capping transitions and IEs — is our patch, not part of ARCANE's pipeline, and it biases the model before refinement even starts. | Stage 2 note, framing A 2nd-pass marks |
| We re-implemented the harness around ARCANE's refiner. Only merge_pta is ARCANE's untouched code. The seed conversion, path enumeration, mutation passes, funnel bookkeeping, encoding and execution harness are our rebuild — a faithful-in-spirit copy, checked against ARCANE where artifacts existed, but not their code. Bugs in it are ours. | Stages 4–6 |
| We raised max_paths 10 → 200. ARCANE's shipped cap never reaches any response branch. Without this deliberate deviation the CU07 path does not exist in framings A and B, and the whole demo would stop at Stage 4 with zero cases. | Stage 4, "the seam worth knowing" |
| ARCANE's container silently drops parallel edges, and we inherit it. The model format allows one transition per (source, target), so 4 of framing A's 11 LLM edges (6 of B's 13; none of C's 7) vanished at conversion — a real fidelity loss we did not fix, only logged. | the 11→7 collapse, ledger "parallel dropped" |
| Most refined states are artifacts, not protocol truth. Four separate F1_SETUP_RESPONSE states and seven _variant copies exist because merge_pta walks each trace independently and one threshold (0.7/0.95/0.9) decides the shape. A human would draw one Response state (Stage 1's expert machine does). We present ARCANE's machine as-built, duplicates included — but nobody should read 41 as "the F1AP procedure has 41 milestones." | Stage 3, "how refinement did it" |
| We filtered the campaign down to one bug. The mutation pass produced 32 framed-A cases; we kept only the 2 unsolicited-Response ones and executed just 1 (case 31). Case 16 sits in the list as kept-but-never-run. This is a CU07 demo, not a reproduction of ARCANE's full multi-bug campaign. | Stage 4 funnel, Stage 6 |
| The crash ran under emulation on one image. The recorded SIGSEGV came from ARCANE's sdran/sdran-mobicom:v1 OAI CU under Intel SDE (the host CPU lacks AVX-512), once, on 2026-08-28. We did not reproduce it natively, across builds, or across runs — one emulated kill is our evidence. | Stage 6 |
What we deliberately did not do: fix ARCANE's direction-less seed format, deduplicate the refiner's artifact states, re-encode the dropped parallel edges, or expand the demo beyond the single bug CU07 needed. The point was to measure the pipeline as it actually is — walls and all — and put every deviation from ARCANE's shipped behavior on this one list.
Three prompts, one pipeline — what the comparison actually shows
The scoreboard first, then what it means. Every number below is the framing's own, from the funnels built on this page.
| framing | what the prompt asked for | seed | dropped at conversion | CU07 cases | executed |
|---|---|---|---|---|---|
| A — faithful modeler | model what the spec says, tag direction + IEs | 4 st · 11 ed | 4 of 11 | 2 | case 31 → CU exit 139 |
| B — formal-methods engineer | per-IE precision, preconditions on every edge | 6 st · 13 ed | 6 of 13 | 1 | no — kept, never run |
| C — the noob | nothing coherent — "give me everything" | 5 st · 7 ed | 0 of 7 | 2 | no — kept, never run |
The prompts, sentence by sentence — which line caused what
Each row pairs one real sentence from the prompt (shown verbatim in Stage 2) with the behavior it produced in that framing's seed JSON, and the measured consequence.
| framing | the sentence | what the LLM did with it | measured consequence |
|---|---|---|---|
| A — faithful modeler | "Model what the spec says the messages are, and tag each message's Information Elements with validity ('valid' / 'invalid' / 'missing') from the message-definition tables." | Authority = the spec itself. The LLM copied the spec's own shape: idle → pending → complete/failure, and every one of the 7 surviving edges carries direction plus a full IE list with validity tags. | Those validity tags are exactly the vocabulary the mutation pass flips; the kill itself (case 31) needed only the sequence-level drop. 9 233 raw chars — but it took 2 passes (pass 1 truncated). |
| contract: "…optional extra keys…" | The contract left the model free to volunteer extra structure (direction, procedure, notes) beyond the schema it was asked for — and it did. | Direction survived in A's seed because "what the spec says" includes the spec's Direction fields — but no sentence in the prompt demanded it. A got it right by persona, not by instruction. | |
| B — formal-methods engineer | "For every transition be pedantic about: (1) message DIRECTION …, (2) IE PRESENCE …, (3) the precondition state. Include any direction/precondition information as explicit fields on each transition." | The LLM became an over-formalist: ~10 fields per edge (Sender, Receiver, Precondition, Effect, Signalling, SpecRef…), 7 parallel copies of F1_SETUP_RESPONSE across two (source, target) pairs, and failure split into two states (FAILURE_RESPONSE + TIME_WAIT_BACKOFF) — the most expert-like seed of the three. | 22 028 raw chars = 2.4× A's JSON for the same 7 surviving pairs, and the container's one-edge-per-pair rule deleted 6 of 13 edges (46%) at the first deterministic step. The pedantry lives in fields no pipeline code ever reads. |
| C — the noob | "give me the states and the messages between them, all of them, whatever you find" + "dont spend too much effort on the details" | 7 thin edges; message names with spaces, not UPPER_SNAKE; and a pseudo-transition "(local) Time To Wait IE present -> start" — a local timer dressed up as wire traffic. | 0 of 7 edges carry IE lists, so downstream only drop-mutations were even possible. 100% conversion survival — survival of content nothing downstream can check. |
| "i always mix up request and response lol" | The model did not mix them up: every sent_by / received_by pair is correct. | Correctness in fields the pipeline never reads is decorative — the model knew better than the prompt, and it still did not help. | |
| "maybe add the failure one too??" | C modeled F1_SETUP_FAILURE, plus a TIME_WAIT state. | The one explicit request in the noob's prompt is the one part of the seed that is reliably there. |
One small metric, and how it is judged
| metric | how it's computed | A | B | C | winner |
|---|---|---|---|---|---|
| discovery yield | CU07 cases ÷ output tokens × 10 000 | 0.65 | 0.33 | 1.01 | C |
| conversion survival | edges the container keeps ÷ raw edges | 64% | 54% | 100% | C |
| seed auditability | surviving edges a reviewer can check against the spec (direction + IE list) | 100% | 100% | 0% | A = B |
| cost per surviving edge | output tokens ÷ surviving edges | 4 385 | 4 361 | 2 846 | C |
Read the metric honestly and it crowns the noob: on discovery per token C wins clearly, and that is not a bug in the metric — it is point 3 below (for this one bug, the pipeline carries discovery, not the prompt). What decides the contest is the judging rule, stated rather than implied: a prompt is worth what its seed is worth on the day the funnel comes back empty. Discovery yield measures a lucky campaign; auditability measures the seed you are left holding when nothing was found and the model must be debugged and extended by hand. By that rule B loses twice — worst yield and worst survival, its precision already deleted by the first deterministic step; C wins the day and nothing else; and A is the only framing that ties the best discovery (2 cases, byte-identical sequences to C's) while remaining fully auditable — every edge traceable to a spec clause.
1 — A models the protocol; the others model opinions about it. A's prompt did the one job a seed model has: turn spec text into the milestone machine a conformance tester would draw — idle → pending → complete/failure, every direction tagged. It never mentions attack, never mentions the fuzzer. The unsolicited Response that kills the CU was not put there by anybody — it was found, later, by ARCANE's mutation code walking a machine that had no opinion about it.
2 — B shows extra precision buying nothing. The formal-methods persona's per-IE structure looked the most rigorous, and 46% of it (6 of 13 edges) vanished at conversion because ARCANE allows one edge per (source, target). One CU07 case survived. Rigor beyond what the container can represent is noise with a confidence problem.
3 — The honest twist: even the noob prompt (framing C) reached 2 CU07 cases — and that is the most important number on this page. C's cases (16 and 31) are the identical sequences to A's, and its funnel counts match A's line for line. The reason is what all three framings share: ARCANE's own merge\_pta grew the traced procedures onto any walkable F1 Setup spine, our max_paths 10 → 200 patch opened the response branches, and the mutation pass is seed-agnostic. For this one bug, discovery is carried by the pipeline, not the prompt — "the LLM found the bug" oversells the prompt, and C is the experiment that shows it. But forgiving is not the same as equivalent: C's seed carried a pseudo-transition ("time to wait") inside a message-type field, no IE lists at all, and a sent\_by / received\_by tag scheme the pipeline had to reinterpret. Nothing downstream could rely on a machine built like that — it just happened not to matter for a single-message bug. The moment the machine's meaning matters — auditing why a case exists, extending the campaign past CU07, comparing the model against the spec — the noob's model gives you nothing to audit.
4 — And A is the faithful choice for a reproduction. ARCANE published one seed model built from a neutral spec→FSM conversion. Framing A is the closest reconstruction of that, it produced the campaign's recorded run (case 31 — one unsolicited Response, exit 139), and its numbers are the ones every other framing is measured against on this page. The pipeline carried the discovery; framing A carries the evidence.