Experiment 1 · Results

Results: 0 of 14 runs

(a) \(\textbf{H}_a\): \(s(N,\text{off})\) is non-decreasing in \(N\). (b) \(\textbf{H}_b\): at equal \(k\), \(E_{\rm miss}\) and \(f\) are lower under paired, with \(s\) not much lower. The definitions of \(s, f, E_{\rm self}, E_{\rm miss}, F\) are in Design §3–4. With \(m=2\) seeds both are descriptions, not tests (§2.8).

Run list

The 14 runs in the order \(\pi=\text{shuffle}_{\,\text{seed }1}(\mathcal C\times\{1,2\})\) (Python random.Random(1).shuffle); the Codex set uses the same order.
\(\pi\)run\((N,v)\)\(k\)status
1exp1-cc-N4-paired-s2(4, paired)8not run
2exp1-cc-N2-paired-s1(2, paired)4not run
3exp1-cc-N1-off-s1(1, off)1not run
4exp1-cc-N4-paired-s1(4, paired)8not run
5exp1-cc-N8-off-s1(8, off)8not run
6exp1-cc-N4-off-s2(4, off)4not run
7exp1-cc-N2-off-s2(2, off)2not run
8exp1-cc-N1-paired-s1(1, paired)2not run
9exp1-cc-N8-off-s2(8, off)8not run
10exp1-cc-N2-paired-s2(2, paired)4not run
11exp1-cc-N4-off-s1(4, off)4not run
12exp1-cc-N1-off-s2(1, off)1not run
13exp1-cc-N1-paired-s2(1, paired)2not run
14exp1-cc-N2-off-s1(2, off)2not run

The pilot is outside the 14; its report stays on the private branch until the 14 runs are filed.

(a) More agents

Empty axes, to be filled: solved rate s from 0 to 1 against agents in the run k = 1, 2, 4, 8 on a log scale, one curve for verifier=off and one for verifier=paired.
Solved rate \(s\) against \(k\), with 95% Wilson intervals; empty until runs exist.

(b) Verifiers

Empty bar chart, to be filled: mistakes per run for the seven conditions, verifier=off at k = 1, 2, 4, 8 and verifier=paired at k = 2, 4, 8; each bar split into caught and missed mistakes, with false alarms beside it.
Mistakes per run by condition: caught, missed, false alarms; empty until runs exist.

Cost

Empty axes, to be filled: tokens per run on a log scale against agents in the run k = 1, 2, 4, 8, one curve per verifier value, with the budget B = 100M and the stop line at 80M drawn.
Tokens per run against \(k\), under the budget line; empty until runs exist.