Efficiency: torus vs stock pi overhead
torus layers a harness (child extension set, tool registrations, system-prompt additions) on top of the stock pi engine. This page documents how that harness-layer context overhead is measured. The comparison the bench makes: stock pi (bare engine, no extensions) vs torus (engine + the same child-extension set delegations get), on identical prompts; the delta is torus’s overhead.
Summary
Headline results first: totals per config across the full task set, the overhead delta (torus − stock), and how that delta moved since the prior release’s run. The table is pasted from the torus 0.14.0 bench output; the HTML report carries the same table, regenerated on every run.
| model | field | stock | torus | Δ overhead | Δ change |
|---|---|---|---|---|---|
| zai/glm-5.3-flash | tokens in | 8,195 | 11,899 | +3,704 | +2,797 vs torus 0.6.0 |
| zai/glm-5.3-flash | cache read | 64,768 | 180,480 | +115,712 | -5,696 vs torus 0.6.0 |
| zai/glm-5.3-flash | cost | $0.0039 | $0.0084 | +$0.0045 | +$0.0007 vs torus 0.6.0 |
| zai/glm-5.3-flash | ms | 72,276 ms | 96,867 ms | +24,591 | -31,036 vs torus 0.6.0 |
| zai/glm-5.3 | tokens in | 25,053 | 14,730 | -10,323 | -664 vs torus 0.6.0 |
| zai/glm-5.3 | cache read | 56,768 | 146,048 | +89,280 | -1,408 vs torus 0.6.0 |
| zai/glm-5.3 | cost | $0.0569 | $0.0646 | +$0.0077 | -$0.0028 vs torus 0.6.0 |
| zai/glm-5.3 | ms | 61,132 ms | 89,135 ms | +28,003 | -10,581 vs torus 0.6.0 |
What the 0.14.0 numbers say: the overhead is structural, not behavioral. Cache read dominates because the extension declaration surface is cached input, billed at a fraction of fresh-input price. A full sweep costs under half a cent more on flash and $0.0077 more on glm-5.3. Both models came in below 0.6.0 on cache read, cost, and wall-clock; the flash tokens-in delta grew by 2,797 tokens, and the per-task verdicts below place most of that movement within run-to-run noise.
Method
- Task set. A fixed, read-only, repo-local prompt set in three difficulty
tiers (trivial lookup / aggregate sweep / multi-file reasoning), defined in
scripts/bench/tasks.mjs. Deterministic inputs, no file writes, no network beyond the model API, so any token delta comes from the harness layer, not the task. - Configs. Per task, the pinned engine (
resolveEngineBin) is spawned with--mode json: stock gets the bare argv, torus gets the same argv plus the canonicalchildExtensionArgslist andTORUS_ENGINE_CHILD=1; byte-for-byte the spawn a delegation receives. - Warmup discarded. One warmup run per task × config executes first and is excluded from statistics, so measured runs start from a steady-state prompt cache and warm extension load path.
- Interleaved rotation. Measured passes alternate config order (stock, torus / torus, stock / …), so provider-side prompt-cache drift and slow-warming trends cancel across configs instead of confounding the delta.
- Medians with spread. Report cells are medians;
±is half the interquartile range over measured runs. A delta row is labeledsignalonly when the median delta exceeds the larger config’s IQR (run-to-run noise); anything smaller is labeledwithin noiserather than passed off as a finding. With fewer than two measured runs there is no dispersion basis and verdicts render as—. - Aggregation. Each run’s JSONL event stream is folded by the shared
reduceEngineEventpipeline (the same reducer roster/team/fleet consume): turns, tokens in/out,cacheRead/cacheWrite, and the engine-computed dollar cost come frommessage_endusage. Malformed event lines are skipped and counted, never fatal. - Cache accounting. zai/GLM speaks the OpenAI-compatible schema, which
reports cache reads (
cached_tokens) but has no cache-write token accounting, so the cache-write column reads 0 on this provider by construction (writes happen implicitly, unbilled and unreported; they surface later as cache reads). On providers with explicit cache-write usage (e.g. Anthropic-stylecache_creationtokens) the column fills. - Metadata pinned. Every report header carries the date, torus version, git commit, engine pin, and model list, so published numbers are reproducible claims.
- Run history. The JSON sidecar accumulates one entry per run, so re-benchmarking at each release appends rather than overwrites; older single-run sidecar files migrate to the envelope on first append. Both reports open with a summary table (newest totals plus movement since the prior release), and the HTML report adds a run-history table of per-release overhead deltas.
Running it
The bench spawns the real engine and calls the model API; run it with valid credentials, from the repo root:
npm run bench:overhead -- --model zai/glm-5.3-flash,zai/glm-5.3| Option | Meaning |
|---|---|
--model <ids> | comma-separated model ids, one full sweep each (default zai/glm-5.3-flash) |
--runs N | measured runs per task × config (default 3) |
--warmup N | warmup runs per task × config, discarded from stats (default 1) |
--tasks <ids> | comma-separated task subset (default: all tasks) |
--stock-only / --torus-only | single-config sweeps (no delta rows) |
--timeout <sec> | per-run wall-clock cap (default 600) |
--json <path> | append the run’s raw records (warmups included) to the sidecar history; delete the file to start fresh |
--html <path> | write the self-contained HTML report (charts + method) |
Exit status is 0 when every run (warmups included) completed, 1 otherwise; failures are reported and the bench keeps going.
Artifacts in this repo: docs/efficiency.html (generated
report with distribution charts and a run-history table) and
docs/bench-records.json (raw per-run records; the
sidecar carries one entry per run across releases, and single-run
files migrate on first append).
Numbers
Each release re-runs the bench and appends the run to the sidecar history; the newest numbers lead and earlier releases stay below, unchanged. The HTML report’s run-history table carries the same per-release deltas.
torus 0.14.0 · 2026-10-18 · engine pin 1.3.0 · commit 8f4da67
model(s): zai/glm-5.3-flash, zai/glm-5.3 · engine: node_modules/.bin/pi (pin 1.3.0) · torus 0.14.0 · commit 8f4da67
generated 2026-10-18T02:37:44.313Z · 1 warmup run(s) per config discarded · config order rotated per pass (interleaved)
3 measured runs per task per config.
Cells are medians; ± is half the interquartile range across measured runs. Verdicts: signal means the delta exceeds run-to-run noise (max IQR of the two configs).
zai/glm-5.3-flash · 0.14.0
| task | config | turns | tokens in | tokens out | cache read | cache write | cost | ms |
|---|---|---|---|---|---|---|---|---|
| readme-name | stock | 2 | 685 ±1,088 | 92 ±5 | 8,704 ±1,088 | 0 | $0.0004 ±$0.0001 | 11,426 ms ±1,298 ms |
| readme-name | torus | 2 | 1,156 ±2,889 | 85 ±3 | 24,320 ±2,848 | 0 | $0.0009 ±$0.0003 | 12,841 ms ±191 ms |
| agents-doc-headings | stock | 2 | 2,406 ±1,104 | 400 ±5 | 8,640 ±1,104 | 0 | $0.0008 ±$0.0001 | 12,280 ms ±812 ms |
| agents-doc-headings | torus | 2 | 3,038 ±3,043 | 467 ±8 | 24,320 ±3,040 | 0 | $0.0014 ±$0.0004 | 17,303 ms ±133 ms |
| register-tool-count | stock | 2 | 81 ±20 | 77 ±28 | 8,704 | 0 | $0.0003 ±$0.0000 | 8,035 ms ±529 ms |
| register-tool-count | torus | 2 | 143 ±9 | 92 ±20 | 24,320 | 0 | $0.0008 ±$0.0000 | 12,539 ms ±708 ms |
| skills-inventory | stock | 2 | 134 ±1 | 114 ±6 | 8,704 | 0 | $0.0003 ±$0.0000 | 11,177 ms ±3,618 ms |
| skills-inventory | torus | 2 | 195 ±2 | 117 ±8 | 24,320 | 0 | $0.0008 ±$0.0000 | 13,136 ms ±892 ms |
| extension-registration | stock | 3 ±1 | 3,857 ±1,409 | 683 ±23 | 16,448 ±3,344 | 0 | $0.0014 ±$0.0003 | 18,151 ms ±2,182 ms |
| extension-registration | torus | 3 ±1 | 5,847 ±2,467 | 660 ±115 | 45,824 ±8,976 | 0 | $0.0029 ±$0.0004 | 24,919 ms ±3,558 ms |
| engine-pin-report | stock | 3 ±0 | 1,032 ±72 | 141 ±5 | 13,568 ±1,200 | 0 | $0.0006 ±$0.0000 | 11,207 ms ±5,445 ms |
| engine-pin-report | torus | 3 ±0 | 1,520 ±289 | 136 ±25 | 37,376 ±3,408 | 0 | $0.0015 ±$0.0001 | 16,129 ms ±1,538 ms |
| totals | stock | 14 ±1 | 8,195 ±3,695 | 1,507 ±71 | 64,768 ±6,736 | 0 | $0.0039 ±$0.0006 | 72,276 ms ±13,882 ms |
| totals | torus | 14 ±1 | 11,899 ±8,699 | 1,557 ±179 | 180,480 ±18,272 | 0 | $0.0084 ±$0.0013 | 96,867 ms ±7,019 ms |
Δ torus − stock (medians) vs run-to-run noise:
| task | field | Δ med | noise (IQR) | verdict |
|---|---|---|---|---|
| readme-name | turns | 0 | 0 | flat |
| readme-name | tokens in | +471 | 5,777 | within noise |
| readme-name | tokens out | -7 | 10 | within noise |
| readme-name | cache read | +15,616 | 5,696 | signal |
| readme-name | cache write | 0 | 0 | flat |
| readme-name | cost | +$0.0005 | $0.0007 | within noise |
| readme-name | ms | +1,415 | 2,595 ms | within noise |
| agents-doc-headings | turns | 0 | 0 | flat |
| agents-doc-headings | tokens in | +632 | 6,087 | within noise |
| agents-doc-headings | tokens out | +67 | 16 | signal |
| agents-doc-headings | cache read | +15,680 | 6,080 | signal |
| agents-doc-headings | cache write | 0 | 0 | flat |
| agents-doc-headings | cost | +$0.0006 | $0.0007 | within noise |
| agents-doc-headings | ms | +5,023 | 1,623 ms | signal |
| register-tool-count | turns | 0 | 0 | flat |
| register-tool-count | tokens in | +62 | 41 | signal |
| register-tool-count | tokens out | +15 | 56 | within noise |
| register-tool-count | cache read | +15,616 | 0 | signal |
| register-tool-count | cache write | 0 | 0 | flat |
| register-tool-count | cost | +$0.0005 | $0.0000 | signal |
| register-tool-count | ms | +4,504 | 1,416 ms | signal |
| skills-inventory | turns | 0 | 0 | flat |
| skills-inventory | tokens in | +61 | 4 | signal |
| skills-inventory | tokens out | +3 | 16 | within noise |
| skills-inventory | cache read | +15,616 | 0 | signal |
| skills-inventory | cache write | 0 | 0 | flat |
| skills-inventory | cost | +$0.0005 | $0.0000 | signal |
| skills-inventory | ms | +1,959 | 7,236 ms | within noise |
| extension-registration | turns | 0 | 1 | within noise |
| extension-registration | tokens in | +1,990 | 4,934 | within noise |
| extension-registration | tokens out | -23 | 231 | within noise |
| extension-registration | cache read | +29,376 | 17,952 | signal |
| extension-registration | cache write | 0 | 0 | flat |
| extension-registration | cost | +$0.0015 | $0.0009 | signal |
| extension-registration | ms | +6,768 | 7,115 ms | within noise |
| engine-pin-report | turns | 0 | 1 | within noise |
| engine-pin-report | tokens in | +488 | 579 | within noise |
| engine-pin-report | tokens out | -5 | 50 | within noise |
| engine-pin-report | cache read | +23,808 | 6,816 | signal |
| engine-pin-report | cache write | 0 | 0 | flat |
| engine-pin-report | cost | +$0.0009 | $0.0002 | signal |
| engine-pin-report | ms | +4,922 | 10,889 ms | within noise |
zai/glm-5.3 · 0.14.0
| task | config | turns | tokens in | tokens out | cache read | cache write | cost | ms |
|---|---|---|---|---|---|---|---|---|
| readme-name | stock | 2 | 666 ±35 | 68 ±1 | 8,640 | 0 | $0.0035 ±$0.0001 | 5,616 ms ±859 ms |
| readme-name | torus | 2 | 960 ±34 | 74 ±7 | 24,320 | 0 | $0.0079 ±$0.0000 | 12,400 ms ±589 ms |
| agents-doc-headings | stock | 3 ±0 | 2,586 ±675 | 459 ±39 | 13,312 ±1,088 | 0 | $0.0091 ±$0.0011 | 11,020 ms ±243 ms |
| agents-doc-headings | torus | 2 | 3,008 ±1 | 268 ±64 | 24,320 | 0 | $0.0117 ±$0.0003 | 14,598 ms ±575 ms |
| register-tool-count | stock | 2 | 104 ±9 | 117 ±10 | 8,704 | 0 | $0.0029 ±$0.0001 | 6,014 ms ±696 ms |
| register-tool-count | torus | 2 | 168 ±9 | 104 ±2 | 24,320 | 0 | $0.0070 ±$0.0000 | 12,227 ms ±563 ms |
| skills-inventory | stock | 2 | 146 ±4 | 155 ±9 | 8,704 | 0 | $0.0031 ±$0.0000 | 8,704 ms ±1,383 ms |
| skills-inventory | torus | 2 | 192 ±33 | 138 ±2 | 24,320 ±32 | 0 | $0.0072 ±$0.0000 | 17,675 ms ±1,887 ms |
| extension-registration | stock | 2 | 20,705 ±5,169 | 953 ±67 | 8,704 ±5,168 | 0 | $0.0344 ±$0.0059 | 16,342 ms ±2,280 ms |
| extension-registration | torus | 2 | 8,958 ±17 | 716 ±13 | 24,384 ±16 | 0 | $0.0220 ±$0.0000 | 19,717 ms ±1,603 ms |
| engine-pin-report | stock | 2 | 846 ±35 | 105 ±2 | 8,704 ±16 | 0 | $0.0039 ±$0.0000 | 13,436 ms ±2,515 ms |
| engine-pin-report | torus | 2 | 1,444 ±4 | 104 ±5 | 24,384 | 0 | $0.0088 ±$0.0000 | 12,518 ms ±1,838 ms |
| totals | stock | 13 ±0 | 25,053 ±5,927 | 1,857 ±129 | 56,768 ±6,272 | 0 | $0.0569 ±$0.0072 | 61,132 ms ±7,975 ms |
| totals | torus | 12 | 14,730 ±98 | 1,404 ±92 | 146,048 ±48 | 0 | $0.0646 ±$0.0004 | 89,135 ms ±7,054 ms |
Δ torus − stock (medians) vs run-to-run noise:
| task | field | Δ med | noise (IQR) | verdict |
|---|---|---|---|---|
| readme-name | turns | 0 | 0 | flat |
| readme-name | tokens in | +294 | 70 | signal |
| readme-name | tokens out | +6 | 14 | within noise |
| readme-name | cache read | +15,680 | 0 | signal |
| readme-name | cache write | 0 | 0 | flat |
| readme-name | cost | +$0.0044 | $0.0001 | signal |
| readme-name | ms | +6,784 | 1,719 ms | signal |
| agents-doc-headings | turns | -1 | 1 | signal |
| agents-doc-headings | tokens in | +422 | 1,351 | within noise |
| agents-doc-headings | tokens out | -191 | 127 | signal |
| agents-doc-headings | cache read | +11,008 | 2,176 | signal |
| agents-doc-headings | cache write | 0 | 0 | flat |
| agents-doc-headings | cost | +$0.0026 | $0.0021 | signal |
| agents-doc-headings | ms | +3,578 | 1,149 ms | signal |
| register-tool-count | turns | 0 | 0 | flat |
| register-tool-count | tokens in | +64 | 19 | signal |
| register-tool-count | tokens out | -13 | 21 | within noise |
| register-tool-count | cache read | +15,616 | 0 | signal |
| register-tool-count | cache write | 0 | 0 | flat |
| register-tool-count | cost | +$0.0041 | $0.0001 | signal |
| register-tool-count | ms | +6,213 | 1,391 ms | signal |
| skills-inventory | turns | 0 | 0 | flat |
| skills-inventory | tokens in | +46 | 66 | within noise |
| skills-inventory | tokens out | -17 | 18 | within noise |
| skills-inventory | cache read | +15,616 | 64 | signal |
| skills-inventory | cache write | 0 | 0 | flat |
| skills-inventory | cost | +$0.0040 | $0.0001 | signal |
| skills-inventory | ms | +8,971 | 3,774 ms | signal |
| extension-registration | turns | 0 | 0 | flat |
| extension-registration | tokens in | -11,747 | 10,339 | signal |
| extension-registration | tokens out | -237 | 135 | signal |
| extension-registration | cache read | +15,680 | 10,336 | signal |
| extension-registration | cache write | 0 | 0 | flat |
| extension-registration | cost | -$0.0123 | $0.0118 | signal |
| extension-registration | ms | +3,375 | 4,560 ms | within noise |
| engine-pin-report | turns | 0 | 0 | flat |
| engine-pin-report | tokens in | +598 | 69 | signal |
| engine-pin-report | tokens out | -1 | 10 | within noise |
| engine-pin-report | cache read | +15,680 | 32 | signal |
| engine-pin-report | cache write | 0 | 0 | flat |
| engine-pin-report | cost | +$0.0049 | $0.0001 | signal |
| engine-pin-report | ms | -918 | 5,029 ms | within noise |
torus 0.6.0 · 2026-10-07 · engine pin 1.0.4 · commit 1c7256e (historical)
Measured 2026-10-07, torus 0.6.0 + engine pin 1.0.4, commit 1c7256e · 3 measured runs per task per config (1 warmup discarded, config order rotated per pass) · npm run bench:overhead -- --model zai/glm-5.3-flash,zai/glm-5.3.
Cells are medians; ± is half the interquartile range across measured runs. Verdicts: signal means the delta exceeds run-to-run noise (max IQR of the two configs).
zai/glm-5.3-flash · 0.6.0
| task | config | turns | tokens in | tokens out | cache read | cache write | cost | ms |
|---|---|---|---|---|---|---|---|---|
| readme-name | stock | 2 | 3,644 ±1,722 | 81 ±9 | 5,888 ±1,696 | 0 | $0.0008 ±$0.0002 | 5,402 ms ±265 ms |
| readme-name | torus | 2 | 3,949 ±3,509 | 110 ±8 | 21,824 ±3,488 | 0 | $0.0013 ±$0.0004 | 12,805 ms ±579 ms |
| agents-doc-headings | stock | 2 | 2,411 ±1,440 | 370 ±37 | 8,448 ±1,440 | 0 | $0.0008 ±$0.0002 | 8,257 ms ±242 ms |
| agents-doc-headings | torus | 2 | 2,908 ±2,880 | 405 ±34 | 24,384 ±2,880 | 0 | $0.0014 ±$0.0004 | 17,335 ms ±835 ms |
| register-tool-count | stock | 2 | 121 ±8 | 65 ±6 | 8,512 | 0 | $0.0003 ±$0.0000 | 4,558 ms ±840 ms |
| register-tool-count | torus | 2 | 19 ±17 | 93 ±9 | 24,512 ±16 | 0 | $0.0008 ±$0.0000 | 12,227 ms ±232 ms |
| skills-inventory | stock | 2 | 158 ±32 | 99 ±6 | 8,512 ±32 | 0 | $0.0003 ±$0.0000 | 5,375 ms ±891 ms |
| skills-inventory | torus | 2 | 126 ±17 | 99 ±2 | 24,448 ±16 | 0 | $0.0008 ±$0.0000 | 11,939 ms ±98 ms |
| extension-registration | stock | 3 ±0 | 7,698 ±827 | 709 ±76 | 20,992 ±1,840 | 0 | $0.0023 ±$0.0002 | 16,718 ms ±1,990 ms |
| extension-registration | torus | 4 ±0 | 7,073 ±513 | 772 ±110 | 54,592 ±4,192 | 0 | $0.0032 ±$0.0002 | 33,399 ms ±3,786 ms |
| engine-pin-report | stock | 3 | 539 ±116 | 84 ±1 | 13,504 ±112 | 0 | $0.0005 ±$0.0000 | 5,417 ms ±319 ms |
| engine-pin-report | torus | 3 | 1,403 ±232 | 118 ±15 | 37,504 ±256 | 0 | $0.0014 ±$0.0000 | 13,649 ms ±879 ms |
| totals | stock | 14 ±0 | 14,571 ±4,144 | 1,408 ±134 | 65,856 ±5,120 | 0 | $0.0050 ±$0.0006 | 45,727 ms ±4,547 ms |
| totals | torus | 15 ±0 | 15,478 ±7,168 | 1,597 ±177 | 187,264 ±10,848 | 0 | $0.0088 ±$0.0011 | 101,354 ms ±6,409 ms |
Δ torus − stock (medians) vs run-to-run noise:
| task | field | Δ med | noise (IQR) | verdict |
|---|---|---|---|---|
| readme-name | turns | 0 | 0 | flat |
| readme-name | tokens in | +305 | 7,018 | within noise |
| readme-name | tokens out | +29 | 18 | signal |
| readme-name | cache read | +15,936 | 6,976 | signal |
| readme-name | cache write | 0 | 0 | flat |
| readme-name | cost | +$0.0005 | $0.0008 | within noise |
| readme-name | ms | +7,403 | 1,158 ms | signal |
| agents-doc-headings | turns | 0 | 0 | flat |
| agents-doc-headings | tokens in | +497 | 5,760 | within noise |
| agents-doc-headings | tokens out | +35 | 74 | within noise |
| agents-doc-headings | cache read | +15,936 | 5,760 | signal |
| agents-doc-headings | cache write | 0 | 0 | flat |
| agents-doc-headings | cost | +$0.0006 | $0.0007 | within noise |
| agents-doc-headings | ms | +9,078 | 1,669 ms | signal |
| register-tool-count | turns | 0 | 0 | flat |
| register-tool-count | tokens in | -102 | 34 | signal |
| register-tool-count | tokens out | +28 | 17 | signal |
| register-tool-count | cache read | +16,000 | 32 | signal |
| register-tool-count | cache write | 0 | 0 | flat |
| register-tool-count | cost | +$0.0005 | $0.0000 | signal |
| register-tool-count | ms | +7,669 | 1,681 ms | signal |
| skills-inventory | turns | 0 | 0 | flat |
| skills-inventory | tokens in | -32 | 64 | within noise |
| skills-inventory | tokens out | 0 | 12 | within noise |
| skills-inventory | cache read | +15,936 | 64 | signal |
| skills-inventory | cache write | 0 | 0 | flat |
| skills-inventory | cost | +$0.0005 | $0.0000 | signal |
| skills-inventory | ms | +6,564 | 1,781 ms | signal |
| extension-registration | turns | +1 | 1 | signal |
| extension-registration | tokens in | -625 | 1,654 | within noise |
| extension-registration | tokens out | +63 | 220 | within noise |
| extension-registration | cache read | +33,600 | 8,384 | signal |
| extension-registration | cache write | 0 | 0 | flat |
| extension-registration | cost | +$0.0009 | $0.0005 | signal |
| extension-registration | ms | +16,681 | 7,572 ms | signal |
| engine-pin-report | turns | 0 | 0 | flat |
| engine-pin-report | tokens in | +864 | 463 | signal |
| engine-pin-report | tokens out | +34 | 29 | signal |
| engine-pin-report | cache read | +24,000 | 512 | signal |
| engine-pin-report | cache write | 0 | 0 | flat |
| engine-pin-report | cost | +$0.0009 | $0.0001 | signal |
| engine-pin-report | ms | +8,232 | 1,758 ms | signal |
zai/glm-5.3 · 0.6.0
| task | config | turns | tokens in | tokens out | cache read | cache write | cost | ms |
|---|---|---|---|---|---|---|---|---|
| readme-name | stock | 2 | 742 ±82 | 63 ±5 | 8,512 | 0 | $0.0035 ±$0.0001 | 4,110 ms ±199 ms |
| readme-name | torus | 2 | 694 ±271 | 76 ±5 | 24,448 ±272 | 0 | $0.0076 ±$0.0003 | 10,978 ms ±82 ms |
| agents-doc-headings | stock | 2 ±0 | 2,374 ±77 | 365 ±62 | 8,512 ±1,632 | 0 | $0.0072 ±$0.0008 | 7,970 ms ±1,075 ms |
| agents-doc-headings | torus | 2 ±0 | 2,843 ±67 | 386 ±73 | 24,448 ±3,088 | 0 | $0.0120 ±$0.0012 | 14,236 ms ±3,195 ms |
| register-tool-count | stock | 2 | 177 ±23 | 150 ±9 | 8,512 ±16 | 0 | $0.0031 ±$0.0001 | 5,453 ms ±569 ms |
| register-tool-count | torus | 2 | 113 ±21 | 128 ±11 | 24,448 | 0 | $0.0071 ±$0.0001 | 10,587 ms ±785 ms |
| skills-inventory | stock | 2 | 168 ±17 | 136 ±6 | 8,512 ±16 | 0 | $0.0031 ±$0.0000 | 5,051 ms ±484 ms |
| skills-inventory | torus | 2 | 135 ±3 | 123 ±10 | 24,448 | 0 | $0.0071 ±$0.0000 | 12,175 ms ±358 ms |
| extension-registration | stock | 2 | 19,273 ±1 | 679 ±67 | 8,576 | 0 | $0.0322 ±$0.0003 | 11,288 ms ±936 ms |
| extension-registration | torus | 2 ±1 | 8,756 ±180 | 644 ±35 | 24,448 ±10,544 | 0 | $0.0218 ±$0.0029 | 19,495 ms ±2,202 ms |
| engine-pin-report | stock | 3 ±0 | 779 ±129 | 140 ±16 | 13,376 ±1,328 | 0 | $0.0047 ±$0.0004 | 6,717 ms ±1,110 ms |
| engine-pin-report | torus | 2 | 1,313 ±320 | 123 ±6 | 24,448 ±320 | 0 | $0.0086 ±$0.0004 | 11,702 ms ±1,479 ms |
| totals | stock | 13 ±1 | 23,513 ±329 | 1,533 ±164 | 56,000 ±2,992 | 0 | $0.0538 ±$0.0017 | 40,589 ms ±4,373 ms |
| totals | torus | 12 ±1 | 13,854 ±861 | 1,480 ±140 | 146,688 ±14,224 | 0 | $0.0643 ±$0.0049 | 79,173 ms ±8,100 ms |
Δ torus − stock (medians) vs run-to-run noise:
| task | field | Δ med | noise (IQR) | verdict |
|---|---|---|---|---|
| readme-name | turns | 0 | 0 | flat |
| readme-name | tokens in | -48 | 542 | within noise |
| readme-name | tokens out | +13 | 10 | signal |
| readme-name | cache read | +15,936 | 544 | signal |
| readme-name | cache write | 0 | 0 | flat |
| readme-name | cost | +$0.0041 | $0.0006 | signal |
| readme-name | ms | +6,868 | 398 ms | signal |
| agents-doc-headings | turns | 0 | 1 | within noise |
| agents-doc-headings | tokens in | +469 | 154 | signal |
| agents-doc-headings | tokens out | +21 | 147 | within noise |
| agents-doc-headings | cache read | +15,936 | 6,176 | signal |
| agents-doc-headings | cache write | 0 | 0 | flat |
| agents-doc-headings | cost | +$0.0049 | $0.0024 | signal |
| agents-doc-headings | ms | +6,266 | 6,390 ms | within noise |
| register-tool-count | turns | 0 | 0 | flat |
| register-tool-count | tokens in | -64 | 46 | signal |
| register-tool-count | tokens out | -22 | 21 | signal |
| register-tool-count | cache read | +15,936 | 32 | signal |
| register-tool-count | cache write | 0 | 0 | flat |
| register-tool-count | cost | +$0.0040 | $0.0002 | signal |
| register-tool-count | ms | +5,134 | 1,570 ms | signal |
| skills-inventory | turns | 0 | 0 | flat |
| skills-inventory | tokens in | -33 | 35 | within noise |
| skills-inventory | tokens out | -13 | 20 | within noise |
| skills-inventory | cache read | +15,936 | 32 | signal |
| skills-inventory | cache write | 0 | 0 | flat |
| skills-inventory | cost | +$0.0040 | $0.0001 | signal |
| skills-inventory | ms | +7,124 | 969 ms | signal |
| extension-registration | turns | 0 | 1 | within noise |
| extension-registration | tokens in | -10,517 | 360 | signal |
| extension-registration | tokens out | -35 | 135 | within noise |
| extension-registration | cache read | +15,872 | 21,088 | within noise |
| extension-registration | cache write | 0 | 0 | flat |
| extension-registration | cost | -$0.0104 | $0.0059 | signal |
| extension-registration | ms | +8,207 | 4,403 ms | signal |
| engine-pin-report | turns | -1 | 1 | signal |
| engine-pin-report | tokens in | +534 | 640 | within noise |
| engine-pin-report | tokens out | -17 | 32 | within noise |
| engine-pin-report | cache read | +11,072 | 2,656 | signal |
| engine-pin-report | cache write | 0 | 0 | flat |
| engine-pin-report | cost | +$0.0039 | $0.0008 | signal |
| engine-pin-report | ms | +4,985 | 2,957 ms | signal |
Reading them
- Cache read is the dominant, consistent overhead signal: the extension declaration surface, cached by the provider at a fraction of input price.
- Non-cached tokens-in deltas are mostly within run-to-run noise, and can be negative: on the multi-file reasoning task the stock engine flails (more turns, more fresh input) while the torus context reaches the answer directly.
- Cost deltas are small but above noise on glm-5.3; on flash pricing they sit closer to the noise floor.
- Wall-clock overhead per run is dominated by extension loading at spawn; amortized in real sessions that run many turns.
- Ambient roles (dream/reflect, session-title) resolve via the fast tier, so a fast-tier override in the config’s
chainssection pointing at a local model moves all ambient work to it. - Consumed ACP agents bypass the chain machinery entirely (they run their own models), so their usage/cost figures are best-effort zeros; see the acp section in
docs/extensions.md. - Compare models only within their own section; pricing and token behavior differ.
- For release gating, cache read and cost deltas are the dependable signals. Each run compares torus with stock on the same engine pin, so the overhead delta is engine-normalized. Tokens in, turns, and wall-clock move with task behavior and network jitter; treat them as context, not gate conditions.