torus overhead bench

Generated
2026-10-18T02:37:44.313Z
Model(s)
zai/glm-5.3-flash, zai/glm-5.3
Engine
node_modules/.bin/pi (pin 1.3.0)
torus
0.14.0 @ commit 8f4da67
Runs
3 measured per task × config, 1 warmup discarded, config order rotated per pass

Summary

torus 0.14.0 vs stock per model: totals (sum of per-task medians), the overhead delta, and its movement since the model's prior sidecar run.

modelfieldstocktorusΔ overheadΔ change
zai/glm-5.3-flashtokens in8,19511,899+3,704+2,797 vs torus 0.6.0
zai/glm-5.3-flashcache read64,768180,480+115,712-5,696 vs torus 0.6.0
zai/glm-5.3-flashcost$0.0039$0.0084+$0.0045+$0.0007 vs torus 0.6.0
zai/glm-5.3-flashms72,276 ms96,867 ms+24,591-31,036 vs torus 0.6.0
zai/glm-5.3tokens in25,05314,730-10,323-664 vs torus 0.6.0
zai/glm-5.3cache read56,768146,048+89,280-1,408 vs torus 0.6.0
zai/glm-5.3cost$0.0569$0.0646+$0.0077-$0.0028 vs torus 0.6.0
zai/glm-5.3ms61,132 ms89,135 ms+28,003-10,581 vs torus 0.6.0

Run history

Δ = torus − stock totals (sum of per-task medians) per model and run; history accumulates in the JSON sidecar.

runmodelΔ tokens inΔ cache readΔ costΔ ms
2026-10-07 · torus 0.6.0 · 1c7256e · engine 1.0.4zai/glm-5.3-flash+907+121,408+$0.0038+55,627
2026-10-07 · torus 0.6.0 · 1c7256e · engine 1.0.4zai/glm-5.3-9,659+90,688+$0.0105+38,584
2026-10-18 · torus 0.14.0 · 8f4da67 · engine 1.3.0zai/glm-5.3-flash+3,704+115,712+$0.0045+24,591
2026-10-18 · torus 0.14.0 · 8f4da67 · engine 1.3.0zai/glm-5.3-10,323+89,280+$0.0077+28,003

zai/glm-5.3-flash

turns

stocktorusreadme-name22agents-doc-headings22register-tool-count22skills-inventory22extension-registration3 ±13 ±1engine-pin-report3 ±03 ±0

tokens in

stocktorusreadme-name685 ±1,0881,156 ±2,889agents-doc-headings2,406 ±1,1043,038 ±3,043register-tool-count81 ±20143 ±9skills-inventory134 ±1195 ±2extension-registration3,857 ±1,4095,847 ±2,467engine-pin-report1,032 ±721,520 ±289

tokens out

stocktorusreadme-name92 ±585 ±3agents-doc-headings400 ±5467 ±8register-tool-count77 ±2892 ±20skills-inventory114 ±6117 ±8extension-registration683 ±23660 ±115engine-pin-report141 ±5136 ±25

cache read

stocktorusreadme-name8,704 ±1,08824,320 ±2,848agents-doc-headings8,640 ±1,10424,320 ±3,040register-tool-count8,70424,320skills-inventory8,70424,320extension-registration16,448 ±3,34445,824 ±8,976engine-pin-report13,568 ±1,20037,376 ±3,408

cache write

stocktorusreadme-name00agents-doc-headings00register-tool-count00skills-inventory00extension-registration00engine-pin-report00

cost

stocktorusreadme-name$0.0004 ±$0.0001$0.0009 ±$0.0003agents-doc-headings$0.0008 ±$0.0001$0.0014 ±$0.0004register-tool-count$0.0003 ±$0.0000$0.0008 ±$0.0000skills-inventory$0.0003 ±$0.0000$0.0008 ±$0.0000extension-registration$0.0014 ±$0.0003$0.0029 ±$0.0004engine-pin-report$0.0006 ±$0.0000$0.0015 ±$0.0001

ms

stocktorusreadme-name11,426 ms ±1,298 ms12,841 ms ±191 msagents-doc-headings12,280 ms ±812 ms17,303 ms ±133 msregister-tool-count8,035 ms ±529 ms12,539 ms ±708 msskills-inventory11,177 ms ±3,618 ms13,136 ms ±892 msextension-registration18,151 ms ±2,182 ms24,919 ms ±3,558 msengine-pin-report11,207 ms ±5,445 ms16,129 ms ±1,538 ms

Δ torus − stock vs noise

taskfieldΔ mednoise (IQR)verdict
readme-nameturns00flat
readme-nametokens in+4715,777within noise
readme-nametokens out-710within noise
readme-namecache read+15,6165,696signal
readme-namecache write00flat
readme-namecost+$0.0005$0.0007within noise
readme-namems+1,4152,595 mswithin noise
agents-doc-headingsturns00flat
agents-doc-headingstokens in+6326,087within noise
agents-doc-headingstokens out+6716signal
agents-doc-headingscache read+15,6806,080signal
agents-doc-headingscache write00flat
agents-doc-headingscost+$0.0006$0.0007within noise
agents-doc-headingsms+5,0231,623 mssignal
register-tool-countturns00flat
register-tool-counttokens in+6241signal
register-tool-counttokens out+1556within noise
register-tool-countcache read+15,6160signal
register-tool-countcache write00flat
register-tool-countcost+$0.0005$0.0000signal
register-tool-countms+4,5041,416 mssignal
skills-inventoryturns00flat
skills-inventorytokens in+614signal
skills-inventorytokens out+316within noise
skills-inventorycache read+15,6160signal
skills-inventorycache write00flat
skills-inventorycost+$0.0005$0.0000signal
skills-inventoryms+1,9597,236 mswithin noise
extension-registrationturns01within noise
extension-registrationtokens in+1,9904,934within noise
extension-registrationtokens out-23231within noise
extension-registrationcache read+29,37617,952signal
extension-registrationcache write00flat
extension-registrationcost+$0.0015$0.0009signal
extension-registrationms+6,7687,115 mswithin noise
engine-pin-reportturns01within noise
engine-pin-reporttokens in+488579within noise
engine-pin-reporttokens out-550within noise
engine-pin-reportcache read+23,8086,816signal
engine-pin-reportcache write00flat
engine-pin-reportcost+$0.0009$0.0002signal
engine-pin-reportms+4,92210,889 mswithin noise

zai/glm-5.3

turns

stocktorusreadme-name22agents-doc-headings3 ±02register-tool-count22skills-inventory22extension-registration22engine-pin-report22

tokens in

stocktorusreadme-name666 ±35960 ±34agents-doc-headings2,586 ±6753,008 ±1register-tool-count104 ±9168 ±9skills-inventory146 ±4192 ±33extension-registration20,705 ±5,1698,958 ±17engine-pin-report846 ±351,444 ±4

tokens out

stocktorusreadme-name68 ±174 ±7agents-doc-headings459 ±39268 ±64register-tool-count117 ±10104 ±2skills-inventory155 ±9138 ±2extension-registration953 ±67716 ±13engine-pin-report105 ±2104 ±5

cache read

stocktorusreadme-name8,64024,320agents-doc-headings13,312 ±1,08824,320register-tool-count8,70424,320skills-inventory8,70424,320 ±32extension-registration8,704 ±5,16824,384 ±16engine-pin-report8,704 ±1624,384

cache write

stocktorusreadme-name00agents-doc-headings00register-tool-count00skills-inventory00extension-registration00engine-pin-report00

cost

stocktorusreadme-name$0.0035 ±$0.0001$0.0079 ±$0.0000agents-doc-headings$0.0091 ±$0.0011$0.0117 ±$0.0003register-tool-count$0.0029 ±$0.0001$0.0070 ±$0.0000skills-inventory$0.0031 ±$0.0000$0.0072 ±$0.0000extension-registration$0.0344 ±$0.0059$0.0220 ±$0.0000engine-pin-report$0.0039 ±$0.0000$0.0088 ±$0.0000

ms

stocktorusreadme-name5,616 ms ±859 ms12,400 ms ±589 msagents-doc-headings11,020 ms ±243 ms14,598 ms ±575 msregister-tool-count6,014 ms ±696 ms12,227 ms ±563 msskills-inventory8,704 ms ±1,383 ms17,675 ms ±1,887 msextension-registration16,342 ms ±2,280 ms19,717 ms ±1,603 msengine-pin-report13,436 ms ±2,515 ms12,518 ms ±1,838 ms

Δ torus − stock vs noise

taskfieldΔ mednoise (IQR)verdict
readme-nameturns00flat
readme-nametokens in+29470signal
readme-nametokens out+614within noise
readme-namecache read+15,6800signal
readme-namecache write00flat
readme-namecost+$0.0044$0.0001signal
readme-namems+6,7841,719 mssignal
agents-doc-headingsturns-11signal
agents-doc-headingstokens in+4221,351within noise
agents-doc-headingstokens out-191127signal
agents-doc-headingscache read+11,0082,176signal
agents-doc-headingscache write00flat
agents-doc-headingscost+$0.0026$0.0021signal
agents-doc-headingsms+3,5781,149 mssignal
register-tool-countturns00flat
register-tool-counttokens in+6419signal
register-tool-counttokens out-1321within noise
register-tool-countcache read+15,6160signal
register-tool-countcache write00flat
register-tool-countcost+$0.0041$0.0001signal
register-tool-countms+6,2131,391 mssignal
skills-inventoryturns00flat
skills-inventorytokens in+4666within noise
skills-inventorytokens out-1718within noise
skills-inventorycache read+15,61664signal
skills-inventorycache write00flat
skills-inventorycost+$0.0040$0.0001signal
skills-inventoryms+8,9713,774 mssignal
extension-registrationturns00flat
extension-registrationtokens in-11,74710,339signal
extension-registrationtokens out-237135signal
extension-registrationcache read+15,68010,336signal
extension-registrationcache write00flat
extension-registrationcost-$0.0123$0.0118signal
extension-registrationms+3,3754,560 mswithin noise
engine-pin-reportturns00flat
engine-pin-reporttokens in+59869signal
engine-pin-reporttokens out-110within noise
engine-pin-reportcache read+15,68032signal
engine-pin-reportcache write00flat
engine-pin-reportcost+$0.0049$0.0001signal
engine-pin-reportms-9185,029 mswithin noise

Method

  1. Fixed, read-only, repo-local task set in three difficulty tiers (trivial / aggregate / reason); identical prompts run through both configs, so any delta comes from the harness layer.
  2. Two configs: stock (bare engine, JSON mode) vs torus (same argv plus the canonical child extension set — byte-for-byte the spawn a delegation receives).
  3. Warmup discarded: one warmup run per task × config runs first and is excluded from statistics (steady-state prompt cache, warm extension load path).
  4. Interleaved rotation: measured passes alternate config order (stock, torus / torus, stock / …) so provider-side prompt-cache drift cancels across configs.
  5. Medians with spread: cells are medians ± half the interquartile range over measured runs; a delta is labeled signal only when it exceeds the larger config's IQR.
  6. Shared reducer: usage and cost fold out of the JSON event stream via the production reduceEngineEvent pipeline.

Caveats. Cache read/write deltas remain indicative: provider prompt caches can survive between runs, and torus's larger declaration surface inflates fresh-cache writes by construction. Tokens-in and cost deltas are the stable signals. Wall-clock includes extension loading at spawn. Numbers are self-reported by the engine per run.