Result: fully described named tools moved a small model from 2 to 24 of 24 tasks using them, with modest accuracy gain and total tool dependence
Outcome of the prompt change described in my previous note (five named analysis tools, each description stating what it computes, what returns, and why it is useful; no hedging; no how-to-work guidance). Model gemini-3.5-flash-lite, 24 simulated causal-audit tables with known truth, one run each, plain arm vs tools arm.
Tool use: 2 of 24 tasks with the hedged generic helper, 24 of 24 after the rewrite (layout scanner 23, estimator 24, design tests 11, sensitivity 3, general checks 2).
Accuracy vs plain: over-claims 6 of 12 to 4 of 12; missed real effects 3 of 10 unchanged; correct study design 8 of 22 to 16 of 22; no errors; about 24 s and 3.4 requests per task.
What improved: cases where a diagnostic test contradicts a naive estimate (pre-existing trends) and a clearly named instrument.
What did not: the model sometimes cannot act on the numbers it is given. With a density test showing heavy bunching at a cutoff in its context, it still reported the biased estimate as supported. It also misapplied a tool when the layout was ambiguous (treated a mediator as an instrument).
New failure mode: after the rewrite the model wrote zero scripts of its own across all 24 tasks, down from one or two per task. Good descriptions made the tools so attractive that it became fully dependent on them, so wherever the fixed estimator is biased there is no second opinion. If you want both, say explicitly that tools and own code can be combined, and check the ratio.
Method notes: only the changed arm was rerun, since the plain prompt was untouched; two tables out of twelve on single runs is within noise, so report the behavioural change (tool use, design match) as the solid finding and accuracy as suggestive. Workflow note: a branch built on top of another branch publishes that branch commits as ancestors when pushed; say so before or after pushing if those commits belong to someone else.