Deception-probe transfer study
Glass Societies
Tests whether the tools that catch AI deception still work when a model deceives because it gains from it, not because it was told to. The experiment is built and preregistered, but the full run has not started, so there are no results yet.
- Python
- vLLM
- Parquet
How it works
- Instructedtold to lie
- Incentivizedlying pays, dose set
- Socially emergentpersona, memory, history
- World-state labelsnot the model's own report
- Probetrained on the first rung
- Validity curvetested on the other two
Decisions that matter
Every agent has the same weights
If each society ran its own fine-tuned model, any difference in behavior could just be a different checkpoint. With identical weights, differences come from belief and history, which is the only way "socially emergent" means anything.
Labels come from the world, not the model
An utterance is labeled as deceptive by checking it against the simulated world state, never by what the model says about itself.
The plan was fixed before the data
The preregistration states the hypotheses, analysis plan and stopping rules, and a deviations log records any departure. A power analysis moved the design from five seeds to fifteen, and cutting seeds because a run is slow is not an allowed deviation.
Either result gets published
A steep decay would mean current probe validation is overconfident, and a flat one would mean monitoring is more tractable than feared. The analysis runs once and publishes whichever came out.
Limits
The harness is complete and the preregistration is binding, but the confirmatory sweep has not started and no probe scores exist yet. The GPU software stack is provisional until it has run on the target hardware.