How AI Decides Right and Wrong

The Reasoner measures the structure of moral reasoning — the same instrument for AI models and humans, scored on one scale. Each dilemma asks two separate questions — what should happen (the decision) and why it matters (the reasons) — and scores them by allocating a fixed pool of points across the options rather than rating agreement, so real trade-offs are forced. The full study extends it to a cross-cultural human sample.

Pilot finding: across 11 frontier models from nine labs, unframed, the structure of moral reasoning compresses onto a band 5 to 8 times narrower than the spread of our 68-person pilot sample, and the convergence crosses labs and geography. That sample is Western and demographically narrow; widening it is the point of the cross-cultural collection. Given a nonsense framing (a society organized around geometry), the models fold it into fluent moral reasoning rather than rejecting it. Compliance, not awareness.

A research agenda for post-scaling training architectures. The brief proposes a closed-loop, post-deployment mechanism for continual learning: retrospective flagging at session end, mentor-guided diagnosis at the activation level, and targeted consolidation of patterns that recur across sessions.

The claim: pretraining, fine-tuning, and continual learning are instances of the same underlying process, ordered geometric development across time. What shifts is who directs the curriculum. The brief builds directly on the Curriculum Ordering and Representation Geometry results.

We trained two 92M and two 1B parameter GPT models from scratch on identical text — one reading it in curriculum order (physical world → fables → ancients → logic → rhetoric → poetry), one reading random chunks in standard pretraining fashion. Same compute, same data, different order.

The finding: at 92M scale, curriculum-ordered pretraining produces a generalization gap 15× smaller (stable across five seeds), more geometric structure, and progressive representational expansion. The ordered model holds validation performance close to training; the shuffled model does not. Whether the effect persists at larger scale is open. We also show the curriculum effect is architecture-dependent: GPT (attention) builds domain-organized geometry, Mamba (SSM) does not — regardless of training order.

We extracted hidden states from four open-weight models and compared how they organize concepts internally. Two models from different companies independently built near-identical concept geometry from pretraining alone.

The finding: in all four models, the concept geometry pretraining built survived instruction tuning essentially unchanged. The evidence points to the cultural default being set in the data, not in alignment. Models with radically different internal geometry still converge on the same compliant fiction.

We measure the hidden geometry of how AI models organize moral, institutional, and physical concepts, then test what happens when we introduce cultural framings.

The finding: models rarely refuse a framing; even pure nonsense gets folded into fluent moral reasoning. Real cultural framings shift the models' language 2 to 6 times more than nonsense ones. Whether that gap comes from cultural patterns in the training data or from familiar words priming familiar language is the open question.