Back to methodology overview

How the verdicts are generated

Every Yes/No verdict on this site — for each theory of consciousness, against each of the seven questions — is produced by the same disciplined, multi-agent AI pipeline described here. This page is the canonical, replicable record of that method, so anyone can see exactly how the AI-generated verdicts were arrived at and reproduce them.

The framework being applied

The seven questions are taken from:

Ohmura, Y. & Kuniyoshi, Y. (2026). A Minimal Set of Questions for Theories of Consciousness: Toward a Unified Explanatory Framework.

The questions are treated as necessary conditions. A theory is complete only if it can answer "Yes" to all seven; a "No" is not a mark against the theory's worth — it identifies a specific, well-defined explanatory gap. Verdicts are strictly binary: Yes or No, never a partial score. (The paper's own illustrative model, the Dual-Laws Model, is not given special standing here — it is judged by the same bar as every other theory.)

The model

| | | |---|---| | Model | Claude Opus 4.8 (claude-opus-4-8) | | Rubric version | 7.0-minimal-questions | | Theories evaluated | 11 | | Questions | 7 | | Cells | 77 (11 × 7) |

Every agent in the pipeline runs on the same model. The model is recorded with each run; if a future run uses a different model, that is noted in the version history below.

The grading standard

A single strict, hard-problem-honest rubric is applied to every cell:

Answer Yes only if the theory, as actually formulated by its proponents, provides a genuine, mechanistic account that addresses the question as written — not a vague gesture, not a neural correlate standing in for an explanation, not a promissory note about future work. When a theory merely identifies when or where consciousness occurs without explaining the asked-for how/why, the verdict is No. Under genuine uncertainty, default to No: a Yes must be defensible directly from the theory's explicit commitments. Do not grade on ambition, elegance, or popularity.

Because the bar is deliberately strict, most theories answer No to most questions. That is the point: the exercise exposes real gaps rather than awarding partial credit.

The four-phase pipeline

The run is a deterministic, multi-agent workflow. Each box is an independent AI agent; agents within a phase run in parallel.

Phase 1 · DOSSIERS    11 agents — one per theory
   └─ web-research a factual dossier: core mechanism + what the theory
      actually says about each of the seven questions + known limitations + sources

Phase 2 · JUDGE       77 agents — one per (theory × question) cell
   └─ apply the strict rubric + that question's YES-bar to one theory,
      using its dossier; return verdict + confidence + justification +
      key evidence + what-would-flip-it

Phase 3 · CALIBRATE    7 agents — one per question
   └─ review all 11 verdicts for a question side by side and enforce ONE
      uniform bar across theories, overriding any cell that was stricter
      or looser than its peers

Phase 4 · VERIFY       1 agent per flagged cell
   └─ an independent skeptic tries to REFUTE every verdict that was changed
      in calibration or marked low-confidence; the verdict stays or flips

Why four phases:

  • Dossiers keep judging grounded in each theory's primary literature, not the model's general impressions.
  • Per-cell judging isolates one theory × one question per agent, so no cell's reasoning bleeds into another's.
  • Calibration fixes the main failure mode of independent judges — cross-agent drift, where the effective bar wanders from theory to theory.
  • Adversarial verification stress-tests exactly the cells most likely to be wrong (changed or low-confidence), defaulting to refutation.

The per-question YES-bars

Each question carries an explicit bar stating what earns a Yes. These are the bars applied in the current rubric (7.0-minimal-questions):

  1. Phenomena — How can subjective experience be accounted for in physical or computational systems? — Yes only with a causal generative mechanism for why experience arises (engaging the hard problem). Correlates or conditions, without a why, are No. Bare panpsychist posits do not count as a mechanism.
  2. Self — Why does the subject of experience coincide with the initiator of action? — Yes only if it explains why the experiencer and the agent are the same entity (distinguishing "I" from "my body"), without a homunculus or infinite regress.
  3. Causation — Does consciousness have causal efficacy within a system, beyond mere correlations or predictability? — Yes only if consciousness is non-epiphenomenal and true causation is distinguished from mere correlation or predictability (apparent inter-level causation detected only via predictability measures does not count).
  4. State — How can differences in levels or states of consciousness be explained? — Yes only with intrinsic internal mechanisms that control conscious level (e.g. wakefulness vs non-REM sleep vs anesthesia), not mere external correlation.
  5. Function — What functional or cognitive roles are associated with consciousness? — Yes only with a specific account of which roles (e.g. semantic integration, association, cognitive control) and why they depend on consciousness.
  6. Contents — How can the diversity, structure, and organization of conscious contents be explained? — Yes only if it explains structure/organization — integration into unified representations and selective segmentation (e.g. binocular-rivalry switches) — not diversity alone.
  7. Universality — Can the theory be applied across different types of systems, including artificial systems? — Yes if the form of explanation applies across systems wherever the relevant causal/functional structures are present. This is not a demand for substrate-independence: a theory may hold that implementation matters and that not all systems are conscious.

Structured outputs

Every agent returns a validated JSON object (no free-text parsing), which is why the verdicts are uniform and machine-importable:

  • Dossier: mechanism, q1…q7 notes, limitations, sources[]
  • Verdict (per cell): verdict (yes/no), confidence (high/med/low), justification, key_evidence, what_would_flip_it
  • Calibration (per question): for every theory — verdict, changed, confidence, note
  • Verification (per flagged cell): final_verdict, refuted, reasoning

Fairness guarantees

  • Identical rubric and prompt template for every theory — only the theory's name and researched dossier change.
  • The same seven questions asked of every theory.
  • The same binary Yes/No standard applied to all, with a single calibrated bar per question.
  • No theory-specific modifications to prompts or criteria.
  • Source-grounded: verdicts are argued from each theory's own literature, not general knowledge.
  • Community-revisable: every verdict is AI-generated and can be challenged, refined, or overridden through the normal edit flow.

Reproducing a run

The pipeline lives in the repository and is meant to be re-run:

| Artifact | Path | |---|---| | Workflow (4-phase pipeline, self-contained) | scripts/verdicts/regenerate-verdicts.workflow.js | | Theory roster evaluated | scripts/verdicts/theories.json | | Importer (JSON → answers table) | scripts/verdicts/import-verdicts.ts | | This methodology | docs/verdict-methodology.md |

Steps

  1. From a Claude Code session on the recorded model (Claude Opus 4.8), run the workflow with the Workflow tool: Workflow({ scriptPath: "scripts/verdicts/regenerate-verdicts.workflow.js" }). To re-judge only some questions, pass args: { questions: ["q3","q6","q7"] } (dossiers still cover all seven for context).
  2. Save the returned { verdicts, dossiers } JSON to Fixes_late/verdicts-<date>.json as the immutable backup of that run.
  3. Import into the database: npx tsx scripts/verdicts/import-verdicts.ts --file Fixes_late/verdicts-<date>.json (add --dry-run first to preview; requires TURSO_DATABASE_URL in .env.local).
  4. Update src/lib/ai-judge-config.ts (lastEvaluated, rubricVersion, modelName) and add a row to the version history below.

Same model + same questions + same theories ⇒ a faithful reproduction; the only intended sources of variation are model sampling and refreshed source literature.

Version history

| Rubric | Date | Model | Change | |---|---|---|---| | 7.0-minimal-questions | 2026-06-27 | Claude Opus 4.8 | Adopted the published Minimal Set of Questions wording. Re-tuned the Q3 (causation vs. predictability), Q6 (structure/organization, not just diversity), and Q7 (cross-system applicability, not substrate-independence) bars, and regenerated all 77 verdicts. | | 6.0-dual-laws | 2026-06-05 | Claude Opus 4.8 | First strict, hard-problem-honest run of the four-phase pipeline over 11 theories × 7 questions. | | 5.0 | 2026 | — | Replaced 1–5 dimensional radar scoring with the seven necessary conditions, answered strictly Yes/No. |