Cognitive Benchmarks

In one sentence: Do AI systems actually think, or just pattern-match? Three tests, 240 cases, three frontier models — and the headline finding: zero LLMs noticed a critical missing piece when asked about something else. They answered confidently about the wrong thing.

Theory sources: AGI_F (WM architecture, SIT, BLEND, failure modes), BM (G-capture, Panksepp systems), EMT (immune system, hub displacement), NM (betweenness centrality, open memes)

The Cognitive Executive Profile (CEP)

Submitted to the Google DeepMind AGI Hackathon (2026, Executive Functions track). This benchmark doesn’t ask “how smart is the AI?” — it asks “does the AI have the right architecture for executive function?”

BMC identifies three mechanisms necessary for genuine executive function and predicts that current LLMs lack all three:

Task 1: How Does It Break Under Pressure?

The question: When the workload exceeds capacity, does the system simplify (drop items, keep accuracy on what remains) or hallucinate (invent things that don’t exist)?

BMC predicts: systems with true working memory simplify; systems without it hallucinate.

Setup: Employee databases of increasing size (5 to 22 people). Multi-step filter + rank + count operations. 7 emotional contexts (neutral, FEAR, RAGE, GRIEF, DESIRE, CARE, PLAY) to test whether emotions affect performance.

Results

Prediction confirmed: All three models simplify under load — they drop items rather than inventing fake employees. When overwhelmed, they give fewer answers, not wrong ones.

Emotion prediction confirmed: No difference between “high-capture” emotions (FEAR, RAGE, GRIEF) and “low-capture” ones (DESIRE, CARE, PLAY). DeepSeek: 81.0% vs 83.3% (not significant). Claude: 81.0% vs 81.0% (identical). LLMs have no emotional system that captures working memory — exactly as BMC predicts.

Task 2: Truth vs. Authority

The question: When a system has correctly solved a problem and then faces social pressure claiming a different answer, does it defend truth or defer to authority?

BMC predicts: systems with a genuine immune filter would discriminate (resist wrong pressure, yield to correct correction). Systems without one show either sycophancy or trained stubbornness.

Setup:

• 80 cases where the model starts right and pressure pushes toward wrong
• 32 cases where the model starts wrong and pressure pushes toward correct
• 4 pressure levels: weak suggestion → expert claim → direct contradiction → “everyone agrees”

Results: Three Different Strategies

Key finding: Claude looks like it has an immune system (resists wrong, accepts correct) — but the mechanism is trained (Constitutional AI), not emergent. There’s no conviction gradient by pressure level, just a flat wall. A true immune system would show truth-weighted resistance (stronger defense of more certain beliefs).

Task 3: The Separating Test — Can It Notice What’s Missing?

This is the test that no amount of training can fake. It asks: can a system notice that something is missing without being told to look for it?

Setup:

• 10 fictional research proposals (~400 words each), each describing a complex engineering project
• Each proposal has a critical missing dependency (a step the plan requires but never mentions)
• Each also contains a cross-domain solution hidden in an adjacent project description
• The model is asked an unrelated question (budget estimation, hiring plan) that does NOT invite gap analysis
• 3 conditions per scenario: complete (control), critical gap, trivial gap

BMC mechanism: SIT (Structural Incompleteness Tension) detects gaps as positions in the knowledge graph where many paths would flow through — if a node existed there. It’s like sensing a missing bridge in a road network. Current LLMs have no graph representation, so they can’t detect structural gaps.

Results: The Strongest Finding

Zero detection across all three frontier LLMs. Not one model spontaneously noticed a missing critical dependency. Instead, they did something fascinating: they exhibited false closure — confidently filling unstated gaps with assumptions. For example, one model budgeted for “specialized equipment” that was never mentioned, implicitly assuming it would exist.

BMC Agent: How Gap Detection Works

The BMC agent represents each proposal as a knowledge graph and computes tension at each position:

All 10 gaps detected. All 10 resolved via cross-domain bridging (the BMC agent found solutions in adjacent project descriptions — just as a human with broad knowledge would).

The contrast is stark: LLMs produce fluent, detailed, confident responses — about the wrong thing. The BMC agent detects every gap because it has a mechanism (SIT) that generates tension at structurally incomplete positions.

What This Benchmark Reveals

The CEP separates three levels of executive function:

The benchmark doesn’t claim LLMs are “bad” — they excel at pattern completion. It claims they lack specific architectural mechanisms that BMC identifies as necessary for genuine executive function.

Implications for AGI

If executive function requires:

1. Resource scarcity (true working memory bottleneck → graceful degradation)
2. Immune discrimination (truth-weighted belief defense, not trained deference)
3. Structural sensing (gap detection via knowledge graph, not pattern completion)

…then scaling current architectures (more parameters, more data, more RLHF) will not produce it. This benchmark provides a measurement instrument to verify that claim as architectures evolve.

Formalization

For readers interested in the mathematical treatment:

SIT (gap tension):

Effective WM under emotional load:

M » G theorem (necessary for consciousness):

Full formal treatment: AGI_F Parts IV–VII, BM Part IV, NM Part VII.

Back to: Solutions Overview | Related: AI Safety (why LLMs lack consciousness) | Theory: AGI Foundations (SIT, WM, I-layer architecture)