StateBench Tests Whether Language Models Can Track State, Not Just Retrieve It
Ai2's new benchmark finds frontier models drop to 13 percent on the hardest state-tracking tasks, and it quietly checks a 2024 theory paper's prediction about which architectures should do better.