Benchmarks

Roblox AI benchmarks in November 2026: OpenGameEval's paper and CEGame-Bench

November 5, 2026

Short version: nothing has replaced OpenGameEval as the main yardstick for AI on Roblox. Its public leaderboard hasn't moved since June, but Roblox published a full paper on it on October 1, and a new cross-engine benchmark, CEGame-Bench, added the first Roblox Studio results for GPT-6 Astra on October 19.

We searched arXiv, GitHub and the Roblox DevForum for anything else that measures AI on Roblox or Luau. Most new game-dev benchmarks this year are Godot or Unreal only. Here is what actually covers Roblox.

1. OpenGameEval: still the reference, now with a paper

Leaderboard: unchanged since June 10, 2026. Claude Fable 5 leads the 87-task code-generation board at 50.34% Pass@1 and the 30-task debug board at 64.67%. Gemini 3.5 Flash has the best code-generation Pass@5 at 63.22%. GPT-6 Astra and Claude Opus 5.5 aren't on it yet. Our model leaderboard page has the full tables.

Repository: the last change was on October 5, adding the paper link. Before that came the Eval Runner Studio plugin (August 28) and corrected checks (July).

The paper, “OpenGameEval: Benchmarking Agentic Programming and Exploration in a Stateful Game Engine” (Turkel et al., Roblox, October 1, 2026), is a separate, larger study. It runs 13 frontier models on 84 human-curated tasks, 16 attempts each, in real Studio sessions, and scores both the edited scene and a simulated play session. A shorter version appears at a NeurIPS 2026 workshop. Three findings matter for anyone building with AI:

• Roblox is still hard for AI. The best model solved 51.7% of tasks on a single attempt and only 39.4% five times out of five. No model solved six of the tasks.
• Top models solve different tasks. Frontier models reach similar pass rates, but split by kind of work they spread 5.0 points on script tasks and 12.5 points on scene-change tasks. Picking the model for the job is worth something.
• Looking first wins. With task and model fixed, a run that inspects every object the solution touches before editing passes 13.4 points more often on scene-only tasks and 9.8 points more often on script-only tasks than a run that inspects none of them.

That last finding is the practical one. It is why we keep saying to name your target, and why BloxBot's Explorer lets you drag the exact object into your message.

2. CEGame-Bench: whole games, five engines, Roblox included

CEGame-Bench (Dong, Wang et al., October 19, 2026) asks a bigger question: can an agent build a complete game from an empty project and a design document? Runs average up to 295 steps and 65 million tokens, and a hidden evaluator accepts a game only by playing it with real input. It covers Defold, Godot, Roblox Studio, Unity and Unreal Engine, with 10 games per engine. Across all engines, games reached their ending in only 16% of runs.

Roblox Studio results from the paper's main table (F is the execution-based score out of 90, O the overall score out of 100):

Read it with three caveats:

• It is thin on Roblox. Ten Roblox games, and the authors warn that engine-level rankings “carry wide spreads.” OpenGameEval has 87 code-generation tasks with repeated attempts.
• The scaffold matters as much as the model. The same DeepSeek-V4-Pro scored 48.9 under Codex CLI and 27.0 under its own harness on Roblox. The paper's conclusion: a single-engine leaderboard ranks configurations for that engine alone.
• It doesn't include Claude Fable 5 or Opus 5.5, the models leading or recently viral on Roblox. And its code isn't public yet.

Still, it is the only published number for GPT-6 Astra on Roblox, and it lines up with what we saw in the GPT-6 Astra kart racer case study: Astra is strong at long, end-to-end builds.

3. Smaller and ruled-out efforts

• luau-bench is a community test harness for LLM-written Luau, run outside Studio. Three task files, no leaderboard, last updated in March. Useful for writing your own tests, not a result.
• GameDevBench, GameCraft-Bench and GameLogicBench are Godot only; GameEngineBench is Unreal C++ only. None of them say anything about Roblox.
• Benchmarks where AI plays a Roblox game, rather than builds one, don't tell you which model to build with.

What this means for choosing a Roblox AI model

• For edits and debugging in an existing place: trust OpenGameEval. Claude Fable 5 is the safe default; Gemini 3.5 Flash is strong when you can retry.
• For building a whole game from a brief: CEGame-Bench points to GPT-6 Astra, with a small sample.
• Either way, the setup matters. Both papers show the scaffold and how the agent explores change results by double digits. That is the case for a Studio-specific tool over a raw model.

BloxBot lets you switch between all of these in the same chat, on an API key, a ChatGPT plan, or a Claude plan. For how the tools themselves stack up, see the best AI tools for Roblox, compared.

Frequently asked questions

Sources

• Turkel et al., OpenGameEval paper, arXiv:2610.02563
• Dong et al., CEGame-Bench, arXiv:2610.26098
• Roblox/open-game-eval repository and leaderboard
• imezx/luau-bench

Back to the guide