WWorldModel GymResearch Benchmark

Release notes

Changelog

How WorldModel Gym evolved — from the first benchmark cut to a hardened, scalable platform with an editorial dashboard. Newest releases first.

Unreleased

In progress

On the bench

Pending release metadata and follow-up benchmark extensions in active development.

Added
  • Release-notes and About pages, wired into the footer and sitemap.
  • Continued work on benchmark coverage and additional planner baselines.

0.6.0

Latest

Editorial UI overhaul & iconic features

A full front-end redesign around an editorial design system, plus the marquee features that make the dashboard feel like a product.

Added
  • A cohesive design system: tokens in globals.css, Tailwind class-based dark mode, and reusable UI primitives (Button, Card, Badge, Stat, Section).
  • Command palette (⌘K) for keyboard-first navigation across tasks, leaderboard, and docs.
  • Run comparison view and an interactive episode-trace player for inspecting agent behavior.
  • Generated Open Graph / Twitter images, a keyboard-shortcuts dialog, and an in-app docs site.
  • Scroll-reveal motion, sparklines, and on-brand environment glyphs, all theme-aware.
Changed
  • Reworked navigation and page layouts into a calmer, editorial information hierarchy.
  • Adopted a serif display + mono body type pairing and reduced decorative box framing.
Fixed
  • Tightened route coverage and resolved site interaction and spacing inconsistencies.
  • Aligned end-to-end smoke tests with the redesigned UI.

0.5.0

API contract, pagination & tracing

Production-grade API surface: stable contracts, cursor pagination, and end-to-end observability.

Added
  • OpenTelemetry tracing across the request lifecycle for distributed debugging.
  • Cursor-based pagination on list endpoints for stable, scalable result paging.
  • ESLint and stricter typing on the web client for a consistent contract surface.
Changed
  • Normalized proxied API paths so the browser client and server share one base.
  • Hardened the API client with zod-validated request and response schemas.

0.4.0

Testing & supply-chain hardening

Confidence in every release: automated tests, CI/CD, and a locked-down dependency supply chain.

Added
  • Unit and end-to-end test suites with Playwright smoke coverage in CI.
  • Continuous integration pipeline gating lint, type-check, tests, and build.
  • Pre-commit tooling and reproducible dependency pinning.
Fixed
  • Stabilized flaky Playwright startup and Python fallback paths in CI.
Security
  • Supply-chain hardening: pinned and audited dependencies to reduce drift and exposure.

0.3.0

Scalability: stateless API & async jobs

Built to scale horizontally — a stateless API tier and an asynchronous job queue for heavy evaluations.

Added
  • Asynchronous job tier so long-running evaluations no longer block the request path.
  • Per-run budget controls and a sliding-window rate limiter to protect shared capacity.
Changed
  • Made the API stateless so instances can scale out behind a load balancer.
  • Moved durable run and artifact state out of process memory into shared storage.

0.2.0

Reliability, data integrity & honest metrics

Production defaults plus a scientific-honesty pass that made the leaderboard trustworthy.

Changed
  • Corrected benchmark metric definitions so the leaderboard reflects honest, reproducible scores.
  • Switched to durable storage with data-integrity guarantees for runs and artifacts.
  • Promoted production-ready defaults across the server and deployment configuration.
Fixed
  • Repaired world-model training gradients and removed demo rows from the public leaderboard.
  • Cleaned backend startup and database migration paths for reliable deploys.
Security
  • Introduced secure API-key authentication with bootstrap key creation on startup.
  • Added rate limiting to guard submission and query endpoints.

0.1.0

Initial benchmark platform

The first public cut: environments, evaluation harness, submission API, and dashboards.

Added
  • Monorepo scaffold spanning core, agents, planners, world models, and the server.
  • Three procedural long-horizon environments: MemoryMaze, SwitchQuest, and CraftLite.
  • A deterministic evaluation harness, stable episode-trace schema, CLI runner, and continual-track metrics.
  • Planner baselines (MCTS, MPC-CEM, trajectory sampling) and world-model baselines (deterministic, stochastic, ensemble).
  • A FastAPI submission service with runs CRUD, artifact uploads, leaderboard queries, and trace downloads.
  • A Next.js dashboard (home, tasks, leaderboard, run viewer) and an Expo mobile viewer.
  • Docker Compose stack, demo upload script, CI workflow, and the paper artifacts.