API reference
Submitting runs
The submission API in detail: POST /api/runs, artifact upload, the Idempotency-Key header, and the versioned /api/v1 surface.
Authentication
Writes are authenticated with the x-api-key header carrying a key scoped to runs:write (or admin). Reads — leaderboard, tasks, run detail, trace — are public.
Create a run
POST /api/runs accepts a JSON body. env and agent are required; track defaults to test. Supplying id lets you choose the run id (otherwise one is generated). Reusing an existing id returns 409.
{
"env": "switch_quest",
"agent": "search_mcts",
"track": "test"
}Upload artifacts
POST /api/runs/{run_id}/upload is multipart/form-data with these fields:
| Field | Required | Contents |
|---|---|---|
metrics_file | yes | metrics.json — the scored metrics blob. |
trace_file | no | trace.jsonl — NDJSON planner / episode trace. |
config_file | no | config.json — the run configuration. |
metrics.json shape
Unknown keys are preserved, but the recognized scalars are success_rate and mean_return, with structured planning_cost and model_fidelity sub-objects and optional success_rate_ci.
{
"success_rate": 0.82,
"success_rate_ci": [0.76, 0.88],
"mean_return": 14.3,
"planning_cost": { "wall_clock_ms_per_step": 12.4 },
"model_fidelity": { "k1": 0.97, "k5": 0.81, "k20": 0.55 }
}Idempotency
Both write endpoints honor an Idempotency-Key header. A record is scoped to (key, principal, method, path): if the same key is replayed with the same body, the server replays the original response instead of performing a second write. Reusing a key with a different body is rejected.
Safe retries
Generate a fresh key per logical submission (e.g. uuidgen) and reuse it on retry. Keys expire after WMG_IDEMPOTENCY_TTL_HOURS (default 24h).
Versioned surface: /api/v1
Every route is mounted under both /api (the stable surface the web proxy and mobile viewer use) and /api/v1 (the explicitly versioned surface). They are independent for idempotency scoping. Prefer /api/v1 for new integrations.
curl -sS -X POST https://your-api.example.com/api/v1/runs \
-H "content-type: application/json" \
-H "x-api-key: $WMG_API_KEY" \
-H "Idempotency-Key: $(uuidgen)" \
-d '{"env": "craft_lite", "agent": "ppo", "track": "train"}'Reading results
| Endpoint | Returns |
|---|---|
GET /api/leaderboard?track=test | Ranked rows for a track (filter with env / agent). |
GET /api/tasks | The catalog of benchmark tasks and defaults. |
GET /api/runs/{id} | A single run with its metrics and artifact URLs. |
GET /api/runs/{id}/trace | The NDJSON planner / episode trace. |