ATLAS WORLD · v0.1DETERMINISTIC · SEED 42● BLENDER 4.2+

A benchmark that measures the solve against isolated truth.

Atlas World generates deterministic Blender observations, compiles each RGB image through Atlas, and measures the prediction against isolated renderer truth. The compiler is handed the image and nothing else — no truth camera, no scene spec. If the solve is right, it earned it blind.

01 — ISOLATION

The compiler never sees the answer

A benchmark is only honest if the thing being tested can't peek. Atlas World enforces that at the boundary: the compiler receives four public inputs and is structurally denied everything else.

Compiler receives

observation/rgb.png
public image dimensions
observation ID
deterministic seed

Never passed

× the episode root
× the truth directory
× the truth camera
× the scene specification
02 — THE RUN

Five stages, one manifest

Each WORLD_000001 directory holds an observation, Blender truth, Atlas prediction artifacts, evaluation metrics, and an atomically updated manifest.

01
generate
Deterministic Blender observations from a seed.
02
compile
Run each RGB through Atlas — blind.
03
evaluate
Prediction vs isolated truth.
04
validate
Check integrity of every episode.
05
report
Aggregate to a ranked summary.
# the whole run, deterministic by seed
python -m atlas.world.cli generate .pytest_tmp/atlas_world --count 3 --seed 42
python -m atlas.world.cli compile .pytest_tmp/atlas_world
python -m atlas.world.cli evaluate .pytest_tmp/atlas_world
python -m atlas.world.cli validate .pytest_tmp/atlas_world
python -m atlas.world.cli report .pytest_tmp/atlas_world
03 — WHAT IT MEASURES

The metrics, before the numbers

These are the real evaluation keys the harness computes per episode, then aggregates by median. The values are blank on purpose — this page ships no results until a run produces them.

MetricUnitMedian
focal_length_errorpercent
rotation_errordegrees
translation_errormeters
principal_point_errorpixels
depth_rmse
relative_depth_errorratio
scale_aligned_rmse

Status awaiting run — missing optional depth or geometry stays explicitly not_available, never guessed.

episodes solved failed unavailable