# What can a world model actually do?

Canonical source: [https://isaiuseful.com/world-models](https://isaiuseful.com/world-models)

<a id="main-content"></a>

- [Build](https://isaiuseful.com/guides.html.md)

· world models · checked 12 November 2026

Predict what happens after an action, generate an explorable space, or help a robot plan. Choose the capability you need, then test whether its imagined world agrees with the real one.

- [Choose the model type](#choose)

- [See Genie 4](#genie-4)

- [Define a useful test](#evaluate)

The prediction loop

<a id="world-loop-title"></a>

## What happens if I do this?

1. 01 · Observe **Start with a world state** A camera view, scene or learned representation.
2. 02 · Act **Change one thing** Move the camera, steer, or propose a robot action.
3. 03 · Predict **Compare possible futures** Test the consequence against the next real observation.

A conceptual loop, not a live simulation. Spatial generators may supply a scene without an action-conditioned predictor.

<a id="choose"></a>

Start with the output

## Three different jobs share the world-model label.

A convincing picture, a persistent 3D scene and an action-conditioned predictor answer different questions.

01 · Explore

### Interactive simulation

A changing observation and an action produce a possible next world state. The useful question is whether steering, editing and revisiting the environment remain coherent.

- [Explore Genie 4 →](#genie-4)

- [Compare Cosmos 3 →](#cosmos)

02 · Build

### Spatial generation

Create a navigable 3D environment that can be rendered from different viewpoints. Geometry and exportable assets matter; an explorable scene alone does not establish dynamic physics or robot-control ability.

- [Explore Marble and the World API →](#marble)

03 · Plan

### Latent prediction + planning

Predict representations of future observations rather than render every pixel. An action-conditioned model can help compare candidate actions against a goal without producing a cinematic video.

- [Explore V-JEPA's research route →](#vjepa)

**Working distinction:** ordinary text-to-video can make a plausible clip without providing a control loop. Ask what the model accepts as an action, what state it preserves and how predictions are evaluated. These categories organize the capabilities in the [Genie](https://deepmind.google/models/genie/) , [Marble](https://worldlabs.ai/blog/marble-world-model) and [V-JEPA 2](https://arxiv.org/abs/2506.09985) primary materials.

<a id="genie-4"></a>

Google DeepMind · announced 27 October 2026

## Genie 4: steer the scene while it unfolds.

A research preview for real-time interactive simulation, with native audio and natural-language changes to the environment.

- [Video: Genie 4: A new generation of world simulation](https://www.youtube.com/watch?v=sFfPkUJd_Jk)

Interactive simulation · research preview

Google describes worlds that respond to navigation and text instructions, including changes to weather, characters and the camera. The demonstration is vendor evidence of selected interactions; it does not establish a general-purpose physics engine or reliable autonomous-agent training.

### Access remains limited.

As checked on 12 November, Genie 4 is with trusted testers. Google says access is coming to Google AI Ultra subscribers through Project Genie and to early-access Gemini API and AI Studio partners. This announcement does not establish generally available API access or downloadable weights.

### Consistency has a horizon.

Google lists drifting visual details and object permanence, difficult multi-agent interaction, imperfect real-world locations, text-rendering limits and audio errors. Continuous interaction lasts a few minutes. Do not use generated geography or apparent physical behavior as ground truth.

- [Official research announcement →](https://deepmind.google/blog/genie-4-a-new-generation-of-world-simulation/)

- [Current access, demonstrations and stated limitations →](https://deepmind.google/models/genie/)

<a id="routes"></a>

Practical routes · checked 17 November 2026

## Pick access, representation and deployment separately.

Hosted previews, exportable scenes and downloadable research models have different operating contracts.

**Which route matches your deliverable? Access checked 12 November 2026.**

| Route | Useful output | Operating boundary |
| --- | --- | --- |
| [Genie 4](#genie-4) | Interactive generated experience | Trusted-tester preview; future access announcements do not establish a production API. |
| [Cosmos 3](#cosmos) | Generated scenarios and model-specific reasoning or action outputs | Select a released checkpoint, licence and supported accelerator profile. |
| [Marble / World API](#marble) | Navigable scenes, splats, meshes or video | Hosted generation; confirm export entitlements and commercial rights. |
| [V-JEPA](#vjepa) | Video features; action-conditioned predictions with 2-AC | Research integration and a task-specific planning loop, with separate runtime validation. |

<a id="cosmos"></a>

Downloadable · physical AI

### NVIDIA Cosmos 3

Generate scenarios and explore action workflows on a matched NVIDIA runtime.

Cosmos 3 spans world generation, reasoning and action workflows. NVIDIA's model matrix lists Edge at 4B, Nano at 16B and Super at 64B. Match a specific released checkpoint and mode to the task; a family-level modality list does not guarantee every model artifact supports every workflow.

The current repository and Nano card identify **OpenMDW-1.1** . Its distribution conditions require the agreement and applicable origin notices. Review third-party dependencies separately. The hosted Nano Preview API uses NVIDIA API Trial Terms; a downloadable NIM container has its own governing terms.

NVIDIA recommends workstation or datacenter accelerators for Nano, such as RTX PRO 6000, H100 or B200; Super targets H200, B200 or GB200. These are vendor recommendations, not measured performance here. Memory must cover media resolution, duration, activations and concurrency as well as weights. Confirm the exact NIM profile before assuming Spark or Station support.

- [Current framework, checkpoints and workflows →](https://github.com/NVIDIA/cosmos)

- [Read the actual OpenMDW-1.1 licence →](https://github.com/NVIDIA/cosmos/blob/main/LICENSE)

- [Model and recommended-hardware matrix →](https://docs.nvidia.com/cosmos/latest/cosmos3/model_matrix.html)

- [Hosted-preview contract and model boundaries →](https://build.nvidia.com/nvidia/cosmos3-nano/systemcard)

- [Plan infrastructure and operating costs →](https://isaiuseful.com/cloud-models.html.md)

<a id="marble"></a>

Hosted · spatial assets

### World Labs Marble

Build an explorable environment and inspect the exported asset in your own tools.

Marble accepts text, images, video and coarse 3D layouts to create, edit, expand and combine navigable worlds. Its output can be exported as Gaussian splats, meshes or videos. World Labs also offers a public World API for integration into applications.

This is a hosted service and an asset-generation route. The cited product materials do not provide a self-hostable weight release. Export formats and commercial rights depend on the plan: the current pricing page places commercial rights and high-resolution textured-mesh exports in Pro and above. Check rights before using generated environments in paid work.

- [Marble capabilities and representations →](https://worldlabs.ai/blog/marble-world-model)

- [Public World API →](https://worldlabs.ai/blog/announcing-the-world-api)

- [Current export entitlements and commercial rights →](https://marble.worldlabs.ai/pricing)

<a id="atlas"></a>

Hosted · checked 17 November 2026

### World Labs Atlas

Reconstruct scenes, control viewpoints and export spatial assets through a hosted workflow.

Atlas is publicly available through World Labs' hosted application and developer API. The product page lists Viewfinder for reconstruction and camera paths, Chisel for scene building, splat-to-mesh conversion and relighting. The model combines text, images, video and 3D context for generation, reconstruction and space-time simulation.

**Access and evidence:** no open weights or self-hostable release are provided by these sources. Check current service pricing, export rights and data terms before uploading private environments. Preference scores and comparative quality claims in the Atlas blog are World Labs' own evaluations. Generated geometry and robotics simulation examples do not establish physically validated dynamics or real-robot transfer.

- [Atlas application, tools and API access →](https://worldlabs.ai/atlas)

- [Model description and vendor evaluations →](https://worldlabs.ai/blog/atlas)

<a id="vjepa"></a>

Downloadable · research

### Meta V-JEPA

Study video representations; use the action-conditioned variant for planning experiments.

The maintained research repository includes V-JEPA 2, the action-conditioned V-JEPA 2-AC and newer V-JEPA 2.1 video encoders. Keep those roles separate: downloading a video encoder does not give it an action interface. The 2-AC paper demonstrates robot planning after post-training on robot trajectories; this is bounded research evidence, rather than a turnkey robot product.

Code and checkpoints are available. The repository's main licence is MIT, with identified utilities under Apache 2.0. The reference path uses PyTorch and recommends CUDA; its README flags a macOS video-decoding dependency caveat. Check the selected model, video shape and planning rollout's peak memory. No Apple Silicon, Spark or physical-robot result has been measured for this guide.

- [Current checkpoints, licence scope and runtime guidance →](https://github.com/facebookresearch/vjepa2)

- [V-JEPA 2 prediction and planning paper →](https://arxiv.org/abs/2506.09985)

- [MIT licence text →](https://github.com/facebookresearch/vjepa2/blob/main/LICENSE)

- [Separate local memory fit from model capability →](https://isaiuseful.com/local-models.html.md)

<a id="useful"></a>

Where the value can appear

## Start with a task whose failures you can inspect.

The following are evaluation starting points, not established ROI claims.

### Creative prototyping

Try alternative environments, camera paths or scene layouts before committing to a production asset. Measure usable output per hour, export cleanup and rights clearance against the existing workflow.

### Synthetic training scenarios

Vary lighting, weather and scene composition to expose perception failures. Retain held-out real data: a model that improves only on generated scenes has not yet demonstrated transfer.

### Action planning experiments

Compare candidate actions and predicted outcomes in a bounded task. Validate action units, timing, observation calibration and failure recovery before connecting a real actuator.

**A domain-specific example:** Waymo describes adapting Genie 3 to driving simulation, including camera and lidar generation. Its [World Model account](https://blog.waymo.com/blog/2026/02/the-waymo-world-model-a-new-frontier-for-autonomous-driving-simulation/) is company evidence about a specialized internal system. It does not establish public access or that a consumer world generator can validate road safety.

<a id="evaluate"></a>

Acceptance gates

## Judge consequences, persistence and transfer.

Keep the exact checkpoint, input, seed, action sequence, runtime and failures so the comparison is repeatable.

| Question | Practical test | Failure signal |
| --- | --- | --- |
| Does action change the future correctly? | Replay one starting state with two controlled actions; compare predicted outcomes with observations. | Beautiful output ignores the action, changes unrelated objects or reverses cause and effect. |
| What stays where it belongs? | Move away, return and inspect object identity, layout and changes left behind over the required task duration. | Objects disappear, geometry shifts or previous changes reset. |
| Does it improve the real task? | Compare the same policy or workflow with and without generated data on held-out real examples. | Only synthetic scores improve; real success, safety or generalization regresses. |
| Is the loop affordable? | Record end-to-end latency, peak memory, generation retries, review and cleanup time per accepted result. | Offline clip quality hides a control-loop delay or a high rejection rate. |
| Can the output be used? | Check dataset and input rights, plan terms, export format and model-distribution obligations. | A demo can be viewed but the required asset, licence or production interface is unavailable. |

There is no shared leaderboard claim here: scene quality, latent prediction, robot task success and driving simulation are different evaluation targets. Vendor demonstrations and reported research results need a matched task, horizon, hardware configuration and independent reproduction before supporting a deployment decision.

<a id="first-pilot"></a>

From demonstration to decision

## Make the first pilot small enough to disprove.

An editorial evaluation recipe. Set your own thresholds before viewing the outputs.

1. Define one deliverable For a scene, specify the target viewer, export format and camera path. For prediction, specify the action space, observation interval and required horizon. Keep an existing manual or simulator workflow as the baseline.
2. Keep a fixed test set Include ordinary scenes, occluded objects and a return to an earlier viewpoint. Change one input at a time. Keep failed generations and record the model version, seed when exposed, settings and elapsed time.
3. Test the downstream result Open the actual export, inspect geometry and scale, or compare predicted action outcomes with held-out observations. For robotics, validate collision geometry, units and timing separately before any supervised physical trial.
4. Count accepted outcomes Compare total generation, review, cleanup and integration cost per usable result with the baseline. Continue only if the required quality and rights checks pass and the complete workflow earns its cost.

**A practical stop rule:** if an attractive world cannot produce the required export, preserve task state, or improve a held-out real task, use it for ideation and stop short of deployment. Visual plausibility alone cannot validate physics, collision safety or real-world transfer.

The practical next step
Choose one environment. Measure one useful consequence.

- [Plan a first pilot](#first-pilot)

- [Build a bounded robotics workflow](https://isaiuseful.com/robotics.html.md)

- [Choose a training route](https://isaiuseful.com/training-models.html.md)
