Matrix-Game 4.0: Harnessing Video Models for Real-Time Playable Game Worlds
En palabras de los autores
Moving from controllable videos to genuinely playable worlds requires persistent world state, executable game rules, and low-latency interaction. We outline a roadmap for the next generation of Matrix-Game that separates game execution from visual generation through an executable game harness, and introduce Matrix-Game 4.0 as the first system built on this design. A lightweight game engine maintains world state and executes interaction rules, a multimodal coding agent translates player ideas into game code and dynamic scene captions, and a renderer based on autoregressive video model transforms engine-rendered low-poly proxies into detailed visuals, conditioned on dynamic captions and memory references. This framework provides continuous guidance for visual generation and enables natural-language creation and modification of scenes and gameplay. Our renderer is trained on aligned low-poly and high-fidelity video pairs and distilled into a causal generator that generates in four-frame chunks with one or two denoising steps. Together with targeted inference optimizations, Matrix-Game 4.0 enables 480p, 60 FPS gameplay on a single NVIDIA RTX5090 with a key-press-to-display latency of 112ms. Single-step generation achieves 83FPS steady-state throughput with 18.2GiB of peak GPU memory usage. With fixed prompts, steady-state throughput also reaches 67FPS on an RTX5080 and 58FPS on an RTX5070Ti. We demonstrate customizable shooter and driving games with engine-enforced mechanics, and release code, model weights, and low-poly game assets to support research and creation of playable game worlds.
En palabras simples Resumen con IA
Los autores construyeron Matrix-Game 4.0, un sistema que separa la ejecución del juego de la generación visual. Un motor de juego ligero rastrea el estado del mundo y las reglas, mientras que un modelo de video convierte formas simples en visuales detallados. Los jugadores pueden crear y cambiar escenas y mecánicas de juego usando lenguaje natural. El sistema corre a 60 cuadros por segundo con 112 milisegundos de latencia en una sola tarjeta gráfica NVIDIA RTX5090. Los autores demuestran juegos de disparos y de manejo personalizables y han publicado su código, modelos y recursos.
Escrito por Kimi K2.5 usando solo el abstract de arriba. Jev verificó que cada oración esté respaldada por las oraciones del abstract que cita y eliminó las que no. Pasá el cursor por una oración para ver de dónde sale.
Apareció: miércoles, 21 de octubre. arXiv. Preprint, todavía sin revisión por pares.
Comentario de los autores: Project page, videos, code and models: https://matrix-game-v4-0.github.io/page/