pipette
ENEnglish

Plan First, Act Later: Streaming Reasoning for Executable Memory in Robot Control

Qinghang Liu, Conglin Wang, Zongsheng Li, Boyong He, Han Gao, Pengcheng Wang, Tao Sun, Lige Liu, Chang Liu

PreprintUso en el mundo real

En palabras de los autores

Long-horizon robot manipulation requires a robot to remember task-relevant information beyond its short observation window. Existing memory-augmented policies typically retrieve relevant information from history and reason over it at execution time, placing a heavy reasoning burden on the high-level model. To address this challenge, we propose PAL (Plan first, Act Later), a hierarchical memory framework for vision-language robot manipulation. It maintains a short-term visual buffer, a semantic-event memory, and an executable subtask_list. A high-level vision-language model (VLM) converts streaming visual observations into the subtask_list. At execution time, the VLM selects the currently relevant subtask from the subtask_list and passes it to a low-level vision-language-action (VLA) policy. To teach the model what to write to memory and when to update it, we introduce a two-stage data generation procedure that first expands sparse update supervision and then uses self-labeling with task-grounded verification to generate training data that better match the state distribution encountered during execution. We evaluate our approach on memory-dependent benchmarks and real-world manipulation tasks. On RoboMME, PAL achieves a 56.05% success rate, outperforming the strongest baseline, FrameSamp-Modul, at 44.51%. In real-world experiments across four manipulation tasks, \pal{} reaches 60.83% overall success, surpassing MemER (45.00%) and FrameSamp-Modul (28.33%). These results show that proactively writing future-relevant subtasks into hierarchical memory improves long-horizon visual reasoning and robot control. Code and videos can be found on our website: https://qinghangliu.github.io/pal-webpage/

Resultado principalEl resumen no menciona limitaciones.

Apareció: jueves, 22 de octubre. arXiv. Preprint, todavía sin revisión por pares.