Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents
In the authors' words
Aerial vision-and-language navigation (VLN) agents are typically trained on detail-rich, trajectory-aligned commands, whereas users issue short, intent-driven instructions; on a frozen OpenFly navigator, this instruction gap drops success rate (SR) from to . To scale translator training, we prompt a language model with human-written style examples to convert original commands into paired, intent-centered Weak commands, which yield SR. We introduce the Trajectory-Grounded Instruction Translator (TGIT), a front-end that keeps the navigator frozen and translates Weak inputs into agent-executable commands by learning from its trajectory outcomes. The resulting Weak-trained translator raises Weak-input SR to and transfers zero-shot to real human instructions (); it also improves held-out OpenFly () and yields recovery on CityNav and AirVLN.
Appeared: Friday, October 9. arXiv. Preprint, not yet peer-reviewed.