Rethinking Action Interface for Agentic Spatial Reasoning

Spatial IntelligenceAI Urban Planning

In the realm of spatial reasoning, the ability to determine where objects are, how they relate, and how they move in 3D remains a fundamental challenge.

TL;DR
• SpatialClaw is a training-free framework for spatial reasoning that adopts code as the action interface.
• It enables the agent to flexibly compose and manipulate perception results and adapt its analysis to both intermediate text and visual observations.
• Evaluated across 20 spatial reasoning benchmarks, SpatialClaw achieves 59.9% average accuracy, outperforming the recent spatial agent by +11.2 points.

Summary
SpatialClaw redefines how vision‑language models (VLMs) perform spatial reasoning by replacing rigid tool‑call interfaces with a flexible code‑based action system. The framework embeds a stateful Python kernel pre‑loaded with perception and geometry primitives, allowing agents to write executable cells that adapt to intermediate outputs and visual observations. This design enables dynamic, task‑specific analysis across 3D/4D environments.
• Adopt code‑as‑action interfaces to increase flexibility in spatial reasoning tasks.
• Pre‑load perception modules in a shared kernel to reduce latency and enable iterative refinement.
• Evaluate across diverse benchmarks to ensure robustness without model‑specific tuning.
• Leverage stateful execution to compose operations that mirror human problem‑solving steps.
• Maintain modularity so new primitives can be added without retraining the VLM.

SpatialClaw achieves 59.9% average accuracy on 20 spatial reasoning benchmarks, surpassing prior agents by 11.2 points. The improvement holds across six VLM backbones from two families, demonstrating the framework’s generality. The training‑free approach eliminates the need for benchmark‑specific adaptation, streamlining deployment.

The study highlights that interface design critically shapes an agent’s reasoning capacity. By allowing agents to iteratively compose and manipulate perception outputs, SpatialClaw mirrors human analytical workflows, leading to higher accuracy and broader applicability. Future work can extend this paradigm to other domains requiring open‑ended reasoning.

As we continue to push the boundaries of spatial reasoning, it's essential to consider the impact of our approaches on the field and its applications.