GORDON: Graph-based Object-centric Rewards for Decomposition of Long-Horizon Manipulation

· Editorial Team estimated
reward learning long-horizon manipulation object-centric graph neural networks reinforcement learning

GORDON learns dense rewards from action-free video demonstrations by representing each scene as a graph of detected objects and spatial relations, then embedding those graphs with a self-supervised graph neural network into a task-aligned latent space. The resulting reward reveals stage-wise object-state transitions in long-horizon tasks, enabling automatic subtask discovery and sequentially composed policies. On seven manipulation tasks across MAGICAL and ManiSkill3 it reaches 74.4% average success on long-horizon tasks, roughly +35 p.p. over the best learned baseline and +25 p.p. over an oracle.

Paper · arXiv:2608.03753

Background

Learning long-horizon manipulation skills with reinforcement learning is hard for three compounding reasons: reward design is complex, sparse rewards provide little guidance, and manual subtask annotation is expensive. Visual demonstrations can supply supervision for reward learning, but rewards learned from raw pixels tend to be brittle — sensitive to visual variation, background appearance, and robot motion. What the field needs is a reward signal that tracks true task progress rather than pixel statistics.

Core Innovation

GORDON learns dense rewards from action-free video demonstrations through an object-centric representation. Each visual scene is encoded as a graph of detected objects and their spatial relations, and a graph neural network is trained in a self-supervised manner to embed these graphs into a task-aligned latent space. An activity-aware weighted pooling mechanism emphasizes task-relevant objects while masking robot-dominated motion, aligning the representation with semantic task progress. The dense reward is then the distance in latent space between the current state and demonstrated goal configurations. In long-horizon tasks, the temporal profile of this reward exposes stage-wise object-state transitions, which enables automatic subtask discovery without manual segmentation; the discovered segments train subtask-specific rewards and specialized policies that are composed sequentially.

Results

Across seven manipulation tasks on the MAGICAL and ManiSkill3 benchmarks, GORDON improves reinforcement learning in short-horizon settings and enables successful policy learning in complex long-horizon tasks through automatic decomposition. It achieves an average success rate of 74.4% across the long-horizon tasks — roughly +35 percentage points over the best learned baseline and about +25 percentage points over an oracle — and the automatic decomposition is a core part of that gain.

Limitations

The framework depends on reliable object detection and spatial-relation extraction, so perception failures propagate into the learned reward. Evaluations are conducted in simulation benchmarks; real-world clutter, occlusion, and appearance shifts are not covered. The activity-aware pooling relies on being able to distinguish task-relevant objects from robot motion, which may be harder in tasks where the robot’s own body is heavily involved in the scene.

Industry Implications

Long-horizon manipulation remains one of the main obstacles between today’s demo-following policies and useful factory and warehouse automation. GORDON attacks the annotation bottleneck: rewards and subtask structure emerge from video data alone, without manual reward engineering or segmentation, which directly reduces the cost of applying RL to multi-stage tasks such as assembly, kitting, and bin-to-bin transfer. For teams evaluating RL against imitation learning, object-centric reward signals that reliably track task progress make RL a more viable option for stages where imitation data is thin.