WayTrace: Following Web-Map Directions Across 1.37 km of Real City

Nearly every city already has a routing engine that produces walking directions: turn left in 80 metres, cross at the signal, continue past the pharmacy. That text is free, constantly updated, and robust to appearance change in a way no map built from robot data can be. The hard part has never been getting the directions. It is following them.

Point-goal navigation policies work well when GNSS is accurate, because the goal coordinate does most of the work. Outdoors, between tall buildings or under trees, it is not. Vision-language navigation models handle short instructions in curated indoor settings, but long outdoor routes compound small errors: one missed crosswalk early on puts the robot on the wrong side of a road for the next kilometre.

Core innovation

Language is the interface between the map and the robot, and the hierarchy keeps it narrow. An off-the-shelf orchestrator VLM receives the turn-by-turn directions, a satellite map and recent camera images, and emits exactly one atomic command at a time — something like “cross the road to the left”. It is not asked to plan a trajectory, and it is not asked to act at control rate. This is a sensible division of labour: language models are good at resolving “which of these two crossings does the instruction mean”, and bad at continuous control.

An onboard trace predictor turns each command into a path drawn in the image. Trained on more than a million automatically labelled samples from street-level images, the predictor draws the path implied by the command onto the current camera view. The robot never needs a metrically accurate global map at this stage, only a locally correct path in image space. That is what makes the system tolerant of GNSS drift: the satellite view is used for disambiguation, not for metric localisation.

A reinforcement learning policy tracks the drawn path. The lowest level is a tracking controller over the visual trace, running at robot rate. Because the trace is recomputed in the current image, the loop closes on what the camera actually sees rather than on a stale global plan.

Automatic labelling is what makes the middle layer trainable. A million-plus samples is not a dataset anyone hand-annotates for a navigation paper. The pipeline’s ability to generate trace supervision at that scale from street-level imagery is arguably the reusable engineering contribution here, more than the VLM prompt.

Results

In long-range simulated missions, WayTrace stays robust to GNSS noise and rarely leaves crosswalks or sidewalks, while point-goal baselines degrade sharply with roughly ten times more violations. The violation metric is the right one to report: a navigation system that reaches the goal by jaywalking across a dual carriageway is not a success.

The field result is the headline. On a wheeled-legged robot, the system completed real-world missions of up to 1.37 km at about 1.1 m/s through a busy city centre, rural trails and a university campus. It handled crosswalks, staircases and construction sites. Nearly a kilometre and a half in mixed traffic, at walking pace, without a human escort described in the abstract, is a genuinely different category of evidence from the usual corridor demo.

Limitations

The abstract does not report how many missions were attempted or the success rate, only the longest completed route. That makes it impossible to tell whether this is a reliable system or an impressive best case. For a field paper, the denominator matters as much as the distance.

The orchestrator is an off-the-shelf VLM, which means network latency, API cost and model-version drift all become operational concerns, and behaviour in ambiguous instructions is not characterised. Satellite imagery is another dependency: it is stale in fast-changing areas, unavailable underground, and its licensing varies by region.

The trace predictor is trained on street-level images, so its notion of a path is a pedestrian’s. Kerbs, bike lanes, loading bays and temporary obstructions are exactly where pedestrian and robot affordances diverge, and the abstract does not say how those are handled beyond “handling construction sites”. Staircases are mentioned, which implies the wheeled-legged platform is doing real work, but the split between navigation intelligence and locomotion competence is not quantified.

Finally, the evaluation is one robot, one city, daytime conditions implied, and no discussion of rain, snow, darkness or crowded pavements.

Industry implications

The most transferable idea is to stop treating the map as geometry and treat it as language. Map data is abundant, maintained by third parties, and semantically rich; robot-built maps are none of those things at city scale. A hierarchy that consumes directions and emits local traces sidesteps the impossible task of building a centimetre-accurate global map of a city.

For last-mile delivery and campus robots, the practical takeaway is that the expensive part of the stack may be the middle layer, not the policy. Investing in automatic labelling of trace supervision looks like a better return than hand-tuning a global planner.

The caution is operational. A system whose high level calls a hosted VLM needs a degraded mode when the network is unavailable, and a system that follows pedestrian directions needs a policy about which rules it is willing to bend. Those are product decisions as much as research ones, and this paper’s contribution is to show the architecture can carry them.