Humanoid Loco-Manipulation With Discrete VLA Model

Discrete action tokens made VLA models work well for arms. A humanoid is a harder target: legs, torso, arms and hands in one high-dimensional, heterogeneous action space. This paper folds all of it into the language model itself.

Background

Discrete action tokens have proven a good fit for VLA models controlling robotic arms: the policy emits tokens the language model already knows how to sequence, so no separate action head is needed. A humanoid breaks that assumption. Its whole-body action space - legs, torso, arms and hands - is far higher-dimensional and more heterogeneous than a single arm, and it stresses tokenization, training and real-time inference at once. Existing VLA designs do not address that setting, and the common remedy of a separate continuous action expert reintroduces a knowledge-insulation problem between the language model and the motor policy.

Core Innovation

Holo-M is presented as the first discrete VLA model for humanoid loco-manipulation that exploits the language model intrinsically by extending its vocabulary with action tokens. A unified action tokenizer splits the humanoid action space into four body-part-specific tokenizers - end-effector, body, hand and kinematics - which allows one model to train across drastically different embodiments and data sources, including humanoid teleoperation, egocentric human video and simulation. Because those tokens live in the model’s own vocabulary, the design sidesteps the knowledge-insulation problem of separate continuous action experts. For real-time control the model decodes each body part’s action tokens with grouped discrete diffusion decoding instead of autoregression on the action tokens.

Results

• On the SIMPLE humanoid loco-manipulation benchmark, Holo-M achieves the highest success rates in both the generalist and specialist evaluations.
• It leads the second-best method by significant margins in those comparisons.
• Training spans humanoid teleoperation, egocentric human video and simulation through one unified tokenizer.
• The authors state they will release all code and model weights.

Limitations

The evaluation is reported on a single benchmark, SIMPLE, so breadth across other humanoid platforms and task families is not yet demonstrated in the abstract. The “first of its kind” claim is the authors’ own framing and is not independently verified here. Real-time capability is argued through the grouped discrete diffusion decoder, but the abstract does not quantify control frequency, latency or hardware alongside the success-rate results. Whether the four body-part tokenizers keep their advantage as embodiments diverge further from the training data is left open, as is the accuracy of the kinematic token stream when whole-body motion is fast.

Industry Implications

For humanoid builders the interesting move is architectural: rather than bolting a separate continuous action head onto a language model, Holo-M folds the whole body into the model’s own token stream, which is the kind of design that lets training data be pooled across teleoperation, human video and simulation at once. If one discrete vocabulary can cover four very different body-part action spaces, it suggests a practical route to reusing a single pretrained model across robot generations instead of retraining a bespoke controller per platform. The promised code and weight release matters more than usual here, since it would let other teams test the tokenizer on their own humanoids and check whether the advantage survives contact with different hardware.