Research Feed
行业研究动态
实时收录 EN / CN 双语研究报告,涵盖人形机器人、工业臂、AMR 等核心赛道。
Teaching a Humanoid to Predict How Body and Terrain Change Together
STEP argues that fusing depth and proprioception is not enough: a locomotion policy should also learn how the fused state evolves under its own actions. The one-stage framework adds bidirectional masked cross-modal prediction and an action-conditioned next-state predictor inside the policy representation. On a 23-DoF Noetix E1 it completed 98 of 100 hardware trials across stairs, a 40 cm platform, a 70 cm gap, stepping stones and backward stair descent without rear terrain visibility.
A World Model Built from Moving Particles
GPWM replaces pixel-space video prediction with generative modelling of 3D particle and rigid-body motion, using one object-aware multi-resolution transformer for rigid, deformable, cloth and granular materials. Trained on a 25.5-million-frame corpus, it cuts prediction error by 31% on average across six physical domains, and a single set of weights matches or beats domain-specific specialists. The same model also supports contact-rich manipulation planning by searching its latent noise space instead of action space.
Distilling RL's Control Knowledge into a Vision Policy
Latent RL Prior splits dexterous manipulation learning into two stages: privileged reinforcement learning in simulation is distilled into a flow-matching action decoder, and a frozen decoder is then paired with a learned visual interface trained only on action supervision. On three simulated tasks it outperforms full-parameter fine-tuning of pi0.5 and GR00T N1.7 in the low-data regime, beating full-data baselines on door lifting with 10x fewer demonstrations and on bimanual assembly with 5x fewer.
让人形机器人学会预测:身体与地形如何一起变化
STEP 的观点是:把深度与本体感知融合起来还不够,运动策略还必须学会这个融合状态在自己的动作下如何演化。这个单阶段框架在策略表示内部加入双向掩码跨模态预测,以及一个以动作为条件的下一状态预测器。在 23 自由度的 Noetix E1 上,它完成了 100 次硬件试验中的 98 次,覆盖楼梯、40 厘米平台、70 厘米沟槽、踏石,以及背后地形不可见时的倒退下楼梯。
把世界模型建在三维粒子上
GPWM 放弃像素空间的视频预测,改为在三维空间直接生成粒子与刚体的运动,用一个物体感知的多分辨率 Transformer 同时覆盖刚性、形变、布料与颗粒四类物质。在 2550 万帧语料上训练后,它在六个物理域上平均降低 31% 的预测误差,单套权重即可匹敌甚至超过分域专家模型;同一模型还能通过在潜噪声空间而非动作空间搜索来完成富接触操作规划。
把强化学习的控制力,蒸馏进只看图像的策略
Latent RL Prior 把灵巧操作拆成两段学:先在仿真里用特权信息做大规模强化学习,把 rollout 蒸馏进一个 flow matching 动作解码器;再冻结解码器,只用一个仅靠动作监督训练的视觉接口把图像映射到它需要的潜上下文。在三个仿真任务上,它在低数据区间超过 pi0.5 与 GR00T N1.7 的全参数微调,开门任务用 1/10 的演示、双臂装配用 1/5 的演示就超过了全量数据基线。
Can a Generalist Robot Policy Keep Improving After Deployment?
Generalist robot policies generalise impressively out of the box but still need a deployment-time adaptation phase, and prior self-improvement work only showed this for one task at a time. This study builds structured evaluations for multi-task generalist improvement and systematically tests value estimation, policy extraction and resource allocation, finding that a generalist policy can be improved online across many tasks at once and that the right recipe also raises performance on tasks never seen during online training. Value-based RL and the training task distribution are identified as the critical ingredients.
HALO: Humanoid Autonomy Learned in a Thousand Synthetic Homes
HALO trains a humanoid autonomy stack on vision-language-motion data synthesized across a thousand procedurally generated homes, so the corpus scales with compute instead of human teleoperation hours. The resulting skill repertoire — sitting on chairs, navigating to named objects, sidling through half-open doors, crawling under tables, pushing boxes — composes into long-horizon tasks and transfers zero-shot to hardware, reaching 91% success over 360 trials in six unmapped real-world environments without any real home in the training set.
cuRoboV2-Fast: 4.4 ms Motion Planning With a 3 kg Payload
cuRoboV2-Fast cuts mean planning time to 4.4 ms with a 3 kg payload on an RTX 5090, 14.7x faster than cuRoboV2, by combining B-spline trajectory optimisation, differentiable inverse dynamics for payload-aware refinement, and a GPU TimeOpt stage that solves banded Newton systems for traversal speed under torque, jerk, velocity and acceleration limits. A Jetson AGX Orin running the new planner beats the previous stack on a datacenter GPU, and a stereo-camera YAM deployment completes pick-and-drop cycles in 5.93 s, 34.6% shorter than before.
通用机器人策略上线之后,还能继续变强吗?
通用机器人策略开箱泛化能力很强,但部署时仍需一个适配阶段,而此前的自我改进工作只在单任务设定下得到验证。本文为多任务通用改进搭建了结构化评测,系统比较价值估计、策略提取与资源分配三类设计选择,结论是通用策略可以在多个任务上同时在线提升,且配方正确时还能提高那些在线训练中从未出现过的任务的表现。价值型强化学习与在线任务分布被认定为关键因素。
HALO:在一千个虚拟住宅里学会的人形自主性
HALO 把视觉-语言-动作数据放到一千个程序化生成的住宅里合成,让训练语料的扩张跟着算力走,而不是跟着遥操作工时走。由此习得的能力集——坐椅子、按名称导航到物体、侧身穿半开的门、蹲下钻过桌子、推箱子——可以编排成长时程任务,并零样本迁移到真机:在六个未建图的真实环境中完成 360 次试验,成功率 91%,而训练集中没有任何一个真实住宅。
cuRoboV2-Fast:带 3 公斤负载,规划只要 4.4 毫秒
cuRoboV2-Fast 把带 3 公斤负载的平均规划时间压到 4.4 毫秒(RTX 5090),比 cuRoboV2 快 14.7 倍。做法是三层叠加:B 样条轨迹优化减少迭代次数;可微逆动力学把负载计入细化过程;GPU 上的 TimeOpt 在速度、加速度、加加速度与力矩约束下求解带状牛顿系统来优化行进速度。Jetson AGX Orin 上运行新版规划器,比上一代在数据中心 GPU 上还快;双目相机 + YAM 的真机部署把抓放循环做到平均 5.93 秒,缩短 34.6%。
WayTrace: Following Web-Map Directions Across 1.37 km of Real City
WayTrace connects turn-by-turn walking directions to robot motion: an off-the-shelf vision-language model reads the directions, a satellite map and recent camera frames, then issues one atomic command at a time, while an onboard trace predictor trained on over a million auto-labelled street-level samples draws the path in the current image and an RL policy tracks it. A wheeled-legged robot completed real missions of up to 1.37 km at about 1.1 m/s through a city centre, rural trails and a campus, handling crosswalks, staircases and construction.
DataFarm: An LLM Agent Replaces the Teleoperator at $0.80 an Episode
DataFarm hands a coding agent one calibration episode of a robot in the target scene and asks it to rebuild the task in simulation: it writes the scene, an expert controller and a success judge, then a video model renders every rollout into photorealistic multi-view footage labelled with exact simulator actions. Policies trained only on this synthetic data match teleoperation-trained policies on five real tasks and hold up far better under visual distribution shift, at a reported $0.80 per episode.
PolicyIK: Reinforcement Learning Finally Makes Inverse Kinematics Millimetre-Accurate
PolicyIK replaces hand-tuned inverse kinematics with a goal-conditioned RL policy that updates joint-position references from pose feedback and learns to hold them at rest. Two-scale pose errors resolve millimetre residuals without sacrificing large motions, reaching 1 mm and 0.2 degrees on 93.5% of held-out simulated tasks and 98.9% with a deployment protocol, beating four classical IK controllers on a real arm and executing pi-0.5 commands at 95.7% in simulated pick-and-place.
WayTrace:让机器人跟着网页地图的步行导航走完 1.37 公里真实城区
WayTrace 把网页地图的逐向步行导航接到机器人运动上:一个现成的视觉语言模型同时读导航文字、卫星图和最近的相机画面,每次只下发一条原子指令;机载的轨迹预测器在超过一百万条街景自动标注样本上训练,把指令画成当前画面里的一条路径;最后由强化学习策略负责跟踪。一台轮足机器人以约 1.1 米/秒完成最长 1.37 公里的真实任务,穿过繁忙市中心、乡间小径和校园,处理了人行横道、楼梯和施工路段。
DataFarm:让 LLM Agent 顶替遥操作员,单条数据 0.8 美元
DataFarm 只向编码 Agent 提供一个目标场景下的机器人标定片段,就让它把整个任务在仿真里重建出来:写场景、写专家控制器、写成功判据;再用视频模型把每次 rollout 渲染成照片级的多视角视频,并附上仿真器给出的精确动作标签。只用这套数据训练的策略在五个真机任务上追平了遥操作数据训练的版本,在未见过的桌布与彩色光照下则明显更稳,论文给出的边际成本约为每条 0.8 美元。
PolicyIK:用强化学习把逆运动学做到毫米级
PolicyIK 用一套目标条件强化学习策略替代手工整定的逆运动学:策略根据位姿反馈更新关节位置参考,并观测自己累积的参考量与实测关节的偏差,从而在静止时保住位置控制器所需的偏置。双尺度位姿误差让大范围运动与毫米级残差收敛互不牺牲,仿真留出任务 93.5% 达到 1 毫米 / 0.2 度,加上部署流程后达到 98.9%,在真机上优于四种经典 IK 控制器,并让 pi-0.5 的末端指令在仿真抓放中达到 95.7% 成功率。
VLALagBench: 100 ms of Inference Latency Can Halve Success, So Price It Before You Deploy
VLALagBench is a benchmark for how VLA inference latency affects task success, varying task type, object speed, synchronous versus asynchronous execution, and policy. The headline finding is that 100 ms of latency can nearly halve success on moving targets, with sensitivity varying substantially across tasks and policies. Asynchronous execution overlaps inference with action execution and recovers part of the loss at the cost of more inference requests. The authors also fit an empirical success predictor from limited calibration data, with 5.7-7.8% RMSE across policies, which turns latency and per-robot throughput into a concrete configuration choice.
PanoTac: A Sub-$50 Three-Axis Tactile Fingertip for Contact-Rich Manipulation
PanoTac is a low-cost compliant tactile fingertip that produces a 12x32x3 representation per finger, with normal and two shear components at every taxel, at 60 Hz and under $50 per fingertip. The authors characterize response repeatability and consistency across sensor instances and contact locations, then integrate the sensor into a visuo-tactile policy pipeline. Across test-tube preparation, grain scooping and salt pouring, tactile policies beat vision-only baselines by 46.7 percentage points on average, and ablations show that removing either spatial contact information or shear feedback hurts performance.
Pixel: Hardware, Data, Control and Learning Have to Be Designed Together
Pixel is a full-stack system for learning general-purpose mobile manipulation from Carnegie Mellon: a low-cost wheeled humanoid with human-aligned kinematics and compliant grippers, several teleoperation and robot-free human data-collection interfaces, a unified whole-body controller, and one policy-learning pipeline shared across human and robot demonstrations. A hierarchical learning stack on top adds language-conditioned visuomotor control and long-horizon execution. Evaluation spans hardware, data collection and policy learning, with behaviors that require dexterity, active perception, navigation and whole-body coordination.
VLALagBench:100 毫秒推理延迟就能让成功率腰斩,VLA 部署得先算这笔账
VLALagBench 是一个系统研究 VLA 推理延迟如何影响任务成功率的基准,覆盖任务类型、物体运动速度、同步与异步执行方式以及多种 VLA 策略。核心发现是:仅仅 100 毫秒的推理延迟,就能让移动目标上的成功率接近腰斩,而敏感程度随任务和策略大幅变化。异步执行通过让推理与动作执行重叠,能挽回部分损失,代价是更高的推理请求频率。作者进一步用少量标定数据拟合出成功率预测模型,跨策略的均方根误差仅 5.7-7.8%,可用于在给定延迟和单机吞吐下选择最合适的算法配置。
PanoTac:不到 50 美元的三轴触觉指尖,让接触密集操作不再只靠看
PanoTac 是一款低成本柔性触觉指尖,每个指尖输出 12×32×3 的触觉表示——每个触点同时给出法向力和两轴剪切力,采样率 60 Hz,单指成本低于 50 美元。作者完成了跨传感器个体、跨接触位置的重复性与一致性标定,并把它接入视触觉策略流水线。在配管准备、舀取颗粒、倒盐三类任务上,融合触觉的策略平均比纯视觉基线高出 46.7 个百分点;消融显示去掉空间接触信息或去掉剪切反馈都会显著掉点。
Pixel:硬件、数据、控制、学习必须一起设计,通用移动操作才有解
Pixel 是 CMU 团队做的一套全栈移动操作系统:低成本轮式人形本体、与人对齐的运动学、柔性夹爪、多种遥操作与无机器人的人类数据采集接口、统一的全身控制器,以及一条同时吃人类演示和机器人演示的策略学习流水线。在此之上再叠一层分层学习结构,支撑语言条件视觉运动控制和长程任务执行。论文按硬件、数据、策略三个层面分别评测,展示了需要灵巧性、主动感知、导航和全身协调的移动操作行为。
Fallen with a Table Above: ROOF Teaches Humanoids to Make Room Before Standing
When a humanoid falls beneath a low obstacle, getting up is not a standalone skill: it must create clearance and organize support at the same time, because rising too early drives the obstacle further down. ROOF uses reinforcement learning with contact-supported geometric escape guidance, rewarding incremental reduction of robot-obstacle overlap without prescribing an escape path. A learned motion prior plus posture-dependent support and joint-stall costs shape one policy that handles the whole sequence from proprioception alone. Simulation recovery reaches 88.5% under plates that can only be lifted (18.9% without geometric guidance) and 98.6% under free plates with randomized size, mass, and placement; on a Unitree G1 the policy recovered in 95 of 150 attempts across seven real obstacle scenes.
Freeze the Policy, Rewrite the Harness: RoboArchitect Lifts Unseen-Scene Success 4.5x
RoboArchitect treats the control code wrapped around a robot policy as a design object that can be optimized. All model weights stay frozen while a coding model watches rollouts, diagnoses failures, and iteratively rewrites the harness that decides when to query the policy, consult vision tools, or command the arm directly. Across multiple benchmarks and policies it raises success from 9.7% to 43.3% on unseen scenes (4.5x) and from 12.7% to 40.2% on unseen tasks (3.2x), and a general-purpose policy inside an evolved harness often beats a version finetuned for that task.
Split Reachability from Control: ID-WM Plans Long Horizons with an Implicit Dynamics World Model
ID-WM is a hierarchical world model that decouples state-space reachability from local control. A high-level model samples feasible transitions across the state manifold, while a low-level world model acts as an inverse dynamics solver that grounds each transition into precise actions. The planner can therefore choose reachable subgoals without being constrained by execution details. The authors report a 38.5% improvement over state-of-the-art policy and diffusion-based planners across three navigation and manipulation suites, and a 135% gain over single-level world-model MPC on complex unseen long-horizon configurations.
摔倒之后头顶还有障碍:ROOF 让人形机器人先清空间再起身
人形机器人倒在低矮障碍下方时,恢复不是单纯的起身问题:必须先制造空间,再组织支撑,否则一起身就把障碍顶得更死。ROOF 用带接触支撑几何引导的强化学习,奖励机器人与障碍重叠量的逐步减少,而不规定逃生路径;配合学习到的运动先验与姿态相关的支撑、关节堵转代价,训练出仅凭本体感知就能完成整套动作的单一策略。仿真中在只能抬起的板下恢复率 88.5%(无几何引导为 18.9%),在随机尺寸、质量与摆放的自由板下达到 98.6%;在 Unitree G1 上,七个真实障碍场景共 150 次尝试中成功 95 次。
不更新权重,只重写外壳:RoboArchitect 让冻结策略在新场景上提升 4.5 倍
RoboArchitect 把机器人策略外面那层控制代码当成可优化的设计对象:冻结全部模型权重,让一个编码模型观察 rollout、诊断失败并迭代改写调度外壳。在多个 benchmark 与多种策略上,未见场景成功率从 9.7% 提升到 43.3%(约 4.5 倍),未见任务从 12.7% 提升到 40.2%(约 3.2 倍);用通用策略加演化外壳,常常胜过在该任务上微调过的专用版本。
把可达性和控制拆开:ID-WM 用隐式动力学世界模型做长程规划
ID-WM 是一个分层世界模型:高层模型在状态流形上采样可行的状态转移,低层世界模型充当逆动力学求解器,把这些转移落地成精确动作。这样规划器只需判断子目标是否可达,不必被底层执行细节绑住。作者在三个导航与操作测试集上报告相对当前策略与扩散规划器 38.5% 的提升,并在复杂未见长程配置上相对单层世界模型 MPC 取得 135% 的增益。
Carrying Touch Across the Embodiment Gap: TouchBridge Turns Human Demos into Robot Data at 7x Speed
TouchBridge pairs a wearable human demonstration interface with a matching robot interface so human demonstrations can be converted directly into robot training data. Its Tac-Retarget method uses measured contact to align fingertip contact patches and needs no object mesh, pose estimate, or physics simulation. Across four real contact-rich tasks, a VLA fine-tuned only on converted human demonstrations reaches 72.5% average success versus 77.5% for an equal amount of teleoperated robot data, while collection is 7.0x faster; under the same training setup it beats DexPilot by 25.0 points and UniDex by 42.5.
Let Coding Agents Build the Simulator: RoboWorld's 25 Scenes and 200 RL Environments
This paper treats simulation authoring itself as a coding-agent task: simulations are selected from U.S. GDP data to cover economically valuable physical work, checked against explicit definitions of a good simulation and a good task through an adversarial create-and-audit loop, and assembled into RoboWorld — 25 simulations with 25 test and 175 training RL environments. Frontier models average only 0.06 reward on every test task; the authors then show four routes to improvement, including in-context retries, a privileged teacher, self-assisted learning, and code policies that generate demonstrations for training VLAs such as π0.5.
Skip the Universal Gripper: SwapBot Swaps Tools and Senses Contact
SwapBot gives up on the single universal gripper and instead pairs a fixed robot-side coupling interface with a family of low-cost tool-side adapters that turn household tools into robot end effectors. Each adapter carries compact tactile sensors and supports fast autonomous swapping. User studies show shorter task completion times and lower operator workload during teleoperation, and imitation policies trained on the collected demonstrations cook tofu, stir salad, and flip meat; tactile-augmented demonstrations raise success by 15 to 25 percentage points over RGB-only policies.
把人的触觉一起搬过去:TouchBridge 让人手示范七倍速变成机器人数据
TouchBridge 用可穿戴采集端与机器人端两套硬件的对应关系,把人类示范直接搬成机器人训练数据。核心方法 Tac-Retarget 以实测接触为条件对齐指尖接触斑块,不需要物体网格、位姿估计或物理仿真。在四项真实接触密集任务上,只用转换后的人类示范微调 VLA 达到 72.5% 平均成功率,与等量遥操作数据的 77.5% 接近,而采集速度快 7 倍;相同训练设置下比 DexPilot 和 UniDex 分别高 25.0 与 42.5 个百分点。
让代码智能体去造仿真:RoboWorld 的 25 套场景与 200 个强化学习环境
这篇论文把"造仿真"本身当成代码智能体的任务:按美国 GDP 数据挑选有经济价值的体力工作,用一套好仿真、好任务的定义加上对抗式创建—审核回路,做出 25 套仿真、25 个测试环境和 175 个训练环境。前沿模型在所有测试任务上平均奖励只有 0.06;论文随后给出上下文学习、特权教师、自辅助学习与代码策略生成示范四条改进路径,并用于训练 π0.5 这类 VLA。
不给万能夹爪,给一套工具接口:SwapBot 用可换末端和触觉做家务
SwapBot 放弃"一只万能夹爪"的路线,改用一个固定的机器人侧耦合接口配一批低成本工具侧适配器,把家用工具直接变成机器人末端,每个适配器都集成触觉传感器并支持快速自动更换。用户研究表明遥操作时任务完成时间与操作员负荷都下降;用采集的示范训练模仿学习策略,可完成煎豆腐、拌沙拉、翻肉等工具密集任务,触觉增强示范比纯 RGB 策略成功率高 15 到 25 个百分点。
One World State, Many Robots: A Shared-Space Action Model for Multi-Robot Teams
World Dreamer v5-lite keeps robot-specific action heads but feeds them from a single shared world state expressed in workspace coordinates, populated by a pretrained video encoder and queried by one vision-language token per robot. Masked-view cooperative training withholds camera views so agents learn to use what teammates observe. Pretrained on over 10,000 hours from 20 robot types, it reaches 74.0% on RoboFactory, 78.8% across four real-robot tasks, and 58.08% on RoboCasa365.
Skipping the Image Entirely: Event Simulation Fast Enough to Learn Flight in 30 Minutes
An analytic event simulator synthesizes event streams directly from scene geometry and ego-motion instead of rendering photorealistic images first, producing a 64x64 observation in 17 microseconds, over 400x faster than a state-of-the-art rendering pipeline. Reinforcement learning for goal-directed obstacle avoidance converges in under 30 minutes, and the resulting policy, quantized to 1.2 MB, flies a 56 g Crazyflie fully onboard at 82% success over three real obstacle courses.
In-Hand Dexterity That Survives Contact: Pose Reaching Across 4,800 Real Goals
A training recipe for a LEAP multi-fingered hand learns in-hand object pose reaching in simulation with no human demonstrations, then transfers zero-shot to hardware using either motion capture or a single wrist-mounted depth camera. In 13.3 hours of real-robot testing over 4,800 unseen goals, the state-based policy reaches 95.2% of targets within 2 cm and 0.4 rad, and a learned residual safety filter cuts object drops by 75% while still reaching 91.5%.
一个世界状态,多台机器人:共享空间表征的多机动作模型
World Dreamer v5-lite 保留每台机器人独立的动作输出,但让它们共用同一个以工作空间坐标表达的"世界状态":该状态由预训练视频编码器的校准特征填充,并由每台机器人一个视觉语言查询令牌读取。训练时采用掩码视角协作策略,随机屏蔽部分相机视角,迫使模型利用队友的观测。在超过1万小时、20种机器人形态的数据上预训练后,RoboFactory 达到74.0%,四项真机任务平均78.8%,RoboCasa365 为58.08%。
不渲染图像,也能训飞行:事件相机仿真快到半小时学会穿障碍
研究者提出一种解析式事件仿真器:跳过"先渲染照片级图像、再转成事件流"的传统流程,直接从场景几何与自身运动合成事件,单张64x64观测只需17微秒,比当前最好的渲染式管线快400倍以上。这让端到端强化学习在30分钟内收敛;量化到1.2MB的int8策略完全机载运行在56克Crazyflie上,在三条真实障碍赛道取得82%平均成功率。
灵巧手不只要抓得住,还要掉得少:4800个真机目标的在手位姿实验
研究团队为LEAP多指手设计了一套完全不用人类示范的训练方案:在仿真中学会把物体在手指可达范围内移动到任意位置与姿态,再零样本迁移到真机,部署时既可用动捕,也可只用一颗腕部深度相机。在13.3小时、4800个未见目标的真机评测中,基于状态的策略有95.2%的目标落在2厘米与0.4弧度误差内;叠加学习式残差安全滤波后,掉物次数减少75%,同时仍有91.5%的目标可达。
Pretraining Knows What, Simulation Teaches How: A Robot Agent Trained Without Real Robot Data
ZAP teaches an off-the-shelf 27B vision-language model to control a robot through a simple tool harness, using supervised fine-tuning and GRPO on simulated tasks only, with no real-world robot data during training. It reaches 96.7% success on real-world tasks, matching a frontier agent harness while running three times faster and seventeen times cheaper, and it generalizes to unseen objects, goals and strategies as well as to a Franka arm it never trained on.
OSWAM: One Denoising Step Is Enough for a World Action Model
World action models generate actions by iterative denoising, which costs tens of steps and blocks real-time control. OSWAM replaces random noise initialization with imagination tokens that carry physical and action-conditioned priors, then distils privileged future information into the model, so high-quality action sequences emerge in a single diffusion step. It matches or beats multi-step baselines on autonomous driving and embodied control, runs about three times faster, and reaches 40 ms compiled inference on an RTX 5090.
FLORE: Teaching a Humanoid to Push, Pull and Carry with Regulated Contact Force
FLORE learns force-reactive humanoid loco-manipulation from robot-free demonstrations collected with force-sensing handheld grippers and wearable body trackers. A diffusion policy predicts whole-body actions plus per-hand force commands from vision, proprioception and measured wrenches, and a hybrid compliance whole-body controller tracks those forces while keeping balance. Across three contact-rich tasks on the robot the force-reactive policy succeeds in 80% of rollouts, against 26.7% for a position-only policy trained on identical demonstrations.
预训练负责"知道要做什么",仿真负责"教会怎么做":不碰真机数据的机器人智能体
ZAP让一个现成的270亿参数视觉语言模型通过简单的工具接口控制机器人:先用预训练获得对世界的理解与推理能力,再仅靠仿真任务上的监督微调与GRPO学会如何运动,训练全程不使用任何真机数据。在真实任务上成功率达96.7%,与前沿智能体在同一框架下持平,速度快3倍、成本低17倍;它还能泛化到训练中未出现的物体、目标与策略,并迁移到从未训练过的Franka单臂。
OSWAM:一次去噪就够了的世界动作模型
世界动作模型靠迭代去噪生成动作,几十步的采样开销让实时控制难以实现。OSWAM不再从随机噪声起步,而是用"想象令牌"为扩散过程提供物理先验与动作条件表示,并通过带未来条件的特权信息蒸馏增强生成质量,使高质量动作序列在单步去噪内完成。在自动驾驶与具身控制任务上,它与需要数十步的基线持平甚至更优,速度约为其3倍,在RTX 5090上编译后推理延迟低至40毫秒。
FLORE:让双手机器人学会"用多大劲",而不只是"走到哪"
FLORE提出一套力反应式人形机器人移动操作框架。它用带力感知的UMI式手持夹爪与可穿戴追踪器,在不占用机器人的情况下采集示范;力反应扩散策略以视觉、本体感觉和实测六维力为输入,同时预测全身动作与每只手的力指令;混合柔顺全身控制器在保持平衡的同时跟踪这些力指令。在三项富接触真机任务上,该策略成功率为80%,而用同一批示范训练的纯位置策略只有26.7%。
From Freehand Demonstrations to Autonomous Robotic Ultrasound
Ultrasound acquisition is strongly operator-dependent, and expert skill couples anatomical landmarks, live image interpretation and probe-tissue contact. This work proposes an anatomy-anchored imitation-learning framework: a placement module identifies a task-specific anatomical start point from external vision and fixes the initial probe pose as the scanning reference frame, while a flow-based policy generates scanning motion from anchor-relative pose, ultrasound images, an egocentric probe camera and contact force, with a termination head predicting completion. On 16 unseen participants it reaches 96%, 94%, 88% and 94% success for kidney, gallbladder, liver standard planes and radial artery sweeping, beats a fine-tuned pi_0 by 10-42 points on abdominal tasks, and transfers to a second robot without retraining.
ViT-H: Generating Humanoid Motion from Video, Then Making It Executable
Learning whole-body humanoid control from monocular human video is a scalable route to diverse behaviours, but reconstruction-retargeting-tracking pipelines optimise each stage separately, accumulate error between stages, and cannot use downstream tracking feedback to make reference motions more executable. ViT-H replaces them with a generation-and-tracking paradigm: a diffusion model emits humanoid reference motion directly from video, bypassing reconstruction and retargeting, and is then post-trained with direct preference optimisation on tracking outcomes. An autoregressive variant enables real-time video teleoperation below 400 ms. On a Unitree G1, ViT-H beats state-of-the-art pipelines on both motion quality and executability, and scales with model and data size.
DexPort: A Hand-Held Skill Interface That Closes the Dexterous Data Gap
DexPort is a portable collection interface for high-DoF robot hands that uses the same hand hardware for demonstration and deployment, removing the hand embodiment gap entirely. The operator moves a hand-held interface to specify wrist motion and composes a small library of skills to coordinate the fingers, instead of driving every joint. The device records both commanded joint targets and measured joint states. Across four real tasks and more than 2,500 demonstrations collected in the lab and in the wild, behaviour-cloned visuomotor policies improve from 11% to 64% success when trained on commands rather than states, and from 36% to 78% when in-the-wild data is added.
从徒手示范到自主机器人超声:解剖锚定的学习框架
超声检查高度依赖操作者手法,而专家技能体现在解剖标志、超声图像与探头-组织接触三者的协同判断上。该工作提出解剖锚定的模仿学习框架:放置模块先从外部视觉识别任务相关的解剖起点,把初始探头位姿定义为扫描参考系;流式策略再以锚定相对位姿、超声图像、探头端相机视角与接触力为输入生成扫描动作,并由终止头判断任务完成。在16名未见被试上,肾脏、胆囊、肝脏标准切面与桡动脉扫查的成功率分别为96%、94%、88%、94%,腹部任务较微调后的pi_0提升10至42个百分点,并可免重训迁移到第二台机器人。
ViT-H:用视频生成人形动作,再用跟踪反馈把它变得可执行
从单目人类视频学习人形全身控制是获取多样行为的可扩展路径,但现有重建-重定向-跟踪流水线各阶段分别优化,中间误差累积,且无法用下游跟踪反馈直接提升参考动作的可执行性。ViT-H改用生成-跟踪范式:扩散模型直接从视频生成人形参考动作,绕过人体运动重建与重定向;再用运动跟踪结果对生成器做直接偏好优化后训练。作者还提出自回归变体,实现低于400毫秒延迟的实时视频遥操作。在Unitree G1上,ViT-H的动作质量与可执行性均优于现有流水线,并随模型规模与训练数据规模良好扩展。
DexPort:把手持式技能接口变成灵巧手的数据飞轮
DexPort是一个面向高自由度灵巧手的手持式数据采集接口,采集与部署共用同一套手部硬件,从根本上消除了手部本体差距。操作者只需移动手持接口指定手腕轨迹,并通过选择、微调少量预定义技能来协调手指,无需逐关节操控;系统同时记录下发的关节目标与实测关节状态。作者在四类真实任务上采集了2500余条实验室与野外示范,行为克隆策略直接学习手腕运动与手部命令:用指令替代状态使成功率从11%提升到64%,加入野外数据后任务成功率从36%提升到78%。
WholeBodyMem: Remembering What the Humanoid Saw Before It Acts
Whole-body vision-language-action policies for humanoids condition only on the present frame, yet the cues that decide a loco-manipulation task often vanish while the robot turns and walks. WholeBodyMem tracks task objects with SAM3, lets object events causally select key history frames, and encodes appearance, geometry and time into object tokens read by the action expert via cross-attention. With the new HuMeGo benchmark (six simulated and six real Unitree G1 tasks) it lifts base-policy success from 29.1% to 72.6% in simulation and from 36.7% to 88.3% on the real robot.
P²: Reading a VLA's Own Activations to Catch It Going Off-Course
Deploying VLA policies requires runtime monitoring, but existing monitors bolt on a vision-language model to supervise the policy, adding complexity and inference cost. P² instead trains linear probes on the policy's own activations from demonstrations to estimate task progress, then flags executions that deviate from the expected trajectory. On the open-source VLA pi-0.5, across simulated and real manipulation tasks, it matches or beats VLM-based monitors at failure detection and outperforms methods that require failure data.
OpenTac: A Tactile Skin You Can Design, Print and Retarget Yourself
OpenTac pairs a resistive array with a structured deformable overlay that turns tangential loading into spatially varying pressure: differential responses across neighbouring taxels decode local shear, while the common-mode response encodes normal force. A companion CAD tool generates fabrication files from 2D outlines, developable 3D surfaces, or even a text description, alongside a simulator, cross-sensor transfer and adaptive acquisition. The authors designed and built more than thirty sensors across four robot embodiments and characterised the 3D tactile field against a commercial force/torque sensor.
WholeBodyMem:人形机器人操作前,先把「看到过什么」记住
人形机器人的全身视觉-语言-动作策略只看当前帧,而搬动、放置这类任务的关键线索往往在机器人转身行走的过程中就消失了。WholeBodyMem 用 SAM3 跟踪任务物体,让物体事件因果地挑选关键历史帧,并把外观、几何与时间编码成物体 token 供动作专家交叉注意力读取。配合新提出的 HuMeGo 基准(6 个仿真 + 6 个真实 Unitree G1 任务),该方法把基础策略成功率从 29.1% 提升到 72.6%(仿真)、从 36.7% 提升到 88.3%(真机)。
P²:不用外部大模型,用 VLA 自己的激活值判断它是否跑偏
部署 VLA 策略需要运行时监控,但现有方案外挂视觉语言模型来监督策略,复杂且昂贵。P² 换了个思路:在演示数据上对策略自身的激活值训练线性探针来估计任务进度,再据此标记偏离预期的执行。在开源 VLA π0.5 上的仿真与真实操作任务中,它的失败检测能力与基于 VLM 的监控器相当或更好,也优于需要失败数据训练的方法。
OpenTac:把触觉皮肤做成一套能自己设计的开源生态
OpenTac 用「结构化可变形覆盖层 + 电阻式阵列」把切向载荷转成空间变化的压力图:相邻 taxel 的差分响应解出局部剪切向量,共模响应编码法向力。配套的 CAD 工具可以从 2D 轮廓、可展 3D 曲面甚至一句文字描述自动生成制造文件,再加仿真器、跨传感器迁移与自适应采集。作者在四种机器人本体上设计并制造了 30 多个传感器,并对三维触觉场做了商用高精度力/力矩传感器的标定对照。
SpatialDex: 3D-Grounded Bimanual Dexterity from RGB-Only Human Video
SpatialDex learns human behavior together with its spatial context: it reconstructs scene geometry from each RGB video, estimates hand motion, and aligns the two at a shared video-level scale using the hand-size prior of 3D hand estimates. The reconstructed geometry is kept as both a policy input and a prediction target; a gated residual fusion grounds a pretrained VLM's token grid in that geometry, and the policy predicts future scene geometry alongside wrist and finger actions. Pretrained on 1M human-video clips and fine-tuned on a small number of robot demonstrations, it leads prior methods by 16.5 points on nine simulated dexterous tasks across two embodiments and by 38.3 points on six real-world bimanual tasks.
CW-WAM: Predicting Contact Wrench Lifts Peg Insertion from 25% to 70%
CW-WAM adds an explicit contact-wrench stream to a world action model, treating motion and force as joint modeling targets and emitting time-aligned motion-wrench chunks as control references. Pair-causal attention captures motion-force dependencies and a fixed-length wrench memory retains past interaction. The predicted wrench chunk serves as the force reference for a 20 Hz outer-loop admittance controller in a slow-fast scheme, lifting square-peg insertion from 25% to 70% and power-adapter insertion from 40% to 95%, while cutting force-reference RMSE from 4.8 to 2.7 N and from 2.0 to 1.3 N.
AW-0: Turning World Action Models into Reusable Skill Libraries via Code
AW-0 models code, future video and robot actions as three coupled generative streams inside a single mixture-of-transformers. A code expert emits parameterized skill routines and control logic, while video and action experts generate future visual states and low-level motor commands. Code plays two roles at once: it structures multi-step prediction inside the world model, and it represents skills as reusable programs for agentic execution. An automated curation pipeline labels robot and human video with aligned programs and consolidates recurring routines into skill libraries; on RoboTwin 2.0 and physical bimanual tasks the framework beats both VLA and WAM baselines.
SpatialDex:只用 RGB 人类视频,预训练出有 3D 空间感的双臂灵巧操作
SpatialDex 从纯 RGB 人类视频中同时学习行为与它所处的空间几何:重建场景结构、估计手部运动,并用 3D 手部估计的手部尺寸先验把两者对齐到统一的视频级尺度。重建出的几何既是策略输入也是预测目标,策略通过门控残差融合把 VLM 的 token 网格锚定到该几何上,并在预测未来场景几何的同时输出手腕与手指动作。在 100 万段人类视频片段上预训练、少量机器人示教微调后,九项仿真灵巧任务领先 16.5 分,六项真实双臂灵巧任务领先 38.3 分。
CW-WAM:让世界动作模型预测接触力,把插入成功率从 25% 推到 70%
CW-WAM 在视频与动作之外显式预测接触力旋量(contact wrench),把运动与力作为同一组建模目标,输出时间对齐的 motion-wrench chunk 作为控制参考。模型用 pair-causal attention 刻画运动与力的依赖,并用定长力记忆保留历史交互;预测出的力序列作为 20 Hz 外层导纳控制器的力参考,构成慢-快双层控制。方形销轴插入成功率从 25% 提升到 70%,电源适配器从 40% 提升到 95%,力参考 RMSE 明显下降。
AW-0:用代码把世界动作模型变成可复用技能库
AW-0 把代码、未来视频和机器人动作建模为同一个混合 Transformer 中的三条耦合生成流:代码专家输出参数化技能例程与控制逻辑,视频与动作专家分别生成未来视觉状态和底层电机指令。代码既是世界模型内部组织多步预测的结构,也是可被复用、可被调用的技能程序。配套的自动标注流程把机器人视频与人类视频对齐到程序,并沉淀为技能库,在 RoboTwin 2.0 与真实双臂任务上优于 VLA 与 WAM 基线。
DirectWAM: De-Diffusing World-Action Models into Single-Step Control
Cascade world-action models condition robot actions on predicted visual futures but inherit the iterative sampling cost of diffusion-pretrained experts. DirectWAM first asks what a single-step predicted future must preserve to support single-step action prediction, showing through perturbation studies that resampling video noise barely affects downstream actions, that naively truncating both streams to one step costs substantial success, and that visual prediction errors have highly unequal consequences for action accuracy. It then converts pretrained video and action experts into single-step predictors via action-sufficient distillation while keeping the explicit future-to-action cascade, training the future predictor by decoding actions from its own predicted futures and backpropagating demonstration action error through the expert. Results reach 97.1% average across four LIBERO suites and 85.4% on RoboTwin 2.0, extend to three real-world tasks, and cut inference latency by 28.0x versus the full iterative teacher.
SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model
SuperMimic amortizes whole-body motion-tracking policy learning across 129,514 motions with a single differentiable world model fit from logged transitions of a standard rigid-body simulator, training the model and the tracking policy in one loop and backpropagating through short imagined rollouts for analytic gradients. Three hours on one GPU surpass the return of a state-of-the-art tracker trained for seven days on 128 GPUs, cutting GPU-hours by more than 1000x and failing on 25 of 129,514 benchmark motions versus 179 for the baseline; 9.8 hours on two GPUs reaches zero failures. The policy transfers to a physical Unitree G1 across diverse and acrobatic motions.
PhysCanvas: A Vision-Tactile World Model for Robot Policy Evaluation and Learning
PhysCanvas is a state-consistent visuo-tactile world model for contact-rich manipulation that places multi-view RGB, tactile images, marker displacement, task state and actions in one shared vision space, with all modalities reusing a single video VAE and pretrained generative backbone so optical touch inherits visual and temporal priors; forward-kinematics motion images provide spatially aligned action conditioning. Consistency-aware post-training with frozen Scene, Modality and State reward models supervises scene preservation, visual-tactile compatibility and agreement with real outcomes. Across four real manipulation tasks, State Reward Model supervision raises the mean task-wise Pearson correlation between world-model and real success rates from 0.602 to 0.885, and GRPO policy post-training inside the model lifts pi-0.5 real-world success from a 59.1% SFT baseline to 70.8%.
DirectWAM:把「先生成未来、再决定动作」的级联模型压成单步,延迟降 28 倍
级联式世界动作模型(WAM)用预测的未来画面来条件化动作,但继承了扩散预训练专家的迭代采样成本。DirectWAM 先问了一个更基础的问题:单步预测的未来需要保留什么,才能支撑单步动作预测?作者发现重采样视频噪声对下游动作几乎没有影响,而把两条流粗暴截断到一步会显著掉成功率,且视觉预测误差对动作精度的影响极不均衡。据此提出 action-sufficient distillation,把预训练的视频与动作专家转成无随机采样的单步预测器,并用「从自己预测的未来解码动作、把示范动作误差反传穿过专家」来训练未来预测器。结果在四个 LIBERO 套件上平均 97.1%、RoboTwin 2.0 上 85.4%,三项真机任务同样有效,推理延迟相比完整迭代教师下降 28.0 倍。
SuperMimic:用一个可微世界模型,把全身动作跟踪的算力门槛砍掉三个数量级
SuperMimic 用一个从刚体仿真器日志中拟合的可微世界模型,把 12 万段人体动作的全身跟踪策略学习摊销到共享动力学上:世界模型与跟踪策略在同一循环里联合训练,通过对短 imagined rollout 做 BPTT 获得解析梯度。单卡 3 小时的回报超过此前 128 卡训练 7 天的动作跟踪基线,GPU 小时数下降超过 1000 倍,129,514 段基准动作中仅 25 段失败(基线 179 段);两张卡训练 9.8 小时后实现零失败。策略可迁移到实体 Unitree G1,执行多样化乃至杂技动作。
PhysCanvas:把视觉与触觉放进同一个世界模型,让策略评估不再靠真机硬跑
PhysCanvas 是一个面向接触密集操作的状态一致视觉-触觉世界模型:它把多视角 RGB、触觉图像、标记点位移、任务状态与动作统一到同一个视觉空间,所有模态共享同一视频 VAE 与预训练生成骨干,使「光学触觉」能复用视觉与时间先验;由正运动学与相机投影生成的运动图像提供空间对齐的动作条件。训练后用冻结的 Scene / Modality / State 三个奖励模型做一致性后训练,监督场景保持、视听触相容性与真实任务结果一致。在四项真机任务上,State Reward Model 监督把世界模型与真实成功率之间的任务级 Pearson 相关均值从 0.602 提升到 0.885;在该世界模型内做 GRPO 策略后训练,把 π0.5 的真机平均成功率从 59.1% 的 SFT 基线提升到 70.8%。
HAF-VLA: Staging Whole-Body Humanoid Control Inside a Flow-Matching Policy
HAF-VLA adapts a pretrained flow-matching vision-language-action model to humanoid whole-body loco-manipulation by replacing single-pass action-chunk generation with a staged denoising schedule: locomotion and head commands first, then waist refinement, then full-body manipulation, with cross-stage KV-cache conditioning carrying representations between stages. It inherits the original vision-language prefix encoder and action expert weights, adds learnable stage embeddings, and reports significant gains over existing models across six real-world tasks.
Physical Words: Giving Latent Robot Reasoning a Vocabulary It Can Reuse
PhysWord is a reasoning-before-acting foundation model that predicts upcoming atomic action verbs — Physical Words — and aligns their contextual hidden states with latent targets derived from the corresponding future observations, producing a temporally structured space with visual and optional tactile variants of each word. Autoregressive prediction of these words across future timesteps captures transitions between execution stages and supplies structured multimodal context for action generation. It reports 90.3% success on visuotactile UniVTAC and 99.6% on vision-based LIBERO-Long, and outperforms competing methods on real-world tasks with bimanual grippers and dexterous hands.
Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
Ropedia W0 is presented as the first egocentric loco-manipulation world foundation model spanning humans and robots: it is trained on Xp-Data, 10,000 hours of egocentric human video plus 3,000 hours of robot data with unified multimodal annotation, and encodes locomotion as camera poses and manipulation as projected skeletons in one kinematic representation. It reports state-of-the-art results on five cross-embodiment benchmarks (DreamDojo, Psi-0, omega-HOME, Xp-Teleop, AgiBot) and on held-out human egocentric video across zero-shot, few-shot and full-data post-training, shows that scaling human pre-training improves both human and robot evaluations, and after distillation supports real-time minute-long rollouts for live VR interaction.
HAF-VLA:把全身人形控制的分层拆进动作流去噪过程
HAF-VLA 把一个预训练的流匹配视觉-语言-动作模型改造用于人形机器人的全身移动操作:它不再一次性生成整段动作块,而是按阶段逐步去噪——先出移动与头部指令,再细化腰部,最后生成全身操作动作,并用跨阶段 KV 缓存条件把前一阶段的动作表示传给后一阶段。模型继承原有的视觉-语言前缀编码器与动作专家权重,仅新增可学习的阶段嵌入,在六个真实世界任务上显著优于现有模型。
Physical Words:给机器人操作的隐式推理配一套可复用的词表
PhysWord 是一个「先推理、后行动」的基础模型:它预测接下来要发生的原子动作词——即「物理词」——并把产生这些预测的上下文隐状态,与由对应未来观测导出的隐目标对齐,从而形成一个按时间组织、且每个动作词都带视觉与可选触觉变体的表示空间。模型对未来时间步自回归地预测物理词,捕捉执行阶段之间的切换,为动作生成提供结构化的多模态上下文。它在视触觉基准 UniVTAC 上报告 90.3% 成功率、在纯视觉的 LIBERO-Long 上报告 99.6%,并在双臂夹爪与灵巧手的真实任务上优于对比方法。
Ropedia W0:用人类经验把跨本体移动操作世界模型规模化
Ropedia W0 自称是首个同时覆盖人类与机器人的第一视角移动操作世界基础模型:训练数据 Xp-Data 由 1 万小时人类第一视角视频与 3 千小时机器人数据构成,配有统一的多模态标注;模型用统一的运动学表示,把移动编码为相机位姿、把操作编码为投影骨架。它在 DreamDojo、Psi-0、omega-HOME、Xp-Teleop、AgiBot 五个跨本体基准以及留出的人类第一视角视频上报告了最优结果,覆盖零样本、少样本与全量后训练三种设定;同时给出关键结论:扩大人类预训练数据能同时提升人类与机器人两侧的评测表现。蒸馏之后模型支持实时、分钟级的推演,可用于 VR 实时交互。
LAW: Closing the Loop Between Language, Action and World Modeling for Long-Horizon Manipulation
Long-horizon manipulation demands three things at once: decomposing a task into subtasks, knowing how far execution has actually progressed, and keeping actions coherent across the whole horizon. LAW links a vision-language model and a world-action model through a semantic subtask-progress signal in a closed loop: the VLM plans remaining subtasks, estimates the value of candidate actions and asynchronously assesses execution status, while the world-action model jointly predicts video and action, with both sharing temporal context through a dual-stream hierarchical visual memory. LAW reaches 94.21% average success across 50 RoboTwin tasks and 95.56% when the world-action backbone is swapped for OpenWAM-alpha, a new open-source state of the art on that benchmark; on three real-robot long-horizon tasks it averages 65% success against 28.33% for pi0.5 and 25% for Fast-WAM. Its 2B-parameter VLM also outperforms variants built on Embodied-R1.5 and GPT-5.5 High.
Self-Building Labs: Agents That Design, Configure and Operate Their Own Experiment Platforms
Self-driving laboratories (SDLs) accelerate discovery by automating experimental execution and optimisation, and AI agents are increasingly good at proposing new experiments — but existing SDLs can only run the procedures they were built for. Adapting one to a new experiment still takes extensive manual engineering, which makes laboratory flexibility, not throughput, the bottleneck for scaling discovery. This work introduces the self-building laboratory (SBL), which designs, configures and operates its own setup given a high-level scientific goal. Starting from whatever already exists, from an empty room to a working SDL, SBL builds the missing capabilities: lab layout, digital twin, robot skills and experimental workflows. The authors demonstrate it across different robots and setups, scaling from a single workstation to laboratories with several robots working together, and validate end-to-end capability with a real-world quantum-dot synthesis experiment.
Blind Precise Assembly: Zero-Shot Peg-in-Hole from Joint Proprioception Alone
Every robot already carries high-resolution contact sensors: its own joints. Legged robots and humanoids exploit joint feedback for running and whole-body motion tracking, but there the contacts are large and forceful. Whether joint sensing suffices for precise, contact-rich assembly — finding a millimetre-scale opening, meshing gear teeth, threading a nut — has remained an open question. This work trains blind assembly policies that observe only a short history of joint positions, velocities and their own actions, with no vision, force/torque or tactile sensing, unknown fixture and part poses, and substantially randomised grasps. Trained with reinforcement learning in simulation over randomised parts, grasps and contacts, the policies learn peg insertion, gear meshing and nut threading; the peg policy transfers zero-shot to hardware and inserts pegs of different shapes and sizes in 73% of trials. Controlled comparisons against matched visual policies in simulation show the blind policies generalise far better to unseen shapes and sizes, at the cost of speed.
LAW:把语言推理、动作生成与世界模型拧成一个闭环,长时程操作拿下 RoboTwin 开源 SOTA
长时程操作要求机器人既能拆解子任务,又能判断执行到哪一步,还要在动作层面保持连贯。LAW 把视觉语言模型与「世界-动作模型」用一个语义子任务进度信号串成闭环:VLM 负责规划剩余子任务、评估候选动作价值并异步判断执行状态,世界-动作模型负责联合预测视频与动作,二者通过双流分层视觉记忆共享时序上下文。在 50 个 RoboTwin 任务上平均成功率 94.21%,换用 OpenWAM-alpha 骨干后升至 95.56%,创下该基准的开源最好成绩;三个真实长时程任务上平均成功率 65%,而 pi0.5 为 28.33%、Fast-WAM 为 25%。值得注意的是,其 2B 参数的 VLM 在多模态推理上还优于使用 Embodied-R1.5 与 GPT-5.5 High 的变体。
自建实验室:给一个科学目标,让智能体自己设计、搭建并运行实验平台
自驱动实验室(SDL)用自动化执行与优化加速科学发现,AI 智能体也越来越会提出新实验——但现有 SDL 只能跑它们当初被造出来时的那套流程,换一个新实验就要投入大量人工工程,实验室的灵活性成了规模化发现的新瓶颈。该工作提出「自建实验室」(SBL):给定一个高层科学目标,系统自行设计、配置并运行实验室装置,从空房间到已有 SDL 都能作为起点,自动补齐缺失能力——包括实验室布局、数字孪生、机器人技能与实验工作流。作者在不同机器人、不同装置上做了演示,规模从单工作站扩展到多机器人协同的实验室,并以一个真实的量子点合成实验完成端到端验证。
盲装精密装配:只用关节本体感知,让机器人零样本插入真实零件
机器人身上其实已经装了高分辨率接触传感器——它自己的关节。腿足与人形机器人靠关节反馈完成跑跳与全身动作跟踪,但那类接触面积大、力量足;在需要找到毫米级开口、啮合齿轮、穿过螺纹的轻接触装配中,关节感知是否够用一直没有答案。该工作训练只观测关节位置、速度与自身动作短历史的「盲」装配策略,不使用视觉、力/力矩或触觉,且夹具与零件位姿均未知、抓取大幅随机化。策略在仿真中通过强化学习覆盖随机零件、抓取与接触,学会了轴孔插入、齿轮啮合与螺母穿线;其中轴孔策略零样本迁移到真机,对不同形状与尺寸的销钉实现 73% 的插入成功率。与匹配的视觉策略对照,盲策略对未见形状与尺寸的泛化明显更好,代价是速度较慢。
Spark-0: Scaling Spatially Aligned Tactile Pretraining for Dexterous Manipulation
In contact-rich dexterous manipulation, force, compliance and incipient slip are hard to observe from images. Spark-0 is a spatially aligned vision-tactile-language-action (VTLA) model: during pretraining, image-space contact maps from tactile gloves, together with features from a frozen 3D geometry teacher, supervise the visual tokens (removed at inference), while a tactile pathway encodes regional force histories and 3D taxel states to condition the action expert, so a single checkpoint serves both vision-only and tactile-conditioned deployment. The team built HumanTouch, a 1,000-hour full-hand tactile human manipulation dataset, and the paired SparkArena benchmark; retaining touch raises Ego Humanoid vision-only success from 52.5% to 71.7% and SparkArena-Real from 44.0% to 52.0%.
Whole-Body Dexterity: Humanoids That Brace and Carry with Arms and Torso
Humanoid dexterous manipulation remains largely confined to end-effectors and simple grasps, far short of the human ability to brace and slide bulky objects against arms and torso. Whole-Body Dexterity (WBD) treats the robot's body and appendages as contact surfaces: a sim-to-real pipeline first generates a dataset of feasible whole-body grasps, then trains reinforcement-learning experts on an any-to-any repositioning objective sampled from that dataset, and finally distills them into a single policy that uses only proprioception and object geometry. The policy transfers zero-shot to a real Unitree G1, repositioning a basketball and a cardboard box and completing a long-horizon task that stows a box under one arm, carries it shelf-to-table, and places it, steered by coarse human teleoperator references. On real whole-body tasks the distilled controller completes 90% of trials, versus 18% for direct retargeting and 4% for a generalist whole-body controller.
Value as Memory: Online Value-Guided Post-training for General Robot Policies
Real-world post-training of generalist robot policies faces two challenges: partial observability makes action selection ambiguous, and offline post-training methods delay feedback from deployment experience. Value as Memory (VaM) addresses both with value caching and online real-world reinforcement learning: value caching conditions actions on recent value predictions as task-progress memory, while online updates combine offline demonstrations, off-policy interventions and on-policy rollouts for sample-efficient adaptation, moving from an offline RL warmup to online autonomous learning with sparse rewards. Across long-horizon tasks with deformable objects and contact-rich manipulation, VaM yields a 29.4% mean absolute success-rate gain from value caching and a 32.8% gain from online autonomous learning over matched offline training; the online schedule performs 8x as many policy iterations under the same gradient-step budget, with over 2x lower recorded time per gradient step.
Whole-Body Dexterity:让人形机器人用身体和手臂一起抱稳物体
当前人形灵巧操作大多停留在末端执行器与简单抓取,远不及人类用整个身体顶住、滑动笨重物体的能力。Whole-Body Dexterity(WBD)把机器人的躯干与手臂当作接触面来抓取和搬运物体:仿真到现实的流水线先生成可行的全身抓取数据集,再用它采样初始与目标抓取来训练「任一到任一」重定位任务的强化学习专家,最后蒸馏成只用本体感知与物体几何的单一策略。该策略零样本迁移到真实的 Unitree G1,能重定位篮球与纸箱,并完成把箱子夹在腋下、从货架搬到桌面再放下的长时任务,由人类遥操作者提供粗略运动参考。真机全身操作任务中蒸馏控制器完成 90% 的尝试,而直接重定向为 18%、通用全身控制器仅 4%。
Spark-0:把触觉当作监督信号,规模化预训练灵巧操作
接触密集的灵巧操作里,力、柔顺与即将发生的滑移都难以从图像观察。Spark-0 是一个空间对齐的视觉-触觉-语言-动作(VTLA)模型:预训练时用触觉手套生成的图像空间接触图,配合冻结的 3D 几何教师特征来监督视觉 token(推理时移除),同时用一条触觉通路编码区域力历史与 3D 触觉单元状态来条件化动作专家,使同一个检查点既能纯视觉部署也能触觉条件化部署。团队构建了 1000 小时全手触觉人类操作数据集 HumanTouch 与配对夹具 SparkArena;保留触觉后,Ego Humanoid 上纯视觉成功率从 52.5% 升至 71.7%,SparkArena-Real 从 44.0% 升至 52.0%。
Value as Memory:把价值预测当作记忆,让通用机器人策略在真实世界在线进化
通用机器人策略的真实世界后训练面临两个难题:部分可观测性让动作选择存在歧义,而离线后训练方法又延迟了来自部署经验的反馈。Value as Memory(VaM)用价值缓存与在线真实世界强化学习来应对:价值缓存把最近的价值预测作为任务进展记忆来条件化动作,在线更新则把离线示范、离策略干预与在策略 rollout 结合,实现样本高效的适应,使系统从离线强化学习热身平滑过渡到稀疏奖励下的在线自主学习。在涉及可变形物体与接触密集操作的长时任务上,价值缓存带来平均绝对成功率 29.4% 的提升,在线自主学习相比匹配的离线训练提升 32.8%;相同的梯度预算下,在线调度执行的策略迭代次数是原来的 8 倍,单梯度步记录时间降低 2 倍以上。
TacClaw-Infinity: A Large-Scale Vision-Tactile-Language-Action Dataset for Dexterous Manipulation
Tactile manipulation data remain hard to scale, and existing multimodal datasets often lack consistent quality control and structured supervision. TacClaw-Infinity presents the largest structured vision-tactile-language-action dataset to date, collected with TacClaw, a tactile-enabled UMI-style handheld interface that records synchronized visual observations, fingertip tactile signals and end-effector trajectories without needing a target robot. Gathered through distributed worldwide collection across homes, labs, industrial workplaces, retail stores and outdoor environments, it spans over 10,000 hours, 1 million episodes and 1 billion frames across more than 1,000 task types. A scalable pipeline adds trajectory reconstruction, multimodal quality control, embodiment-aware filtering and structured annotation — contact-centric segmentation, subtask description and spatial grounding. Validation covers the collection interface, action-space compatibility and annotation quality at the data level, and cross-embodiment transfer across three heterogeneous robots without robot-specific demonstrations at the model level.
When Do Discrete Actions Make Good Continuous Controllers?
Today's robot foundation models represent actions as continuous vectors, as if that were a given. This paper asks directly whether continuous parametrization is actually superior to discrete. Across simulation and real hardware, and in both from-scratch single-task and multi-task VLA training, the authors systematically compare continuous and discrete policies in single-step regression and multi-step flow variants. The finding is consistent: discrete action prediction — even the most naive uniform-bin quantization — matches or exceeds continuous policies in task success rate. They further identify the true bottleneck as factorization error from product-distribution predictions across time and action dimensions, not action precision. Paired with the right interventions — iterative computation via discrete flows, or distribution sharpening through classifier-free guidance — this weakness becomes a strength, enabling more responsive action changes within a predicted action chunk. Explicitly injecting geometric information into discrete policies, by contrast, does not yield consistent gains.
PingpongDuo: Humanoid Table Tennis Doubles with Safe Team Coordination
Humanoid table tennis doubles forces two robots to strike in strict alternation while sharing a confined workspace, where fast repositioning sacrifices striking accuracy and uncoordinated movement risks collisions. PingpongDuo resolves the conflict with a hierarchical framework: at the team level a planner jointly selects positioning targets and filters risky proposals with learned risk estimates, preferentially preserving the current striker's target; at the individual level separate footwork and striking skills are trained from one another's states and guided by human motion references, including footwork from human doubles play; a racket-arm residual then refines strikes using feedback from a simulator calibrated with real ball-flight and contact measurements. In simulation it improves return success by 13.2% over the state of the art at roughly 2 cm positioning error; on hardware one robot completes a 2.15 m clearance-and-recovery cycle in 2.4 s at a 2.23 m/s peak, and two robots sustain more than 10 consecutive strikes in strict alternation.
TacClaw-Infinity:面向灵巧操作的大规模视—触—语—动作数据集
触觉操作数据难以规模化,现有多模态数据集又普遍缺少一致的质量控制与结构化监督。TacClaw-Infinity 给出了迄今规模最大的结构化视—触—语—动作(VTLA)数据集:用 TacClaw 这一触觉增强、UMI 风格的手持采集接口,同步记录视觉观测、指尖触觉与末端轨迹,且采集时无需目标机器人。数据来自居家、实验室、工厂、零售与户外等全球多点协同采集,合计超过 1 万小时、100 万条回合、10 亿帧,覆盖 1000 多种任务。团队还构建了轨迹重建、多模态质控、本体感知过滤与结构化标注(接触分割、子任务描述、空间定位)的可扩展流水线。数据层面验证了采集接口、动作空间兼容性与标注质量;模型层面则展示了对三台异构机器人的跨本体迁移,且无需机器人专属演示。
离散动作何时能当好连续控制器?挑战机器人基础模型的连续动作范式
当今主流机器人基础模型把动作表示为连续向量,仿佛这是理所应当的。这篇工作直接质问:连续动作参数化真的优于离散吗?作者在仿真与真实硬件上,对单步回归与多步流两类变体、从零单任务与多任务 VLA 训练两种设定下,系统比较了连续与离散策略。结论一致:离散动作预测——即便是最朴素的均匀分箱量化——都能在任务成功率上追平甚至超过连续策略。作者进一步指出,离散策略的瓶颈并非动作精度,而是跨时间与动作维度的乘积分布预测带来的分解误差。配合正确的干预(离散流迭代计算,或用无分类器引导锐化分布),这一弱点可被反转为优势,使策略在预测的动作块内做出更灵敏的动作变化。反之,试图把几何信息显式注入离散策略(如模拟连续流匹配的加噪过程)并不能带来稳定收益。
PingpongDuo:双打乒乓球里,人形机器人学会安全协同
乒乓球双打要求两名机器人严格轮转击球,还要共享一块狭小场地——快速移动会牺牲击球精度,不协调的走位又会带来碰撞风险。PingpongDuo 用分层框架化解这一矛盾:团队层联合选择站位目标,并用学到的风险估计过滤危险方案、优先保住当前击球者;个体层把步法与击球拆成两套相互训练的技能,并以人类双打动作(含真实双打步法)为参考;球拍手臂残差再借助经真实球飞行与接触测量校准的仿真反馈精修击球。仿真中回球成功率比当前最优基线提高 13.2%,让位定位误差约 2 cm;真机上单机以峰值 2.23 m/s 在 2.4 s 内完成 2.15 m 让位—回位,两台机器人可严格交替连续击球 10 次以上。
Steertuning: New Robot Tasks from Five Human Demonstrations in Five Minutes
Current robot-learning recipes need massive demonstration volumes for robust, adaptive policies, which is a poor fit for deployment where tasks and environments keep changing. Steertuning has two parts: a 400M-parameter diffusion policy pretrained on 100 hours of densely annotated egocentric human interaction, which makes the later step robust to background differences and enables spatial and object generalization; and a finetuning stage that, given just 5 demonstrations of a new task, trains for 2 minutes (2000 updates) and steers the policy's actions during reverse sampling using the demonstration closest to the deployment environment. With a data pipeline for embodiment transfer, it reaches 71% success across 12 tasks in 12 new environments, from labs to kitchens in new homes, with the whole pipeline from data collection to deployment taking under 5 minutes.
GreenWAM: A World Foundation Model for Robotics
A robot needs a world model that predicts how its actions change the scene and a policy that chooses those actions, but the two are usually built separately and every robot records actions in its own format. GreenWAM is a world foundation model that serves both roles across bodies: a shared anatomical action space describes 20 robot families, from bimanual arms to dexterous humanoids, in common joint coordinates; the model is trained jointly in visual, action-conditioned and world-action modes; and policies split an action expert off the generator. It reaches top-tier trajectory, depth and 3D accuracy on WorldArena 1.0, ranks among the top entries on RoboTwin 2.0, has the highest zero-shot LIBERO-plus success among compared methods, and is competitive on CALVIN and LIBERO. Adapted to a real humanoid it beats policies adapted from pi-0.5 and Cosmos 3 Edge, and distills to a single-step policy for real-time control.
LoopWAM: Turning Deployment Failures and Successes into a Better Robot Policy
Scaling expert demonstrations alone does not let generalist robot policies improve from deployment, and real rollouts are a mix of successes, suboptimal behavior and failures: imitating everything reinforces poor behavior, while dropping failures discards evidence of physical consequences. LoopWAM is a closed-loop world-action model that learns from this full quality spectrum, using rollouts from itself and other deployed policies. Source-dependent noise sampling unifies behavior learning and action-conditioned world modeling, while quality steering and quality-weighted supervision keep full video supervision across qualities and regulate action imitation; video and action tokens interact throughout parallel denoising. On RoboChallenge Table30-V2 three deployment-and-training rounds raise success from 23.67% to 43.33% (final score 56.80); it reaches 94.5% average success on RoboTwin 2.0 and 49.5% on RoboCasa365 without extra embodied pretraining, and shirt folding improves from 13.20% to 55.56%.
Steertuning:用五段人类演示、五分钟学会新任务
当前机器人学习配方需要海量演示,才能得到稳健、自适应的策略,这与「任务和环境不断变化」的真实部署格格不入。Steertuning 分两部分:其一,在 100 小时、稠密标注的自视角人类交互数据上预训练一个 4 亿参数的扩散策略——这段预训练让后续步骤耐受背景差异,并在新任务上实现空间与物体泛化;其二,给定新任务的 5 段演示,仅微调 2 分钟(2000 次更新),并在反向采样中用「观测最接近部署环境」的那段演示引导策略动作。配合一条面向本体迁移的数据流水线,它在 12 种新环境、12 个任务上取得 71% 成功率,环境从实验室延伸到新住宅的厨房,而整条流程从采集到部署不足 5 分钟。
GreenWAM:面向机器人的世界基础模型
机器人既需要预测「动作如何改变场景」的世界模型,也需要挑动作的策略,但二者通常分开构建,且每台机器人用自己的格式记录动作。GreenWAM 是一个能跨本体同时承担这两种角色的世界基础模型:一套共享的解剖学动作空间,用统一关节坐标描述 20 个机器人族(从双臂机械臂到灵巧人形);模型以视觉、动作条件、世界-动作三种模式联合训练;策略则通过从生成器切分出动作专家而形成。它在 WorldArena 1.0 上的轨迹、深度与三维精度位列第一梯队,进入 RoboTwin 2.0 排行榜前列,在对比方法中取得最高的 LIBERO-plus 零样本成功率,并在 CALVIN、LIBERO 上具备竞争力。适配到真实人形机器人后,其表现优于由 pi-0.5 与 Cosmos 3 Edge 迁移的策略,并可蒸馏为单步策略用于实时控制。
LoopWAM:让部署中的成功与失败共同改进通用机器人策略
仅靠扩充专家演示,无法让通用机器人策略从部署中提升;而真实 rollout 是成功、次优行为与失败的混合体——全盘模仿会强化糟糕行为,丢弃失败又会丢掉物理后果的证据。LoopWAM 是一个闭环世界-动作模型,直接从这个完整的质量谱中学习,数据既有它自己的 rollout,也有其它已部署策略的。它用与来源相关的噪声采样统一行为学习与动作条件的世界建模,用质量引导与质量加权监督在全部质量上保留完整视频监督、并调节动作模仿,且视频与动作 token 在并行去噪全过程中持续交互。在 RoboChallenge Table30-V2 上,三轮部署训练把成功率从 23.67% 提升到 43.33%(最终得分 56.80);无需额外具身预训练,RoboTwin 2.0 平均成功率 94.5%,RoboCasa365 为 49.5%;叠衬衫任务成功率从 13.20% 提升到 55.56%。
EndoRSI: Letting a Robot's Own Experiments Drive Its Policy Research
Physical robot-policy research still depends on humans to interpret experimental failures and decide how a controller should be revised. EndoRSI is a human-governed, self-improving robotic research system in which an embedded coding agent, EndoForge, iteratively develops and refines endoscopic robot policies using evidence from physical experiments, with execution isolated in a safety-constrained runtime under human authorisation. In motorised endoscopy the loop proved productive rather than decorative: validation evidence led the agent to reject an initial action representation, and later physical experiments falsified an offline-supported insertion strategy by exposing hazardous closed-loop behaviour absent from the source data. After failed rollouts were fed back into policy revision, insertion success rose from 2/21 in the first generation to 17/21 in the fourth. The frozen final policy held up across visual and hardware shifts and was the best-performing autonomous policy across six independent ex-vivo porcine specimens.
SmolWAM: A World-Action Model Small Enough for the Edge
World-action models predict a robot's actions together with the observations those actions will produce, but the strongest of them rely on video diffusion transformers that are far too heavy for edge devices. SmolWAM derives a latent world-action model from the 2B-parameter Cosmos Policy by keeping its pretrained trunk and rebuilding the inference interface: single-frame visual latents, cached text features that preserve the trunk's attention sinks, 12 of 28 transformer blocks, and 1,008 tokens from which actions and future visual latents are read out in a single rectified-flow step. Trained in 3.6 GPU-hours, it succeeds on 95.5% of 400 LIBERO episodes at 57 ms and 2.3 TFLOP per 16-action chunk using 2.4 GiB of GPU memory, about 11x faster than its teacher with 18.5x fewer FLOPs and 3.2x less memory, and its world head still forecasts scene motion.
Splat2Act: Personalizing a Generalist VLA with a Single Phone Scan
A generalist manipulation policy that works in a lab can score 0% pickup success in a real home or office, and collecting demonstrations in every user's space is prohibitively expensive. Splat2Act takes only a handheld phone scan and images of the user's objects and turns them into a photorealistic digital twin with 3D Gaussian Splatting and collision geometry, automatically finds tabletop and non-tabletop receptacles, builds feasible pickup tasks, and generates successful demonstrations with a privileged simulation oracle. Across 101 scanned campus environments with a median navigable area of 104.87 m^2 it produces 852 receptacles and over 68,000 pickup demonstrations. Two-stage fine-tuning of pi-0.5, first on public object assets and then on the user's own objects at the target scene's receptacles, yields 62.5% aggregate pickup success in the target scene, and the same policy reaches 63.9% on a 12-task subset when deployed on a real Stretch 3, close to its 66.7% simulated success.
EndoRSI:让机器人自己的实验来驱动策略研究
物理机器人的策略研究至今仍依赖人来解读实验失败、并决定该怎样修改控制器。EndoRSI 是一个由人监督、可自我改进的机器人研究系统:内嵌的编码智能体 EndoForge 以物理实验证据为依据,反复开发与改进内窥镜机器人策略,而物理执行被隔离在受安全约束的运行时中、并须经人类授权。在电动内窥镜任务中,这个闭环是实打实地在起作用:验证证据让智能体否决了最初的动作表示;后续的物理实验又否定了离线评估支持的插入策略,因为它暴露出源数据中不存在的危险闭环行为。把失败的物理 rollout 反馈进策略修订后,插入成功率从第一代的 2/21 提升到第四代的 17/21。冻结后的最终策略在视觉与硬件变化下仍保持能力,并在六份独立的离体猪标本上取得最佳自主策略表现。
SmolWAM:小到能跑在边缘设备上的世界-动作模型
世界-动作模型会同时预测机器人该做什么、以及这些动作将带来怎样的观测,但最强的模型依赖视频扩散 Transformer,对边缘设备来说重得无法承受。SmolWAM 从 2B 参数的 Cosmos Policy 出发,保留其预训练主干、重建推理接口,得到一个潜在世界-动作模型:单帧视觉潜变量、可保留主干注意力汇(attention sink)的缓存文本特征、28 个 Transformer 块中只取 12 个,以及 1,008 个 token——动作与未来视觉潜变量都在一次整流流(rectified-flow)步骤中读出。它只需 3.6 GPU 小时训练,在 400 个 LIBERO 回合上成功率 95.5%,每个 16 动作块的推理耗时 57 毫秒、算力 2.3 TFLOP、显存占用 2.4 GiB;相比教师模型约快 11 倍、FLOPs 少 18.5 倍、显存少 3.2 倍,而世界预测头仍然能预测场景运动。
Splat2Act:一次手机扫描,把通用 VLA 策略适配到真实场景
一个在实验室里好用的通用操作策略,到了真实的家庭或办公室可能连一次抓取都做不成——论文中的强通用模型 π0.5 在未做适配的部署场景里抓取成功率为 0%,而在每个用户环境里采集真实演示的成本又高得难以承受。Splat2Act 只要求用户用手机扫描一次场景、并拍摄若干物体照片,就能用 3D 高斯泼溅加碰撞几何构建出照片级数字孪生,自动发现桌面与非桌面置物面、构造可行的抓取任务,并借助带有特权信息的仿真预言机批量生成成功演示。在 101 个校园环境中(可通行面积中位数 104.87 平方米),该流程产出 852 个置物面与超过 68,000 条抓取演示。对 π0.5 的两阶段微调——先在公开物体资产上适配,再在目标场景置物面上针对用户自己的物体个性化——在目标场景取得 62.5% 的整体抓取成功率;同一策略部署到真机 Stretch 3 后,在 12 个任务子集上达到 63.9%,与仿真中的 66.7% 十分接近。
ARC: A Reasoning Recipe That Lifts Robot Foundation Models Without New Data
The default path to better robot foundation models is bigger models, more demonstrations and costlier training. ARC argues there is an efficient complement: the right reasoning recipe can substantially improve the zero-shot performance of existing state-of-the-art models. It has three parts. Effective reasoning traces are grounded in the robot's next action and explain its causal structure, why the action is appropriate and what effect it should produce. A scalable automatic labeling pipeline generates those traces from existing demonstrations, so ARC-Trace-DROID is built from DROID without collecting new robot data. And a strategy adapts pretrained models to use the traces for control. State-of-the-art VLAs such as pi_0.5 and world-action models such as Cosmos3-Nano-Policy learn from them: ARC sets a new state of the art on RoboLab-120 and MolmoSpaces, gains up to 50 points on RoboLab-Reasoning-50, and improves pi_0.5's real-robot task success by 82.2 percentage points.
VioLA: A Generalist Humanoid Policy Learned from Human Motion
Whole-body instruction following on a humanoid is hard for two reasons: the action space couples legs, arms and fingers under balance, and humanoid demonstrations are scarce, so generalist policies have to be fine-tuned per task. VioLA changes what the generalist policy predicts. It outputs body and hand motion latents, executed by pretrained controllers, and motion encoders put human and robot motion in the same latent space, so a human recording is already labeled in the policy's action space. That turns a 140.6M-frame pool, 93.2% of it human, into usable training data. The resulting policy follows locomotion instructions on the real robot zero-shot at 100% success, where GR00T N1.7 reaches 16.7% and Psi_0 reaches 0%, and reaches 88.6% manipulation success without task-specific fine-tuning, across two VLA and one world-action backbone.
DreamTrue: A Robot World Model That Follows Actions and Learns from Counterfactuals
Robot world models should predict what happens next faithfully, but training them on real datasets runs into imprecise calibration that weakens action following and a bias toward successful outcomes because failures are rare. DreamTrue is a multi-view, cross-embodiment world model that attacks both. It renders action trajectories into image-space conditions and uses offline geometric calibration to align them with the target video, improving action following across embodiments. To widen interaction coverage it adds counterfactual post-training, rewriting recorded trajectories and generating futures under a broader range of actions and contacts. To score predictions without paired ground truth, it builds a human-annotated dataset of robot, object and interaction defects and trains an embodied video reward model that steers reinforcement-learning post-training toward physically plausible outcomes. On AgiBot it attains state-of-the-art action following and cuts the human-assessed interaction defect rate from 48.12% to 6.25%, and it ranks first in the world-model track of the AgiBot World Challenge 2026.
ARC:无需新数据即可提升机器人基础模型的「推理配方」
提升机器人基础模型的默认路径是:更大的模型、更多的示范、更昂贵的训练。ARC 主张存在一种高效的补充手段——只要「推理配方」对,就能显著改善现有最先进模型的零样本任务表现。这套配方由三部分组成。第一,有效的推理轨迹必须扎根于机器人的下一个动作,并解释这个动作的因果结构:它为何恰当、会产生什么效果。第二,一条可扩展的自动标注流水线能从既有示范中生成这些轨迹,于是 ARC-Trace-DROID 完全从 DROID 构建,无需采集任何新的机器人数据。第三,一套策略让预训练模型学会用这些轨迹来控制。π0.5 这类最先进的 VLA 与 Cosmos3-Nano-Policy 这类世界动作模型都能从中学习:ARC 在 RoboLab-120 与 MolmoSpaces 上刷新了最先进水平,在 RoboLab-Reasoning-50 上提升高达 50 个百分点,并把 π0.5 的真机任务成功率提升了 82.2 个百分点。
VioLA:用人类运动数据训练通用的全身人形控制策略
让一个人形机器人用全身跟随指令,会同时撞上两堵墙:它的动作空间大而强耦合,腿、臂、手必须在维持平衡的前提下协同运动;而人形示范又极其稀缺,导致现有通用策略开箱即用不了,必须针对每个任务用遥操作示范微调。VioLA 改变了通用策略去预测的对象——它不再输出关节指令,而是预测身体与手部的运动隐变量,交由预训练的身体控制器与手部控制器在真机上执行;配套的运动编码器把人类运动和机器人运动映射到同一隐变量空间,于是一段人类录制天然就带着策略动作空间下的标签。这让一个 1.406 亿帧、其中 93.2% 来自人类的训练池变得可用。由此得到的策略能在真机上零样本跟随移动指令,成功率达 100%,而 GR00T N1.7 为 16.7%、Psi_0 为 0%;在无需任务级微调的情况下,操作成功率也达到 88.6%,且同一套做法可跨两种 VLA 与一种世界动作模型骨干复用。
DreamTrue:既忠于动作又能从反事实中学习的机器人世界模型
机器人世界模型应当忠实地预测接下来会发生什么,但在真实数据集上训练时,它会遇到两个麻烦:标定不够精确会削弱对动作的跟随,而数据集中失败样本的稀缺会把预测偏向「成功」这一边。DreamTrue 是一个多视角、跨形态的世界模型,同时针对这两点。它把动作轨迹渲染成图像空间的条件,再用离线几何标定把它们与目标视频对齐,从而改善跨形态的动作跟随;为拓宽交互覆盖,它引入反事实后训练,改写录制轨迹,在更广的动作与接触配置下生成未来视频;为了在没有配对真值未来的情况下给出反馈,它构建了一个覆盖机器人、物体与交互缺陷的人工标注视频数据集,并据此训练一个具身视频奖励模型,用其打分把强化学习后训练导向更符合物理的交互结果。在 AgiBot 上,它取得了当前最好的动作跟随,并把人工评估的交互缺陷率从 48.12% 降到 6.25%,同时在 AgiBot World Challenge 2026 的世界模型赛道中排名第一。
HuMBLE: Human Motion-Driven Behavior Learning for Embodied Locomotion
Humanoid locomotion controllers tend to fail in one of two ways: ones optimized purely for command tracking and robustness produce mechanical gaits, while ones tied to human motion data fail to generalize outside that data. HuMBLE is a learning framework that balances the two. It first learns a natural locomotion prior through teacher-student distillation, training a full-body reference-conditioned policy with reinforcement learning and then distilling it into a lightweight prior conditioned only on proprioception and a planar torso-velocity steering command. That prior is then fine-tuned with multi-task RL that pairs a goal-conditioned command-tracking task with a reference-guided task that keeps the human data as an explicit style regularizer. The framework is validated on Boston Dynamics Atlas R1, Atlas D1 and Unitree G1, including real-world user-controlled indoor and outdoor walking and integration as the locomotion layer within hierarchical control stacks.
RoboJEPA: Scaling Robotic Latent World Models
Latent world models can predict future states and plan in the real world, but the field has lacked a principled way to estimate how those abilities scale with model size, data and compute. RoboJEPA is a world model built on the Joint Embedding Predictive Architecture and trained on a large-scale dataset spanning 12 robotic embodiments. Its imagination error, the error of its latent rollouts, follows a second-order power law in compute, so model quality can be predicted well beyond the scale at which the law is fit, while downstream planning performance improves predictably with compute and correlates strongly with imagination error. The model can also be deployed zero-shot as a robotic agent, planning toward a single goal image to solve long-horizon tasks on real hardware. At 8B parameters, RoboJEPA is the largest JEPA predictor trained to date, and the authors release all checkpoints together with training and deployment code.
Long-WAM: Scaling the Context of World-Action Models
Real-time robot control needs enough visual history to infer motion and task progress, but processing that history costs latency. Long-WAM argues that access to history is not the same as using it: longer contexts pay off far more when the video foundation is pretrained autoregressively. The framework learns causal prediction from robot and egocentric video without action labels, preserves that history-to-future structure during world-action adaptation, then adds streaming observation encoding, asynchronous execution and hardware-specific acceleration so it still runs in real time. On RoboCasa GR-1, increasing context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%, and it leads on LIBERO-Long, RoboTwin 2.0 and DOMINO. It deploys on RTX 5090, DGX Spark and Jetson AGX Thor, where each action chunk takes 107.4 ms on an RTX 5090, and reaches 95% on dynamic cup stacking with Unitree G1 and YAM, where Pi0.5 and Fast-WAM succeed in none of 20 trials.
HuMBLE:以人类动作为驱动的可操控人形运动学习
人形运动控制器往往以两种方式之一失败:纯粹为指令跟踪与鲁棒性优化的控制器会产生机械感的步态;而与人类动作数据绑定的控制器则无法泛化到该数据分布之外。HuMBLE 是一个平衡这两者的学习框架。它先通过教师—学生蒸馏学习一个自然运动先验:用强化学习训练一个全身参考条件策略,再把它蒸馏成仅依赖本体感知与躯干平面速度操控指令的轻量先验。随后用多任务强化学习微调该先验,同时配对两类目标——一个跟踪任意指令的目标条件任务以扩展数据分布之外的覆盖与鲁棒性,以及一个以人类数据为显式风格正则项的参考引导任务。该框架在波士顿动力 Atlas R1、Atlas D1 与 Unitree G1 三台人形上完成验证,包括真实环境中用户直接操控的室内外行走,以及作为分层控制栈中运动层的集成。
RoboJEPA:为机器人隐空间世界模型建立规模定律
隐空间世界模型已能预测未来状态并在真实世界中规划,但这个领域一直缺少一种有原则的方法,去估计这些能力如何随模型规模、数据与算力变化。RoboJEPA 是一个基于联合嵌入预测架构(JEPA)的世界模型,训练于覆盖 12 种机器人形态的大规模数据集。它的「想象误差」,即隐空间 rollout 的误差,随算力呈二阶幂律,因此可以在超出拟合规模的范围内预测模型质量;与此同时,下游规划性能随算力可预测地提升,并与想象误差强相关。该模型还能作为机器人智能体被零样本部署,朝单张目标图像规划,在真实硬件上完成需要长程规划的任务。RoboJEPA 拥有 80 亿参数,是目前训练过的最大 JEPA 预测器,且作者公开了全部检查点以及训练与部署代码。
Long-WAM:把长视觉历史真正用起来的世界动作模型
实时机器人控制需要足够长的视觉历史来判断运动与任务进度,但处理这段历史本身就要付出延迟。Long-WAM 的核心判断是:能访问历史,和真正用上历史,是两回事——当视频基座以自回归方式预训练时,更长的上下文带来的收益要大得多。该框架先在无动作标签的机器人视频与第一视角视频上学习因果预测,再在适配为世界动作模型的过程中保留这种「历史到未来」的结构,并辅以流式观测编码、异步执行和硬件级加速,使其仍能实时运行。在 RoboCasa GR-1 上,上下文从 0.0 秒增至 19.2 秒,成功率由 63.3% 提升到 78.7%,并在 LIBERO-Long、RoboTwin 2.0 和 DOMINO 上取得所比较方法中的最佳结果。模型可部署于 RTX 5090、DGX Spark 与 Jetson AGX Thor,在 RTX 5090 上每个动作块耗时 107.4 毫秒;在 Unitree G1 与 YAM 上,动态叠杯子达到 95% 成功率,而 Pi0.5 与 Fast-WAM 在 20 次尝试中无一成功。
EgoLAP: Turning Human Video into Robot Skill via Language Actions
Egocentric human video is an abundant and cheap alternative to costly robot demonstrations, but the embodiment gap makes raw human trajectories a poor target for control. EgoLAP's insight is that although low-level actions are embodiment-specific, the motion intent behind them is not — it captures task structure that transfers between humans and robots. The framework pre-trains a vision-language-action model on human and robot trajectories through a shared language-based action chain-of-thought, expressing motion intent as structured, temporally abstracted language actions paired with motion-level reasoning grounded in scene geometry, physics, and object affordances. Across real-world and simulated experiments it reaches 80.1% mean real-world task progress, a 2.3x gain over alternative action representations, and its motion-level reasoning outperforms a composite format that adds subtask, object-box, and visual-trace reasoning.
PEARS: Cheap Real-World Adaptation for Tactile Manipulation
Pretrained robot policies degrade under out-of-distribution conditions, and reinforcement-learning post-training can repair them — but in manipulation, where every trial is slow, costly, or destructive, the interaction budget is the binding constraint. PEARS is a physics-prior-guided hybrid RL framework for sample-efficient online adaptation with tactile feedback. Its physics-guided force reasoning module uses priors encoded in a vision-language model to diagnose failures from the visual outcome and tactile history, then updates task-appropriate contact-force bounds that a high-frequency hybrid force-position controller enforces. In parallel, tactile-conditioned diffusion steering adjusts the latent noise of a frozen flow-matching policy to fix free-space motion and contact timing without retraining the base model. In simulation it lifts success rates by 12.4-37.4 percentage points over the strongest per-task baselines and cuts the episodes needed to reach a success threshold by up to 53.2%; on real robots it reaches 95% on Whiteboard Erasing and 90% on Pipette Liquid Aspiration.
QF3: Making Flow Policies Trainable with Off-Policy Reinforcement Learning
Flow policies have become a default way to turn demonstrations into robot behavior, but improving them with reinforcement learning has usually meant on-policy methods that pay a steep interaction cost. QF3 is an online off-policy algorithm that trains a flow policy with flow matching plus the critic's action gradient, backpropagated through a one-step prediction of the flow's output. Its key trick is filtering: the critic gradient is applied only to action dimensions that stay near the replay action, keeping updates where the critic and the one-step prediction are trustworthy. The authors report the first off-policy flow RL method to train humanoid locomotion from scratch and transfer it zero-shot to hardware, a 10x wall-clock speedup over the on-policy FPO++, and successful fine-tuning of pretrained flow manipulation policies.
EgoLAP:用「语言动作」把人类视频转化为机器人技能
第一视角人类视频是昂贵机器人示范的一种充足而廉价的替代,但具身差距使得原始人类轨迹并不适合直接作为控制目标。EgoLAP 的洞见在于:尽管低级动作与具体具身绑定,其背后的「运动意图」却不是——意图承载着能在人与机器人之间迁移的任务结构。该框架通过一条共享的、基于语言的动作思维链,在人类与机器人轨迹上联合预训练视觉-语言-动作模型:把运动意图表达为结构化、时间上抽象的语言动作,并配上扎根于场景几何、物理与物体可供性的运动级推理。在真实与仿真实验中,它取得 80.1% 的真实世界平均任务进度,相比其他动作表示有 2.3 倍提升,且其运动级推理优于叠加了子任务、物体框与视觉轨迹推理的复合格式。
PEARS:以低成本实现接触式操作的实机自适应
预训练的机器人策略在分布外条件下会明显退化,而用强化学习做后训练可以修复它——但在操作任务中,每一次试验都缓慢、昂贵甚至具有破坏性,交互预算才是真正的约束。PEARS 是一个以物理先验引导的混合强化学习框架,借助触觉反馈实现样本高效的在线自适应。它的「物理引导的力推理」模块利用视觉语言模型中编码的物理先验,从视觉结果与触觉交互历史中诊断失败,并据此更新合适的接触力上下界,由一个高频力-位混合控制器在接触过程中执行。与此同时,触觉条件的扩散转向在不重训基础模型的前提下,调整冻结的流匹配策略的隐噪声,以修正自由空间运动与接触时机。在仿真中,它把成功率相比各任务最强基线提升 12.4–37.4 个百分点,并把达到某成功率阈值所需的交互回合数最多减少 53.2%;在真机上,白板擦拭达到 95%、移液吸液达到 90%。
QF3:让流策略可被离策略强化学习训练
流策略已经成为「把示范变成机器人行为」的默认做法之一,但要用强化学习进一步改进它,通常只能采用在策略方法,付出高昂的交互代价。QF3 是一种在线离策略算法:它同时用流匹配目标与评论家的动作梯度来训练流策略,而该梯度是经由对流输出的一步预测反向传播得到的。其关键技巧在于「过滤」——只把评论家梯度作用于那些仍停留在回放动作附近的行为维度,从而把更新限制在评论家与一步预测都可信的范围内。作者报告:这是首个能从零训练人形机器人运动策略并零样本迁移到硬件的离策略流强化学习方法,相比在策略的 FPO++ 取得 10 倍墙钟加速,并能成功微调预训练的流式操作策略。
RealtimeWAM: One-Step Asynchronous World Action Models
World action models incorporate visual representations from video generation backbones to guide action prediction, and recent efficient ones adopt Mixture-of-Transformers architectures that compute the video representation once for reuse by the action expert. Two bottlenecks still limit inference: intra-expert iteration, where the action expert runs multi-step denoising, and inter-expert waiting, where the video and action experts execute sequentially. RealtimeWAM removes both. Teacher-Anchored Consistency Distillation addresses a local-global error gap — low local consistency error alone does not guarantee accurate final actions — by supplementing local consistency with explicit supervision from the frozen teacher's multi-step rollout endpoint, enabling accurate one-step action generation. Cross-Expert Wavefront Pipelining then overlaps the two experts by sharing the video KV cache block-wise and synchronizing only immediately before the corresponding action attention consumes it. Across LIBERO, LIBERO-Plus and RoboTwin and variants such as Fast-WAM and Faster-WAM, RealtimeWAM keeps near-lossless performance (under 1% drop) while delivering a roughly 25x end-to-end speedup on an H100, with code and checkpoints released.
H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
Long-horizon planning with latent world models has to reason across timescales and levels of abstraction, yet existing task-agnostic JEPA world models predict and plan at a single timescale, or with several horizons inside one shared latent space. H-JEPA is an end-to-end recipe for training a hierarchy of action-conditioned JEPAs in which each level predicts farther ahead in its own learned latent space. Planning is top-down: the top level optimizes progress toward the goal, and each level's predictions become subgoals for the planner below it. When the factors in the data evolve at separated timescales, higher levels discard fast, unpredictable detail and retain slower task-relevant state. Across four simulated navigation and manipulation environments hierarchical planning beats a flat JEPA; on Visual AntMaze a three-level hierarchy raises success from 18% to 73% using less planner compute, and ablations attribute the gains to both temporal decomposition and higher-level goal representations. With inverse-dynamics supervision the approach extends to diverse real-robot videos from DROID, improving offline planning fidelity at lower planner compute.
InterMimicGen: Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitation
Human-object interaction captures are a rich but awkward source of supervision for humanoid loco-manipulation: they are sparse, heterogeneous, and not directly executable by a robot. InterMimicGen is a self-evolving motion-imitation framework in which robot motion data and a tracking policy improve one another. It consolidates motion-captured human-object interaction datasets and retargets them into humanoid references that preserve whole-body coordination and dexterous hand-object relationships, then trains a physics-based generalist tracker that executes those references in simulation on a humanoid with dexterous hands. A data flywheel then makes small, task-preserving edits to where an interaction takes place and how the body performs it, fine-tunes the tracker on the variants, and keeps only those whose simulated execution completes the task, so coverage compounds around the sparse original demonstrations while task semantics and motion quality are preserved. The authors report contact-preserving retargeting across robot configurations, broad tracking from a single generalist policy, motions that keep growing across augmentation rounds, and transfer to real robots.
RealtimeWAM:单步异步的世界-动作模型
世界-动作模型借助视频生成骨干的视觉表征来引导动作预测,近期的高效方案采用 Mixture-of-Transformers 架构,把视频表征只计算一次、供动作专家复用。但推理效率仍被两处瓶颈拖累:专家内的迭代,即动作专家的多步去噪;以及专家间的等待,即视频专家与动作专家只能先后串行执行。RealtimeWAM 把这两处一并移除。它提出的「教师锚定一致性蒸馏」(TACD)针对一个局部-全局误差差:仅有较低的局部一致性误差,并不保证最终动作准确,因此 TACD 在局部一致性之外,额外用冻结教师的多步 rollout 终点做显式监督,从而实现准确的单步动作生成。随后「跨专家波前流水线」(CEWP)通过分块共享视频 KV 缓存来重叠两个专家,只在对应的动作注意力即将消费它之前才同步。在 LIBERO、LIBERO-Plus 与 RoboTwin 及 Fast-WAM、Faster-WAM 等变体上,RealtimeWAM 在近乎无损(下降不到 1%)的同时,于 H100 上取得约 25 倍的端到端加速,并开源了代码与权重。
H-JEPA:面向视觉规划的层次化世界模型端到端学习
用隐空间世界模型做长时程规划,需要在多个时间尺度与抽象层级上推理,而现有的任务无关 JEPA 世界模型要么只在单一时间尺度上预测与规划,要么把多个时域塞进同一个共享隐空间。H-JEPA 提供了一套端到端训练「动作条件化 JEPA 层级」的方案:每一层都在自己学到的隐空间里预测得更远。规划自上而下进行:顶层优化朝目标推进的进度,而每一层的预测又成为下一层规划器的子目标。当数据中的因素在彼此分离的时间尺度上演化时,高层会丢弃快速、不可预测的细节,只保留较慢的任务相关状态。在四个仿真导航与操作环境中,分层规划都优于扁平的 JEPA;在 Visual AntMaze 上,三层层级把成功率从 18% 提升到 73%,且规划算力更低,消融实验把收益同时归因于时间分解与高层目标表征。借助逆动力学监督,该方法还能扩展到 DROID 的真实机器人视频,在更低规划算力下提升离线规划保真度。
InterMimicGen:用自进化动作模仿扩展人形机器人的移动-操作
采集到的人-物交互数据是人形机器人移动-操作训练中极有价值的监督信号,但它们稀疏、异构,且无法被机器人直接执行。InterMimicGen 是一套自进化的动作模仿框架,让机器人运动数据与跟踪策略彼此促进。它先整合动作捕捉的人-物交互数据集,在保留全身协调性与灵巧手-物关系的前提下重定向为人形机器人参考动作,得到大规模且多样的参考集合;随后训练一个基于物理的通用跟踪器,在仿真中用带灵巧手的仿人机器人执行这些参考,规模与多样性都超过以往用于移动-操作的仿人跟踪系统。最后它闭合一条数据飞轮:每一轮都对待交互发生的位置与身体的执行方式做微小的、保任务的改动,用这些变体微调跟踪器,并只保留仿真执行能够完成任务的变体,它们再作为下一轮的种子。作者报告了跨机器人构型的接触保持重定向、单一通用策略的广覆盖跟踪、随增强轮次不断增长的可执行动作,以及向真机的迁移。
KungfuAthleteBot: Learning High-Dynamic Humanoid Motion from Video with Unified Robust Recovery
Video is a cheap, abundant source of human motion, and martial-arts leaps and landings embody the high-dynamic capabilities humanoid robots want most, yet video motion is physically inconsistent, carries no actuation information, and is silent on failure and recovery. KungfuAthleteBot (KAB) resolves each failure mode: physics-guided parabolic trajectory correction removes floating, ground penetration and jitter; physics-driven pseudo-low-kinetic-energy sampling biases policy initialization toward dynamically feasible states; and disturbance rejection plus fall recovery are learned in the same policy that tracks the video motion. The humanoid thus learns high-dynamic skills from video and recovers from arbitrary falls in about 0.7s.
I2CD: Direct Image-to-Convex Decomposition for Simulation-Ready Collision Geometry
Physics simulators and motion planners need convex collision geometry, but image-to-3D generative models output dense, often non-manifold visual meshes, bridged only by a slow, brittle reconstruct-then-decompose pipeline. I2CD predicts a convex decomposition directly from a single RGB image: it freezes pretrained Hunyuan3D-2 and trains only a 38M-parameter cross-attention head whose convex-slot tokens emit the halfplane parameters of K convex polytopes, yielding geometry that is convex by construction and loads into physics engines with no post-processing at about 0.5s per image. It attains the highest volumetric IoU on 227 held-out instances at 6 to 37 times speedup, and generates planner-ready geometry for a 20-object real scene in 11s versus 328s for the strongest baseline.
EyeRobot 2.0: Active Gaze for Precise Manipulation without Wrist Cameras
Wrist cameras give fine bimanual manipulation a reliable close-up of the hand, until a grasped object blocks the lens. EyeRobot 2.0 borrows from human vision: a pair of steerable viewpoints actively fixate a 3D point in the scene and process the foveal images with center-weighted token allocation. Hierarchical reinforcement learning coordinates gaze through a low-level servoing policy plus a task-progress-driven target selector, and gripper information is canonicalized into a fixation-relative SE(3) frame. Across 7 real and 6 simulated tasks and over 1000 physical plus 1800 simulated trials, stereo alone beats passive stereo by 40% in the real world and 20% in sim, more than doubling success under wrist occlusion (48% vs 22%).
KungfuAthleteBot:从视频学习高动态人形动作,并统一鲁棒的跌倒恢复
视频是廉价而丰富的人体动作来源,尤以武术的腾空落地最能代表人形机器人渴望的高动态能力,但视频运动物理不自洽、不含驱动力信息、也不谈失败恢复。KungfuAthleteBot(KAB)逐一化解这三种失效模式:用物理引导的抛物线轨迹校正去除漂浮、穿地与抖动;用物理驱动的伪低动能采样把策略初始化偏到动力学可行状态;并让抗扰与跌倒恢复和视频动作跟踪在同一策略里学习。人形机器人由此从视频学到高动态技能,且能在约 0.7 秒内从任意跌倒中恢复。
I2CD:从单张图像直接预测可用于仿真的凸碰撞体分解
物理仿真与运动规划需要凸碰撞几何,而图像到三维生成模型输出的是稠密、常非流形的视觉网格,二者之间只能靠缓慢脆弱的「先重建再分解」流水线衔接。I2CD 直接从单张 RGB 图像预测凸分解:它冻结预训练的 Hunyuan3D-2,只训练一个 3800 万参数的交叉注意力头,让「凸槽」token 输出 K 个凸多面体的半平面参数,产出按构造即凸、可被物理引擎直接加载的几何,单张约 0.5 秒。在 227 个留出实例上体素 IoU 最高、端到端快 6 到 37 倍;真机 20 物体场景生成可规划几何 11 秒(最强基线 328 秒),抓放成功率相当。
EyeRobot 2.0:用主动注视在没有腕部相机的情况下完成精细双臂操作
腕部相机一直是精细双臂操作的可靠近距视角,但一旦抓持物挡住镜头便直接失效。EyeRobot 2.0 借鉴人眼,用一对可转动的视点主动把注视点对准场景中的三维目标,并以中央凹方式分配视觉 token;它用分层强化学习协调注视(低层伺服策略加任务进展驱动的目标选择器),并把夹爪信息规范化到与注视相关的 SE(3) 坐标系。在 7 真实 6 仿真任务、1000+ 真机与 1800 仿真试验中,仅用立体相机即比被动立体真实成功率高 40%、仿真高 20%,在腕部遮挡时成功率翻倍以上(48% vs 22%)。
GlassGuard: Verified Glass Plane Mapping for Robot Navigation
Transparent and specular surfaces break LiDAR navigation: laser returns can pass straight through glass, so collision boundaries are simply absent from the map. Prior work tries to reconstruct the missing surfaces, but inaccurate placement creates the opposite failure — it contaminates traversable free space. GlassGuard is a navigation-oriented framework that reconstructs planar architectural glass from complementary visual and LiDAR evidence while treating both success criteria explicitly: glass coverage and free-space contamination. A foundation vision model supplies glass-instance masks, structural 3D cues generate metric plane hypotheses, and depth-free 2D projective geometry checks their orientation before they enter a consolidated global map. Evaluated in nine building-scale scenes with over an hour and 2.1 km of real-world robot traversal, the panoramic version reaches 85% of total glass coverage; under identical pinhole inputs GlassGuard reaches 82% coverage versus at most 61% for the evaluated baselines, while producing 5-17x fewer false voxels per frame. Qualitative examples with a navigation planner show the reconstructed planes blocking paths through glass while leaving traversable routes open.
InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulation
A controller that was never trained for a task still holds much of the competence that task needs. InterEvolve exploits this for humanoid loco-manipulation by treating test-time evolution as a problem of interface design: give a planner an expressive but executable interface to a broad controller, and execution feedback can guide the search for new behaviors. Its first component is an object-aware forward-backward behavioral foundation model whose object residuals, added to a frozen body prior, turn a newly specified reward about the body or the objects into loco-manipulation behavior at test time. Its second component specifies tasks as reward programs — staged rewards with completion conditions and tunable constants — which an LLM agent revises in context using execution feedback and a library of verified programs while a numerical optimizer tunes the constants. Every candidate is verified across parallel simulation scenarios before it is kept, so the program keeps discovering ways to induce, repurpose and compose the controller's existing motor competence. Human-designed rewards leave much of the foundation model's competence untapped; the evolved programs release it, sometimes through novel strategies, produce behaviors for diverse tasks, complex scenes and long-horizon compositions, and run autonomously on a physical Unitree G1 from egocentric onboard perception.
Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents
Building reliable robot competence across many tasks usually costs a great deal of human effort spent maintaining skills, designing rewards and wiring perception to control. RPG (Reconstruct, Practice, Go Real) is a framework that improves a robot execution system autonomously without updating any model weights. It identifies manipulation capabilities already latent in an offline dataset and constructs related practice tasks in simulation; during practice it diagnoses failures from execution feedback, privileged simulator state and available dataset videos, then develops new reusable symbolic skills, refines existing ones and revises its own system prompt. Cross-task evaluation admits each candidate change, and merged revisions, only after they pass. At test time a multimodal LLM uses the resulting system prompt and skill library to coordinate perception and robot control. On held-out initializations of 22 manipulation tasks, success rises from 28.6% after the first practice round to 95.0% after fifteen, beating every evaluated baseline including ASPIRE (75.5%) and GPT-6 Astra Pro-driven CaP-Agent0 (60.0%). After a common calibration and hardware-adaptation procedure, the frozen system succeeds in all 30 physical trials, ten each on three tasks.
GlassGuard:面向机器人导航的可信玻璃平面建图
透明与高反光表面对基于激光雷达的导航是一个顽疾:激光回波会径直穿过玻璃,于是碰撞边界根本不会出现在地图里。已有工作尝试重建这些缺失的表面,但若位置放得不准,反而制造出相反方向的失败——把可通行的自由空间也污染成障碍。GlassGuard 是一个面向导航的框架,它用视觉与激光雷达的互补证据重建建筑中的平面玻璃,并把两个成功判据都明确纳入:玻璃覆盖率与自由空间污染。一个基础视觉模型给出玻璃实例掩码,结构性 3D 线索生成带度量尺度的平面假设,再由不依赖深度的 2D 投影几何在校验其朝向之后,才允许其进入全局地图。在九个建筑尺度的场景中评测、累计超过一小时与 2.1 公里的真实机器人行驶,其全景版本达到 85% 的玻璃总覆盖率;在相同针孔输入下,GlassGuard 达到 82% 的覆盖率,而所评测基线最高仅 61%,同时每帧产生的虚假体素少 5–17 倍。与导航规划器结合的定性示例显示,重建出的平面会挡住穿玻璃而过的路径,同时保留可通行的路线。
InterEvolve:面向人形全身操作的任务奖励程序测试时演化
一个从未为某个任务训练过的控制器,其实往往已经握有该任务所需的大部分能力。针对人形全身操作(loco-manipulation),InterEvolve 把测试时演化当成一个接口设计问题来处理:只要给规划层一个足够有表达力、又足够可执行的接口去对接一个通用控制器,执行反馈就能引导系统去搜索新的行为。它的第一个组件是一个物体感知的前向—后向(forward-backward, FB)行为基础模型:在被冻结的身体先验之上叠加物体残差,就能在测试时把一个关于身体或物体的新奖励转化为全身操作行为。第二个组件把任务表达为「奖励程序」——带完成条件与可调常数的分阶段奖励;一个 LLM 智能体借助执行反馈与一个已验证程序库在上下文中修改程序结构,同时由数值优化器调整其中的常数。每个候选方案都会被放到并行的仿真场景中验证,通过后才被保留,于是程序不断发现新的方式去诱导、复用与组合控制器已有的运动能力。人工设计的奖励会让基础模型的大量操作能力闲置,而演化出的程序把它们释放出来,有时甚至给出全新的策略;它能为多种任务、复杂场景与长程组合生成行为,并在一台真实的 Unitree G1 上、仅凭第一视角机载感知自主运行。
重构、练习、走向真实:面向具身智能体的引导式自我改进
让机器人在众多任务上具备可靠能力,通常要耗费大量人力:开发并维护技能、设计奖励、再把感知接入控制。RPG(Reconstruct, Practice, Go Real)是一个在不更新任何模型权重的前提下让机器人执行系统自主改进的框架。它先从离线数据集中识别出系统本就潜藏的可操作能力,再在仿真中构造相关的练习任务;练习过程中,它借助执行反馈、特权仿真状态与可用的数据集视频来诊断失败,据此开发新的可复用符号技能、精修既有技能,并修订自身的系统提示词。跨任务评测会对单个候选改动以及合并后的修订逐一放行,只有通过才予以保留。测试时,一个多模态大语言模型依据最终的系统提示词与技能库来协调感知与机器人控制。在 22 个操作任务的留出初始化上,成功率从第一轮练习后的 28.6% 提升到第十五轮后的 95.0%,超过所评测的全部基线,包括 ASPIRE(75.5%)与由 GPT-6 Astra Pro 驱动的 CaP-Agent0(60.0%)。在统一的标定与硬件适配流程之后,冻结的系统在全部 30 次物理试验中成功,三个任务各十次。
When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models
Vision-language-action (VLA) models can clear 90% success on in-distribution tasks and tolerate nuisance changes that preserve the required action, yet fail under counterfactual changes that demand a different action. The authors name this failure instruction-action binding: the policy responds to both language and vision but does not combine them to select the action the task actually requires. Behavioral analysis of fine-tuned π0.5 and GR00T-N1.7 policies shows failed rollouts either retain the source task's trajectory family or switch to another demonstrated task, which means language is not simply ignored. State readouts and interventions link these choices to task-conditioned internal states, and an analysis of the imitation objective shows that when the conditional action support is narrow, grounded and instruction-keyed solutions become indistinguishable on the demonstrations. Equivariant Counterfactual Training (ECT) therefore acts at two levels: its data supply valid demonstrations in which the same instruction requires different actions in distinguishable scenes, and its loss trains each demonstration together with its counterpart in the same update. In a controlled LIBERO-PRO comparison, full ECT raises π0.5's mean position-swap success from 36% to 59%; on CALVIN, the ECT loss improves five-task completion without new demonstrations; and on a real UR5e under a fixed demonstration budget, it raises unseen-position success from 8% to 88%.
EWAM: Emergent Depth-Wise Specialization in a Unified Embodied Model -- From Semantic Understanding through Visual Foresight to Action
Vision-language-action (VLA) policies emphasize semantic understanding, whereas world-action models (WAMs) learn predictive representations of environment dynamics. Systems that expose a policy to both sources often still concentrate action computation on a single expert. EWAM is an action-centric unified embodied model whose asymmetric joint attention lets action tokens read semantic, current-visual, predicted-future and action information at every layer while the perceptual experts retain their distinct roles. Notably, with no layer-wise supervision EWAM develops an emergent depth-wise specialization: action queries attend mainly to vision-language features in shallow layers, to predicted future frames in intermediate layers, and to action tokens themselves in deep layers. The handoff replicates across tasks and is stable across denoising steps, and checkpoint tracking plus causal interventions show it is learned and that action generation depends on it. EWAM is pretrained in two regimes, one on cross-embodiment robot trajectories and one on human egocentric video. In simulation and real-robot experiments it surpasses existing VLA, WAM and hybrid baselines; human egocentric data improve cross-embodiment transfer and real-robot robustness, and subtask-phase supervision improves long-horizon completion.
DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot Agents
Pretrained robot policies provide useful action priors, but long-horizon manipulation still requires coordination between semantic reasoning and physical execution. The two operate at different timescales — semantic reasoning is coarse, physical interaction fine — and episode-level failures give only vague feedback that cannot answer which part of the system should be revised. DynaHarness proposes a dynamic physical harness that couples semantic reasoning with physical governance through a shared execution contract and turns failure evidence into validated capability revisions. A slow brain proposes capabilities and symbolic arguments, while a fast brain grounds and monitors commands, refuses unresolved actions, substitutes capabilities and requests replans when needed. The physical execution contract bounds each accepted command and records execution evidence across analytic skills, recovery skills and the frozen VLA. Failure attribution localizes faults within these records and directs targeted revisions of reusable capabilities or execution mechanisms, after which paired regression checks govern admission and rejection, closing the self-evolution loop. On LIBERO-Pro, DynaHarness reaches 75.2% on 800 newly sampled initial states versus 17.5% for the frozen policy; with the same capability library, full dynamic execution reaches 74.0% versus 63.9% under nominal one-step replanning.
当指令只是在检索轨迹:诊断并缓解 VLA 模型的泛化失效
视觉-语言-动作(VLA)模型在同分布任务上可以做到 90% 以上的成功率,也能承受那些不改变所需动作的扰动,但在要求它换一个动作的反事实变化面前却会失效。作者把这种失效命名为「指令-动作绑定」:策略确实对语言和视觉都有反应,却没有把二者结合起来,去选出任务真正要求的动作。对微调后的 π0.5 与 GR00T-N1.7 的行为分析显示,失败的 rollout 往往会保留原任务的轨迹族,或者切换到另一个演示过的任务——也就是说,语言并没有被忽略。作者用状态读出与干预把这种选择关联到任务条件化的内部状态,并从模仿目标出发指出:当条件动作的支撑过窄时,演示数据上「真正接地气」的解与「仅凭指令检索熟悉轨迹」的解无法区分。为此提出等变反事实训练(ECT),它在两个层面同时发力:数据上给出同一指令在可区分场景中要求不同动作的有效演示,损失上则让每条演示与其对应样本在同一次更新中一起训练。在受控的 LIBERO-PRO 对比中,完整 ECT 把 π0.5 的平均位置交换成功率从 36% 提升到 59%;在 CALVIN 上,无需新增演示即可改善五任务完成率;在真机 UR5e 上,固定演示预算下未见位置的成功率从 8% 提升到 88%。
EWAM:统一具身模型中涌现的逐层分工——从语义理解、视觉预见到动作生成
视觉-语言-动作(VLA)模型强调语义理解,而世界-动作模型(WAM)学习对环境动力学的预测性表征。那些同时把两类信息暴露给策略的系统,往往仍把动作计算集中在单一专家之上。EWAM 提出一个以动作为中心的统一具身模型,它的非对称联合注意力让动作 token 在每一层都能读取语义、当前视觉、预测未来与动作本身的信息,同时让各感知专家保留各自不同的角色。值得注意的是,在没有任何逐层监督的情况下,EWAM 自发形成了「逐层分工」:动作查询在浅层主要关注视觉-语言特征,在中间层转向预测的未来帧,在深层则主要关注动作 token 自身。这一接力在多个任务上复现,并在不同去噪步之间保持稳定。检查点追踪与因果干预表明,这种分工是学出来的,且动作生成确实依赖它。EWAM 在两种范式下预训练:一种是跨本体的机器人轨迹,另一种是人类第一视角视频。在仿真与真机实验中,它超过了现有 VLA、WAM 与混合基线;人类第一视角数据同时改善了跨本体迁移与真机鲁棒性,子任务阶段监督则提升了长时程任务的完成度。
DynaHarness:面向自我演化机器人智能体的动态物理治理框架
预训练机器人策略提供了有用的动作先验,但长时程操作仍然需要在语义推理与物理执行之间做协调。二者的时间尺度并不一致——语义推理更粗,物理交互更细——而片段级的失败只能给出模糊的反馈,无法回答「究竟该修改系统的哪个部件」。DynaHarness 提出一个动态物理治理框架,通过一份共享的执行契约把语义推理与物理治理耦合起来,并把失败证据转化为经过验证的能力修订。具体而言,慢脑负责提出能力与符号参数,快脑负责落地与监视命令、拒绝未解析的动作、替换能力,并在必要时请求重规划。物理执行契约对每条被接受的动作作出边界约束,并跨解析技能、恢复技能与冻结的 VLA 记录执行证据。失败归因在这些记录中定位故障,指向对可复用能力或执行机制的有针对性修订,再由成对的回归检查决定接纳或拒绝,从而闭合自我演化的循环。在 LIBERO-Pro 上,DynaHarness 在 800 个新采样初始状态上达到 75.2%,而冻结策略只有 17.5%;在同样的能力库下,完整的动态执行达到 74.0%,而名义单步重规划为 63.9%。
WorldLine: A Video World Model That Predicts Manipulation Before the Robot Moves
Robot learning is throttled by the cost of collecting experience and testing candidate behaviours on hardware. Video generation promises a visual simulator that predicts outcomes before execution, but existing models favour visual plausibility over faithful action-following and coherent robot-object dynamics, while action-conditioned simulators need scarce, embodiment-specific data. WorldLine decouples transferable dynamics learning from heterogeneous action grounding: it learns manipulation dynamics from over 10,000 hours of action-free robot video and grounds them with more than 2,000 hours of action trajectories across more than ten embodiments, using an image-space action representation as a shared control interface. Multi-view and failure-enriched training with relational regularization sharpens interaction-sensitive prediction, and robot-focused few-step distillation keeps causal rollout cheap. On failed trajectories it improves robot-mask IoU by 0.1626 over the strongest baseline, predicts trajectory success with 74% mean accuracy across RoboTwin and AgiBot, and - without any RoboTwin training or adaptation - its rollouts lift task success by up to 21.4 percentage points over direct policy execution.
Rho: Open-Weights VLA Models That Adapt to a New Arm in Fifteen Corrected Episodes
General-purpose robot models must combine broad visual and linguistic competence with precise control across embodiments and cheap adaptation to new tasks. Rho is a family of open-weights vision-language-action models for bimanual manipulation, aimed at data-light task adaptation on three representative dual-arm platforms: YAM Box, UR AI Trainer and FR3 Duo. The authors ablate the action-expert architecture and training recipe and show, in controlled simulation and physical-robot experiments, that embodiment midtraining improves downstream adaptation. The resulting Rho variants match or beat existing open-weights VLAs on the tasks, embodiments and baselines evaluated. Rho also carries a built-in online-adaptation mechanism: an internal latent policy learns from corrective feedback to select observation-conditioned noise inputs for a frozen flow-matching action expert, so that as few as 15 corrected episodes are enough to handle situations at the fringe of the offline finetuning distribution. The base model and embodiment-specific checkpoints are released.
PRISM: Turning a Handful of Real Videos into Scalable Humanoid Loco-Manipulation
Teaching a humanoid to carry objects by watching people is limited less by policy design than by data: useful clips must show a whole body and an unoccluded interaction with the object, and such footage is scarce. PRISM is a real-to-sim-to-real framework that amplifies a handful of real videos into a large, diverse training set. It generates hundreds of counterfactual human-object interaction videos with video-to-video generation, then uses a contact-anchored real-to-sim pipeline to reconstruct human and object motion and retarget the imperfect video into physically plausible trajectories. The intra-class variability across these videos lets a single policy generalize to unseen objects within each category. Deployed on a real humanoid with no real-world fine-tuning, using only onboard depth, the policy picks up, carries and drops boxes, barrels, bins and balls across novel instances, sizes and initial configurations.
WorldLine:在机器人动之前,先预测操作结果的视频世界模型
机器人学习的瓶颈,既在于采集经验的成本,也在于在真机上验证候选行为的代价。视频生成有望提供一种「执行前先预测结果」的视觉仿真器,但现有模型往往为了画面逼真而牺牲了对动作的忠实跟随与连贯的机器人—物体动力学;而依赖动作条件的仿真器,又需要稀缺的、与特定本体绑定的数据。WorldLine 把「可迁移的动力学学习」与「异构动作的落地(grounding)」解耦:它从超过 10,000 小时无动作标注的机器人视频中学习操作动力学,再用跨越十余种本体、超过 2,000 小时的动作轨迹把它落地,并以图像空间的动作表示作为共享控制接口。多视角与失败富集训练配合同关系正则化,强化了对交互敏感片段的预测;面向机器人的少步蒸馏则让因果推演足够廉价。在失败轨迹上,它把机器人掩码 IoU 相对最强基线提升 0.1626;在 RoboTwin 与 AgiBot 上以 74% 的平均准确率预测轨迹成败;并且在不做任何 RoboTwin 训练或适配的前提下,其推演把任务成功率相对直接执行策略最多提升 21.4 个百分点。
Rho:开源权重的 VLA 模型,用十五次纠正示范适配一条新机械臂
通用机器人模型必须同时具备宽泛的视觉与语言能力、跨不同本体结构的精准控制,以及针对新任务的低成本适配。Rho 是一系列面向双臂操作的开源权重视觉-语言-动作(VLA)模型,目标是在三种具有代表性的双臂平台上实现数据友好型任务适配:YAM Box、UR AI Trainer 与 FR3 Duo。作者系统消融了动作专家(action expert)的架构与训练配方,并在受控仿真与真机实验中表明,本体结构中训练阶段(embodiment midtraining)能提升下游适配效果。由此得到的各 Rho 变体在所评估的任务、本体与基线上达到或超过现有开源权重 VLA。Rho 还内建了在线自适应机制:一个内部隐策略从纠正反馈中学习,为冻结的流匹配动作专家挑选以观测为条件的噪声输入,因此少至 15 段纠正示范就足以应对处于离线微调分布边缘的情形。基础模型与各本体的检查点均已开源。
PRISM:用少量真实视频,放大出可扩展的人形机器人移动操作能力
让人形机器人通过观看人类视频学会搬运物体,主要瓶颈不在策略设计,而在数据:有用的视频片段必须同时拍到人的全身,以及人与物体之间不被遮挡的交互,而这类素材恰恰非常稀缺。PRISM 是一条「实—虚—实」框架,它把少量真实视频放大成一个规模大、多样性高的训练集:先用视频到视频(V2V)生成数百段反事实的人—物交互视频,再用一条以接触为锚的实-虚流程重建人与物体的运动,把这些不完美的视频数据重定向为物理上合理的轨迹。这些反事实视频带来的类内差异,让同一条策略能够泛化到每个类别中未见过的物体。该方法部署在真机人形机器人上,不做任何真实世界微调,仅依靠机载深度观测,即可完成箱子、油桶、周转箱与球类等物体的抓取、搬运与放置,并适应新的实例、尺寸与初始构型。
Humanoid Loco-Manipulation With Discrete VLA Model
Whole-body humanoid control spans legs, torso, arms and hands, an action space far more heterogeneous than the single arm that discrete-token VLA models were built for. Holo-M extends the language model's own vocabulary with action tokens, decomposing the humanoid action space into four body-part tokenizers (end-effector, body, hand, kinematics) so a single model can train across teleoperation, egocentric human video and simulation without the separate continuous action experts that insulate knowledge. Grouped discrete diffusion decoding replaces autoregression to meet real-time control needs. On the SIMPLE humanoid loco-manipulation benchmark it takes the top spot in both generalist and specialist evaluations, ahead of the second best by significant margins, and the authors plan to release code and weights.
RoboCompiler: Graph-Native Compilation of Closed-Chain Robots for Consistent Modeling, Control, and Simulation
Robots with kinematic loops, coupled actuators and changing contacts need models of configuration, motion, force and dynamics that agree with one another, yet those interfaces are usually rebuilt separately for control and for simulation, so closure and actuation consistency is hard to hold. RoboCompiler compiles a canonical mechanism graph into one shared mechanical interface. It builds closure paths and analytic residual Jacobians from bodies, joints, frames, inertias and actuator ports, assembles feasible configurations with rank-checked continuation, and uses a tangent lift plus paired actuator-port maps to keep velocities and virtual work consistent, extending the reduction to accelerations and projected rigid-body dynamics. Validated on a Komatsu excavator, a Unitree Go2, a Franka Panda, Kangaroo and a six-UPS Stewart platform, it cuts residual-and-Jacobian evaluation time by 96.7% and closed-loop rollout wall time by 66.8% on Kangaroo, with dynamics and control held fixed.
DexRoam: Learning Mobile Bimanual Dexterous Manipulation from Egocentric Whole-Body Human Demonstrations
Mobile bimanual dexterous manipulation must coordinate locomotion, whole-body posture and finger-level dexterity inside a single trajectory, and that coupling is exactly why real-robot data is so scarce. DexRoam learns such skills from egocentric human demonstrations captured with only a consumer VR headset and a head-mounted stereo camera, with no motion-capture rig. Three explicit alignment stages (embodiment, action-semantic, temporal) map the human motion into the robot action space while keeping whole-body motion continuous and coupled, so standard VLA policies can train on humans and robots together. On real robots, human demonstrations consistently help: average success rises from 29% to 56% on GR00T N1.7 and from 32% to 57% on pi0.5, and robot-only performance can be matched using half the robot demonstrations.
用离散 VLA 模型实现人形机器人移动操作
人形机器人的全身控制横跨腿、躯干、手臂与手,其动作空间远比离散动作词元 VLA 模型最初面向的单臂复杂且异质。Holo-M 把动作词元直接扩展进语言模型自身的词表,将人形动作空间分解为四个身体部位分词器(末端执行器、身体、手、运动学),使单个模型得以跨遥操作、第一视角人类视频与仿真训练,而无需会隔离知识的独立连续动作专家。为满足实时控制,模型用分组离散扩散解码取代对动作词元的自回归。在 SIMPLE 人形移动操作基准上,它在通用与专用两项评测中都取得最高成功率,且显著领先第二名;作者计划开源全部代码与权重。
RoboCompiler:面向闭链机器人的图原生编译,统一建模、控制与仿真
带有运动学闭环、耦合驱动与接触变化的机器人,需要构型、运动、力与动力学之间彼此一致的模型;但这些接口往往分别为控制与仿真各自重建,导致闭环与驱动的一致性难以维持。RoboCompiler 把一份规范化机构图编译成一个共享的力学接口。它由刚体、关节、坐标系、惯量与驱动端口出发构造闭环路径与解析残差雅可比,用秩校验的连续延拓与修正组装可行构型,并通过切映射与成对的驱动端口映射保持速度与虚功一致,进而把约简扩展到加速度与投影刚体动力学。在 Komatsu 挖掘机、宇树 Go2、Franka Panda、Kangaroo 与六-UPS Stewart 平台上验证后,它在 Kangaroo 上把残差与雅可比的求值时间降低 96.7%、把闭环推演的墙钟时间降低 66.8%,且动力学与控制保持不变。
DexRoam:从第一视角全身人类演示中学习移动双臂灵巧操作
移动双臂灵巧操作要求在单条轨迹内同时协调行走、全身姿态与手指级灵巧性,这种耦合正是机器人数据如此稀缺的原因。DexRoam 从仅用消费级 VR 头显加头戴立体相机采集的第一视角人类演示中学习这类技能,无需动作捕捉设备。三个显式对齐阶段(具身对齐、动作语义对齐、时序对齐)把人类动作映射进机器人动作空间,同时保持全身运动的连续与耦合,使标准 VLA 策略能够联合学习人类与机器人演示。在真机上,人类演示一致带来提升:GR00T N1.7 上平均成功率从 29% 升到 56%,π0.5 上从 32% 升到 57%,且用一半的机器人演示即可达到纯机器人训练的水平。
InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
Generalist manipulation wants one model that predicts how a scene will change and what to do about it, but the useful priors - visual dynamics, scene semantics, geometry, motion - usually live in separate pretrained models. InternW0-Δ fuses them in a Mixture-of-Transformers world action model: a video expert and an action expert interact under guidance from a frozen VLM, a pretrained 4D foundation model injects geometric and motion priors through training-only distillation, and a Causal Imprint head learns future-relevant scene changes without any future-video rollout at inference. It is pretrained on a heterogeneous corpus the authors describe as 20K+ hours, the largest open-source corpus of its kind, and reports gains across simulation benchmarks and real-robot platforms, with code, weights and data promised for release.
Learning to Leverage Compliance: A Policy-Admittance Learning Framework for Robotic Insertion
Policy learning and compliant control are both routes to reliable autonomous assembly under pose error and contact uncertainty, but simply combining them does not make them coordinate: the policy can keep pushing against contact while the admittance controller yields, producing sustained loading with little progress. LeCo turns that interaction into a reward signal. A multirate feedback mechanism aggregates high-rate contact records into policy-transition rewards, an integrated conflict cost penalizes sustained policy-loading versus controller-unloading opposition, and a directional high-force tail cost catches continued-loading events within a transition. On four real connector-assembly tasks LeCo reaches 94% aggregate success, with successful-trial force and torque peaks roughly 30% and 64% below the comparison baseline and about 53% lower contact-conditioned conflict density once conflict shaping is added.
Generate, Track, Improve: Perceptive Multi-Skill Humanoid Locomotion with RL-Fine-Tuned Motion Generators
A general-purpose humanoid needs one controller that can walk, run, jump and climb stairs over whatever terrain it meets, without leaning on odometry or a prebuilt height map. This work splits the job into two layers: a perceptive flow-matching motion generator that plans whole-body trajectories directly from raw depth images, and a perceptive tracking policy trained with control-guided reinforcement learning that follows those motions. Both are trained on terrain-consistent motion clips derived from dynamically optimized human data. The central contribution is an off-policy RL fine-tuning loop that improves the generator itself, using structured search to gather data for advantage-weighted regression - far more sample-efficient than fine-tuning the tracker with on-policy residuals. Terrain traversals improve by up to 25 percentage points and skill selection by up to 80 percentage points, and a single policy pair drives a Unitree G1 to walk, run, stand, jump on and off boxes and climb outdoor stairs.
InternW0-Δ:以 2 万小时以上开放数据连接预测动力学与动作的世界动作模型
通用操作希望用同一个模型既预测场景将如何变化、又决定该采取什么动作,但有价值的先验——视觉动力学、场景语义、几何与运动——通常分散在若干各自独立的预训练模型中。InternW0-Δ 用混合 Transformer(MoT)把它们融合进一个世界动作模型:视频专家与动作专家在一个冻结 VLM 的语义引导下交互,一个预训练的 4D 基础模型通过仅训练期使用的蒸馏注入几何与运动先验,而 Causal Imprint 模块学出与未来相关的场景变化,推理时完全不推出未来视频。它在作者称为同类中最大开源规模的 2 万小时以上异构语料上预训练,并在仿真基准与真机平台上报告了性能提升,同时承诺开源代码、权重与数据。
生成、跟踪、改进:基于 RL 微调运动生成器的感知型多技能人形机器人运动控制
通用人形机器人需要一个控制器,能在遇到的各种地形上行走、奔跑、跳跃与爬楼梯,而不依赖里程计或预先建好的高度图。这项工作把任务拆成两层:一个感知型流匹配运动生成器直接从原始深度图像规划全身轨迹,一个以控制引导强化学习训练的感知型跟踪策略负责跟踪这些运动;两者都建立在由动力学优化的人类数据生成的地形一致动作片段上。其核心贡献是一个离策略 RL 微调回路:用结构化搜索采集数据并做优势加权回归来改进生成器本身,样本效率远高于对跟踪器做在策略残差微调。地形穿越成功率提升最多 25 个百分点,技能选择提升最多 80 个百分点,且单个策略对即可让一台宇树 G1 行走、奔跑、站立、跳上跳下箱体,并在户外环境中攀爬楼梯。
学会顺应柔顺性:面向机器人插装的策略-导纳学习框架
策略学习与柔顺控制都是在位姿误差和接触不确定性下实现可靠自主装配的路径,但把两者简单组合并不保证它们协调:策略可能持续顶住接触面,而导纳控制器却在退让,于是产生持续加载却进展有限。LeCo 把这种交互转化为奖励信号。一个多速率反馈机制把高频率的接触交互记录聚合成策略转移奖励;一项积分冲突代价刻画持续的策略加载与控制退让之间的对抗;一项方向性高力尾代价则捕捉单次转移内持续加载的事件。在四个真实连接器装配任务上,LeCo 取得 94% 的整体成功率,成功试验的力与力矩峰值相对对比基线分别下降约 30% 与 64%,加入冲突塑形后接触条件化冲突密度中位数下降约 53%。
More Room for Magnets: Expanding Electromagnetic Navigation Workspace with Feedback Control
Magnetically guided surgical tools are limited less by what magnets can do than by how much current a coil array can push before it overheats: the usable workspace of an electromagnetic navigation system (eMNS) is carved out by power and thermal budgets. This paper shows that the shape of that workspace is largely a control-design choice. By moving from field-centric field alignment to motion-centric torque and force objectives, allocating coil currents in an energy-optimal way, closing the loop with real-time pose estimation, and using high-bandwidth hardware, the authors stabilise a 3D inverted pendulum on an eight-coil OctoMag system at 0.1–0.2 A instead of 8–14 A. The same ideas extend to multi-agent control, with two inverted pendulums balanced in a shared workspace, and transfer to the clinically oriented Navion platform, which maintains stable balancing up to 50 cm from the coils.
World In Your Hands: A Thousand Hours of In-the-Wild Manipulation Data, Made Open
Robot manipulation is gated less by policy architecture than by data, and World In Your Hands (WIYH) attacks that constraint at the ecosystem level: a wearable capture kit with an automatic labelling pipeline, a dataset of more than 1,000 hours of in-the-wild multimodal manipulation spanning hundreds of skills, and annotations plus benchmarks that reach from perception to action. The authors report millimetre-scale motion accuracy from the capture hardware and show that folding WIYH's human data into robot training lifts manipulation success in cluttered scenes from 8% to 60%. Both the dataset and the hardware designs are released openly, positioning WIYH as infrastructure for cross-embodiment policy learning rather than a single-task result.
Whole-Body UMI: Turning Hand-Held Manipulation Demos into Humanoid Whole-Body Skills
Humanoid whole-body skills are usually taught by teleoperation, which is expensive and hard to scale. The Universal Manipulation Interface (UMI) offers a cheaper, scalable way to record hand-held manipulation, but end-effector trajectories alone do not determine how a humanoid should coordinate its entire body. Whole-Body UMI (WB-UMI) closes that gap with a task-agnostic, real-time, end-effector-conditioned motion generator that learns whole-body coordination separately from task semantics and shares only the end-effector interface between them. A diffusion policy consumes native UMI demonstrations while WB-UMI learns from retargeted motion capture, so task-specific collection needs neither body trackers nor paired image and whole-body data. On a G1 humanoid the stack transfers in closed loop across four tasks, reaching 90% success on drawer closing and 80% on shelf pick-and-place, though the more dynamic ball toss (30%) and loco-manipulation (40%) remain difficult.
StenoVLA-3D: A 3D-Aware VLA for Navigation Through Gastrointestinal Stenoses
Autonomous navigation through stenotic segments asks a policy to predict actions from texture-poor monocular observations, make safe control decisions, and retain evidence of lesions after they leave the field of view. Existing VLAs lean on visual appearance and short-term context, which limits geometric grounding and episode-level reporting. StenoVLA-3D injects point-maps into a Cosmos-Reason 2 backbone through learned geometry-gated fusion and adds a temporal state branch to model traversal progress; the reasoning-and-action backbone emits grounded reasoning alongside actions, while dedicated heads estimate stenosis shape and generate the final lesion report. The authors also release EndoCausal, an episode-level dataset with lesion annotations, actions, and temporally grounded reasoning. On 40 held-out recorded test episodes StenoVLA-3D reaches 95.2% semantic accuracy and 83.4% action accuracy, and on a physical 3-DoF endoscope it attains 88.9% and 77.8% task success in esophageal and colonic phantoms (36 trials each), substantially ahead of the evaluated baselines.
JAMB: Denoising Bimanual Actions and Future Geometry Together
The hard part of bimanual manipulation is that two arms share one 3D scene: whatever either arm does rewrites the spatial relations the other arm depends on. JAMB's answer is to denoise bimanual actions and future 3D point tracks together inside a single Transformer, so the two hypotheses correct each other at every denoising step — geometry constrains the action, and the action reshapes the predicted geometry — instead of demoting future state to auxiliary supervision or fixed conditioning, as earlier predictive diffusion policies do. Multimodal representations are grounded in a shared spatiotemporal coordinate system so that this cross-modal interaction is geometrically meaningful. Across 16 bimanual tasks in RoboTwin 2.0, JAMB averages 83.4% success, 23.9 percentage points above the strongest baseline, and on a real robot it beats action-only and auxiliary geometry-prediction methods by 50.0 and 21.2 percentage points, with better generalization to cluttered scenes and out-of-distribution backgrounds.
Learning Surgery From the Surgeon's Own Hands: In-Vivo Laparoscopic Appendectomy Without Robot-Collected Demonstrations
Most minimally invasive surgery is still performed with hand-held laparoscopic instruments, and when the operation ends almost all of the surgeon's instrument motion is discarded — only the endoscope video is kept. This paper builds an end-to-end pipeline that recovers that lost motion in the operating room and turns it into a surgical robot policy, validated on live animals. The sensing half is a surgical instrument-state logger that mounts on the shaft of a standard laparoscopic instrument and recovers its pose and jaw state from an inertial sensor, a time-of-flight sensor and a Hall sensor, with no external camera or tracker. A data pipeline then measures the latency of every sensor channel against robot ground truth and aligns the channels before forming observation-action pairs. On those demonstrations the authors train a diffusion policy with a fine-tuned DINOv3 backbone, choosing its design by closed-loop rollouts in a physics simulator reconstructed from depth maps of an ex-vivo rabbit appendix. The policy is retrained on 849 in-vivo demonstrations from four live rabbits and then deployed on four additional live rabbits with electrosurgery armed; with the surgeon selecting the surgical phase, it completed the appendectomy in three of the four animals. Both demonstration corpora are released.
MachEmbodied-U0: One Foundation Model That Both Understands the Task and Predicts How the Scene Will Change
General-purpose robot control asks a model to do four things at once: understand what the user wants, work out where to interact, anticipate how the scene will evolve, and produce precise actions. Vision-language-action models supply strong semantic priors but do not explicitly model scene dynamics; world-action models couple visual prediction to control without necessarily exposing the semantic and spatial structure that fine-grained manipulation needs. MachEmbodied-U0 (ME-U0) is a unified embodied foundation model that connects an understanding expert and a generation expert through a Mixture-of-Transformers architecture. Subtask prediction and affordance grounding steer joint visual-dynamics and action generation through flow matching, where the predicted visual dynamics span future RGB, depth, surface normals and optical flow, giving complementary supervision over appearance, geometry and motion. A Multi-rate Rotary Position Encoding scheme aligns those visual dynamics with fine-grained control. ME-U0 is pretrained on roughly 4,200 hours of curated demonstrations drawn from robotic and egocentric datasets, and evaluated with only the supervision each downstream benchmark natively provides: an average score of 17.66 on the RoboDojo simulation benchmark, and average success rates of 99.0 percent on LIBERO and 82.5 percent on LIBERO-Plus. It is additionally validated on real-world manipulation tasks, and shows zero-shot subtask prediction, affordance grounding and visual dynamics on simulated and real observations.
PAKT: Kinesthetic Teaching That Respects What the Robot Can Actually Do
Real-world reinforcement learning still struggles with contact-rich industrial manipulation, where the demands are micrometre-level precision, success rates above 99 percent and human-level cycle times. Off-policy methods can get there by exploiting demonstrations and human interventions, but the bottleneck is a missing interface: never mind the algorithm, how does a person on a factory floor supply guidance that is both intuitive and physically consistent with what the robot and the policy can actually reproduce? PAKT is a framework for kinesthetic teaching in reinforcement learning. Instead of teleoperation it uses kinesthetic guidance, the method industry already relies on, but it fixes that method's classic weakness: an operator can drag the robot along trajectories whose velocities, accelerations and jerk the robot or policy cannot physically reproduce. In PAKT the operator guides the robot through admittance control, which maps human-applied forces to motion, and a downstream reference generator applies the same kinematic limits used during policy execution, so every collected trajectory stays inside what the system can reproduce. Supporting this interface is a high-performance control stack that maps low-frequency RL actions to high-frequency torque commands through a reference generator and an impedance controller, where the reference generator preserves the impedance controller's tracking performance while improving contact handling and producing smoother policy actions. Across reported runs on four insertion and industrial assembly benchmarks, including a data centre compute tray, the end-to-end system cuts cycle time by 23 to 48 percent and cumulative intervention count by 62 to 86 percent relative to the HIL-SERL baseline.
MedVLA: Function-Constrained Execution Lifts Precision Medical Manipulation From Single Digits to 95%
Precision medical robotics demands adaptive decision-making under strict safety, interpretability, and execution constraints. Vision-language-action models show strong multimodal reasoning, but their continuous action-generation paradigm fits precision medical tasks poorly, where reliable closed-loop operation may also depend on non-action system function calls. MedVLA addresses this with a hierarchical framework that couples high-level multimodal reasoning to low-level function-constrained execution, paired with a scalable multi-agent pipeline that generates skill-oriented chain-of-thought data for structured training. Built on several multimodal large-model backbones, it improves consistently after fine-tuning. Across 100 closed-loop flexible electrode implantation trials under identical initial conditions, MedVLA reaches a 95.0% task success rate, substantially outperforming representative VLA baselines — OpenVLA at 8% and pi0 at 15% — in accuracy, stability, and safety.
StrataVLA: Layering 3D Geometry Into the VLA Backbone for Up to 88% Fewer Geometry Calls
Vision-language-action models inherit strong semantic priors from large-scale vision-language pretraining but remain limited in robotic manipulation by insufficient 3D spatial awareness. Existing remedies either require explicit depth or point-cloud inputs, compress geometry into training-time supervision, or inject it only at the model input or action expert, so the vision-language backbone never has persistent access to task-relevant spatial information. StrataVLA introduces a plug-and-play framework for hierarchical geometric grounding: a frozen geometry foundation model extracts shared geometric features from RGB observations, and sparse, layer-specific Geometry Adapters let visual representations at selected backbone depths retrieve geometric evidence through cross-attention. Task-aware routing and an LRU feature cache exploit temporal redundancy during manipulation. On LIBERO, SimplerEnv, and real-world manipulation it delivers consistent gains over strong VLA baselines, reaching 98.53% average success on the LIBERO suites while reducing geometry-model invocations by up to 88%.
Silent Sabotage: A Stealthy Backdoor Triggered by a Robot's Own Action History
Wiring large language models into robot control systems unlocks complex reasoning and planning, and with it a class of security risks that remain largely unexplored. Existing LLM backdoor research revolves almost entirely around external stimuli — particular words, visual objects, or environmental states. Those attacks are potent, but they overlook a more insidious vulnerability in which the trigger lives inside the agent's own operating logic. This paper presents the first comprehensive study of history-based backdoor attacks on LLM-powered robotic systems: by manipulating its instructions, an attacker can embed a stealthy backdoor in an LLM-based robot controller whose trigger is not an external cue but a specific, rare sequence of the robot's own past actions. The backdoor stays dormant during normal operation and preserves the robot's utility, yet can be activated to induce malicious behavior such as a complete stop or a collision. In simulated experiments spanning a variety of robots and LLMs, the history-based attack achieves a near-perfect success rate while remaining exceptionally difficult to detect.
RACaP: Move Coding to Evolution Time, Call Frozen Policy APIs at Run Time
Code-as-Policies methods let a model write and repair programs at run time, paying for it in latency and entangling reusable mechanisms with task-specific decisions. RACaP moves code generation out of deployment and into evolution: at run time a Reasoning-and-Acting loop calls a set of frozen, typed policy APIs, and tasks are adapted through arguments rather than rewritten source. A two-phase strategy — capability curriculum learning followed by autonomous self-evolution — improves the APIs, the ReAct harness, and an experience memory together. RACaP reaches 54.4% success on LIBERO-90, 45.0% zero-shot on LIBERO-PRO, and 46.0% on LIBERO-Long, where Code-as-Policies baselines top out at 4.0% on long-horizon tasks; on LIBERO-PRO that is 2.5x the baseline success rate and a 1.9-fold speedup in median policy time. For on-robot deployment, rejection-sampled fine-tuning distils GPT-5.6 ReAct decisions into Qwen3-VL-8B-Instruct, giving a 13.2x per-decision speedup and cutting repeated physical calls from 16 to 4.
Self-Adaptive VLA: Keeping Robot Policies Calibrated When the Hardware Drifts
Vision-language-action models manipulate impressively but are memoryless, which makes them brittle to test-time environment shifts — especially hardware shifts caused by wear or imperfect calibration. Enabling these policies to self-adapt during deployment, without continuous on-site recalibration, is a critical bottleneck for real-world scalability. Self-Adaptive VLA is a post-training recipe that lets a policy iteratively adapt to deployment-time hardware shifts using its own rollouts as context. It first collects rollouts under deliberately injected hardware shifts, then transforms the base policy's training data into shift-conditioned expert demonstrations by pre-compensating expert actions for the known shifts. A lightweight plug-in context encoder compresses the visual observation, proprioception, and actions from the shifted environment into a latent context token, which modulates the policy through adaptive layer normalization. The context tokens can also be ensembled, letting the policy self-correct and mitigate failures step by step. Across four precision-critical bi-manual and dexterous manipulation tasks, Self-Adaptive VLA recovers more than 80% of the base policy's performance under shifts such as actuation bias and joint encoder offsets, and deploys more robustly to new workstations than the base policy.
RAPID: Turning a Single Human Demo Into a Reusable Robot Program
Coding agents have proven effective at complex programming tasks, and RAPID asks whether that strength can be redirected at robot systems. Given a single visual human demonstration, RAPID automatically generates, verifies, and refines robot programs in an iterative agentic loop. To make this work it infers three ingredients from the demonstration alone: a testable task specification, the action primitives for execution, and an interactive environment for program execution and verification. To let the resulting program generalize beyond the demonstration setting, RAPID uses an object-centric relational program representation that captures the structure of the demonstrated strategy rather than the specific motion, expressing primitives as trajectory-optimization programs that realize object-level motion effects and composing them through relational constraints resolved against scene geometry at run time. It was evaluated in simulation on eight challenging contact-rich nonprehensile manipulation tasks and on LIBERO-Pro, and deployed on a real Franka arm for all eight nonprehensile tasks, showing generalization over object pose, shape, material, and environment.
AD-WM: Teaching World Models to Tell Actions Apart, Not Just Predict the Next Frame
Latent world models are usually trained to predict the factual transition that actually happened, but model predictive control needs to compare actions that never happened. A model can therefore reach low factual prediction error while barely distinguishing candidate actions. AD-WM addresses this with residual latent dynamics plus predictor-level action-recovery regularization — an inverse-dynamics head and a normalized recovery objective motivated by conditional mutual information — so that imagined transitions retain the information needed to identify the action that produced them. The auxiliary heads are dropped at test time, leaving the MPC loop unchanged. On OGBench-Cube, hard-start success rises from 3.7% to 52.0% over a matched baseline, mean success improves in four of five simulated environments, and with a frozen V-JEPA 2 encoder and matched DROID post-training, zero-shot Franka pick-and-place improves from 42.2% to 71.1% without lab-specific adaptation.
World In Your Hands:把上千小时「野外」人类操作数据开放出来
制约机器人操作能力的,往往不是策略网络,而是数据。World In Your Hands(WIYH)选择从生态层面动刀:一套可穿戴采集套件加自动标注流水线,一个超过 1000 小时、覆盖数百种技能的野外多模态操作数据集,以及从感知一直贯通到动作的标注与基准。作者报告采集硬件达到毫米级运动精度,并证明把 WIYH 的人类数据接入机器人训练后,杂乱场景中的操作成功率从 8% 提升到 60%。数据集与硬件设计均对外开放,WIYH 的定位因此不是某个任务的成果,而是跨本体策略学习的基础设施。
磁导航的「可用面积」还能更大:用反馈控制把电流降两个数量级
磁导航手术工具的真正瓶颈,往往不是磁体能做什么,而是线圈阵列在过热之前能推出多大电流:电磁导航系统(eMNS)的可用工作空间,是被功率与散热预算「切」出来的。这篇论文指出,这块空间的大小很大程度上取决于控制设计。把以磁场为中心的「对准」策略换成以运动为中心的力矩/力目标,做能量最优的线圈电流分配,用实时位姿估计闭合回路,再配上高带宽硬件,作者在八线圈 OctoMag 平台上稳定一个三维倒立摆,所需电流从 8–14 A 降到 0.1–0.2 A。同样的思路还能扩展到多智能体——在共享空间里同时平衡两个倒立摆——并迁移到面向临床的 Navion 平台,在距线圈最远 50 cm 处仍能稳定平衡。
Whole-Body UMI:把手持演示迁移成人形机器人的全身操作技能
人形机器人的全身操作技能长期靠遥操作教会,既昂贵又难以规模化。Universal Manipulation Interface(UMI)提供了一种更便宜、可扩展的手持操作录制方式,但末端轨迹本身并不足以确定人形机器人该如何协调整个身体。Whole-Body UMI(WB-UMI)用一个与任务无关、实时、以末端为条件的动作生成器补上这一环:它把全身协调的学习与任务语义的学习彻底解耦,两者之间只共享末端接口。扩散策略吃原生 UMI 演示,WB-UMI 则独立地从重定向动捕中学习,因此采集具体任务时既不需要动捕服,也不需要「图像—全身动作」的配对数据。在 G1 人形机器人上,整套系统在四个任务上实现了实时闭环迁移:关抽屉 90%、货架取放 80%,但更动态的抛球(30%)与移动操作(40%)依然困难。
StenoVLA-3D:在消化道狭窄中导航的三维感知 VLA
在狭窄段自主导航的内镜,要求策略模型能从纹理贫乏的单目观测中预测动作、做出安全的控制决策,并在病灶离开视野后仍然保留证据。现有 VLA 主要依赖外观与短期上下文,几何接地与整段级报告能力有限。StenoVLA-3D 把点图经由「几何门控融合」注入 Cosmos-Reason 2 主干,并引入一条时序状态分支来建模通过进度;主干在输出动作的同时给出接地的推理,专用头则负责估计狭窄形状并生成最终的病灶报告。作者同时发布了 EndoCausal——一个带病灶标注、动作与时序接地推理的整段级数据集。在 40 段留出测试 episode 上,语义准确率 95.2%、动作准确率 83.4%;在实体 3 自由度内镜上,食管与结肠体模各 36 次试验的任务成功率为 88.9% 与 77.8%,明显优于所评测的基线。
JAMB:让动作与未来几何一起去噪的双臂操作扩散策略
双臂操作的核心难点在于两条手臂共享同一个三维场景,任一条手臂的动作都会改写另一条手臂所依赖的空间关系。JAMB 的做法是让动作与未来三维点轨迹在同一个 Transformer 里联合去噪:二者在每一轮去噪中相互修正——几何假设约束动作假设,动作假设又反过来塑造几何预测——而不是像此前的预测式扩散策略那样,把未来状态降级为辅助监督或固定条件。作者进一步把所有多模态表示统一到共享的时空坐标系,使跨模态交互在几何上有共同参照。在 RoboTwin 2.0 的十六个双臂任务上平均成功率 83.4%,高出最强基线 23.9 个百分点;在真实机器人上,相较纯动作策略与辅助几何预测方法分别高出 50.0 与 21.2 个百分点,并在杂乱场景与分布外背景下展现出更强的泛化。
从外科医生自己的器械里学手术:不依赖机器人采集示教的活体腹腔镜阑尾切除
微创手术至今仍以手持腹腔镜器械为主,而一台手术结束后,医生操作器械的运动信息几乎全部丢失——能留下来的只有内窥镜视频。本文搭建了一条端到端流水线,在手术室里把这段丢失的运动重新拿回来,并把它变成手术机器人的策略,并在活体动物上完成了验证。感知那一半是一枚卡在标准腹腔镜器械杆上的「器械状态记录器」:它靠惯性传感器、飞行时间传感器和霍尔传感器恢复器械的位姿与钳口开合状态,不需要任何外部相机或追踪器。数据流水线则逐一测量每个传感器通道相对机器人真值的延迟,在配对「观测—动作」之前先把各通道对齐。作者用这些示教训练了一个以微调 DINOv3 为骨干的扩散策略,其设计是通过一个由离体兔阑尾深度图重建出的物理仿真器做闭环 rollout 来选定的。策略随后用四只活体兔子采集的 849 条活体示教重新训练,并部署到另外四只活体兔子上,且电外科设备处于激活状态;在由外科医生选择手术阶段的前提下,策略在四只动物中的三只完成了阑尾切除。两套示教数据集均已公开。
MachEmbodied-U0:一个既理解任务、又能预测场景如何变化的具身基础模型
通用机器人控制要求一个模型同时做四件事:理解用户想做什么、判断该在哪里交互、预见场景将如何演变、并给出精确动作。视觉-语言-动作模型提供了很强的语义先验,却不显式建模场景动态;世界-动作模型把视觉预测与控制耦合起来,却未必暴露出细粒度操作所需的语义与空间结构。MachEmbodied-U0(ME-U0)是一个统一的具身基础模型,通过 Mixture-of-Transformers 架构把「理解专家」和「生成专家」连接起来。子任务预测与可供性定位共同引导「视觉动态 + 动作」的联合生成,生成过程采用流匹配;其中视觉动态涵盖未来的 RGB、深度、表面法线与光流,从而在表观、几何与运动三个维度上提供互补监督。作者进一步提出多速率旋转位置编码(MRPE),把视觉动态与细粒度控制对齐。ME-U0 在约 4200 小时的机器人数据与第一视角数据上完成预训练,评测时只使用各下游基准自身原生的监督:在 RoboDojo 仿真基准上取得 17.66 的平均分,在 LIBERO 与 LIBERO-Plus 上平均成功率分别为 99.0% 和 82.5%;此外还在真实机器人操作任务上做了验证,并在仿真与真实观测上展示了零样本的子任务预测、可供性定位与视觉动态能力。
PAKT:尊重机器人真实能力的示教式教学
真实世界强化学习在接触密集的工业操作上依然吃力:这一场景同时要求微米级精度、99% 以上的成功率,以及接近人力的节拍时间。离策略方法可以靠示教与人工干预做到这一点,但瓶颈在于缺少一个合适的接口——抛开算法不论,产线上的操作者究竟怎样提供「既直观、又在物理上与机器人及其策略真正可复现的能力一致」的指导?PAKT 是一个用于强化学习的示教式教学框架。它不用遥操作,而是用工业界本来就依赖的「拖动示教」,并修掉了后者最典型的毛病:操作者可以把机器人拖过一些轨迹,但其中的速度、加速度与加加速度,机器人或策略在物理上根本无法复现。在 PAKT 中,操作者通过导纳控制引导机器人,即把人的手施加的力映射为运动;下游的参考轨迹生成器则施加与策略执行时相同的运动学限制,于是采集到的每条轨迹天然落在系统可复现的范围内。支撑这一接口的是一套高性能控制栈:通过参考轨迹生成器与阻抗控制器,把低频的强化学习动作映射为高频力矩指令,其中参考生成器在保持阻抗控制器跟踪性能的同时,改善了接触处理并让策略动作更平滑。在四个插入与工业装配基准(含一个数据中心算力托盘)上,端到端系统相对 HIL-SERL 基线将节拍时间缩短 23% 至 48%,累计干预次数减少 62% 至 86%。
MedVLA:用「函数约束执行」把精密医疗操作成功率从个位数拉到 95%
精密医疗机器人需要在严格的安全、可解释性与执行约束下做自适应决策。视觉语言动作(VLA)模型虽然展现出很强的多模态推理能力,但其「连续动作生成」的范式并不适合精密医疗任务——在这种场景里,可靠的闭环运行往往还依赖非动作类的系统功能调用。MedVLA 因此提出一个层级式框架,把高层多模态推理与底层「函数约束执行」耦合起来,并配套一条可扩展的多智能体流水线,用于生成面向技能的思维链(CoT)数据以支撑结构化训练。该框架构建在多种多模态大模型主干之上,微调后均取得一致提升。在相同初始条件下进行的 100 次柔性电极植入闭环试验中,MedVLA 达到 95.0% 的任务成功率,在准确性、稳定性与安全性上显著优于代表性 VLA 基线(OpenVLA 8%、pi0 15%)。
Silent Sabotage:用机器人自己的动作历史触发的隐蔽后门
把大语言模型接进机器人控制系统,带来了复杂推理与规划能力,也带来了尚未被充分研究的安全风险。现有的 LLM 后门研究几乎都围绕「外部刺激」展开——特定的词、特定的视觉目标、特定的环境状态。这类攻击虽然有效,却忽略了一类更隐蔽的漏洞:触发条件位于智能体自身的运行逻辑内部。本文首次系统研究了对 LLM 驱动机器人的「基于历史」的后门攻击:攻击者只需篡改指令,就能在机器人控制器中植入后门,而其触发器不是任何外部线索,而是机器人自己过去动作的一段罕见序列。后门在正常运行中保持休眠、不损害机器人的实用性,却能被激活以诱发恶意行为,例如完全停住或制造碰撞。在覆盖多种机器人与多种 LLM 的仿真实验中,这种基于历史的攻击达到了近乎完美的攻击成功率,同时极难被检测。
StrataVLA:把 3D 几何分层注入 VLA 主干,几何调用量最多省下 88%
视觉语言动作(VLA)模型从大规模视觉语言预训练中继承了对语义的先验理解,却在机器人操作上持续受制于 3D 空间感知不足。现有做法要么要求显式的深度图或点云输入,要么把几何压缩成训练期监督,要么只在模型输入或动作专家处注入几何——共同结果是视觉语言主干在推理时始终拿不到任务相关的空间信息。StrataVLA 提出一种即插即用的分层几何接地框架:用一个冻结的几何基础模型从 RGB 观测中提取共享几何特征,再通过稀疏、按主干深度分层的 Geometry Adapter,让选定层的视觉表征以交叉注意力按需取用几何证据;同时用任务感知路由与 LRU 特征缓存利用操作过程中的时间冗余。在 LIBERO、SimplerEnv 与真实机器人操作上,该方法在保持对强 VLA 基线一致领先的同时,把几何模型的调用次数最多降低 88%,其中 LIBERO 系列平均成功率达到 98.53%。
RACaP:把代码生成挪到进化阶段,运行时只调用冻结的策略 API
「代码即策略」(Code as Policies)让模型在运行时现写现修程序,代价是推理延迟,并把可复用的物理机制与具体任务决策纠缠在一起。RACaP 把代码生成从部署时挪到进化时:部署阶段由一个 ReAct 循环调用一组冻结的、带类型的策略 API,任务适配通过参数完成,而不是重新改代码。两阶段策略用能力课程学习加自主自进化,同时改进 API、ReAct harness 与经验记忆。在 LIBERO-90 上达到 54.4% 成功率,零样本 LIBERO-PRO 45.0%,LIBERO-Long 46.0%,而 Code as Policies 基线在长时序任务上最高只有 4.0%;在 LIBERO-PRO 上成功率是基线的 2.5 倍,中位策略时间快 1.9 倍。为了上机部署,作者用拒绝采样微调把 GPT-5.6 的 ReAct 决策蒸馏进 Qwen3-VL-8B-Instruct,单次决策推理提速 13.2 倍,重复物理调用从 16 次降到 4 次。
Self-Adaptive VLA:硬件漂移时,让 VLA 策略自己重新校准
视觉语言动作(VLA)模型的操作能力令人印象深刻,但它们天生「无记忆」,因此在测试时环境变化面前很脆弱——尤其是由磨损或标定不准带来的硬件偏移。让这类策略在部署过程中自适应、而无需持续现场重新标定,是能否大规模落地的一个关键瓶颈。Self-Adaptive VLA 是一套后训练方案:它让策略把自己产生的 rollout 当作上下文,迭代地适应部署期的硬件偏移。具体做法是先刻意注入硬件偏移并采集 rollout,再把基座策略原有的训练数据「预补偿」成带偏移条件的专家示范;随后用一个轻量、可插拔的上下文编码器,把偏移环境下的视觉观测、本体感觉与动作压缩成一个隐上下文 token,通过自适应层归一化(AdaLN)调制策略。这些上下文 token 还可以被集成(ensemble),让策略逐步自我纠正、逐帧缓解失败。在四个精度敏感的双臂与灵巧操作任务上,Self-Adaptive VLA 在作动偏置、关节编码器偏移等硬件偏移下恢复了基座策略 80% 以上的性能,并且相比基座策略能更稳健地部署到新的工位。
RAPID:把一次人类演示变成可复用的机器人程序
编码智能体在复杂编程任务上已经证明了自己的能力,RAPID 追问的是:这种能力能不能被引到机器人系统上。给定一段人类视觉演示,RAPID 通过一个迭代式的智能体循环自动生成、验证并精化机器人程序。为了让这件事成立,它只从演示中推断出三样东西:可测试的任务规格、用于执行的动作原语、以及可供程序运行与验证的交互式环境。为了让产出的程序不止于演示场景本身,RAPID 采用了一种「以物体为中心的关系式程序表示」——它关注被演示策略的底层结构,而不是具体的那段运动:把动作原语写成实现「物体级运动效果」的轨迹优化程序,再用关系式约束把它们组合起来,而这些约束在运行时才结合具体场景几何求解。论文在仿真中评测了八个高难度、接触丰富的非抓取操作任务以及 LIBERO-Pro 的通用抓取操作,并在真实 Franka 机械臂上完成了全部八个非抓取任务,表现出对物体位姿、形状、材质与环境的泛化能力。
AD-WM:让世界模型学会分辨动作,而不只是预测下一帧
隐空间世界模型通常被训练去预测「真实发生」的下一个状态,但模型预测控制真正要做的是比较那些「没有发生」的动作。于是会出现一种尴尬情况:模型的事实预测误差很低,却几乎分辨不出不同候选动作。AD-WM 针对这一点,在残差隐空间动力学之上加入「预测器层面的动作可恢复」正则化——一个逆动力学头,加上一个有条件互信息动机的归一化恢复目标——迫使想象出来的状态转移仍保留足以反推出动作的信息。这些辅助头在测试阶段被丢弃,MPC 主循环完全不变。在 OGBench-Cube 上,其 hard-start 成功率从基线的 3.7% 提升到 52.0%;在五个仿真环境中有四个的平均成功率优于复现基线;在使用冻结的 V-JEPA 2 编码器并做匹配的 DROID 后训练后,零样本迁移到作者自建的 Franka 平台,基础抓放成功率从 42.2% 提升到 71.1%,且没有任何针对实验室的适配。
Skytopia: Monocular Drone Navigation with Action-Conditioned Latent World Models
World models for robot navigation are normally built to be executed: a prediction is produced at deployment and fed back into action generation at every control step. Skytopia argues that this is the wrong contract — what a policy needs from a world model is the representation required to produce a prediction, not the prediction itself, because in flight the executed action already explains most of the change between observations and prediction reduces to reprojecting a static scene under a known displacement. The policy is therefore built on an action-conditioned latent world model, trained on a purpose-built 3D Gaussian Splatting platform, with a forward objective predicting the next observation's representation from intended motion and an inverse objective recovering that motion from the predicted transition. Because the prediction never reaches action generation, the predictor is discarded at deployment, and one policy serves point-goal, image-goal, and goal-free navigation at 57.8, 66.0, and 49.0 percent success while cutting inference cost by 59.4 percent. The same policy flies on a physical drone without fine-tuning in indoor, open outdoor, and woodland settings.
MATE: Multi-Agent Virtual Teleoperation Platform for Humanoid Collaboration Data Collection
Humanoid robots need embodied experience of collaborative work, but physical multi-robot data collection does not scale: it demands several robots, a dedicated space, and repeated resets. MATE replaces the hardware with a shared physics simulation in which multiple geographically distributed operators simultaneously teleoperate whole-body humanoids, preserving the physically coupled interactions between humanoids, objects, and environments that make collaboration data valuable in the first place. Using MATE the authors build a multi-humanoid collaboration dataset of 24.1 hours across 2,500 joint episodes and five long-horizon tasks, including object handover, relay delivery, environment interaction, and cooperative transport, and introduce EAIS, an execution-aligned interaction sampling strategy that prioritizes task-progressing and interaction-critical behavior. Imitation learning and vision-language-action policies trained on this data learn effectively and transfer zero-shot to a physical humanoid without real-world fine-tuning.
ε4P: Upcycling Imperfect Data for High-Precision Robotic Manipulation
High-precision manipulation is where vision-language-action policies break down, and the usual remedy is task-specific teleoperation data, which is slow and expensive to collect. ε4P attacks the data problem instead of the model: it upcycles two sources that are normally thrown away, namely low-precision data from the target task and high-precision data from mismatched tasks, and controls where each source contributes along the flow-matching trajectory rather than mixing them uniformly. Low-precision target-task data is applied at high noise to preserve high-level task context, while high-precision mismatched-task data is applied at low noise to transfer low-level action precision. On real-robot sub-millimeter precision tasks and coarse-grained tasks, the method improves policy performance by up to 31.7 percentage points and can replace an equal amount of task-specific high-quality data with an average drop of only 4.2 points.
Skytopia:用动作条件隐世界模型实现单目无人机导航
机器人导航中的世界模型通常被设计成「要被执行」的:部署时先产生预测,再在每一个控制步把预测反馈给动作生成。Skytopia 认为这个契约本身是错的——策略真正需要的不是预测结果,而是产生该预测所必需的表示;因为在飞行中,已执行的动作几乎解释了相邻观测之间的全部变化,预测退化为「在已知位移下重投影一个静态场景」。于是策略建立在一个动作条件的隐世界模型之上,并用专门搭建的三维高斯泼溅平台训练:前向目标从意图运动预测下一观测的表示,逆向目标则从预测出的状态转移中恢复该运动。由于预测从不进入动作生成,预测器在部署时被直接丢弃,一个策略同时覆盖点目标、图像目标与无目标导航,成功率分别为 57.8%、66.0% 与 49.0%,同时把推理开销降低 59.4%。同一策略无需微调即可在实体无人机上飞行,覆盖室内、开阔室外与林地环境。
MATE:面向人形机器人协作数据采集的多操作者虚拟遥操作平台
人形机器人需要关于协作的具身经验,但真机多机器人数据采集无法规模化:它需要多台机器人、专用场地,还要反复重置。MATE 用共享物理仿真替代硬件,让多位地理位置分散的操作者同时遥操作全身人形机器人,同时保留了人形机器人、物体与环境之间真实物理耦合的交互——而这恰恰是协作数据价值的来源。借助 MATE,作者构建了包含 24.1 小时、2500 个联合回合、五类长时序任务的多机协作数据集,覆盖物体交接、接力配送、环境交互与协同搬运;并提出 EAIS(执行对齐的交互采样)策略,优先采样推进任务进度与交互关键的行为。基于该数据训练的模仿学习与视觉语言动作策略学习效果良好,并能零样本迁移到实体人形机器人,无需真机微调。
ε4P:把不完美数据「升级再造」为高精度操作能力
高精度操作是视觉语言动作(VLA)策略最容易失手的地方,而它依赖的任务专用高质量数据通常来自遥操作,采集既慢又贵。ε4P 选择从数据侧而非模型侧解题:它把两类通常被直接丢弃的数据「升级再造」——目标任务上的低精度数据,以及任务不匹配的高精度数据——并且不是把它们均匀混进联合训练,而是控制每种数据在流匹配轨迹上的作用区间。低精度的目标任务数据用在高噪声段,保留任务层面的上下文;高精度但任务不匹配的数据用在低噪声段,迁移底层的动作精度。在真实机器人的亚毫米级高精度任务与粗粒度任务上,该方法把策略表现最多提升 31.7 个百分点,并且可以用不完美数据替换等量的任务专用高质量数据,平均仅损失 4.2 个百分点。
Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policies
Vision-Language-Action models map observations to actions with no objective that accounts for how the world actually responds, so their robustness is bounded by data coverage; world models carry exactly that missing objective but cost seconds per decision and cannot sit in a control loop. This paper argues the two can be separated: physical grounding lives in a world model's internal features, while future generation is merely the objective that produced those features, so the grounding can be inherited and the generative machinery discarded. One feature-alignment term is added to ordinary VLA training, with a frozen world model run once over the training frames and cached; no teacher is loaded during training, the projector is dropped afterwards, and the deployed policy is identical to the undistilled baseline at 32 ms and 1.86 GB on a consumer RTX 5090. A 0.8B student reaches 97.9% on LIBERO, lifts RoboCasa-GR1 humanoid manipulation from 48.2% to 50.5%, and transfers to real single-arm and bimanual hardware.
PredActor: Predictive Action Diffusion for Steerable Onboard Humanoid Control
Diffusion models generate flexible motion, but turning that flexibility into feedback-responsive humanoid control is awkward: hierarchical systems steer through motion references that a separate tracker may not be able to execute, action-only diffusion has no explicit future-state trajectory to steer, and joint state-action diffusion controllers typically depend on privileged full-body state estimates while leaving behavior selection and test-time steering fragmented. PredActor consolidates these capabilities into one directly executed policy driven by proprioceptive observations: conditioned on proprioceptive history and optional task context, it jointly generates executable actions and an internal future-state trajectory, with classifier-free guidance strengthening text-conditioned behavior and classifier guidance steering predicted states toward test-time objectives. Only the actions are executed, with no separate motion-reference tracker and no externally estimated full-body state as input. In simulation it reaches all 15 destination targets with a text retrieval score of 0.580 against 0.373 for conditional action diffusion; rolling denoising and computation-preserving runtime optimizations bring the complete callback to 16.790 ms median and 19.383 ms p95 on a Jetson Orin NX, both inside the 20 ms control period. It is deployed on a Unitree G1.
DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation
Dexterous manipulation rests on contact dynamics that cameras only partially reveal, yet the current generation of World-Action Models couples predictive video with action generation while remaining vision-centric, so the signal that decides success in contact-rich tasks never enters the predicted world state. DexTacWAM encodes each fingertip independently, aggregates the features through a finger- and pose-aware tactile compressor, and injects the resulting tactile latent into a video diffusion world model so contact evolution becomes part of the imagined future. Across six contact-rich tasks on a 22-DoF bimanual platform it scores highest on every task, averaging 70.6 against 38.0 for the strongest baseline; an ablation that removes tactile world modeling while keeping the same tactile features and action expert collapses the four-task mean from 74.7 to 26.6, locating the gain in the world state rather than in tactile conditioning. Adaptation to touch takes roughly four hours and around 100 demonstrations per task with a frozen pretrained vision VAE.
像世界模型一样思考,像 VLA 一样行动:把世界模型表征蒸馏进轻量机器人策略
视觉-语言-动作(VLA)模型把观测映射成动作,训练目标里没有任何一项去刻画「世界会如何回应」,于是它的鲁棒性本质上被数据覆盖范围所限定:分布内可以很好,稍出分布就完全没有对「后果」的概念。世界模型恰好带着这个缺失的目标——它是被训练来预测下一刻会发生什么的,因此也更有物理根据——但把未来推进一步要花上秒级的计算,根本进不了控制回路。这篇文章主张二者本可分离:世界模型对物理场景的认知存放在它的内部特征里,而「生成未来」只是产生这些特征时所用的训练目标。于是接地能力可以被继承,生成机器则可以丢开。做法是在普通 VLA 训练中增加一项特征对齐损失:冻结的世界模型先在训练帧上跑一遍并缓存,学生只需去对齐这份缓存。训练时不加载教师,投影器用完即弃,部署策略与未蒸馏的基线完全相同,在消费级 RTX 5090 上仅 32 ms、1.86 GB。0.8B 的学生模型在 LIBERO 上达到 97.9%,把 RoboCasa-GR1 人形操作从 48.2% 提到 50.5%,并迁移到真实的单臂与双臂平台。
PredActor:可导向的机载人形机器人预测式动作扩散策略
扩散模型能生成灵活多样的动作,但要把这种灵活性变成能响应反馈的人形机器人控制器,中间隔着不少麻烦:分层方案通过动作参考来引导,而参考动作可能超出单独跟踪器的能力,于是恢复与物理执行都落到了跟踪器身上;只生成动作的扩散没有显式的未来状态轨迹可供测试期导向;联合状态-动作扩散虽然提供了这条轨迹,但代表性的控制器依赖特权式的全身状态估计,而行为选择与测试期导向又分散在不同环节里。PredActor 把这些能力收进一个直接用本体感知观测驱动的策略:以本体感知历史与可选任务上下文为条件,联合生成可执行动作和一条内部未来状态轨迹,用无分类器引导强化文本条件下的行为选择,用分类器引导把预测状态导向测试期目标;执行时只执行动作,既不需要单独的动作参考跟踪器,也不把外部估计的全身状态作为策略输入。仿真中它到达全部 15 个目标点,文本检索得分 0.580,而条件式动作扩散为 0.373;配合滚动去噪与保算力的运行时优化,完整回调在 Jetson Orin NX 上达到中位 16.790 ms、p95 19.383 ms,都落在 20 ms 控制周期之内。策略已部署在宇树 G1 上。
DexTacWAM:把接触动力学写进世界模型的视触觉灵巧操作框架
灵巧操作的成败由接触决定,而接触恰恰是相机看不全的部分:指尖压力、起滑瞬间、手指间的载荷转移,视觉几乎无法直接观测。当前的世界动作模型(World-Action Model)把预测式视频生成与动作生成耦合在一起,思路本来对路,但整体仍是「视觉中心」的,于是「预测出来的未来」里根本没有决定任务成败的接触动力学。DexTacWAM 让每个指尖独立编码,再用一个感知手指身份与手部姿态的触觉压缩器做聚合,最后把触觉 latent 注入视频扩散世界模型,使接触演化本身成为被预测的世界状态。在一个 22 自由度的双臂平台上,它在六个强接触任务上每一项都取得最高分,平均 70.6,对最强基线为 38.0;一个消融实验把「触觉世界建模」去掉而保留同样的触觉特征与动作专家,四项任务均值从 74.7 掉到 26.6,说明收益来自「预测接触」而非「用触觉做条件」。适配触觉只需约四小时、每任务约 100 条演示。
LunaDrive: Pushing a Flat BLDC Motor to a 96 V Bus with GaN FETs
How fast a dynamic robot moves is ultimately capped by the rotational speed limit of its actuators, and most commercially available servo drives assume a bus voltage of 48 V or lower, which puts a hard ceiling on instantaneous maximum speed. The obstacle is a property of the power devices themselves: voltage rating and current capacity trade off against each other in semiconductors, so conventional Si MOSFET drivers struggle to deliver both high voltage and high current. GaN FETs are promising, yet compact drivers that can be mounted on the rear side of a flat BLDC motor remain scarce. This work develops a GaN FET driver for high-power flat BLDC motors and studies the effect of delay compensation in the high-speed region beyond the rated operating range. Under 96 V operation the driver achieves 30 A continuous current with a heat sink, a peak current of 80 A, and a maximum electrical frequency of 3110 Hz, and a high-speed load-lifting experiment powered by a 24S (100 V) LiPo battery demonstrates applicability to dynamic robot operation.
Packing the Gripper Into the Plan: Closed-Loop Dense Packing of Irregular Objects
Automatic dense packing is what warehouse operations actually want, and it has stayed largely in simulation: prior work treats the object as an isolated rigid body with idealized contact, inserts the gripper as a discrete post-hoc feasibility check if at all, and leaves perception and contact drift during execution unaddressed. This paper delivers a closed-loop pipeline that runs on real hardware, integrating perception, gripper-aware placement optimization, and force-guided execution. The optimizer represents the object together with the gripper as one composite body of hierarchical sphere trees and searches five degrees of freedom on a GPU inside a CMA-ES framework, with the vertical coordinate grounded analytically against the current heightmap. During execution a force-monitored vertical descent stops on first contact, and a post-release consolidation push closes the residual lateral clearance that gripper-aware planning leaves behind, while the container is re-perceived between placements so drift does not accumulate. The system is validated on a Franka Emika Panda packing 3D-printed flat, curved, and concave objects plus a YCB subset, with an ablation isolating gripper-aware optimization, the consolidation push, and mesh-derived geometry against end-to-end success, achieved density, and computational cost, and a benchmark against a heightmap-minimization baseline representative of prior work.
LIMBO: Learning Safety Into the Policy Instead of Filtering It at Runtime
Safe whole-body control has to hold collision avoidance and balance together under high-dimensional nonlinear dynamics, and safety certificates are typically hand-designed per behavior, then re-derived and re-argued whenever the task changes. LIMBO moves certificate synthesis from design to learning. It learns a state-action control barrier function around a frozen base controller, over residual actions, with a state-based failure specification, which makes Q-CBF synthesis tractable in the full control dimension while placing the certificate in the control space of the task policy. The learned safety value then drives risk-guided sampling near the estimated boundary of recoverability, and later serves as a teacher supplying action-level safety feedback during task learning, so the safety structure is internalized and no online safety filter is needed at deployment. The authors demonstrate the framework on a 29-degree-of-freedom humanoid performing dodgeball avoidance and locomotion beneath low obstacles, with both policies transferring to hardware without online filtering; notably, under a fixed safety specification, varying only the sampling concentration yields strategies ranging from crouching to a novel backward-leaning limbo maneuver.
把抓取器算进摆放里:不规则物体的密集码放闭环系统
仓储场景真正想要的密集码放,一直停留在仿真里:既有工作大多把物体当成孤立的刚体,接触被理想化,抓取器往往只是一个事后的可行性检查,执行过程中累积的感知误差与接触漂移也没人管。这篇工作给出了一条在真机上跑通的闭环流水线,把感知、抓取器感知的摆放优化与力引导执行串在一起。优化器把物体和抓取器合成一个由层级球体树表示的复合刚体,在 GPU 上以 CMA-ES 搜索五个自由度,竖直方向则依据当前高度图解析求解;执行阶段监控力的垂直下压会在首次接触时停止,随后一次释放后的压实推压(consolidation push)用来消除抓取器感知规划留下的侧向间隙,而每次摆放之间容器都会被重新感知,使漂移不会累积。作者在 Franka Emika Panda 上码放 3D 打印的平面、曲面与凹面物体以及一部分 YCB 物体,并通过消融实验分别量化了抓取器感知优化、压实推压与网格几何对成功率、实际密度与计算开销的贡献,同时与高度图最小化方法做了对照。
LIMBO:把安全证书学进策略本身,让 29 自由度人形机器人贴着可恢复边界运动
安全全身控制要在高维非线性动力学下同时管住碰撞与平衡,而安全证书(控制屏障函数)通常是为某一类行为手工设计的,换任务就得重新设计、重新验证。LIMBO 把证书的合成从「设计」改成「学习」:在一个冻结的底层控制器周围学习状态-动作控制屏障函数,把安全判据写成基于状态的失败规范,从而在完整的残余动作维度上完成合成;合成阶段学到的安全价值函数还会引导风险采样,主动把样本撒向估计的可恢复边界附近。随后它充当教师,在任务学习阶段给出动作级的安全反馈,安全结构因此被内化进任务策略,部署时不再需要在线安全滤波器。作者在 29 自由度人形机器人上演示了躲闪球与低障碍下穿行两个任务,两者都迁移到真机且不带在线滤波;一个额外的发现是,在同一个安全规范下仅改变采样集中度,就能得到从下蹲到一种新的后仰钻越动作的策略谱。
LunaDrive:把扁平无刷电机的母线电压推到 96V 的 GaN 驱动方案
动态机器人的动作速度最终受限于执行器的转速上限,而市面上多数伺服驱动器假定母线电压不超过 48V,这一约束直接压住了瞬时最高转速。问题在于功率器件本身的取舍:耐压与通流能力在半导体上是相互挤压的指标,传统 Si MOSFET 驱动器很难同时做到高压与大电流;GaN FET 虽被看好,但能贴在扁平无刷电机背面这种紧凑位置的驱动器仍然很少。这篇工作开发了一款面向高功率扁平无刷电机的 GaN FET 驱动器,并研究了超出额定工作范围的高速区间里延迟补偿的效果。实验在 96V 母线电压下实现 30A 连续电流(加装散热片)、80A 峰值电流与 3110 Hz 的最高电频率,并用 24S(100V)锂聚合物电池完成了一次高速举升负载实验,验证其在动态机器人场景下的可用性。
Agile-WAM: A Lightweight Tactile World Action Model for Contact-Rich Control
World Action Models (WAMs) go beyond conventional visuomotor policies by jointly predicting future world states and robot actions, letting the policy learn the physical dynamics that support control. Recent tactile WAMs, however, tend to lean on large-scale pretrained generative backbones to capture contact-rich dynamics, which limits inference efficiency and flexible deployment. Agile-WAM addresses this by encoding visual and tactile observations into a shared latent that serves as the source of a direct vision-tactile-to-action flow-matching process, jointly generating latent representations of action chunks and of future visual and tactile latents. The key observation is that vision and touch evolve at inherently different timescales: adjacent visual frames are highly similar, while tactile signals can change abruptly upon contact. Agile-WAM therefore uses multi-horizon multimodal prediction, supervising the visual latent at a larger temporal offset while predicting the tactile latent for the next frame. Across nine simulated and five real-world contact-rich manipulation tasks it outperforms the strongest baseline in success rate while maintaining low inference latency; in five real-world experiments it delivers a 29.4% relative gain in overall success rate at 11.9 ms inference latency.
GeoAAC: Geometry-Based Adaptive Action Chunking for Flow-Matching VLA Policies
Action chunking is standard practice for generating and executing actions in vision-language-action (VLA) policies, yet existing methods use a fixed action horizon. Within a single rollout, different task stages call for different levels of action continuity, control precision and closed-loop feedback, so one fixed horizon cannot accommodate all of them. The authors propose GeoAAC, a geometry-based adaptive action chunking method for flow-based VLA policies that sets the action horizon from the reliability of the current prediction. The observation behind it is that the geometry of flow-matching denoising trajectories carries process-level information about prediction reliability, with geometric variation across action prefixes positively correlated with predictive uncertainty. GeoAAC turns prefix-wise geometry into a horizon-wise geometric profile and determines the action horizon adaptively within a single generation, with no additional training. Experiments with GR00T N1.5 and pi0.5 on LIBERO, LIBERO-Pro, RoboCasa365 and real-world manipulation show consistent gains over fixed-horizon baselines and existing adaptive methods: up to 8.7 percentage points in simulation and real-world average success rising from 53.3% to 74.4%.
SafeHarness: Making Safety a Planning Priority for Coding Agents on Robots
Coding agents are emerging as a route to robot manipulation: a language model writes the controller as a program, and agents built this way operate robots with no robot-specific training. Whether that route is safe had not been asked. The authors pair each manipulation goal with an obstacle the robot must not touch, and find the agent collides with it in most cases. The agent discusses the obstacle in its traces and the prompt already forbids touching it, so the fault lies neither in perception nor in instruction but in planning, where the constraint never becomes a priority. Decomposing manipulation into a route phase and a contact-rich moment localizes the failure: along the route the model has no notion of a clearing route and no replanning once a route becomes infeasible, and at contact it is unaware that execution is bounded by the same constraint. SafeHarness supplies two obstacle-aware harnesses: route planning grounds objects as bounding boxes and draws candidate waypoint routes over them so the agent plans, verifies and replans before executing, while contact execution selects the contact position so the contact itself avoids the obstacle. The result is 71.9% task success and 87.5% collision avoidance, beating the previous state of the art by 6.5 and 27.0 points and reaching 2.3x and 1.5x the same agent without harnesses.
Agile-WAM:把触觉世界模型做「轻」——不必生成未来视频,也能预判接触
世界动作模型(World Action Model, WAM)通过同时预测未来的世界状态与机器人动作,让策略学到支撑控制所需的物理动力学,因而超越了传统的视觉运动策略。但近期的触觉 WAM 往往依赖大规模预训练的生成式主干来刻画接触丰富的物理动力学,代价是推理效率低、部署不灵活。本文提出 Agile-WAM,一个用于接触丰富操作的高效触觉世界动作模型:它把视觉与触觉观测编码进一个共享隐空间,并以该隐空间为起点做直接的「视觉-触觉到动作」流匹配,从而联合生成动作块的隐表示与未来的视觉、触觉隐表示。其关键观察是视觉与触觉信号的时间尺度本就不同——相邻视觉帧高度相似,而触觉信号在接触发生时会突变——因此提出多时域多模态预测:以更大的时间偏移监督视觉隐表示,同时只预测下一帧的触觉隐表示以捕捉精细接触动力学。在九个仿真与五个真实世界接触丰富操作任务上,Agile-WAM 在成功率上优于最强基线,同时保持低推理延迟;在五个真实实验中整体成功率相对提升 29.4%,推理延迟为 11.9 毫秒。
GeoAAC:用去噪轨迹的几何形状,给 VLA 策略自适应决定动作块长度
动作分块(action chunking)已成为视觉-语言-动作(VLA)策略生成与执行动作的常规做法,但现有方法普遍使用固定的动作时域。同一次执行中,不同任务阶段对动作连续性、控制精度与闭环反馈的需求并不相同,固定时域无法适应这种变化。作者提出 GeoAAC:一种面向流匹配(Flow Matching)式 VLA 策略、基于几何的自适应动作分块方法,按当前动作预测的可靠程度调整动作时域。其核心观察是,流匹配去噪轨迹的几何形状提供了刻画预测可靠性的过程级信息——动作前缀之间的几何变化量与预测不确定性正相关。GeoAAC 用这种前缀级几何构造「时域级几何剖面」,在单次生成内自适应确定动作时域,且不需额外训练。在 GR00T N1.5 与 π0.5 上、于 LIBERO、LIBERO-Pro、RoboCasa365 及真实机器人操作任务上的实验显示,方法一致优于固定时域基线与已有自适应方法:仿真中最多提升 8.7 个百分点,真实世界平均成功率从 53.3% 提升到 74.4%。
SafeHarness:让「不要碰到障碍」真正进入代码智能体的规划与接触
代码智能体正在成为机器人操作的一条新路径:语言模型把控制器写成程序,由此构建的智能体无需机器人专门训练即可驱动机器人。但这条路径是否安全,此前无人追问。作者为每个操作任务配一个「绝不能触碰」的障碍物,发现智能体在大多数情况下仍然撞上障碍;它在推理轨迹中明确讨论了障碍,提示词也早已禁止触碰,因此问题既不在感知也不在指令,而在规划——这条约束从未成为优先事项。作者把操作拆成「路径阶段」与「接触时刻」两段来定位失效来源,并据此提出 SafeHarness:以障碍感知的路径规划(把物体落成包围盒,在其上把候选路径画成路点序列,先规划、再校验、必要时重规划、最后执行)与障碍感知的接触执行(选择接触位置,使接触本身避开障碍)两副外挂,让模型把安全约束排到前面。结果是 71.9% 的任务成功率与 87.5% 的避障率,分别比此前 SOTA 高 6.5 与 27.0 个百分点,也是同一智能体不加外挂时的 2.3 倍与 1.5 倍。
VLA-ULAP: Interleaving Cloud VLA Calls with Ultra-Lightweight Local Action Prediction at the Edge
Billion-parameter vision-language-action (VLA) policies demand substantial onboard power, while putting inference in the cloud runs into communication delay. VLA-ULAP interleaves remote VLA calls with an Ultra-Lightweight Local Action Predictor (ULAP). At roughly 7.4M parameters including a frozen vision encoder, ULAP fuses the current view, proprioception and executed action history in a single pass to predict action chunks, and trains independently: it needs no VLA hidden states, no online verification and no server round trips. On a Jetson Orin Nano, ULAP takes 19.9 ms and 0.183 J per inference, against 284.3 ms and 50.55 J for GR00T on an RTX A6000. Across three simulated base-policy/benchmark pairs, selected operating points remove 48.8-76.7% of VLA calls while retaining 95.0-97.5% of baseline success. On physical SO-101 experiments the system retains 95.2-100% of baseline success across seen and held-out placements, cutting estimated inference time by 47.9-58.0% and inference-device energy by 52.1-62.5%. In latency-aware LIBERO-Safety simulation it also exceeds pi-0.5 by 11.0 and 15.5 percentage points on two dynamic tasks.
PASSAGE: Scaling Scene-Aligned Motion Learning for Perceptive Humanoid Traversal in Cluttered Environments
PASSAGE is a perception-conditioned planner-tracker framework for humanoid traversal of cluttered environments. The authors collect 100 hours of scene-aligned human motion with virtual reality and inertial motion capture across 1,500 cluttered scenes; a conditional flow-matching planner generates short-horizon references from motion history, a local destination and a robot-centric multi-layer elevation map, and a perceptive whole-body tracker executes them at 50 Hz with geometric feedback. Real-time chunking promotes inter-chunk consistency, and planner-side reinforcement learning on top of the frozen tracker further improves closed-loop performance. The system needs no skill annotations and no obstacle-specific policies: one planner-tracker pair learns to select and compose stepping, squeezing and ducking on geometries it has never seen. Scaling captured data from 6 to 100 hours raises mean contact-free success on held-out scenes from 48.1% to 68.9%, and 70.3% with validated scene augmentation. The fully onboard system integrates egocentric 3D LiDAR, online occupancy mapping, 6.25 Hz planning and 50 Hz control on a Jetson AGX Orin, traversing 50 unseen physical layouts without prebuilt maps or offboard computation.
PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics
World models let perceptual systems predict how a scene will evolve under interaction, and they benefit most from large and diverse training volumes. Existing methods typically need robot action labels to learn action-conditioned 3D dynamics, which excludes web video from the training pool. This paper studies 3D point track completion as a pretraining objective that learns transferable 3D dynamics without any robot data: given a single RGB-D observation and sparse partial 3D trajectories, the model must predict the future 3D tracks of all observed points. The authors contribute a 2.9-million-frame synthetic dataset spanning deformable, articulated and rigid objects, and train PointZero, an expressive transformer, on it. Post-trained for action-conditioned 3D dynamics, PointZero beats baselines on the recent PGND benchmark; post-trained to predict robot actions and 3D tracks, it outperforms or matches baselines on 6 of 7 simulated and real-world manipulation tasks. Training from scratch isolates architecture gains from those of the objective and dataset, and the dataset, checkpoints and full training recipe are released.
VLA-ULAP:把云端 VLA 调用与端侧超轻量动作预测交替执行
十亿参数级的视觉—语言—动作(VLA)策略既需要可观的机载功耗,又受制于远程推理的通信延迟。VLA-ULAP 提出把远程 VLA 调用与一个超轻量本地动作预测器(ULAP)交替执行:ULAP 连同冻结的视觉编码器共约 740 万参数,一次性融合当前视图、本体感知与已执行动作历史来预测动作块,可独立训练,不需要 VLA 的隐状态、在线校验或服务端往返。在 Jetson Orin Nano 上,ULAP 单次推理为 19.9 毫秒与 0.183 焦耳,而 GR00T 在 RTX A6000 上是 284.3 毫秒与 50.55 焦耳。在三组仿真「基座策略—基准」组合上,所选工作点可去掉 48.8% 至 76.7% 的 VLA 调用,同时保留基线成功率的 95.0% 至 97.5%。在 SO-101 真机上,系统在已见与留出摆放上保留基线成功率的 95.2% 至 100%,推理时间估算降低 47.9% 至 58.0%,推理设备能耗降低 52.1% 至 62.5%;在感知延迟的 LIBERO-Safety 仿真中,其动态任务成功率还比 π0.5 高出 11.0 与 15.5 个百分点。
PASSAGE:用 100 小时场景对齐人体动作,训练能在杂乱环境中穿行的人形机器人
PASSAGE 是一个感知条件化的规划器—跟踪器框架,用于人形机器人在杂乱场景中的穿越。作者用虚拟现实与惯性动捕采集了 1500 个杂乱场景中 100 小时场景对齐的人体动作;条件流匹配规划器依据运动历史、局部目标点与机器人中心多层高程图生成短时域参考,具备感知能力的全身跟踪器再以 50 Hz 带几何反馈执行。实时分块保证块间一致性,规划器在冻结的跟踪器之上做强化学习后训练以提升闭环性能。整套系统不需要技能标注,也不用为每种障碍单独训练策略:同一对规划器—跟踪器即可在下台阶、侧身挤过、低头钻过之间自主选择与组合。把采集数据从 6 小时扩到 100 小时,held-out 场景的平均无接触成功率由 48.1% 提升到 68.9%,加入经校验的场景增强后达到 70.3%;全机载版本集成第一视角三维激光雷达、在线占据栅格建图、6.25 Hz 规划与 50 Hz 控制,在 Jetson AGX Orin 上于 50 个未见真实布局中完成穿越。
PointZero:把三维点轨迹补全当作预训练目标,学习可迁移的三维动力学
世界模型让感知系统能够预测场景在交互下如何演化,而这类模型最受益于大规模多样化数据。既有方法通常需要机器人动作标注才能学习「以动作为条件」的三维动力学,于是把网络视频排除在训练池之外。本文研究把三维点轨迹补全(3D point track completion)作为预训练目标,从而在没有机器人数据的情况下学习可迁移的三维动力学:给定一次 RGB-D 观测与稀疏的部分三维轨迹,模型需要预测所有已观测点未来的三维轨迹。作者贡献了一个覆盖可变形、铰接与刚性物体的 290 万帧合成数据集,并在其上训练了 Transformer 模型 PointZero。在下游的「以末端执行器位姿为条件的三维动力学预测」与「模仿学习」两项任务上,PointZero 在 PGND 三维动力学基准上优于基线,在 6/7 项仿真与真实机器人操作任务上优于或持平基线;作者还从零训练以分离架构、预训练目标与数据集各自的贡献,并公开数据集、检查点与完整训练配方。
TIO-Former: Ultra-Lightweight 6-Directional ToF-Inertial Odometry for Nano-UAVs
Autonomous nano-UAV navigation demands accurate ego-motion estimation under severe size, weight, power and computing constraints, yet cameras and LiDAR exceed payload limits, optical flow degrades in low-texture scenes, and inertial-only estimation drifts. Multi-zone time-of-flight arrays are a lightweight metric complement, but recovering 6-DoF motion from only 384 ranges per frame must contend with invalid returns, anisotropic observability and computation that scales with flight time. TIO-Former is a camera-free, optical-flow-free and mapless range-inertial odometry framework driven by an IMU and a 15 g payload of six orthogonal 8x8 ToF arrays. Its frontend pairs consecutive range grids with a bilateral gated difference, while IMU-guided cross-attention routes directional features conditioned on platform kinematics; a streaming causal transformer couples an uncompressed local KV cache with compressed Chunk-FIFO memory so inference cost and memory stay bounded regardless of flight duration. In real flights it cuts open-loop position error by 54.4% versus nano-UAV optical flow and by 66.4%-89.1% versus learned inertial baselines, running on an edge RISC-V companion at 10.466 ms P95 latency and 6.324 MiB peak resident memory. Code is released.
XPACE: Joint World and Action Modeling from Heterogeneous Experience
XPACE is a unified embodied world model that serves at once as a world action model, jointly predicting executable robot actions and future video, and as a world simulator that predicts the visual consequences of prescribed actions. A shared video backbone lets action-unlabeled video teach visual dynamics while action-labeled human and robot demonstrations jointly teach video and action prediction. A coarse-to-fine curriculum progressively shifts emphasis toward robot control while retaining human experience, so the policy acquires behaviors absent from robot demonstrations. The model then adapts its simulator to its own generated context, synthesizes deviation-recovery trajectories around expert demonstrations, and fine-tunes the policy on filtered recovery data. On XPENG's IRON humanoid, heterogeneous training improves robustness and transfers human-observed skills, and self-generated recovery data raises real-world task completion.
ModAR: Modality-Autoregressive World-Action Models
World-action models (WAMs) jointly model future observations and actions, and they almost always represent the predicted future as RGB images; yet depth maps, pretrained visual features and point tracks capture geometric, semantic and motion information far more efficiently. How best to combine these modalities inside a WAM has remained open. ModAR is the first WAM to autoregressively denoise multiple future modalities before predicting actions, so every prediction conditions on the modalities generated so far. Trained from scratch, it systematically studies training-data mixtures, which modalities to predict, and WAM formulations. The findings are that predicting point tracks, DINO features and depth maps helps, while additionally predicting future RGB gives no consistent benefit. ModAR's sequential generation attains the highest average success rate at every evaluated data scale, and when a video-model-initialized WAM, Flex-pi, is fine-tuned on the same data, ModAR reaches a slightly higher observed average success rate (75% vs 72%) using roughly 20x fewer training FLOPs and no pretraining. On three real-world bimanual tasks it beats baselines and improves further with human videos.
TIO-Former:为纳米无人机打造的极轻量六向 ToF-惯性里程计
自主纳米无人机需要在严苛的尺寸、重量、功耗与算力约束下完成精确自运动估计,而相机与激光雷达超重、光流在低纹理场景退化、纯惯性解算又会漂移。多区飞行时间(ToF)阵列提供了轻量的度量补充,但仅凭每帧 384 个测距值做 6 自由度估计,要面对无效读数、各向异性的可观测性与随时间增长的计算量。本文提出 TIO-Former:一个无相机、无光流、无地图的测距-惯性里程计框架,仅依赖 IMU 与六个正交 8x8 ToF 阵列组成的 15 克载荷。前端用双边门控差分对齐相邻距离网格,并以 IMU 引导的交叉注意力按平台运动学动态路由方向特征;流式因果 Transformer 把未压缩的本地 KV 缓存与压缩的分块 FIFO 记忆结合,使推理开销与内存占用不随飞行时长增长。真实飞行中,它把开环位置误差相比纳米无人机光流降低 54.4%,相比学习型惯性基线降低 66.4%-89.1%,并在 RISC-V 边缘计算单元上实现 10.466 毫秒 P95 延迟与 6.324 MiB 峰值常驻内存。代码已开源。
XPACE:把世界模型与动作模型合成一个模型,用异构经验训练人形机器人
本文提出 XPACE,一个把世界模型与世界-动作模型合二为一的统一具身世界模型:它既能联合预测可执行的机器人动作与未来视频,也能作为模拟器预测给定动作的视觉后果。作者用无动作标注的视频学习视觉动力学,用带动作标注的人类与机器人演示联合学习视频与动作预测,并采用由粗到细的课程逐步把训练重心转向机器人控制,从而使策略学到机器人演示未覆盖的行为。该模型还用自身模拟器合成偏离—恢复轨迹来补充监督,在 XPENG IRON 人形机器人上提升了鲁棒性、跨任务技能迁移与真实任务完成率。
ModAR:先自回归去噪多种未来模态,再预测动作的世界-动作模型
世界-动作模型(WAM)通常联合建模未来观测与动作,并把未来观测默认为 RGB 图像;但深度图、预训练视觉特征与点轨迹能更高效地承载几何、语义与运动信息,如何在 WAM 中组合这些模态仍是开放问题。本文提出 ModAR,第一个在预测动作之前先自回归地去噪多种未来模态的 WAM,使每一次预测都以上一步生成的模态为条件。作者从零开始训练,系统研究训练数据配比、预测模态与 WAM 形式的影响,发现预测点轨迹、DINO 特征与深度图能带来收益,而额外预测未来 RGB 并没有一致的增益。ModAR 的平均成功率在所有评测数据规模上都高于既有 WAM 形式;在与用视频模型初始化的 Flex-π 同数据微调对比时,它以约二十分之一的训练算力、不做预训练,取得略高的观测平均成功率(75% 对 72%)。在三个真实双臂任务上,ModAR 优于基线并能从人类视频中继续获益。
Pelican-Sim 1.0: A General World Model Simulator for Embodied Intelligence
Pelican-Sim 1.0 is a general world model simulator for embodied intelligence: it predicts future observations from visual context and robot actions to support downstream learning and decision making. Four design choices carry the system — a 28-dimensional action value space covering most mainstream embodiments so that one model stays valid across heterogeneous devices, action-visual injection that bridges actions and pixels with URDF- and camera-rendered action videos, sparse mixture-of-experts layers that absorb the action modality while reducing inter-modality conflict, and a four-step autoregressive simulator produced by causal adaptation and few-step distillation. Trained on roughly one million real-world and simulated trajectories, it improves video quality and action controllability substantially; in four downstream RoboTwin applications, adding 500 generated trajectories per task to 50 demonstrations raises policy success from 70% to 93%.
Size Doesn't Matter: Material-State Reinforcement Learning for Transferable Excavator Soil Manipulation
The paper casts autonomous earthmoving as material-state-conditioned reinforcement learning, training policies inside a GPU-parallel Material Point Method particle simulation so the controller can condition on soil shape and compactness, use multiple contact faces of the bucket, and displace material both into and out of the shovel. Policies are learned in a normalized end-effector space and deployed through a calibrated machine interface, so the same learned weights drive an 11.5 t hydraulic excavator and a 500 g tabletop robot. In the field the system autonomously built a 42 m long, 2.1 m high embankment, completing 201 consecutive policy strokes in 45 minutes with no failures, retries, or operator intervention, matching an expert operator's pace.
Dynin-Robotics: An Omnimodal Unified Diffusion Vision-Language-Action Model
The paper brings visual goal prediction and dynamics prediction into the generation and selection of actions by sharing a single trajectory model between them. It is built on Dynin-Omni, an omnimodal masked-diffusion backbone that represents language, visual observations, goals, and actions as discrete tokens. By varying only the conditioning and target spans, the same model learns action prediction, action-conditioned next-observation prediction, terminal goal-state prediction, and trajectory-to-instruction reconstruction. Those interfaces support test-time scaling through goal prediction, action-candidate evaluation, and joint refinement of action and future-state predictions. The authors continually pretrain on roughly 1.33 million trajectories from 48 Open X-Embodiment datasets and reach a 78.4% average success rate across four manipulation conditions on a Franka Research 3 robot.
Pelican-Sim 1.0:面向具身智能的通用世界模型仿真器
Pelican-Sim 1.0 是一个面向具身智能的通用世界模型仿真器:它根据视觉上下文与机器人动作预测未来观测,以支撑下游的学习与决策。模型包含四项关键设计——覆盖主流本体的 28 维统一动作空间、用 URDF 与相机渲染的动作视频打通动作与像素的动作-视觉注入、用于吸收异构动力学并缓解模态冲突的稀疏混合专家层,以及借助因果适配与少步蒸馏得到的四步自回归仿真器。作者在约一百万条真实与仿真轨迹上训练,并在多个数据集上取得视频质量与动作可控性的显著提升;在 RoboTwin 的四个下游应用中,每任务仅需补充 500 条生成轨迹即可把策略成功率从 70% 提升到 93%。
Size Doesn't Matter:材料状态强化学习让挖掘机学会可迁移的土壤作业
本文把土方作业的自主控制建模为材料状态条件化的强化学习问题,在 GPU 并行的材料点法(MPM)颗粒仿真中训练策略,使其能依据土壤形状与密实度等状态选用铲斗的不同接触面,把土体在铲斗内外双向搬运。策略在归一化末端执行器空间中学习,并通过标定后的机器接口跨机型部署:同一组权重既驱动 11.5 吨液压挖掘机,也驱动 500 克桌面机器人。实机测试中系统自主修筑了一条 42 米长、2.1 米高的土堤,45 分钟内连续完成 201 次策略动作,全程无需人工干预,推进速度与熟练操作员持平。
Dynin-Robotics:把动作、目标与动力学预测统一进一个全模态扩散 VLA
本文把视觉目标预测与动力学预测带进动作的生成与选择环节,方法是通过一个共享的轨迹模型来统一表达它们。模型构建在 Dynin-Omni 全模态掩码扩散骨干之上,将语言、视觉观测、目标与动作都表示为离散 token;只需改变条件区间与目标区间,同一个模型就能同时学会动作预测、动作条件下的下一观测预测、终末目标状态预测,以及轨迹到指令的重建。这些接口支持推理期扩展:通过目标预测、动作候选评估与动作-未来状态的联合细化来提升成功率。作者在约 133 万条、来自 48 个 Open X-Embodiment 数据集的轨迹上继续预训练,并在四个操作条件下于 Franka Research 3 机器人上取得 78.4% 的平均成功率。
ActSafeGuard: Training-Aligned Hard-Constraint Enforcement for Flow-Matching Robot Policies
Vision-language-action models and world-action models generate capable manipulation behavior, but their actions can violate hard physical constraints and therefore be unsafe or infeasible to deploy. Existing safety approaches either optimize statistical safety objectives without deterministic per-step guarantees, or correct unsafe actions only at inference time, creating a mismatch between policy training and execution. ActSafeGuard is a differentiable, training-aligned safeguard layer for flow-matching policies that integrates hard action feasibility into policy learning rather than treating safety as an inference-time add-on. An analytic ray-scaling operator yields boundary-aware gradients that guide the model to learn constrained manifolds. Across multiple standard foundation backbones (π0.5 and Fast-WAM) and a range of tasks, it consistently achieves a 100 percent step safety rate while fully preserving or even boosting task success.
In-Hand Pen Writing for an Anthropomorphic Hand, Learned in Real Time Without Demonstrations
The authors present an embodied control approach that estimates the task Jacobian of the combined hand-and-object system in real time on the physical robot. Using only a laptop CPU, the controller begins in-hand pen writing after roughly 18 seconds of initialization and keeps adapting online, with no analytic hand-object kinematic or contact model, no simulation training, and no precollected demonstrations. The same estimator/controller formulation works on three anthropomorphic hands (one physical, two simulated), demonstrating an embodiment-independent method. On the physical robot it reaches sub-millimeter in-plane precision (mean 0.6 mm across runs) for letters and shapes written both in the air and on paper, which the authors describe as the first demonstration of an anthropomorphic hand writing arbitrary single-stroke trajectories with a grasped pen through purely in-hand motion.
EVPeriscope: Event-Camera Propeller Tracking Gives Air and Ground Robots Mutual Perception
EVPeriscope equips a ground robot with an upward-facing event camera that picks up the high-frequency visual signature of a quadrotor's propellers, enabling detection, localization, and closed-loop control of the aerial vehicle without fiducial markers and without the motion-blur and lighting sensitivity of frame-based cameras. The quadrotor can therefore act as an extended perception system for the ground robot when its own sensors degrade or are occluded, forming a marsupial air-ground team. The authors run field experiments in winds up to 15 mph, in daylight and at night, and demonstrate localization and closed-loop navigation through dense foliage where the ground robot's sensors are blocked; quadrotor control runs at 200 Hz entirely on onboard sensing and computation.
ActSafeGuard:把硬约束「训进」流匹配策略,实现 100% 单步安全
视觉-语言-动作(VLA)与世界-动作模型(WAM)在通用机器人操作上表现出色,但其生成的动作可能违反硬性物理约束,从而在部署中不安全或不可行。现有安全方法要么只优化统计意义上的安全目标、缺乏逐步骤的确定性保证,要么只在推理时纠正不安全动作,造成策略训练与执行之间的错位。ActSafeGuard 是一个面向流匹配(flow matching)策略的可微、且与训练对齐的安全防护层,把硬性动作可行性融进策略学习本身,而非把安全当作推理期的外挂。作者设计的解析射线缩放(ray-scaling)算子使梯度具有边界感知能力,引导模型自然地学到受约束的动作流形。在多个标准基础骨干(π0.5 与 Fast-WAM)与多种任务上的实验显示,ActSafeGuard 持续取得 100% 的单步安全率,同时完整保持甚至提升任务成功率。
拟人机械手实现手内握笔书写:实时雅可比估计,无需仿真训练与示教数据
本文提出一种具身控制方法,在真实机器人上实时估计「手+被握持物体」组合系统的任务雅可比。控制器只用一台笔记本的 CPU,在约 18 秒初始化后即开始手内握笔书写,并在运行中持续在线自适应;整个过程不需要手-物体的解析运动学或接触模型、不需要仿真训练,也不需要预先采集的任务演示。同一套估计器/控制器在三套拟人机械手(一台实物、两套仿真)上均成立,说明该表述与具体本体无关。在实物机器人上,无论在空中还是在纸面书写字母与图形,都能达到亚毫米级的面内精度(多轮运行均值 0.6 mm);作者称这是首次演示拟人机械手仅靠纯手内运动、用握持的笔写出任意单笔画轨迹。
EVPeriscope:地面机器人抬头,用事件相机追踪螺旋桨与无人机互为空地感知延伸
EVPeriscope 让地面机器人用一台朝上的事件相机捕捉四旋翼螺旋桨的高频视觉特征,从而在不需要基准标记、也不受运动模糊与光照变化拖累的前提下完成对无人机的检测、定位与闭环控制。无人机由此可以在自身传感器退化或被遮挡时充当地面机器人的「延伸感知」,形成地空「袋鼠式」协同系统。作者在上至 15 mph 风速的野外条件下、白天与夜间都做了实验,并演示了在地面机器人传感器被茂密植被遮挡时依靠该系统完成定位与闭环导航;对无人机的控制以 200 Hz 运行,完全依赖机载传感与计算。
GTA-2: A Multi-VLM Framework for Synthesizing Robot Manipulation Skills via Grounded Task Axes
GTA-2 is a modular multi-VLM framework that constructs executable, task-bespoke manipulation skills from reusable object-centric task-axis components rather than predicting actions end-to-end or composing fixed primitives. Each skill is represented as semantic subtasks comprising task-relevant keypoints and axes, controller compositions, and scene-dependent parameters. Four specialized VLM agents separately decompose the task, construct an abstract task-axis skill, assign controller parameters, and ground required visual features from RGB-D observations, enabling zero-shot skill generation with no task-specific demonstrations, policy training, or fine-tuning. On 14 real-robot manipulation tasks it achieves 73.9% average zero-shot success, exceeding the strongest baseline by 31.4 percentage points, and targeted human feedback refines an incorrect stage to raise the average to 90.7%.
FolDeX: A Physical-World Benchmark for Long-Horizon Robotic Manipulation of Deformable Objects
FolDeX is a physical-world benchmark built entirely from real-robot data, centered on garment folding, that targets long-horizon deformable-object manipulation where policies must track changing states and execute reliable multi-stage bimanual interactions. Because real-robot data is costly, it studies efficient reuse of heterogeneous physical experience along four axes: human intervention and recovery data collected during deployment; transfer across tasks including garment categories and rigid-to-deformable manipulation; reuse across scenes with changing lighting, background, and layout; and transfer across embodiments. The benchmark provides 2,000+ hours of real-robot data spanning 20+ tasks and 10+ embodiments, plus a fair evaluation platform for externally submitted policies with standardized tasks, held-out physical objects, controlled initializations, and a unified execution protocol.
Learning Terrain-Adaptive Humanoid Locomotion on Granular Terrain
This work builds a physics-grounded granular contact model from three-dimensional resistive force theory (3D RFT) and simulates granular terrain efficiently for reinforcement learning, replacing rigid or heuristic contact solvers with physically accurate intrusion dynamics that capture penetration and tangential drag. A terrain-adaptive controller is trained via teacher-student RL, with a variational autoencoder compressing terrain information into a compact latent for zero-shot terrain identification and adaptation. MPM/NVIDIA Newton studies show generalization to unseen granular terrains and higher success rates than baselines, and hardware experiments on basalt, dry sand, and beach sand deliver what the authors state is the first demonstration of agile humanoid locomotion on real-world granular terrain.
A Traffic Management System for Large and Heterogeneous Vehicles in Narrow Industrial Environments
Coordinating automated guided vehicles (AGVs) in high-density industrial environments is a critical challenge within Logistics 4.0, where traditional negotiation-based priority assignment in traffic management often causes inefficiency. This paper presents an AGV traffic management system based on lifelong multi-agent path finding (L-MAPF) operating on roadmaps generated with non-uniform rational B-spline (NURBS) curves, guaranteeing locally optimal coordination and safe operation of large, heterogeneous AGVs. The framework integrates a modified bounded-horizon conflict-based search (CBS) within a rolling-horizon conflict resolution strategy, using an extended time horizon per agent to resolve conflicts effectively in corridors identified by a topological map. Unlike state-of-the-art AGV fleet traffic management, the solution targets real-world, non-standardized (non-grid-like) industrial settings with narrow bidirectional corridors and high traffic density, where AGVs of various sizes and capabilities operate simultaneously. Key contributions include an anytime conflict resolution strategy with adaptive time-horizon regulation, an execution layer for safe and standard-compliant interaction with real AGVs, and an advanced deadlock detection and resolution mechanism. Experiments in realistic industrial environments show throughput improvements of up to 11% over a conventional rule-based traffic management system, a state-of-the-art industrial method, and a priority-based L-MAPF variant, while maintaining continuous operation.
Frequency-Conditioned Flow Matching for Vision-Language-Action Models
Robot actions are temporally correlated trajectories whose frequency components encode motion at different scales with highly non-uniform energy distributions, yet flow-matching vision-language-action models typically generate actions in temporal coordinates without explicitly modeling or systematically exploiting this frequency heterogeneity. FreqFM raises action frequency from an implicit trajectory property to an explicit conditioning dimension that spans the entire generation pipeline. Concretely, in DCT frequency coordinates it constructs a spectrum-matched source distribution, adaptively balances the objective across frequencies, and constrains per-frequency guidance residuals using the corresponding reference transport scales. FreqFM integrates into existing flow-matching action experts without changing the VLA backbone, improving performance consistently across LIBERO, LIBERO-Plus, and VLA-Arena, including a 9.3-point gain on LIBERO-Plus, and further demonstrating effectiveness on six real-robot tasks.
Show-Harness: Just a VLM Agent Can Play Robots
Show-Harness is an embodied harness that converts the world knowledge of foundation vision-language models into robot control through a compact semantic interface. The VLM reasons over discrete semantic action units, and embodiment-specific interpreters deterministically ground those decisions into local robot actions, keeping the VLM directly responsible for fine-grained physical decisions. The same interface directly unlocks closed-source frontier VLMs for zero-shot robot control and adapts small-scale open-source VLMs for low-cost deployment with only a few GPU-hours of fine-tuning. Its companion GUMI (GUI Manipulation Interface) extends the same semantic action space to GUI-based demonstration collection, so humans and agents can play robots across embodiments without specialized teleoperation hardware. Extensive experiments show that Show-Harness-equipped VLM agents generalize robustly across tasks, embodiments, and environments, outperforming representative agentic and VLA paradigms.
GTA-2:基于具身任务轴的多VLM机器人操作技能合成框架
GTA-2是一个模块化多VLM框架,从可复用的、以物体为中心的任务轴组件出发,构建可执行、为具体任务量身定制的操作技能,而非端到端预测动作或组合固定原语。每项技能被表示为语义子任务,包含与任务相关的关键点与轴、控制器组合以及场景相关参数。四个专门化的VLM智能体分别负责:分解任务、构建抽象任务轴技能、分配控制器参数、并从RGB-D观测中落地所需视觉特征,从而实现零样本技能生成——无需任务专用示教、策略训练或微调。在14项真机操作任务上,平均零样本成功率达73.9%,比最强基线高出31.4个百分点;针对性的阶段级人工反馈可将平均成功率提升至90.7%。
FolDeX:面向可形变物体长时程机器人操作的真实世界基准
FolDeX是一个完全基于真机数据构建的真实世界基准,以衣物折叠为核心任务,面向长时程可形变物体操作——策略须跟踪不断变化的状态并执行可靠的多阶段双臂交互。鉴于真机数据采集成本高昂,该基准围绕四条研究轴考察异构物理经验的高效复用:利用部署中采集的人类干预与恢复数据;跨任务迁移(含衣物类别之间及刚体到可形变物体);跨场景复用(光照、背景与布局变化);以及跨本体迁移。基准提供2000+小时真机数据,覆盖20+任务与10+本体,并给出对外部提交策略的公平评测平台:标准化任务、留出实物、受控初始化与统一执行协议。
面向颗粒地形的自适应人形机器人运动学习
本文基于三维阻力理论(3D RFT)构建了物理上可靠的颗粒接触模型,并高效仿真颗粒地形用于强化学习训练,以能刻画下陷与切向拖曳的物理精确侵入动力学,取代刚体接触模型与启发式颗粒模型。控制器通过教师—学生强化学习训练,并用变分自编码器把地形信息压缩为紧凑隐变量,实现零样本地形识别与自适应。基于MPM与NVIDIA Newton的仿真研究表明方法可泛化到未见颗粒地形、成功率显著高于基线;在玄武岩、干沙与海滩沙等多种真实颗粒地形上的硬件实验,作者称是首次实现真实颗粒地形上的敏捷人形机器人运动。
面向狭窄工业环境的大型异构车辆交通管理系统
高密度工业环境中的自动导引车(AGV)协同是物流 4.0 的核心难题,传统基于协商的优先级分配式交通管理常因协商开销导致效率损失。本文提出一种基于终身多智能体路径规划(L-MAPF)的 AGV 交通管理系统,运行在由非均匀有理 B 样条(NURBS)曲线生成的路网上,保证局部最优协同并确保大型异构 AGV 的安全运行。框架把改进的有界时域冲突搜索(CBS)嵌入滚动时域冲突消解策略:为每个智能体设置更长的时域,以便在拓扑地图标定的走廊中有效消解冲突。与现有 AGV 车队交通管理方法不同,该方案面向真实、非标准化(非网格状)的工业场景——狭窄双向走廊、高交通密度、多种尺寸与能力的 AGV 同时运行。关键贡献包括带自适应时域调节的 anytime 冲突消解、面向真实 AGV 的安全合规执行层,以及死锁检测与消解机制。在真实工业环境中的实验显示吞吐量最高提升 11%,优于常规规则式交通管理与基于优先级的 L-MAPF 变体,同时保持连续运行。
面向视觉-语言-动作模型的频率条件流匹配
机器人动作是时间上相关的轨迹,其频率分量在不同尺度上编码运动,能量分布高度不均;而基于流匹配的视觉-语言-动作(VLA)模型通常在时间坐标中生成动作,并未显式建模或系统利用这种频率异质性。本文提出 FreqFM,一种面向 VLA 的频率条件流匹配框架:它把动作频率从轨迹的隐式属性提升为贯穿整个生成流程的显式条件维度。具体而言,在 DCT 频率坐标下,FreqFM 构造谱匹配的源分布、自适应地在各频率之间平衡训练目标,并用对应的参考传输尺度约束逐频率的引导残差。FreqFM 可无缝接入现有流匹配动作专家而不改动 VLA 主干,在 LIBERO、LIBERO-Plus 与 VLA-Arena 上一致提升,其中 LIBERO-Plus 提升 9.3 个百分点,并在六个真实机器人任务上验证有效。
Show-Harness:一个 VLM Agent 就能「玩」机器人
Show-Harness 提出一种「具身载体」接口,把基础视觉语言模型(VLM)的世界知识转化为机器人控制能力:它向 VLM 暴露一组离散的语义动作单元供其自然推理,再由本体相关的解释器确定性地把这些单元落地为局部机器人动作,使 VLM 直接对细粒度物理决策负责。同一接口既能零样本解锁闭源前沿 VLM 的机器人控制,也能用数 GPU 小时微调小规模开源 VLM 以实现低成本部署;配套的 GUMI(图形界面操作接口)把同一语义动作空间扩展到基于 GUI 的示教采集,让人与智能体无需专用遥操作硬件即可跨本体「玩」机器人。大量实验显示,搭载 Show-Harness 的 VLM 智能体在任务、本体与环境间稳健泛化,优于代表性 agentic 与 VLA 范式。
Ostrich: Taking Large Strides Through Stiff Contact in Differentiable Dynamics
Ostrich is a GPU-accelerated differentiable rigid-body simulator that resolves hard contacts and friction with non-smooth Newton iteration at large timesteps (~0.1 s) and differentiates the converged residual via the implicit function theorem, reusing the forward Schur complement to compute the adjoint at O(1) memory per timestep. On real-robot trajectories over a pallet obstacle, Ostrich holds MuJoCo's sim-to-real accuracy at up to 50x larger timesteps. Its gradients converge from random initializations where MJX descends slowly and Newton Semi-Implicit stalls; a warm iteration runs 211x faster than MJX and 4.7x faster than Semi-Implicit. Ostrich differentiates 8,192 parallel worlds on a single 24 GB GPU, sustaining 29x checkpointed MJX's optimization throughput. It closes with gradient-based trajectory optimization over triangle-mesh terrain across a 10 s horizon.
Proxy Policy Steering
Proxy Policy Steering (PPS) adapts a frozen generalist robot policy to a new task at inference time. Instead of modifying the base policy, PPS trains two lightweight proxy policies — a reference proxy that models the frozen base's behavior on target-task observations and a task proxy capturing how behavior changes under task supervision — and uses their calibrated velocity-space difference to steer the base sampler at every denoising step. Because the base is never modified, its broad capabilities remain intact. On 8 real-world and 4 simulation manipulation tasks, PPS lifts the state-of-the-art pi0.5 base policy by 53% absolute success rate on average, including zero-to-one gains on tasks the base never solved, outperforming LoRA fine-tuning, from-scratch specialists, residual policies, and prior inference-time steering methods.
TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model
TANGO is the first whole-body vision-language navigation framework for humanoid robots in cluttered indoor environments. Unlike 2D path-planning approaches, it treats traversal as a continuous geometry-aware whole-body problem: given a natural-language instruction and egocentric RGB observations, TANGO directly predicts 29-DoF joint-space actions. It is trained entirely in simulation using synthesized collision-free traversal behaviors built from global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. TANGO achieves state-of-the-art vision-language navigation results in simulation, outperforming strong modular baselines in obstacle-negotiation scenes, and deploys zero-shot on a Unitree G1 humanoid in cluttered real-world scenes without any real-world navigation training data.
Ostrich:在可微动力学中以大步长穿越刚硬接触
Ostrich是一个GPU加速的可微刚体仿真器:它以大步长(约0.1秒)用非光滑牛顿迭代求解硬接触与摩擦,并借助隐函数定理对收敛残差求导,复用前向Schur补以每时间步O(1)内存计算伴随梯度。在跨越托盘障碍的真实机器人轨迹上,Ostrich以高达50倍于MuJoCo的时间步长保持与其相当的仿真到现实精度。其梯度能从随机初始化收敛,而MJX下降缓慢、Newton Semi-Implicit直接停滞;一次热启动迭代比MJX快211倍、比Semi-Implicit快4.7倍。Ostrich可在单张24 GB GPU上并行求导8192个世界,达到带检查点MJX优化吞吐的29倍。论文最后展示了在三角网格地形上、10秒时域内的基于梯度的轨迹优化。
代理策略引导:面向冻结通用策略的推理时任务适配
Proxy Policy Steering(PPS,代理策略引导)是一种推理时适配方法,让冻结的通用机器人策略无需修改即可适应新任务。它训练两个轻量代理策略——一个「参考代理」建模冻结基座策略在目标任务观测上的行为,一个「任务代理」刻画任务监督带来的行为变化——用二者经校准的速度空间差值在每个去噪步引导基座采样器。由于基座策略从未被修改,其广泛能力得以完整保留。在8个真实世界与4个仿真操作任务上,PPS将当前最强的pi0.5基座策略的平均绝对成功率提升53%,在基座从未解决的任务上实现从零到一的突破,并全面优于LoRA微调、从零训练的专用策略、残差策略及此前的推理时引导方法。
TANGO:基于全身视觉-语言-动作模型的杂乱环境人形导航
TANGO是首个面向杂乱室内环境下人形机器人的全身视觉-语言导航框架。与二维路径规划不同,它把穿行视为连续的、几何感知的全身问题:给定自然语言指令与第一视角RGB观测,TANGO直接预测29自由度关节空间动作。模型完全在仿真中训练,通过全局路径规划、运动学全身动作生成、障碍感知动作编辑与基于强化学习的跟踪,合成多样的无碰撞穿行行为。TANGO在仿真视觉-语言导航中取得当前最优性能,在需要障碍协商的挑战性场景中优于强模块化基线,并在未使用任何真实导航数据的情况下,零样本部署到Unitree G1人形机器人上,在杂乱真实场景中实现稳健的语言引导穿行。
Collision Snapshot Guided Time-Reversed Safety-Critical Scenario Generation
COSTER is a framework for generating safety-critical traffic scenarios by leveraging learned traffic priors to determine plausible collision times and locations. It constructs a collision snapshot by inserting a new vehicle in contact with the target vehicle at an identified collision state, then uses a conditional variational autoencoder to perform a time-reversed rollout that reconstructs the inserted vehicle's trajectory backward toward earlier timesteps. Experiments show COSTER outperforms existing methods in plausibility, diversity, and data efficiency; agents trained on COSTER-generated scenarios reduce collision rates by 31% on safety-critical scenarios from the Waymo Open Motion Dataset while also improving ego task completion.
碰撞快照引导的时间反演安全关键场景生成
COSTER是一种利用学习到的交通先验来确定合理碰撞时间与位置的安全关键交通场景生成框架。它先在识别出的碰撞状态下插入一辆与目标车辆接触的新车,构造"碰撞快照",再由条件变分自编码器执行时间反演展开,把插入车辆的轨迹向更早时刻逆向重建。实验表明COSTER在合理性、多样性与数据效率上均优于现有方法;在COSTER生成场景上训练的智能体,于Waymo Open Motion Dataset的安全关键场景中碰撞率降低31%,同时提升了主车任务完成度。
GLoRI: Closed-Loop Whole-Body Tracking with Global-Local Reference Interaction for Humanoid Loco-Manipulation
GLoRI is a closed-loop whole-body controller for humanoid loco-manipulation that integrates structured global reference and feedback with local motion guidance. Its GLoRI-Net uses Global-Local Cross Attention (GLCA) to refine local keypoint features with global target and pose-difference features, preserving motion structure while correcting world-frame placement. GLoRI achieves 100% completion and a g-MPJPE of 6.44 cm on held-out HuMoTo motions, and this accuracy remains robust under direct Isaac Gym-to-MuJoCo transfer without fine-tuning. The accuracy and generalization enable autonomous loco-manipulation with a single policy on a real Unitree G1 interacting with diverse unseen objects.
GloVLA: Let Geometry Move and Local VLA Interact for Robust Object-Centric Manipulation in Unstructured Environments
GloVLA is a hybrid framework that separates object-centric manipulation into two complementary regimes: a geometric transport controller moves the end-effector into interaction-centric handoff regions, while local Vision-Language-Action (VLA) policies handle only the short-horizon, contact-rich interaction phases. It is model-agnostic, integrates with different VLA backbones without additional demonstrations, and requires no changes to the action space or success predicate. Under clutter, distractors, illumination changes, visual shifts, and obstruction, full-trajectory GR00T N1.6 execution degrades to 20.9% average success while GloVLA retains 88.5%. On a physical UR10e, overall success improves from 35.6% to 90.0% while mean inference time is more than halved.
GLoRI:全局-局部参考交互的闭环全身跟踪——人形机器人移动操作
GLoRI是一种面向人形机器人移动操作的闭环全身控制器,将结构化全局参考与反馈同局部运动引导相结合。其GLoRI-Net采用全局-局部交叉注意力(GLCA),用全局目标与位姿差特征精化局部关键点特征,在保持动作结构的同时纠正世界坐标系下的空间位置。GLoRI在保留的HuMoTo动作上达到100%完成率、全局MPJPE 6.44厘米,且该精度在Isaac Gym到MuJoCo的直接迁移中无需微调依然稳健。凭借这一精度与泛化能力,GLoRI可在真实Unitree G1上以单一策略自主完成与多种未见物体的移动操作交互。
GloVLA:让几何负责移动、局部VLA负责交互——非结构化环境下的稳健以物为中心操作
GloVLA是一种混合式框架,将以物为中心的操作显式拆分为两个互补阶段:几何传输控制器负责把末端执行器移动到交互为中心的交接区域,局部视觉-语言-动作(VLA)策略只处理短时程、重接触的交互阶段。该方法与模型无关,可无需额外演示地集成不同VLA骨干,且不改变动作空间或成功判据。在杂乱、干扰物、光照变化、视觉偏移与遮挡等挑战条件下,全轨迹GR00T N1.6的平均成功率降至20.9%,而GloVLA仍保持88.5%;在真实UR10e机械臂上,整体成功率从35.6%提升到90.0%,平均推理时间缩短一半以上。
TacPAC: Tactile Prediction and Real-Time Action Correction in World-Action Models for Contact-Rich Manipulation
TacPAC turns tactile prediction into real-time action correction inside world-action models. Vision-centric predictions miss the local contact cues that decide contact-rich manipulation, yet naively predicting future tactile observations as extra views recovers only about a third of the achievable gain in the authors' experiments — a timing mismatch, since predictions precede execution while tactile feedback arrives during it. TacPAC caches the predicted contact a planned action chunk was conditioned on, together with the plan's own representation, and a tactile expert reads each newly observed tactile image against that cache to correct only the not-yet-executed actions. Feedback is interpreted against what the plan anticipated rather than in isolation, and a single correction pass costs 20.7x less than regenerating the chunk. On five real-robot tasks spanning precision insertion, fragile-object handling, object reorientation, and long-horizon manipulation, TacPAC leads every task and lifts average success from 22% for its vision-only base model to 64%. Code is open-sourced.
RoboSPA: Can VLA Models Go Beyond Simple Scenes and Short-Horizon Tasks?
RoboSPA (Robot Spatial-Procedural Assessment) is a large-scale robotic manipulation dataset and diagnostic benchmark for embodied reasoning in VLA models. It focuses on two core dimensions — fine-grained spatial reasoning and long-horizon procedural planning — covering 10 task categories and 56 base tasks, each instantiated across five difficulty levels to yield 280 variants with increasing spatial ambiguity and procedural complexity. The dataset contains 527K trajectories across multiple embodiments and diverse scenes. Beyond binary success rate, RoboSPA introduces diagnostic metrics for finer-grained evaluation. Experiments on representative VLA models show current systems still struggle with complex spatial relations, precise low-level execution, and memory-intensive planning, establishing RoboSPA as a challenging diagnostic benchmark for more capable, reliable, and generalizable embodied agents. Data and code are public.
FIRE-LIVWO: Robust LiDAR-Inertial-Visual-Wheel Odometry via Failure-Immune mmWave Radar Enhancement
FIRE-LIVWO is a failure-immune multi-modal odometry framework for large-scale underground coal mines. Built on an iterated error-state Kalman filter (IESKF), it tightly couples 4D mmWave radar, LiDAR, visual features, and wheel odometry inside a unified VoxelMap, jointly constructing LiDAR-radar point-to-plane residuals and sparse visual photometric residuals. In smoke-filled environments it exploits mmWave radar penetration with pointwise Doppler velocity constraints to preserve observability; in geometrically degenerate corridors it couples wheel odometry via non-holonomic constraints with online lever-arm compensation. Its central contribution is a degeneration-detection and adaptive fusion-model switching strategy grounded in geometric and visual observability analysis that quantifies observability online and dynamically adjusts modality weights. Real-world underground coal-mine experiments show reliable failure-boundary identification and modality switching with a 5.677 m average localization error, outperforming baselines; code is open-sourced.
What Matters, When? Diagnosing and Improving Conditional Visual Grounding in Visuomotor Imitation Policies
This paper diagnoses why visuomotor imitation policies fail when visually similar distractors appear, framing the problem as conditional visual grounding: the visual target needed for control changes with the manipulation phase and, in complex tasks, with the observed task state. Using Action Chunking with Transformers (ACT), the authors systematically introduce distractors with controlled color and shape similarity and localize failures to picking and placement stages, finding sensitivity depends on both similarity type and stage. Guided by the diagnosis, they combine distractor augmentation, phase-dependent attention regularization, and appearance-based visual prompting, substantially improving robustness in simulation and on a physical UR3e, and show the same failure pattern in a pretrained vision-language-action policy on a state-conditioned instrument-handling task.
Same Trajectory, Contradictory Rewards (ROBORMBENCH): Paraphrase Fragility in Vision Language Reward Models
ROBORMBENCH measures a neglected failure mode in VLM-based reward models for robot learning: paraphrase fragility. Using 2,390 real-robot trajectories with ground-truth progress labels and 21,673 verified paraphrases (lexical, syntactic, and action-goal rewrites), the authors show that merely rewording an instruction can substantially change predicted reward and even flip identical behavior between success and failure. The instability is widespread across proprietary and open-source VLMs, grows with more divergent rewrites, and is not reliably fixed by model scale or explicit reasoning; dedicated reward models trained with trajectory-grounded supervision are markedly more stable. The work establishes paraphrase robustness as a core requirement for reliable VLM reward modeling.
TacPAC:面向接触丰富操作的世界动作模型触觉预测与实时动作校正
TacPAC把触觉预测转化为世界动作模型中的实时动作校正。纯视觉世界模型对未来的预测缺少决定接触丰富操作成败的局部接触线索,而朴素地把未来触觉观测当作额外视角预测,在实验中只能收回约三分之一的可实现增益——根源在于时间错配:预测先于执行,触觉反馈却在执行中才到达。TacPAC在基础模型规划出动作块后,缓存该规划所依据的预测接触及其表征,由触觉专家模型将每个新触觉图像与该缓存比对,仅校正尚未执行的动作,单次遍历缓存即完成校正,比重新生成动作块便宜20.7倍。在精密插入、易碎物操作、物体重定向与长时程操作五类真实机器人任务上全面领先,平均成功率由纯视觉基础模型的22%提升至64%,代码已开源。
RoboSPA:VLA模型能否超越简单场景与短视距任务?
RoboSPA(机器人空间-程序化评估)是一个大规模机器人操作数据集与诊断基准,用于检验VLA模型在空间与程序复杂度递增条件下的具身推理能力。基准聚焦两个核心维度:细粒度空间推理与长时程程序化规划,覆盖10个任务类别、56个基础任务,每个任务在五个难度级别上实例化,共280个变体,空间歧义与程序复杂度逐级递增。数据规模为527K条轨迹,覆盖多种本体与多样场景。除二值成功率外,RoboSPA引入诊断性指标进行更细致的评估。对代表性VLA模型的实验显示,现有系统在复杂空间关系、精确低层执行与记忆密集型规划上仍力不从心,确立了RoboSPA作为开发更强、更可靠、更泛化具身智能体的挑战性诊断基准,数据与代码已公开。
FIRE-LIVWO:毫米波雷达增强的煤矿井下多模态鲁棒里程计
FIRE-LIVWO是一种面向大型地下煤矿的故障免疫多模态里程计框架:在迭代误差状态卡尔曼滤波(IESKF)中紧耦合4D毫米波雷达、LiDAR、视觉与轮式里程计,于统一VoxelMap中构建LiDAR-雷达点到面残差与稀疏视觉光度残差。烟尘环境下利用毫米波雷达强穿透并引入逐点多普勒速度约束维持可观性,退化巷道中通过非完整约束与在线杆臂补偿抑制漂移;核心贡献是基于几何与视觉可观性分析的退化检测与自适应融合模型切换策略,在线量化可观性并动态调整模态权重。真实煤矿井下实验表明其能准确识别失效边界并可靠切换模态,平均定位误差5.677米,优于对比基线,代码已开源。
什么在何时起作用?视觉运动模仿策略中条件视觉定位的诊断与改进
本文诊断了视觉运动模仿策略在出现视觉相似干扰物时失效的原因,将问题归结为条件视觉定位:控制所需的视觉目标会随操作阶段变化,在复杂任务中还会随观测到的任务状态变化。作者以 ACT(Action Chunking with Transformers)为对象,系统引入颜色与形状相似度受控的干扰物与容器,将失败定位到抓取与放置阶段,发现敏感性同时取决于相似类型与操作阶段。基于这一诊断,他们组合了干扰物增强、阶段相关注意力正则化与外观视觉提示三种手段,在仿真与实体 UR3e 上显著提升鲁棒性,并在预训练 VLA 的状态条件器械操作任务中复现了同样的失效模式。
同一轨迹,互相矛盾的奖励:ROBORMBENCH 揭示视觉语言奖励模型的改写脆弱性
ROBORMBENCH 系统度量了 VLM 奖励模型的一个被忽视的失效模式——改写脆弱性。基于 2,390 条带真实进度标签的真实机器人轨迹和 21,673 条经人工验证的改写指令(涵盖词汇、句法与动作目标层面),研究发现:仅仅换一种说法描述目标,就可能显著改变预测的奖励分数,甚至把完全相同的机器人行为从成功判成失败。这种不稳定在闭源与开源 VLM 中普遍且严重,随改写偏离程度加剧,且无法靠增大模型规模或显式推理可靠消除;而用轨迹监督训练的专用奖励模型则稳定得多。工作将改写鲁棒性确立为可靠 VLM 奖励建模的核心要求。
BRIDGE: An Open-Source Humanoid Platform via Morphology-Control Co-Design for Physical AI
BRIDGE is an open-source 88 cm humanoid platform produced by a data-driven morphology-control co-design framework that optimizes humanoid morphology for human-like movement. The framework introduces a metric jointly measuring kinematic retargeting fidelity to human motion and dynamic tracking performance, and the resulting platform outperforms baseline humanoids (Bumi, K1, Toddlerbot) across all reported metrics. The robot is released alongside its whole-body control policy, demonstrating foundational locomotion, robust balance, and highly dynamic maneuvers while capturing human motion data with superior fidelity.
Scaling Bimanual Household Manipulation from 1,500 Hours of Demonstrations to On-Policy Corrections
This work releases 1,500 hours of diverse bimanual household manipulation demonstrations and trains XR-2, a vision-language-action model, on the corpus through a purpose-built high-throughput data pipeline and a multi-stage training paradigm. Systematic experiments show strong task performance with favorable training efficiency and data utilization, and along two scaling axes — demonstration volume and post-training on DAgger correction data from real-time human interventions — task success improves steadily, exhibiting a clear scaling trend at current data scale. The dataset is open-sourced to support reproducible bimanual manipulation research.
MINERVA: How Small Can a Manipulation Policy Be and Still Solve LIBERO?
MINERVA measures the task-specific capacity floor of the LIBERO benchmark with deliberately compact visuomotor policies: a 0.54M-parameter policy reaches 95.1% average success over 2,000 rollouts across the four standard LIBERO suites — only 2.4 points below the reported LeRobot pi0.5 result with 7,700x fewer parameters. Performance saturates near 1M parameters and collapses below 0.25M. A task-ID permutation probe shows standard LIBERO instruction conditioning primarily selects among memorized tasks, and the tiny policy replans every control step in 5-9 ms on a laptop CPU — 113x faster than SmolVLA and 1,400x faster than pi0.5.
LaPla: Continuous Actions from Discrete Minds — Latent-Aligned Planning for End-to-End Autonomous Driving
LaPla is a unified Vision-Language-Action (VLA) framework with latent-aligned planning that grounds semantic understanding in precise motion execution. It builds an action tokenizer from a residual vector-quantized VAE that captures vehicle kinematics, then repurposes that representation as a continuous physical prior rather than a discrete codebook: concurrent action queries attend to multimodal context in a single forward pass and project hidden states directly into the pretrained latent space, where a frozen decoder translates them into actions — eliminating quantization error and bypassing autoregressive generation. On nuScenes, LaPla reduces long-horizon L2 error by 15.52% versus state-of-the-art VLA methods; closed-loop evaluation in NVIDIA AlpaSim improves success rate by 33.34 percentage points with significantly lower inference latency.
AdaRoboVLG: Adaptive Vision-Language Grasping via Composable Foundation Priors and Generalizable Grasp Synthesis
AdaRoboVLG is a task-adaptive Vision-Language-Grasp (VLG) framework supporting generalizable grasp synthesis across different robotic hands. Unlike VLG methods that tightly couple foundation models with end-to-end grasp policies, it learns an efficient generalizable base policy that generates and evaluates physically feasible grasp candidates through explicit kinematic mapping and force-closure-based stability estimation, while task-dependent understanding is offloaded to specialized foundation-model modules that provide composable spatial, cognitive, and temporal priors — enabling contextually adaptive grasping without retraining the underlying grasp policy. Extensive simulation and real-world experiments show efficient base-policy learning with strong cross-hand generalization, effective use of the three priors on representative grasping challenges without compromising synthesis performance versus state-of-the-art methods, and joint operation of the priors for functional grasping in cluttered and dynamic environments.
GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation
GIFT addresses the action-sufficiency gap — the mismatch between the visual richness of vision-language pretraining and what control actually needs — by supervising intermediate features with three control-relevant structures: geometry governing motion feasibility, affordances encoding instruction-relevant entities, and goals grounding instructions in task-relevant regions. The framework is architecture-flexible and was instantiated in a VLA policy, a direct-action World-Action Model, and an inverse-dynamics WAM without changing each model's action formulation. GIFT variants outperform their counterparts by 4.6/12.6/5.2 points on zero-shot LIBERO-Plus (79.6%/72.6%/87.8%) and by 12.6/9.0/8.4 points on RoboCasa, with the largest gains on articulated objects and high-precision real-world manipulation under unseen visual and spatial perturbations.
BRIDGE:面向物理 AI 的形态-控制协同设计开源人形平台
BRIDGE 是一个开源 88 厘米人形平台,由数据驱动的形态-控制协同设计框架生成:该框架以类人运动为目标联合优化人形机器人的形态。新提出的评价指标同时度量了运动重定向到人类动作的保真度与动态跟踪性能。相比 Bumi、K1、Toddlerbot 等基线人形机器人,BRIDGE 在所有报告指标上达到最先进水平,并随全身控制策略一同开源发布,展示了基础 locomotion、鲁棒平衡与高动态机动能力,同时以更优保真度复现人类运动数据。
从 1500 小时示教到在策略纠正:双臂家务操作的规模化之路
该工作开源发布 1500 小时多样化双臂家务操作示教数据,并基于专用高通量数据管线与精心设计的多阶段训练范式,在这些数据上训练了视觉-语言-动作模型 XR-2。系统实验表明其任务表现强劲,训练效率与数据利用率俱佳;沿两条规模化轴线——专家示教数据量与基于实时人工干预的 DAgger 在策略纠正后训练——任务成功率均稳步提升,在当前数据规模上呈现清晰一致的扩展趋势。数据集已开源,以支持双臂操作学习的可复现研究。
MINERVA:操作策略究竟可以做到多小还能解决 LIBERO?
MINERVA 用刻意压缩的视觉运动策略度量 LIBERO 基准的任务专属容量下限:仅 0.54M 参数的策略在四个标准 LIBERO 套件的 2000 次 rollout 中取得 95.1% 平均成功率——比论文报道的 LeRobot pi0.5 仅低 2.4 个百分点,参数量却少 7700 倍。性能在约 1M 参数处饱和,低于 0.25M 则崩溃。任务 ID 置换探针显示标准 LIBERO 指令条件化主要是在已记忆任务间做选择;该微型策略在笔记本电脑 CPU 上每控制步重规划仅需 5-9 毫秒,比 SmolVLA 快 113 倍、比 pi0.5 快 1400 倍。
LaPla:离散思维与连续动作——端到端自动驾驶的潜在对齐规划
LaPla 是一个带潜在对齐规划的视觉-语言-动作(VLA)框架,把语义理解落地为精确的运动执行。它先用残差向量量化 VAE 构建捕捉车辆运动学的动作分词器,再把该表示当作连续物理先验而非离散码本:并发动作查询在单次前向中因果地关注多模态上下文,把隐状态直接投影到预训练潜在空间,由冻结解码器转译为动作——既消除了量化误差,也绕开了自回归生成。在 nuScenes 上,LaPla 相比最先进 VLA 方法将长时域 L2 误差降低 15.52%;在 NVIDIA AlpaSim 闭环评测中成功率提升 33.34 个百分点,推理延迟显著降低。
AdaRoboVLG:基于可组合基础模型先验与通用抓取合成的自适应视觉-语言抓取
AdaRoboVLG 是一个任务自适应的视觉-语言-抓取(VLG)框架,支持跨不同机器人手的通用抓取合成。与把基础模型紧耦合进端到端抓取策略的既有 VLG 方法不同,它学习一个高效、可泛化的基础策略:通过显式运动学映射与力闭合稳定性估计来生成并评估物理上可行的抓取候选;任务相关的理解则交给专门的基础模型模块,以可组合的空间、认知与时间先验形式融入抓取合成,无需重训底层抓取策略即可实现上下文自适应的抓取。大量仿真与真实实验表明:基础策略学习高效且跨手泛化强;三类先验可分别应对代表性抓取挑战且不损害相较最先进方法的合成性能;先验联合运作可在杂乱动态环境中实现功能性抓取。
GIFT:面向机器人操作的动作导向结构化监督引导中间特征训练
GIFT 针对"动作充分性缺口"——视觉-语言预训练提供的视觉丰富性与控制真正所需信息之间的错配——提出对中间特征施加三类与控制相关的结构监督:决定运动可行性的几何、编码指令相关实体的可供性、以及把指令 grounding 到任务相关区域的目标。该框架与架构无关,已在 VLA 策略、直接动作世界动作模型(WAM)与逆动力学 WAM 上实例化,且不改动各模型原有的动作表达。在零样本 LIBERO-Plus 上,GIFT 各变体以 79.6%/72.6%/87.8% 超出对应基线 4.6/12.6/5.2 分;在 RoboCasa 上以 61.4%/83.6%/82.3% 超出 12.6/9.0/8.4 分;在关节物体任务与受未见视觉/空间扰动的高精度真实操作中增益最大。
Spatially Aware World Action Model via Geometric Latent Diffusion
SA-WAM repurposes a pretrained video diffusion model into a spatially aware World Action Model that predicts actions, RGB, and depth within a single diffusion backbone. A nonlinear depth encoding maps unbounded depth into the bounded input domain of the frozen VAE tokenizer, adding geometric awareness without 3D-specific fine-tuning. SA-WAM achieves state-of-the-art results on RoboCasa and LIBERO-Plus while improving future-state predictions, and outperforms strong baselines on a real UR5 arm with large gains in randomized environments.
World-Model-Augmented Visual Locomotion for Humanoids on Foothold-Constrained Terrain
WM-LOCO jointly trains a recurrent world model and a PPO policy so a humanoid can traverse foothold-constrained terrain — stepping stones, gaps, and narrow stair treads — using only proprioception and a single onboard depth image, with no explicit foothold labels. In simulation it succeeds on gaps and stepping stones where a matched baseline fails completely, and the same policy deployed on a physical Unitree G1 traverses all three terrain classes with a 93.3% average success rate.
ZETA: A Controlled Study of Zero-Shot Cross-Embodiment VLA Transfer for Tabletop Manipulation
ZETA is the first controlled study of zero-shot cross-embodiment transfer for Vision-Language-Action models. It separates strict zero-shot transfer (target embodiment absent from all training data) from pretrain-exposed zero-shot transfer (target embodiment seen only in pretraining), and introduces a benchmark spanning 14 held-out target embodiments in simulation and the real world. Controlled factor analysis shows local end-effector representations (+15pp), source embodiment diversity (+18pp), and auxiliary co-training (+7pp) all help, while adding just 5% target-embodiment data during pretraining improves target-embodiment progress by 13.4 percentage points.
SA-WAM:几何潜在扩散的空间感知世界动作模型
SA-WAM 将预训练视频扩散模型改造为空间感知的世界动作模型:在单一扩散骨干内同时预测动作、RGB 与深度。非线性编码把无界的深度信号映射到冻结 VAE tokenizer 的有界输入域,无需三维微调即可引入几何信息。SA-WAM 在 RoboCasa 与 LIBERO-Plus 上取得最先进结果,同时改善未来状态预测,并在真实 UR5 机械臂上优于强基线,在随机化环境中增益尤为显著。
WM-LOCO:立足点受限地形上的世界模型增强人形机器人运动
WM-LOCO 联合训练循环世界模型与 PPO 策略,使人形机器人仅凭本体感觉与单张机载深度图即可穿越立足点受限地形——踏脚石、沟隙与窄台阶——全程无需显式落足标签。仿真中它在沟隙与踏脚石上成功通过(匹配基线完全失败);同一策略部署到实体 Unitree G1 后,在全部三类地形上平均成功率达 93.3%。
ZETA:桌面操作零样本跨本体 VLA 迁移的受控研究
ZETA 是对 VLA(视觉-语言-动作)模型零样本跨本体迁移的首个受控研究:它区分“严格零样本”(目标本体完全不出现在训练数据中)与“预训练暴露式零样本”(目标本体仅在预训练中出现),并提出覆盖仿真与真机、含 14 个留出目标本体的评测基准。受控因素分析显示:局部末端执行器表征(+15 个百分点)、源本体多样性(+18)与辅助共训练(+7)均有助益;预训练中仅加入 5% 目标本体数据即可使目标本体进度提升 13.4 个百分点。
Non-Prehensile Throwing: A Reinforcement Learning Perspective
This paper casts non-prehensile (grasp-free) throwing as a reinforcement learning problem: the policy explicitly exploits sliding and rolling contact modes and directly optimizes joint-space trajectories, requiring no analytical contact models or custom parameterizations. The policy reaches 99% simulated success across thousands of configurations and generalizes to unseen objects; deployed zero-shot on a UR5e running near its physical limits at 5 m/s end-effector velocity, it throws heavy 790 g and large 20x20x28 cm objects to targets up to 350 cm away or 180 cm high with a 97% real-world success rate.
A System for Fast, Resilient, and Adaptable Loco-Manipulation Behaviors on Humanoid Robots
This paper presents a robot-local, runtime-editable behavior authoring and runtime system that treats behavior architecture as a primary enabler of humanoid capability: object-centric Affordance Templates provide organization and logic, a behavior scene with primitive scene actions makes perception editable at runtime, and a continuously synchronized operator interface supports runtime authoring, monitoring, and repair. Demonstrated on Unitree H1-2 and Alex across six task variants, the system executes a push-door traversal in 34 seconds and color-sorts six balls under human disturbance in 45 seconds, with new loco-manipulation behaviors authored from scratch in hours.
Facet-0: A Robotic Foundation Model for Contact-Rich Precise Manipulation
Facet-0 is a robotic foundation model that predicts and values the contact consequences of its actions: it aligns a causal wrench history with vision-language semantics and kinematic state, uses flow matching to generate action chunks jointly with the wrist-wrench profile they are expected to induce, and concentrates reinforcement learning on decisive interactions via a distributional Action-Wrench Critic and contact-selective credit. Trained on ManuFacet-1K, a 1,000-hour force-synchronized corpus spanning three embodiments, it reaches 82% mean success on five sub-millimeter computer-assembly tasks versus 15% for the strongest baseline, with 0.5 mm placement accuracy and 50 ms command latency.
非抓取抛掷:一种强化学习视角
本文用强化学习解决非抓取(无抓取)抛掷问题:策略显式利用滑移与滚动接触模式,直接优化关节空间轨迹,无需解析接触模型或自定义参数化。策略在仿真中达到数千构型 99% 的成功率并能泛化到未见物体;零样本部署到 UR5e 后,以接近物理极限的 5 m/s 末端速度,可将 790 g 重物与 20×20×28 cm 大物体抛至最远 350 cm、最高 180 cm 的目标,真实世界成功率 97%。
面向人形机器人的快速、鲁棒、可适配移动操作行为系统
本文提出一套机器人本地的、运行时可编辑的行为编排与运行系统,把人形机器人的移动操作能力建立在"行为架构"之上:以物体为中心的 Affordance Templates 提供组织与逻辑,行为场景与原始场景动作实现运行时可编辑感知,操作员界面与机器人持续同步以支持运行时编写、监控与修复。系统在 Unitree H1-2 与 Alex 上完成六种任务变体演示,推门穿越 34 秒、人类干扰下按颜色分拣六个球 45 秒,新行为可在数小时内从零创建。
Facet-0:面向接触丰富精密操作的机器人基础模型
Facet-0 是一个会"预测并评估接触后果"的机器人基础模型:它把视觉-语言语义、运动学状态与因果腕力历史对齐,用流匹配同时生成动作块与预期诱发的腕部力/力矩曲线,并通过分布式的 Action-Wrench Critic 与接触选择性奖励把强化学习集中在决定性交互上。在 1,000 小时力同步语料 ManuFacet-1K 上训练后,该系统在五项亚毫米计算机装配任务上取得 82% 的平均成功率(最强基线仅 15%),放置精度 0.5 mm、指令延迟 50 ms。
LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation
LightNav-0 is a compact generalist navigation model that elicits the spatial intelligence of a pretrained VLM and aligns it with navigation, without task-specific prediction heads. A unified token interface expresses task-, scene-, and embodiment-agnostic spatial intent through dual-channel pointing, while a residual vector-quantized action tokenizer maps that intent to precise, embodiment-specific trajectories. Trained on 2K+ scenes and 4K+ hours of embodied navigation data, LightNav-0 achieves state-of-the-art monocular success rates across all 10 public navigation simulation settings and demonstrates zero-shot generalization across robot embodiments and dynamic targets in the real world.
Driving on Memory
This study probes how much of a NAVSIM score actually comes from reacting to the dynamic traffic scene. The authors remove a driving model's camera input and replace it with memories from prior drives at the same location — persistent scene information such as road layout, but no current traffic state. Memory alone reaches or even exceeds leading end-to-end methods on NAVSIM, showing that a high NAVSIM score does not require reacting to the evaluated scene. The effect is benchmark-dependent: driving from memory causes substantially larger drops on Bench2Drive and RealEngine. Code is released.
SUN: Persistent Programs For Language-Grounded Control-to-Learning-to-Real Policies
Kuafu introduces Semantically UNified (SUN) Programs — typed executables that define geometric and contact relations once and compile them into aligned MPC costs, RL rewards, satisfaction predicates, transition guards, and diagnostics. A large vision-language system synthesizes SUN Programs from language and scene semantics, MPC screens feasibility, and the retained semantics train stage-conditioned policies. Across nine tasks Kuafu reaches 82.03% macro-success (vs. 35.67% sparse-reward and 24.75% Stage-BC baselines), generates 10.57x more successful trajectory time per teleoperation hour at 8192-way scale, and its 500-trajectory datasets train DP3 policies to 46.0% simulation and 34.7% physical Franka/Kinova success — all without demonstrations or dense rewards.
LightNav-0:激发VLM空间智能的通用具身导航模型
LightNav-0是一个紧凑的通用导航模型,它激发预训练VLM的空间智能并将其与导航对齐,无需任何任务专用预测头。统一token接口通过双通道指向表达与任务、场景、形态无关的空间意图,残差向量量化动作分词器再把意图映射为精确的、形态专属的轨迹。基于2000+场景、4000+小时具身导航数据的训练,LightNav-0在全部10个公开导航仿真设置上取得最先进的单目成功率,并在真实世界中展示了跨机器人形态、跨场景、面向静态与动态目标的零样本泛化能力。
Driving on Memory:仅凭记忆完成驾驶评测
这项研究探明了一个问题:NAVSIM分数中有多少真正来自对动态交通场景的反应。作者移除驾驶模型的摄像头输入,改用同一地点先前驾驶的记忆——即道路布局等持久场景信息,但不含任何当前交通状态。仅凭记忆,模型在NAVSIM上就能达到甚至超越领先的端到端方法,说明高分并不需要模型对被测场景作出反应。该效应依赖具体基准:在Bench2Drive与RealEngine上,仅凭记忆驾驶会造成显著更大的性能下降。代码已开源。
SUN:面向语言接地控制—学习—实机策略的持久化程序
Kuafu提出语义统一(SUN)程序——一种类型化可执行程序,几何与接触关系只需定义一次,即可编译为对齐的MPC代价、满足谓词、RL奖励、转移守卫与诊断信号。大型视觉语言系统从语言与场景语义自动合成SUN程序,MPC先进行可行性筛查,保留的语义再用于训练分阶段策略。在九个任务上Kuafu取得82.03%的宏观成功率(稀疏奖励基线35.67%、Stage-BC基线24.75%),8192路并行规模下每小时遥操作产生的成功轨迹时间是传统方式的10.57倍;每个任务500条轨迹训练出的DP3策略在仿真中达到46.0%、在真实Franka与Kinova机械臂上达到34.7%的成功率——全程无需示范数据或人工稠密奖励。
Riemann-1.0: An Embodied World Action Model for Physical AI
Riemann-1.0 is a fully causal autoregressive World Action Model that jointly models multi-view visual observations, robot states, and embodiment-specific actions as causal state transitions in a single sequence, functioning as both an executable robot policy and a multi-embodiment visual world simulator. Progressive embodied pretraining over 200K+ hours of human and robot interaction data yields 94.3% on RoboTwin2.0, 99.0% on LIBERO, 62.6% on RoboCasa-365 (+8.4% over previous best), and 85.0% SR / 94.4% PSR on long-horizon real-world manipulation, exceeding the strongest open-source baseline by 15% SR.
CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical Simulators
CLAP trains action-conditioned video world models on internet-scale heterogeneous video spanning humans and diverse robots, reconciling incompatible action spaces via end-effector poses, language instructions, and latent actions under a curriculum-based cross-embodiment learning recipe. It approaches or surpasses single-embodiment state of the art on DROID, compounds gains through few-shot adaptation, and ships the most comprehensive suite of action-conditioned video world models to date, with all code and models open-sourced.
Riemann-1.0:面向物理AI的具身世界动作模型
Riemann-1.0是一个完全因果的自回归世界动作模型(WAM),将多视角视觉观测、机器人状态与形态特定动作作为因果状态转移统一建模在单一序列中,同一模型既可在线执行机器人策略,又可充当多形态视觉世界模拟器。基于20万+小时人类与机器人交互数据的渐进式具身预训练,在RoboTwin2.0上达到94.3%、LIBERO上99.0%、长时程组合基准RoboCasa-365上62.6%(较此前最优提升8.4%);真实世界长时程操作任务成功率达85.0%、进度成功率达94.4%,较最强开源基线高出15个百分点。
CLAP:跨形态视频世界模型,零样本的物理模拟器
CLAP在互联网规模的异构视频(涵盖人类与多种机器人)上训练动作条件视频世界模型,通过末端执行器位姿、语言指令与潜在动作三种表征统一互不兼容的动作空间,并采用基于课程学习的跨形态训练范式:先用潜在动作从无标注视频学习基础物理先验,再将其接地到末端执行器动作空间实现零样本部署。在DROID等挑战性环境中达到或超越单形态SOTA,并通过少样本适应进一步放大优势;所发布套件是目前覆盖动作条件空间与机器人形态最全面的视频世界模型,代码与模型全部开源。
Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning
Q-Planning equips a large visuomotor behavior-cloning policy with a small off-policy Q-function, enabling value-guided action selection at inference and online self-improvement that fine-tunes only the Q-function. On LIBERO and bimanual RoboTwin, ten self-improvement iterations lift every benchmark score (LIBERO-10 93% to 99%, RoboTwin 83.8% to 91.4%); on two contact-rich bimanual real-robot tasks the same loop improves purely from its own deployment rollouts (stack-cups 40% to 90%, insert-wallet 25% to 80%), whereas filtered SFT on successful rollouts stalls at 55% and 30%.
Beyond Imitation:基于离策略Q-规划的自改进机器人策略
Q-Planning为大规模视觉运动行为克隆策略配备一个小型离策略Q函数,推理时用Q加权选择动作,在线自改进时只微调Q函数而不动策略权重。在LIBERO与双臂RoboTwin上,十轮自改进迭代提升所有基准分数(LIBERO-10从93%升至99%,RoboTwin从83.8%升至91.4%);在两个接触密集的双臂真实机器人任务中,同一循环仅凭部署rollout即可自改进(叠杯从40%升至90%、插钱包从25%升至80%),而仅用成功rollout的SFT停滞在55%和30%。
BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving
BrainWAM is a structured action-space coordination framework that unifies semantic reasoning (VLA priors) and predictive dynamics (world models) for end-to-end autonomous driving, reaching state-of-the-art 89.5 PDMS on NAVSIM v1 and 89.6 EPDMS on NAVSIM v2 while shortening inference latency via asynchronous rectified-flow inference.
DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation
DreamX-Phi 1.0 is an action-conditioned video world model that predicts future observations for robotic manipulation from a frame, a language instruction, and an action sequence, using per-arm SE(3) geometric encoding, a depth branch, and SAM3+V-JEPA object-consistency constraints; it ranked first on Track 1 and second on Track 2 of the WorldArena 2.0 Challenge.
HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark
HumanTracker is a large-scale benchmark containing approximately 153 hours of professional optical motion trajectories for evaluating humanoid motion tracking, paired with HumanScore, a preference-aligned metric trained on 12K motion pairs that better predicts human perception and reveals contact and stability failures that kinematic metrics miss.
BrainWAM:面向自动驾驶的语义先验与预测动力学动作空间协同
BrainWAM是一种结构化的动作空间协同框架,将语义推理(VLA先验)与预测动力学(世界模型)统一用于端到端自动驾驶,在NAVSIM v1上达到89.5 PDMS、NAVSIM v2上达到89.6 EPDMS的领先水平,并通过异步整流流推理缩短推理延迟。
DreamX-Phi 1.0:面向机器人操作的动作条件视频世界模型
DreamX-Phi 1.0是一种动作条件视频世界模型:给定观测帧、语言指令与动作序列(末端位姿与夹爪状态),预测未来的观测结果;通过逐臂SE(3)几何编码、深度分支与SAM3+V-JEPA物体一致性约束保证预测忠实度,在WorldArena 2.0挑战赛中获赛道一第一、赛道二第二。
HumanTracker:面向全面且符合人类感知的人形运动跟踪基准
HumanTracker是一个大规模基准,包含约153小时来自多位专业表演者的光学运动轨迹,用于评估人形运动跟踪;配套提出HumanScore——一种基于1.2万对运动数据训练、与人类偏好对齐的指标,能更好地预测人类感知,并揭示运动学指标容易忽略的接触与稳定性失败。
StellaVLA: In-Context Structured Demonstration for Generalizable Vision-Language-Action Models
StellaVLA adapts at test time by conditioning on a single retrieved demonstration: an automated offline pipeline converts raw trajectories into structured demonstrations (task plans, sub-goal descriptions, verbalized 3D motion) that let the policy reason about the task rather than mimic pixels. It ranks first on the VLA-Arena leaderboard (Aug 1, 2026) with 0.63 vs 0.44 for pi0.5, reaches 98.8% on LIBERO, and adds zero inference latency.
G0.5: One Autoregressive Stream for Robot Reasoning and Action
G0.5 unifies robot reasoning and action in a single autoregressive transformer decoder that emits reasoning and action tokens under one objective, surpassing state-of-the-art models across 7 independent regimes, including real-robot fine-tuning (76.7% vs 53.3% for pi0.5 on R1lite/R1pro), the 2025 BEHAVIOR Challenge, DROID zero-shot transfer (82.5%), and LIBERO (98.9%).
StellaVLA:上下文结构化演示驱动的可泛化视觉-语言-动作模型
StellaVLA在测试时仅凭一条检索到的演示即可完成分布外适应:离线管线将原始轨迹自动转化为任务计划、子目标描述与口语化3D动作等结构化演示,作为上下文指导让策略“理解任务”而非“模仿像素”。在VLA-Arena排行榜(2026-08-01)以0.63总分登顶(π0.5为0.44),LIBERO平均成功率98.8%,且推理阶段零额外延迟。
G0.5:单自回归流统一机器人推理与动作生成
G0.5将推理与动作生成统一到单一自回归Transformer解码器中,推理token与动作token在同一目标下联合生成;在7个独立评测场景中全面超越现有SOTA,包括真实机器人微调(R1lite/R1pro上76.7%对π0.5的53.3%)、2025 BEHAVIOR挑战赛、DROID零样本迁移(82.5%)与LIBERO(98.9%)。
Hidden in Plain Sight: Diffusion-Based Unrestricted Robotic Attacks on Vision-Language-Action Models
DURA generates visually natural adversarial patches for Vision-Language-Action models via diffusion-based unrestricted attacks, working in both white-box and black-box settings and steering robots toward attacker-specified actions in simulation and the physical world.
Flex-π: A Multi-Stream World-Action Model with Compute Flexibility
Flex-π is a 6B-parameter world-action model that jointly denoises 3D geometry, object-centric semantics, RGB, and actions in a shared latent space, using a frozen video VAE that encodes pointmaps almost losslessly — achieving 2-7x gains over strong baselines on dexterous real-world bimanual manipulation while running faster than π0.5.
Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning
Surgical WAM pretrains a generative world-action model on action-free endoscopic video, then fine-tunes it on a fixed budget of action-labeled demonstrations, improving average success from 63.5% to 77.8% across four simulated surgical manipulation tasks.
MiDAS: Adaptation of Generalist Robot Policies with Minimal Data
MiDAS studies minimal-data adaptation: a pretrained policy learns a new task from as little as one demonstration, anchored by behavior cloning and then improved through value-based online RL on a residual policy parameterization. It recovers strong performance on LIBERO and RoboCasa from a single demo and improves a fragile policy on a bimanual YAM platform over ~6 hours of online interaction — per the paper, the first reliable robot policy adaptation from a single task demonstration.
藏在明处:面向视觉-语言-动作模型的扩散式无限制机器人攻击
DURA通过基于扩散模型的无限制攻击为视觉-语言-动作模型生成视觉上自然的对抗补丁,支持白盒与黑盒两种设置,能在仿真与真实物理世界中引导机器人执行攻击者指定的动作。
Flex-π:具备计算弹性的多流世界-动作模型
Flex-π是一个60亿参数的世界-动作模型,在共享潜空间中联合去噪3D几何、物体中心语义、RGB与动作信号;利用冻结的视频VAE几乎无损地编码点图,在灵巧的真实世界双臂操作任务上相较最强基线取得2-7倍提升,且推理速度快于π0.5。
Surgical WAM:面向数据高效手术机器人学习的世界-动作模型
Surgical WAM先在无动作标注的内镜视频上预训练生成式世界-动作模型,再在固定预算的动作标注演示上微调,在四个模拟手术操作任务上将平均成功率从63.5%提升至77.8%。
MiDAS:单条演示即可启动的通用机器人策略最小数据适应
MiDAS提出“最小数据适应”范式:预训练策略仅需一条演示,经行为克隆锚定后再通过残差策略上的基于价值的在线强化学习自主提升。在LIBERO与RoboCasa上仅凭单条演示即恢复强任务性能,并在双臂YAM平台上用约6小时在线交互将脆弱策略提升为鲁棒策略——据论文所述,这是首次实现可靠的单条演示策略适应。
FactorDrive: Adaptive Multi-Step Reasoning Driven by Planning-Critical Factors for End-to-End Autonomous Driving
FactorDrive is an end-to-end autonomous driving framework that grounds planning reasoning in trajectory-relevant spatial-physical evidence and adapts reasoning depth to scene-specific planning demands. It builds on large-scale driving-domain instruction tuning, constructs PCF-CoT, a chain-of-thought dataset organized around planning-critical factors (PCFs), and introduces Quality Search-Guided Group Relative Policy Optimization (QS-GRPO), which uses Monte Carlo Tree Search guided by trajectory-level planning rewards to discover higher-quality reasoning paths and optimize the policy via GRPO. Experiments on open-loop nuScenes and closed-loop-oriented NAVSIM benchmarks demonstrate state-of-the-art planning performance.
SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation
SLIM is a compact 0.5B-parameter latent interaction policy that learns action-grounded predictive latents — representations capturing both action-conditioned future transitions and the actions that explain observed changes — through self-supervised masked trajectory prediction combining action reconstruction with future-latent prediction. A compact Mixture-of-Transformers backbone models interactions between observation latents and action tokens, and the policy is trained with flow matching for language-conditioned action generation. Across simulation benchmarks and real-world evaluation, SLIM matches or exceeds representative large-scale VLA and world-action-model baselines with fewer parameters, no additional embodied pretraining, lower inference latency, and substantially lower GPU memory usage.
RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance
RynnValue is an open-source value foundation model for robotic manipulation that replaces preference or progress annotations with temporal distance — the directed cost-to-go from an observation to a language-specified goal — as the supervision target. Trained on over 7,000 hours and roughly 3 million instruction-conditioned clips without preference labels, it attains a Kendall's tau_a of 0.675 on RBM-EVAL-OOD, surpassing the fully preference-supervised state of the art (0.655) and more than doubling a progress-only baseline (0.292), while generalizing zero-shot to unseen tasks, embodiments, and viewpoints. Converted into dense rewards via potential-based shaping, it lifts real-world policy success from 52.5% to 72.5% online and from 63.8% to 82.5% offline.
FactorDrive:由规划关键因子驱动的自适应多步推理端到端自动驾驶
FactorDrive 是一个端到端自动驾驶框架,将规划推理锚定在与轨迹相关的空间-物理证据上,并根据场景特定的规划需求自适应推理深度。它以大规模驾驶域指令微调为基础,构建了围绕规划关键因子(PCF)组织的 PCF-CoT 思维链数据集,并提出 QS-GRPO:用轨迹级规划奖励引导蒙特卡洛树搜索(MCTS)发现更高质量的推理路径,再通过 GRPO 优化策略。在开环 nuScenes 与面向闭环的 NAVSIM 基准上的大量实验表明,FactorDrive 取得了最先进的规划性能。
SLIM-0.5B:面向机器人操作的行动锚定预测隐空间学习
SLIM 是一个仅 0.5B 参数的紧凑隐空间交互策略,通过自监督掩码轨迹预测学习行动锚定的预测隐变量——同时刻画行动条件下的未来状态转移与能解释观测变化的行动——结合动作重建与未来隐变量预测。紧凑的 Mixture-of-Transformers(MoT)骨干网络建模观测隐变量与动作 token 之间的交互,策略采用流匹配训练以生成语言条件下的动作。在仿真基准与真实世界评估中,SLIM 以更少的参数、无需额外的具身预训练、更低的推理延迟与显著更低的 GPU 内存占用,达到或超过代表性的大规模 VLA 与世界行动模型基线。
RynnValue:用时间距离扩展机器人价值基础模型
RynnValue 是一个开源的机器人操作价值基础模型,用时间距离——从观测到语言指定目标的定向代价——替代偏好或进度标注作为监督信号。该方法无需偏好标注,在超过 7,000 小时、约 300 万条指令条件片段上训练,在 RBM-EVAL-OOD 上取得 0.675 的平均 Kendall tau_a,超过完全偏好监督的最先进方法(0.655),并比仅用进度的基线(0.292)提升一倍以上,且能零样本泛化到未见任务、具身与视角。通过基于势能的奖励整形转为稠密奖励后,真实世界策略成功率在线从 52.5% 提升到 72.5%,离线从 63.8% 提升到 82.5%。
AtlasVLA: Persistent World-Ego State Modeling for Vision-Language-Action Models
AtlasVLA addresses the reactive limitations of Vision-Language-Action (VLA) models in partially observable, long-horizon tasks by introducing a dual-memory architecture: a 4D Persistent World State Memory that lifts transient 2D observations into a globally updated voxel-hashed spatial state, and an Ego-Working State Memory that tracks historical ego state and task progress. A diffusion transformer is conditioned on this joint world-ego state for proactive reasoning. Using only a single wrist-mounted camera, AtlasVLA achieves state-of-the-art results on LIBERO, RLBench, and real-world benchmarks, outperforming multi-view baselines by 9.4% on LIBERO-Long and 17.5% in real-world long-horizon tasks.
Detection and Ranging of Transient Extrinsic Contacts Based on 6D Dynamic Tactile Sensing
TECDAR detects and localizes transient extrinsic contacts (collisions between a grasped object and the environment) using dynamic tactile sensing built on a single 2.5x3 mm 6D inertial measurement unit in each gripper tip. The sensor captures sub-millisecond tip deformations at 7 kHz while streaming only 84 KB/s; fusing tactile data with robot pose through an extended Kalman filter localizes contacts to roughly 7 mm accuracy within 180 ms. This near-instantaneous localization lets a robot correct its trajectory on a millisecond scale, enabling precise tool manipulation and touch-driven exploration for precision assembly, surgical assistance, and autonomous exploration.
Learning Fault-Tolerant Locomotion with Adaptive Gait Timing
This work proposes a deep reinforcement learning approach for fault-tolerant quadruped locomotion under actuator power loss. An asymmetric actor-critic architecture lets the critic use privileged information during training while the actor reconstructs a corresponding latent representation from proprioception, enforced by a latent-alignment loss. A learnable gait frequency parameter augments the action space, enabling adaptive gait timing under terrain variation and actuator degradation without predefined faulty-leg strategies. The method is validated in high-fidelity simulation on uneven terrain and in real-world experiments on flat ground using a 68 kg quadruped.
AtlasVLA:面向视觉-语言-动作模型的持久世界-自我状态建模
AtlasVLA 针对视觉-语言-动作(VLA)模型在部分可观测、长时程任务中的反应式局限,提出双记忆架构:4D 持久世界状态记忆将瞬态 2D 观测提升为全局更新的体素哈希空间状态,自我工作状态记忆则追踪历史自我状态与任务进度;扩散 Transformer(DiT)以该联合世界-自我状态为条件进行主动推理。仅使用单个腕部相机,AtlasVLA 即在 LIBERO、RLBench 与真实世界基准上取得 SOTA 成绩,在 LIBERO-Long 上超出多视角基线 9.4%,在真实长时程任务上超出 17.5%。
基于 6D 动态触觉传感的瞬时外部接触检测与测距
TECDAR 利用嵌在夹爪指尖的单个 2.5×3 mm 6D 惯性测量单元(IMU)实现动态触觉传感,用于检测并定位被抓物体与环境之间的瞬时外部接触。传感器以 7 kHz 采样率捕捉亚毫秒级的指尖形变,而数据流仅为 84 KB/s;将触觉数据与机器人位姿经扩展卡尔曼滤波融合后,可在 180 ms 内将接触定位到约 7 mm 的精度。这种近乎实时的定位使机器人能在毫秒尺度上修正轨迹,为精密装配、手术辅助以及依赖触觉探索的自主作业提供了关键能力。
学习自适应步态时序的容错运动控制
本文提出一种面向执行器断电故障的深度强化学习容错四足运动控制方法。非对称 actor-critic 架构让 critic 在训练时使用特权信息,而 actor 仅从本体感觉观测中重建对应的潜在表征,并通过潜在对齐损失保证两者一致。动作空间被扩充了一个可学习的步态频率参数,使策略能够根据地形变化与执行器退化自适应调整步态时序,而无需预定义故障腿策略。该方法在崎岖地形的高保真仿真与 68 kg 四足机器人的平地真实实验中均得到验证。
Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robots
This study shows that adversarial text placed inside a robot's visual field can act as an indirect prompt injection against VLM-controlled manipulation. Across 5,670 trials on GPT-4o, Gemini 2.5 Flash, and Qwen3-VL-32B, physical signage attacks succeeded at 27.0%, 29.4%, and 5.0% respectively, with authority-impersonation and negation attacks transferring across all three models. Simple defenses — prompt hardening, two-stage verification, and text masking — reduce risk substantially (up to 100%), though each carries trade-offs.
DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation
DyPES-VLA is a cross-embodiment vision-language-action framework that separates shared dynamics priors from embodiment-specific control. A future-prediction objective trains the vision-language backbone to capture object motion, contact, and interaction-induced scene changes across heterogeneous data, while an embodiment-specific Mixture-of-Experts action head translates those priors into actions directly in each robot's native action space, eliminating manual action pre-alignment. As a single generalist policy it reaches 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin 2.0.
ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation
ω-0 is a whole-body world-action model that lets a single humanoid policy move, rebalance, and manipulate objects concurrently from a language instruction and visual observation. Instead of reconstructing future videos, it learns compact latent foresight embeddings coupled to diffusion-based whole-body action generation, and uses controller-based simulation replay to ground human motion priors into executable action latents. Trained on the new 40+ hour ω-HOME dataset, one model produces smooth manipulate-while-moving behaviors across 11 real-world household tasks, consistently beating imitation learning, VLA, humanoid, and world-action-model baselines.
一张纸劫持机器人:VLM 控制机器人物理提示注入的系统性研究
该研究证明,放置在机器人视野内的对抗性文字可作为针对 VLM 操控的间接提示注入。在 GPT-4o、Gemini 2.5 Flash 与 Qwen3-VL-32B 上进行的 5,670 次试验中,物理标识攻击分别以 27.0%、29.4% 与 5.0% 的成功率得手,其中权威冒充与否定类攻击可跨三种模型迁移。简单的防御手段——提示加固、两阶段验证与文本掩码——可将风险大幅降低(最高达 100%),但各有取舍。
DyPES-VLA:面向跨本体操控的共享动力学先验与本体特定控制学习
DyPES-VLA 是一种跨本体视觉-语言-行动(VLA)框架,将共享的动力学先验与本体特定的控制解耦。未来预测目标使视觉-语言骨干在异构数据上捕获物体运动、接触与交互引起的场景变化;本体特定的混合专家(MoE)动作头则直接在每种机器人的原生动作空间中把这些先验转化为可执行动作,免去手动动作对齐。作为单一通用策略,它在 LIBERO 上达到 98.0% 成功率,在 RoboCasa-GR1 上为 59.25%,在 RoboTwin 2.0 上为 89.02%。
ω-0:面向人形机器人同步移动-操控的隐空间预测世界行动模型
ω-0 是一种全身世界行动模型,使单一人形机器人策略能够根据语言指令和视觉观测,在移动、调整姿态、保持平衡的同时完成物体操控。它不重建未来视频,而是学习紧凑的隐空间未来观测嵌入,并与基于扩散的全身动作生成耦合,再通过基于控制器的仿真回放将人类运动先验落地为可执行的行动隐变量。基于新构建的 40 多小时 ω-HOME 数据集,单一模型即可在 11 项真实家庭任务上生成流畅的边移动边操控行为,持续超越模仿学习、VLA、人形机器人与世界行动模型基线。
PRIMAL3: Pathfinding via Reinforcement and Imitation Multi-Agent Learning — Leveraging LaCAM3
PRIMAL3 is an ultra-large-scale learning-based multi-agent pathfinding framework combining reinforcement learning, topology-aware communication, LaCAM3-guided training, and PIBT-based action refinement. It targets failures at topologically critical states such as bottlenecks and dead ends, scales to city-level instances with up to 100,000 agents, substantially outperforms state-of-the-art learning-based baselines, and is validated on physical robotic systems.
DreamWAM: Beyond RGB Future Prediction for World Action Models
DreamWAM reformulates future prediction in world action models as structured modeling beyond RGB — jointly predicting appearance, motion, geometry, and semantics during training while staying RGB-only at inference. It improves matched RGB-only baselines on LIBERO (97.30% to 98.40%), shows much larger gains under unseen LIBERO-Plus perturbations (51.36% to 63.44%), and lifts real-world success from 55.6% to 74.4% across unseen lighting, background, and layout changes. Code and models are publicly released.
BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation
BridgeVLA++ equips the data-efficient BridgeVLA 3D manipulation framework with a unified spatio-temporal memory that models persistent spatial context and temporal interaction history, enabling policies to reason over observation histories without sacrificing data efficiency or generalization. It achieves state-of-the-art results on two memory-dependent manipulation benchmarks, works in bimanual settings, and is validated on an additional real-world robotic platform.
PRIMAL3:借助 LaCAM3 的大规模多智能体强化与模仿学习路径规划
PRIMAL3 是一个超大规模的学习式多智能体路径规划(MAPF)框架,融合了强化学习、拓扑感知通信、LaCAM3 引导训练与基于 PIBT 的动作精化。它专门针对瓶颈、死胡同、持续冲突等拓扑关键状态下的失败,可扩展至多达 10 万个智能体的城市级实例,显著优于现有最优的学习式基线,并已在真实机器人系统上完成验证。
DreamWAM:超越 RGB 的未来预测——世界动作模型的新范式
DreamWAM 将世界动作模型中的未来预测重构为超越 RGB 的结构化建模——训练时联合预测外观、运动、几何与语义,推理时仍只依赖 RGB。它在 LIBERO 上将匹配的 RGB-only 基线从 97.30% 提升至 98.40%,在未见过的 LIBERO-Plus 扰动下增益更为显著(51.36% 提升至 63.44%),并在真实世界中把平均成功率从 55.6% 提升至 74.4%(面对未见过的光照、背景与物体布局变化)。代码与模型已开源。
BridgeVLA++:具备数据高效、强泛化与记忆增强能力的 3D 操作视觉-语言-动作框架
BridgeVLA++ 在数据高效的 BridgeVLA 三维操作框架之上引入统一的时空记忆架构,同时建模持久的空间上下文与时间交互历史,使策略能够基于观测历史进行推理,且不牺牲数据效率与泛化能力。该方法在两个依赖记忆的操作基准上达到当前最优,支持双臂操作场景,并在另一套真实机器人平台上完成验证。
PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud
PhyAI is a unified Physical AI inference engine with a single runtime for vision-language-action (VLA) and world-action models (WAMs) across onboard, edge, and cloud deployments. Model adapters encapsulate architecture-specific conditioning, solver, cache, and output logic while graph execution, kernels, and memory management are shared, yielding 1.40x–4.65x speedups over the official implementations of pi0, pi0.5, GR00T N1.7, and MiniCPM-Robot, plus a 2.08x latency reduction on Cosmos3-Nano-Policy-DROID. The paper also introduces the control-time Roofline to distinguish inference-bound from environment-bound control.
GORDON: Graph-based Object-centric Rewards for Decomposition of Long-Horizon Manipulation
GORDON learns dense rewards from action-free video demonstrations by representing each scene as a graph of detected objects and spatial relations, then embedding those graphs with a self-supervised graph neural network into a task-aligned latent space. The resulting reward reveals stage-wise object-state transitions in long-horizon tasks, enabling automatic subtask discovery and sequentially composed policies. On seven manipulation tasks across MAGICAL and ManiSkill3 it reaches 74.4% average success on long-horizon tasks, roughly +35 p.p. over the best learned baseline and +25 p.p. over an oracle.
Bimanual Manipulation Within an 8 GB Budget: Zero-Copy Sensing and Quantized ACT on an Entry-Level Jetson
This paper runs a complete bimanual SO-101 pick-and-place system — sensing, policy, and control — on an entry-level Jetson Orin Nano Super (8 GB), using a desktop GPU only for offline training. A GStreamer/NVMM zero-copy pipeline cuts worst-case latency from 117.31 ms to 101.52 ms and peak single-core CPU from 98.0% to 77.0%; TensorRT conversion brings ACT inference from 114.02 ms to 17.93 ms in FP16 (6.4x) and 12.65 ms in INT8 (9.0x) with task success preserved (19/20, 18/20, 19/20). Two undocumented ACT findings: INT8 calibration quantizes the ResNet18 backbone but accepts zero of 145 transformer layers, and quantization need is conditional on action-chunking configuration.
PhyAI:一套运行时打通物理 AI 的边缘实时推理与云端规模化部署
PhyAI 是一个统一的物理 AI 推理引擎,以单一运行时在机载、边缘与云端部署视觉-语言-动作(VLA)模型和世界动作模型(WAM)。模型适配器封装了架构相关的条件计算、求解器、缓存与输出逻辑,而图执行、算子、内存管理与并行服务全部共享,相较 pi0、pi0.5、GR00T N1.7、MiniCPM-Robot 的官方实现取得 1.40–4.65 倍加速,并在 Cosmos3-Nano-Policy-DROID 上实现 2.08 倍延迟降低。论文还提出控制时 Roofline,用于区分推理受限与环境受限的控制回路。
GORDON:基于图的物体中心奖励,实现长时程操作任务的自动分解
GORDON 从无动作的视频示教中学习稠密奖励:将每个视觉场景表示为检测物体及其空间关系构成的图,并用自监督图神经网络把这些图嵌入到与任务对齐的隐空间。该奖励的时间曲线在长时程任务中会显露出阶段式的物体状态迁移,从而支持自动子任务发现与顺序组合策略。在 MAGICAL 与 ManiSkill3 的七个操作任务上,长时程任务平均成功率达 74.4%,较最佳学习基线提升约 35 个百分点、较 oracle 提升约 25 个百分点。
8 GB 预算内的双臂操作:零拷贝感知与量化 ACT 登上入门级 Jetson
本文在入门级 Jetson Orin Nano Super(8 GB)上完整运行了一套双臂 SO-101 抓取-放置系统——感知、策略与控制全部在边缘完成,桌面 GPU 仅用于离线训练。基于 GStreamer/NVMM 的零拷贝流水线将最坏延迟从 117.31 ms 降至 101.52 ms,单核 CPU 峰值占用从 98.0% 降至 77.0%;TensorRT 转换使 ACT 推理延迟由 114.02 ms 降至 FP16 的 17.93 ms(6.4 倍)与 INT8 的 12.65 ms(9.0 倍),任务成功率保持不变(19/20、18/20、19/20)。论文还报告了两项 ACT 此前未被记录的发现:INT8 校准只量化 ResNet18 主干、对 145 个 transformer 层全部拒收;以及量化的必要性取决于动作分块配置。
StableMimic: Smooth Human-Like Recovery for Humanoid Motion Tracking - Learning Beyond the Tracking Distribution for Structured Post-Fall Behavior
StableMimic is a unified humanoid motion-tracking policy trained beyond the nominal tracking distribution so it can recover from falls. Perturbed resets around human get-up references expose prone, supine, and off-balance states; dedicated tracking and recovery experts are blended by a proprioceptive gate, and a hidden successor-state objective shapes human-like recovery without exposing reference identity to the deployed policy. On the retargeted LAFAN1 dance subset it achieves the lowest errors on all four tracking metrics among five methods, recovers 100/100 in matched push-to-fall trials, and is validated on real Unitree G1 robots.
Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control?
This paper rigorously investigates why action chunking improves behavioral cloning. Experiments in simulation and the real world refute the dominant hypotheses (temporal consistency, horizon reduction, representation learning), showing the real benefits come from non-Markovian expressivity, reduced compounding error, and a newly identified effect called implicit ensembling. The authors match action-chunking performance without chunking by deploying policies as ensembles of randomized delays, and propose an explicit-ensemble policy class that significantly outperforms action chunking in many domains.
Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data
Ego2Robot is a scalable pipeline that converts egocentric human manipulation videos into robot training data through action retargeting, robot-arm visual synthesis, and multi-level quality curation, producing 18,561 hours of data across 15 robot morphologies — the largest ego-to-robot dataset to date. Joint pretraining on Ego2Robot-synthesized and robot data consistently improves out-of-distribution generalization across visual appearance, scene layout, embodiment morphology, and task semantics, with benefits validated on real-robot deployment.
StableMimic:人形机器人动作跟踪的平滑类人跌倒恢复——在跟踪分布之外学习结构化倒地后行为
StableMimic 是一个在名义跟踪分布之外训练的统一人形机器人动作跟踪策略,能够在跌倒后自主恢复。围绕多个人类起身参考的扰动重置暴露了俯卧、仰卧与失衡等状态;专用的跟踪与恢复专家由本体感觉门控持续混合,隐藏后继状态目标则在不向部署策略暴露参考身份的前提下塑造类人恢复行为。在重定向后的 LAFAN1 舞蹈子集上,它在五种方法中取得全部四项跟踪指标的最低误差;在匹配的推倒实验中 100/100 次成功恢复,并在真实宇树 G1 机器人上得到验证。
动作分块为何能提升机器人控制中的行为克隆性能?
本文对动作分块(action chunking)为何能提升行为克隆性能进行了严谨剖析。仿真与真实世界实验推翻了主流假说(时间一致性、视界缩短、表征学习),指出真正收益来自更强的非马尔可夫表达能力、更低的误差累积,以及一个新识别的机制——隐式集成(implicit ensembling)。作者在完全不使用动作分块的情况下,通过将策略部署为随机延迟策略的集成,复现了分块策略的性能,并提出一种显式实例化集成的策略类,在多个领域显著超越动作分块。
Ego2Robot:从第一视角人类数据规模化合成机器人数据
Ego2Robot 是一条可扩展的数据合成流水线,通过动作重定向、机械臂视觉合成与多级质量筛选,将第一视角人类操作视频转化为机器人训练数据,产出横跨 15 种机器人形态、共计 18,561 小时的训练数据——这是迄今规模最大的 ego-to-robot 数据集。在合成数据与真实机器人数据上的联合预训练,能在视觉外观、场景布局、本体形态与任务语义等多个扰动维度上持续提升分布外泛化能力,并在真实机器人部署中得到验证。
CLIFT: Turning Gemini Robotics On-Device into Humanoid Specialists via Non-Invasive Closed-Loop Iterative Fine-Tuning
CLIFT shows how closed-weight robot foundation models can be specialized for agile, contact-rich humanoid tasks through managed supervised fine-tuning APIs. By converting deployment-time reward feedback into API-compatible supervised data, CLIFT pushes Gemini Robotics On-Device to near-perfect success after two flywheel cycles — without ever accessing weights, gradients, or losses.
BWM: A Low-Cost High-Fidelity World Simulator for Robot Learning
BWM is an open-source, low-cost, action-conditioned world model that predicts future observations for robot manipulation, serving both as a data engine that augments imitation-learning data with action-aligned rollouts and as a policy evaluator for closed-loop assessment, risk anticipation, and policy ranking. It ranks first overall in the WorldArena Challenge across Track 1 and two Track 2 applications, with checkpoints, code, and interfaces released.
WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning
WCM replaces scalar-return critics in VLA reinforcement-learning post-training with a lightweight LeJEPA-based World Critic that jointly predicts future latent states and estimates values, giving the critic explicit temporal structure. Across 149 tasks on four benchmarks and seven real-world manipulation tasks, WCM delivers state-of-the-art in-distribution and out-of-distribution performance on Pi0, Pi0.5, and OpenVLA-OFT.
CLIFT:通过非侵入式闭环迭代微调,将 Gemini Robotics On-Device 打造成人形机器人专家
CLIFT 展示了如何通过受管监督微调(SFT)API,将闭源权重的人形机器人基础模型定制为敏捷、接触密集任务的专家。通过把部署期的奖励反馈转化为 API 兼容的监督数据,CLIFT 在两个飞轮周期内将 Gemini Robotics On-Device 推向近乎完美的成功率——全程无需接触权重、梯度或损失。
BWM:面向机器人学习的低成本高保真世界模拟器
BWM 是一个开源、低成本、动作条件的世界模型,可预测机器人操作任务的未来观测,既可作为数据引擎,用动作对齐的 rollout 扩充模仿学习数据,也可作为策略评估器,用于闭环评估、风险预判与策略排序。它在 WorldArena 挑战赛中 Track 1 与两个 Track 2 应用中总排名第一,并已开源模型权重、训练与推理代码及接口。
WCM:面向视觉-语言-动作模型强化学习的 World Critic 模型
WCM 用基于轻量级 LeJEPA 架构的 World Critic 取代 VLA 强化学习后训练中的标量回报评论家,在估计价值的同时联合预测未来潜在状态,为评论家赋予显式的时间结构。在四个基准的 149 个任务与 7 个真实操作任务上,WCM 在 Pi0、Pi0.5 与 OpenVLA-OFT 上均取得分布内与分布外的最优性能。
TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction
TacWAM is a world action model that predicts future tactile states — appearance, dense force fields, and deformation flow — to supervise contact-rich manipulation learning, reaching 75.0% average success across four real-world tasks and outperforming the strongest baseline by 37.5 percentage points.
FasTac: A Curved Multispectral Vision-Based Tactile Sensor for High-Speed High-Precision 3D Shape and Force Perception
FasTac is a curved vision-based tactile sensor combining multispectral photometric stereo, dynamic-convolution force estimation, and FPGA acceleration in a compact form. It reduces depth error to 0.0415mm MAE, estimates three-axis forces with ~2.4-2.7% NMAE, and cuts processing latency to 1.09ms on FPGA.
PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball
PAC-MAN couples control-barrier-function safety with deployment-realistic onboard sensing for whole-body humanoid evasion. A policy trained with training-time CBF guidance and a head-mounted camera only achieves near-oracle dodge performance, and zero-shot deployment on the Unitree G1 succeeds on 95% of real-world throws.
TacWAM:基于锚点引导与力学感知触觉预测的世界动作模型
TacWAM 是一个预测未来触觉状态(外观、密集力场与形变流)来监督接触丰富操作学习的世界动作模型,在四个真实世界任务上平均成功率达 75.0%,比最强基线高出 37.5 个百分点。
FasTac:面向高速高精度三维形貌与力感知的曲面多光谱视觉触觉传感器
FasTac 是一款曲面视觉触觉传感器,将多光谱光度立体、动态卷积力估计与 FPGA 硬件加速集成于紧凑形态。深度误差降至 0.0415mm MAE,三轴力估计 NMAE 约 2.4%-2.7%,FPGA 部署将处理延迟缩短至 1.09ms。
PAC-MAN:面向全身安全的人形机器人躲避球感知感知 CBF-RL 框架
PAC-MAN 将控制屏障函数(CBF)安全机制与贴近部署的真实机载感知相结合,用于人形机器人全身闪避。仅凭头戴相机与训练期 CBF 引导训练的策略即可达到接近特权状态oracle的闪避表现,零样本部署到宇树 G1 后在真实世界投掷中成功率达 95%。
SymmGrid: Super-Scaling On-Robot Learning with Parallelized Symmetries and Egocentric-Exocentric Visual Perception
SymmGrid uses parallelized symmetry transformations to dramatically accelerate on-robot reinforcement learning. Real-world manipulation tasks achieve 1.37-2.17× wall-clock training speedup, with convergence in as little as 10.9-16.6 minutes for peg-insertion and cable routing, and up to 2.59× improvement in normalized AUC.
HumanCLAW: Can Vision-Language Models Act Through a Body?
HumanCLAW introduces an evaluation framework that decouples action decision-making from motor execution for embodied AI, benchmarking 9 state-of-the-art VLMs across 1,218 egocentric find-navigate-interact episodes in 41 scenes. The best model achieves only 16.8% — revealing that current VLMs fundamentally lack embodied self-awareness.
TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM
TurboVLA reformulates the conventional V→L→A VLA paradigm into a direct V+L→A mapping, eliminating the LLM as the central bottleneck. With only 0.2B parameters, it achieves 97.7% average success on LIBERO at 32 Hz inference with just 0.9 GB VRAM on an RTX 4090, matching or outperforming much larger VLA policies.
SymmGrid:利用并行化对称性与第一人称-第三人称视觉感知加速机器人真实学习
SymmGrid 利用并行化对称变换大幅加速机器人真实强化学习。在真实机器人操作接触任务上实现 1.37-2.17 倍的训练加速(壁钟时间),插销插入和缆线布设在 10.9-16.6 分钟内收敛,归一化 AUC 提升最高达 2.59 倍。
HumanCLAW:视觉语言模型能否通过身体行动?
HumanCLAW 提出了一种将动作决策与运动执行解耦的具身 AI 评估框架,在 41 个场景中的 1,218 个第一人称“寻找-导航-交互”长程任务上评估了 9 个最先进的 VLM。最佳模型仅达 16.8%——揭示了当前 VLM 从根本上缺乏具身自我意识。
TurboVLA: 在RTX 4090上以32Hz、小于1GB显存运行的实时视觉-语言-动作模型
TurboVLA 将传统的 V→L→A 范式重构为直接的 V+L→A 映射,消除了大语言模型作为中心瓶颈。仅 0.2B 参数,在 LIBERO 上达到 97.7% 平均成功率,RTX 4090 上以 32Hz 推理,仅需 0.9GB 显存,匹配或超越远超其规模的 VLA 策略。
HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone
HiFi-UMI raises the fidelity of robot-free UMI data collection to 3 mm end-effector accuracy using head-mounted stereo-inertial SLAM, native relative pose estimation, and wide-angle stereo cameras. Policies post-trained solely on this data match in-domain teleoperation across three backbone families, and a 4000-hour pretraining corpus lowers action error on unseen tasks by 41%.
Pictura: Perspective-View Self-Play at Scale for Driving
Pictura is a GPU-accelerated multi-agent driving simulator providing egocentric perspective rendering at every step, enabling the first large-scale driving self-play policy trained directly from perspective images without privileged observations. The resulting Alberti policy approaches privileged vectorized performance and transfers zero-shot to Waymo layouts.
$\pi\mathbf{R}^2$: Reactive Real-time Flow Policies
πR² makes large-backbone action-chunking flow policies reactive and real-time by splitting conditioning into a fast proprioception channel (fresh every tick) and an asynchronously updated slow vision-language channel, combined with a latency-adaptive flow schedule. Applied to GR00T-N1.7, it replans at ~25Hz on an A5000, improving success by up to 30% in real-world tasks.
HiFi-UMI: 仅使用高保真UMI数据训练可部署的机器人操作策略
HiFi-UMI 通过头戴式立体惯性SLAM、原生相对位姿估计和超广角立体相机,将机器人无需遥操作干预的UMI数据采集精度提升至末端执行器3毫米。仅基于此数据微调的策略在三个骨干网络家族上与同场景遥操作基线性能持平,4000小时预训练语料将未见任务的动作误差降低41%。
Pictura: 基于第一视角的大规模自博弈自动驾驶策略学习
Pictura 是一个 GPU 加速的多智能体驾驶模拟器,在每个时间步渲染每个智能体的自我中心视角,首次实现了无需特权观测、直接基于第一视角图像的大规模驾驶自博弈策略训练。训练出的 Alberti 策略性能接近使用特权向量化观测的对应策略,并能零样本迁移至 Waymo 数据集。
$\pi\mathbf{R}^2$: 具有反应能力与实时性的流策略
πR² 通过将条件输入拆分为快速本体感知通道(每控制周期更新)和异步更新的慢速视觉语言通道,并结合延迟自适应流调度,使基于大骨干网络的动作分块流策略获得反应能力和实时性。应用于 GR00T-N1.7 后,在 A5000 GPU 上以约25Hz频率重规划,真实场景任务成功率最高提升30%。
Learning Reusable Hybrid Motion Priors for Humanoid Locomotion from Motion Imitation
A three-stage pipeline that turns motion-imitation skills into a reusable Hybrid Motion Prior (HMP) for humanoid locomotion. An expert policy learns to imitate retargeted human motion-capture clips, is distilled into a frozen RVQ codebook architecture, and task-level policies then solve locomotion tasks by selecting discrete codebook entries. The velocity-tracking policy is deployed on a real Unitree G1 robot.
FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning
A hierarchical visuo-tactile world model that jointly predicts future visual latents and three tactile states (contact state, 3D tactile latent encoding force info, slip state). Uses contact-gated asymmetric attention to maintain visual-only prediction before contact and enable joint visuo-tactile prediction during contact. Achieves LPIPS 61% lower than visual baseline after 80-step autoregressive rollout and 81.7% zero-shot planning success.
Data Pyramid for Embodied Manipulation
A systematic framework organizing the embodied data ecosystem into a five-tier pyramid: real-robot data, UMI-style data, egocentric and exocentric data, simulation data, and general vision-language data. Analyzes recent embodied foundation models through the lens of their data recipes, examining how different sources are selected, aligned, and mixed during pretraining.
Learning Reusable Hybrid Motion Priors for Humanoid Locomotion from Motion Imitation: 从运动模仿学习可复用的混合运动先验用于人形机器人 locomotion
提出三阶段流水线将运动模仿技能转化为可复用的混合运动先验(HMP)。先训练专家策略模仿人类动作捕捉数据,再蒸馏为冻结的RVQ码本架构,最后训练任务级策略通过选择离散码本条目实现 locomotion。速度追踪策略成功部署到真实Unitree G1机器人上。码本结构呈现可解释的层次化步态模式。
FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning: 视觉-触觉世界模型用于层次化接触预测与规划
提出层次化视觉-触觉世界模型FeelWorld,联合预测未来视觉潜在特征与三种触觉状态(接触、三维触觉潜在编码、滑动)。引入接触门控非对称注意力机制,在自由空间运动中保持纯视觉预测路径,接触后激活触觉预测。在芯片抓取、水果抓取和USB插入任务上,80步自回归滚动后LPIPS比纯视觉基线低61%,零样本规划成功率81.7%。
Data Pyramid for Embodied Manipulation: 具身操作的数据金字塔
本文系统性地梳理了具身智能体的数据生态系统,将其组织为五层数据金字塔:真实机器人数据、UMI风格数据、自我中心与外中心数据、仿真数据、通用视觉语言数据。围绕可扩展性与机器人对齐之间的张力展开分析,并从数据配方角度审视了近期具身基础模型的能力来源。
One Hand Watches The Other: Dynamic Multi-Agent Cooperation for Sample-Efficient Bimanual Manipulation in Dynamic Environments
DynaMAC resolves the causal limitation of multi-stream policies in dynamic settings by treating the opposite arm as a dynamic task parameter, enabling unified dynamic manipulation and bimanual coordination without an explicit leader-follower relationship. It outperforms leading baselines by 35+ percentage points while requiring 20× fewer samples, and generalizes zero-shot from static demonstrations to dynamic environments.
ViTacWorld: Scaling Visuo-Tactile World Models for Contact-Rich Robot Manipulation
ViTacWorld is the first visuo-tactile world model framework for contact-rich manipulation. It leverages public real tactile datasets and simulated environments to pretrain visuo-tactile-action trajectories, exploiting the fact that tactile signals exhibit a smaller sim-to-real gap than visual observations. It serves both as a data augmentation engine and a policy evaluation tool, generating physically meaningful rollouts that improve downstream tactile policies.
DynaMAC:面向动态环境的高样本效率双臂协作操作
DynaMAC 通过将对手臂视为动态任务参数,解决了多流策略在动态环境中的因果限制,为动态操作和双臂协调提供了统一表述,无需显式的主从关系。它在领先基线上提升 35+ 个百分点,同时所需样本减少 20 倍,并能从静态演示零样本泛化到动态环境。
ViTacWorld:面向接触丰富机器人操作的视触觉世界模型规模化
ViTacWorld 是首个面向接触丰富操作的视触觉世界模型框架。它利用公开真实触觉数据集和仿真环境预训练视触觉-动作轨迹,利用触觉信号比视觉观测具有更小的仿真-现实差距这一特性,同时作为数据增强引擎和策略评估工具,生成物理上有意义的轨迹以提升下游触觉策略性能。
RL-MACRO: A Cybernetic Closed-Loop Intelligence Framework for Multimodal Adaptive Robotic Craniotomy
RL-MACRO is a closed-loop cybernetic framework for autonomous robotic craniotomy that couples multimodal perception (force and sound), adaptive decision-making via offline Implicit Q-Learning (IQL), and robotic execution. A CNN-LSTM observer reconstructs the hidden temperature state from partial sensory feedback (R²=0.939, MAE=1.717°C). A dual-head Actor dynamically coordinates feed rate, spindle speed, and cutting depth. Experiments on bovine ribs and ex vivo goat skulls validate robust perception, adaptive recovery from force/temperature excursions, and smooth execution on irregular surfaces.
AXIS: A Growable Community-Driven Data Engine for Scalable Robot Manipulation
AXIS is a scalable community-driven data engine and benchmark for robot manipulation that enables browser-based teleoperation, automatically generates and validates new manipulation tasks, and transforms community-collected demonstrations into training-ready data. The AXIS dataset contains 207 diverse tasks and 50K+ trajectories. Continual pretraining on AXIS improves π0.5 by 5.8% overall, outperforms RoboCasa365 by 37.3%, and exhibits consistent scaling with increasing data volume.
Addressing the Orchestration Gap in Generalist Robots via Physical Agency
Pigey introduces a closed-loop physical agent orchestrator that decomposes goals into subgoals, commands low-level VLA policies, tracks outcomes, and recovers from failures — all without additional data or fine-tuning. It achieves 4x SOTA on LIBERO-PRO (12.8%→53.3%) and lifts frozen policies from near-zero to 90% on real-world reasoning-limited tasks, demonstrating the orchestration gap between motor skills and agentic reasoning.
RL-MACRO:面向多模态自适应机器人颅骨切除的控制论闭环智能框架
RL-MACRO是一个用于自主机器人颅骨切除的闭环控制论框架,整合了多模态感知(力和声音)、基于离线隐式Q学习(IQL)的自适应决策和机器人执行。CNN-LSTM观测器从部分感知反馈中重建隐藏温度状态(R²=0.939,MAE=1.717°C)。双头Actor动态协调进给速度、主轴转速和切割深度。在牛肋骨和离体山羊头骨上的实验验证了鲁棒的感知能力、对力/温度异常的适应恢复能力以及在不规则表面上的平滑执行。
AXIS:面向可扩展机器人操作的可生长社区驱动数据引擎
AXIS是一个可扩展的社区驱动机器人操作数据引擎和基准,支持基于浏览器的遥操作、自动生成并验证新操作任务、以及将社区收集的演示数据转化为训练就绪数据。AXIS数据集包含207个多样化任务和5万+条轨迹。在AXIS上持续预训练使π0.5的整体成功率提升5.8%,超越在RoboCasa365上预训练的模型37.3%,并随数据量增长呈现一致的扩展特性。
Pigey:通过物理智能体编排弥合通用机器人的编排鸿沟
Pigey 提出了一种闭环物理智能体编排器,能够将高层目标分解为子目标、调用低层 VLA 策略、跟踪执行结果并从失败中恢复——整个过程无需额外数据采集或微调。在 LIBERO-PRO 上达到 SOTA 的 4 倍(12.8%→53.3%),在真实机器人推理受限任务中将冻结策略从近乎零提升至 90% 的成功率。
Diffusion ReRoll: Revisable Denoising for Robotic Sequential Prediction
Diffusion ReRoll introduces structured re-noising for revisable denoising over prediction horizons, enabling earlier and later segments of a sequence to revise one another. It achieves relative gains of 21% over Diffusion Forcing in guided planning on OGBench, 56.5% relative improvement over Diffusion Policy on LIBERO-10, and best action-video consistency in unified video-action prediction.
HOST: Robots Acquire Manipulation Skills in Seconds from a Single Human Video
HOST enables a robot to acquire novel manipulation skills in an average of 29 seconds from a single human video, while retaining previously mastered skills. It achieves 62% average success rate across tasks — exceeding zero-shot baselines by 45% and even surpassing models fine-tuned on 50 robot demonstrations per task, requiring 50x fewer demonstrations and acquiring each skill 507x faster.
DEED: A Data-Efficient Post-Training and Experience-Driven Learning VLA Framework for Retail Humanoids
DEED is a systems-level approach to bridging the lab-to-store gap for humanoid VLA robots, validated on a supermarket chip-restocking task with a Unitree G1-Edu. It demonstrates that careful data design and targeted post-training with a single GPU can transform a policy that fails under naive fine-tuning into a competent real-world system, suggesting the lab-to-store gap is primarily a systems integration challenge.
Robostral Navigate: Monocular RGB-Only Visual Navigation with an 8B VLM
Robostral Navigate is an 8B vision-language model for visual navigation that consumes only monocular RGB images — the most ubiquitous sensor across robotic platforms — and predicts waypoints by pointing to the next target in the current camera view. Operating purely in image space makes it robust to changes in camera intrinsics and scene scale, enabling zero-recalibration deployment across wheeled, legged, and aerial robots. It sets a new state of the art on R2R-CE (77.4% success rate) and RxR-CE (75.1%), surpassing the best monocular methods by 10.5 points and even depth- or multi-camera systems by 5.3 points.
Diffusion ReRoll:面向机器人序列预测的可修正去噪方法
Diffusion ReRoll提出结构化重噪声方法,使预测序列中的早段和晚段能够相互修正。在OGBench上,它比Diffusion Forcing的引导规划成功率相对提高21%;在LIBERO-10上,比Diffusion Policy的动作预测相对改进56.5%;在统一视频-动作预测中取得了最佳的动作-视频一致性。
DEED:面向零售人形机器人的数据高效后训练与经验驱动学习VLA框架
DEED是一个弥合人形机器人VLA系统从实验室到商店部署鸿沟的系统级框架,在超市货架补货任务中基于Unitree G1-Edu和GR00T N1.6进行了验证。它表明,通过精心设计数据和使用单GPU进行定向后训练,可以将朴素微调下失败的策略转化为可在真实世界中运行的系统,证明实验室到商店的鸿沟主要是一个系统集成挑战。
Robostral Navigate:基于8B视觉语言模型的单目RGB纯视觉导航
Robostral Navigate是一个8B参数的视觉语言导航模型,仅需单目RGB图像——这是所有机器人平台上最普遍的传感器——即可通过在当前视野中指向下一个目标位置来预测路点。纯图像空间操作使其对相机内参和场景尺度变化具有自然鲁棒性,可在轮式、足式和空中机器人之间零校准部署。在R2R-CE上达到77.4%的成功率,在RxR-CE上达到75.1%,超越最佳单目方法10.5个百分点,甚至超过基于深度或多相机的系统5.3个百分点。
HOST:机器人从单段人类视频中秒级习得操控技能
HOST使机器人能够从单段人类视频中平均29秒习得新的操控技能,同时保留已掌握技能。它在各任务上达到62%的平均成功率——超过零样本基线45%,甚至超过每个任务用50段机器人演示微调的基线,且所需演示量少50倍,技能习得快507倍。
Eversion-Growing Robots Enable Safe Navigation and Endoscopic Imaging Within the Spinal Subarachnoid Space
This paper presents a 2 mm diameter eversion-growing robotic platform for low-friction navigation within the human spinal subarachnoid space. Validated through computational modeling, phantom experiments, and intact human cadaver studies, the robot achieved 65.2% reduction in mean interaction force and 48.0% reduction in peak interaction force compared to push-based insertion, with 150 mm of controlled intrathecal extension and no observable dural or neural damage.
Masked Visual Actions: A Pixel-Space Control Interface for Unified World Modeling
This paper introduces Masked Visual Actions, a pixel-space control interface that expresses robot actions as partially revealed trajectories of an arbitrary entity in a video. Finetuned with only 15 hours of masked examples, a single checkpoint achieves strong visual fidelity, forward dynamics prediction, inverse modeling, and model-based planning across diverse scenes and multiple embodiments.
Koopman DCM: Unstable Eigenfunctions as Data-Driven Representations for Legged Balancing
This paper generalizes Divergent Components of Motion as Koopman eigenfunctions, learning unstable eigenpairs from only 1 hour of real robot data. On a real biped, the learned Koopman DCMs improve reference walking pattern tracking and provide state-based viability constraints when combined with model predictive control.
Koopman DCM:基于不稳定本征函数的双足平衡数据驱动表示
本文首次将运动发散分量(DCM)推广为Koopman本征函数,利用仅1小时的真实机器人数据学习不稳定特征对。在真实双足机器人上,学习得到的Koopman DCM改善了参考步态跟踪性能,并与模型预测控制结合提供基于状态的生存性约束。
基于外翻生长机器人的脊髓蛛网膜下腔安全介入与内镜成像
本文报道了一种直径2毫米的外翻生长机器人平台,能够在人体脊髓蛛网膜下腔实现低摩擦推进和转向。通过尸体实验验证,该机器人在150毫米鞘内延伸过程中,平均交互力降低65.2%,峰值力降低48.0%,且未对硬脊膜或神经结构造成可观察的宏观损伤。
掩膜视觉动作:面向统一世界建模的像素空间控制接口
本文提出掩膜视觉动作(Masked Visual Actions),一种将机器人动作表示为视频中部分揭示轨迹的像素空间控制接口。仅用15小时掩膜示例微调,单个检查点即可在多样场景和多种机器人形态上实现高保真前向动力学预测和逆建模,并支持基于排名的模型预测规划和策略评估。
Manifold-Guided Motion Planning for Tight Assemblies
CMG-RRT proposes critical-manifold guided sampling-based planning, achieving 100% success on all tight assembly benchmarks including the first automatic solution of the Elk disentanglement puzzle, with a probabilistic completeness proof.
RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model
RynnBrain 1.1 introduces an embodied foundation model family spanning 2B to 122B parameters with contact-point prediction and native 3D grounding. It outperforms all proprietary and open-source models on VSI-Bench, MMSI, and RefSpatial-Bench, and demonstrates cross-embodiment VLA deployment on Unitree G1, Astribot-S1, and Tianji-Wuji robots.
Importance Sampling and PCA for Finding Failures in Commercial Autonomous Vehicles
This paper applies Adaptive Stress Testing (AST) and Diffusion-based Failure Sampling (DiFS) to a commercial autonomous trucking stack, combined with PCA-based statistical analysis for failure mode classification. Both algorithms discover simulated collisions in merge and cut-in maneuvers that traditional Monte Carlo simulation cannot reach.
流形引导的精密装配运动规划
CMG-RRT 提出基于临界流形引导的采样规划方法,在精密装配任务中实现 100% 成功率,首次自动求解 Elk 益智拆卸谜题,并提供概率完备性证明。
RynnBrain 1.1:迈向更强大、更通用的具身基础模型
RynnBrain 1.1 发布涵盖 2B 至 122B 参数的具身基础模型家族,新增接触点预测与原生 3D 定位能力,在 VSI-Bench、MMSI、RefSpatial-Bench 等基准上超越所有开源及商业模型,并在 Unitree G1、Astribot-S1、天玑无极三款机器人上完成跨本体 VLA 部署验证。
重要性采样与主成分分析在商用车自动驾驶故障发现中的应用
本文将适应性压力测试(AST)和扩散故障采样(DiFS)应用于真实商用自动驾驶卡车栈,结合PCA分析对故障模式进行分类与诊断,在汇入和切入场景中发现了传统蒙特卡洛仿真无法触及的罕见碰撞故障。
Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning
Open-AoE is a large-scale egocentric manipulation dataset with roughly 2,000 hours of real-world video, structured annotations, and an end-to-end toolchain designed to lower the data barrier for embodied AI research.
AC-VLA: Robust Out-of-Distribution Action Execution via Compositional Learning
AC-VLA is a plug-and-play compositional learning framework that addresses trajectory overfitting and perceptual shortcuts in VLA models, achieving ~28% absolute improvement on compositional OOD tasks while maintaining near-perfect in-distribution performance.
Orbis 2: A Hierarchical World Model for Driving
Orbis 2 introduces a hierarchical driving world model with a two-stage training paradigm combining diffusion forcing and teacher forcing, achieving SOTA results in long-horizon generation fidelity, steering responsiveness, and internal representation quality.
Handroid: Bridging Dexterous Hand and Humanoid
Handroid is a compact 27-DoF reconfigurable robot that functions as either a dexterous anthropomorphic hand or a 0.33m desktop humanoid, validated on manipulation, locomotion, and long-horizon reconfiguration tasks.
Open-AoE:面向具身学习的开源自我中心操作数据集与工具链
Open-AoE 是一个大规模的自我中心操作数据集,包含约2000小时的真实环境操作视频,提供完整的标注和工具链支持,降低具身智能研究的数据门槛。
AC-VLA:通过组合学习实现鲁棒的分布外动作执行
AC-VLA 是一个即插即用的组合学习框架,解决了 VLA 模型中的轨迹过拟合和感知捷径问题,在组合分布外任务上实现约 28% 的绝对提升,同时保持近乎完美的分布内性能。
Orbis 2:面向驾驶的分层世界模型
Orbis 2 提出了一种分层驾驶世界模型,采用扩散强迫预训练与教师强迫微调相结合的两阶段训练范式,在长时序生成质量、转向响应能力和内部表征质量上达到当前最优水平。
Handroid:灵巧手与人形机器人的桥梁
Handroid 是一款紧凑型 27 自由度可重构机器人,能够在一套硬件上分别充当灵巧的仿人手掌或 0.33 米高的桌面人形机器人,并在操作、运动规划与长时序重构任务上得到实验验证。
Reflex: Real-Time VLA Control through Streaming Inference
Reflex enables real-time streaming inference for flow-matching Vision-Language-Action (VLA) policies by exploiting the timestep-invariance property, achieving 2.58x speedup and 50Hz stable streaming on standard benchmarks.
BridgeFlow: Fast and Robust SE(2)-Equivariant Motion Planning with Flow Matching
BridgeFlow achieves fast SE(2)-equivariant motion planning via lightweight task-centric canonicalization paired with flow matching, delivering up to 15x inference speedup and 2x higher valid trajectory rates over diffusion baselines.
AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight
AeroAct is the first world-action model (WAM) demonstrated on a real-world quadrotor, adapting a pretrained video diffusion Transformer to predict trajectory-action chunks from egocentric vision, proprioception, and natural language for closed-loop aerial navigation.
DriftWorld: Fast World Modeling through Drifting
DriftWorld replaces diffusion with drifting generative models for single-step world model inference at 30+ FPS — 17x faster than diffusion baselines — achieving state-of-the-art decision-making across multiple robotic manipulation benchmarks.
Scaling Behavior Foundation Model for Humanoid Robots
A systematic scaling recipe for Behavior Foundation Models in humanoid control, achieving 10-82% MPKPE reduction through motion tracking paradigm, strategic data synergy, and the Humanoid Transformer architecture.
Stigmergic Graph Memory: An Environment-Aware Approach for Many-to-Many Multi-Agent Pickup and Delivery
Stigmergic Graph Memory (SGM) records recent execution signals on warehouse nodes and edges to rank feasible endpoints and route preferences, achieving 20.5-36.7% throughput gains across 15 map-load conditions without altering collision constraints or planner validity.
RoboTTT: Context Scaling for Robot Policies — A New Scaling Axis for Foundation Models
NVIDIA and Stanford researchers propose RoboTTT, scaling visuomotor context to 8K timesteps — three orders of magnitude beyond prior work — enabling one-shot in-context imitation from human video, on-the-fly policy improvement, and full completion of a five-minute, ten-stage assembly task with 87% improvement over single-step baselines.
Reflex:基于流式推理的实时VLA控制
Reflex通过利用时间步不变性(timestep-invariance)属性,实现了流匹配VLA策略的实时流式推理,在标准基准上达到2.58倍加速和50Hz稳定流式输出。
BridgeFlow:基于流匹配的快速鲁棒SE(2)等变运动规划
BridgeFlow通过轻量级任务中心规范化与流匹配相结合,实现了快速的SE(2)等变运动规划,在扩散基线方法上取得高达15倍推理加速和2倍有效轨迹率提升。
AeroAct:以动作为中心的世界动作模型用于语言条件四旋翼飞行
AeroAct是首个在实际四旋翼无人机上验证的世界动作模型(WAM),通过适配预训练的视频扩散Transformer,从第一视角视觉、本体感知和自然语言中预测轨迹-动作片段,实现闭环空中导航。
DriftWorld:漂移生成模型驱动的快速世界建模
DriftWorld利用漂移生成模型替代扩散模型,实现30+ FPS的单步世界模型推理,速度提升17倍,并在多个机器人操作基准上取得最先进决策性能。
扩展行为基础模型:迈向通用人形机器人控制
本文提出扩展行为基础模型(BFM)的系统性框架,通过运动跟踪学习范式、策略-数据协同和Humanoid Transformer架构,在仿真和真实场景中将人形机器人控制精度提升10%至82%。
Stigmergic Graph Memory:基于执行记忆的多对多多智能体货物搬运
本文提出Stigmergic Graph Memory (SGM),一种有界衰退记忆层,通过记录仓库节点和边的近期执行信号来优化端点选择和路由偏好,在15种地图负载条件下实现20.5-36.7%的吞吐量提升。
RoboTTT:上下文长度缩放——机器人策略模型的新维度
NVIDIA联合斯坦福大学团队提出RoboTTT,首次将机器人视觉-语言-动作策略的上下文窗口扩展至8000时间步,实现从人类视频演示中一次性模仿学习、在线策略改进以及长时序复杂装配任务的完全自主完成,性能较单步基线提升87%。
基于V2X的机器人多机协同系统:迈向智能交通互联新范式
基于视觉语言模型的DexSim2Real:实现灵巧操作的零样本虚实迁移
基于V2X技术的社会机器人多机协作架构
触觉引导的精密装配:突破亚毫米级插入难题
DexSim2Real:利用基础模型实现通用灵巧操作的虚实迁移
触觉增强:迈向亚毫米精度精密装配的新范式
用于 ISAC 支持的机器人避障的面向目标的语义通信
最新机器人研究突破:Goal-Oriented Semantic Communication for ISAC-Enab...
多智能体AI赋能自适应AR机器人训练:超越静态指令
界面感知轨迹重构:让受限演示“进化”为高效机器人策略
OmniTrack:基于物理一致性引导的人形机器人通用运动追踪框架
OmniTrack:基于物理一致性引导的人形机器人通用运动追踪框架
SODA-CitrON:基于在线聚类的多模态传感器静态目标数据关联新算法
意图重构:让低维交互界面实现高维机器人精准控制
突破限制:面向低维控制接口的辅助机器人轨迹重构算法
打破接口枷锁:基于接口感知轨迹重构的辅助机器人高效学习
ZEST:波士顿动力重磅发布!人形机器人零样本动作迁移新突破
ZEST:波士顿动力重磅发布!人形机器人零样本动作迁移新突破...
SPARR框架:打破“仿真到现实”壁垒,实现无监督高精度机器人组装
SPARR框架:打破“仿真到现实”壁垒,实现无监督高精度机器人组装...
从受限操控到自由灵动:界面感知轨迹重构助力残障人士操控高自由度机器人
从受限操控到自由灵动:界面感知轨迹重构助力残障人士操控高自由度机器人...
SODA-CitrON:基于在线聚类的多模态异构传感器静态目标关联新框架
SODA-CitrON:基于在线聚类的多模态异构传感器静态目标关联新框架...
具身智能最新进展:2026年人形机器人论文综述
2026年初人形机器人领域论文综述,涵盖运动控制、感知融合等核心方向。
人形机器人核心执行器 BOM 成本拆解:2025 年供应链全景
基于对 12 家核心供应商的深度访谈,拆解人形机器人关节执行器的完整 BOM 成本结构。