WT-UMI

Whole-Body Tactile UMI for Force-Supervised Humanoid Manipulation

Conference on Robot Learning (CoRL) 2026

Jaehwi Jang*, Zhaoyuan Gu*, Alfred Cueva, Zimeng Chai, Junjie Sheng, Thong Nguyen, Himank Galundia, Yifan Wu, Huishu Xue, Isaac Legene, Ojas Mediratta, Davin Doan, Andrew Collins, Sarah Sadegh, KyoungMok Kim, Rishita Dhalbisoi, Zun Chen, Ye Zhao

* Equal Contribution

Arxiv Paper Code (Coming Soon) Dataset Hardware Guide Hardware Repo

Whole-Body Manipulation

Teleoperation Data Collection

Yoga Ball
Box Moving
Box Reorientation
Babysitting
Box Unloading

Force-supervised planner trained on different data sources

Human Only Teleoperation + Human

Abstract

Whole-body humanoid manipulation of bulky, deformable, and shared-load objects requires distributed contact sensing and explicit force regulation, yet most imitation policies treat contact force only implicitly. On the other hand, different demonstration sources provide complementary modalities with inherent trade-offs: human demonstrations capture natural contact forces but not robot-executable actions, while teleoperation directly records robot actions but with less natural force regulation. This paper presents WT-UMI, a wearable whole-body tactile interface worn by human operators or mounted on humanoids, providing accurate observations of tactile images, contact forces, and end-effector poses across both human demonstration and humanoid teleoperation modes. We introduce a force-conditioned target-pose correction module that converts measured human poses into contact-aware robot targets by learning corrections from teleoperation data. To leverage the natural force interaction in human data, we propose a force-supervised planner that predicts end-effector pose chunks and contact-force trajectories. The predicted contact force serves as the reference for a tactile-based admittance controller. Across five contact-rich tasks spanning deformable objects, bulky rigid objects, and human–humanoid collaboration, WT-UMI improves success rate and reduces contact-position tracking error over four policy baselines.

Framework

framework diagram

Hardware

WT-UMI hardware diagram

Target-Pose Correction

Target-pose correction

Force Prediction Evaluation

Source Force RMSE [N] Lag [ms] Force Rate RMS [N/s]
Measured Predicted
Human 1.05 68 5.86 3.74
Teleoperation 2.07 151 30.62 19.80

Force signal quality. Human demonstrations exhibit more accurate and smoother force profiles than teleoperation, improving the quality of supervision for learning.

Force prediction evaluation

Effect of Data Source

Task Succ. (%) Cont. Drift (mm) Cont. Force (N) Translational Smoothness (m/s2) ↓ Rotational Smoothness (rad/s2) ↓
Data → Tel.Comb. Tel.Comb. Tel.Comb. Tel.Comb. Tel.Comb.
T1 100.0100.0 20.4118.12 3.274.77 4.083.14 22.9320.29
T2 100.0100.0 24.6121.04 0.170.52 6.031.87 34.5412.63
T3 60.080.0 26.0125.00 0.660.96 3.491.85 27.8214.05

Effect of data source over tasks T1–T3. Best values are highlighted in bold.

Results

Policy Task Succ. Rate (%) ↑ Cont. Drift (mm) ↓ Cont. Force (N) Translational Smoothness (m/s2) ↓ Rotational Smoothness (rad/s2) ↓
Admi. (Ours) → w/ow/ w/ow/ w/ow/ w/ow/ w/ow/
ViT-FMT T1 100100 18.1215.67 4.775.50 3.142.98 20.2918.56
T2 100100 21.0419.44 0.520.13 1.871.95 12.6313.61
T3 8092 25.0022.08 0.961.61 1.851.29 14.0510.38
ViT-DiT T1 5252 21.7918.00 2.212.57 2.922.54 18.7314.97
T2 100100 21.2219.61 0.190.76 4.672.41 25.9315.17
T3 5252 26.2222.20 1.740.93 2.332.05 16.4714.79
π0.5 T1 8892 21.5111.78 2.532.80 4.694.82 29.3329.79
T2 6876 19.7815.97 0.500.50 5.065.03 31.8631.22
T3 8476 20.1819.40 2.573.38 4.113.53 27.2623.56
Ψ0 T1 8892 15.8013.56 3.143.03 4.203.27 27.0623.01
T2 8996 20.4818.69 2.502.50 4.945.44 32.1233.04
T3 00 –– –– –– ––

Admittance ablation across all policy backbones over tasks T1 (Yogaball), T2 (Pillow), and T3 (Bucket). Each metric is evaluated without (w/o) and with (w/) our admittance control. The better or tied value is bolded for success rate, contact drift, and motion smoothness.

Success Rate (%) ↑

w/o admittance w/ admittance
100
100
52
52
88
92
88
92
ViT-FMTViT-DiTπ0.5Ψ0
T1 · Yogaball
100
100
100
100
68
76
89
96
ViT-FMTViT-DiTπ0.5Ψ0
T2 · Pillow
80
92
52
52
84
76
0
0
ViT-FMTViT-DiTπ0.5Ψ0
T3 · Bucket

Contact Drift (mm) ↓

w/o admittance w/ admittance
18.12
15.67
21.79
18.00
21.51
11.78
15.80
13.56
ViT-FMTViT-DiTπ0.5Ψ0
T1 · Yogaball
21.04
19.44
21.22
19.61
19.78
15.97
20.48
18.69
ViT-FMTViT-DiTπ0.5Ψ0
T2 · Pillow
25.00
22.08
26.22
22.20
20.18
19.40
n/a
n/a
ViT-FMTViT-DiTπ0.5Ψ0
T3 · Bucket

Contact Force (N)

w/o admittance w/ admittance
4.77
5.50
2.21
2.57
2.53
2.80
3.14
3.03
ViT-FMTViT-DiTπ0.5Ψ0
T1 · Yogaball
0.52
0.13
0.19
0.76
0.50
0.50
2.50
2.50
ViT-FMTViT-DiTπ0.5Ψ0
T2 · Pillow
0.96
1.61
1.74
0.93
2.57
3.38
n/a
n/a
ViT-FMTViT-DiTπ0.5Ψ0
T3 · Bucket

BibTeX

@inproceedings{jang2026wtumi,
  title={{WT-UMI}: Tactile-based Whole-Body Manipulation via Force-Supervised Contact-Aware Planning},
  author={Jang, Jaehwi and Gu, Zhaoyuan and Cueva, Alfred and Chai, Zimeng and Sheng, Junjie and Nguyen, Thong and Galundia, Himank and Wu, Yifan and Xue, Huishu and Legene, Isaac and Mediratta, Ojas and Doan, Davin and Collins, Andrew and Sadegh, Sarah and Kim, KyoungMok and Dhalbisoi, Rishita and Chen, Zun and Zhao, Ye},
  booktitle={Proceedings of the 10th Conference on Robot Learning},
  year={2026}
}