WT-UMI

Whole-Body Tactile UMI for Force-Supervised Humanoid Manipulation

In submission

Arxiv Paper Code (Coming Soon) Dataset Hardware Guide Hardware Repo

Whole-Body Manipulation

Teleoperation Data Collection

Yoga Ball
Box Moving
Box Reorientation
Babysitting
Box Unloading

Force-supervised planner trained on different data sources

Human Only Teleoperation + Human

Abstract

Whole-body humanoid manipulation of bulky, deformable, and shared load objects requires distributed contact sensing and explicit force regulation, yet most imitation policies treat contact force only implicitly. Demonstration sources provide complementary modalities with inherent trade-offs: human demonstra tions capture natural contact forces but not robot-executable actions, while teleop eration directly records robot actions but with less natural force regulation. This paper presents WT-UMI, a wearable whole-body tactile interface worn by human demonstrators or mounted on humanoids, providing accurate observations of tactile images, contact forces, and end-effector poses across both collection modes. We introduce a force-conditioned target-pose correction module that con verts measured human hand trajectories into contact-aware robot targets through learned corrections from measured tactile, pose, and force inputs, and a force supervised planner that predicts end-effector pose chunks and contact-force tra jectories. The predicted contact force serves as the reference for a tactile-based admittance controller. Across five contact-rich tasks spanning deformable objects, bulky rigid objects, and human–humanoid collaboration, WT-UMI improves success rate and reduces contact-position tracking error over four policy baselines.

Framework

framework diagram

Hardware

WT-UMI hardware diagram

Target-Pose Correction

Target-pose correction

Force Prediction Evaluation

Source Force RMSE [N] Lag [ms] Force Rate RMS [N/s]
Measured Predicted
Human 1.05 68 5.86 3.74
Teleoperation 2.07 151 30.62 19.80
Force prediction evaluation

Importance of IK Correction and Admittance Controller

Metric Raw Human Raw Teleoperation Human Correction Human Correction + Admittance
Avg. Success Rate (%) ↑ Failed 85.35 89.29 96.15
Motion Smoothness (m/s2) ↓ — 1.10 1.09 1.18
Contact Drift (mm) ↓ — 18.60 18.69 12.47
Contact Establishment Time (s) ↓ — 1.00 1.08 0.58
Contact Lingering Time (s) ↓ — 0.12 0.10 0.14

Results

Policy Task Succ. Rate (%) ↑ Cont. Drift (mm) ↓ Cont. Force (N) Smooth.-Trans. (m/s2) ↓ Smooth.-Rot. (rad/s2) ↓
Admi. (Ours) → w/ow/ w/ow/ w/ow/ w/ow/ w/ow/
ViT-FMT T1 100100 18.1215.67 4.775.50 3.142.98 20.2918.56
T2 100100 21.0419.44 0.520.13 1.871.95 12.6313.61
T3 8092 25.0022.08 0.961.61 1.851.29 14.0510.38
ViT-DiT T1 5252 21.7918.00 2.212.57 2.922.54 18.7314.97
T2 100100 21.2219.61 0.190.76 4.672.41 25.9315.17
T3 5252 26.2222.20 1.740.93 2.332.05 16.4714.79
π0.5 T1 8892 21.5111.78 2.532.80 4.694.82 29.3329.79
T2 6876 19.7815.97 0.500.50 5.065.03 31.8631.22
T3 8476 20.1819.40 2.573.38 4.113.53 27.2623.56
Ψ0 T1 8892 15.8013.56 3.143.03 4.203.27 27.0623.01
T2 8996 20.4818.69 2.502.50 4.945.44 32.1233.04
T3 00 –– –– –– ––

Admittance ablation across all policy backbones over tasks T1 (Yogaball), T2 (Pillow), and T3 (Bucket). Each metric is evaluated without (w/o) and with (w/) our admittance control. The better or tied value is bolded for success rate, contact drift, and motion smoothness.

Success Rate (%) ↑

w/o admittance w/ admittance
100
100
52
52
88
92
88
92
ViT-FMTViT-DiTπ0.5Ψ0
T1 · Yogaball
100
100
100
100
68
76
89
96
ViT-FMTViT-DiTπ0.5Ψ0
T2 · Pillow
80
92
52
52
84
76
0
0
ViT-FMTViT-DiTπ0.5Ψ0
T3 · Bucket

Contact Drift (mm) ↓

w/o admittance w/ admittance
18.12
15.67
21.79
18.00
21.51
11.78
15.80
13.56
ViT-FMTViT-DiTπ0.5Ψ0
T1 · Yogaball
21.04
19.44
21.22
19.61
19.78
15.97
20.48
18.69
ViT-FMTViT-DiTπ0.5Ψ0
T2 · Pillow
25.00
22.08
26.22
22.20
20.18
19.40
n/a
n/a
ViT-FMTViT-DiTπ0.5Ψ0
T3 · Bucket

Contact Force (N)

w/o admittance w/ admittance
4.77
5.50
2.21
2.57
2.53
2.80
3.14
3.03
ViT-FMTViT-DiTπ0.5Ψ0
T1 · Yogaball
0.52
0.13
0.19
0.76
0.50
0.50
2.50
2.50
ViT-FMTViT-DiTπ0.5Ψ0
T2 · Pillow
0.96
1.61
1.74
0.93
2.57
3.38
n/a
n/a
ViT-FMTViT-DiTπ0.5Ψ0
T3 · Bucket

Translational Smoothness (m/s2) ↓

w/o admittance w/ admittance
3.14
2.98
2.92
2.54
4.69
4.82
4.20
3.27
ViT-FMTViT-DiTπ0.5Ψ0
T1 · Yogaball
1.87
1.95
4.67
2.41
5.06
5.03
4.94
5.44
ViT-FMTViT-DiTπ0.5Ψ0
T2 · Pillow
1.85
1.29
2.33
2.05
4.11
3.53
n/a
n/a
ViT-FMTViT-DiTπ0.5Ψ0
T3 · Bucket

Rotational Smoothness (rad/s2) ↓

w/o admittance w/ admittance
20.29
18.56
18.73
14.97
29.33
29.79
27.06
23.01
ViT-FMTViT-DiTπ0.5Ψ0
T1 · Yogaball
12.63
13.61
25.93
15.17
31.86
31.22
32.12
33.04
ViT-FMTViT-DiTπ0.5Ψ0
T2 · Pillow
14.05
10.38
16.47
14.79
27.26
23.56
n/a
n/a
ViT-FMTViT-DiTπ0.5Ψ0
T3 · Bucket

BibTeX

@article{wtumi2026,
  title={WT-UMI: Whole-Body Tactile UMI for Force-Supervised Humanoid Manipulation},
  author={Anonymous},
  journal={arXiv preprint arXiv:2606.13232},
  year={2026}
}