Geometry
Reconstruct and align meshes, particles, and Gaussian appearance.
FROM VISUAL IDEAS TO PHYSICAL INTERACTION
DeformSmithPhysics Harness-Guided Hierarchical Generation
of Deformable Assets for Robot Manipulation
1 Nankai University 2 Rightly Robotics, A4x
MANIPULATION IN THE LOOP
Robot Pick-and-Place in the Generation Loop
More text-to-manipulation demos coming soon.
Scene and close-up views show the robot handling a generated deformable object.
Qualitative interaction replay. Task acceptance and material consistency are evaluated separately in the paper.
OVERVIEW
Generating Deformable Assets for Robot Manipulation
Text and images tell us what an object looks like. Interaction reveals how it behaves.
DeformSmith constructs deformable assets from text or a single image. A hierarchy of agents builds geometry, establishes physical models, configures material behavior, and plans robot interaction. A shared physics-grounded harness uses simulation probes and manipulation feedback to guide construction and refinement.
The outputs bring together geometry, appearance, physical parameters, validation records, and replayable interaction data in a common asset representation.
THE METHOD
Four layers. One feedback loop.
Progressively construct each part of an asset, then examine its behavior through simulation.
Physical modeling
Establish mass, volume, contact conditions, and the initial state.
Material behavior
Evaluate material candidates with deformation and recovery probes.
Robot interaction
Use pick-and-place feedback to revise actions and material choices.
QUALITATIVE COMPARISON
From airborne to contact.
More demos coming soon.
Explore the same object across four generation methods during a five-second gravity drop.
Loading comparison videos…
REFERENCE
BibTeX
@misc{li2026deformsmith,
title = {{DeformSmith}: Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation},
author = {Li, Can and Gu, Jie and Deng, Zishun and Chen, Jingmin and Sun, Lei},
year = {2026},
eprint = {2609.18620},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2609.18620}
}