Text input
A soft teddy toy
FROM VISUAL IDEAS TO PHYSICAL INTERACTION
DeformSmithPhysics Harness-Guided Hierarchical Generation
of Deformable Assets for Robot Manipulation
1 Nankai University 2 Rightly Robotics, A4x
MANIPULATION IN THE LOOP
Robot Pick-and-Place in the Generation Loop
From a text prompt or an image to robot interaction. Explore each input alongside the scene replay and deformation detail.
Teddy · text · Original playback speed
Text prompt:
“a soft teddy toy”
Scene
Detail
Qualitative interaction replay. Task acceptance and material consistency are evaluated separately in the paper.
OVERVIEW
Generating Deformable Assets for Robot Manipulation
Text and images tell us what an object looks like. Interaction reveals how it behaves.
DeformSmith constructs deformable assets from text or a single image. A hierarchy of agents builds geometry, establishes physical models, configures material behavior, and plans robot interaction. A shared physics-grounded harness uses simulation probes and manipulation feedback to guide construction and refinement.
The outputs bring together geometry, appearance, physical parameters, validation records, and replayable interaction data in a common asset representation.
THE METHOD
Four layers. One feedback loop.
Progressively construct each part of an asset, then examine its behavior through simulation.
Geometry
Reconstruct and align meshes, particles, and Gaussian appearance.
Physical modeling
Establish mass, volume, contact conditions, and the initial state.
Material behavior
Evaluate material candidates with deformation and recovery probes.
Robot interaction
Use pick-and-place feedback to revise actions and material choices.
QUALITATIVE COMPARISON
From airborne to contact.
More demos coming soon.
Explore falling, contact, and recovery. Inspect DeformSmith in slow motion, or switch to the original comparison.
Loading comparison videos…
REFERENCE
BibTeX
@misc{li2026deformsmith,
title = {{DeformSmith}: Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation},
author = {Can Li and Jie Gu and Zishun Deng and Jingmin Chen and Lei Sun},
year = {2026},
eprint = {2609.18620},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2609.18620}
}