pipette
ESEspañol

From Sketch to Scene: Relightable Garment Reconstruction from Marked Sketches

Ahmet Ne\c{c}irvan Do\u{g}an, Eren Acar Ba\c{s}aran, Metin Sezgin, Zahra Montazeri

Preprint

In the authors' words

Sketching is one of the most natural ways for people to express visual ideas. Unlike the text prompts that most generative systems rely on today, a sketch captures spatial structure and intent directly, including where a designer wants highlights and shadows to appear. Yet current sketch-to-3D pipelines have no input for that intent: they bake the drawn shading into the texture as dark paint, and the light cannot be moved. We propose a system that transforms a rough garment sketch, marked with sparse annotations (a star for a highlight and a moon for a shadow), and a short text prompt describing the texture, into a textured 3D mesh with an explicit directional light, as a scene that can be relit and edited after generation. The two symbols are the most frequent proposals in an elicitation study with art students. From the marked sketch the system predicts the scene's directional light and, separately, a displacement field that bends the surface so that the moon-marked regions fall into shadow under that light, rather than being painted dark in the texture.

Main resultThe abstract does not state a limitation.

Appeared: Friday, October 16. arXiv. Preprint, not yet peer-reviewed.

Authors' comment: 11 pages, 5 figures. Ahmet Ne\c{c}irvan Do\u{g}an and Eren Acar Ba\c{s}aran contributed equally