Lifting Freehand Concept Sketches into 3D

Yulia Gryaditskaya Univesité Côte d'Azur, Inria, University of Surrey, CVSSP
Felix Hähnlein Univesité Côte d'Azur, Inria
Chenxi Liu University of British Columbia
Alla Sheffer University of British Columbia
Adrien Bousseau Univesité Côte d'Azur, Inria
ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 2020.

[Paper]

Given a raw vector 2D design drawing containing both scaffold and surface curves, our algorithm lifts the drawing into 3D by distinguishing 3D intersections from occlusions. The recovered depth enables rendering the drawing from novel viewpoints.

Abstract

We present the first algorithm capable of automatically lifting real-world, vector-format, industrial design sketches into 3D. Targeting real-world sketches raises numerous challenges due to inaccuracies, use of overdrawn strokes, and construction lines. In particular, while construction lines convey important 3D information, they add significant clutter and introduce multiple accidental 2D intersections. Our algorithm exploits the geometric cues provided by the construction lines and lifts them to 3D by computing their intended 3D intersections and depths. Once lifted to 3D, these lines provide valuable geometric constraints that we leverage to infer the 3D shape of other artist drawn strokes. The core challenge we address is inferring the 3D connectivity of construction and other lines from their 2D projections by separating 2D intersections into 3D intersections and accidental occlusions. We efficiently address this complex combinatorial problem using a dedicated search algorithm that leverages observations about designer drawing preferences, and uses those to explore only the most likely solutions of the 3D intersection detection problem. We demonstrate that our separator outputs are of comparable quality to human annotations, and that the 3D structures we recover enable a range of design editing and visualization applications, including novel view synthesis and 3D-aware scaling of the depicted shape.

Code

The code for the paper: github repo.

Data

BibTex

@Article{Lift3D_SA20,
  author       = "Yulia Gryaditskaya and Felix H\"{a}hnlein and Chenxi Liu and Alla Sheffer and Adrien Bousseau",
  title        = "Lifting Freehand Concept Sketches into 3D",
  journal      = "ACM Transactions on Graphics (Proc. SIGGRAPH Asia)",
  year         = "2020",
  volume       = "39",
  month        = "12",
  publisher    = {ACM}
}