Learning Hierarchical Shape Segmentation and Labeling
from Online Repositories



Li Yi1    Leonidas Guibas1   Aaron Hertzmann2   Vladimir G. Kim2    Hao Su1   Ersin Yumer2  
1Stanford University    2Adobe Research   


Figure 1: Large online model repositories contain abundant additional data beyond 3D geometry, such as part labels and artist's part decompositions, flat or hierarchical. We tap into this trove of sparse and noisy noisy data to train a network for simultaneous hierarchical shape structure decomposition and labeling. Our method learns to take new geometry, and segment it into parts, label the parts, and place them in a hierarchy. In this paper, we visualize scene graphs with a circular visualization, in which the root node is near the center. Blue lines indicate parent-child relationships, and red dashed arcs connect siblings. The input geometry in online databases are broken as connected components, visualized in the input as random colors.



We propose a method for converting geometric shapes into hierarchically segmented parts with part labels. Our key idea is to train category-specific models from the scene graphs and part names that accompany 3D shapes in public repositories. These freely-available annotations represent an enormous, untapped source of information on geometry. However, because the models and corresponding scene graphs are created by a wide range of modelers with different levels of expertise, modeling tools, and objectives, these models have very inconsistent segmentations and hierarchies with sparse and noisy textual tags. Our method involves two analysis steps. First, we perform a joint optimization to simultaneously cluster and label parts in the database while also inferring a canonical tag dictionary and part hierarchy. We then use this labeled data to train a method for hierarchical segmentation and labeling of new 3D shapes. We demonstrate that our method can mine complex information, detecting hierarchies in man-made objects and their constituent parts, obtaining finer scale details than existing alternatives. We also show that, by performing domain transfer using a few supervised examples, our technique outperforms fully-supervised techniques that require hundreds of manually-labeled models.




Figure 2: Hierarchical segmentation results. In each case, the input is a geometric shape. Our method automatically determines the segmentation into parts, the part labels and the hierarchy.





Code and Data





Author = {Li Yi and Leonidas Guibas and Aaron Hertzmann and Vladimir G. Kim and Hao Su and Ersin Yumer},
Journal = {SIGGRAPH},
Title = {Learning Hierarchical Shape Segmentation and Labeling from Online Repositories},
Year = {2017}}