Abstract

Automated assembly of 3D fractures is essential in orthopedics, archaeology, and our daily life. This paper presents Jigsaw, a novel framework for assembling physically broken 3D objects from multiple pieces. Our approach leverages hierarchical features of global and local geometry to match and align the fracture surfaces. Our framework consists of three major components: (1) surface segmentation to separate fracture and original parts, (2) multi-parts matching to find correspondences among fracture surface points, and (3) robust global alignment to recover the global poses of the pieces.
We show how to jointly learn segmentation and matching and seamlessly integrate feature matching and rigidity constraints. We evaluate Jigsaw on the Breaking Bad dataset and achieve superior performance compared to state-of-the-art methods. Our method also generalizes well to diverse fracture modes, objects, and unseen instances. To the best of our knowledge, this is the first learning-based method designed specifically for 3D fracture assembly over multiple pieces.

Learning Framework

Overall pipeline for Jigsaw. The method consists of four parts: front-end feature extractor, surface segmentation, multi-part matching, and global fracture alignment. In the front-end feature extraction, we use a multi-scale grouping PointNet++, and one self-attention layer followed by one cross-attention layer to extract features for each point. Surface segmentation is used to locate all fracture points in one broken object, and multi-part matching finds correspondences of fracture points among multiple pieces. The matching results will be used for pairwise pose recovery and joint alignment to retrieve an assembled object.

Results

Qualitative results of baseline methods and Jigsaw on the Breaking Bad dataset. The coordinate system of the green piece is set as the global coordinate system.

Citation

Acknowledgements

The website template was borrowed from Michaël Gharbi.