Panoptic segmentation is the combination of semantic and instance segmentation: assign the points in a 3D point cloud to semantic categories and partition them into distinct object instances. It has many obvious applications for outdoor scene understanding, from city mapping to forest management. Existing methods struggle to segment nearby instances of the same semantic category, like adjacent pieces of street furniture or neighbouring trees, which limits their usability for inventory- or management-type applications that rely on object instances. This study explores the steps of the panoptic segmentation pipeline concerned with clustering points into object instances, with the goal to alleviate that bottleneck. We find that a carefully designed clustering strategy, which leverages multiple types of learned point embeddings, significantly improves instance segmentation. Experiments on the NPM3D urban mobile mapping dataset and the FOR-instance forest dataset demonstrate the effectiveness and versatility of the proposed strategy.
From a research paper by Xiang, et al.
Modern panoptic segmentation techniques are often built upon a 3D deep network backbone that extracts per-point features, followed by network branches that segment the points into semantic categories and into object instances, based on those features. The backbone network is not the focus of this paper. We treat it as a plug-in module of our overall network that ingests a point cloud and returns a feature vector of fixed length for every point. Multiple well-proven, trainable feature extractors exist for the task (Thomas et al., 2019, Choy et al., 2019). Semantic segmentation also has reached a certain level of maturity and can be regarded as a commodity. Technically, the associated network branch is a classifier that maps the feature representation to a list of seudo-)probabilities per point and is typically trained by minimising the cross-entropy loss. We follow that
practice, but do not deeply delve into the details. The focus of the present paper is on the instance segmentation branch, arguably the least explored part of the problem and the current performance bottleneck. There are two different strategies to identify object instances in point clouds.
For the complete paper CLICK HERE.
Note – If you liked this post click here to stay informed of all of the 3D laser scanning, geomatics, UAS, autonomous vehicle, Lidar News and more. If you have an informative 3D video that you would like us to promote, please forward to editor@lidarnews.com and if you would like to join the Younger Geospatial Professional movement click here
