REN Bufan,HUANG Xiaoyan,WU Sidong,et al.Research on Panoptic Segmentation of 3D Point Clouds based on Semantic Information[J].Journal of Chengdu University of Information Technology,2023,38(05):535-542.[doi:10.16836/j.cnki.jcuit.2023.05.007]
基于语义信息的三维点云全景分割方法研究
- Title:
- Research on Panoptic Segmentation of 3D Point Clouds based on Semantic Information
- 文章编号:
- 2096-1618(2023)05-0535-08
- 分类号:
- TP391
- 文献标志码:
- A
- 摘要:
- 针对端到端点云全景分割网络精度不足的问题，设计一种基于点云语义信息的全景分割算法。首先利用语义分割模型获取点云数据语义信息，然后结合点云语义和空间信息，对前景目标(车、人等)进行聚类。具体地，为避免同类别相邻目标被聚类为一个目标，提出融合法向量夹角特征、空间位置、语义信息的聚类算法进行准确的前景实例分割。最后，提出一种新的类别划分方法，在不影响后续决策处理情况下，显著增加分割质量。SemanticKITTI数据集上的实验结果表明，提出的方法在全景质量、分割质量、识别质量、平均交并比4个指标上取得了较好的效果，分别达到56.6%、82.3%、68.2%、68.1%，并保持较快的速度(175 ms)，充分证明其有效性和实用性。
- Abstract:
- The accuracy of end-to-end point cloud panoptic segmentation network is insufficient, this paper designs a panoptic segmentation algorithm based on point cloud semantic information to address this problem. This paper uses a semantic model to obtain the semantic information of point cloud, and then combined with spatial information to cluster the foreground objects(cars, people, etc.). Specifically, to avoid the adjacent objects of the same category from being clustered into one object, a clustering algorithm that integrates normal angle, spatial localization, and semantic information is proposed to accurately segment foreground instance. Finally, through further analysis and research on data categories, a new category classification method is proposed, which can significantly increase the segmentation quality without affecting the subsequent decision processing. The experimental results on the SemanticKITTI dataset show that the proposed method has achieved good results in four indicators: panoptic quality, segmentation quality, recognition quality, and average intersection ratio, respectively reaching 56.6%, 82.3%, 68.2%, 68.1%, and maintain a relatively fast speed(175 ms), which further indicates the effectiveness and practicability of this method.
参考文献/References:
[1] Zermas D,Izzat I,Papanikolopoulos N.Fast segmentation of 3d point clouds:A paradigm on lidar data for autonomous vehicle applications[C].2017 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2017:5067-5073.
[2] Shin M O,Oh G M,Kim S W,et al.Real-time and accurate segmentation of 3-D point clouds based on Gaussian process regression[J].IEEE Transactions on Intelligent Transportation Systems,2017,18(12):3363-3377.
[3] Yurtsever E,Lambert J,Carballo A,et al.A survey of autonomous driving:Common practices and emerging technologies[J].IEEE access,2020,8:58443-58469.
[4] Kirillov A,Girshick R,He K,et al.Panoptic feature pyramid networks[C].Proceedings of the IEEE/CVF conference on computer vision and pattern recognition,2019:6399-6408.
[5] Zhou Z,Zhang Y,Foroosh H.Panoptic-polarnet:Proposal-free lidar point cloud panoptic segmentation[C].Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:13194-13203.
[6] Gasperini S,Mahani M A N,Marcos-Ramiro A,et al.Panoster: End-to-end panoptic segmentation of lidar point clouds[J].IEEE Robotics and Automation Letters,2021,6(2):3216-3223.
[7] Hong F,Zhou H,Zhu X,et al.Lidar-based panoptic segmentation via dynamic shifting network[C].Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:13090-13099.
[8] Thomas H,Qi C R,Deschaud J E,et al.Kpconv: Flexible and deformable convolution for point clouds[C].Proceedings of the IEEE/CVF international conference on computer vision,2019:6411-6420.
[9] Lang A H,Vora S,Caesar H,et al.Pointpillars: Fast encoders for object detection from point clouds[C].Proceedings of the IEEE/CVF conference on computer vision and pattern recognition,2019:12697-12705.
[10] Shi S,Guo C,Jiang L,et al.Pv-rcnn: Point-voxel feature set abstraction for 3d object detection[C].Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:10529-10538.
[11] Singh G,Gupta S,Lease M,et al.Range-Net: A High Precision Streaming SVD for Big Data Applications[J].arXiv preprint arXiv:2010.14226,2020.
[12] Zhou H,Zhu X,Song X,et al.Cylinder3d:An effective 3d framework for driving-scene lidar semantic segmentation[J].arXiv preprint arXiv:2008.01550,2020.
[13] Zermas D,Izzat I,Papanikolopoulos N.Fast segmentation of 3d point clouds:A paradigm on lidar data for autonomous vehicle applications[C].2017 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2017:5067-5073.
[14] He L,Chao Y,Suzuki K.A run-based two-scan labeling algorithm[J].IEEE transactions on image processing,2008,17(5):749-756.
[15] Behley J,Garbade M,Milioto A,et al.Semantickitti: A dataset for semantic scene understanding of lidar sequences[C].Proceedings of the IEEE/CVF International Conference on Computer Vision,2019:9297-9307.
[16] Schumann O,Hahn M,Scheiner N,et al.RadarScenes:A real-world radar point cloud data set for automotive applications[C].2021 IEEE 24th International Conference on Information Fusion(FUSION).IEEE,2021:1-8.
[17] Kirillov A,He K,Girshick R,et al.Panoptic segmentation[C].Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:9404-9413.
相似文献/References:
[1]卢 丽,许源平,卢 军,等.基于社会力异常检测改进算法的人群行为模型[J].成都信息工程大学学报,2018,(01):1.[doi:10.16836/j.cnki.jcuit.2018.01.001]
LU Li,XU Yuan-ping,LU Jun,et al.A Crowd Behavior Model based on an ImprovedSocial Force Anomaly Detection Algorithm[J].Journal of Chengdu University of Information Technology,2018,(05):1.[doi:10.16836/j.cnki.jcuit.2018.01.001]
[2]胡 婕,陶宏才.基于深度学习的领域问答系统的设计与实现[J].成都信息工程大学学报,2019,(03):232.[doi:10.16836/j.cnki.jcuit.2019.03.004]
HU Jie,TAO Hongcai.Design and Implementation of Domain Question Answering System based on Deep Learning[J].Journal of Chengdu University of Information Technology,2019,(05):232.[doi:10.16836/j.cnki.jcuit.2019.03.004]
[3]王 强,李孝杰,陈 俊.基于He-Net的卷积神经网络算法的图像分类研究[J].成都信息工程大学学报,2017,(05):503.[doi:10.16836/j.cnki.jcuit.2017.05.007]
WANG Qing,LI Xiao-jie,CHEN Jun.Research on Image Classification based on HE-Net Convolutional Neural Networks[J].Journal of Chengdu University of Information Technology,2017,(05):503.[doi:10.16836/j.cnki.jcuit.2017.05.007]
[4]冉元波,孙 敏,高梦清,等.双偏振天气雷达水凝物识别研究[J].成都信息工程大学学报,2017,(06):590.[doi:10.16836/j.cnki.jcuit.2017.06.003]
RAN Yuan-bo,SUN Min,GAO Meng-qing,et al.Study on Hydrometeor Identification based on Deep Learning[J].Journal of Chengdu University of Information Technology,2017,(05):590.[doi:10.16836/j.cnki.jcuit.2017.06.003]
[5]周 咏,万 垚.基于无人机的监控系统设计[J].成都信息工程大学学报,2021,36(02):159.[doi:10.16836/j.cnki.jcuit.2021.02.006]
ZHOU Yong,WAN Yao.Design of Surveillance System based on UAV[J].Journal of Chengdu University of Information Technology,2021,36(05):159.[doi:10.16836/j.cnki.jcuit.2021.02.006]
[6]谭诗雨,杨 玲,师春香,等.复杂背景下银行卡号识别方法研究[J].成都信息工程大学学报,2021,36(03):280.[doi:10.16836/j.cnki.jcuit.2021.03.007]
TAN Shiyu,YANG Ling,SHI Chunxiang,et al.Bank Card Number Recognition System under the Complex Background based on Deep Learning[J].Journal of Chengdu University of Information Technology,2021,36(05):280.[doi:10.16836/j.cnki.jcuit.2021.03.007]
[7]郭楠馨,林宏刚,张运理,等.基于元学习的僵尸网络检测研究[J].成都信息工程大学学报,2022,37(06):615.[doi:10.16836/j.cnki.jcuit.2022.06.001]
GUO Nanxin,LIN Honggang,ZHANG Yunli,et al.Botnet Detection Method based on Meta-Learning Network[J].Journal of Chengdu University of Information Technology,2022,37(05):615.[doi:10.16836/j.cnki.jcuit.2022.06.001]
[8]李 静,鲜 林,王海江.基于YOLOv3的船只检测算法研究[J].成都信息工程大学学报,2023,38(01):37.[doi:10.16836/j.cnki.jcuit.2023.01.006]
LI Jing,XIAN Lin,WANG Haijiang.Research on Ship Detection Algorithm based on YOLOv3[J].Journal of Chengdu University of Information Technology,2023,38(05):37.[doi:10.16836/j.cnki.jcuit.2023.01.006]
[9]毛 波,杨 昊,周世杰,等.基于CMA-REPS格点预报数据的深度学习风速订正方法[J].成都信息工程大学学报,2023,38(03):264.[doi:10.16836/j.cnki.jcuit.2023.03.003]
MAO Bo,YANG Hao,ZHOU Shijie,et al.A Deep Learning Method for Wind Speed Grid Point Forecasting Data Correction based on CMA-REPS[J].Journal of Chengdu University of Information Technology,2023,38(05):264.[doi:10.16836/j.cnki.jcuit.2023.03.003]
[10]张卓然,张 倩,宋 智,等.基于残差Swin Transformer的天气图像识别技术研究[J].成都信息工程大学学报,2023,38(06):637.[doi:10.16836/j.cnki.jcuit.2023.06.003]
ZHANG Zhuoran,ZHANG Qian,SONG Zhi,et al.Research on Weather Image Recognition based on Residual Swin Transformer[J].Journal of Chengdu University of Information Technology,2023,38(05):637.[doi:10.16836/j.cnki.jcuit.2023.06.003]
备注/Memo
收稿日期:2022-09-22
基金项目:国家自然科学基金资助项目(62103064); 四川省科技厅资助项目(2021YFG0295、2021YFG0133、2021YFN0104、2021YFH0069、2022YFN 0020、2022YFS0565); 四川省无人系统智能感知控制技术工程实验室开放课题资助项目(WRXT2020-001、WRXT2020-002、WRXT2020-005)
通信作者:吴思东.E-mail:wsd@cuit.edu.cn