Artificial Intelligence (AI) – based strategies for point cloud data and digital twins
DOI:
https://doi.org/10.55779/ng33138Keywords:
deep learning (DL), geospatial AI, geospatial digital twin, machine learning (ML), point cloud dataAbstract
Artificial Intelligence (AI), specifically machine learning (ML) and deep learning (DL), is causing a paradigm shift in coding practices and software solutions across diverse fields. This study focuses on harnessing the potential of ML/DL strategies in the geospatial domain, where geodata possesses characteristics that align with the concept of a “lingual manuscript” in aesthetic theory. By employing ML/DL techniques, such as feature evaluation and extraction from 3D point clouds, we can derive concepts that are specific to software, geographical areas, and tasks. ML/DL-based interpretation of 3D point clouds extends geospatial modelling beyond implicit representations, enabling the resolution of complex heuristic-based reconstructions and abstract concepts. These advancements in artificial intelligence have the potential to optimize and expedite geodata computation and geographic information systems. However, ML/DL encounters notable challenges in this domain, including the need for abundant training data, advanced statistical methods, and the development of effective feature representations. Overcoming these challenges is essential to enhance the performance and efficacy of ML/DL systems. Additionally, ML/DL-based solutions can simplify software engineering processes by replacing certain aspects of current adoption and implementation practices, resulting in reduced complexities in development and management. Through the adoption of ML/DL, many of the existing explicitly coded GIS implementations may gradually be replaced in the long term. Overall, this research illustrates the transformative capabilities of ML/DL in geospatial applications and underscores the significance of addressing associated challenges to drive further advancements in the field.
Metrics
References
Allamanis M, Barr ET, Devanbu P, Sutton C (2018). A survey of machine learning for big code and naturalness. ACM Computing Surveys 51(4):1-37. https://doi.org/10.1145/3212695
Borth M, Verriet J, Muller G (2019). Digital twin strategies for SoS 4 Challenges and 4 Architecture Setups for Digital Twins of SoS. IEE 2019 14th Annual Conference System of Systems Engineering (SoSE) pp 164-169. https://doi.org/10.1109/SYSOSE.2019.8753860
Boschert S, Rosen R (2016). Digital twin – The simulation aspect. In: Hehenberger P, Bradley D (Eds). Mechatronic futures: Challenges and solutions for mechatronic systems and their designers. Springer International Publishing pp 59-74. https://doi.org/10.1007/978-3-319-32156-1_5
Brock A, Lim T, Ritchie JM, Weston N (2016). Generative and discriminative voxel modeling with convolutional neural networks. arXiv:1608.04236. https://doi.org/10.48550/arXiv.1608.04236
Chen H, Hou L, Zhang G, Wu S (2023). Using context-guided data augmentation, lightweight CNN, and proximity detection techniques to improve site safety monitoring under occlusion conditions. Safety Science 158:105958. https://doi.org/10.1016/j.ssci.2022.105958
Cheng R, Hou L, Xu S (2023). A review of digital twin applications in civil and infrastructure emergency management. Buildings 13(5):1143. https://doi.org/10.3390/buildings13051143
Diab A, Kashef R, Shaker A (2022). Deep learning for LiDAR point cloud classification in remote sensing. Sensors 22(20):7868. https://doi.org/10.3390/s22207868
Dogan Ö, Sahin O, Karaarslan E (2021). Digital twin based disaster management system proposal: DT-DMS. Journal of Emerging Computer Technologies 1(2):25-30.
Döllner J (2020). Geospatial Artificial Intelligence: Potentials of machine learning for 3D point clouds and geospatial digital twins. PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science 88(1):15-24. https://doi.org/10.1007/s41064-020-00102-3
Dong CZ, Catbas FN (2021). A review of computer vision–based structural health monitoring at local and global levels. Structural Health Monitoring 20(2):692-743. https://doi.org/10.1177/1475921720935585
Gorokhovatskyi O, Peredrii O (2018). Shallow convolutional neural networks for pattern recognition problems. 2018 IEEE Second International Conference on Data Stream Mining & Processing (DSMP) 18133631. https://doi.org/10.1109/DSMP.2018.8478540
Grieves M, Vickers J (2017). Digital Twin: Mitigating unpredictable, undesirable emergent behavior in complex systems. In: Kahlen J, Flumerfelt S, Alves A (Eds). Transdisciplinary Perspectives on Complex Systems. Springer, Cham. https://doi.org/10.1007/978-3-319-38756-7_4
Gross M, Pfister H (2007). Point-Based Graphics. Morgan Kaufmann Publishers Inc. San Francisco, CA, USA.
Guo M-H, Xu T, Liu J-J, Liu Z-N, Jiang P-T, Mu T-J, Zhang S-H, Martin R, Cheng M-M, Hu S-M (2022). Attention mechanisms in computer vision: A survey. Computational Visual Media 8. https://doi.org/10.1007/s41095-022-0271-y
Haenlein M, Kaplan A (2019). A brief history of Artificial Intelligence: On the past, present, and future of Artificial Intelligence. California Management Review 61(4):5-14. https://doi.org/10.1177/0008125619864925
Hindle A, Barr ET, Su Z, Gabel M, Devanbu P (2012). On the naturalness of software. ICSE '12: Proceedings of the 34th International Conference on Software Engineering pp 837-847.
Janowicz K, Gao S, McKenzie G, Hu Y, Bhaduri B (2020). GeoAI: spatially explicit artificial intelligence techniques for geographic knowledge discovery and beyond. International Journal of Geographical Information Science 34(4):625-636. https://doi.org/10.1080/13658816.2019.1684500
Jiang F, Ma L, Broyd T, Chen W, Luo H (2022). Building digital twins of existing highways using map data based on engineering expertise. Automation in Construction 134:104081. https://doi.org/10.1016/j.autcon.2021.104081
Jung D, Sundstrom C (2017). A combined data-driven and model-based residual selection algorithm for fault detection and isolation. IEEE Transactions on Control Systems Technology 27(2):616-630. https://doi.org/10.1109/TCST.2017.2773514
Jurafsky D, Martin JH (2000). Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition (1st ed.). Prentice Hall PTR.
Kaewunruen S, Lian Q (2019). Digital twin aided sustainability-based lifecycle management for railway turnout systems. Journal of Cleaner Production 228:1537-1551. https://doi.org/10.1016/j.jclepro.2019.04.156
Kaewunruen S, Sresakoolchai J, Ma W, Phil-Ebosie O (2021). Digital Twin aided vulnerability assessment and risk-based maintenance planning of bridge infrastructures exposed to extreme conditions. Sustainability 13(4):2051. https://doi.org/10.3390/su13042051
Kanevski M, Foresti L, Kaiser C, Pozdnoukhov A, Timonin V, Tuia D (2009). Machine learning models for geospatial data. In Bavaud F, Mager C (Eds). Handbook of Theoretical and Quantitative Geography. FGSE, University of Lausanne, Switzerland, Workshop series N° 2 pp 175-227.
Khajavi SH, Motlagh NH, Jaribion A, Werner LC, Holmstrom J (2019). Digital Twin: Vision, benefits, boundaries, and creation for buildings. IEEE Access 7:147406-147419. https://doi.org/10.1109/ACCESS.2019.2946515
Kwon JW, Yun SJ, Kim WT (2022). A semantic data-based distributed computing framework to accelerate Digital Twin services for large-scale disasters. Sensors 22(18):6749. https://doi.org/10.3390/s22186749
Lee D, Lee SH, Masoud N, Krishnan MS, Li VC (2021). Integrated digital twin and blockchain framework to support accountable information sharing in construction projects. Automation in Construction 127:103688. https://doi.org/10.1016/j.autcon.2021.103688
Lei F, Liu X, Dai Q, Ling B (2020). Shallow convolutional neural network for image classification. SN Applied Sciences 2:97. https://doi.org/10.1007/s42452-019-1903-4
Li Y, Pirk S, Su H, Ruizhongtai Qi C, Guibas L (2016). FPNN: Field probing neural networks for 3D data. Advances in Neural Information Processing Systems 29 (NIPS 2016). https://proceedings.neurips.cc/paper_files/paper/2016
Liu C, Le Roux L, Körner C, Tabaste O, Lacan F, Bigot S (2022). Digital Twin-enabled collaborative data management for metal additive manufacturing systems. Journal of Manufacturing Systems 62:857-874. https://doi.org/10.1016/j.jmsy.2020.05.010
Löwner M-O, Gröger G, Joachim B, Biljecki F, Nagel C (2016). Proposal for a new LOD and multi-representation concept for CityGML. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences: 4:3-12. https://doi.org/10.5194/isprs-annals-IV-2-W1-3-2016
Lu Q, Parlikad AK, Woodall P, Ranasinghe GD, Xie X, Liang Z, Konstantinou E, Heaton J, Schooling J (2020). Developing a Digital Twin at building and city levels: Case study of West Cambridge Campus. Journal of Management in Engineering 36(3):5020004. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000763
Luis Lugo J, Gallegos Reyes JC, Weidner J, Raheem A, Cheu RL (2022). Evaluation of 3D reconstruction methods from terrestrial LiDAR point cloud data to create Digital Twins of civil infrastructure projects. International Conference on Transportation and Development pp 81-92. https://ascelibrary.org/doi/abs/10.1061/9780784484319.008
Majumdar P, Haider MF, Reifsnider K (2013). Multi-physics response of structural composites and framework for modeling using material geometry. 54th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. https://doi.org/10.2514/6.2013-1577
Maturana D, Scherer S (2015). VoxNet: A 3D convolutional neural network for real-time object recognition. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) pp 922-928. https://doi.org/10.1109/IROS.2015.7353481
Meierhofer J, Schweiger L, Lu J, Züst S, West S, Stoll O, Kiritsis D (2021). Digital twin-enabled decision support services in industrial ecosystems. Applied Sciences 11(23):11418. https://doi.org/10.3390/app112311418
Motawa P, Almarshad A (2013). A knowledge-based BIM system for building maintenance. Automation in Construction 29:173-182. https://doi.org/10.1016/j.autcon.2012.09.008
Naderi H, Shojaei A (2023). Digital twinning of civil infrastructures: Current state of model architectures, interoperability solutions, and future prospects. Automation in Construction 149. Elsevier B.V. https://doi.org/10.1016/j.autcon.2023.104785
Pang TY, Pelaez Restrepo JD, Cheng CT, Yasin A, Lim H, Miletic M (2021). Developing a digital twin and digital thread framework for an ‘Industry 4.0’ Shipyard. Applied Sciences 11(3):1097. https://doi.org/10.3390/app11031097
Purdy M, Daugherty P (2017). How AI boosts industry profits and innovation. Accenture pp 1-28.
Qi CR, Su H, Niessner M, Dai A, Yan M, Guibas LJ (2016). Volumetric and multi-view CNNs for object classification on 3D data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp 5648-5656. http://arxiv.org/abs/1604.03265
Qi Q, Tao F (2018). Digital Twin and big data towards smart manufacturing and industry 4.0: 360 degree comparison. IEEE Access 6:3585-3593. https://doi.org/10.1109/ACCESS.2018.2793265
Richter R, Döllner J (2013). Concepts and techniques for integration, analysis and visualization of massive 3D point clouds. Computers, Environment and Urban Systems, 45:114-124. https://doi.org/10.1016/j.compenvurbsys.2013.07.004
Riegler G, Ulusoy AO, Geiger A (2016). OctNet: Learning deep 3D representations at high resolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp 3577-3586.
Rosen R, Wichert G, Lo G, Bettenhausen K (2015). About the importance of autonomy and Digital twins for the future of manufacturing. IFAC-PapersOnLine 48:567-572. https://doi.org/10.1016/j.ifacol.2015.06.141
Roveri R, Rahmann L, Oztireli A, Gross M (2018). A network architecture for point cloud classification via automatic depth images generation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp 4176-4184. https://doi.org/10.1109/CVPR.2018.00439
Rozsa A, Günther M, Boult TE (2016). Are accuracy and robustness correlated. 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA) pp 227-232. https://doi.org/10.1109/ICMLA.2016.0045
Ruizhongtai Qi C, Su H, NieBner M, Dai A, Yan M, Guibas L (2016). Volumetric and multi-view CNNs for object classification on 3D data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp 5648-5656. https://doi.org/10.1109/CVPR.2016.609
Ruizhongtai Qi C, Yi L, Su H, Guibas L (2017). PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. Advances in Neural Information Processing Systems 30 (NIPS 2017).
Shah K, Prabhakar TV, Sarweshkumar CR, Abhishek SV (2021). Construction of a digital twin framework using free and open-source software programs. IEEE Internet Computing 26(5):50-59. https://doi.org/10.1109/MIC.2021.3051798
Simonyan K, Zisserman A (2014). Very deep convolutional networks for large-scale image recognition. Computer Vision and Pattern Recognition arXiv:1409.1556. https://doi.org/10.48550/arXiv.1409.1556
Stojanovic V, Trapp M, Richter R, Döllner J (2019). Generation of approximate 2D and 3D floor plans from 3D Point Clouds. Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 1:177-184. https://doi.org/10.5220/0007247601770184
Stojanovic V, Trapp M, Richter R, Döllner J (2019). Classification of indoor point clouds using multiviews. The 24th International Conference on 3D Web Technology pp 1-9. https://doi.org/10.1145/3329714.3338129
Tao F, Cheng J, Qi Q, Zhang M, Zhang H, Sui F (2018). Digital twin-driven product design, manufacturing and service with big data. International Journal of Advanced Manufacturing Technology 94:3563-3576. https://doi.org/10.1007/s00170-017-0233-1
VoPham T, Hart JE, Laden F, Chiang Y-Y (2018). Emerging trends in geospatial artificial intelligence (geoAI): potential applications for environmental epidemiology. Environmental Health 17(1):40. https://doi.org/10.1186/s12940-018-0386-x
Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E (2018). Deep learning for computer vision: A brief review. Computational Intelligence and Neuroscience 2018:7068349. https://doi.org/10.1155/2018/7068349
Wang P-S, Liu Y, Guo Y-X, Sun C, Tong X (2017). O-CNN: Octree-based convolutional neural networks for 3D shape analysis. ACM Transactions on Graphics 36:72. https://doi.org/10.1145/3072959.3073608
Wang Y, Sun Y, Liu Z, Sarma SE, Bronstein MM, Solomon JM (2019). Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics 38:146. https://doi.org/10.1145/3326362
Wu J, Zhang C, Xue T, Freeman W, Tenenbaum J (2016). Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. Advances in Neural Information Processing Systems 29.
Wu S, Zhong S, Liu Y (2018). Deep residual learning for image steganalysis. Multimedia Tools and Applications 77:10437-10453. https://doi.org/10.1007/s11042-017-4440-4
Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015). 3D ShapeNets: A deep representation for volumetric shapes. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp 1912-1920. https://doi.org/10.1109/CVPR.2015.7298801
Xiao A, Huang J, Guan D, Lu S (2022). Unsupervised representation learning for point clouds: A survey. arXiv:2202.13589.
Xie X, Merino J, Moretti N, Pauwels P, Chang J, Parlikad AK (2023). Digital twin enabled fault detection and diagnosis process for building HVAC systems. Automation in Construction 146:104695. https://doi.org/10.1016/j.autcon.2022.104695
Xue F, Lu W, Chen Z, Webster CJ (2020). From LiDAR point cloud towards digital twin city: Clustering city objects based on Gestalt principles. ISPRS Journal of Photogrammetry and Remote Sensing 167:418-431. https://doi.org/10.1016/j.isprsjprs.2020.07.020
Yeum CM, Choi J, Dyke SJ (2018). Automated region-of-interest localization and classification for vision-based visual assessment of civil infrastructure. Structural Health Monitoring 18(3):675-689. https://doi.org/10.1177/1475921718765419
Zhang H-B, Zhang Y-X, Zhong B, Lei Q, Yang L, Du J-X, Chen D-S (2019). A comprehensive survey of vision-based human action recognition methods. Sensors 19(5):1005. https://doi.org/10.3390/s19051005
Zhang K, Zhang Z, Li Z, Qiao Y (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23:1499-1503. https://doi.org/10.1109/LSP.2016.2603342
Zhang Q, Hou J, Qian Y, Zeng Y, Zhang J, He Y (2023). Flattening-net: Deep regular 2D representation for 3D Point Cloud analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(8):9726-9742. https://doi.org/10.1109/TPAMI.2023.3244828
Zhao H, Tan C, OBrien EJ, Zhang B, Uddin N, Guo H (2022). Developing digital twins to characterize bridge behavior using measurements taken under random traffic. Journal of Bridge Engineering 27(1):4021101. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001814
Zhuang C, Miao T, Liu J, Xiong H (2021). The connotation of digital twin, and the construction and application method of shop-floor digital twin. Robotics and Computer-Integrated Manufacturing 68:102075. https://doi.org/10.1016/j.rcim.2020.102075
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Ifra AFTAB, Mohammad DOWAJY, Kristof KAPITANY, Tamas LOVAS
This work is licensed under a Creative Commons Attribution 4.0 International License.
Distribution - Permissions - Copyright
Papers published in Nova Geodesia are Open-Access, distributed under the terms and conditions of the Creative Commons Attribution License.
© Articles by the authors; licensee SMTCT, Cluj-Napoca, Romania. The journal allows the author(s) to hold the copyright/to retain publishing rights without restriction.
License:
Open Access Journal - the journal offers free, immediate, and unrestricted access to peer-reviewed research and scholarly work, due to SMTCT support to increase the visibility, accessibility and reputation of the researchers, regardless of geography and their budgets. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles, or use them for any other lawful purpose, without asking prior permission from the publisher or the author.
