Semantic segmentation based stereo visual servoing of nonholonomic mobile robot in intelligent manufacturing environment
Само за регистроване кориснике
2022
Чланак у часопису (Објављена верзија)
Метаподаци
Приказ свих података о документуАпстракт
In the interest of developing an intelligent manufacturing environment with an agile, efficient, and optimally utilized transportation system, mobile robots need to achieve a certain level of autonomy as they play an important role in carrying out transportation tasks. Bearing this in mind, in the paper we propose a novel stereo visual servoing method for nonholonomic mobile robot control based on semantic segmentation. Semantic segmentation provides a rich body of information required for an adequate decision-making process in a clustered, dynamic, and ever-changing manufacturing environment. The innovative idea behind the new visual servoing system is to utilize semantic information of the scene for visual servoing, as well as for other mobile robot tasks, such as obstacle avoidance, scene understanding, and simultaneous localization and mapping. Semantic segmentation is carried out by exploiting fully convolutional neural networks. The new visual servoing algorithm utilizes an inten...sity-based image registration procedure, which results in the image transformation matrix. The transformation matrix encompasses the relations of images taken at the current and desired pose, and that information is directly used for visual servoing. The developed algorithm is deployed on our own developed wheeled differential drive mobile robot RAICO (Robot with Artificial Intelligence based COgnition). The experimental evaluation is carried out in the 3D simulation environment and in the laboratory model of the real manufacturing environment. The experimental results show that the accuracy of the proposed approach is improved when compared to the state-of-the-art approaches while being robust to the partial occlusions of the scene and illumination changes.
Кључне речи:
Visual servoing / Stereo camera system / Semantic segmentation / Nonholonomic mobile robot / Intelligent manufacturing systems / Image registrationИзвор:
Expert Systems With Applications, 2022, 190, 116203-Издавач:
- Pergamon-Elsevier Science Ltd, Oxford
Финансирање / пројекти:
- Министарство науке, технолошког развоја и иновација Републике Србије, институционално финансирање - 200105 (Универзитет у Београду, Машински факултет) (RS-MESTD-inst-2020-200105)
- MISSION4.0 - Deep Machine Learning and Swarm Intelligence-Based Optimization Algorithms for Control and Scheduling of Cyber-Physical Systems in Industry 4.0 (RS-ScienceFundRS-AI-6523109)
DOI: 10.1016/j.eswa.2021.116203
ISSN: 0957-4174
WoS: 000724554000006
Scopus: 2-s2.0-85119331942
Колекције
Институција/група
Mašinski fakultetTY - JOUR AU - Jokić, Aleksandar AU - Petrović, Milica AU - Miljković, Zoran PY - 2022 UR - https://machinery.mas.bg.ac.rs/handle/123456789/3757 AB - In the interest of developing an intelligent manufacturing environment with an agile, efficient, and optimally utilized transportation system, mobile robots need to achieve a certain level of autonomy as they play an important role in carrying out transportation tasks. Bearing this in mind, in the paper we propose a novel stereo visual servoing method for nonholonomic mobile robot control based on semantic segmentation. Semantic segmentation provides a rich body of information required for an adequate decision-making process in a clustered, dynamic, and ever-changing manufacturing environment. The innovative idea behind the new visual servoing system is to utilize semantic information of the scene for visual servoing, as well as for other mobile robot tasks, such as obstacle avoidance, scene understanding, and simultaneous localization and mapping. Semantic segmentation is carried out by exploiting fully convolutional neural networks. The new visual servoing algorithm utilizes an intensity-based image registration procedure, which results in the image transformation matrix. The transformation matrix encompasses the relations of images taken at the current and desired pose, and that information is directly used for visual servoing. The developed algorithm is deployed on our own developed wheeled differential drive mobile robot RAICO (Robot with Artificial Intelligence based COgnition). The experimental evaluation is carried out in the 3D simulation environment and in the laboratory model of the real manufacturing environment. The experimental results show that the accuracy of the proposed approach is improved when compared to the state-of-the-art approaches while being robust to the partial occlusions of the scene and illumination changes. PB - Pergamon-Elsevier Science Ltd, Oxford T2 - Expert Systems With Applications T1 - Semantic segmentation based stereo visual servoing of nonholonomic mobile robot in intelligent manufacturing environment SP - 116203 VL - 190 DO - 10.1016/j.eswa.2021.116203 ER -
@article{ author = "Jokić, Aleksandar and Petrović, Milica and Miljković, Zoran", year = "2022", abstract = "In the interest of developing an intelligent manufacturing environment with an agile, efficient, and optimally utilized transportation system, mobile robots need to achieve a certain level of autonomy as they play an important role in carrying out transportation tasks. Bearing this in mind, in the paper we propose a novel stereo visual servoing method for nonholonomic mobile robot control based on semantic segmentation. Semantic segmentation provides a rich body of information required for an adequate decision-making process in a clustered, dynamic, and ever-changing manufacturing environment. The innovative idea behind the new visual servoing system is to utilize semantic information of the scene for visual servoing, as well as for other mobile robot tasks, such as obstacle avoidance, scene understanding, and simultaneous localization and mapping. Semantic segmentation is carried out by exploiting fully convolutional neural networks. The new visual servoing algorithm utilizes an intensity-based image registration procedure, which results in the image transformation matrix. The transformation matrix encompasses the relations of images taken at the current and desired pose, and that information is directly used for visual servoing. The developed algorithm is deployed on our own developed wheeled differential drive mobile robot RAICO (Robot with Artificial Intelligence based COgnition). The experimental evaluation is carried out in the 3D simulation environment and in the laboratory model of the real manufacturing environment. The experimental results show that the accuracy of the proposed approach is improved when compared to the state-of-the-art approaches while being robust to the partial occlusions of the scene and illumination changes.", publisher = "Pergamon-Elsevier Science Ltd, Oxford", journal = "Expert Systems With Applications", title = "Semantic segmentation based stereo visual servoing of nonholonomic mobile robot in intelligent manufacturing environment", pages = "116203", volume = "190", doi = "10.1016/j.eswa.2021.116203" }
Jokić, A., Petrović, M.,& Miljković, Z.. (2022). Semantic segmentation based stereo visual servoing of nonholonomic mobile robot in intelligent manufacturing environment. in Expert Systems With Applications Pergamon-Elsevier Science Ltd, Oxford., 190, 116203. https://doi.org/10.1016/j.eswa.2021.116203
Jokić A, Petrović M, Miljković Z. Semantic segmentation based stereo visual servoing of nonholonomic mobile robot in intelligent manufacturing environment. in Expert Systems With Applications. 2022;190:116203. doi:10.1016/j.eswa.2021.116203 .
Jokić, Aleksandar, Petrović, Milica, Miljković, Zoran, "Semantic segmentation based stereo visual servoing of nonholonomic mobile robot in intelligent manufacturing environment" in Expert Systems With Applications, 190 (2022):116203, https://doi.org/10.1016/j.eswa.2021.116203 . .