Intelligent Robotic Knowledge-Supported Visual Recognition of Handled Objects in Condictions of Acquiring Incomplete Information
Abstract
The paper presents a knowledge-supported approach to the synthesis of visual-perceptual system of industrial bi-manual robot for collaborative work with human, intended for performing service tasks in random task space in conditions of collecting incomplete data set on processing objects within technological operation. This system is intended for Industry 4.0, i.e. smart industrial production, that connects (integrates) cyber-physical systems with modern information and communication technologies. In Industry 4.0, machine vision systems and knowledge-based perceptions involving artificial intelligence are of great importance. The new principle of visual perception, described in this paper, will be used by the robot controller for faster and more accurate planning of the movement and manipulative actions of the robot in unstructured conditions, less tidy workspace and normal ambient lighting. The visual perception system uses the methods of classical 3D machine vision, a structured data...base containing CAD models of objects, technical drawings and photographs of bodies made from different angles of observation. The database also stores useful information on the technological stages of the implementation of operations on read objects using a syntax that is understandable to a robotic controller. The controller uses recorded images of objects and/or video streams from the scene or from the task workspace. In doing so, the robot uses artificial intelligence algorithms to reconstruct the scene from an incomplete set of object geometry data and determine unknown objects and their positions, and then create a strategy for capturing and manipulating bodies in accordance with the requirements of the technological process. The experimental system of a bi-manual collaborative robot, for the verification of the concept of knowledge-supported visual perception, is in the phase of physical integration of the system, so simulation results will be demonstrated here. For this purpose, the Robotic Vision Toolbox for the Mathworks’ Matlab/Simulink was used, which is connected to one Microsoft RealSense RGB-D camera as a video sensor via the appropriate software interface. The results of the experimental verification are presented in the corresponding figures in the paper with expert comments attached. In conclusion, future directions of development related to the inclusion of complementary, tactile perception on the robot gripper are given.
Keywords:
Knowledge-supported perception / Intelligent grasping / Industry 4.0 / Industrial humanoid / Bi-manual collaborative robotSource:
Mechanisms and Machine Science, 2022, 120 MMS, 122-132Publisher:
- Springer Science and Business Media B.V.
Funding / projects:
- Acknowledgements. Development of the robot prototype is the result of the bilateral R&D project entitled “Development and experimental verification of performance of mobile dual-arm robot for cooperation with humans”
Collections
Institution/Community
Mašinski fakultetTY - CONF AU - Šumarac, J. AU - Ilić, U. AU - Rodić, Aleksandar AU - Xu, X. PY - 2022 UR - https://machinery.mas.bg.ac.rs/handle/123456789/3798 AB - The paper presents a knowledge-supported approach to the synthesis of visual-perceptual system of industrial bi-manual robot for collaborative work with human, intended for performing service tasks in random task space in conditions of collecting incomplete data set on processing objects within technological operation. This system is intended for Industry 4.0, i.e. smart industrial production, that connects (integrates) cyber-physical systems with modern information and communication technologies. In Industry 4.0, machine vision systems and knowledge-based perceptions involving artificial intelligence are of great importance. The new principle of visual perception, described in this paper, will be used by the robot controller for faster and more accurate planning of the movement and manipulative actions of the robot in unstructured conditions, less tidy workspace and normal ambient lighting. The visual perception system uses the methods of classical 3D machine vision, a structured database containing CAD models of objects, technical drawings and photographs of bodies made from different angles of observation. The database also stores useful information on the technological stages of the implementation of operations on read objects using a syntax that is understandable to a robotic controller. The controller uses recorded images of objects and/or video streams from the scene or from the task workspace. In doing so, the robot uses artificial intelligence algorithms to reconstruct the scene from an incomplete set of object geometry data and determine unknown objects and their positions, and then create a strategy for capturing and manipulating bodies in accordance with the requirements of the technological process. The experimental system of a bi-manual collaborative robot, for the verification of the concept of knowledge-supported visual perception, is in the phase of physical integration of the system, so simulation results will be demonstrated here. For this purpose, the Robotic Vision Toolbox for the Mathworks’ Matlab/Simulink was used, which is connected to one Microsoft RealSense RGB-D camera as a video sensor via the appropriate software interface. The results of the experimental verification are presented in the corresponding figures in the paper with expert comments attached. In conclusion, future directions of development related to the inclusion of complementary, tactile perception on the robot gripper are given. PB - Springer Science and Business Media B.V. C3 - Mechanisms and Machine Science T1 - Intelligent Robotic Knowledge-Supported Visual Recognition of Handled Objects in Condictions of Acquiring Incomplete Information EP - 132 SP - 122 VL - 120 MMS DO - 10.1007/978-3-031-04870-8_15 ER -
@conference{ author = "Šumarac, J. and Ilić, U. and Rodić, Aleksandar and Xu, X.", year = "2022", abstract = "The paper presents a knowledge-supported approach to the synthesis of visual-perceptual system of industrial bi-manual robot for collaborative work with human, intended for performing service tasks in random task space in conditions of collecting incomplete data set on processing objects within technological operation. This system is intended for Industry 4.0, i.e. smart industrial production, that connects (integrates) cyber-physical systems with modern information and communication technologies. In Industry 4.0, machine vision systems and knowledge-based perceptions involving artificial intelligence are of great importance. The new principle of visual perception, described in this paper, will be used by the robot controller for faster and more accurate planning of the movement and manipulative actions of the robot in unstructured conditions, less tidy workspace and normal ambient lighting. The visual perception system uses the methods of classical 3D machine vision, a structured database containing CAD models of objects, technical drawings and photographs of bodies made from different angles of observation. The database also stores useful information on the technological stages of the implementation of operations on read objects using a syntax that is understandable to a robotic controller. The controller uses recorded images of objects and/or video streams from the scene or from the task workspace. In doing so, the robot uses artificial intelligence algorithms to reconstruct the scene from an incomplete set of object geometry data and determine unknown objects and their positions, and then create a strategy for capturing and manipulating bodies in accordance with the requirements of the technological process. The experimental system of a bi-manual collaborative robot, for the verification of the concept of knowledge-supported visual perception, is in the phase of physical integration of the system, so simulation results will be demonstrated here. For this purpose, the Robotic Vision Toolbox for the Mathworks’ Matlab/Simulink was used, which is connected to one Microsoft RealSense RGB-D camera as a video sensor via the appropriate software interface. The results of the experimental verification are presented in the corresponding figures in the paper with expert comments attached. In conclusion, future directions of development related to the inclusion of complementary, tactile perception on the robot gripper are given.", publisher = "Springer Science and Business Media B.V.", journal = "Mechanisms and Machine Science", title = "Intelligent Robotic Knowledge-Supported Visual Recognition of Handled Objects in Condictions of Acquiring Incomplete Information", pages = "132-122", volume = "120 MMS", doi = "10.1007/978-3-031-04870-8_15" }
Šumarac, J., Ilić, U., Rodić, A.,& Xu, X.. (2022). Intelligent Robotic Knowledge-Supported Visual Recognition of Handled Objects in Condictions of Acquiring Incomplete Information. in Mechanisms and Machine Science Springer Science and Business Media B.V.., 120 MMS, 122-132. https://doi.org/10.1007/978-3-031-04870-8_15
Šumarac J, Ilić U, Rodić A, Xu X. Intelligent Robotic Knowledge-Supported Visual Recognition of Handled Objects in Condictions of Acquiring Incomplete Information. in Mechanisms and Machine Science. 2022;120 MMS:122-132. doi:10.1007/978-3-031-04870-8_15 .
Šumarac, J., Ilić, U., Rodić, Aleksandar, Xu, X., "Intelligent Robotic Knowledge-Supported Visual Recognition of Handled Objects in Condictions of Acquiring Incomplete Information" in Mechanisms and Machine Science, 120 MMS (2022):122-132, https://doi.org/10.1007/978-3-031-04870-8_15 . .