Intelligent Robotic Knowledge-Supported Visual Recognition of Handled Objects in Condictions of Acquiring Incomplete Information

Abstract

The paper presents a knowledge-supported approach to the synthesis of visual-perceptual system of industrial bi-manual robot for collaborative work with human, intended for performing service tasks in random task space in conditions of collecting incomplete data set on processing objects within technological operation. This system is intended for Industry 4.0, i.e. smart industrial production, that connects (integrates) cyber-physical systems with modern information and communication technologies. In Industry 4.0, machine vision systems and knowledge-based perceptions involving artificial intelligence are of great importance. The new principle of visual perception, described in this paper, will be used by the robot controller for faster and more accurate planning of the movement and manipulative actions of the robot in unstructured conditions, less tidy workspace and normal ambient lighting. The visual perception system uses the methods of classical 3D machine vision, a structured database containing CAD models of objects, technical drawings and photographs of bodies made from different angles of observation. The database also stores useful information on the technological stages of the implementation of operations on read objects using a syntax that is understandable to a robotic controller. The controller uses recorded images of objects and/or video streams from the scene or from the task workspace. In doing so, the robot uses artificial intelligence algorithms to reconstruct the scene from an incomplete set of object geometry data and determine unknown objects and their positions, and then create a strategy for capturing and manipulating bodies in accordance with the requirements of the technological process. The experimental system of a bi-manual collaborative robot, for the verification of the concept of knowledge-supported visual perception, is in the phase of physical integration of the system, so simulation results will be demonstrated here. For this purpose, the Robotic Vision Toolbox for the Mathworks’ Matlab/Simulink was used, which is connected to one Microsoft RealSense RGB-D camera as a video sensor via the appropriate software interface. The results of the experimental verification are presented in the corresponding figures in the paper with expert comments attached. In conclusion, future directions of development related to the inclusion of complementary, tactile perception on the robot gripper are given.