Application of convolutional neural networks for visual control of intelligent robotic systems

Abstract

Intelligent mobile robots are foreseen as one of the possible solutions to efficiently performing transportation and manipulation tasks in intelligent manufacturing systems (IMS) of Industry 4.0. In the last few decades, deep learning models have been recognized as a promising technique to enable the intelligent behavior of mobile robots for performing such tasks. For the particular problems of object detection and classification, a class of deep learning models, namely Convolutional Neural Networks (CNN), is the most widely used. This chapter presents an application of Region-based CNN (R-CNN) for advanced object identification tasks by using transfer learning. The proposed learning approach is further used for the improvement of Image- Based Visual Servoing (IBVS) algorithm used to control an intelligent mobile robot. The proposed algorithms are implemented in the MATLAB software package, and both simulation and the experimental verification of the proposed concept are performed on intelligent mobile robot, DOMINO (Deep learning Omnidirectional Mobile robot with INtelligent cOntrol). Four different CNN models are trained for object detection and classification, and the most suitable CNN model is ResNet-18, with the best recorded mean Average Precision (mAP) of 77%. Achieved experimental results show the applicability of CNN for accurate detection and classification of different manufacturing entities and the IBVS algorithm for efficient mobile robot control within IMS.