Intelligent Robotic Systems: Theory, Control and Applications in Manufacturing
Apstrakt
Image-based visual servo (IBVS) control algorithm is one of the approaches used to control the motion of robot manipulators in structured manufacturing environment.
The main challenges facing systems for visual servo control in real world applications are the following:
(i) maintenance of visual features within the field of view of the camera,
(ii) incorrect camera calibration parameters,
(iii) white image noise, and
(iv) modelling error.
- Unknown disturbances during the motion can result in none of the visual features in the image plane.
- Robustness to camera calibration parameters and image noise is very important in real world applications.
- The key problems in visual servo control of robots are related to the performance analysis of the system considering measurement and modeling errors;
Solution: By implementing machine learning techniques e.g. neural network Reinforcement Learning into the vision based control scheme, the robot is enabled to improve its perfo...rmance online and to adapt to the changing real time conditions in the environment.
Intelligent visual control for robot manipulator;
Two independent steps: one with the neural network Reinforcement Learning controller and the other with the switching scheme between the neural network Reinforcement Learning and the traditional IBVS control;
Firstly, the error between the feature state in the current image plane and the feature state in the prerecorded target image is calculated.
Step#1: a correction in the robot pose for aligning the base with the object of interest is conducted using the neural network Reinforcement Learning controller (for finding the optimal policy of actions)
Step#2: the approaching behavior of the robot is carried out.
The robotic system used for experimental validation consists of a 6 DOF robot NeuroArm Manipulator System from NeuroRobotics and a low cost camera.
The scene is composed of three black blob patterns in order to simplify and speed up the camera performance in real time. Captured low resolution grayscale images are in the size of 177 x 144 pixels. SURF algorithm proved out to be fairly accurate regarding feature detection in a low resolution image.
Conclusions (IBVS control):
- The experimental results show that an intelligent controller (Q-learning and SARSA, coupled with neural networks) is able to select an optimal action despite challenging conditions such as the presence of the calibration error, modeling error, and image noise.
- Real world experiments are conducted on a 6 DOF NeuroArm Manipulator System and a low-cost camera in an eye-in-hand configuration.
- The experimental results demonstrate that the proposed method can provide a high accuracy of a manipulator positioning in a situation when the low resolution image is used.
Ključne reči:
Image-based visual servo (IBVS) control algorithm / Manufacturing environment / Camera calibration parameters / White image noise / Modelling error / Maintenance of visual features within the field of view of the camera / Reinforcement Learning / Artificial neural networks / 6 DOF robot NeuroArm Manipulator System / Intelligent controller / Accuracy of a manipulator positioningIzvor:
The Fifth China Robot Summit and Intelligent Economic Talents Summit, The Provincial Government of Zhejiang Provincial Party Committee and Ningbo Municipal Government, 2018Izdavač:
- The Fifth China Robot Summit and Intelligent Economic Talents Summit, Ningbo, China, May, 2018.
Finansiranje / projekti:
- Inovativni pristup u primeni inteligentnih tehnoloških sistema za proizvodnju delova od lima zasnovan na ekološkim principima (RS-MESTD-Technological Development (TD or TR)-35004)
Napomena:
- This invited lecture was a keynote part of the Fifth China Robot Summit.
Kolekcije
Institucija/grupa
Mašinski fakultetTY - GEN AU - Miljković, Zoran AU - Petrović, Milica PY - 2018 UR - https://machinery.mas.bg.ac.rs/handle/123456789/6623 AB - Image-based visual servo (IBVS) control algorithm is one of the approaches used to control the motion of robot manipulators in structured manufacturing environment. The main challenges facing systems for visual servo control in real world applications are the following: (i) maintenance of visual features within the field of view of the camera, (ii) incorrect camera calibration parameters, (iii) white image noise, and (iv) modelling error. - Unknown disturbances during the motion can result in none of the visual features in the image plane. - Robustness to camera calibration parameters and image noise is very important in real world applications. - The key problems in visual servo control of robots are related to the performance analysis of the system considering measurement and modeling errors; Solution: By implementing machine learning techniques e.g. neural network Reinforcement Learning into the vision based control scheme, the robot is enabled to improve its performance online and to adapt to the changing real time conditions in the environment. Intelligent visual control for robot manipulator; Two independent steps: one with the neural network Reinforcement Learning controller and the other with the switching scheme between the neural network Reinforcement Learning and the traditional IBVS control; Firstly, the error between the feature state in the current image plane and the feature state in the prerecorded target image is calculated. Step#1: a correction in the robot pose for aligning the base with the object of interest is conducted using the neural network Reinforcement Learning controller (for finding the optimal policy of actions) Step#2: the approaching behavior of the robot is carried out. The robotic system used for experimental validation consists of a 6 DOF robot NeuroArm Manipulator System from NeuroRobotics and a low cost camera. The scene is composed of three black blob patterns in order to simplify and speed up the camera performance in real time. Captured low resolution grayscale images are in the size of 177 x 144 pixels. SURF algorithm proved out to be fairly accurate regarding feature detection in a low resolution image. Conclusions (IBVS control): - The experimental results show that an intelligent controller (Q-learning and SARSA, coupled with neural networks) is able to select an optimal action despite challenging conditions such as the presence of the calibration error, modeling error, and image noise. - Real world experiments are conducted on a 6 DOF NeuroArm Manipulator System and a low-cost camera in an eye-in-hand configuration. - The experimental results demonstrate that the proposed method can provide a high accuracy of a manipulator positioning in a situation when the low resolution image is used. PB - The Fifth China Robot Summit and Intelligent Economic Talents Summit, Ningbo, China, May, 2018. T2 - The Fifth China Robot Summit and Intelligent Economic Talents Summit, The Provincial Government of Zhejiang Provincial Party Committee and Ningbo Municipal Government T1 - Intelligent Robotic Systems: Theory, Control and Applications in Manufacturing UR - https://hdl.handle.net/21.15107/rcub_machinery_6623 ER -
@misc{ author = "Miljković, Zoran and Petrović, Milica", year = "2018", abstract = "Image-based visual servo (IBVS) control algorithm is one of the approaches used to control the motion of robot manipulators in structured manufacturing environment. The main challenges facing systems for visual servo control in real world applications are the following: (i) maintenance of visual features within the field of view of the camera, (ii) incorrect camera calibration parameters, (iii) white image noise, and (iv) modelling error. - Unknown disturbances during the motion can result in none of the visual features in the image plane. - Robustness to camera calibration parameters and image noise is very important in real world applications. - The key problems in visual servo control of robots are related to the performance analysis of the system considering measurement and modeling errors; Solution: By implementing machine learning techniques e.g. neural network Reinforcement Learning into the vision based control scheme, the robot is enabled to improve its performance online and to adapt to the changing real time conditions in the environment. Intelligent visual control for robot manipulator; Two independent steps: one with the neural network Reinforcement Learning controller and the other with the switching scheme between the neural network Reinforcement Learning and the traditional IBVS control; Firstly, the error between the feature state in the current image plane and the feature state in the prerecorded target image is calculated. Step#1: a correction in the robot pose for aligning the base with the object of interest is conducted using the neural network Reinforcement Learning controller (for finding the optimal policy of actions) Step#2: the approaching behavior of the robot is carried out. The robotic system used for experimental validation consists of a 6 DOF robot NeuroArm Manipulator System from NeuroRobotics and a low cost camera. The scene is composed of three black blob patterns in order to simplify and speed up the camera performance in real time. Captured low resolution grayscale images are in the size of 177 x 144 pixels. SURF algorithm proved out to be fairly accurate regarding feature detection in a low resolution image. Conclusions (IBVS control): - The experimental results show that an intelligent controller (Q-learning and SARSA, coupled with neural networks) is able to select an optimal action despite challenging conditions such as the presence of the calibration error, modeling error, and image noise. - Real world experiments are conducted on a 6 DOF NeuroArm Manipulator System and a low-cost camera in an eye-in-hand configuration. - The experimental results demonstrate that the proposed method can provide a high accuracy of a manipulator positioning in a situation when the low resolution image is used.", publisher = "The Fifth China Robot Summit and Intelligent Economic Talents Summit, Ningbo, China, May, 2018.", journal = "The Fifth China Robot Summit and Intelligent Economic Talents Summit, The Provincial Government of Zhejiang Provincial Party Committee and Ningbo Municipal Government", title = "Intelligent Robotic Systems: Theory, Control and Applications in Manufacturing", url = "https://hdl.handle.net/21.15107/rcub_machinery_6623" }
Miljković, Z.,& Petrović, M.. (2018). Intelligent Robotic Systems: Theory, Control and Applications in Manufacturing. in The Fifth China Robot Summit and Intelligent Economic Talents Summit, The Provincial Government of Zhejiang Provincial Party Committee and Ningbo Municipal Government The Fifth China Robot Summit and Intelligent Economic Talents Summit, Ningbo, China, May, 2018... https://hdl.handle.net/21.15107/rcub_machinery_6623
Miljković Z, Petrović M. Intelligent Robotic Systems: Theory, Control and Applications in Manufacturing. in The Fifth China Robot Summit and Intelligent Economic Talents Summit, The Provincial Government of Zhejiang Provincial Party Committee and Ningbo Municipal Government. 2018;. https://hdl.handle.net/21.15107/rcub_machinery_6623 .
Miljković, Zoran, Petrović, Milica, "Intelligent Robotic Systems: Theory, Control and Applications in Manufacturing" in The Fifth China Robot Summit and Intelligent Economic Talents Summit, The Provincial Government of Zhejiang Provincial Party Committee and Ningbo Municipal Government (2018), https://hdl.handle.net/21.15107/rcub_machinery_6623 .