Neural network Reinforcement Learning for visual control of robot manipulators
Само за регистроване кориснике
2013
Чланак у часопису (Објављена верзија)
Метаподаци
Приказ свих података о документуАпстракт
It is known that most of the key problems in visual servo control of robots are related to the performance analysis of the system considering measurement and modeling errors. In this paper, the development and performance evaluation of a novel intelligent visual servo controller for a robot manipulator using neural network Reinforcement Learning is presented. By implementing machine learning techniques into the vision based control scheme, the robot is enabled to improve its performance online and to adapt to the changing conditions in the environment. Two different temporal difference algorithms (Q-learning and SARSA) coupled with neural networks are developed and tested through different visual control scenarios. A database of representative learning samples is employed so as to speed up the convergence of the neural network and real-time learning of robot behavior. Moreover, the visual servoing task is divided into two steps in order to ensure the visibility of the features: in the ...first step centering behavior of the robot is conducted using neural network Reinforcement Learning controller, while the second step involves switching control between the traditional Image Based Visual Servoing and the neural network Reinforcement Learning for enabling approaching behavior of the manipulator. The correction in robot motion is achieved with the definition of the areas of interest for the image features independently in both control steps. Various simulations are developed in order to present the robustness of the developed system regarding calibration error, modeling error, and image noise. In addition, a comparison with the traditional Image Based Visual Servoing is presented. Real world experiments on a robot manipulator with the low cost vision system demonstrate the effectiveness of the proposed approach.
Кључне речи:
Robot manipulator / Reinforcement Learning / Neural network / Intelligent hybrid control / Image Based Visual Servo controlИзвор:
Expert Systems With Applications, 2013, 40, 5, 1721-1736Издавач:
- Pergamon-Elsevier Science Ltd, Oxford
Финансирање / пројекти:
- Иновативни приступ у примени интелигентних технолошких система за производњу делова од лима заснован на еколошким принципима (RS-35004)
- Одрживост и унапређење машинских система у енергетици и транспорту применом форензичког инжењерства, еко и робуст дизајна (RS-35006)
DOI: 10.1016/j.eswa.2012.09.010
ISSN: 0957-4174
WoS: 000314737600030
Scopus: 2-s2.0-84872011415
Колекције
Институција/група
Mašinski fakultetTY - JOUR AU - Miljković, Zoran AU - Mitić, Marko AU - Lazarević, Mihailo AU - Babić, Bojan PY - 2013 UR - https://machinery.mas.bg.ac.rs/handle/123456789/1690 AB - It is known that most of the key problems in visual servo control of robots are related to the performance analysis of the system considering measurement and modeling errors. In this paper, the development and performance evaluation of a novel intelligent visual servo controller for a robot manipulator using neural network Reinforcement Learning is presented. By implementing machine learning techniques into the vision based control scheme, the robot is enabled to improve its performance online and to adapt to the changing conditions in the environment. Two different temporal difference algorithms (Q-learning and SARSA) coupled with neural networks are developed and tested through different visual control scenarios. A database of representative learning samples is employed so as to speed up the convergence of the neural network and real-time learning of robot behavior. Moreover, the visual servoing task is divided into two steps in order to ensure the visibility of the features: in the first step centering behavior of the robot is conducted using neural network Reinforcement Learning controller, while the second step involves switching control between the traditional Image Based Visual Servoing and the neural network Reinforcement Learning for enabling approaching behavior of the manipulator. The correction in robot motion is achieved with the definition of the areas of interest for the image features independently in both control steps. Various simulations are developed in order to present the robustness of the developed system regarding calibration error, modeling error, and image noise. In addition, a comparison with the traditional Image Based Visual Servoing is presented. Real world experiments on a robot manipulator with the low cost vision system demonstrate the effectiveness of the proposed approach. PB - Pergamon-Elsevier Science Ltd, Oxford T2 - Expert Systems With Applications T1 - Neural network Reinforcement Learning for visual control of robot manipulators EP - 1736 IS - 5 SP - 1721 VL - 40 DO - 10.1016/j.eswa.2012.09.010 ER -
@article{ author = "Miljković, Zoran and Mitić, Marko and Lazarević, Mihailo and Babić, Bojan", year = "2013", abstract = "It is known that most of the key problems in visual servo control of robots are related to the performance analysis of the system considering measurement and modeling errors. In this paper, the development and performance evaluation of a novel intelligent visual servo controller for a robot manipulator using neural network Reinforcement Learning is presented. By implementing machine learning techniques into the vision based control scheme, the robot is enabled to improve its performance online and to adapt to the changing conditions in the environment. Two different temporal difference algorithms (Q-learning and SARSA) coupled with neural networks are developed and tested through different visual control scenarios. A database of representative learning samples is employed so as to speed up the convergence of the neural network and real-time learning of robot behavior. Moreover, the visual servoing task is divided into two steps in order to ensure the visibility of the features: in the first step centering behavior of the robot is conducted using neural network Reinforcement Learning controller, while the second step involves switching control between the traditional Image Based Visual Servoing and the neural network Reinforcement Learning for enabling approaching behavior of the manipulator. The correction in robot motion is achieved with the definition of the areas of interest for the image features independently in both control steps. Various simulations are developed in order to present the robustness of the developed system regarding calibration error, modeling error, and image noise. In addition, a comparison with the traditional Image Based Visual Servoing is presented. Real world experiments on a robot manipulator with the low cost vision system demonstrate the effectiveness of the proposed approach.", publisher = "Pergamon-Elsevier Science Ltd, Oxford", journal = "Expert Systems With Applications", title = "Neural network Reinforcement Learning for visual control of robot manipulators", pages = "1736-1721", number = "5", volume = "40", doi = "10.1016/j.eswa.2012.09.010" }
Miljković, Z., Mitić, M., Lazarević, M.,& Babić, B.. (2013). Neural network Reinforcement Learning for visual control of robot manipulators. in Expert Systems With Applications Pergamon-Elsevier Science Ltd, Oxford., 40(5), 1721-1736. https://doi.org/10.1016/j.eswa.2012.09.010
Miljković Z, Mitić M, Lazarević M, Babić B. Neural network Reinforcement Learning for visual control of robot manipulators. in Expert Systems With Applications. 2013;40(5):1721-1736. doi:10.1016/j.eswa.2012.09.010 .
Miljković, Zoran, Mitić, Marko, Lazarević, Mihailo, Babić, Bojan, "Neural network Reinforcement Learning for visual control of robot manipulators" in Expert Systems With Applications, 40, no. 5 (2013):1721-1736, https://doi.org/10.1016/j.eswa.2012.09.010 . .