Q-Learning Framework as a Solution for an Obstacle Avoidance Problem in Unknown Environment
Abstract
This paper presents machine learning approach as a solution for an obstacle avoidance problem. Q-learning, as one of the reinforcement learning algorithms, imparts the learning process based on trial and error and a corresponding reward into the behaviour of an intelligent agent - a mobile robot. The adaptable actions of a mobile robot in situations when that behaviour is necessary are the main advantage over conventional methods for designing a navigational path. The implemented algorithm characterizes simplicity and efficiency, and certainty in terms of reaching optimal behaviour after the certain number of learning episodes. Experimental results show proper exploration strategy with gradually improving mobile robot state to action mapping by adjusting Q-value function in a described manner. With more episodes conducted this adaptable control system could lead to a fully autonomous mobile robot, which is one of the main demands in modern intelligent manufacturing systems in which sto...chastic changes in the environment can results with failure in the entire production process.
Keywords:
Intelligent mobile robot / Q-learning / Reinforcement machine learning / Obstacle avoidanceSource:
Introduction paper presented at the 6th International Working Conference ”Total Quality Management – Advanced and Intelligent Approaches”, Published in International Journal Total Quality Management & Excellence, 7th – 11th June, 2011, Belgrade, 2011, 39, 2, 21-25Publisher:
- Belgrade : JUSQ
Funding / projects:
- An innovative ecologically based approach to implementation of intelligent manufacturing systems for production of sheet metal parts (RS-MESTD-Technological Development (TD or TR)-35004)
Collections
Institution/Community
Mašinski fakultetTY - JOUR AU - Mitić, Marko AU - Miljković, Zoran AU - Babić, Bojan AU - Majstorović, Vidosav PY - 2011 UR - https://machinery.mas.bg.ac.rs/handle/123456789/4470 AB - This paper presents machine learning approach as a solution for an obstacle avoidance problem. Q-learning, as one of the reinforcement learning algorithms, imparts the learning process based on trial and error and a corresponding reward into the behaviour of an intelligent agent - a mobile robot. The adaptable actions of a mobile robot in situations when that behaviour is necessary are the main advantage over conventional methods for designing a navigational path. The implemented algorithm characterizes simplicity and efficiency, and certainty in terms of reaching optimal behaviour after the certain number of learning episodes. Experimental results show proper exploration strategy with gradually improving mobile robot state to action mapping by adjusting Q-value function in a described manner. With more episodes conducted this adaptable control system could lead to a fully autonomous mobile robot, which is one of the main demands in modern intelligent manufacturing systems in which stochastic changes in the environment can results with failure in the entire production process. PB - Belgrade : JUSQ T2 - Introduction paper presented at the 6th International Working Conference ”Total Quality Management – Advanced and Intelligent Approaches”, Published in International Journal Total Quality Management & Excellence, 7th – 11th June, 2011, Belgrade T1 - Q-Learning Framework as a Solution for an Obstacle Avoidance Problem in Unknown Environment EP - 25 IS - 2 SP - 21 VL - 39 UR - https://hdl.handle.net/21.15107/rcub_machinery_4470 ER -
@article{ author = "Mitić, Marko and Miljković, Zoran and Babić, Bojan and Majstorović, Vidosav", year = "2011", abstract = "This paper presents machine learning approach as a solution for an obstacle avoidance problem. Q-learning, as one of the reinforcement learning algorithms, imparts the learning process based on trial and error and a corresponding reward into the behaviour of an intelligent agent - a mobile robot. The adaptable actions of a mobile robot in situations when that behaviour is necessary are the main advantage over conventional methods for designing a navigational path. The implemented algorithm characterizes simplicity and efficiency, and certainty in terms of reaching optimal behaviour after the certain number of learning episodes. Experimental results show proper exploration strategy with gradually improving mobile robot state to action mapping by adjusting Q-value function in a described manner. With more episodes conducted this adaptable control system could lead to a fully autonomous mobile robot, which is one of the main demands in modern intelligent manufacturing systems in which stochastic changes in the environment can results with failure in the entire production process.", publisher = "Belgrade : JUSQ", journal = "Introduction paper presented at the 6th International Working Conference ”Total Quality Management – Advanced and Intelligent Approaches”, Published in International Journal Total Quality Management & Excellence, 7th – 11th June, 2011, Belgrade", title = "Q-Learning Framework as a Solution for an Obstacle Avoidance Problem in Unknown Environment", pages = "25-21", number = "2", volume = "39", url = "https://hdl.handle.net/21.15107/rcub_machinery_4470" }
Mitić, M., Miljković, Z., Babić, B.,& Majstorović, V.. (2011). Q-Learning Framework as a Solution for an Obstacle Avoidance Problem in Unknown Environment. in Introduction paper presented at the 6th International Working Conference ”Total Quality Management – Advanced and Intelligent Approaches”, Published in International Journal Total Quality Management & Excellence, 7th – 11th June, 2011, Belgrade Belgrade : JUSQ., 39(2), 21-25. https://hdl.handle.net/21.15107/rcub_machinery_4470
Mitić M, Miljković Z, Babić B, Majstorović V. Q-Learning Framework as a Solution for an Obstacle Avoidance Problem in Unknown Environment. in Introduction paper presented at the 6th International Working Conference ”Total Quality Management – Advanced and Intelligent Approaches”, Published in International Journal Total Quality Management & Excellence, 7th – 11th June, 2011, Belgrade. 2011;39(2):21-25. https://hdl.handle.net/21.15107/rcub_machinery_4470 .
Mitić, Marko, Miljković, Zoran, Babić, Bojan, Majstorović, Vidosav, "Q-Learning Framework as a Solution for an Obstacle Avoidance Problem in Unknown Environment" in Introduction paper presented at the 6th International Working Conference ”Total Quality Management – Advanced and Intelligent Approaches”, Published in International Journal Total Quality Management & Excellence, 7th – 11th June, 2011, Belgrade, 39, no. 2 (2011):21-25, https://hdl.handle.net/21.15107/rcub_machinery_4470 .