Open Access Journal

ISSN : 2394 - 6849 (Online)

International Journal of Engineering Research in Electronics and Communication Engineering(IJERECE)

Monthly Journal for Electronics and Communication Engineering

Open Access Journal

International Journal of Engineering Research in Electronics and Communication Engineering(IJERECE)

Monthly Journal for Electronics and Communication Engineering
Call For Paper : Vol 11, Issue 03, March 2024

ISSN : 2394-6849 (Online)

Call for Paper

Vol 11, Issue 03, March 2024

Indexing

Reinforcement Learning based path planning controller for collision avoidance and goal seeking of a mobile robot in complex dynamic environment

Author : Arun Shankar M 1 Dr P S Lalpriya 2

PDF

Date of Publication :25th March 2020

Abstract: A path planning controller of a mobile robot is very critical considering the fact that the robot navigation should happen reaching the target without hitting the obstacles. But the complexity of the task drastically varies with the type of environment through which the robot navigates. In certain environments like airports, shopping complex, bus terminals etc. the environment is so dynamic such that the robot navigation done by different path planning approaches which works well in static environments won’t be suitable for these dynamic applications. In this paper a path planning controller for collision avoidance and goal seeking of a mobile robot is presented utilizing deep Q reinforcement learning algorithm. A dynamic environment is created using Robot operating system Gazebo simulator and one of the most popular open source robot Turtlebot3 is used for simulation. The hyper-parameters are selected in such a way that the reinforcement learning path planner will train the robot to form a policy which will maximize the reward function. A hardware model also developed utilizing ultrasonic sensors for obstacle avoidance and UWB technology for localization and goal tracking. Similar results are obtained from the hardware model also when it is trained using the path planner.

Reference :

    1. F. Haro and M. Torres, ‘‘A comparison of path planning algorithms for omni-directional robots in dynamic environments,’’ in Proc. IEEE 3rd Latin Amer. Robot. Symp., Oct. 2006, pp. 18–25.
    2. A. Alexopoulos, A. Kandil, P. Orzechowski, and E. Badreddin, ‘‘A comparative study of collision avoidance techniques for unmanned aerial vehicles,’’ in Proc. IEEE Int. Conf. Syst., Man, Cybern., Oct. 2013, pp. 1969–1974
    3. A. Mohammadi, M. Rahimi, and A. A. Suratgar, ‘‘A new path planning and obstacle avoidance algorithm in dynamic environment,’’ in Proc. 22nd Iranian Conf. Elect. Eng. (ICEE), May 2014, pp. 1301–1306
    4. I. Susnea, V. Minzu, and G. Vasiliu, ‘‘Simple, real-time obstacle avoidancealgorithmformobilerobots,’’inProc.8thWSEASI nt.Conf.Comput. Intell., Man-Mach. Syst. Cybern. (CIMMACS), Stevens Point, WI, USA, 2009, pp. 24–29.
    5. Z.-Q. Ma and Z.-R. Yuan, ‘‘Real-time navigation and obstacle avoidance based on grids method for fast mobile robots,’’ Eng. Appl. Artif. Intell., vol. 8, no. 1, pp. 91–95, 1995.
    6. G. Yen and T. Hickey, ‘‘Reinforcement learning algorithms for robotic navigation in dynamic environments,’’ in Proc. Int. Joint Conf. Neural Netw. (IJCNN), vol. 2, May 2002, pp. 1444–1449
    7. X.-T. Truong, H. T. Dinh, and C. D. Nguyen, ‘‘An efficient navigation frameworkforautonomousmobilerobotsindynamicenviron mentsusing learningalgorithms,’’J.Comput.Sci.Cybern.,vol.33,no.2,pp .107–118, 2018
    8. Y. Wang, H. He, and C. Sun, ‘‘Learning to navigate through complex dynamic environment with modular deep reinforcement learning,’’ IEEE Trans. Games, to be published
    9. V. F. da Silva and A. H. R. Costa, ‘‘Compulsory flow Q-learning: An RL algorithmforrobotnavigationbasedonpartialpolicyandmacro-states,’’ J. Brazilian Comput. Soc., vol. 15, pp. 65–75, Sep. 2009. 
    10. S. Gu, T. Lillicrap, I. Sutskever, and S. Levine, “Continuous Deep QLearning with Model-based Acceleration,” in Proceedings of The 33rd International Conference on Machine Learning, 2016, pp. 2829–2838.
    11. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015
    12. A. Ng, A. Coates, M. Diel, V. Ganapathi, J. Schulte, B. Tse, E. Berger, and E. Liang, “Autonomous inverted helicopter flight via reinforcement learning,” Experimental Robotics IX, pp. 363–372, 2006.
    13. M.J.Segura,F.A.A.Cheein,J.M.Toibero,V.Mut,an dR.Carelli,‘‘Ultra widebandlocalizationandSLAM:Acomparativestudyformobiler obot navigation,’’ Sensors, vol. 11, no. 2, pp. 2035–2055, 2011.
    14. G. Dissanayake, S. Huang, Z. Wang, and R. Ranasinghe, ‘‘A review of recentdevelopmentsinsimultaneouslocalizationandmappin g,’’inProc. 6th Int. Conf. Ind. Inf. Syst., Aug. 2011, pp. 477–482.
    15. S. Kumar, V. Dhiman, M. R. Ganesh, and J. J. Corso, ‘‘Spatiotemporal articulated models for dynamic SLAM,’’ CoRR, vol. abs/1604.03526, pp. 1–10, Apr. 2016.
    16. M. Henein, G. Kennedy, V. Ila, and R. Mahony, ‘‘Simultaneous localization and mapping with dynamic rigid objects,’’ CoRR, vol. abs/1805.03800, pp. 1–10, May 2018.
    17. I. Randria, M. M. B. Khelifa, M. Bouchouicha, and P. Abellard, ‘‘A comparative study of six basic approaches for path planning towards an autonomous navigation,’’ in Proc. 33rd Annu. Conf. IEEE Ind. Electron. Soc. (IECON), Taipei, Taiwan, Nov. 2007, pp. 2730–2735
    18. I. Chaari, A. Koubaa, H. Bennaceur, A. Ammar, M. Alajlan, and H. Youssef, ‘‘Design and performance analysis of global path planning techniques for autonomous mobile robots in grid environments,’’ Int. J. Adv. Robot. Syst., vol. 14, no. 2, pp. 1–15, 2017.

Recent Article