Learning from Multiple Demonstrations with Different Modes of Operations

Utku Bozdogan; Emre Ugur

doi:10.18201/ijisae.2020158887

Authors

Utku Bozdogan Boğaziçi University https://orcid.org/0000-0002-8912-9446
Emre Ugur Bogazici University https://orcid.org/0000-0001-9597-2731

DOI:

https://doi.org/10.18201/ijisae.2020158887

Keywords:

Gaussian Mixture Regression, Hidden Markov Models, Learning from Demonstration, trajectory reproduction

Abstract

In this paper, teaching multiple types of complex trajectories at once to a robot in a robust, easy to train model using Learning from Demonstration is studied where the robot is expected to gain the capacity to differentiate between different types of demonstrated trajectories and be able to reproduce these trajectories correctly. Demonstrated trajectories are used to train a Hidden Markov Model (HMM) and a modified version of Gaussian Mixture Regression (GMR) -which utilizes state transition probabilities between states of the HMM, the most probable state the end effector of the robot belongs to in the current reproduction of the trajectory, and previous points in the current reproduction of the trajectory- is used to estimate the trajectory iteratively. A Proportional Derivative (PD) controller is employed for the reproduction. Starting points that are intended to correspond to different types of trajectories which the robot is expected to differentiate between are tested on numerical and simulation experiments. Multiple numerical experiments and simulation experiments showed that our modified algorithm produced comparable results to previous work, and in certain complex trajectories our algorithm was successful where previous work has failed to produce expected results.

Downloads

Download data is not yet available.

Author Biography

Emre Ugur, Bogazici University

Emre Uğur is an assistant professor in Department of Computer Engineering Department, Bogazici
University, Turkey. After receiving his PhD in Computer Engineering from Middle East, he worked at
ATR Japan as a researcher (2009-2013), at University of Innsbruck as a senior researcher (2013-2016),
and at Osaka University as specially appointed assistant professor (2015, 2016). He participated in several EU funded projects, including Xperience, ROSSI, MACS and Swarm-bots, and is currently PI of IMAGINE project supported by European Commission, Horizon 2020 Programme. He is interested in developmental and cognitive robotics, and intelligent and adaptive manipulation.

References

A. Billard, S. Calinon, R. Dillmann, and S. Schaal, “Robot programming by demonstration,” in Handbook of Robotics, B. Siciliano and O. Khatib, Eds. Secaucus, NJ, USA: Springer-Verlag, 2008, pp. 1371–1394.

B. D. Argall, S. Chernova, M. Veloso, and B. Browning, “A survey of robot learning from demonstration,” Robotics and autonomous systems, Vol. 57, No. 5, pp. 469-483, 2009.

M. Khansari and A. Billard, “BM: An iterative method to learn stable non-linear dynamical systems with Gaussian mixture models,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Anchorage, Alaska, USA, May 2010.

A. Vakanski, F. Janabi-Sharifi, I. Mantegh, and A. Irish, “Trajectory learning based on conditional random fields for robot programming by demonstration,” in Proceedings of the IASTED International Conference on Robotics and Applications (RA’2010).

T. Alizadeh, S. Calinon, and D. G. Caldwell, “Learning from demonstrations with partially observable task parameters,” 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 3309-3314, IEEE, 2014.

S. Calinon, F. D'halluin, E. L. Sauser, D. G. Caldwell and A. G. Billard, “Learning and reproduction of gestures by imitation,” IEEE Robotics & Automation Magazine, Vol. 17, No. 2, pp. 44-54, 2010.

E. Ugur and H. Girgin, “Compliant Parametric Dynamic Movement Primitives,” Robotica, vol. 38, no. 3, pp. 457–474, 2020.

H. Girgin, and E. Ugur, “Associative skill memory models,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6043-6048, IEEE, October 2018.

S. Schaal, “Dynamic movement primitives-a framework for motor control in humans and humanoid robotics,” in Adaptive motion of animals and machines, pp. 261-280, Springer, Tokyo, 2006.

S. Calinon, and A. Billard, “Stochastic gesture production and recognition model for a humanoid robot,” in 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), Vol. 3, pp. 2769-2774, IEEE, September 2004.

A. Vakanski, I. Mantegh, A. Irish and F. Janabi-Shari, “Trajectory learning for robot programming by demonstration using hidden Markov model and dynamic time warping,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), Vol. 42, No. 4, pp. 1039-1052, 2012.

S. Calinon, and A. Billard, “Recognition and reproduction of gestures using a probabilistic framework combining PCA, ICA and HMM,” Proceedings of the 22nd international conference on Machine learning, pp. 105-112, ACM, 2005.

L. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257–285, Feb. 1989.

Z. Ghahramani, and M. I. Jordan, “Supervised learning from incomplete data via an EM approach,” Advances in neural information processing systems, pp. 120-127, 1994.

S. Calinon, E. L. Sauser, A. G. Billard and D. G. Caldwell, “Evaluation of a probabilistic approach to learn and reproduce gestures by imitation,” 2010 IEEE International Conference on Robotics and Automation, pp. 2671-2676, IEEE, 2010.

A. K. Tanwani and S. Calinon, “Learning robot manipulation tasks with task-parameterized semitied hidden semi-markov model,” IEEE Robotics and Automation Letters, vol. 1, no. 1, pp. 235-242, Jan 2016.

S. Calinon, F. Guenter, and A. Billard, “On learning, representing, and generalizing a task in a humanoid robot,” Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 37, no. 2, pp. 286-298, 2007.

C. G. Atkeson, A. W. Moore, and S. Schaal, “Locally weighted learning for control,” in Lazy learning. Springer, 1997, pp. 75-113.

D. Nguyen-Tuong, M. Seeger, and J. Peters, “Model learning with local Gaussian process regression,” Adv. Robot., vol. 23, pp. 2015–2034, 2009.

A. Ude, A. Gams, T. Asfour, and J. Morimoto, “Task-specific generalization of discrete and periodic dynamic movement primitives,” IEEE Transactions on Robotics, vol. 26, no. 5, pp. 800-815, 2010.

S. Vijayakumar and S. Schaal, “Locally weighted projection regression: Incremental real time learning in high dimensional space,” in Proceedings of the Seventeenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc., 2000, pp. 1079-1086.

S. Calinon, “A tutorial on task-parameterized movement learning and retrieval,” Intelligent Service Robotics, vol. 9, no. 1, pp. 1-29, 2016.