Hostname: page-component-cb9f654ff-h4f6x Total loading time: 0 Render date: 2025-08-26T02:45:53.365Z Has data issue: false hasContentIssue false
  • English
  • Français

A sketch drawing humanoid robot using image-based visual servoing

Part of: Robot Athletes and Entertainers

Published online by Cambridge University Press:  25 November 2019

Meng-Cheng Lau Open the ORCID record for Meng-Cheng Lau [Opens in a new window] ,
John Anderson  and
Jacky Baltes
Meng-Cheng Lau*
Affiliation:
Autonomous Agents Laboratory, Department of Computer Science, University of Manitoba, Winnipeg MB R3T2N2, Canada; e-mails: mengcheng.lau@gmail.com, andersj@cs.umanitoba.ca
John Anderson
Affiliation:
Autonomous Agents Laboratory, Department of Computer Science, University of Manitoba, Winnipeg MB R3T2N2, Canada; e-mails: mengcheng.lau@gmail.com, andersj@cs.umanitoba.ca
Jacky Baltes
Affiliation:
Educational Robotics Center, Department of Electrical Engineering, National Taiwan Normal University, 129, He-ping East Road, Section 1, Taipei, Taiwan 10610, Republic of China; e-mail: jacky.baltes@ntnu.edu.tw
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper presents our sketch drawing artist humanoid robot research. One of the limitations of the existing artist humanoid robot is the lack of feedback on the error that occurs during the drawing process. The contribution of this research is the development of a humanoid robot artist with drawing error correction capability. Based on our previous work with open-loop control pen-and-ink humanoid robot artist, we have implemented a closed-loop visual servoing approach to address this problem. Our experimental results show that this approach is sufficient to correct drawing errors that occur due to mechanical limitation of a robot.

Information

Type
Research Article
Information
The Knowledge Engineering Review , Volume 34 , 2019 , e18
Copyright
© Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Akinlar, C. & Topal, C. 2011a. Edlines: a real-time line segment detector with a false detection control. Pattern Recognition Letters 32(13), 1633–1642. http://www.sciencedirect.com/science/article/pii/S0167865511001772 CrossRefGoogle Scholar
Akinlar, C. & Topal, C. 2011b. Edlines: real-time line segment detection by edge drawing (ed). In 18th IEEE International Conference on Image Processing (ICIP), 2837–2840.Google Scholar
Atoofi, P., Hamker, F. H. & Nassour, J. 2018. Learning of central pattern generator coordination in robot drawing. Frontiers in Neurorobotics 12. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6064740/ CrossRefGoogle Scholar
Baltes, J., Cheng, C. T., Lau, M. C. & Anderson, J. E. 2011. Cost oriented automation approach to upper body humanoid robot. In Proceedings of the 18th IFAC World Congress, Milano, Italy.Google Scholar
Baltes, J., Tu, K.-Y., Sadeghnejad, S. & Anderson, J. 2017. Hurocup: competition for multi-event humanoid robot athletes. The Knowledge Engineering Review 32, e1.CrossRefGoogle Scholar
Bourquardez, O., Mahony, R., Hamel, T. & Chaumette, F. 2006. Stability and performance of image based visual servo control using first order spherical image moments. In 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, 4304–4309.Google Scholar
Brown, P., Bigge, B., Bird, J., Husbands, P., Perris, M. & Stokes, D. 2005. The drawbots. http://www.sussex.ac.uk/Users/philh/pubs/drawbots-muta-final-small.pdf [Accessed 25-July-2011].Google Scholar
Calinon, S., Epiney, J. & Billard, A. 2005. A humanoid robot drawing human portraits. In Proceedings of the IEEE-RAS International Conference on Humanoid Robots (HUMANOID 2005), IEEE-RAS, Tsukuba, Japan.Google Scholar
Chaumette, F. & Hutchinson, S. 2006. Visual servo control, part I: Basic approaches. IEEE Robotics and Automation Magazine 13(4), 82–90.CrossRefGoogle Scholar
Chaumette, F. & Hutchinson, S. 2007. Visual servo control, part II: Advanced approaches. IEEE Robotics and Automation Magazine 14(1), 109–118.CrossRefGoogle Scholar
Chu, H. K., Mills, J. K. & Cleghorn, W. L. 2011. Image-based visual servoing through micropart reflection for the microassembly process. Journal of Micromechanics and Microengineering 21(6), 065016. http://stacks.iop.org/0960-1317/21/i=6/a=065016 CrossRefGoogle Scholar
Clever, M. 2013. Makelangelo 2. http://www.marginallyclever.com/blog/drawbot [Accessed 31-July-2013].Google Scholar
Cohen, H. 1994. The further exploits of aaron, paiter. http://crca.ucsd.edu/~hcohen/cohenpdf/furtherexploits.pdf [Accessed 13-June-2011].Google Scholar
De Luca, A., Oriolo, G. & Giordano, P. R. 2007. On-line estimation of feature depth for image-based visual servoing schemes. In Proceedings 2007 IEEE International Conference on Robotics and Automation, 2823–2828. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4209517 CrossRefGoogle Scholar
Franklin, G. F., Powell, D. J. & Emami-Naeini, A. 2001. Feedback Control of Dynamic Systems, 4th edition, Prentice Hall PTR.Google Scholar
Gommel, M., Haitz, M. & Zappe, J. 2004. Autoportrait project: portrait drawings with a robotic arm. http://www.robotlab.de/auto/portrait.htm [Accessed 20-May-2011].Google Scholar
Hill, J. & Park, W. 1979. Real time control of a robot with a mobile camera. In Proceedings of 9th ISIR, Washington, D.C., 409–417.Google Scholar
Hodges, S. E. & Hall, T. 1996. Looking for a cheaper robot: visual feedback for automated PCB manufacture. http://www.ifm.eng.cam.ac.uk/automation/publications/papers/seh-thesis.pdf Google Scholar
Hutchinson, S., Hager, G. & Corke, P. 1996. A tutorial on visual servo control. IEEE Transactions on Robotics and Automation 12(5), 651–670.CrossRefGoogle Scholar
Instruments, N. 2011. Control laws. http://www.ni.com/white-paper/8156/en/ [Accessed 04-April-2012].Google Scholar
Kennedy, D. & Osuga, R. 2012. Calligraphy robot uses a motion copy system to reproduce detailed brushwork. http://www.diginfo.tv/v/12-0181-r-en.php [Accessed 31-July-2013].Google Scholar
Kragic, D. & Christensen, H. 2002. Survey on Visual Servoing for Manipulation, Technical report, Centre for Autonomous Systems, Numerical Analysis and Computer Science.Google Scholar
Kudoh, S., Ogawara, K., Ruchanurucks, M. & Ikeuchi, K. 2009. Painting robot with multi-fingered hands and stereo vision. Robotics and Autonomous Systems 57, 279–288.CrossRefGoogle Scholar
Lau, M. C. & Baltes, J. 2010. The real-time embedded system for a humanoid: Betty. In Proceedings of the 13th FIRA Robot World Congress, Communications in Computer and Information Science 103, 122–129, Springer-Verlag.Google Scholar
Lin, C. Y., Chuang, L. W. & Mac, T. T. 2009. Human portrait generation system for robot arm drawing. In Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Singapore. IEEE, 1757–1762.Google Scholar
Liu, Y., Mejias, L. & Li, Z. 2012. Fast power line detection and localization using steerable filter for active uav guidance. ISPRS12 XXXIX–B3, 491–496.Google Scholar
Lu, Y., Lam, J. H. M. & Yam, Y. 2009. Preliminary study on vision-based pen-and-ink drawing by a robotic manipulator. In Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Singapore. IEEE, 578–583.Google Scholar
Nomura, H. & Naito, T. 2000. Integrated visual servoing system to grasp industrial parts moving on conveyer by controlling 6DOF arm. In 2000 IEEE International Conference on Systems, Man, and Cybernetics 3, 1768–1775.Google Scholar
Olsson, T., Bengtsson, J., Johansson, R. & Malm, H. 2002. Force control and visual servoing using planar surface identification. In Proceedings of the 2002 IEEE International Conference on Robotics and Automation, Washington, DC. IEEE, 4211–4216.Google Scholar
Ruchanurucks, M., Kudoh, S., Ogawara, K., Shiratori, T. & Ikeuchi, K. 2007. Humanoid robot painter: Visual perception and high-level planning. In Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy. IEEE, 3028–3033.Google Scholar
Sasaki, K., Noda, K. & Ogata, T. 2016. Visual motor integration of robots drawing behavior using recurrent neural network. Robotics and Autonomous Systems 86, 184–195. http://www.sciencedirect.com/science/article/pii/S0921889016305383 CrossRefGoogle Scholar
Singh, A. K., Baranwal, N. & Nandi, G. C. 2017. Development of a self reliant humanoid robot for sketch drawing. Multimedia Tools and Applications 76(18), 18847–18870.CrossRefGoogle Scholar
Siradjuddin, I., Behera, L., McGinnity, T. & Coleman, S. 2010. Image based visual servoing of a 7 dof robot manipulator using a distributed fuzzy proportional controller. In 2010 IEEE International Conference on Fuzzy Systems (FUZZ), 1–8.Google Scholar
Srikaew, A., Cambron, M. E., Northrup, S., Peters, R. A., II, Ii, R. A. P., Wilkes, D. M. & Kawamura, K. 1998. Humanoid drawing robot. In Proceedings of the IASTED International Conference on Robotics and Manufacturing.Google Scholar
Team, O. D. 2013a. Geometric image transformations. http://docs.opencv.org/modules/imgproc/doc/geometric_transformations.html# [Accessed 18-July-2013].Google Scholar
Team, O. D. 2013b. Structural analysis and shape descriptors. http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html [Accessed 20-June-2013].Google Scholar
Tresset, P. & Leymarie, F. F. 2005. Generative portrait sketching. In Proceedings of the 11th International Conference on Virtual Sytems and MultiMedia (VSMM’05), VSMM, Ghent, Belgium, 739–748.Google Scholar
Tresset, P. & Leymarie, F. F. 2006. Aikon: The artistic/automatic ikonograph. In ACM SIGGRAPH 2006 Research Posters, SIGGRAPH’06. ACM. http://doi.acm.org/10.1145/1179622.1179664.CrossRefGoogle Scholar
Tresset, P., Leymarie, F. F. & Khaorapapong, N. 2011. Skediomata: guinea pig and performer. In Proceedings of the 17th International Symposium on Electronic Art, Istanbul, Turkey. ISEA Press.Google Scholar
Tsunashima, N. & Katsura, S. 2010. Reproduction of human motion using motion-copying system based on coordinate modification. In IECON 2010 - 36th Annual Conference on IEEE Industrial Electronics Society, 1609–1614.Google Scholar
Vahrenkamp, N., Boge, C., Welke, K., Asfour, T., Walter, J. & Dillmann, R. 2009. Visual servoing for dual arm motions on a humanoid robot. In 9th IEEE-RAS International Conference on Humanoid Robots, 2009. Humanoids 2009, 208–214.Google Scholar
Vahrenkamp, N., Wieland, S., Azad, P., Gonzalez, D., Asfour, T. & Dillmann, R. 2008. Visual servoing for humanoid grasping and manipulation tasks. In 8th IEEE-RAS International Conference on Humanoid Robots, 2008. Humanoids 2008, 406–412.Google Scholar
Weiss, L. & Sanderson, A. 1987. Dynamic sensor-based control of robots with visual feedback. IEEE Journal of Robotics and Automation 3(5), 404–417.CrossRefGoogle Scholar
Weiss, L., Sanderson, A. & Neuman, C. 1985. Dynamic visual servo control of robots: An adaptive image-based approach. In Proceedings of IEEE International Conference on Robotics and Automation, 2, 662–668.Google Scholar
Wilson, W., Williams Hulls, C. & Bell, G. 1996. Relative end-effector control using cartesian position based visual servoing. IEEE Transactions on Robotics and Automation 12(5), 684–696.CrossRefGoogle Scholar