Tracking and Recognition of Fist Move on Musical Excerpts at the Piano

Abstract

This works presents a method for teaching assistance in exercises of piano playing with hand tracking and gesture recognition. A tutor defines exercises to be performed and the system detects and tracks the hands of student and checks the correct perform of exercises. To hand detection and tracking are used coloured markers on the back of hands, captured by a regular webcam and for gesture recognition is used Multidimensional Dynamic Time Warping (MDTW), a n-dimensional version of Dynamic Time Warping (DTW). To verify the performance of the method, we have made videos of exercises, which were filtered for some adjusts and noise reduction. Results shows that the method has been capable to detect and recognize gestures successfully. Although the system needs some improvements, first impression about the system has been satisfactory. Thus, we hope, until the end of this work, implement a real-time version of the method for build a tutor system for piano classes.

Publication
In Proceedings of 11th Brazilian Workshop on Virtual and Augmented Reality (WRVA)

Text in Portuguese, with English title and abstract.

Thyago Carvalho
Thyago Carvalho
PhD Student
and former Master’s Student

Fabrizzio Soares is bla bla bla.

Fabrizzio Soares
Fabrizzio Soares
Associate Professor and CS Chair

Fabrizzio Soares is a professor of Computer Science, Information Systems and Software Engineering at INF/UFG. His research interests include Computer Vision, Human Computer Interaction, Machine Learning and Programming topics. He is the leader of the Pixellab group, which develops solutions for accesibilty, Precision Agriculture, and Interactive Systems.

Cristiane Ferreira
Cristiane Ferreira
PhD Student

Fabrizzio Soares is bla bla bla.

Related