Published Paper Details:

In this paper, a new virtual piano system is proposed, with an emphasis on the use of a monocular camera system for gesture-based musical input with no dependence on keys. The system utilizes state-of-art hand and fingertip tracking built upon the principles of computer vision. One enhancement, in particular, is the velocity-sensitive key press detection mechanism used to detect rapid downward finger movement as the musical note triggers, much like the action of a piano. The concepts of dynamic calibration applied to an idealized reference line representing the edge of the desk and the study of accidental key presses resulting from hand movements not involved in keying enhance accuracy by clearly defining an exclusion zone that must not be crossed and minimizing interference from false contact with the keyboard. The system incorporates dynamic calibration so as to account for differing heights of the desk together with camera direction to make certain that it performs optimally in several situations. Efficiencies in tracking algorithms and feedback systems reduce latency delivering a dynamic and engaging application. Additional features include real-time fingertip highlighting and note names to facilitate user participation and to give feedback support. The proposed system shows that monocular camera-based solutions have the ability to provide an efficient way of constructing portable and accessible virtual musical instruments. The mentioned concepts include its use in music learning and teaching, as gesture-controlled devices, and augmented- or virtual-reality-based music applications. This paper discusses features of gesture recognition, interactive music systems, and computer vision with a focus on building a new approach to virtual instrument control.