PowerPoint slideshow navigation control with hand gestures using Hidden Markov Model method

Abstract

Gesture is the easiest and most expressive way of communication between humans and computers, especially gestures that focus on hand and facial movements. Users can use simple gestures to communicate their ideas with a computer without interacting physically. One form of communication between users and machines is in the teaching and learning process in college. One of them is the way the speakers deliver material in the classroom. Most speakers nowadays make use of projectors that project PowerPoint slides from a connected laptop. In running the presentation, the speaker needs to move a slide from one slide to the next or to the previous slide. Therefore, a hand gesture recognition system is needed so it can implement the above interactions. In this study, a PowerPoint navigation control system was built. Digital imaging techniques use a combination of methods. The YCbCr threshold method is used to detect skin color. Furthermore, the morphological method is used to refine the detection results. Then the background subtraction method is used to detect moving objects. The classification method uses the Hidden Markov Model (HMM). With 526 hand images, the result shows that the accuracy of the confusion matrix is 74.5% and the sensitivity is 76.47%. From the accuracy and sensitivity values, it can be concluded that the Hidden Markov Model method can detect gestures quite well as a PowerPoint slide navigation control.