A Lip Reading Application on MS Kinect Camera

YARGIÇ A., DOĞAN M.

IEEE International Symposium on INnovations in Intelligent SysTems and Applications (INISTA), Bulgaristan, 19 - 21 Haziran 2013 identifier identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/inista.2013.6577656
  • Basıldığı Ülke: Bulgaristan
  • Anahtar Kelimeler: Lip reading, MS Kinect camera, 3D face tracking, K-nearest neighbor classifier, lip activation, lip motion detection
  • Anadolu Üniversitesi Adresli: Evet

Özet

Hearing-impaired people can read lips and lip reading applications may help them to improve their lip imitation skills. Speech of normal people can be recognized by even cellular phones but lip reading systems using only visual features remain important for hearing-impaired people. This paper aims to develop an application using MS Kinect camera to recognize Turkish color names to be used in the education of hearing-impaired children. Predefined lip points are located with depth information by the MS Kinect Face Tracking SDK. Words are segmented from the speech and the angles between the lip points are used as features to classify the words. Angles are computed using the 3D coordinates of the lip points. The KNN classifier is used to classify the words with Manhattan and Euclidean distances and the best feature vectors are tried to be found. As a result, the isolated words are classified with the success rate of 78.22%.