[Kde-accessibility] Working Demonstration for Simon[GSoC]

[Sina Bahram]

Some thoughts from universal design:

 

I would suggest that perhaps the blinking might be too restrictive. I'm thinking of blind users that might want to use this but for
whom blinking might not be the best way to communicate.

 

Also, it might be nice to activate it in some other way than focus of visual gaze, as again the blind user in this scenario won't
know where to look.

 

Take care,

Sina

 

Website: www.SinaBahram.com

Twitter: @SinaBahram

 

From: kde-accessibility-bounces at kde.org [mailto:kde-accessibility-bounces at kde.org] On Behalf Of Yash Shah
Sent: Tuesday, March 20, 2012 4:28 PM
To: kde-accessibility at kde.org
Subject: [Kde-accessibility] Working Demonstration for Simon[GSoC]

 

Hi Peter,

I am working on project of face detection/recognition for Simon since last few days. I have implement some part of it. I would like
to demonstrate that to you.

As the major obstacle for command and control speech recognition systems is to differentiate commands from background noise, We will
activate the recognition only when the user is actively looking at the screen / robot and we will also detect whether the user is
speaking or not by recognizing mouth movements. So now we are not only detecting face, We are also detecting whether he is speaking
or not. We can also activate/deactivate the microphone on eye blinks. For example, If the user blinks eyes continuously for two
times, we can activate/deactivate the microphones. We will also allow user defined gestures to control it. This matches a natural
"human to human" communication.

I have uploaded the working video on Youtube.

http://www.youtube.com/watch?v=wGI4lYXxlWg


I am able to detect:

1. Face.
2 .Eyes 
3. Mouth
4. Whether the user is speaking or not.

I am able to track the cropped image of the face accurately which can be seen in the Cropped Window of the video. I am also able to
track the size of the face and also for tilted face. Also, The processing of images hardly takes much CPU usages. We are checking
for users every 2 seconds which makes it fast and efficient.

This is just the demonstration of how things will be done. We will be using libKface library for efficient face detection. It was
developed by my friend Aditya Bhatt from my college in Gsoc 2010. I will extend that for detecting mouth and other parts. It is not
just about 3 months of GSoC, We will keep linking Computer vision to Simon even after that.
 




-- 
Regards,
Yash Shah

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-accessibility/attachments/20120320/290608ad/attachment.html>