[Kde-accessibility] Working Demonstration for Simon[GSoC]

[Sina Bahram]

Sounds good. I'd be interested in contributing more as that happens.

 

The only remaining thing from me would be to possibly lower the threshold to 1 second instead of 2 for perceived responsiveness.

 

Also, is the system recording the past 500 to 1,000 ms of audio in a temporary buffer so as to be able to start from there?

 

This will allow for an impression of "smoothness" from the user as they won't encounter the possible frustration of speaking before
the system is ready, even though as you point out, you inform them.

 

Btw, let's keep all conversations on list.

 

Take care,

Sina

 

Website: www.SinaBahram.com

Twitter: @SinaBahram

 

From: Yash Shah [mailto:blazonware at gmail.com] 
Sent: Tuesday, March 20, 2012 4:50 PM
To: Sina Bahram
Subject: Re: [Kde-accessibility] Working Demonstration for Simon[GSoC]

 

Hey Sina,
Thanks for your Inputs.
Yeah it is restrictive, I was just giving the idea. Some people will like to use that as activating/deactivating microphone so it
will be kind of optional feature for them. According to survey conducted by me in my campus, People really appreciated the blink
feature. My main focus will be on mouth movements detection. 
For blind people, We can go for fingers/hands gesture detections.

On Wed, Mar 21, 2012 at 2:06 AM, Sina Bahram <sbahram at nc.rr.com> wrote:

Some thoughts from universal design:

 

I would suggest that perhaps the blinking might be too restrictive. I'm thinking of blind users that might want to use this but for
whom blinking might not be the best way to communicate.

 

Also, it might be nice to activate it in some other way than focus of visual gaze, as again the blind user in this scenario won't
know where to look.

 

Take care,

Sina

 

Website: www.SinaBahram.com

Twitter: @SinaBahram

 

From: kde-accessibility-bounces at kde.org [mailto:kde-accessibility-bounces at kde.org] On Behalf Of Yash Shah
Sent: Tuesday, March 20, 2012 4:28 PM
To: kde-accessibility at kde.org
Subject: [Kde-accessibility] Working Demonstration for Simon[GSoC]

 

Hi Peter,

I am working on project of face detection/recognition for Simon since last few days. I have implement some part of it. I would like
to demonstrate that to you.

As the major obstacle for command and control speech recognition systems is to differentiate commands from background noise, We will
activate the recognition only when the user is actively looking at the screen / robot and we will also detect whether the user is
speaking or not by recognizing mouth movements. So now we are not only detecting face, We are also detecting whether he is speaking
or not. We can also activate/deactivate the microphone on eye blinks. For example, If the user blinks eyes continuously for two
times, we can activate/deactivate the microphones. We will also allow user defined gestures to control it. This matches a natural
"human to human" communication.

I have uploaded the working video on Youtube.

http://www.youtube.com/watch?v=wGI4lYXxlWg


I am able to detect:

1. Face.
2 .Eyes 
3. Mouth
4. Whether the user is speaking or not.

I am able to track the cropped image of the face accurately which can be seen in the Cropped Window of the video. I am also able to
track the size of the face and also for tilted face. Also, The processing of images hardly takes much CPU usages. We are checking
for users every 2 seconds which makes it fast and efficient.

This is just the demonstration of how things will be done. We will be using libKface library for efficient face detection. It was
developed by my friend Aditya Bhatt from my college in Gsoc 2010. I will extend that for detecting mouth and other parts. It is not
just about 3 months of GSoC, We will keep linking Computer vision to Simon even after that.
 



-- 
Regards,
Yash Shah

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-accessibility/attachments/20120320/c245c100/attachment-0001.html>