[digiKam-users] Question about the face recognition - how does matching work?

[Thomas D]

Hi,

First of all: Thanks for a fantastic piece of software! I love it!

I have a question about the "under the hood" stuff and some suggestions.

How does the matching in DK's face recognition actually work?
I mean lets say I have the following persons:


   - Alice (10 faces/images tagged)
   - Bob (1 face/image tagget, it is rather blurry)
   - Charlie (1000 faces/images tagged)


Then, I have the following new images added to our collection:
Image 1: Contains Charlie
Image 2. Contains Bob
Image 3: Contains  Alice

So now we have DK scan for images.
It finds 3 faces in these 3 images.

How does it go then?
Does it generate a matching score for each image to each person? And then
sees that Image 1 matches Alice with 4%, Bob with 83% and Charlie with 92
%? And then figures out that Alice must be the match?

Or does it generate a matching score for Alice and then see if this is over
the specified threshold? And if it is, then  assigns this image to Alice?
And if it is not, then continues to generate score against Bob, and if it
is over threshold then it matches this image to Bob?

The reason I am asking is that I have a number of people in my collection.
But the number if images for each person seems like a Pareto Distribution:
Very few people has a LOT of  photos   (e.g. close family and friends) and
many people have very few photos (e.g. acquaintances, random people I met
once, etc.)

And it seems that when I have a person like Bob that has very few photos.
And maybe even photos in poor quality (blurry, poor lighting, etc.) then
that person receives an huge amount of incorrect matches. EVEN when there
are much better candidates available (like the actual person on the
picture).

I am guessing this happens because the FR engine just accepts a match when
a score is over a threshold and does not keep searching for other matches
(possible to avoid an O(n^2) complexity.

If this is the case, I would think this could be improved greatly by one or
more of the following approaches:


1: When searching for matches, sort the list of people to try  by the
amount of photos already found and accepted for this person (descending

2: Assign an "image quality score" to each face. This should include stuff
like
 - if the image is blurry or not. (a fourier transformation could probably
be useful here)
 - How well something on this image does actually look like an image (use
data from the face detection engine)
 - if the  lighting is good (look at contrast/levels)

3: Store a list on each image/face with the last three persons this face
was matched to. For example if the face was previously matched to Bob, but
then it was rejected as a match, then store this information so that this
will reduce the likelihood of matching this face to Bob again. It is my
impression that this signal (rejection of a match) is completely lost in
the current version of DK. Maybe there should be a way to reject a match
and store this decision as well as reject a match and not store this
decision. The latter is useful when you just want to reject a lot of stuff
and start over with a higher threshold.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/digikam-users/attachments/20210908/357939fa/attachment.htm>