[digiKam-users] Face recognition woes

[Jon]

Hi,

First I just want to thank for the really nice piece of software. Just
started using it but must say that I'm really impressed, and the general
scan for faces is a really nice feature which makes it a lot easier finding
good photos of people. The version I'm using
is digikam-7.2.0-beta2-x86-64.appimage.

However, like Frederic I'm struggling a bit with face recognition and
before jumping into my current problems I have four general questions on
preparing things as good as possible:

1. Roughly how many photos of a person should I manually tag before it's
realistic to expect automatic matches?

2. At what point is it "safe" to tag photos of a person where they look
really different?
I'm thinking of photos where a person is in the background out of focus,
motion blur, people dressing out and so on where there's a value for me to
tag that person A is in the photo, even if it isn't the focus of the photo.
Or should I avoid that and use a duplicate tag that isn't a person-tag to
make life easier for the face recognition code?

3. When manually tagging images is there a specific strategy which makes it
more efficient for the learning?
Should I try to find images from different angles and different years to
give a wider definition of the person, or should I only select images
showing the whole face?

4. When I have images with crowds, restaurants or sightseeing where the
same random person is in one max 2 images, should I mark them as "not
faces"? Having them as ignored doesn't make  much sense as there will be
thousands of such faces being found.


Now over to my struggles with face recognition:

I have a collection of about 20k images with around 50 people being my main
focus of trying to have automatic face recognition. I have, through manual
tagging and multiple runs of automatic detection, tagged these persons in
between 5 and 1500 images per person. I also have 5000 faces marked as
unknown. Out of those I'd say about 2000 are high quality photos of the
people I've identified.

I've tried the following things:

I read the mail thread started by Frederic, and followed the suggestion
there to use tools -> maintenance -> clear and rebuild training data.

Then I set the threshold to 90% with YOLO v3 checked and I only got 10
suggestions, neither being the suggested person.

Then I ran it again with 70% and YOLO v3 checked, got 1254 suggestions, but
only 294 of those where correct suggestions. The wrong faces seems very
random as well, such as suggesting a black man when the person is a white
woman or a 90+ years old lady who had 5 suggestions; 3 different men and a
toddler. I confirmed the 294 correct photos.

Ran it again with 80% and YOLO v3, it suggested 343 and only 13 were
correct. I rejected them all.

Ran again with 70% without YOLO, it suggested 1043 and I didn't go through
them all but it seems like about 5-10% were correct.

In all these runs the wrong ones are in many cases of other identified
people.
Is this roughly the performance I can expect from the face recognition? I
don't really know what's reasonable to expect, but if this is the level it
would seem that it won't optimise the manual workflow a lot.

Thanks in advance
Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/digikam-users/attachments/20210206/4876524d/attachment-0001.htm>