[Digikam-devel] [digikam] [Bug 323888] New: Face recognition makes digikam fill all the available memory

[eric]

Hello,

Have a few more tests.

I was having some problem with ZRAM, but the bug still occur when do a
scan with option :  Clear unconfirmed results and rescan. (restart scan
for beginning except for result wich have been confirmed - I suppose).

Detection and recognition begin from root of the collection to the few 1
rst albums then crash with same error as my precedent posts :

GDB : 
error :
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x94db5b40 (LWP 2502)]
0xb6efd89a in Digikam::DImg::DImg (this=0x94db5124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:165
165                    dptr[0] = qBlue(*sptr);

(gdb) backtrace :

#0  0xb6efd89a in Digikam::DImg::DImg (this=0x94db5124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:165
#1  0xb70d5d7d in Digikam::PreviewLoadingTask::execute (this=0xaca8d38)

at /home/eric/digikamgit/dk/core/libs/threadimageio/previewtask.cpp:288
#2  0xb70c5205 in Digikam::LoadSaveThread::run (this=0xb033c38)

at /home/eric/digikamgit/dk/core/libs/threadimageio/loadsavethread.cpp:136
#3  0xb70ecd58 in Digikam::DynamicThread::DynamicThreadPriv::run (
    this=0xb033cd0)
    at /home/eric/digikamgit/dk/core/libs/threads/dynamicthread.cpp:186
#4  0xb53d446b in QThreadPoolThread::run (this=0x96bc2a0)
    at concurrent/qthreadpool.cpp:107
#5  0xb53e1eb0 in QThreadPrivate::start (arg=0x96bc2a0)
    at thread/qthread_unix.cpp:307
#6  0xb36c1d4c in start_thread (arg=0x94db5b40) at pthread_create.c:308
#7  0xb510fdde in clone ()
at ../sysdeps/unix/sysv/linux/i386/clone.S:130 


Valgrind :

digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif
color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation:
Orientation =>
Exif.Image.Orientation =>  1

<--CRASH-->

==2423== Thread 7:
==2423== Invalid read of size 4
==2423==    at 0x4D7C89A: Digikam::DImg::DImg(QImage const&)
(dimg.cpp:165)
==2423==    by 0x4F54D7C: Digikam::PreviewLoadingTask::execute()
(previewtask.cpp:288)
==2423==    by 0x4F44204: Digikam::LoadSaveThread::run()
(loadsavethread.cpp:136)
==2423==    by 0x4F6BD57:
Digikam::DynamicThread::DynamicThreadPriv::run()
(dynamicthread.cpp:186)
==2423==    by 0x6A1246A: QThreadPoolThread::run() (in
/usr/lib/i386-linux-gnu/libQtCore.so.4.8.2)
==2423==    by 0x6A1FEAF: QThreadPrivate::start(void*) (in
/usr/lib/i386-linux-gnu/libQtCore.so.4.8.2)
==2423==    by 0x96CCD4B: start_thread (pthread_create.c:308)
==2423==    by 0x6F34DDD: clone (clone.S:130)
==2423==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==2423== 


This is on digikam 3.5 build from git with opencv 2.4.5 built from
opencv repository.

Build date: Sep 30 2013 (target: debugfull)
Rev.: 04ec199e75e8a6bc01f29bac3a9476e3b72960e0

Components :

digiKam version 3.5.0
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Pgf: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 supports XMP metadata: Yes
LibCImg: 130
LibEigen: 3.0.5
LibExiv2: 0.22
LibJPEG: 80
LibJasper: 1.900.1
LibKDE: 4.11.1
LibKExiv2: 2.3.1
LibKGeoMap: 2.0.0
LibKdcraw: 2.3.1
LibLCMS: 2040
LibLensFun: 0.2.7-0
LibPGF: 6.12.24 - external shared library
LibPNG: 1.2.46
LibQt: 4.8.2
LibRaw: 0.15.4
LibTIFF: LIBTIFF, Version 3.9.5 Copyright (c) 1988-1996 Sam Leffler
Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble Widget: 0.16.1 (stable version)
Parallelized PGF codec: No
Parallelized demosaicing: Yes
RawSpeed codec support: Yes
Database backend: QMYSQL
Database internal server: No
Kipi-Plugins: 3.5.0
LibGphoto2: 2.4.14
LibKface: 3.0.0
LibKipi: 2.1.0
LibOpenCV: 2.4.5


digiKam version 3.5.0
Images: 
BMP: 5
GIF: 56
JP2: 16
JPG: 22730
PNG: 130
RAW-CR2: 3432
XCF: 209
total: 26578
: 
: 
Videos: 
AVI: 29
MOV: 185
MP4: 43
MPEG: 2
WMV: 1
total: 260
: 
: 
Total Items: 26838
Albums: 2749
Tags: 492
Database backend: QMYSQL
Database internal server: No





Greatings,

Eric









Le jeudi 22 août 2013 à 10:10 +0000, Alberto a écrit :
> https://bugs.kde.org/show_bug.cgi?id=323888
> 
>             Bug ID: 323888
>            Summary: Face recognition makes digikam fill all the available
>                     memory
>     Classification: Unclassified
>            Product: digikam
>            Version: 3.3.0
>           Platform: Fedora RPMs
>                 OS: Linux
>             Status: UNCONFIRMED
>           Severity: major
>           Priority: NOR
>          Component: Face Management
>           Assignee: digikam-devel at kde.org
>           Reporter: alberto.ferrante at tiscali.it
> 
> When recognizing faces on large albums, after a while the recognition.db starts
> growing a lot, up to 200-something MB. After that size face recognition becomes
> impossible: for each modification, while digikam updates the face DB, it fills
> all available memory (28GB on my system, including swap space) and then it gets
> terminated by the OS.
> 
> Reproducible: Always
> 
> Steps to Reproduce:
> 1. Use a large set of photos (15-20'000 in my case)
> 2. Perform face detection
> 3. Tag people
> 4. Perform face recognition
> Actual Results:  
> The program starts filling all available memory and gets terminated by the
> operating system
> 
> Expected Results:  
> Update the face db and recognize faces
>