[Digikam-devel] Collection Scanning of Specific Directories

[Todd Goodman]

* Marcel Wiesweg <marcel.wiesweg at gmx.de> [100624 11:51]:
> > Hi,
> > 
> > I'm having problems with 1.3.0 (built from SVN.)
> > 
> > I have a large number of pictures spread out over a number of
> > directories.
> 
> 10^6? 10^7?

About 5.5million

> 
> 
> > 
> > When Digikam starts up with a new DB it wants to scan the entire
> > collection.
> > 
> > If I allow it to do so it performs the scan (over many hours) and then
> > when it looks like it's completed the scan, it gets a realloc (memory
> > corruption) error and hangs.
> > 
> > I have to kill Digikam and the next time I restart it still wants to
> > rescan everything.
> > 
> > If I cancel out of the collection scan dialog box then it ends up with
> > a QLIST out of memory error.
> 
> Can you give a backtrace? Would also be nice to have a backtrace for the first 
> crash, but I can understand that is not done in a minute.

Yes for the QList out of memory problem if I let the scan take place
without cancelling the dialog box.  It's below.

I haven't been able to reproduce the realloc() problem

> 
> > 
> > I was wondering if I could either:
> > 
> > 1)  Just scan each directory individually so as not to run out of memory
> > 
> > or
> > 
> > 2)  Perform some kind of incremental scan so that if it crashes during
> > the initial scan I could restart and pic up where it left off
> > (eventually getting past the out of memory problems.)
> 
> The solution should be not to get out of memory at all ;-)

Yes, that would be ideal.  :-)

Thank you,

Todd

Application: digiKam (digikam), signal: Aborted
[Current thread is 0 (LWP 27729)]

Thread 2 (Thread 0xb27a9b90 (LWP 27731)):
[KCrash Handler]
#5  0xffffe424 in __kernel_vsyscall ()
#6  0xb5445c41 in raise () from /lib/libc.so.6
#7  0xb5447428 in abort () from /lib/libc.so.6
#8  0xb56d08f5 in qt_message_output () from /usr/lib/qt4/libQtCore.so.4
#9  0xb56d09aa in qFatal () from /usr/lib/qt4/libQtCore.so.4
#10 0xb56fd8c0 in QListData::realloc () from /usr/lib/qt4/libQtCore.so.4
#11 0xb56fd9a7 in QListData::append () from /usr/lib/qt4/libQtCore.so.4
#12 0xb57b9b57 in ?? () from /usr/lib/qt4/libQtCore.so.4
#13 0x084f30e8 in ?? ()
#14 0x0000002b in ?? ()
#15 0xb57b65cc in QCoreApplication::postEvent () from /usr/lib/qt4/libQtCore.so.4
#16 0xb57b667a in QCoreApplication::postEvent () from /usr/lib/qt4/libQtCore.so.4
#17 0xb57c74d8 in ?? () from /usr/lib/qt4/libQtCore.so.4
#18 0x085a8ce0 in ?? ()
#19 0x80196010 in ?? ()
#20 0x0859f690 in ?? ()
#21 0x0000000b in ?? ()
#22 0xb57c76db in QMetaObject::activate () from /usr/lib/qt4/libQtCore.so.4
#23 0xb57c7d44 in QMetaObject::activate () from /usr/lib/qt4/libQtCore.so.4
#24 0xb6da7980 in Digikam::DatabaseWatch::imageChange (this=0x859f690, _t1=@0x85a81d4, _t2=@0x85a81d8, _t3=@0xb27a915c) at /root/digikam-svn/graphics/digikam/build/digikam/moc_databasewatch.cpp:193
#25 0xb6da8152 in Digikam::DatabaseWatch::sendImageChange (this=0x859f690, cset=@0xb27a915c) at /root/digikam-svn/graphics/digikam/libs/database/databasewatch.cpp:241
#26 0xb6daea51 in Digikam::DatabaseBackendPrivate::sendToWatch (this=0x85a2058, changeset=@0xb27a915c) at /root/digikam-svn/graphics/digikam/libs/database/databasebackend_p.h:59
#27 0xb6daeb4a in Digikam::DatabaseBackendPrivate::ChangesetContainer<Digikam::ImageChangeset>::sendOut (this=0x85a20b8) at /root/digikam-svn/graphics/digikam/libs/database/databasebackend_p.h:94
#28 0xb6daec34 in Digikam::DatabaseBackendPrivate::transactionFinished (this=0x85a2058) at /root/digikam-svn/graphics/digikam/libs/database/databasebackend_p.h:112
#29 0xb70f4876 in Digikam::DatabaseCoreBackend::commitTransaction (this=0x85a05e0) at /root/digikam-svn/graphics/digikam/libs/database/databasecorebackend.cpp:1151
#30 0xb6daf53a in ~DatabaseTransaction (this=0xb27a9274) at /root/digikam-svn/graphics/digikam/libs/database/databasetransaction.cpp:57
#31 0xb6d932c6 in Digikam::CollectionScanner::completeScan (this=0xb27a92f8) at /root/digikam-svn/graphics/digikam/libs/database/collectionscanner.cpp:277
#32 0x08335a0f in Digikam::ScanController::run (this=0x8588f60) at /root/digikam-svn/graphics/digikam/digikam/scancontroller.cpp:541
#33 0xb56d7fc1 in ?? () from /usr/lib/qt4/libQtCore.so.4
#34 0x08588f60 in ?? ()
#35 0x00000000 in ?? ()

Thread 1 (Thread 0xb3bae700 (LWP 27729)):
#0  0xffffe424 in __kernel_vsyscall ()
#1  0xb567e8d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0xb56d739a in ?? () from /usr/lib/qt4/libQtCore.so.4
#3  0x0859fd80 in ?? ()
#4  0x0859fd68 in ?? ()
#5  0x00000000 in ?? ()