[Digikam-devel] [digikam] [Bug 323888] Face recognition makes digikam fill all the available memory (Qt SQlite plugin relevant)
Gilles Caulier
caulier.gilles at gmail.com
Thu Jul 10 11:11:26 BST 2014
https://bugs.kde.org/show_bug.cgi?id=323888
--- Comment #89 from Gilles Caulier <caulier.gilles at gmail.com> ---
I progressed well in this entry about investigations.
Here under Mageia4, i use Qt 4.8.2 and libsqlite 3.8.0.2
I written a small CLI program to check digiKam database init with valgrind.
Installing qt and sqlite debug package, i can identify now where memory is
leak.
In fact it's a know problem already reported in this entry :
https://bugs.kde.org/show_bug.cgi?id=329697
... where users complain that digiKam crash into libsqlite memory management
functions.
So now with valgrind, we can see where memory is leak on my computer :
==32332== 7,680 bytes in 6 blocks are possibly lost in loss record 2,049 of
2,066
==32332== at 0x4C266ED: malloc (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332== by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332== by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332== by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332== by 0x1DDE5DCA: pcache1Alloc (sqlite3.c:36875)
==32332== by 0x1DDE6021: pcache1Fetch (sqlite3.c:36959)
==32332== by 0x1DDE08E8: sqlite3PcacheFetch (sqlite3.c:36286)
==32332== by 0x1DE0B255: sqlite3PagerAcquire (sqlite3.c:43632)
==32332== by 0x1DE0B50C: btreeGetPage (sqlite3.c:51175)
==32332== by 0x1DE0D701: getAndInitPage (sqlite3.c:51230)
==32332== by 0x1DE0D9EE: moveToRoot (sqlite3.c:53987)
==32332== by 0x1DE0F0E4: sqlite3BtreeMovetoUnpacked (sqlite3.c:54204)
==32332==
==32332== 9,000 bytes in 45 blocks are possibly lost in loss record 2,056 of
2,066
==32332== at 0x4C266ED: malloc (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332== by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332== by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332== by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332== by 0x1DDE3450: sqlite3DbMallocRaw (sqlite3.c:19406)
==32332== by 0x1DDF06DB: exprDup (sqlite3.c:76262)
==32332== by 0x1DE2EEA3: sqlite3Parser (sqlite3.c:76350)
==32332== by 0x1DE34459: sqlite3RunParser (sqlite3.c:115140)
==32332== by 0x1DE34A81: sqlite3Prepare (sqlite3.c:96189)
==32332== by 0x1DE34D64: sqlite3LockAndPrepare (sqlite3.c:96280)
==32332== by 0x1DE34DF4: sqlite3_prepare (sqlite3.c:96344)
==32332== by 0x1DE34EAE: sqlite3InitCallback (sqlite3.c:95658)
==32332==
==32332== 9,564 (1,920 direct, 7,644 indirect) bytes in 20 blocks are
definitely lost in loss record 2,058 of 2,066
==32332== at 0x4C26BF5: operator new(unsigned long) (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332== by 0x5C39FA2: ??? (in /usr/lib64/libQtCore.so.4.8.6)
==32332== by 0x5C34567: QFactoryLoader::updateDir(QString const&,
QSettings&) (in /usr/lib64/libQtCore.so.4.8.6)
==32332== by 0x5C364C6: QFactoryLoader::update() (in
/usr/lib64/libQtCore.so.4.8.6)
==32332== by 0x5C366E9: QFactoryLoader::QFactoryLoader(char const*, QString
const&, Qt::CaseSensitivity) (in /usr/lib64/libQtCore.so.4.8.6)
==32332== by 0x6456754: ??? (in /usr/lib64/libQtGui.so.4.8.6)
==32332== by 0x6456BBF: ??? (in /usr/lib64/libQtGui.so.4.8.6)
==32332== by 0x6458891: ??? (in /usr/lib64/libQtGui.so.4.8.6)
==32332== by 0x6459DE7: QImageReader::read(QImage*) (in
/usr/lib64/libQtGui.so.4.8.6)
==32332== by 0x6459FB3: QImageReader::read() (in
/usr/lib64/libQtGui.so.4.8.6)
==32332== by 0x644E5D8: operator>>(QDataStream&, QImage&) (in
/usr/lib64/libQtGui.so.4.8.6)
==32332== by 0x64676D8: operator>>(QDataStream&, QPixmap&) (in
/usr/lib64/libQtGui.so.4.8.6)
==32332==
==32332== 15,288 bytes in 39 blocks are possibly lost in loss record 2,059 of
2,066
==32332== at 0x4C266ED: malloc (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332== by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332== by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332== by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332== by 0x1DDE3450: sqlite3DbMallocRaw (sqlite3.c:19406)
==32332== by 0x1DE31C06: sqlite3Parser (sqlite3.c:83487)
==32332== by 0x1DE34459: sqlite3RunParser (sqlite3.c:115140)
==32332== by 0x1DE34A81: sqlite3Prepare (sqlite3.c:96189)
==32332== by 0x1DE34D64: sqlite3LockAndPrepare (sqlite3.c:96280)
==32332== by 0x1DE34DF4: sqlite3_prepare (sqlite3.c:96344)
==32332== by 0x1DE34EAE: sqlite3InitCallback (sqlite3.c:95658)
==32332== by 0x1DE35258: sqlite3_exec (sqlite3.c:92433)
==32332==
==32332== 38,400 bytes in 30 blocks are possibly lost in loss record 2,063 of
2,066
==32332== at 0x4C266ED: malloc (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332== by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332== by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332== by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332== by 0x1DDE5DCA: pcache1Alloc (sqlite3.c:36875)
==32332== by 0x1DDE6021: pcache1Fetch (sqlite3.c:36959)
==32332== by 0x1DDE08E8: sqlite3PcacheFetch (sqlite3.c:36286)
==32332== by 0x1DE0B255: sqlite3PagerAcquire (sqlite3.c:43632)
==32332== by 0x1DE0B50C: btreeGetPage (sqlite3.c:51175)
==32332== by 0x1DE0D701: getAndInitPage (sqlite3.c:51230)
==32332== by 0x1DE0D779: moveToChild (sqlite3.c:53869)
==32332== by 0x1DE0D867: moveToLeftmost (sqlite3.c:54049)
==32332==
==32332== 64,008 bytes in 1 blocks are possibly lost in loss record 2,064 of
2,066
==32332== at 0x4C266ED: malloc (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332== by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332== by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332== by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332== by 0x1DDE9AE1: setupLookaside.part.235 (sqlite3.c:116154)
==32332== by 0x1DE5D2A6: openDatabase (sqlite3.c:93720)
==32332== by 0x2BA5A81E: ??? (in
/usr/lib64/qt4/plugins/sqldrivers/libqsqlite.so)
==32332== by 0x6E8CD60: QSqlDatabase::open() (in
/usr/lib64/libQtSql.so.4.8.6)
==32332== by 0x5173AA8:
Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&)
(databasecorebackend.cpp:214)
==32332== by 0x51733D0:
Digikam::DatabaseCoreBackendPrivate::databaseForThread()
(databasecorebackend.cpp:120)
==32332== by 0x5176861:
Digikam::DatabaseCoreBackend::open(Digikam::DatabaseParameters const&)
(databasecorebackend.cpp:769)
==32332== by 0x572FBD1:
Digikam::DatabaseAccess::checkReadyForUse(Digikam::InitializationObserver*)
(databaseaccess.cpp:286)
==32332==
==32332== 64,008 bytes in 1 blocks are possibly lost in loss record 2,065 of
2,066
==32332== at 0x4C266ED: malloc (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332== by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332== by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332== by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332== by 0x1DDE9AE1: setupLookaside.part.235 (sqlite3.c:116154)
==32332== by 0x1DE5D2A6: openDatabase (sqlite3.c:93720)
==32332== by 0x2BA5A81E: ??? (in
/usr/lib64/qt4/plugins/sqldrivers/libqsqlite.so)
==32332== by 0x6E8CD60: QSqlDatabase::open() (in
/usr/lib64/libQtSql.so.4.8.6)
==32332== by 0x5173AA8:
Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&)
(databasecorebackend.cpp:214)
==32332== by 0x51733D0:
Digikam::DatabaseCoreBackendPrivate::databaseForThread()
(databasecorebackend.cpp:120)
==32332== by 0x517939B: Digikam::DatabaseCoreBackend::getQuery()
(databasecorebackend.cpp:1512)
==32332== by 0x5178FD0: Digikam::DatabaseCoreBackend::prepareQuery(QString
const&) (databasecorebackend.cpp:1467)
==32332==
==32332== 64,008 bytes in 1 blocks are possibly lost in loss record 2,066 of
2,066
==32332== at 0x4C266ED: malloc (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332== by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332== by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332== by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332== by 0x1DDE9AE1: setupLookaside.part.235 (sqlite3.c:116154)
==32332== by 0x1DE5D2A6: openDatabase (sqlite3.c:93720)
==32332== by 0x2BA5A81E: ??? (in
/usr/lib64/qt4/plugins/sqldrivers/libqsqlite.so)
==32332== by 0x6E8CD60: QSqlDatabase::open() (in
/usr/lib64/libQtSql.so.4.8.6)
==32332== by 0x5173AA8:
Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&)
(databasecorebackend.cpp:214)
==32332== by 0x51733D0:
Digikam::DatabaseCoreBackendPrivate::databaseForThread()
(databasecorebackend.cpp:120)
==32332== by 0x5176861:
Digikam::DatabaseCoreBackend::open(Digikam::DatabaseParameters const&)
(databasecorebackend.cpp:769)
==32332== by 0x5181716:
Digikam::ThumbnailDatabaseAccess::checkReadyForUse(Digikam::InitializationObserver*)
(thumbnaildatabaseaccess.cpp:216)
==32332==
==32332== LEAK SUMMARY:
==32332== definitely lost: 8,000 bytes in 690 blocks
==32332== indirectly lost: 9,358 bytes in 81 blocks
==32332== possibly lost: 391,691 bytes in 1,652 blocks
==32332== still reachable: 336,972 bytes in 4,708 blocks
==32332== suppressed: 0 bytes in 0 blocks
==32332== Reachable blocks (those to which a pointer was found) are not shown.
==32332== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==32332==
==32332== For counts of detected and suppressed errors, rerun with: -v
==32332== ERROR SUMMARY: 957 errors from 957 contexts (suppressed: 2 from 2)
[gilles at localhost tests]$
So i suspect this entry to be fully relevant of Sqlite implementation as it's
have already reported as UPSTREAM #329697
Note the difference : #329697 is about libsqlite:: sqlite3MemCompare() when
this file is about libsqlite::sqlite3MemMalloc().
#329697 have been fixed with a patch release of libsqlite has it's explained in
this Ubuntu report :
https://bugs.launchpad.net/ubuntu/+source/sqlite3/+bug/1317449
- Packagers and Users must update libsqlite to last stable 3.8.5
(http://www.sqlite.org/news.html)
- Users must try to run FaceManagement again and look if memory leak still
present or not.
- If yes, this file must be reported as UPSTREAM to SQLite bug tracker.
Giles Caulier
--
You are receiving this mail because:
You are the assignee for the bug.
More information about the Digikam-devel
mailing list