[Digikam-devel] extragear/graphics/digikam

[Gilles Caulier]

Marcel,

Perhaps we can try to mesure performance of thumbs DB using mysql Qt4
plugin ? This DB schema is simple and can be ported easily...

What do you think ?

Gilles


2009/6/27 Andi Clemens <andi.clemens at gmx.net>:
> SVN commit 988322 by aclemens:
>
> Removing this code again, it will not make things faster.
>
> I did a lot of I/O performance testing now and at least with our sqlite3
> solution, we really have reached the limit, it is just not performing
> well on bigger collections.
>
> My thumbsDB is 970MB big, and when the disk cache has not been already
> filled, querying the database is painfully slow.
> This is not digiKam's fault, it can be easily reproduced in the sqlite3
> command line tool.
>
> as root:
> sync
> echo 1 > /proc/sys/vm/drop_caches
>
> Now your disk cache is cleared, as if the system just started up.
> Run a simple query on a bigger thumbsDB like:
>
> select count(*) from thumbnails where type = 0;
>
> Even though this query delivers no results, it takes up to 20 seconds to
> completely parse the table.
>
> When used in digiKam, the application is not responding at all during
> this operation, you get the impression that digiKam just hung up.
>
> I have a large RAM here (4GB), but I don't know what happens if you run
> digiKam 1.0.0 on an average system with 512MB or 1GB RAM.
>
> For me the only solution is to go with embedded MySQL or even a real
> MySQL server solution.
> Because now we don't have just one, but two rather big databases.
>
> CCMAIL:digikam-devel at kde.org
>
>  M  +0 -22     libs/database/thumbnaildb.cpp
>  M  +0 -1      libs/database/thumbnaildb.h
>  M  +0 -18     utilities/batch/batchthumbsgenerator.cpp
>
>
> --- trunk/extragear/graphics/digikam/libs/database/thumbnaildb.cpp #988321:988322
> @@ -150,28 +150,6 @@
>     return filePaths;
>  }
>
> -QHash<QString, int> ThumbnailDB::getInvalidFilePaths()
> -{
> -    QSqlQuery query;
> -    query = d->db->prepareQuery(QString("SELECT path, id "
> -                                        "FROM FilePaths "
> -                                        "   INNER JOIN Thumbnails ON FilePaths.thumbId=Thumbnails.id "
> -                                        "WHERE type BETWEEN %1 AND %2;")
> -                                .arg(DatabaseThumbnail::UndefinedType)
> -                                .arg(DatabaseThumbnail::NoThumbnail));
> -
> -    if (!d->db->exec(query))
> -        return QHash<QString, int>();
> -
> -    QHash <QString, int> filePaths;
> -
> -    while (query.next())
> -    {
> -        filePaths[query.value(0).toString()] = query.value(1).toInt();
> -    }
> -    return filePaths;
> -}
> -
>  void ThumbnailDB::insertUniqueHash(const QString &uniqueHash, int fileSize, int thumbId)
>  {
>     d->db->execSql("REPLACE INTO UniqueHashes (uniqueHash, fileSize, thumbId) VALUES (?,?,?)",
> --- trunk/extragear/graphics/digikam/libs/database/thumbnaildb.h #988321:988322
> @@ -104,7 +104,6 @@
>     void replaceThumbnail(const DatabaseThumbnailInfo &info);
>
>     QHash<QString, int> getValidFilePaths();
> -    QHash<QString, int> getInvalidFilePaths();
>
>  private:
>
> --- trunk/extragear/graphics/digikam/utilities/batch/batchthumbsgenerator.cpp #988321:988322
> @@ -121,26 +121,8 @@
>
>  #ifdef USE_THUMBS_DB
>
> -    /* Not working at the moment, it seems that the NoThumbnail type is not (yet) used, so no results
> -     * are returned from getInvalidFilePaths()
> -
>     if (!d->rebuildAll)
>     {
> -        QHash<QString, int> filePaths = ThumbnailDatabaseAccess().db()->getInvalidFilePaths();
> -
> -        QStringList::iterator it = d->allPicturesPath.begin();
> -        while (it != d->allPicturesPath.end())
> -        {
> -            if (!filePaths.contains(*it))
> -                it = d->allPicturesPath.erase(it);
> -            else
> -                ++it;
> -        }
> -    }
> -    */
> -
> -    if (!d->rebuildAll)
> -    {
>         QHash<QString, int> filePaths = ThumbnailDatabaseAccess().db()->getValidFilePaths();
>
>         QStringList::iterator it = d->allPicturesPath.begin();
> _______________________________________________
> Digikam-devel mailing list
> Digikam-devel at kde.org
> https://mail.kde.org/mailman/listinfo/digikam-devel
>