[KPhotoAlbum] Performance issue: KPA too slow when clicking on "big" supercategories
Jesper K. Pedersen
blackie at blackie.dk
Tue Dec 26 11:05:05 GMT 2006
One of my goals is to push all this onto a worker thread, so it is
backgrounded. Esp md5sum calculation is tedious at when new images are found.
On Sunday 24 December 2006 17:16, Robert L Krawitz wrote:
| From: "Jesper K. Pedersen" <blackie at blackie.dk>
| Date: Sun, 24 Dec 2006 11:32:17 +0100
| Seems like you are right indeed. That just shows how little I use
| subcategories myself.
| I spent the better of an hour profiling it, and came to the
| conclusion that it unfortunately is the algorithms that sucks
| rather than the implementation. In my database with ~7000 images,
| it takes billions of string compares to search for a large enough
| category :-/
| This is probably what I reported about a month ago.
| It is unfortunately too late to do something about that for the
| next release. I do, however, plan to spent some time to make a
| patch level release after the next one, which will include bug
| fixes and optimizations only.
| Another thing I'd like to look at is scanning for images. It takes me
| about 100 seconds on a database of 20000 images the first time I do it
| (succeeding times are very fast -- a fraction of a second). This is
| using reiserfs with a single 250 GB 7200 RPM drive. It's likely that
| different filesystems will yield different -- possibly radically
| different -- results.
| This is due to the fact that every file is being stat()'ed, even if
| it's already in the database. I tried rearranging the code in
| question to not call isReadable() unless the file is not registered in
| the database, but that didn't help. The problem is that QDir always
| calls stat() on every directory entry even if no filters are
| specified. Since we're not going to be able to change Qt, we may want
| to devise our own directory class (or derive a class from QDir) for
| this purpose.
| There's one other issue: people who use RAW+JPEG and choose not to
| index both, or who use cameras that generate thumbnail files, will
| still have to stat all of these files. There may be workarounds, such
| lists of extensions to ignore outright (which of course would break if
| someone actually named a directory foo.thm or the like), or some other
| way to ignore JPEG files if the RAW files are already indexed.
Having trouble finding a given image in your collection containing
thousands of images?
http://www.kphotoalbum.org might be the answer.
More information about the Kphotoalbum