[KPhotoAlbum] Speed up new image load time
Robert Krawitz
rlk at alum.mit.edu
Mon May 29 22:27:49 BST 2017
I've improved loading new images into kpa by batching up the EXIF
information into a single transaction, as I expected.
Some timings, for loading 1133 images:
Old New
20 MP 5:41 0:32
0.2 MP 6:00+ 0:07
Not sure why it took longer to read in the 0.2 MP images than the
full-size ones; probably due to whatever else was going on on my
system. But for the realistic case, I have about 10x speedup
overall.
It appears that there were actually multiple places in the code
reading the EXIF data and storing it in the database, one file at a
time. I found what looked like one, and the time for the 0.2 MP files
dropped to 1:45 or so, so that may have actually been more than one place.
It looks like storing the EXIF data in the database takes about 3
seconds. The next big time consumer is file version detection; if I
turn that off, the total time drops off to about 7 seconds. At that
point, in a realistic scenario, I'd likely be I/O-bound; if I were
loading 3000 images (30 GB, typically), I'd need on the order of
250-300 seconds just to read the data from disk. But if someone were
storing their images on nVME, it might matter.
In any event, this will make loading new images a lot more bearable.
I'm used to having to wait 30 minutes to load images into kpa; this
should get that down to 5'ish (with I/O being the limiting factor).
--
Robert Krawitz <rlk at alum.mit.edu>
*** MIT Engineers A Proud Tradition http://mitathletics.com ***
Member of the League for Programming Freedom -- http://ProgFree.org
Project lead for Gutenprint -- http://gimp-print.sourceforge.net
"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-fast-new-images.patch
Type: application/octet-stream
Size: 8328 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kphotoalbum/attachments/20170529/f005caee/attachment.obj>
More information about the Kphotoalbum
mailing list