[Tellico-users] Memory consumption issue

[Valerio Ricci]

Hi Robby,

2016-03-22 2:05 GMT+01:00 Robby Stephenson <robby at periapsis.org>:

> Hi Valerio,
>
> On 03/21/2016 05:54 AM, Valerio Ricci wrote:
>
>> I am running tellico 2.3.9 under kubuntu 15.10 and I noticed a memory
>> usage issue when browsing my book collection.
>>
>> I have a book collection of ca. 650 entries and about 220 of them have
>> cover images.
>>
>> The xml file is about 2MB and all the images, stored in a separate folder
>> are about 130MB in total.
>>
>> I made the collection some time ago and I started adding the cover images
>> recently.
>> To see how many of them where missing I made a filter that follows the
>> following criteria: cover field contains "."
>>
>> As expected I end up with my 220 entries listed.
>>
>> The problem is that when I click on the newly created filter the memory
>> usage jumps from 44.2 MB to 2.8 GB.
>>
>> I suspect this is because the images are loaded into memory (and maybe
>> they are also decompressed from their original .jpg format).
>>
>
> They are loaded and scaled, but they shouldn't be held in memory. Or at
> least, that's not my intent. There's an internal cache, too, that should
> help manage the memory usage.
>
While this can be an expected behavior it can be a problem when you have to
>> manage a large collection with many images.
>> I don't have all the images to populate all my 650 entries but making a
>> simple proportion I can infer that it will take over 8 GB of ram.
>>
>
> My book collection is a bit smaller, 46MB or so of jpeg images. When I
> create a filter as you describe, my memory usage does jump up by about 200
> MB, but not into GB as you are experiencing. I'll poke at it some.
>

I want to point out that the memory is filled when I click on the filter
and not before I do. This lets me believe it could be an issue related to
how the pictures are loaded in memory to be displayed on screen. But
obviously it is just an hypothesis.

For the sake of completeness I can also say that all the images are
relatively big, having a resolution around 1500x2500px and are compressed
jpeg files about 400KB to 1MB in size.

Again, I'm not sure about how tellico (or the C++ back-end or whatever)
handles compressed images but it seems like the memory consumption
increases exponentially.

I just made some testing:

Collection with 1 entry, no images -> memory consumption 35 MB
Collection with 1 entry, 1 image (11487x2649px, 3.5 MB jpeg      -> memory
consumption 72.3 MB
Collection with 1 entry, 1 image (7024x4687px, 4.3 MB jpeg    -> memory
consumption 79.6 MB
Collection with 2 entries, 2 images (the above 2 together)   -> memory
consumption 351.7 MB

The two images are respectively at http://i.imgur.com/hYdBawf.jpg and
https://upload.wikimedia.org/wikipedia/commons/4/47/Canonization_2014-_The_Canonization_of_Saint_John_XXIII_and_Saint_John_Paul_II_%2814036894824%29.jpg

The second image is 24MB, but I noticed that when tellico creates the copy
of the file in the collection folder the size is reduced to 4.3MB. I
suppose this is due to a different compression ratio in jpeg. I've tried to
add the original file and the "resized" file just to notice that memory
consumption is not changing.



> Just to confirm your setting, your Image Storage is set to "Store images
> in directory relative to data file" ?
>

Yes it is. Same for the previous testing.


>
> Robby
>

Hope this helps

Valerio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/tellico-users/attachments/20160322/5bc95085/attachment-0001.html>