[KPhotoAlbum] Speaking of performance...

[Robert Krawitz]

On Thu, 31 Jan 2019 11:12:43 +0100, Manfred Usselmann wrote:
> Am 27.01.2019 16:28, schrieb Robert Krawitz: 
>
>> We really do need to attack the startup performance somehow, for
>> people (like myself) with big collections. I currently have a little
>> over 300,000 shots in my collection, and depending upon how well we do
>> in the postseason that could increase by another 10% over the next few
>> months. It takes about 13 seconds for kpa to start up, largely due to
>> the XML parsing. Since the XML file is "only" 57 MB, it's clearly not
>> I/O-limited.
>> 
>> I understand (and agree with) the desire for a readable and editable
>> file format. I've fixed things up myself on occasion. But I don't
>> want to pay that kind of startup price every time.
>> 
>> What I'm thinking in terms of is to save the file in two formats, a
>> fast format (which could be an SQL database, a binary serialization,
>> or such) and the XML format. The fast format would have an embedded
>> timestamp; if the XML file were newer, it would be used instead, or
>> the user would be prompted to choose which.
>> 
>> Autosave would save only the fast format (possibly only a delta, but
>> that would likely be quite difficult). Full save would save both
>> formats; if we were really clever, we might be able to parallelize the
>> two operations.
>
> Just a quick suggestion, don't know if it's feasable: 
>
> Store a binary serialization, which can be loaded fast, at the beginning
> of the xml together with a hash key of the xml. If the key is still
> valid use this part for fast loading else the editable xml. 

In order to be valid XML (needed to be editable by many editors), it
would have to be stored base64-encoded or similar, which would negate
at least some of the performance gain.
-- 
Robert Krawitz                                     <rlk at alum.mit.edu>

***  MIT Engineers   A Proud Tradition   http://mitathletics.com  ***
Member of the League for Programming Freedom  --  http://ProgFree.org
Project lead for Gutenprint   --    http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton