[KPhotoAlbum] Startup performance

[Robert Krawitz]

On 10/13/20 1:50 AM, Martin Höller wrote:
> Hi!
> 
> Am 12. Okt. 2020 schrieb Robert Krawitz:
> 
>> It really is the startup that's the most visible aspect of all of this.  Again, that either means
>> pretty fundamental changes in the storage backend, a faster XML parser, or maybe a way of bringing
>> up the UI while images are still loading (and since you're often going to want to look at your most
>> recent photos, not even very helpul).  If we stored photos in the database in _reverse_ order of
>> time, though, that might be workable (I'd love to have instant access to my most recent basketball
>> game while the rest of the file is being loaded).
> 
> Just an idea: what about "partitioning" the database. I mean we could
> have a single XML file per year, per decade, per 10k images or whatever.
> That way, loading the more recent images first as Robert suggested would
> be easier.

One of the goals of KPA is for the database to be human readable and editable.  Splitting up the
image file in that way makes the structure more complicated to move/copy and edit.

The decision of a human-readable database certainly makes parallelizing parsing more difficult
Human-readable (with no tools beyond a pager or text editor) files have to be line-delimited plain
text.  In this case, since the file metadata can be arbitrarily long (many categories and members),
we can't require a fixed line length, which is problematic for editing anyway.  So we've set
ourselves a bit of a challenge here.

Breaking the file up would allow for parallel parsing, certainly.

Maybe a way to do it is to store the data in an RDBMS, preferably with the raw storage format
matching the underlying data format so we don't have to do a lot of conversions on the way in, and
provide a tool to convert from/to XML format.  This would also allow for backups in the current
manner.  Saving is at present about twice as fast as startup, not including time to make a backup
(which is a little more than 1/3 of the time to otherwise save the database) and is less
objectionable other than autosave.

Back to day job...