optimization proposal -- playlist stored in the DB

[Ovidiu Gheorghioiu]

Hi guys,

I was looking for opportunities to optimize Amarok, and there's minor
things here and there, but one thing was sticking out like a store
thumb, the playlist's XML format takes a lot of time to load an save
on a huge playlist.

Take a look at these callgrind snapshots:

http://ovidiug.googlepages.com/loadxml.png
http://ovidiug.googlepages.com/savexml.png

, a full 38% of the time is taken by the xml operations! And this is
after the recent MetaBundle::exactColumnName optimization that shaved
10% or so off of the total time. And it wasn't even my full collection
which is about double that.

Let me qualify the run: I started up Amarok, it loaded a playlist of
4800 items, then I did some searches, reloaded the playlist from a
smart playlist, did a couple of searches, removed an item, scrolled
around a little bit, then quit amarok. saveXml's happened on the
playlist reload and the removal, to support undo, and again on
quitting, and I think somewhere in between on the autosave timer.

Now you might consider this skewed, because in normal usage a huge
chunk of time is spent just playing songs, but that doesn't really
matter because I'm not waiting for anything, interactivity is not a
concern. On the other hand, after loading a smart playlist, or
starting up, I am waiting, and that's what I'm trying to improve.

Also, personally I use and switch between smart playlists a lot. E.g.
my girlfriend doesn't understand Romanian, and when I'm with her I
usually exclude the Romanian songs that are in the playlist. Well,
except the numa-numa-yeah song :)

I think having the playlist in the DB would eliminate these issues.
Loading would definitely be faster, as can be seen from the smart
playlist loading time (1.1%in SqlLoader) -- and the smart playlist is
actually a fairly complicated query. Saving will likely be much faster
too, and most importantly it can be done incrementally for things like
undo or autosave.

And there are other fringe benefits, too. For example, another 6% was
spend loading song ratings one by one as I was scrolling, on the first
instance of the playlist which came from XML -- these can probably be
stored right in the playlist table.

So, what do you guys think? I'd be willing to do a significant amount
of the work, but I would likey need / use some help from somebody
who's familiar with the DB. And also some testing on the other two
db's, which I don't have installed or configured.

Regards,
Ovy