GSoC update

[Vishesh Handa]

On Fri, Jul 13, 2012 at 3:06 PM, Phalgun Guduthur <
phalgun.guduthur at gmail.com> wrote:

> On Fri, Jul 13, 2012 at 1:02 PM, Bart Cerneels <bart.cerneels at gmail.com>wrote:
>
>> On Wed, Jul 4, 2012 at 4:37 PM, Phalgun Guduthur
>> <phalgun.guduthur at gmail.com> wrote:
>> > Hey All!
>> >
>> > An update on my GSoC project : Semantic Collection for Amarok
>> >
>> > I have figured out a way to reuse a lot of existing Amarok code. So it
>> made
>> > a part of the code I wrote till now redundant.
>> >
>> > I will be using MemoryCollection as other collections do. The Meta QMaps
>> > will be constructed using queried results from Nepomuk. This will make
>> all
>> > the collections consistent and hopefully avoids a lot of bugs too.
>> > Similarly, I will also be reusing the MemoryQueryMaker and the other
>> support
>> > classes that go with it.
>> >
>> > Right now, the Nepomuk Queries are taking too long. So I will have to
>> run
>> > them in the background using Threadweaver jobs.
>>
>
There are optimization that can be done, but you're loading a lot of tracks
during startup, it should be done in another thread.


>  > Apart from this, the NepomukCollection looks complete with the Meta
>> classes
>> > implemented.
>>
>> This worries me a little bit. MemoryCollection obviously uses a lot of
>> memory. Where possible we should avoid having to place the complete
>> set of tracks in memory and rather rely on the design goals of
>> QueryMaker to create them on demand. I would think Nepomuk is the
>> perfectly suited for that. Can you explain what blocked you and
>> decided to switch to MemoryCollection?
>>
>> Bart
>>
>
> Oh I haven't checked my memory usage when I ran Amarok Collection with
> Nepomuk.
> If all of the MetaMaps are being stored in memory, it should indeed eat up
> a lot, if we consider collections of size more than 2000 tracks. But then,
> wouldn't all the existing collections that use MemoryCollection also suffer
> from the same problem? Say for example the iPodCollection.
>

My amarok has a memory usage of about 200mb for ~5000 tracks. Not *that*
much.


>
> I did speak to Edward (dr_lepper) on the possibility of caching resource
> URI's (These are the unique identifiers of each resource in Nepomuk, just
> strings) of each track so that they can be looked up quickly. But if this
> caching is indeed required, I'll have to postpone it for now and work on
> them after completing other milestones.
>
> And as to why I wanted to reuse the MemoryCollection, I was going to
> implement my NepomukQueryMaker on the same lines as MemoryQueryMaker,
> including the *QueryMakerInternal and other intricacies. Then I realized I
> could make use of MemoryCollection as other collections and maintain the
> same consistency among the collections.
>
> Now I do start to wonder if constructing tracks on demand, say when a user
> clicks on the artist or a genre was indeed an option. Both approaches have
> their pros and cons, 'cos the 'construct at beginning' approach causes a
> lag only during the bootup of Amarok whereas the 'on demand' approach would
> cause a lag every time a query is requested. ( time lag for the query to
> run and enumerate )
>

The 'construct at the beginning approach' has a huge initial load time.
This can be optimized, and will be eventually, cause other applications
(Teleapthy and PIM) need the ability to load large chunks of data in one
go, but it should ideally be done in another thread. The other problem with
loading everything in one go is that some other application could change
the Nepomuk data, and amarok would need to be informed. While the overhead
is very small, it's added work.

In contrast, loading everything on demand would cause a minute lag (30 ms
on system) per track being loaded. These are numbers based on the current
nepomuk-core, with one optimization which is still in the review board.

I obviously prefer the second approach, but I'm okay with whichever
approach the Amarok developers choose. I can optimize the required code
paths.


> Best,
> Phalgun
>

PS: Please keep me cced, I'm not on the amarok mailing list.

-- 
Vishesh Handa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/amarok-devel/attachments/20120713/dbb50fe9/attachment.html>