<br><br><div class="gmail_quote">On Fri, Jul 13, 2012 at 3:06 PM, Phalgun Guduthur <span dir="ltr"><<a href="mailto:phalgun.guduthur@gmail.com" target="_blank">phalgun.guduthur@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div class="gmail_quote">On Fri, Jul 13, 2012 at 1:02 PM, Bart Cerneels <span dir="ltr"><<a href="mailto:bart.cerneels@gmail.com" target="_blank">bart.cerneels@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On Wed, Jul 4, 2012 at 4:37 PM, Phalgun Guduthur<br>
<<a href="mailto:phalgun.guduthur@gmail.com" target="_blank">phalgun.guduthur@gmail.com</a>> wrote:<br>
</div><div>> Hey All!<br>
><br>
> An update on my GSoC project : Semantic Collection for Amarok<br>
><br>
> I have figured out a way to reuse a lot of existing Amarok code. So it made<br>
> a part of the code I wrote till now redundant.<br>
><br>
> I will be using MemoryCollection as other collections do. The Meta QMaps<br>
> will be constructed using queried results from Nepomuk. This will make all<br>
> the collections consistent and hopefully avoids a lot of bugs too.<br>
> Similarly, I will also be reusing the MemoryQueryMaker and the other support<br>
> classes that go with it.<br>
><br>
> Right now, the Nepomuk Queries are taking too long. So I will have to run<br>
> them in the background using Threadweaver jobs.<br></div></blockquote></div></div></blockquote><div><br>There are optimization that can be done, but you're loading a lot of tracks during startup, it should be done in another thread.<br>
<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
> Apart from this, the NepomukCollection looks complete with the Meta classes<br>
> implemented.<br>
<br>
</div>This worries me a little bit. MemoryCollection obviously uses a lot of<br>
memory. Where possible we should avoid having to place the complete<br>
set of tracks in memory and rather rely on the design goals of<br>
QueryMaker to create them on demand. I would think Nepomuk is the<br>
perfectly suited for that. Can you explain what blocked you and<br>
decided to switch to MemoryCollection?<br>
<span><font color="#888888"><br>
Bart<br>
</font></span></blockquote></div><br></div><div>Oh I haven't checked my memory usage when I ran Amarok Collection with Nepomuk. <div>If all of the MetaMaps are being stored in memory, it should indeed eat up a lot, if we consider collections of size more than 2000 tracks. But then, wouldn't all the existing collections that use MemoryCollection also suffer from the same problem? Say for example the iPodCollection. </div>
</div></blockquote><div><br>My amarok has a memory usage of about 200mb for ~5000 tracks. Not *that* much.<br> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div><br></div><div>I did speak to Edward (dr_lepper) on the possibility of caching resource URI's (These are the unique identifiers of each resource in Nepomuk, just strings) of each track so that they can be looked up quickly. But if this caching is indeed required, I'll have to postpone it for now and work on them after completing other milestones. </div>
</div><div><div><br></div><div>And as to why I wanted to reuse the MemoryCollection, I was going to implement my NepomukQueryMaker on the same lines as MemoryQueryMaker, including the *QueryMakerInternal and other intricacies. Then I realized I could make use of MemoryCollection as other collections and maintain the same consistency among the collections. </div>
</div><div><br></div><div>Now I do start to wonder if constructing tracks on demand, say when a user clicks on the artist or a genre was indeed an option. Both approaches have their pros and cons, 'cos the 'construct at beginning' approach causes a lag only during the bootup of Amarok whereas the 'on demand' approach would cause a lag every time a query is requested. ( time lag for the query to run and enumerate )</div>
</blockquote><div><br>The 'construct at the beginning approach' has a huge initial load time. This can be optimized, and will be eventually, cause other applications (Teleapthy and PIM) need the ability to load large chunks of data in one go, but it should ideally be done in another thread. The other problem with loading everything in one go is that some other application could change the Nepomuk data, and amarok would need to be informed. While the overhead is very small, it's added work.<br>
<br>In contrast, loading everything on demand would cause a minute lag (30 ms on system) per track being loaded. These are numbers based on the current nepomuk-core, with one optimization which is still in the review board.<br>
<br>I obviously prefer the second approach, but I'm okay with whichever approach the Amarok developers choose. I can optimize the required code paths.<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div><br></div><div>Best, </div><span><font color="#888888"><div>Phalgun</div>
</font></span></blockquote></div><br>PS: Please keep me cced, I'm not on the amarok mailing list.<br clear="all"><br>-- <br><span style="color:rgb(192,192,192)">Vishesh Handa</span><br><br>