<div dir="ltr"><span style="color:rgb(80,0,80)">On Thu, May 9, 2013 at 3:21 PM, Ignacio Serantes </span><span dir="ltr" style="color:rgb(80,0,80)"><<a href="mailto:kde@aynoa.net" target="_blank">kde@aynoa.net</a>></span><span style="color:rgb(80,0,80)"> wrote:</span><br>
<div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">
<div class="gmail_extra"><div class="gmail_quote"><div class="im">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br></div></blockquote></div><div>Yeah. I was just thinking about the file indexer. If you incorporate the web-miner, then it would be the web-miner's responsibility to update the full text index (or nie:plainTextIndex) for the resource who it attaches resources to. <br>
</div></div></div></div></blockquote><div><br></div><div style>You can't expect this. Nepomuk it's the middleware so this is a Nepomuk's job and you can't trust that your clients follow right logic because logic could change. For example, nepomuk webminer could be abandoned or some users could still use Bangarang that seems abandoned. Even worse, some distros could be packaging and old version without your required changes, for example openSUSE still is packaging webminer but the old version.<br>
<br>You can use clients to process data (sorting, filtering, etc...) but all business logic must be written in the middleware and never in the clients. </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>
</div><div class="im"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr">
<div><br></div><div>I read some comments about Nepomuk it's not a data store that concern me. I'm using Nepomuk as a data store extensively, tags, comments, rating and other stuff using Nepoogle, because without it Nepomuk less more useful for me and, honestly, I can't understand it without this functionality. Some time ago some effort was spend explaining that Nepomuk it's not a file search so don't transform it in a resources search tool.</div>
</div></blockquote><div><br></div></div><div>I was hoping it would be more of a full text + structured data storage tool. It is not a place to store plain text such as the file's contents and expect to get them back exactly as they were.</div>
</div></div></div></blockquote><div style><br>No, but you want to store any kind of metadata and retrieve information.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>I'm worried that if we do not put all the plain text in one place we cannot reliably solve the searching problem. Users mostly just provide plain text when searching. They just provide words "blah blah blah". Most users would not know about higher semantics such as "hasTag:" or "performer:". That's only for more technical users.<br>
</div></div></div></div></blockquote><div><br></div><div style>The main problem here is the lack of a proper search interface. There was one in KDE 4.5, I learned about hastag and other stuff that days, but was removed. Maybe with this year's GSOC we have one.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>Doing a search through for x words leads 2x unions which is very very slow. In the case I highlighted in the first email, it takes a good 26 seconds on my system. That's just too slow. The user expects feedback in MAX a second. Generally, even less than a second.<br>
</div></div></div></div></blockquote><div><br></div><div style>Yes, I know it but where are you doing this search? In Dolphin or KRunner? Then maybe something like mail:"my search text" could be useful to optimize this kind of search. Consider that most of the time you can optimize queries if you guide user but, it's true, full text search always will be slow :). In applications, like KMail, you can always add right filters to optimize the search.</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div></div><div>
<br>Do you have any suggestions on how to fix that?<br></div></div></div></div></blockquote><div><br></div><div style>Sadly I don't know how to optimize this in triplestore databases, with relational databases you use views and indexes but you can't use it here. Maybe Virtuoso people could help you. </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div> <br></div><div>Additionally, with the query I showed above you also have a problem of stuff like this -<br><br></div><div>res a nmm:MusicPiece .<br></div><div>nmm:MuiscPiece rdfs:comment "Used to assign music-specific properties such a BPM to video and audio"</div>
<div><br></div><div>searching for 'assign music' can give me music results which have nothing to do with music. I'm not sure how to solve this.<br></div></div></div></div></blockquote><div><br></div><div style>
You can't, a full text search it's a full text search :).</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div><br>If we want to change how plain text is stored now is the time to do because with the 4.11 release most users will already have to re-index all their files and PIM data.<br>
</div></div></div></div></blockquote><div><br></div><div style>When 4.11 will be released? This is not a trivial change and must be tested.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>
<br>[1] <a href="http://xapian.org/" target="_blank">http://xapian.org/</a> <br></div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div dir="ltr"><div></div></div>
<div class="gmail_extra"><div><div><br><br><div class="gmail_quote">On Sat, May 4, 2013 at 7:09 PM, Vishesh Handa <span dir="ltr"><<a href="mailto:me@vhanda.in" target="_blank">me@vhanda.in</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">
<div><div>On Sat, May 4, 2013 at 9:18 PM, <span dir="ltr"><<a href="mailto:phreedom@yandex.ru" target="_blank">phreedom@yandex.ru</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div>On Суббота 04 мая 2013 20:14:37 Vishesh Handa wrote:<br>
> On Sat, May 4, 2013 at 7:47 PM, Ivan Čukić <<a href="mailto:ivan.cukic@kde.org" target="_blank">ivan.cukic@kde.org</a>> wrote:<br>
> > > <res> nie:plainTextContent "title artist album whatevereElse" .<br>
> ><br>
> > For me, the plainTextContent of a song would be the lyrics. This seems<br>
> > like a<br>
> > misuse of the property. With a very good reason behind it, but still a<br>
> > misuse.<br>
> ><br>
> > I remember when I wanted to keep all activities in one string property as<br>
> > a \n<br>
> > terminated list to make it speedy :D<br>
> ><br>
> > I'd say go for it, but only as a last resort.<br>
><br>
> I would not like Nepomuk to be a data store. It's not the place to store<br>
> your lyrics to fetch them later, same for emails and files. It is a place<br>
> to store structured data.<br>
><br>
> In the case of lyrics, the main reason we are storing them is to be able to<br>
> be search through them, not to display them to the user. So we can<br>
> potentially append other data.<br>
<br>
</div>Yes and no. Until discardable graphs were introduced, there was even no<br>
distinction between primary storage and cached stuff. The real life is even<br>
more complicated, you can have local data indexed, you can have remote data<br>
indexed(and it would be very very nice to have it cached) and for some tuff<br>
nepomuk is used as the primary storage.<br>
<br>
The reason people are trying to stuff nepomuk with their blobs is very simple:<br>
there's a very real demand for this functionality and nepomuk ontologies as-is<br>
already allow you to store your whole filesystem, including all byte<br>
streams/file contents, so it looks like a very reasonable approach, especially<br>
since nobody actually offers an alternative. Ok akonadi is the only exception<br>
which provides caching of remote data but it's domain-specific.<br>
<br>
Imagine a user finding a music video by its lyrics, opening the video only to<br>
discover that (s)he can't see any lyrics, because nepomuk got its lyrics from<br>
some web extractor. Thus the motivation to use nepomuk at least as a cache of<br>
data, not only for search purposes.<br></blockquote><div><br></div></div></div><div>You do have a point. In this case they should be able to access the lyrics.<br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
There's no primary storage for user-generated rdf at all, so the data is<br>
stored in nepomuk and users are disappointed when something breaks or<br>
disappears.<br></blockquote><div><br></div></div><div>If we treat Nepomuk as a data store, then you have to deal with keeping the store up to date. Specifically in the case of Akonadi - what are applications supposed to use? Nepomuk or Akonadi? And then we also need a 2 way sync to keep both the databases up to date.<br>
<br></div><div>So I prefer treating Nepomuk as a cache just for searching, but I get that it isn't in the case of tags, and ratings, and other specific rdf. So it's weird.<br></div><div><div><br></div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
I'm currently experimenting with solutions to some of these issues, but I<br>
can't do it fast due to time constraints. I don't expect anything worth going<br>
public with in the next couple of months at least and that's if I'm lucky :(<br></blockquote><div><br></div></div><div>Could you elaborate?<br> <br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div><div>_______________________________________________<br>
Nepomuk mailing list<br>
<a href="mailto:Nepomuk@kde.org" target="_blank">Nepomuk@kde.org</a><br>
<a href="https://mail.kde.org/mailman/listinfo/nepomuk" target="_blank">https://mail.kde.org/mailman/listinfo/nepomuk</a><br>
</div></div></blockquote></div></div><span><font color="#888888"><br><br clear="all"><br>-- <br><span style="color:rgb(192,192,192)">Vishesh Handa</span><br>
</font></span></div></div>
<br>_______________________________________________<br>
Nepomuk mailing list<br>
<a href="mailto:Nepomuk@kde.org" target="_blank">Nepomuk@kde.org</a><br>
<a href="https://mail.kde.org/mailman/listinfo/nepomuk" target="_blank">https://mail.kde.org/mailman/listinfo/nepomuk</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br></div></div>Best wishes,<div>Ignacio</div><div><br></div>
</div>
<br>_______________________________________________<br>
Nepomuk mailing list<br>
<a href="mailto:Nepomuk@kde.org" target="_blank">Nepomuk@kde.org</a><br>
<a href="https://mail.kde.org/mailman/listinfo/nepomuk" target="_blank">https://mail.kde.org/mailman/listinfo/nepomuk</a><br>
<br></blockquote></div></div></div><span class=""><font color="#888888"><br><br clear="all"><br>-- <br><span style="color:rgb(192,192,192)">Vishesh Handa</span><br>
</font></span></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Best wishes,<div>Ignacio</div><div><br></div>
</div></div>