[Nepomuk] Fwd: Patches and issues with search/strigi

[Sebastian Trüg]

On Friday 01 January 2010 17:35:53 Daniel Winter wrote:
> Filewatch.patch: (against filewatch service)
> 
> The nfo::fileName wasn't updated when moving file. (The search ioslave
> uses that to get the filename). Patch fixes this

Please apply. But for completeness the extension property should be updated, 
too. Or we simply throw it out completely. After all it should be deprecated 
by the mimetype and the rdf type...


> dolphinsearch.patch: (against dolphin)
> 
> Two isses fixed by this patch:
> - Date - Today was an "is equal query", but it works on times. So it
> gives no results most of the time. Change it to is equal or greater.
> In theory it will also hit files from the future. Put they shouldn't
> exist anyway.

looks good.

> (There are more similar but harder to fix issues for example search
> for files eqal or less 1.1.2010 will not find files from today.
> (because it compares it with the start of the current day. And so
> files from today are not equal or less that..)

So we need to use start/end of day depending on the comparator. Should be an 
easy fix, shouldn't it?

> - Searches in NIE::lastModified instead of NAO::lastModified with the
> patch.  I think that is what the user cares about. (The modifed date
> of the file, not the information when tagged it or something.) This
> way one actually get the expected results. (or do you really think NAO
> is correct? )

correct.

> - strigiservice  monitoring:
> 
> It doesn't catch changes in files. It recurses over the directorys for
> changed modifed time. The problem is: The directory modified time only
> gets changed when one renames, creates or deletes file in the
> directory. Not when a file just gets changed (like the modification
> date or content). I think catching this is important.
> 
> There is no obvious solution to this. Inotify would be one. But there
> is the limit on handles. I think it is the right way anway. Most users
> will not add all directories to strigi/nepomuk and therefore never hit
> the limit. Nepomuk could count the number of directories the user has
> in his indexed dirs and check if there are enough inotify handles set
> in the system. (It should leave at least 30% of them to other
> applications) If there are enough (respecting the reserver for the
> rest) it should use them.

We have a patch for that in Mandriva. Now if you give me code to determine the 
number of free inotify descriptors left I can finish it and get it into 4.4.1 
maybe.

> The kde filewatch thing (the one the filewatch service uses) could be
> a fallback for systems with not enough or no inotify at all.

Strigi already uses that.

> - Search result caching/auto updating:
> 
> There seam to be some regression there. You have to wait a lot of time
> or restart Nepomuk to find newly tagged files for example.
> 
> 1. Search for files tagged with "x"
> 2. Tag another file with "x"
> 3. search for files tagged with "x"
> 
> The file will not show up for a quite some time. (or restart of
> Nepomuk) It used to get updated in realtime in 4.3.

I did not test this after the change to Virtuoso yet. It is on my bugfix todo 
for 4.4.

> -- Nepomuk IO-Slave  URL/Forwaring/file handling
> 
> Not sure what exactly is happening or if it is intented to work this
> way. From my point of view (as a user of it) it is a regression.
> 
> Opening search results from Dolphin opens the nepomuk://  uri or
> something. You get some really weird filename shown in the
> Applications. One could live with that. The problem is: Non KDE
> Applications are getting some tmp file. It doesn't look like writing
> to this file is possible. (KIO asks if I want to upload the file, and
> if I say yes it tells me that writing to Nepomuk is not supported).

Hm, that would mean that put is broken. I need to look at that. Please create 
a bugreport.

> I believe related to this is the really slow preview of files (for
> example Images) in Dolphin when there are a lot. Every single preview
> results in a request to Nepomuk wich opens and closes a connection to
> soprano on every request (this seems to be a general issue?)

I dont see another way ATM.

> Also somehow the caching of Nepomuk::ResourceManager doesn't really
> work? It opens a new Soprano connection to virtuoso on every single
> mouse over on the results of a search in Dolphin.

This is a KIO issue with reusing of KIO slaves. Each slave runs in its own 
process and as such gets a new connection.

> -- Dolphin search (or the search io slave) should have a (default)
> limit of results.

It did before. Not sure if it should now....