[Nepomuk] Nepomuk Digest, Vol 9, Issue 1

[James Smith]

Here's a transcript after an ideastorm on irc this morning.

i just did a quick look through the code.
it seems to me filehandles are a tough issue to sidetrack.
if nepomuk used some extra diskspace to hold only one file with the
entire contents of the currently indexable directories.
filewatch could instead watch only that handle for updates and parse
on gamin emits to find out what recently changes.
kind of like a metadata journal but called say nepomuk ledger.
the same idea anyway.
fsync would be a bitch more than probably on a notebook, however most
of it can be seen in ram.
compcache with the equivalent of a full '# find / .' and filehandle
pageswap after pageswap.
it's great to have handles open after a find command because the reuse
is killer, but not all apps (ie amarok) pick up on why.
i don't think amarok uses kio.
e.g. fileledger service: hash of file, current path, file name,
optional filesize. 4 columbs.(sic)
so the ledger service picks up on a new notify on fileledger.xml or
what have you, and spits the latest entr(ies) having been moved or
modified to nepomuk-core. Database ends up updated in-DB. I think tail
does moderately well as far as memory / filewatch usage, so it should
be an easy way around the issue.
-I may have a QT oversight, I'm not certain but the main glib loop
supposedly in qt should help matters as far as filewatch events
actually being triggered by an automatic parse on inotify.-

practical experience, with a full file open by a service who would
bother saving :) A bitch. (Freud-Daemon would be the closest the
kernel would go IMO...a daemon. Again.)

unpremeditated (tail -f ) practical terminology, a follow.
write-then-read, not C-O-W.

off-topic: I wonder if Quassel has a direct-to-printer option?

hmm, brainstorm.
how does kio handle inotify events and can they actually emit a
signal? e.g. a file was moved.
i'm thinking if the desktop actually was separated via kio (ie gated
to not actually bother with files not being interacted with on the
desktop shell) it should be easier to separate systems.

i'm reminded of a plane that hit a swamp which they blamed on a
shipper piling boxes in a doorway.
it's termed a system accident or a cascade failure or something similar.
anyway, isolating systems; if kio was to trigger an inotify event on
move and increment the ledger, nepomuk was to pick up the notification
via filewatch on ledger.xml and do a quick parse on the last few
entries the systems should be isolated enough to actually grep a feel
for what they would be doing in tandem at any one time.

nepomuk updates aren't necessarily needed to be instantaneous.
if kio can be rigged, we can call it a morning and blame it all on
fuse for not being 'there'.

On 5/1/10, nepomuk-request at kde.org <nepomuk-request at kde.org> wrote:
> Send Nepomuk mailing list submissions to
> 	nepomuk at kde.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://mail.kde.org/mailman/listinfo/nepomuk
> or, via email, send a message with subject or body 'help' to
> 	nepomuk-request at kde.org
>
> You can reach the person managing the list at
> 	nepomuk-owner at kde.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Nepomuk digest..."
>
>
> Today's Topics:
>
>    1. Re: Bugs, Patches and Scripts. - StrigiService & KInotify
>       (Vishesh Handa)
>    2. Re: Bugs, Patches and Scripts. - StrigiService & KInotify
>       (Carlos Ruiz)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 30 Apr 2010 17:37:03 +0530
> From: Vishesh Handa <handa.vish at gmail.com>
> Subject: Re: [Nepomuk] Bugs, Patches and Scripts. - StrigiService &
> 	KInotify
> To: Sebastian Tr?g <trueg at kde.org>
> Cc: nepomuk at kde.org
> Message-ID:
> 	<u2j38ef5bd51004300507o246bb36bs719b757215c57319 at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> *This patch doesn't work perfectly.*
>
> I seem to to be duplicating code from StrigiServiceConfig, which is not
> something I'm too happy about.
>
> The moment some watched folder is moved/renamed, metadata mover fires up,
> and starts moving the metadata. Meanwhile the strigi config file is updated
> and the strigi service detects that the file has been changed (exact same
> code as in the testpatch) and then proceeds to delete unnecessary metadata
> via removeOldAndUnwantedEntries(). Which removes some of the indexed data.
> And since some of the metadata has been removed the filewatch service can't
> move it. After that the strigi service proceeds to index the moved files
> who's metadata isn't intact.
>
> :-/
>
> It's doing what we want, but not how we want it. We need to delay writing to
> the config file until the metadata mover has finished is job. I could just
> slap on a signal which is fired when it changes from a Busy to Idle state
> (and that'll work) but for all you know the user could be performing other
> tasks and the metadata mover doesn't become idle for a long time. (Maybe
> he's monitoring a network drive where other users are doing stuff).
> This method will ultimately result in what we want except when then this
> happens -
> 1. The user moves some folder which is being indexed.
> 2. The metadata mover is active.
> 3. He edits the files being watched via kcm.
> 4. The metadata mover becomes idle.
>
> There is one more issue. Say I have an indexed folder *Boo*, and it has a
> sub-folder *Yoda*. Yoda is moved somewhere outside Boo which isn't being
> indexed. Should Yoda still be indexed? Or Not? I think it should be. What do
> you think?
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.kde.org/pipermail/nepomuk/attachments/20100430/9d80a1d1/attachment-0001.htm
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: patch.diff
> Type: text/x-patch
> Size: 3198 bytes
> Desc: not available
> Url :
> http://mail.kde.org/pipermail/nepomuk/attachments/20100430/9d80a1d1/attachment-0001.diff
>
> ------------------------------
>
> Message: 2
> Date: Fri, 30 Apr 2010 12:44:03 +0000
> From: "Carlos Ruiz" <cruiz at isoco.com>
> Subject: Re: [Nepomuk] Bugs, Patches and Scripts. - StrigiService &
> 	KInotify
> To: "Vishesh Handa" <handa.vish at gmail.com>, " Sebastian Tr?g "
> 	<trueg at kde.org>
> Cc: nepomuk at kde.org
> Message-ID:
> 	<2093337748-1272631370-cardhu_decombobulator_blackberry.rim.net-137219992- at bda188.bisx.produk.on.blackberry>
> 	
> Content-Type: text/plain; charset="Windows-1252"
>
> F
> Carlos
>
> -----Original Message-----
> From: Vishesh Handa <handa.vish at gmail.com>
> Date: Fri, 30 Apr 2010 17:37:03
> To: Sebastian Tr??g<trueg at kde.org>
> Cc: <nepomuk at kde.org>
> Subject: Re: [Nepomuk] Bugs, Patches and Scripts. - StrigiService & KInotify
>
> _______________________________________________
> Nepomuk mailing list
> Nepomuk at kde.org
> https://mail.kde.org/mailman/listinfo/nepomuk
>
>
> ------------------------------
>
> _______________________________________________
> Nepomuk mailing list
> Nepomuk at kde.org
> https://mail.kde.org/mailman/listinfo/nepomuk
>
>
> End of Nepomuk Digest, Vol 9, Issue 1
> *************************************
>