[Nepomuk] Nepomuk Digest, Vol 9, Issue 1

Sat May 1 19:53:43 CEST 2010

A pastor,  minister, and a British pirate all walk into a bar.

On 5/1/10, James Smith <smithjd15 at gmail.com> wrote:
> Here's a transcript after an ideastorm on irc this morning.
>
> i just did a quick look through the code.
> it seems to me filehandles are a tough issue to sidetrack.
> if nepomuk used some extra diskspace to hold only one file with the
> entire contents of the currently indexable directories.
> filewatch could instead watch only that handle for updates and parse
> on gamin emits to find out what recently changes.
> kind of like a metadata journal but called say nepomuk ledger.
> the same idea anyway.
> fsync would be a bitch more than probably on a notebook, however most
> of it can be seen in ram.
> compcache with the equivalent of a full '# find / .' and filehandle
> pageswap after pageswap.
> it's great to have handles open after a find command because the reuse
> is killer, but not all apps (ie amarok) pick up on why.
> i don't think amarok uses kio.
> e.g. fileledger service: hash of file, current path, file name,
> optional filesize. 4 columbs.(sic)
> so the ledger service picks up on a new notify on fileledger.xml or
> what have you, and spits the latest entr(ies) having been moved or
> modified to nepomuk-core. Database ends up updated in-DB. I think tail
> does moderately well as far as memory / filewatch usage, so it should
> be an easy way around the issue.
> -I may have a QT oversight, I'm not certain but the main glib loop
> supposedly in qt should help matters as far as filewatch events
> actually being triggered by an automatic parse on inotify.-
>
> practical experience, with a full file open by a service who would
> bother saving :) A bitch. (Freud-Daemon would be the closest the
> kernel would go IMO...a daemon. Again.)
>
> unpremeditated (tail -f ) practical terminology, a follow.
> write-then-read, not C-O-W.
>
> off-topic: I wonder if Quassel has a direct-to-printer option?
>
> hmm, brainstorm.
> how does kio handle inotify events and can they actually emit a
> signal? e.g. a file was moved.
> i'm thinking if the desktop actually was separated via kio (ie gated
> to not actually bother with files not being interacted with on the
> desktop shell) it should be easier to separate systems.
>
> i'm reminded of a plane that hit a swamp which they blamed on a
> shipper piling boxes in a doorway.
> it's termed a system accident or a cascade failure or something similar.
> anyway, isolating systems; if kio was to trigger an inotify event on
> move and increment the ledger, nepomuk was to pick up the notification
> via filewatch on ledger.xml and do a quick parse on the last few
> entries the systems should be isolated enough to actually grep a feel
> for what they would be doing in tandem at any one time.
>
> nepomuk updates aren't necessarily needed to be instantaneous.
> if kio can be rigged, we can call it a morning and blame it all on
> fuse for not being 'there'.
>
> On 5/1/10, nepomuk-request at kde.org <nepomuk-request at kde.org> wrote:
>> Send Nepomuk mailing list submissions to
>> 	nepomuk at kde.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> 	https://mail.kde.org/mailman/listinfo/nepomuk
>> or, via email, send a message with subject or body 'help' to
>> 	nepomuk-request at kde.org
>>
>> You can reach the person managing the list at
>> 	nepomuk-owner at kde.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Nepomuk digest..."
>>
>>
>> Today's Topics:
>>
>>    1. Re: Bugs, Patches and Scripts. - StrigiService & KInotify
>>       (Vishesh Handa)
>>    2. Re: Bugs, Patches and Scripts. - StrigiService & KInotify
>>       (Carlos Ruiz)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Fri, 30 Apr 2010 17:37:03 +0530
>> From: Vishesh Handa <handa.vish at gmail.com>
>> Subject: Re: [Nepomuk] Bugs, Patches and Scripts. - StrigiService &
>> 	KInotify
>> To: Sebastian Tr?g <trueg at kde.org>
>> Cc: nepomuk at kde.org
>> Message-ID:
>> 	<u2j38ef5bd51004300507o246bb36bs719b757215c57319 at mail.gmail.com>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> *This patch doesn't work perfectly.*
>>
>> I seem to to be duplicating code from StrigiServiceConfig, which is not
>> something I'm too happy about.
>>
>> The moment some watched folder is moved/renamed, metadata mover fires up,
>> and starts moving the metadata. Meanwhile the strigi config file is
>> updated
>> and the strigi service detects that the file has been changed (exact same
>> code as in the testpatch) and then proceeds to delete unnecessary
>> metadata
>> via removeOldAndUnwantedEntries(). Which removes some of the indexed
>> data.
>> And since some of the metadata has been removed the filewatch service
>> can't
>> move it. After that the strigi service proceeds to index the moved files
>> who's metadata isn't intact.
>>
>> :-/
>>
>> It's doing what we want, but not how we want it. We need to delay writing
>> to
>> the config file until the metadata mover has finished is job. I could
>> just
>> slap on a signal which is fired when it changes from a Busy to Idle state
>> (and that'll work) but for all you know the user could be performing
>> other
>> tasks and the metadata mover doesn't become idle for a long time. (Maybe
>> he's monitoring a network drive where other users are doing stuff).
>> This method will ultimately result in what we want except when then this
>> happens -
>> 1. The user moves some folder which is being indexed.
>> 2. The metadata mover is active.
>> 3. He edits the files being watched via kcm.
>> 4. The metadata mover becomes idle.
>>
>> There is one more issue. Say I have an indexed folder *Boo*, and it has a
>> sub-folder *Yoda*. Yoda is moved somewhere outside Boo which isn't being
>> indexed. Should Yoda still be indexed? Or Not? I think it should be. What
>> do
>> you think?
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://mail.kde.org/pipermail/nepomuk/attachments/20100430/9d80a1d1/attachment-0001.htm
>> -------------- next part --------------
>> A non-text attachment was scrubbed...
>> Name: patch.diff
>> Type: text/x-patch
>> Size: 3198 bytes
>> Desc: not available
>> Url :
>> http://mail.kde.org/pipermail/nepomuk/attachments/20100430/9d80a1d1/attachment-0001.diff
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Fri, 30 Apr 2010 12:44:03 +0000
>> From: "Carlos Ruiz" <cruiz at isoco.com>
>> Subject: Re: [Nepomuk] Bugs, Patches and Scripts. - StrigiService &
>> 	KInotify
>> To: "Vishesh Handa" <handa.vish at gmail.com>, " Sebastian Tr?g "
>> 	<trueg at kde.org>
>> Cc: nepomuk at kde.org
>> Message-ID:
>> 	<2093337748-1272631370-cardhu_decombobulator_blackberry.rim.net-137219992- at bda188.bisx.produk.on.blackberry>
>> 	
>> Content-Type: text/plain; charset="Windows-1252"
>>
>> F
>> Carlos
>>
>> -----Original Message-----
>> From: Vishesh Handa <handa.vish at gmail.com>
>> Date: Fri, 30 Apr 2010 17:37:03
>> To: Sebastian Tr??g<trueg at kde.org>
>> Cc: <nepomuk at kde.org>
>> Subject: Re: [Nepomuk] Bugs, Patches and Scripts. - StrigiService &
>> KInotify
>>
>> _______________________________________________
>> Nepomuk mailing list
>> Nepomuk at kde.org
>> https://mail.kde.org/mailman/listinfo/nepomuk
>>
>>
>> ------------------------------
>>
>> _______________________________________________
>> Nepomuk mailing list
>> Nepomuk at kde.org
>> https://mail.kde.org/mailman/listinfo/nepomuk
>>
>>
>> End of Nepomuk Digest, Vol 9, Issue 1
>> *************************************
>>
>