problem with KIO::NetAccess and KDirLister/KFileTreeView

[David Faure]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

[Hmm, I still had this in my drafts folder, from some time ago, I forgot to send it.]

On Tuesday 08 October 2002 13:02, Josef Weidendorfer wrote:
> On Tuesday 08 October 2002 10:59, David Faure wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > On Tuesday 08 October 2002 01:33, Josef Weidendorfer wrote:
> > > Hi,
> > >
> > > I did some large hacking on KDirWatch before release of KDE 3.0...
> > > So I feel kind of responsible for this class ;-)
> > >
> > > And relying on KDirWatch should be no problem at all. What are
> > > the exact issues?
> > >
> > > One problem perhaps is that KDirWatch triggers to often
> > > when a file is written to in short parts over a longer period of time,
> >
> > Yes that's a big problem, when FAM is enabled.
> > I tested copying an image over an other (when fixing that "it should update
> > the preview" bug), and I got TONS of FAM events (FileChanged),
> > so the code would keep stopping and relaunching the preview, making the
> > whole thing quite slow. Some buffering is needed, and IMHO the best would
> > be to do that buffering in KDirWatch itself (delaying and compressing the
> > events). Even better would be to have a way to know when the operation (the
> > copying, from a terminal) is finished, but I guess that's not possible ;)
>
> I suppose the original use of start/stop was to suppress these events. But
> that's no real option as another Konqui process could watch the directory
> where to copy happens. The best would be to make these start/stops global for
> all apps on your box. A way to do this is to make the copy in fact into a
> hidden subdirectory and rename the created file afterwards to the file to be
> overwritten. KDirWatch would only report a file change at the end ;-)

In my case, as I said, the copying was done from a terminal. Nothing KDE can act upon.

> Of course "delaying and compressing" is needed in KDirWatch as the above is
> not generally possible. Note that even delivering these huge amounts of
> "changed" events only to KDirWatch can be very CPU-intensive when the
> notifications are created by the kernel. The current LINUX DNOTIFY
> implemention sends a signal on every file change (e.g. each write() system
> call). Here, the "delaying and compression" should be already done in the
> kernel.
I agree. Is there any chance of this happening?

> Also note there are FAM daemons using DNOTIFY: Here the problem
> is even bigger as each file change will lead to an IPC socket communication.
I see.

> We need 2 time interval values:
> - One for the normal time between event ariving and delayed event emitting. If
> events arrive in shorter intervals, we delay the delayed event emitting even
> further. That way the delay in the usual case (1 event) can be kept short.
> - A maximal delay.
>
> Suppose normal time 0.5 s and maximal time 5 secs:
> - One single event is delayed 0.5 s (acceptable?)
> - Two events with e.g. 0.3 s between create only one emitted event 0.8 s after
> the first one arrived.
> - A long period of writing to a file: Every 5 seconds a change event is
> delivered, and 0.5 seconds after finishing the writing.
>
> Is this OK?

Ok, this is more complex that what I thought of (a simple short max delay
from the first event). But indeed this reacts nicer on long copy operations.

> > I guess this is done by KDirLister (which sort of "quits" the dir, and then
> > "re-enters" it). Some kdBacktrace() investigation by Klaas would help ;)
>
> Or some tracing with cachegrind and analysing with KCachgrind. BTW, are you
> using it sometimes (my baby, hehe)?

Not yet - KWord seemed too big for it, last time I tried. (cachegrind exited after
eating all available memory).

> OT: I should do a KPart for
> inode/directory showing space partitioning on your harddisc using my TreeMap
> widget. As (tiny, like 50 lines) application it's already on my HD: It's
> unbelievable how fast you see your space wasters...
Yes this would be cool. I still have to use "du" for this.

> OK. Seems reasonable: In fact you should never miss events this way. Note that
> misses would be possible when deleting the watch and adding again.
Yes, we have to figure out why this happens.

> The implementation in KDE 3.0 (I think) had the "bug" that a watch deletion in
> the slot invoked by the "delete" signal of this watch lead to a crash (same
> as with QListViewItems in QListViews...). I think this was already fixed in
> 3.0.1 by delaying the emitting. It could be that this is NOT fixed for the
> FAM case. Support for the above "delaying and compressing" could fix it...
> The backtrace suggests something like this happening (a dangling pointer), but
> I'm not quite sure...
Yes, same impressions here.

> The problem with reproducability with KDirLister is its cache: You have to
> reproduce your browsing history...
Right.

> This must be the portmapper refusing the connection ("sgi_fam" is the
> portmapper service I suppose). Isn't there an option for the portmapper to
> always allow connections from the local host?
Hmm, I guess this is related to /etc/hosts.allow. I'll keep investigating
(but this shouldn't be soo hard to set up :(.

> Have you NFS working on this system (it uses the portmapper, too)?
Yup, works ok.

> > But Konqueror (which needs to notice files growing) would then be stat'ing
> > every file it's showing? This could be a lot of files. Do we really want
> > that?
>
> If you have FAM running, the FAM daemon already does this :-)
Hmm, I thought there was a kernel part of it....

> You say: Konqueror needs to notice files growing. So there's no real way
> around this, is there? And stat() in LINUX is really fast in my experience
> (they put a lot of effort into this).
It's very slow over NFS, but that's another story ;)
Hmm. I can see from here the reports from people who notice that konqueror
is stat'ing every file it's showing, many times per second.... (maybe a much longer
period like 1-5 seconds would be enough, but it gives an impression of slowness....)

- --
David FAURE, david at mandrakesoft.com, faure at kde.org
http://people.mandrakesoft.com/~david/
Contributing to: http://www.konqueror.org/, http://www.koffice.org/
Get the latest KOffice - http://download.kde.org/stable/koffice-1.2/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9rG3k72KcVAmwbhARAo55AJ9XvIxWEStBGJbMUlqKVupq0JUBqQCgrd4m
aBUTu3BOKO+dcNazB35MUnM=
=dHdH
-----END PGP SIGNATURE-----