KIO::listRecursive() with filters

[Milian Wolff]

David Faure, 30.10.2009:
> On Thursday 09 July 2009, Milian Wolff wrote:
> > Hi all!
> >
> > I'm one of the guys hacking on KDevelop and ported the generic project
> >  manager to KIO to make it work for remote projects (think web projects
> > on some server or similar).
> >
> > I decided to use KIO::listRecursive() which works wonderfully. One big
> > sore thumb I have with it, is performance. Perse it's wonderful, but
> > users might/want/will filter what is considered part of their project. By
> > default e.g. I filter hidden files and .svn|.git|CVS folders. One could
> > think of a lot more.
> >
> > Right now, I have to filter what KIO::listRecursive() gives me. But this
> > is potentially a lot of unwanted stuff since the list job recurses into
> > filtered directories. Hence my sore thumb regarding performance...
> >
> > How do you suggest should I fix this? Since listRecursive essentially
> > just creates a ListJobPrivate I'd need to change that one. Esp.
> >  slotListEntries() if I see it correctly. Thing is: It's private, hence I
> >  can't simply overload it...
> 
> I can think of two solutions.

Wow, been some time :) We ported KDevelop to mutliple listDir() calls now, put 
together in a ComposeJob or how it was called.

> 1) The simple one, adding a KIO::ListJob::setExcludedFilenames(const
> QStringList&)

Woudln't this break BC? Or is KIO::ListJob considered "private"?

> 2) The more flexible one, adding a "hook" where the application can define
>  which files to exclude. Actually... thanks to Carsten Pfeiffer I have some
>  class doing this kind of stuff on my harddisk... Attached. Needs to be
>  integrated into ListJob and then tested, of course ;)
> IIRC he wrote it for kde3, but it was never used anywhere, so it was
>  cleaned up for kde4, but I kept a copy for a day like today.
> 
> (Note that KDirLister has this kind of functionality already, with a
>  virtual bool matchesFilter, but this won't help the fact that the listjob
>  recurses into unwanted directories; the dirlister filtering happens after
>  the fact).
> 
> In fact, Carsten's idea was to port KDirLister to that KFileFilter class
>  (but given the virtual stuff, that's for KDE5 now, I guess). At least this
>  gives a good plan ahead: in KDE5 we can use that filter class from both
>  ListJob and KDirLister (doing it in the listjob isn't enough for the
>  kdirlister use case, since it lets the user change the filter at runtime,
>  e.g. when showing/hiding dot files in the file dialog, and we don't want
>  to re-list the directory at that point, so I think both uses are needed).

Yes, the class looks good. When a pointer to it can be passed to the 
listRecursive e.g. I could overload passesFilter for my needs. Or would you 
think it should go into the mainline:

What is always / often lacking from filters like the one here, is that it only 
matches against the _name_ of the file (ok, and mimetype and stuff, but that's 
not my point). Problem with matching name only, is that you cannot match stuff 
like "dir called foo" (when matching against path simply add filter for 
"foo/"). Or "file called bar somewhere in path XYZ". For KDevelop/Quanta I 
added this, since I find it sometimes useful to have such kind of filtering for 
projects.

But of course, it's arguably whether one would need such stuff anywhere else.

> Looking forward to your patch that will make KIO::ListJob use KFileFilter
>  :-)

But: How can I make KIO::ListJob use KFileFilter without breaking BC (see 
above)?
-- 
Milian Wolff
mail at milianw.de
http://milianw.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20091030/500d5904/attachment.sig>