KDirModelV2, KDirListerV2 and UDSEntryV2 suggestions

[David Faure]

On Saturday 29 December 2012 15:21:26 Mark wrote:
> Just a little side note, is it possible to get rid of KFileItem and
> use QFileInfo? Though i'm guessing it's not possible due to Qt not
> being as powerful as KIO in the file stuff.

No. QFileInfo is only for local files.

> > UDSEntry is the raw storage [for things that come from a kioslave],
> > KFileItem is the API on top [plus support for things that don't come from
> > a kioslave].
> > 
> > Better keep this separated (e.g. UDSEntry is enough on the slave side),
> > and fix the "memory usage" of KFileItem by changing its implementation.
> 
> This kinda depends on it. Why would you cache? Because something is
> slow. If the reason for it being slow is gone then you don't need a
> cache at all. That's my aim here. Making it fast enough for the cache
> to be removed. Perhaps the UDSEntry should have all the values of
> KFileItem where KFileItem would just access those values. That would
> make the UDSEntry a little more heavier but would make KFileItem
> nearly free.

Yes, that's exactly what I meant.

Currently KFileItem can be created from a local path (without any UDSEntries 
as input), and will determine the size/mtime/owner/etc. by looking at the 
filesystem directly, and storing that into its own variables. It could store 
into the UDSEntry array instead, to save memory.

> Missed that one. Oke, should not be removed/deprecated. Though i do
> wonder how QFileSystemModel is doing this..

Well QFileSystemModel's job is easier, it can list subdirs synchronously, so 
expanding to a specific path can easily be done, even from the outside.
In KDirModel, directory listing is async (KIO::listDir job), so this requires 
keeping intermediate state ... much harder, and therefore has to be done 
internally for sure.

> So i think KDirLister needs two lists.
> 1 list with all the UDSEntry entries (AKA UDSEntryList)
> 1 list with the constructed KFileItem lists + lazy loading them when needed.

I don't see what this gains. It only makes KDirLister manage even more data 
structures -- which is a step in the wrong direction, given that KDirLister's 
complexity comes from managing too many data structures already.

If KFileItem is just convenience API around UDSEntryList, and doesn't require 
more memory, the above gains nothing.

> >> Remove:
> >> -  void started( const QUrl& _url ); // You know the object and url
> >> your're listening to. No need to emit the url.
> >> -  void completed( const QUrl& _url ); // see above
> >> -  void canceled( const QUrl& _url ); // see above
> >> -  void clear( const QUrl& _url ); // see above
> > 
> > Not true, in the case of trees.
> 
> Right, with this i also wonder how QFileSystemModel is doing this. It
> doesn't have those functions but does "function" the same way.. Those
> functions at least don't have to be public, right?

Those are signals, coming from the async nature of the directory listing.
QFSM is sync, no signals.

This being said, it might very well be that some of these signals are not 
really used anymore these days. Requires investigation.

> > Well, although filtering is conceptually better done higher up ... it
> > sounds like a directory chooser would be much slower if KDirLister was
> > emitting all files too, just for them to be filtered afterwards. This
> > sounds like an possibly useful optimisation... To be tested, I guess.
> 
> I think you're mixing things up a bit now. The intention is to set
> flags (like in QDir). If you set the flag to list directory only then
> you obviously only get the signals for directories. Same for files,
> dot files and whatever else is possible.

Ah OK, so same feature, just a different API.

> >> -  virtual KFileItem findByUrl( const QUrl& _url ) const;
> >> -  virtual KFileItem findByName( const QString& name ) const;
> > 
> > Should not be virtual, that's for sure. About deprecating/removing, maybe,
> > needs to be investigated (hard to use lxr for that, better compile KDE SC
> > with this private, and see who uses it).
> 
> My guess it that it's used quite a bit. It's convenience, but
> something that should be done by the user. Just get the items from the
> listener and search in that list. Not something that should be placed
> in the listener itself.

It's more than a listener, it's also the storage for the items.

findByUrl can search in the entire tree, so "get the items and search in that 
list" doesn't work... it's not one list, but a whole tree of lists.

findByName, on the other hand, breaks in case of a tree with multiple files of 
the same name.... please look into whether anyone is using that, and kill it
(deprecate it) if, as I suspect, nobody is (at least nothing in kdelibs itself 
calls that). And nothing in kde-baseapps... Yeah, you can deprecate findByName.

===

If you want to think "bigger redesign", one could consider adding the 
necessary API to KDirModel so that direct usage of KDirLister isn't necessary 
anymore, i.e. encapsulating it completely. Then looking at whether all the app 
code can be ported away from KDirLister, to use either KDirModel or 
KIO::listDir (when no storage is required, just a one-time listing).

A quick look at lxr.kde.org (for "KDirLister") seems to indicate that this 
should be possible.
Most cases are using a KDirModel already, and just use KDirLister to call 
openUrl or to enable "directories only" mode.
On the other hand, 
http://lxr.kde.org/source/kde/kdeedu/kstars/kstars/ekos/capture.cpp
should use KIO::listDir directly, I think.
http://lxr.kde.org/source/kde/kdegraphics/gwenview/importer/documentdirfinder.cpp
too, AFAICS. (Aurélien: storing the items isn't needed, in that code, right?)

The most problematic case I found is KParts::ListingExtension, and its use in
http://lxr.kde.org/source/kde/kde-baseapps/dolphin/src/dolphinpart_ext.cpp
which is based on the signals from KDirLister (to notify of new items). 
KDirModel could relay that signal, maybe.

The longer term goal with encapsulating KDirLister as an implementation detail 
of KDirModel, could be to then merge the data structures, to avoid the current 
overhead due to layering (KDirModel creates tree nodes on top of KDirLister's 
own tree-like structure). But one step at a time.

-- 
David Faure, faure at kde.org, http://www.davidfaure.fr
Working on KDE, in particular KDE Frameworks 5