KDirModelV2, KDirListerV2 and UDSEntryV2 suggestions

[Mark]

On Thu, Jan 10, 2013 at 3:53 PM, Aaron J. Seigo <aseigo at kde.org> wrote:
> On Thursday, January 10, 2013 14:12:30 you wrote:
>> part very rapidly. No issue thus far. But when i start scrolling
>> _while_ the data is dripping in it's ... not very responsive.
>
> after a quick look:
>
> the DirModel is doing all the data retrieval in the main thread as well as
> instantly on demand (no batching, e.g.). this means:
>
> * every allocation of a FileItem and all the work to populate it takes cycles
> away from the UI thread at the same time as the list is trying to scroll
> * KFileItems are created *and* populated regardless of whether they will be
> used or not, which means during scrolling things will get slower the more you
> scroll. they are also fully populated on creation.
>
> neither of these things is good. (and some of these issues have already been
> touched on in this thread)
>
> this probably requires changes to KDirModel such as:
>
> * put data processing into a thread; whether that means putting the whole of
> KDirModel into a worker thread or just having a thread pool for processing new
> KDirModelNodes / KFileItems .. others more familiar with the code would be
> able to answer that far better than i can right now :) but there should be
> near-zero time spent in the GUI thread for actually getting the file
> information
>
> * KFileItem needs to support lazy loading to avoid unecessary work when the
> information is not actually requested. there is even a TODO about this in
> KFileItemPrivate::init() in the 4.10 branch :)
>
> * methods like KDirModelPrivate::_k_slotNewItems need be faster and/or have
> their work moved into a thread. the number of times
> beginInsertRows/endInsertRows is called should also be minimized as that will
> innevitably create pauses in the GUI thread while it reacts to this
>
> perhaps some of this is already done or being done in the frameworks branch
> (haven't looked yet; apologies in advance for that)
>
> as to the DirModel class:
>
> * the image cache is done in the main thread. that means updating a database
> and writing to disk in the UI thread .. when the goal is 60fps or better, this
> is likely going to be too slow.
>
> * KIO::PreviewJobs are also in the main GUI thread. could be another cause for
> GUI latency.
>
> * DirModel::rebuildUrlToIndex is O(n) and called whenever there are new items
>
> * DirModel::updatePreview calls setData for each preview that it gets back.
> that causes dataChanged to be emitted. again, for each index. this needs to be
> batched up and/or DirModel should override data() itself to return
> Qt::DecorationRole when it has a preview.

Would a 300ms timer be acceptable in user experience? So if there are
new thumbnails available, batch them and update the model only once
every 300ms. Or is 300ms too much?
>
> btw, DirModel::itemForIndex looks like it is leaking memory.
>
> so it seems DirModel is need of a scalibility refactor/rewrite, and KDirModel
> needs to be made read for use with threads in some fashion as well have its
> base performance improved with more lazy loading and batching of requests.
>
> (this feels earily similar to the model rewrites Marco ended up doing for
> Plasma Active ... threads and batching, threads and batching. the UI thread
> must not be disturbed! :)

Interesting! My DirModel is coming directly from plasma-mobile [1]
which is not doing the threading you describe. Is there another
dirmodel fork somewhere that does that? Link please :)

>
> --
> Aaron J. Seigo
> _______________________________________________
> Kde-frameworks-devel mailing list
> Kde-frameworks-devel at kde.org
> https://mail.kde.org/mailman/listinfo/kde-frameworks-devel
>

You just verified the need for KDirModel, KDirLister and KFileItem to
be optmized for scalibility :-)
My DirModel is basically a KDirModel but with some "convenient" parts.

How much work would it be to let KDirModel run in it's separate
thread? I'm guessing the public KDirModel would internally be nothing
more then a signal/slot mechanism to a KDirModelPrivate class which is
sent to it's own thread? Or would you say that sending KDirModel to
it's own thread it too risky and should be done by the developer using
KDirModel (like dolphin and my app) rather then internally in the
private API?

[1] http://quickgit.kde.org/?p=plasma-mobile.git&a=blob&h=1539ae3d6cf54f96af216593e5c7517df820a6b8&hb=50664f4fae8243280f1e7362447a864b37a2039b&f=components%2Fdirmodel%2Fdirmodel.cpp