KDirModelV2, KDirListerV2 and UDSEntryV2 suggestions

[Mark]

On Thu, Jan 10, 2013 at 9:23 AM, Frank Reininghaus
<frank78ac at googlemail.com> wrote:
> Hi,
>
> 2013/1/9 Mark:
>> On Wed, Jan 9, 2013 at 2:51 PM, David Faure wrote:
>>> On Wednesday 09 January 2013 11:15:20 Mark wrote:
>>>> A little more in depth questions for KDirLister and KFileItem. In my
>>>> profiling KFileItem ends up high due to various reasons, but
>>>> KDirLister is also a bit of a heavy resource hog due to it's default
>>>> behavior of fetching all file information (thus at least 1 stat call
>>>> per file) which severely slows down the dir listing process for large
>>>> folders.
>>>
>>> This stat call happens in kio_file though, not in the GUI process where
>>> KDirLister lives, right?
>>> So I'm surprised that you see that when profiling...
>>> or is there a nasty stat() in KDirLister somewhere?
>>
>> ehm, well i'm not really monitoring stat calls. I'm monitoring the
>> time it takes for a directory listing on 1 million files to be
>> available in my application.
>
> I agree that we should try to behave well even if there are many
> files. However, if it is ensured if all algorithms involved in loading
> item information and view setup are O(N) or at most O(N*log(N)) (like
> sorting), then the remaining slowness for many files is something that
> the user might just have to live with IMHO.
>
> I'm not saying that everything is good as it is. If there are some
> operations with quadratic or worse complexity involved, then this
> should be looked into, of course. And I also do realise that Dolphin
> could benefit from making use of your listJob batching improvements
> (https://git.reviewboard.kde.org/r/107520/, very nice work BTW!).

Thank you for the compliment :)
That could be a nice improvement in dolphin indeed. However, while
testing that in my application (with 1 million files) i found an issue
that you are very likely to have as well. While the files do show up
very rapidly, the actual view is still barely usable as long as files
are dripping in. This is the same in QML (ListView) as in QWidgets
(QTreeView). The issue here is that the view - even though it's not
showing the items - is doing all the calculations for every item
added. I have no clue how to fix that issue and if it's fixed it
likely has to be done upstream in Qt itself. For Dolphin that might be
different since it has it's own fully implemented view so you probably
have to look at it there.
>
> But making extremely intrusive changes to well-tested code which are
> only motivated by the (IMHO rather exotic) "1 million files" use case
> and which would require large changes everywhere in KDE might not be
> such a good idea. Just my opnion, of course.

Well, in my test cases my initial goal of 100.000 - 200.000 files is
reached :) So then i upped my demands to 1 million. But you already
notice slowdowns in dolphin with 10.000+ files, it kinda depends on
your hardware as well. I know that i'm testing extremes right now :)
>
> Best regards,
> Frank
>
> P.S.: I agree that the "lazy folder icon loading" issue mentioned
> elsewhere in this thread can be disturbing sometimes and could be
> improved.