[frameworks-kio] [Bug 471018] Terrible browsing performance with mounted remote FS on links with high latency

[Pedro V]

https://bugs.kde.org/show_bug.cgi?id=471018

Pedro V <voidpointertonull+bugskdeorg at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|general                     |general
             Status|REPORTED                    |CONFIRMED
           Assignee|dolphin-bugs-null at kde.org   |kio-bugs-null at kde.org
            Product|dolphin                     |frameworks-kio
           Keywords|usability                   |efficiency, reproducible
                 CC|                            |kdelibs-bugs at kde.org,
                   |                            |voidpointertonull+bugskdeor
                   |                            |g at gmail.com
     Ever confirmed|0                           |1
            Version|22.12.3                     |5.110.0

--- Comment #1 from Pedro V <voidpointertonull+bugskdeorg at gmail.com> ---
I'm not sure if the issue I have is exactly the same, but given that the
symptom is familiar, figured I'd do some digging.
I tend to use Krusader with sftp which is a better target to reproduce such
troubles anyway given that it doesn't even require mounting. Found
`sftp://testhost/usr/share/man/man1/` to be a good test target on an Ubuntu
host with high latency.

Running `strace -p $(pidof sftp-server)` on the target host shows the slow
iteration with readlink and newfstatat alternating.
What's interesting though is that it starts with a large batch of newfstatat,
and apparently the slow iteration is only done over symbolic links. This seems
to be confirmed by being able to check out directories consisting of many
regular files rather quick.

I think my issue lies in SFTPWorker::listDir -> SFTPWorker::createUDSEntry ->
sftp_stat being called for each symlink back to back with synchronous I/O.
Likely a more generic solution is desired though than dealing with this
specifically, or additional framework features would be needed which would need
to be implemented for each KIO slave individually.
These approaches could help:
- Asynchronous I/O support to allow KIO slaves to queue up all I/O work in one
pass so there's only one latency hit per operation group
- Progressive directory listing especially as the file list is already
available early, and filling in the details later would be more desirable than
waiting for everything even though the extra info may not be needed at all

Likely no easy feat due to the need to work on slaves individually, but
apparently sftp was getting some related goods already with Harald Sitter
introducing some light coroutine usage and mentioning "sftp_aio is currently
being worked on upstream" which is definitely an encouraging direction.

Setting CONFIRMED as it's really a reproducible problem, and moving to
frameworks-kio as this isn't really Dolphin specific, at least I don't think
that's doing anything interesting to make the problem worse than what I can
observe with Krusader.
This isn't really a bug though as it's more of a design shortcoming, so
recommending dropping importance to wishlist.

-- 
You are receiving this mail because:
You are on the CC list for the bug.