[frameworks-kio] [Bug 434175] Move to wastebin functionality is so slow that is unusable when it already has a lot of stuff in it

[Pedro V]

https://bugs.kde.org/show_bug.cgi?id=434175

Pedro V <voidpointertonull+bugskdeorg at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |voidpointertonull+bugskdeor
                   |                            |g at gmail.com

--- Comment #15 from Pedro V <voidpointertonull+bugskdeorg at gmail.com> ---
I also got to "enjoy" this problem quite a few time as I found KDE's trash
function neat to avoid the rare but quite serious mistake of accidentally
deleting the wrong file, so it was not uncommon to have a ton of waste in the
trash that was likely not needed.
Having something there already is not a requirement though, just trashing tens
or hundreds of thousands of files also leads to a slowdown over time.

Started digging into what could be the problem, and while I'm not familiar with
the details, I've found some quite interesting answers to even questions I
thought would be unrelated here:
https://invent.kde.org/frameworks/kio/-/blob/3942b233a777a4fa9e9ab45c8c3b9da58309822f/src/kioworkers/trash/trashimpl.cpp#L1269

So apparently the trash settings regarding size and time limit are enforced
every time a file is trashed which is somewhat sensible, the process is just
not too efficient.
Initially I thought that the time limit was implemented with some time-based
scheduling as that one doesn't actually need to be checked every time there's a
file operation, but then on the other hand something needs to take care of that
which might not exist.

Do note that apparently the code assumes that trash directories can just appear
(
https://invent.kde.org/frameworks/kio/-/blob/3942b233a777a4fa9e9ab45c8c3b9da58309822f/src/kioworkers/trash/trashimpl.cpp#L686
), so we'll run with the idea of assuming that the filesystem is our database
and we can't have anything faster, also allowing files from the trash to just
simply disappear which does have its benefits.
Given that, we have these problems and/or possible ways to improve performance:
- As mentioned earlier, the time limit doesn't really need to be checked every
time a file is trashed, that could be moved elsewhere, likely there's just no
appropriate time-based scheduler for this currently. Also, the earlier
mentioned "hotplug" assumption can throw a wrench into such scheduling
- The `useTimeLimit` and `useSizeLimit` checks lead to separate iterations,
essentially going through the trash twice to gather file information
- The filesystem could be also used more efficiently than this, there's no need
to keep on rescanning, inotify could be used opportunistically (realistically
it should almost always work), and rescanning could be used only as a fallback
- Synchronous I/O operations could be replaced with asynchronous ones getting
pumped into a work queue (io_uring has all the required goods for that
already), so the slow QD1 userspace<->kernelspace dance could be replaced with
a high throughput solution utilizing a deep I/O queue welcome by even fast SSDs

Chances are high that this is a significant amount of work in an area which
isn't exactly looking too troublesome for common users as-is, so not expecting
a whole lot of magic here to happen any soon.
Given that, I'm recommending the following workaround when trashing a ton of
files at once which I just discovered and it seems like it will be neat the
next time I'm doing more than just an artificial test:
- Verify that there's enough space in the trash before beginning the operation
(assuming the files are on the same mount point as the trash, this is a no-op)
- Start the trashing process
- Open trash settings ("Configure trash settings")
- Remove ticks from "Cleanup:" and "Size:" options, hit "Apply"
- Wait for the large trashing operation to finish which should be significantly
faster at this point
- Add back ticks to the earlier unticked options, hit "OK" to apply them and
exit settings

-- 
You are receiving this mail because:
You are on the CC list for the bug.