KIO directory listing - CPU slows down SSD

[David Faure]

On Sunday 11 May 2014 21:57:58 Mark Gaiser wrote:
> My theoretical solution (and i really hope to get feedback on this) is
> to introduce a worker thread in slavebase.cpp.

First feedback: threads create complexity, so I'm very wary of using them 
unless absolutely necessary. In any case, *if* there's a good reason for one, 
do it in kio_file itself, don't impose it on all slaves (which are not ready 
for it).

> The slave should be
> reduced in functionality. It should not make a UDSEntry anymore.
> Instead, it should send the STAT object to slavebase.cpp (no function
> for it yet). SlaveBase then puts it in a QVector object which would be
> shared with the actual worker thread. The thread would only read from
> the QVector which takes away the need to care for thread safety.

Wrong. A read and a write to the same area of memory create a race condition, 
by definition. You need to protect the vector with a mutex.

Think of the case where append() has to reallocate, too... (but even if you 
take that case away with a reserve() call, you still have a race on the 
individual vector items, unless you use a mutex).

> The
> workerthread should then process the entries (in a new short lived
> vector i guess), create UDSEntry objects and send them over the socket
> connection to the client.

Your analysis is that currently this happens:
I/O, CPU, I/O, CPU, I/O, CPU, I/O.
You want to use threads so that one thread does
I/O, I/O, I/O and the other receives that and does CPU, CPU, CPU.
I'm not sure this is going to even be faster, the additional overhead (context 
switching on single-cpu machines, intermediate data structures, mutex 
protection) will probably be bigger than the gain, since you're not using 
thread for what they were meant for, when it comes to performance: CPU 
operations being run in parallel over multiple threads.

I would favour much more an approach that reduces the amount of CPU operations 
needed (-> making UDSEntry faster) than an approach that uses threads for 
this.

-- 
David Faure, faure at kde.org, http://www.davidfaure.fr
Working on KDE Frameworks 5