Thread Scheduling

[Milian Wolff]

David Nolden, 14.12.2010:
> I don't see the basic issue. The jobs will get as much file I/O as the
> system can deliver, so where is the problem? Some thread will always
> starve, when the CPU is faster than the disks.. however by using
> multiple threads, we can at least have some threads working while
> others are starving.
> 
> Such a cap would not be useful at all. Those I/O intensive jobs
> (especially parsing jobs) won't be doing I/O all the time, after all.
> Also I/O does not only have a "bandwidth" which is shared and allows
> threads to starve each other, but it also has _latencies_ which can be
> more or less independent of the number of threads. On my machine at
> work for example, we have the disks mounted through NFS, and I have to
> raise the thread-count to 8 (!) so that the project-parsing has an
> acceptable speed (due to network latency I guess).

Interesting, this is pretty much the opposite of what Qt teaches in its 
trainings. There is an example shipped with Qt that basically creates 
thumbnails for images. According to the training, it will be slower to create 
lots of threads that all thrash the IO instead of having only a few.

The reasoning is this: With fewer concurrent accesses to the HDD it will spin 
less and read more. When two threads read from the same HDD but at different 
places, the kernel will deliver those requests and the HDD will read a few 
bytes here, spin, read a few bytes there, spin back and repeat...

Is this not correct, or did I misunderstand something? I find it very logical.

Note: I'm not in favor of removing threads, I just want to make them all use 
the global threadweaver and configure that centrally. I.e. use $user-defined-
number threads for all IO operations, be it parse jobs, import jobs, replace 
jobs.

Bye
-- 
Milian Wolff
mail at milianw.de
http://milianw.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20101214/d61fff3b/attachment.sig>