Thread Scheduling

[David Nolden]

I don't see the basic issue. The jobs will get as much file I/O as the
system can deliver, so where is the problem? Some thread will always
starve, when the CPU is faster than the disks.. however by using
multiple threads, we can at least have some threads working while
others are starving.

Such a cap would not be useful at all. Those I/O intensive jobs
(especially parsing jobs) won't be doing I/O all the time, after all.
Also I/O does not only have a "bandwidth" which is shared and allows
threads to starve each other, but it also has _latencies_ which can be
more or less independent of the number of threads. On my machine at
work for example, we have the disks mounted through NFS, and I have to
raise the thread-count to 8 (!) so that the project-parsing has an
acceptable speed (due to network latency I guess).

Greetings, David