Yet more "preloading"

[Oswald Buddenhagen]

On Fri, Mar 19, 2004 at 04:12:39PM +0100, Michael Matz wrote:
> Simply dumping the filled pagecache looks more elegant ;-)
> 
yes, but i suspect this is _the_ bottleneck of the concept, and the
reason why it gave only 10% improvement. if you order the data and
request it all at once, you can get two results: 1) if you do it sync,
the system will do nothing but waiting for the disk for a while. the
total time will be smaller due to the ordering, but not too much. 2) if
you do it async, the requested data often won't be preloaded in time, as
the disk was busy fetching other requested data in order. towards the
end of the boot process the hit rate will increase, obviously.

the solution is obvious: to maximize the effect, the data needs to be
read in the order it will be needed. "sequencializing" the i/o should be
done only if it does not affect the local deadlines (much). in fact,
this is a usual optimization problem with quite some parameters ...
happy solving. :)
the implementation of the proper fetch scheduling can be done in two
ways: 1) request the data not before the deadline minus the expected
latency. this could be done from user space. 2) request all at once,
but put a priority on every prefetch request. this has the advantage,
that the kernel can sequencialize more (i.e., execute the really cheap
reads longer before the deadline if they are "on the way"), but it needs
kernel support, obviously.

as for the parallelization of init.d: i think this is counter-productive
as far as disk i/o is concerned; we explicitly unparallelized part of
the kde startup because of trashing. of course a better disk i/o
scheduler (and the above prefetching) re{duce,move} the trashing, so
more parallelity makes sense again.
conceptually, i like the idea of a Makefile-based init.d; i pondered
with a dependency-based init.d years ago, but came to the conclusion
that maintaining it concurrently to the evolution of the used
distribution would be a nightmare, and i had no chance to get such a
radical change of the init.d paradigm into any distribution ...

greetings

-- 
Hi! I'm a .signature virus! Copy me into your ~/.signature, please!
--
Chaos, panic, and disorder - my work here is done.