preloading of binaries

[Dr. Juergen Pfennig]

On Monday 08 March 2004 14:19, Lubos Lunak wrote:

>  I'd need some people to test the the following hack. In startkde, before
> KDE is loaded, a small utility mmap()'s basic large libraries used by KDE
> and walks over them, causing them to be all loaded into memory, and they'll
> stay cached (I think OO.o uses a similar hack). As the mmap() is done with
> MADV_WILLNEED, this should cause sequential reads of large chunks from the
> disk[1], instead of small random access reads which kernel uses otherwise
> as the parts of the binaries are paged in[2].

It's good to make things better, but please don't do this hack. I am the 
author of a little video edit application (zapcut and zapcvt) that is using 
mmap. I played a lot with mmap. At least the 2.4 Kernel seems to ignore 
MADV_WILLNEED (or it has no effect). Most of the time used to load a library 
will be spent matching import/export symbols. Have GNU/KDE people ever read 
the documentation about the M$s rebase utility?

You mention OO.o - but please be aware that M$ Office also plays that trick - 
I believe that they load their DLLs to their Office quick-starter. Maybe they 
have a patent on that trick?

Did you also try just to open the .so files to make the Kernel cache the 
directory entries? In the M$ case finding a DLL took up to 200ms some years 
ago. This was the reason for IBM to add the bind command to their OS/2. With 
M$ the system DLLs are bound. It could be that GNU does this via ldconfig.

But please: under no circumstance read the whole files via mmap. 30'% of it is 
symbol info and large parts may be messages or code that is never used. What 
about just reading the .dynamic section (or what the precise name is). Your 
hack (reading the whole file) could also cause a desaster on machines that 
are under memory stress!

Here a suggestion: I believe that it was Coolo who was extending the kdeinit 
launch trick for KDE 3.2. That works very nice because the time to match 
import/export symbol tables is saved (kdeinit preloads some .so objects?). 
Why don't you work on some tricks to extend kdeinit further?

Yours
Jürgen