Using madvise for ld.so

[Lubos Lunak]

On Wednesday 17 of March 2004 19:21, Andrew Morton wrote:
> Michael Matz <matz at kde.org> wrote:
> > Hi Andrew,
> >
> > On Tue, 16 Mar 2004, Andrew Morton wrote:
> > > > On Tue, 16 Mar 2004 21:00:26 +0100, Dirk Mueller <mueller at kde.org> 
wrote:
> > > > > might be a stupid question, but how did you "clear the cache" ?
> > > >
> > > > the usual "use memory till oom" thing (included in an earlier email
> > > > iirc) i wish the kernel guys would sort this out which a proper
> > > > solution.
> > >
> > > In 2.6 you can run posix_fadvise(fd, start, length,
> > > POSIX_FADV_DONTNEED).
> > >
> > > I've attached a little proggy which uses fadvise to shoot down some or
> > > all of a file's pagecache.
> >
> > This is only for one file, though.  I.e. one would need to do /proc
> > parsing magic to really clean the whole file cache (for all files not
> > open anymore).  I think the "alloc all mem, touch all pages" approach is
> > easier ;)
>
> True.
>
> You want to kill all pagecache to simulate cold startup?  No, we don't have
> a way of doing that apart from the force-a-swapstorm approach which you're
> using.  That's more valid anyway, because it will also cause reclaim of the
> inode and directory entry caches.

 I have a somewhat opposite question, related to the 'yet more preloading' 
post. Just like this ld.so patch (and the hack that started this) speeds up 
loading of large binaries by loading them sequentially in a large chunk, is 
there some possibility to do the equivalent for inodes?

 I.e. I used the scan binary to simply recursively search and stat every file 
under the given path. Would it be possible to do simply do a large read from 
the HDD that would read all that information? With the MSDOS fs, if I still 
remember that thing a bit, I could basically just read the whole area that's 
FAT, although that wouldn't help with directories much, as they can be 
scattered if I'm not mistaken.

>
> > Which reminds me of a question I asked myself: If I open the file,
> > fadvise a WILLNEED on it, all its pages are scheduled for IO.  Completion
> > is not waited for.  So, if the program then exits, the file gets closed,
> > and assuming that was the last ref to that inode: will the not yet
> > completed IO be canceled or will it still be completed?
>
> The I/O will not be cancelled.

 Is this the case also with mmap+madvise+munmap? It seems a bit strange, 
although for these preloading tricks it's indeed handy.

-- 
Lubos Lunak
KDE developer
---------------------------------------------------------------------
SuSE CR, s.r.o.  e-mail: l.lunak at suse.cz , l.lunak at kde.org
Drahobejlova 27  tel: +420 2 9654 2373
190 00 Praha 9   fax: +420 2 9654 2374
Czech Republic   http://www.suse.cz/