Some libraries are not found without LD_LIBRARY_PATH mangling

[Thiago Macieira]

Em Tuesday 30 October 2007 14:44:51 Alexander Neundorf escreveu:
> > We're talking about running uninstalled applications that link to
> > uninstalled libraries. Linking to installed libraries has never been a
> > problem -- the RPATH to the install tree is present anyways.
>
> But not necessarily to RPATH to libraries that library links to, see below.

It doesn't have to. See below.

> > > then A will link just to that and not know about other libraries
> > > libsoprano.so links to (and their directories, which are required for
> > > the RPATH). If libsoprano.so doesn't find its libraries without RPATH
> > > (or RUNPATH), there are two options:
> >
> > Again, stop. libsoprano.so's dependencies are loaded by libsoprano.so
> > itself, not by the application. So it's the dependent library's job to
> > find the libraries the right way (environment variable, ld.so.conf
> > modification, RPATH, whatever).
>
> Yes, we agree here.
> What I still don't know is for which steps is the RPATH valid ?

DT_RPATH is only valid for the libraries loaded by that module's DT_NEEDED 
tags. It doesn't work indirectly.

> Say APP links to libA, and libA links to libB, and APP has an RPATH, is
> this RPATH also valid for finding libB from libA ?

No. libA has to know how to find libB on its own.

Test case:
$ ls -lR
.:
total 12
-rw-r--r-- 1 tmacieir tmacieir   45 2007-10-30 14:57 main.c
drwxr-xr-x 2 tmacieir tmacieir  100 2007-10-30 14:57 ONE/
drwxr-xr-x 2 tmacieir tmacieir  100 2007-10-30 14:58 TWO/

./ONE:
total 12
lrwxrwxrwx 1 tmacieir tmacieir   11 2007-10-30 14:57 libONE.so -> libONE.so.1*
-rwxr-xr-x 1 tmacieir tmacieir 5236 2007-10-30 14:54 libONE.so.1*
-rw-r--r-- 1 tmacieir tmacieir   21 2007-10-30 14:54 main.c

./TWO:
total 12
lrwxrwxrwx 1 tmacieir tmacieir   11 2007-10-30 14:58 libTWO.so -> libTWO.so.1*
-rwxr-xr-x 1 tmacieir tmacieir 5375 2007-10-30 14:57 libTWO.so.1*
-rw-r--r-- 1 tmacieir tmacieir   53 2007-10-30 14:55 main.c

$ cat ONE/main.c
int symbol_ONE() { }
$ ldd ONE/libONE.so
        linux-gate.so.1 =>  (0xb7fe7000)
        libc.so.6 => /lib/i686/libc.so.6 (0xb7e85000)
        /lib/ld-linux.so.2 (0x80000000)

$ cat TWO/main.c
int symbol_ONE(); int symbol_TWO() { symbol_ONE(); }
$ ldd TWO/libTWO.so
        linux-gate.so.1 =>  (0xb7fcd000)
        libONE.so.1 => not found
        libc.so.6 => /lib/i686/libc.so.6 (0xb7e6b000)
        /lib/ld-linux.so.2 (0x80000000)

So, at this point above, you can see that libTWO was compiled linking to 
libONE, but *without* any RPATH setting whatsoever.

If we try to link our test program:
$ cat main.c
int symbol_TWO(); int main() { symbol_TWO; }
$ gcc -o main -LTWO -lTWO main.c
/usr/bin/ld: warning: libONE.so.1, needed by TWO/libTWO.so, not found (try 
using -rpath or -rpath-link)
TWO/libTWO.so: undefined reference to `symbol_ONE'
collect2: ld returned 1 exit status

Oh, well, let's add the -rpath:
$ gcc -o main -LTWO -lTWO main.c -Wl,-rpath,$PWD/ONE:$PWD/TWO
$ ./main
./main: error while loading shared libraries: libONE.so.1: cannot open shared 
object file: No such file or directory

So you see that main's RPATH does *not* apply to libTWO's dependencies.

Now, if we make it a direct dependency:
$ gcc -o main -LTWO -LONE -lTWO -lONE main.c -Wl,-rpath,$PWD/ONE:$PWD/TWO
$ ./main

Works just fine.

Also works if -rpath is set in the dependent library:
$ ldd TWO/libTWO.so
        linux-gate.so.1 =>  (0xb7f50000)
        libONE.so.1 => /tmp/testdir/TWO/../ONE/libONE.so.1 (0xb7f4b000)
        libc.so.6 => /lib/i686/libc.so.6 (0xb7dec000)
        /lib/ld-linux.so.2 (0x80000000)
[rpath was added, so it finds its dependency]
$ gcc -o main -LTWO -lTWO main.c -Wl,-rpath,$PWD/TWO
$ ./main


Conclusions:
1) when linking to uninstalled libraries, we should link to them *all* (so as 
to make all uninstalled dependencies direct ones)
2) each one should clean up after itself: each library should be loadable on 
its own after installed. Having a "not found" in ldd's output means you 
messed up.

> Yes, but for anything except kdelibs any library from kdelibs is an
> "already installed library". Currently they don't get an RPATH at all
> (except with FULL_RPATH option), but when linking to them the full list of
> libs is used (which should produce the correct RPATH for the executable).
> Once the libraries get an RPATH, they already have it in the build tree or
> they will be relinked.

I didn't understand.

> > > > - on ELF platforms without DT_RUNPATH (IRIX, HP-UX):
> > > >   RUN_UNINSTALLED executables are relinked at install time
> > >
> > > This could make some things easier, I think we could even completely
> > > get rid of the RUN_UNINSTALLED argument. Everything, apps and libs
> > > would get the full RPATH (RUNPATH) for the install tree, the shell
> > > scripts which set LD_LIBRARY_PATH are created anyway.
> >
> > That's what I proposed for the platforms using DT_RUNPATH. But mind you:
>
> So we agree, right ?

Yes.

> > the real target (the ELF executable) should have a different name -- the
> > shell script should have the target's name inside the build tree.
>
> I don't feel like changing this now and I don't see a real reason for it.
> The shell script is currently <target>.shell, if you use tab completion you
> notice that there are two executables.

I see a reason for it: if those programs are run uninstalled, it's because 
there's a rule in CMake somewhere that is running them. Are we absolutely 
sure that all places calling them have the ".shell" if it's inside kdelibs 
but no ending if it's not?

> Yes, FULL_RPATH is ok, my only problem is the time the relinking takes.
> It's even better than RUNPATH+LD_LIBRARY_PATH (because you can run it
> directly and don't have to use a script).

FULL_RPATH+RUNPATH+LD_LIBRARY_PATH is the best solution I see. Distributions, 
however, probably won't use RPATH at all, so LD_LIBRARY_PATH also applies to 
them and works fine. (Then again, distributions don't have the installed 
libraries in their build environments, so even if RPATH is there, it will 
find no library)

As far as I can see, relinking is necessary only on AIX and on ELF without 
RUNPATH.

> I'll put chrpath on my TODO.

I did take a look at it once and my conclusion was that increasing the RPATH 
entry would require rewriting the entire ELF file. That's not an easy task 
and requires the tool to actually understand ELF. Don't forget that there are 
some ELF binaries that are several megabytes in size.

However, since we're talking about *removing* path entries from RPATH 
(removing the build tree path), it should be always possible, since you 
decrease the length of the tag.

-- 
  Thiago Macieira  -  thiago (AT) macieira.info - thiago (AT) kde.org
    PGP/GPG: 0x6EF45358; fingerprint:
    E067 918B B660 DBD1 105C  966C 33F5 F005 6EF4 5358
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url : http://mail.kde.org/pipermail/kde-buildsystem/attachments/20071030/a2674712/attachment.pgp