Some libraries are not found without LD_LIBRARY_PATH mangling

[Alexander Neundorf]

Hi,

On Tuesday 30 October 2007, Thiago Macieira wrote:
> Discussion moved from kde-core-devel. Link to thread:
> http://lists.kde.org/?t=119343241900001&r=1&w=2
>
> Em Monday 29 October 2007 16:45:01 Alexander Neundorf escreveu:
...
> By the way, this is how Qt solves this problem for moc and uic: it
> *doesn't* link them to the Qt libraries. Those programs are "bootstrapped":
> that is, they add the .cpp files directly to their compilation, therefore
> linking statically to the Qt stuff they need.
>
> So in a typical Qt build, qstring.cpp is compiled 5 times: once for QtCore,
> then one more time per tool (moc, uic, rcc and qmake).

Sure ? How about creating a static lib for these tools ?
While seems to be a good approach, do we want that for KDE ? I doubt that, 
there are several generators, not only in kdelibs.

> > > Another option: build those tools only with RPATH to the build tree,
> > > then relink them (and only them) during installation. This requires a
> > > full library list during linking: e.g., "-lkio -lkdeui -lkdecore", not
> > > just "-lkio". Doesn't CMake already do that?
> >
> > Well, the issue is that potentially any of the libraries could be used by
> > these tools, and it doesn't make sense to mark libraries as
> > RUN_UNINSTALLED, because the developer of the library doesn't necessarily
> > know that.
> >
> > CMake doesn't automatically detect which shared libraries are linked to
> > executables which have an RPATH which points into the build tree, in
> > order to guess to build these libraries also with RPATH pointing into the
> > build tree.
>
> What happens to a library marked RUN_UNINSTALLED?

Libraries are not marked as "RUN_UNINSTALLED".

> My point is: only the program has to be modified to find the libraries when
> they aren't installed yet. The libraries themselves don't need to be
> modified at all.
>
> So, when building a RUN_UNINSTALLED program, it is linked with full rpath
> to the build tree and full library list. That way, all libraries will be
> found in the build tree, even if the libraries themselves don't know
> anything about RUN_UNINSTALLED.

Apparently not. If app A links to soprano (or another 3rd party library), and 
the soprano cmake module just sets
SET(SOPRANO_LIBS /opt/soprano/lib/libsoprano.so)
then A will link just to that and not know about other libraries libsoprano.so 
links to (and their directories, which are required for the RPATH). If 
libsoprano.so doesn't find its libraries without RPATH (or RUNPATH), there 
are two options:

1) it needs an RPATH itself
2) the cmake module must provide a variable which either lists all dependent 
libraries with their full path and this must be used when linking to 
libsoprano.so, or it must provide a SOPRANO_LINK_FLAGS which lists the 
additional RPATH directories

The first option sounds much better to me.

If the library comes from within the same build tree, CMake should add all the 
directories for the RPATH.

> > > Maybe an even better option: set LD_LIBRARY_PATH when running
> > > uninstalled. That variable overrides RUNPATH and any distribution not
> > > using that (read: using RPATH only) should be shot in the head.
> > > DT_RUNPATH has been available for more than 5 years.
> >
> > While it probably is available on most Linux systems today, I don't know
> > if this is also true for *BSD, Solaris, HP-UX etc.
>
> FreeBSD has been using RUNPATH exclusively for years. They have already
> dumped the old and conceptually broken DT_RPATH. This is what irks me: why
> the hell hasn't Linux done the same? Support for DT_RUNPATH has been around
> for 8 years in glibc, some 5 years or more in the linker.
>
> Of the platforms we can support, we have two groups:
>
> AIX, MacOS X, Win32: their own executable formats
> All others (Linux, the BSDS, Solaris, IRIX, HP-UX): ELF
>
> I have no idea about AIX's file format. All I know is it's COFF-based (it's
> called XCOFF). Reading through the ld(1) man page on AIX indicates that the
> linker hardcodes the -L switches to the final executable. In other words:
> the AIX linker does not like linking to uninstalled libraries at all. It
> would indicate that relinking at install time is mandatory on AIX (so as to
> get rid of the build-tree references).
>
> I also try to steer away from MacOS X's Mach-O format. It's very confusing.
> But I know it supports changing the linker paths of already-created
> executables without relinking.

Yes, that's cool, it does the right thing and it does it very fast :-)

> As for Win32's COFF Portable Executable format, the interesting thing is
> that all we need to do is put the DLLs in the same directory as the
> executable itself. There's no need to relink: the same executable and same
> libraries are valid for uninstalled- and installed-execution. We do place
> them all in bin/, don't we?

We don't, see the lengthy discussion on k-c-d and here about it. We may do it 
or KDE 4.1. Setting PATH is required or the installer has to do it.

> For IRIX, it looks like DT_RUNPATH is not supported. It's not mentioned
> anywhere in the ld(1) or rld(5) manuals.
>
> The same seems to apply to HP-UX.
>
> For Solaris, it's using DT_RUNPATH on normal builds (Solaris 9 tested).
>
> > I don't think CMake right now supports setting only RUNPATH but not RPATH
> > due to this. Even if it would be implemented today, it wouldn't help us
> > since we require CMake 2.4.5 for KDE 4.0.x.
>
> There's no way to set only RUNPATH (without RPATH) without modifying the
> linker. But this is not anything that affects us: if DT_RUNPATH is present,
> the DT_RPATH entry *must* be ignored by the dynamic linker.

Ah, I didn't know that.
So if enable-new-dtags is in the link flags, -Wl,-rpath sets the RUNPATH 
instead of the RPATH ?

> So, the search order is this, on ELF platforms:
> 	RPATH, unless RUNPATH is present
> 	system-specific environment variables (LD_LIBRARY64_PATH, etc.)
> 	LD_LIBRARY_PATH
> 	RUNPATH
> 	system configuration
> 	system defaults
>
> > Having something like install_name_tool would be nice :-)
> > There is chrpath but this is not widely installed and works only if the
> > new RPATH is not longer than the original one, so it would need some
> > support from the linker, so enough space is reserved. OS X does that.

...
> So, let me summarise what I am proposing:
>
> - on MacOS X: install_name_tool
> - on Windows: no action necessary - at most, set PATH
> - on ELF platforms with DT_RUNPATH (Solaris, Linux, BSDs):
>   set LD_LIBRARY_PATH from a shell script and exec
>   no relinking necessary

> - on AIX: always relink at install time, regardless of any CMake settings

I would just assume that CMake does the right thing, Kitware builds almost all 
of their projects/products on almost all platforms, e.g. VTK is built every 
night also on AIX.
The first person who will try to build KDE4 on AIX will tell us :-)

> - on ELF platforms without DT_RUNPATH (IRIX, HP-UX):
>   RUN_UNINSTALLED executables are relinked at install time

This could make some things easier, I think we could even completely get rid 
of the RUN_UNINSTALLED argument. Everything, apps and libs would get the full 
RPATH (RUNPATH) for the install tree, the shell scripts which set 
LD_LIBRARY_PATH are created anyway. 
I'd consider AIX, IRIX and HP-UX not our main target platforms, so I'd say it 
shouldn't be too much of a problem if on these platforms relinking always 
happens, for everything which has an RPATH.

Alex

P.S. actually the platform is intended to be frozen, and this is a significant 
change...