Reducing library dependancies

[Thiago Macieira]

Maksim Orlovich wrote:
>> Yes, results attached. Command was:
>>
>> eu-readelf -r libkdeui.so | \
>> 	awk '/386_(32|GLOB_DAT)/ {print $4}' | \
>> 	uniq -c | \
>> 	sort -n -r | \
>> 	c++filt
>>
>> The number to the left is the number of times the symbol is used.
>> There are
>> 1551 distinct symbols in use, 936 of which are used only once, 261
>> twice, 91
>> three times (or 1288 symbols for 1731 relocations).
>>
>> You will notice that the symbols most used are those required for the
>> virtual
>> tables. That means we have 244 non-trivial classes in kdeui, and at
>> least 226
>> of those derive ultimately from QObject.
>
>Thanks for the numbers; however isn't multiplicity far less relevant
> these days?
>IIRC, the relocations are sorted to have duplicates adjacent, and ld.so
>has a last-entry cache.

That's correct. If you examine the output of eu-readelf -r, you'll see 
that the symbols are all sequential in naming.

That means ld.so has to calculate the relocation's value only once, but it 
still has to replace the value many times (technically, add to what's in 
the executable already).

>I am also getting different numbers with the 
>kubuntu build  here --- nearly 3000 distinct entries. The difference
>appears to be due to various virtuals inside kdeui proper. Did you build
>kdeui with -Bsymbolic or something? Still, 3000 isn't quite as bad as
>10,000...

Yes, I built with -Bsymbolic-functions. That means kdeui has a simpler 
relocation type for anything that resolves to itself at link time.

Instead of having to look up a symbol, all ld.so has to do is add the 
value of the base address the library was loaded to. About 5000 times.

That also means you cannot do symbol interposition.

>It does seem like there isn't much we can do on our end (there do seem
> to be about 75 symbols from private classes leaking out, but that's
> certainly not a huge fraction).
>It seems that even w/o a new relocation type it should be possible to
>avoid the in-library symbolic relocations to vtables and the like,
>however. Out-of-library they would still be needed, of course.

-Bsymbolic-functions helps with in-library relocations.

The more general -Bsymbolic option has side-effects due to the fact that 
x86 final applications aren't position independent. Platforms with 
position independent executables could use -Bsymbolic and gain a few more 
relocations.

>> The only way to reduce this count is to convince the gcc and binutils
>> people
>> that a relocation type specific for virtual tables is a must.
>
>This definitely does seem optimizable at that level, since slots in the
>child vtable
>can be merely copied from the parent one, so there isn't really need to
> do symbolic resolution beyond the initial vtable symbol.  It does
> introduce a recursive resolution dependency, however, but that symbol
> can be determined statically, too.

It's probably not an easy problem to solve, like you mentioned. For one 
thing, ld.so would have to resolve vtables in the proper order (base 
class to derived class). And it has to also leave intact the overridden 
functions.

>(I am also not sure of things like ti-merging and single definition rule
>with stuff emitted all over the place...).
>
>[I suppose one can also override every single virtual with a forwarding
>stub, which would result in a lazy PLT entry; I guess objprelink used to
>do something similar?]

That could work. The entire vtable would have only relative relocations -- 
either to functions defined in the module being loaded, or to its PLT. 
The emitted typeinfo class would still require an absolute relocation to 
the parent class -- the same way that the meta object does.

But it could require an update to the C++ ABI, since it could affect the 
way pointer-to-member-functions are represented.

Alternatively, they could dump completely the current vtable format and 
instead do it completely in code, via a dispatcher function not unlike 
qt_metacall (except done in assembly).

-- 
  Thiago Macieira  -  thiago (AT) macieira.info - thiago (AT) kde.org
    PGP/GPG: 0x6EF45358; fingerprint:
    E067 918B B660 DBD1 105C  966C 33F5 F005 6EF4 5358
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20080804/5f8611d0/attachment.sig>