modular kdelibs: packagers' view

[Heinz Wiesinger]

Sebastian Kügler <sebas <at> kde.org> writes:
> Let me quickly outline the situation. There has been a trend and desire to
> ship kdelibs (and other parts of our development platform, I'm just writing
> kdelibs here for convenience) as separate libraries, the reasons for this are:
>
>  * can be individually consumed by 3rd party developers, no either or decision
>    for kdelibs, helps 3rd party adoption
>  * kdelibs' purpose is not anymore just desktop, mobile use cases become 
more
>    important, more modularity helps to create leaner systems

I can totally see how modularity in code can help there. However, I don't quite
see why this has to affect packaging.

The biggest beneficiary of modular source tarballs are probably the developers.
Smaller source tarballs allows them to have more components of their everyday
life based on stable releases (maybe even distro packages, if the modularity
is passed down the chain), while only having to care about those small parts
they are actually developing on.

Looking at packaging the beneficiaries of small tarballs are most likely source
based distributions. If the upstream tarball is a big monolithic collection of
apps I can imagine them to have a hard time splitting it up, as it involves a
lot of logic duplication across build scripts. Smaller source tarballs could
help there.
Package based distributions are probably better off with those big monolithic
collections. Every distro has different packaging rules. Some distributions split
more than others, but splitting at packaging level is way easier to handle than
at source level. You only have to build *once* and split the compiled files into
separate packages, rather than having to issue *multiple* build steps to do the
same.

Being a packager myself I would say it is really hard to please both audiences.
Since developers are comfortable with using git already, and those repositories
are already split, it would make sense to concentrate on monolithic tarballs for
package based distributions. That leaves out source based distros, but maybe 
they can be convinced to use git checkouts as well.

As for the different platforms, maybe it makes sense to release different
monolithic sets of source tarballs per target platform. That would also avoid
confusion. We would then have one set of tarballs for kde-desktop, one for
kde-mobile and one for kde-tablet (or maybe a combined one for kde-active),
one for kde-server (who knows :) ), etc. Just thinking aloud here...

> I see a couple of things we can do here:
>
>  * Provide "traditional" tarballs, leading to relatively little disruption,
>    but means duplication as we provide different sets of tarballs that
>    overlap, might be more confusing than worth it
>
>  * Do the split once, try to prevent the git migration mess where we've
>    clearly not thought through the ramifications for release management, which
>    lead to much confusion and frustration among packagers

I guess the biggest fear around modular source tarballs is about giving birth
to a big fat mess. We've seen it in the past, and not once, but several times.
The example of GNOME was already mentioned, but there is X.org as well.
They've modularized the code base with one aim being getting driver releases
out to the users more quickly. Have they achieved their goal? Maybe. Did it
improve the overall situation? Not really. Today it's pretty much based on
luck whether you get a working set of mesa, libdrm and all other xorg drivers.

With kde shipping modular source tarballs as part of a kde-sc, we are not far
off from a X11 release. Right now release criteria are quite strong, but
convenience might win over time and who tells us that single components don't
start shipping intermediary releases at some point?

Dependencies are another big issue. KDE has never been very good at
documenting its dependencies. There was an entry a while ago on techbase
listing some dependencies per module. I couldn't find it anymore, maybe
it's still up somewhere, but it was largely incomplete to begin with.
For KDE 4.0 and 4.1 I went over all the CMakeLists.txt in the kde source
tarballs and noted dependencies to ease initial Slackware packaging of
those releases, and it was a huge list, far longer than what ever was
on techbase. Now consider this were around 10/15 source tarballs, imagine
having to do this for 100 or more.
This is certainly something KDE as upstream can help with, maybe as a
combined effort with distribution packages, but will it be done? How long
will it be maintained? Who will update such a list? Who checks that the
listed dependencies are actually correct and there's not one little
application that slipped through the cracks and messes everything up?

Most importantly, who checks that two modules don't depend on conflicting
versions of one third-party lib. Right now this is pretty easy to check,
since such dependency issues should be caught rather quickly within
a monolithic tarball, and conflicts between two monolithic tarballs
should be fairly easy and fast to find as well. The same procedures simply
don't work anymore if you're dealing with 50/75/100/... source tarballs.

I'm aware that I'm asking way more questions than I deliver possible
solutions for, but asking questions is the first step at getting issues
resolved :)

Cheers,
Heinz