[Kde-scm-interest] Package splitting

[Maciej Mrozowski]

On Thursday 28 of January 2010 03:05:21 Maciej Mrozowski wrote:
> On Thursday 28 of January 2010 00:58:59 Oswald Buddenhagen wrote:
> > On Wed, Jan 27, 2010 at 10:46:52PM +0100, Thiago Macieira wrote:
> > > [...] The build rules are: [...]
> > 
> > why not (mostly) keep our current layout in form of a meta structure (be
> > it git submodules or not)? build order would be determined by cmakelists
> > which declare subdirectories (and *nothing* else) as ever. breaking out
> > of that layout would happen at own peril, but any breakage will be
> > quickly detected anyway (as the projects' cmakelists must be
> > self-contained and dependency-complete).
> > 
> > On Wed, Jan 27, 2010 at 10:56:21PM +0100, Thiago Macieira wrote:
> > > And here's a suggestion to address those points, so a modification to
> > > the rules in the earlier email.
> > > 
> > > Instead of having 4 stages per category, we actually have:
> > > 	libs
> > > 	libs2
> > > 	libs3
> > > 	...
> > 
> > you know, sysvinit is now moving away from fixed priorities to a
> > dependency-based system ... ;)
> > anyway, my private build script does effectively the same, so it isn't
> > that bad as such. :D
> 
> I'll drop in some comments here as in Gentoo we run into similar problems
> when trying to migrate to git (we did not do it yet, details follows).
> 
> Mentioned atomicity
> 
> This is actually what (I think - unfortunately) caused that our package
> repository is not going to be split into subprojects (like KDE, GNOME,
> WebApps, Java, and such). Our repository is in following format:
> category1/package1/package1-version.ebuild (along with some other files)
> category2/package2/package2-version.ebuild
> ...
> profiles/<subprofile> (along with many other files)
> 
> (skip to "Git dependencies" if you don't care)
> There are inter-dependencies (also mutual - like for instance blocks)
> between packages, and what's the most important - profiles/ subdirectory
> contains repository-wide settings, so the simplest approach is to provide
> atomicity by keeping all in one repository.
> It's all fine with this, but one problem - purely technical - repository is
> huge enough (not as big as KDE one though) to make cloning it directly
> impractical as repacking on server side will simply kill the server (thus
> clones has to be delivered in alternative way - like git bundles or via
> rsync - and direct cloning explicitly disabled on server by the means of
> patching git).
> This is what actually holds our migration as splitting repository would
> need extensive changes to our package manager - repository dependencies,
> making profiles/ local to repository, and creating for instance master
> repository holding only profiles/ - global settings - and adding it as
> dependency for any other.
> 
> Git "dependencies"
> 
> This is quite similar to what you're trying to achieve with KDE - as you
> need to have some means to 'fake' git repository dependencies - either by
> the meaning "exactly this version of kdelibs.git" or more likely -
> "version of kdelibs.git at least".
> 
> To be frank I don't exactly understand how would you achieve it
> automatically - manually - it would need to create some metadata file in
> toplevel directory of particular git repository for instance with
> following format (here as bash array but that's just example - file
> metadata toplevel repo dir):
> 
> need=(
>    <path_to_repo>:<branch>[:<commit>]
>    ...
> )
> 
> Where <path_to_repo> would be relative path to particular repository - for
> instance 'kdelibs.git' or 'kdegames/libkdegames.git', <branch> being the
> name of the branch ("4.4", "master" etc) and optional <commit> would be
> any name that git understands as a commit identifier - so tag, sha1. For
> correctness branch 'tip' names (like 'master') should probably be
> forbidden.
> 
> I really don't think there's need to invent new layout - your module layout
> on server is quite nice already (maybe just add 'playground' and review
> 'subdirectories' there like Thiago proposed_, so (layout on server) :
> cmakemodules.git (read below)
> kdelibs.git
> kdegames/
>   libkdegames.git
>   kpat.git
>   review/
>     somnewshinygame.git
>   playground/
>     somebleedingedgegame.git
> 
> Of course git itself doesn't understand any dependencies so you'd need to
> provide some scripts to clone repository AND recursively its dependencies
> (along with local hooks maybe to update those dependencies to match
> requirements in metadata - also this metadata would need to be updated
> manually when needed (for all remote branches independently).
> 
> CMake modules
> 
> I'd propose separate repository for them and adding it as a dependency of
> all other repositories that need it (also with version specifier) - it
> would prevent .cmake modules duplication. Update/clone script would need
> to clone this cmake modules repo to known local location (actually
> locations of all local clones need to be well defined if one is to
> include_directories from them for instance), so that all other projects
> (repositories) would just add in first line of CMakeLists.txt:
> 
> set (CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../cmakemodules/")
> 
> All this still need writing script to recursively create tags, tarballs
> (there's still possibility to package it the way it is now - as modules),
> recursively update dependencies (for instance working) or just update
> dependencies.

@SaroEngels

Couldn't your emerge script (used for Win32 KDE packaging) be extended to 
support git cloning/pulling and implement git dependencies?

> As for work flow
> 
> I think Thiago is on the spot that it's much safer to work on stable branch
> and forward-port bugfixes instead. Personally - there was very few of them
> - but it happened that I needed to reopen KDE bug marked as RESOLVED/FIXED
> just after fix has been commited to stable branch.
> 
> My .5

-- 
regards
MM