Module layout proposal: Split kdegames-data from kdegames

[Alexander Neundorf]

On Friday 14 October 2011, Stefan Majewsky wrote:
> [CC kde-scm-interest for notification only]
> [CC kde-buildsystem for feedback on the proposed build system changes]
> [CC kde-packagers for feedback on the implied changes to package layouts]
> [@CC: please keep discussion on k-c-d and k-g-d only]
> 
> Moin moin,
> 
> ==== EXECUTIVE SUMMARY ====
> 
> I propose to move the data files from the kdegames module into a new
> kdegames-data module to
> 1. facilitate the move of the remaining source code to Git (while a
> method of storing binary data files in Git efficiently is still being
> worked on) and
> 2. enable developers to fetch and compile the kdegames source without
> having to download the data files again.
> 
> ==== DETAILED PROPOSAL ====
> 
> kdegames is among the few modules that have not yet switched to Git.
> The main concern is that the kdegames source tree contains tons [1] of
> binary data files, which Git is known not to handle well. All
> discussions about moving to Git (on scm-interest and games-devel) have
> just let to bikeshedding about how to handle the binary files with
> git. I propose to postpone this specific problem and move on with the
> Git transition *now*, especially since the solution I want to propose
> has added benefit.
> 
> I propose to split a new module kdegames-data from kdegames, meaning that:
> 
> 1. kdegames-data should be built and installed before kdegames.
> 2. Any kdegames application will refuse to start when the
> corresponding data files have not been installed.
> 
> Number 2 is a protective measure because, currently, about all games
> won't run correctly or look utterly broken without proper indication
> of the problem.
> 
> If this proposal is implemented, the kdegames module, that is: the
> source code, can move to Git without worrying about the data files.
> These can move at any later point in time, when a wise solution has
> been worked out by our Git specialists. I'll repeat to get this clear:
> I'm not proposing to let the data stay in SVN forever. What I want is
> to disentangle the fates of the source code and the data files, for
> the benefit of both.
> 
> The added benefit of this solution is that distributions will be
> encouraged to package data files separately. Because this data (and
> esp. its format) changes very seldomly only, developers will not need
> to checkout this giant mass of data from our SCM servers, but can
> instead use the packages provided by their distribution. 

IMO today with usually broadband internet access this shouldn't be much of a 
concern (especially if these files change only rarely).

> The same
> holds true for drive-by contributors: They only have to clone a tiny
> repo containing the source code of the game which they want to hack
> on, without fetching megabytes of data files which they already had
> installed.

Personally I'd say "who cares". At least I don't check anymore how big a 
checkout will be before doing the checkout.
 
> If this proposal is accepted not later than by the end of next week, I
> will be able to implement the following changes before the soft
> feature freeze (Thu, October 27):
> 
> 1. Create the new module in SVN, move data files from the current
> kdegames module. The toplevel directory is still divided by
> applications, of course.

I'm not sure I really like this.
The general trend is to split per-application, so there would be e.g. one 
repository kolf and one for ksquares.
If data/ goes into one separate repository, this will make it harder once the 
sources are split into multiple repositories.
Either each small game would depend on all data files, or each small game 
would consist of two packages, one for the data and one for the actual 
program.
Or will packager take care of this and create nice packages ?

> 2. Setup the buildsystem for kdegames-data using
> macro_optional_add_subdirectory() as in kdegames, so that data can be
> selected for installation on a per-application basis. Each set of
> application data will additionally install an empty marker file which
> indicates that data is available for this application (something like
> ${DATA_INSTALL_DIR}/${APP}/hasdata; please suggest better schemes if
> there are).

Would it be possible to simply check whether the directory exists ?
${DATA_INSTALL_DIR}/${APP}/Data/ or something like this ?
 
> My roadmap also includes the following changes to be implemented
> before the hard feature freeze (Thu, November 10):
> 
> 3. Adjust the buildsystem of kdegames to exclude applications from
> compilation when required data files are missing. The exclusion can be
> overridden by a documented CMake switch. This is consistent with how
> other runtime dependencies of kdegames are handled (see
> kdegames/kajongg/CMakeLists.txt).

Hmm, not sure.
I'd prefer if it would also build when the data files are not present (since 
they are not required for building).

> 4. Make all applications abort with a visible warning when data files
> are missing.

This sounds good.

Overall I think it looks like a reasonable plan, if it is really necessary to 
keep the binary files out of git (I'd much prefer if they could simply be put 
into git).
 
Alex