[Kde-games-devel] Re: A Thought About The Version Control System

[Shlomi Fish]

Hi Wolfgang,

see below for my response.

On Monday 11 Apr 2011 09:55:10 Wolfgang Rohdewald wrote:
> On Montag 11 April 2011, Shlomi Fish wrote:
> > there was some discussion here about whether and how kdegames
> > should  transition from Subversion to Git, including a
> > discussion whether to keep everything in a single repository
> > or to split each games into a different repository. The
> > reason for the entire discussion is because due to the large
> > history and many binary files in the svn repository, the
> > resultant git repository will be extremely large.
> > 
> > What I suggest considering is keep the repository as a
> > Subversion one and  working with it remotely using Git,
> > Mercurial, Bzr or whatever, possibly while starting from a
> > later snapshot. The reason is that distributed version
> > control (DVCS) is not very suitable for games (though it is
> > much more apparent in large-scale commercial games than for
> > the KDE games package):
>
> I am glad you re-started the discussion, it seems to have
> fallen asleep somehow.
> 
> But I disagree with you - very much so.
> 
> Please define "extremely large" - how large is that? And
> how much does it matter? The full repository has to be
> downloaded only once. Updates do not mind how big the
> repository is.

Yes, but then it has to be kept on disk with the entire history.

> 
> Processors, download speed and storage get faster
> and cheaper by the day - so in 2 or 3 years people might
> find this discussion rather amusing in hindsight. And git
> also moves on - look at the git mailing list. Better
> support for binary files is very much discussed there
> and the git development rate is still very fast.

I discuss the argument that "we should expect processor speed/network 
speed/storage to keep up with our demands" here:

http://en.wikibooks.org/wiki/Optimizing_Code_for_Speed/Factor_Optimizations

(see "The Motivation for Factor Optimizations")

The problem is that checking out everything from a git repository may be very 
slow due to either poor bandwidth, or the so-called ISP "traffic-shaping". For 
example, my connection should be something like 300 KiloBytes/s , but I'm 
getting a much lower bandwidth from most sites outside Israel, due to my ISP 
being sucky. 

Furthermore, git does an all-or-nothing clone - if you interrupt a git clone, 
the partial repository is deleted, and you need to restart the clone all over 
again. This is while a Subversion working copy checkout (or its svnsync 
command for cloning a repository from remote) *can* be resumed from a partial 
state.

I don't know how well the git people are planning to better support binary 
files inside a history, but I don't think there is a silver bullet for that. 
There's a limit to how much it can be done.

And regarding being on the git mailing list, see:

http://www.shlomifish.org/lkml-letter-f6y9/some-requests-from-kernel.org.xhtml

> 
> I am using git svn locally for kajongg. Yes, this is better
> than pure svn but many advantages of git are not usable over
> those bridges - especially when it comes to branching and
> merging.

OK.

> 
> binary files could be put into separate repositories -
> either one per game or one for all. That should be decided
> by the individual game developers. There are already data
> packages like kdegames-mahjongg-data - this could be one
> repository. Kajongg has some audio files but that is
> currently not much data - if and when that gets too much
> we can still start a separate repository kdegames-kajongg-data

That will add a lot of complexity to the process.

> 
> I also claim - without proof - that binary files are not
> changed as often as source files. So I do not believe they
> are a real problem. 

The problem is that the many binary files take a lot of space in the history, 
where they are stored as big binary blobs. Cloning a repository with this is 
time-consuming and occupies a lot of space on the disk.

> And if it later turns out that they are,
> they can still be split out to binary file repositories. 

Making the "git clone" of the repository non-usable, and adding complexity.

> Yes,
> that means rewriting history (git filter-branch) and yes,
> that means all developers have to rebase their local work
> because all git commits change. 

I don't know git intimately enough (and would rather avoid learning it too 
much because it's so complex, idiosyncratic and counter-intuitive).

> But honestly - kdegames
> development is really not very active, only a few developers
> will notice at all.

OK.

> 
> I forgot to mention that I am all for splitting kdegames
> into a repository per game. Otherwise splitting out binaries
> at a later point might be more problematic. I am sure mechanims
> will develop for automatically resolving dependencies between
> repositories so the argument that this makes it too difficult
> for newcomers to git into KDE development is not that strong.
> 

Regarding "I am sure" - "[citation needed]". Splitting the games into a 
different repository per game will add complexity to the process, and will 
make maintenance harder (including branching, tagging, refactoring, etc.) and 
developers won't like it.

> and of course nobody from the outside (of the games developer
> community) will understand and accept it if kdegames stays
> with svn. The entire infrastructure of all KDE parts
> (except i18n for now) is moving to git - AFAIK. If kdegames
> stays with svn, a lot of the infrastructure has to be
> maintained twice. Guess who will maintain the svn part - nobody!

I volunteer to help in maintaining this. And a Subversion service does not 
require a lot of maintenance. In any case, the KDE services are a tool and 
should serve the needs of the users and developers, and we should not go to 
too many extra lengths to twist our workflow so it will match what is a bit 
easier for the admins. Tools are tools, and should not be our master.

Regards,

	Shlomi Fish

-- 
-----------------------------------------------------------------
Shlomi Fish       http://www.shlomifish.org/
My Aphorisms - http://www.shlomifish.org/humour.html

Satan condemned Hitler for a million years of writing XSLT.

Please reply to list if it's a mailing list post - http://shlom.in/reply .