[Kde-scm-interest] Meeting minutes

[Oswald Buddenhagen]

On Mon, Nov 16, 2009 at 06:19:19PM +0100, Thiago Macieira wrote:
> Em Segunda-feira 16 Novembro 2009, às 09:28:31, você escreveu:
> > Git scales well with repositories, it does not scale well with the number
> >  or files inside a repository. That applies to the performance as well as
> >  the merging woes that you get.
> 
> I agree on the number-of-files issue, but not on the merge one.
> 
i disagree. see below.

> Besides, the number of files only applies per repository. If you look
> at your development model, you'll still have the same number of files.
> They'll just be split across multiple Git repositories.
> 
correct.
the merge problem is similar to the "n people in a room, how
likely is it that two have birthday on the same day" problem.
by making the rooms smaller and thus having fewer people in them
(because most people are interested only in particular "booths"), the
probability of a clash goes down *drastically*.

> By doing that, you'll introduce other issues:

> - git status no longer shows other files modified (svn doesn't either,
> but in Git it helps)
>
that's minor, and should be automated by the meta repo management tool
anyway.

> - the sizes of the Git object dbs are larger
>
i think you posted some numbers before, but would you mind refreshing
our memory?
irrespective of the actual numbers i'd venture the guess that on
average, individual developers would still save disk space, as they
wouldn't have to download stuff they have no interest in. that applies
to app developers in particular.

> - the module doesn't match the tarball
> 
> Let me emphasise the last point: you MUST check out today kdelibs, kdepimlibs, 
> kdeutil, kdesdk, etc. You cannot checkout from SVN anything under that and 
> expect it to work.
> 
> So if we do split, then we need one of two things:
> a) split the tarballs too; or
> b) use some kind of script that restores the build structure and maintains it
>   (like repo or git-submodule)
> 
> Splitting the tarballs would be suicide. Instead of nice 19 tarballs for the 
> main KDE modules, we'd end up with 322 (crude find run). Even if my count is 
> exaggerated, we'd have over 200. Not to mention that the dependency checking 
> that CMake is doing now for us would have to be done by the packagers, in 
> order to build KDE in the proper order.
> 
what you are presenting as a reason not to do the split is in fact
probably the most requested feature for / most reported bug against the
kde distribution process. i don't think we'll ever get a better
opportunity (well, motivation) to fix it.
apart from that, the dependency problem isn't new, it will just become
bigger (to the point of not being manageable without tools).
it doesn't have to be left to packagers, anyway. in fact, it musn't, as
otherwise kde devs would be unable to work with it. but that's no biggie
- when things are split up, the modules must have individual cmake
checks for their deps anyway. and this is *good*, as it clarifies the
relationships. and has the nice side effect, that a nightly script could
auto-collect the dependencies and enter them into the meta-module database.

> Besides, the model we chose for Qt development is that developers do
> _not_ merge their changes to the newer branches when they push.
> Someone (like you or I) will do it later.
> 
yes

> For KDE, I'd recommend you always merge up (i.e., p4i). That would
> solve the conflict issues.
> 
yes, it would solve that issue, indeed.
at the cost of being more work (well, ok, who cares) and producing a
rather terrible merge history.
if the modules were split up, the merging policy could be determined by
the respective module maintainers for optimal results.

> > In addition with Qt the use of git is slow on non-Linux platforms.
> 
> That can't be avoided. And having more files to deal with (remember,
> more repositories means more .git/config, .git/objects/pack/pack-*,
> etc.) will not help.
> 
that's bogus. sure it is more files in total, but the active working set
is *much* smaller. possibly to the point where "git status -u" wouldn't
be prohibitively expensive.

> You're proposing redoing the entire layout of the KDE structure. That
> will take a lot longer than we're trying to do now. It would have to
> be done in SVN *before* any attempt at Git conversion be performed.
> 
correct. you (*) wanted the job, now do it right.

(*) that's a plural 'you'.

> Besides, if we relayout, we can forget any hope of having a full
> history import. If we just split, without changing the layout, we'd
> have full history for trunk only (no separate branches imported or
> tags), but if we change the layout, then we will have history only up
> until the layout change.
> 
> Adapting the import rules to follow each library/app instead of the
> module is far too much effort.
> 
sorry, but that argumentation is based on very well fixable shortcomings
of the import script. in fact, i consider it pretty much insane to
create the rules by hand even for the smaller set. i did that for the
cvs2svn migration and it was a truly daunting experience, which i went
through only because i utterly failed at automating it. now we have an
almost perfect rename history (and the cases where it is not can be
relatively easily detected and corrected with manual rules), so it is
plain silly not to use it.
there is also another thing to consider ... most libs/apps never moved
*within* a module. which means that splitting the modules will add only
few additional cases needing rename tracking.

> The chances that you don't know how to resolve a conflict that you run
> into are much larger in Qt.
> 
well, that's the core mistake which makes all those nice numbers
entirely meaningless. for one, the average qt developer knows much more
of qt than the average kde developer knows of kde (and i think that your
numbers suggest that to be true). from that follows also, that the
chance to find somebody who can help you are much better in qt. not to
mention that most qt developers are usually much more available due to
the structure of the organization.
the bottom line is that a finer granularity is *much* more important for
kde than for qt, and it is already painful in qt. the size advantage by
a factor of three will be completely dwarfed by the relative impact of
the problems.