[Kde-scm-interest] Git roadmap

[Thiago Macieira]

Johannes Sixt wrote:
>Thiago Macieira schrieb:
>> 2) the conversion from Subversion to git
>>   Chris's work on the git-svn-fastexport tool is a good start, but our
>> repository is extremely complex. We need to develop the tool further
>> to be able to import the repository correctly.
>
>Sorry, but I have to ask: What can be "complex" in an SVN repository?

All moves.

We've done once a "trunk move" -- see revisions 409200 through 409209. No 
svn-to-git tool that I have tried so far can handle that. We need a tool 
that understands that.

We've also got partial branches: when you make a copy of a subtree of a 
module. That's a valid branch and should be handled too. I have no idea 
if:
a) we want to keep those branches (the official ones are always full-tree)
b) any tool can handle it

There's also the issue of issue of copy-with-history across modules. This 
is the main feature missing in git for the KDE conversion, as far as I 
can see. The conversion tool will have to be updated with this concept 
once it's introduced into git.

Another issue is the use of svn:externals. We now have several uses of 
that, not just the official kde-common/admin. I'm tempted to say we have 
to "fix" those externals first and simply not try to get past revisions 
correct. You can't get those in SVN anyways -- it always updates the 
external to HEAD.

>>   Someone on #git has suggested we import all of SVN into one
>> monstruous git repository and then work from there, inside git. I
>> believe, however, we can extract some more metadata from Subversion
>> (read: file-copy relationships)
>
>git does not represent file-copy relationships, so the don't need to be
>converted.

Unfortunately, they have. It's the copy across modules, like I said above.

>The most important information to be recovered are where merges
> happened. It would be invaluable to represent them in the converted git
> repository.

These, I think, can be done manually. I.e., after converting to git, we 
can use grafts to mark the commits that were merges and then do a single, 
major filter-branch to make the grafts permanent.

Subversion does not record that information, so we can't extract that 
information from it. It can be automated somewhat, though. (Heuristic 
matching the commit log messages, possibly even the svnmerge and svk 
properties).

>This alone gives an idea what can be done and how the final conversion
>should *not* look like: It shouldn't be a single repository because it's
> to clumsy to be handled that way. Probably, it is also wise to keep
> historical parts in a separate repository (maybe everything before the
> KDE 4 porting efforts and all KDE 3 branches).

Yes, I agree.

>> 3) repository layout, ACLs, hooks, etc.
>>   First and foremost, switching to git brings the question of whether
>> we should have multiple repositories in KDE (as opposed to a single
>> one like we have with Subversion). I'm not talking about clones. If we
>> have multiple repositories -- which is the recommended way in git --
>> we have to decide where to split and how to glue everything to gether
>> with submodule support.
>
>I think it's pretty clear:
>- Have each major module (kdelibs, kdebase, kdenetwork, ...) in its own
>repository. Also kdeadmin (which in svn is linke in as external) should
> be in its own repository.

kde-common/admin is no longer used as an external in KDE 4. But, in order 
to properly represent history of KDE 1 through 3, you're right: we need 
a "kde-common-admin" module so that we can use git-module with it.

>- Have an "umbrella" repository that basically only consists of
> submodule references to those major modules.

Agreed again.

>I don't think that it's necessary to have a finer-grained separation.

I do think that applications in extragear, kdereview and playground can 
each have a module of its own. The progress of an application, leading to 
extragear, would be a simple change in submodule relationships.

The progression of kdereview->main modules would be handled with the 
copy-with-history tool.

>>   Then we need to think about the "social" side: who gets push access?
>> Who gets fast-forward-only push access? Will we require "SignedOff
>> by"? Where will we keep our post-commit hooks, like the
>> kde-commits at kde.org mailer? How will we protect the official tags? How
>> will we protect the Root CA files in kdelibs? Etc.
>
>IMHO, there should exist "the one and only official KDE repository".
>
>And only few people have push access to it.
>
>Those people are the "integrators". They pull from the "trusted
> lieutenants" (who are probably in charge of the individual submodules)
> and publish to this official repository.

Unfortunately, I don't agree (read through, please). This goes against the 
KDE developer culture.

We have lots of people going through all modules and fixing bugs. The 
whole KDE 4 porting effort has led to that: if you change something in 
kdelibs that makes other modules break, you are responsible for fixing 
ALL other modules too -- KDE modules, koffice, extragear, etc.

Where would those people push to?

There are also people doing periodic compilations and fixing all build 
problems. I know I do that. Will I have to send patches to people instead 
of just fixing the problem? If I have to create an email, attach a patch, 
send and then wait several days for the fix to be in, it'll certainly 
demotivate me for doing those kinds of fixes and cleanups.

Or will I have more rights than other developers?

I do think, though, we should have a "protected" area -- be it a branch or 
a separate repository. Only the official module maintainers can 
commit/push there. This is how we handle in SVN already: the KDE branches 
and tags are only read-write to the KDE sysadmins. 

But there has to be a "free for all" area.

Maybe we can have that after all:
- a set of repositories that are protected and are read-write only for a 
couple of people: sysadmins, module maintainers. Those would contain the 
official branches (4.0 branch, 4.1 branch, etc.) as well as the latest 
mostly-stable master.

- a second set of repositories that are "free for all". Anyone can commit 
there, but without the ability to remove commits. Only the sysadmins and 
module maintainers can do "push -f".

During normal development, hopefully developers will have each their own 
repository where they publish their changes. A developer group working on 
a given feature or application would coordinate among themselves, pushing 
and pulling from each other.

Every now and then, they would merge their work into the "free for all" 
repository. Other developers would then build those new features and make 
the fixes, cleanups, etc., that they are known to do. They'd simply push 
the changes into the repository again. No need for patch review.

Finally, the module maintainer would periodically merge "blessed" changes 
into the protected repository. (where "blessed" probably means "it 
compiles for me")

>Other points you raised:
>- fast-forward-only push access: don't know what you mean

People who can do:
   git push
VS people who can do:
   git push -f

>- protect Root CA: If "protect" means "cannot be changed behind our
> back": As long as you regard SHA1 as sufficiently secure, this is
> impossible with git.

In the context of a "free for all" repository, there are some files that 
mustn't be changed. The only I can think of are the Root CA files in 
kdelibs. Though the "www" module is also very limited: you get access 
only to the files you need to.

-- 
  Thiago Macieira  -  thiago (AT) macieira.info - thiago (AT) kde.org
    PGP/GPG: 0x6EF45358; fingerprint:
    E067 918B B660 DBD1 105C  966C 33F5 F005 6EF4 5358
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url : http://mail.kde.org/pipermail/kde-scm-interest/attachments/20070918/0100a2a0/attachment.pgp