Sysadmin Load Reduction: Subversion Infrastructure

[Luigi Toscano]

Alexander Potashev ha scritto:
> вс, 10 нояб. 2019 г. в 18:09, Luigi Toscano <luigi.toscano at tiscali.it>:
>>
>> Nate Graham ha scritto:
>>> On 11/9/19 4:50 PM, Nicolás Alvarez wrote:
>>>> El sáb., 9 de nov. de 2019 a la(s) 20:29, Nate Graham (nate at kde.org) escribió:
>>>>>
>>>>> Not knowing anything about the translation system we use... what are the
>>>>> blockers to moving it over to git?
>>>>>
>>>> Not necessarily in order:
>>>> - Experiment conversion to git and see if the resulting repo(s) are of
>>>> a reasonable size (I vaguely recall seeing bad results with this)
>>>> - Convince and teach every translator to use git (expect lots of friction here)
>>>> - Make scripty use git instead of svn (likely lots of work)
>>>> - Some language teams have their own tooling (local for translation,
>>>> or web for statistics) that would need to be gitified too. For
>>>> example, the Spanish team developed KSvnUpdater.
>>>> - Update lots of documentation in different languages (eg.
>>>> https://es.l10n.kde.org/repositorio.php)
>>>>
>>>> You may even need to migrate languages one by one (as you convince
>>>> each language team?), so in the meantime all tooling would need to
>>>> support both repos at the same time.
>>>>
>>>> It doesn't sound like fun...
>>>
>>> Thanks for the background.
>>>
>>> If we're going to keep SVN, I think we need to fully support it. If we don't
>>> have the resources to do this on an ongoing basis, then we need to invest the
>>> resources now/soon to move away from it. I don't see any other realistic options.
>>
>> New resources is the solution. Please see the other emails: the problem
>> discussed is not subversion itself, but the web interface.
>>
>>>
>>> Again, from a totally non-translation perspective, I am somewhat surprised to
>>> hear that individual translators are required to be proficient in a source
>>> control system. Perhaps de-coupling the workflow from direct interaction with
>>> the SCM would be beneficial? Isn't this what GUI apps like KLocalize do? If
>>> not, can it be modified to do the SCM interaction? This seems like a solvable
>>> problem.
>>
>> Most of translators are not so technical as the developers. And even
>> developers can shoot them in the foot with git, I see many issues coming from
>> unwanted merges.
>> Also, in addition to some of the problem described above (not all of them are
>> blocker IMHO), more relevant: how would you convert the SVN repositories into git?
>> - a unique repository would be the easier way, and required by people who do
>> changes to all the languages (renamed and moves) but I can only foresee tons
>> of merging issues when committing (see above).
>> - a repository per language would mean a tons of work whenever you need to do
>> a global operation, and right now it's a no go.
> 
> Hi Luigi,
> 
> We can block unwanted merged on the server with a Git hook like this:
> https://github.com/FabreFrederic/git-hook-pre-receive-reject-merge-commit

When I talk about local merges, I foresee a bit of mess when resolving a merge
locally, which may result in proper commits whose content has an invalid syntax.

> 
>>>> - Experiment conversion to git and see if the resulting repo(s) are of
>>>> a reasonable size (I vaguely recall seeing bad results with this)
> 
> I expect that a single Git repository would be several GBs in size.

For the record, the current checkout (without svn metadata) of trunk/l10n-kde4
(ok, no more needed), branches/stable/l10n-kde4, branches/stable/l10n-kf5,
branches/stable/l10n-kf5-plasma-lts, trunk/l10n-kf5 and trunk/l10n-summit is
~11 GiB. The trunk/l10n-kf5 branch alone is 5.3 GiB.


> Cloning the whole repo would be too much for some translators, however
> the new Git feature "git clone --filter" might be a solution:
> https://unix.stackexchange.com/a/468182

Documentation does not seem to help (or at least I don't seem to find it in
the man pages for git 2.24). Do you know how it could filter just some
directories? It seems more focused on filtering by object type.

> Shall I create a new Phabricator task "[WIP] Migrate translations from
> SVN to Git" so we can put relevant ideas in one place?

I don't know. For me having a board and tasks (it's not just a single task,
it's an entire project) sounds like there is something that can be
implemented, and I don't think it's the case.
I think we should first document all the answers to this recurring questions
first.
-- 
Luigi