CI Requirements - Lessons Not Learnt?

[Martin Gräßlin]

Am 2017-01-12 08:32, schrieb Ben Cooksley:
> On Thu, Jan 12, 2017 at 4:54 AM, Martin Gräßlin <mgraesslin at kde.org> 
> wrote:
>> Am 2017-01-11 10:46, schrieb Ben Cooksley:
>>> 
>>> On Wed, Jan 11, 2017 at 6:57 PM, Martin Gräßlin <mgraesslin at kde.org>
>>> wrote:
>>>> 
>>>> 
>>>> 
>>>> Am 10. Januar 2017 22:42:35 MEZ schrieb Ben Cooksley 
>>>> <bcooksley at kde.org>:
>>>>> 
>>>>> On Mon, Jan 9, 2017 at 8:22 AM, Martin Gräßlin <mgraesslin at kde.org>
>>>>> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Am 6. Januar 2017 17:46:30 MEZ schrieb Alexander Neundorf
>>>>> 
>>>>> <neundorf at kde.org>:
>>>>>>> 
>>>>>>> On 2017 M01 6, Fri 22:39:22 CET Ben Cooksley wrote:
>>>>>>>> 
>>>>>>>> See my notes above re. why tying this to the dependency freeze 
>>>>>>>> date
>>>>>>> 
>>>>>>> is
>>>>>>>> 
>>>>>>>> a bad idea and won't really work.
>>>>>>>> 
>>>>>>>> Given that we potentially have to take into account Qt version
>>>>> 
>>>>> bumps
>>>>>>>> 
>>>>>>>> and base system rebuilds - i'll give a timeline of 1 month's
>>>>> 
>>>>> advance
>>>>>>>> 
>>>>>>>> notice (this is an absolute worst case scenario time). In most
>>>>>>>> instances we can turn things around in much shorter time (about 
>>>>>>>> a
>>>>>>>> week), especially where it's just a case of installing an 
>>>>>>>> already
>>>>>>>> available package / adding a PPA/Equivalent Repository.
>>>>>>>> 
>>>>>>>> The significant variation in time above is why I don't want to
>>>>>>>> actually specify a time frame which is set in stone. Some things
>>>>> 
>>>>> are
>>>>>>>> 
>>>>>>>> just easier to provide / upgrade than others.
>>>>>>> 
>>>>>>> 
>>>>>>> how about some simple rule like "new dependencies every first of 
>>>>>>> the
>>>>>>> month",
>>>>>>> and 1 week notice. For the developer this would mean in the worst
>>>>> 
>>>>> case
>>>>>>> 
>>>>>>> 5 weeks
>>>>>>> waiting, which is probably quite a lot.
>>>>>>> E.g. if you need a new dependency, and announce it let's say 
>>>>>>> January
>>>>>>> 6th,
>>>>>>> you'll get it February 1st.
>>>>>>> If you announce it January 29th, you get it March 1st.
>>>>>> 
>>>>>> 
>>>>>> In general I like the more formal approach to dependency handling.
>>>>>> 
>>>>>> This concrete proposal is in my opinion to restrictive to 
>>>>>> developers
>>>>> 
>>>>> - in case of plasma it could mean only one date per release 
>>>>> schedule.
>>>>>> 
>>>>>> That's difficult to plan and makes it easy for devs to miss. If 
>>>>>> for
>>>>> 
>>>>> whatever reason you cannot push that one day you have to wait a
>>>>> complete release cycle.
>>>>>> 
>>>>>> 
>>>>>> And I don't see how this addresses the problem of CI needs time. 
>>>>>> As
>>>>> 
>>>>> Ben told us one week might not be enough.
>>>>>> 
>>>>>> 
>>>>>> It can address the problem for distro consumers, but for 
>>>>>> developers
>>>>> 
>>>>> on older distros it wouldn't help either.
>>>>> 
>>>>> Can we have a proposal from your side then Martin?
>>>>> 
>>>>> As i've said previously, what i'm asking for here is fundamentally
>>>>> incompatible with integration into a release schedule, as 
>>>>> dependencies
>>>>> can be bumped at essentially any time from the point of a version
>>>>> being released (the unfreeze) to the point dependencies are frozen 
>>>>> in
>>>>> the lead up to a release. The advance notice i've asked for needs 
>>>>> to
>>>>> be given before said bump actually happens.
>>>>> 
>>>>> Note that the amount of notice needed is very dependent on the 
>>>>> nature
>>>>> of the bump - some bumps could be sorted in 24 hours... while 
>>>>> others
>>>>> require extensive planning and need weeks.
>>>> 
>>>> 
>>>> Honestly, that is an inflexibility I cannot work with. I don't see 
>>>> how I
>>>> as a dev can
>>>> plan to upgrade a dependency if all we get is something between a 
>>>> day and
>>>> months.
>>>> Please remember that our release schedule is only about 3 months. 
>>>> With
>>>> the
>>>> requirements presented here it means I have to announce in the last
>>>> release cycle that I want to upgrade a dependency.
>>> 
>>> 
>>> I'm sorry but that is how it is - we're providing CI to the entire of
>>> KDE, covering Frameworks, Applications and Extragear - in addition to
>>> Plasma.
>>> Jenkins currently has ~1300 jobs loaded into it - so KWin is a
>>> miniscule blip in comparison, considering it makes up only 2 of those
>>> jobs.
>>> 
>>> This makes any change in the base platform a complex one - requiring 
>>> a
>>> full rebuild of everything (takes more than a day last time we did it
>>> - and that's not taking into account the current Jenga tower state of
>>> the dependency tree for some parts of KDE). This is where the "weeks"
>>> part I mentioned above comes in - as depending on what you are after
>>> we may have to replace the system base. That isn't something we can
>>> just do at a drop of a hat because you've decided to bump 
>>> dependencies
>>> on something, and it isn't something I can predict with any certainty
>>> either.
>>> 
>>> How difficult a dependency bump or introduction is to accommodate 
>>> very
>>> much depends on the availability of supplementary repositories 
>>> (PPAs),
>>> the difficulty of building it ourselves (bear in mind this has a
>>> corresponding maintenance cost as well) and the ABI impact on the 
>>> rest
>>> of the system (even Qt patch releases demand a full rebuild of things
>>> due to the usage of private API in various parts of KDE).
>> 
>> 
>> From what I get here we have different classes of dependencies. And 
>> they
>> will take a different amount of time.
> 
> That's correct.
> 
>> 
>> Can we classify this in a way that we as developers can derive how 
>> long it
>> takes.
>> My saying about inflexibility is only about the "can take as long as 
>> it
>> takes".
>> That is something I cannot plan with. I need to know how long it will 
>> take.
>> 
>> So if we have categories like:
>> * if present in Ubuntu 16.10, but not yet installed: 24 h
>> * if not present in Ubuntu 16.10, but available through PPA: 48 h
>> * if not present through a PPA: evaluation by sysadmin is needed
>> * Qt version increase: tell us 3 months ahead
>> 
>> That would be something I can work with. This would give me a clear 
>> planning
>> and I could categorize the required dependency myself.
> 
> Something a little more than 24 hours would be nice in case we have
> other Sysadmin requests or work that needs attending to (several
> instances needed Wordpress upgrades this afternoon for instance)

all right! Can we put that down in a wiki page together with Eike's 
suggestion
so that we have it formalized?

> 
> It would probably be a good idea to announce it for other developers
> to know about as well so they can sort their systems out.

that's what we have code review for :-)


Cheers
Martin