Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

[Elvis Angelaccio]

On sabato 6 maggio 2017 11:37:51 CEST, Ben Cooksley wrote:
> Hi everyone,
>
> This is going to be quite a long email, my apologies in advance for that.
> If you use the CI system regularly as part of your development flow it
> is important you read this email in it's entirety as your action will
> probably be required. If you are aware of a list I have missed in the
> above, please feel free to forward it there.
>
> As some will have been aware of for some time we currently have a
> problem where changing the system base is an incredibly disruptive
> activity. We also have issues where builds sometimes fail due to files
> disappearing mid-build or installations being inconsistent, and
> excessive numbers of emails are generated. To top this off, everyone
> has to use the same underlying "base" system for performing builds
> which sets up conflicts between projects. Oh, and we also only perform
> CI for Linux.
>
> To solve these problems we've been working on a Next Generation CI
> system which introduces a new concept called 'Products'. In short, a
> Product is a release unit, such as Frameworks, Plasma, KDE
> Applications, which groups a series of repositories which are
> developed and subsequently released together.
>
> A preview of this system can be viewed at https://build-sandbox.kde.org/
>
> Products as part of their definition are able to define a set of
> 'Platforms' on which they build. At the moment we have three Platforms
> available: Ubuntu Xenial Qt 5.7, Windows Qt 5.7 and FreeBSD Qt 5.7.
> Adding additional Platforms to this mix is fairly easy, as long as the
> code can be built there. Qt will now be considered as part of the base
> system, and is something we will no longer build ourselves.
>
> Each Product / Platform definition is fully independent. This will
> allow for different products to build against different versions of Qt
> among other things for instance.
>
> This is the first part that requires your action: we'd like
> developers, particularly those whose development relies on bleeding
> edge system software to assist in creating and maintaining (Linux)
> Platform images. If you're interested in this, please let me know.
>
> As part of the shift to the Products paradigm, a number of changes
> have been made to how projects are built and dependencies managed.
> This particularly affects dependencies on projects which are outside
> your Product (Frameworks is outside of Plasma and Applications for
> instance). These dependencies will now be satisfied through
> "Dependency Build" jobs, which will be executed once a week. As a
> result the master version of Frameworks will no longer be available
> outside Frameworks itself.
>
> This change is one of the big ones which massively reduces the effort
> required to change base systems and thus hugely improves the
> maintainability of the CI system. It is therefore quite important and
> necessary, and also isolates the rest of the CI system from breakages
> lower down in the stack which have happened in the past.
>
> This is the second point that requires your attention. If your
> development process is dependent on using the latest development
> version of something which is located in another product, then we will
> need to add that to your Product. If this affects you, please start a
> new thread (CC'ing sysadmin and kde-core-devel along with your
> Product's main list) stating which specific repositories you need and
> providing one to two lines of explaination for each.
>
> Note that requests for the entirety of Frameworks won't be accepted -
> each must be requested on an individual basis with an explanation
> given for why your development process is dependent on the master
> version of that Framework.
>
> With the change to Products as a core concept for driving the CI
> system, this does leave Extragear somewhat in a bit of an unusual
> situation. At the moment we're planning to deal with this by having
> Extragear be a 'Product' with all of them combined together. If anyone
> in Extragear has any comments to make on this they're certainly
> welcome here.
>
> Which brings me to the third point of attention. We'll only be adding
> projects to the Next Gen CI system at their request going forward. For
> Frameworks, Applications and Plasma this is something which we're
> essentially assuming we're going to receive from their release
> managers, so we'll take care of defining the initial Products for
> those. For Extragear projects, please respond to this thread if you'd
> like CI coverage (to continue) to be provided to you.

What's the role of kde-build-metadata.git in the new CI? If an extragear 
project is already defined there, won't it be automatically built by the 
new CI?

>
> Thanks for all who have reached this point - my apologies again for
> it's length.
>
> Note that the Next Gen system isn't finished yet - Notifications are
> yet to be setup and there are a few other touches left to be made.
>
> Regards,
> Ben Cooksley
> KDE Sysadmin

Cheers,
Elvis

>
>