Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

[Mario Fux]

Am Samstag, 6. Mai 2017, 21:37:51 CEST schrieb Ben Cooksley:
> Hi everyone,

Morning Ben and Co

> This is going to be quite a long email, my apologies in advance for that.
> If you use the CI system regularly as part of your development flow it
> is important you read this email in it's entirety as your action will
> probably be required. If you are aware of a list I have missed in the
> above, please feel free to forward it there.
> 
> As some will have been aware of for some time we currently have a
> problem where changing the system base is an incredibly disruptive
> activity. We also have issues where builds sometimes fail due to files
> disappearing mid-build or installations being inconsistent, and
> excessive numbers of emails are generated. To top this off, everyone
> has to use the same underlying "base" system for performing builds
> which sets up conflicts between projects. Oh, and we also only perform
> CI for Linux.
> 
> To solve these problems we've been working on a Next Generation CI
> system which introduces a new concept called 'Products'. In short, a
> Product is a release unit, such as Frameworks, Plasma, KDE
> Applications, which groups a series of repositories which are
> developed and subsequently released together.
> 
> A preview of this system can be viewed at https://build-sandbox.kde.org/
> 
> Products as part of their definition are able to define a set of
> 'Platforms' on which they build. At the moment we have three Platforms
> available: Ubuntu Xenial Qt 5.7, Windows Qt 5.7 and FreeBSD Qt 5.7.
> Adding additional Platforms to this mix is fairly easy, as long as the
> code can be built there. Qt will now be considered as part of the base
> system, and is something we will no longer build ourselves.
> 
> Each Product / Platform definition is fully independent. This will
> allow for different products to build against different versions of Qt
> among other things for instance.
> 
> This is the first part that requires your action: we'd like
> developers, particularly those whose development relies on bleeding
> edge system software to assist in creating and maintaining (Linux)
> Platform images. If you're interested in this, please let me know.
> 
> As part of the shift to the Products paradigm, a number of changes
> have been made to how projects are built and dependencies managed.
> This particularly affects dependencies on projects which are outside
> your Product (Frameworks is outside of Plasma and Applications for
> instance). These dependencies will now be satisfied through
> "Dependency Build" jobs, which will be executed once a week. As a
> result the master version of Frameworks will no longer be available
> outside Frameworks itself.
> 
> This change is one of the big ones which massively reduces the effort
> required to change base systems and thus hugely improves the
> maintainability of the CI system. It is therefore quite important and
> necessary, and also isolates the rest of the CI system from breakages
> lower down in the stack which have happened in the past.
> 
> This is the second point that requires your attention. If your
> development process is dependent on using the latest development
> version of something which is located in another product, then we will
> need to add that to your Product. If this affects you, please start a
> new thread (CC'ing sysadmin and kde-core-devel along with your
> Product's main list) stating which specific repositories you need and
> providing one to two lines of explaination for each.
> 
> Note that requests for the entirety of Frameworks won't be accepted -
> each must be requested on an individual basis with an explanation
> given for why your development process is dependent on the master
> version of that Framework.
> 
> With the change to Products as a core concept for driving the CI
> system, this does leave Extragear somewhat in a bit of an unusual
> situation. At the moment we're planning to deal with this by having
> Extragear be a 'Product' with all of them combined together. If anyone
> in Extragear has any comments to make on this they're certainly
> welcome here.
> 
> Which brings me to the third point of attention. We'll only be adding
> projects to the Next Gen CI system at their request going forward. For
> Frameworks, Applications and Plasma this is something which we're
> essentially assuming we're going to receive from their release
> managers, so we'll take care of defining the initial Products for
> those. For Extragear projects, please respond to this thread if you'd
> like CI coverage (to continue) to be provided to you.

So Simon is such an extragear project that likes to keep CI.

> Thanks for all who have reached this point - my apologies again for
> it's length.
> 
> Note that the Next Gen system isn't finished yet - Notifications are
> yet to be setup and there are a few other touches left to be made.
> 
> Regards,
> Ben Cooksley
> KDE Sysadmin

Thanks for your work on this very useful service
Mario