KDevelop CI for creating Linux AppImage + Windows installers

[Ben Cooksley]

On Sat, Mar 18, 2017 at 3:36 PM, Kevin Funk <kfunk at kde.org> wrote:
> On Monday, 13 March 2017 16:12:19 CET Aleix Pol wrote:
>> On Sat, Mar 4, 2017 at 3:01 AM, Kevin Funk <kfunk at kde.org> wrote:
>> > Heya,
>> >
>> > you might have noticed the 5.0.4 took /quite/ long again to get out. This
>> > time this has been merely b/c of serious breakages in KDE's Craft (the
>> > tool we use to create the Windows installers) we needed to fix before
>> > re-spinning the Windows installers.
>> >
>> > Needless to say, this has been quite a frustrating experience to both Sven
>> > & me, since that's mundane and unrewarding work.
>> >
>> > In order to improve on this situation, and to be able to release more
>> > quickly, I've set up a self-hosted CI which builds the following
>> > configurations for us automatically:
>> >
>> > - KDevelop AppImage 64 bit
>> > - KDevelop Windows Installer 64 bit
>> > - KDevelop Windows Installer 32 bit
>> >
>> > It's all automated, the CI is set to track a certain branch (5.0 right
>> > now)
>> > for all of kdevplatform, kdevelop, kdev-python, kdev-php and rebuild all
>> > that on a nightly basis.
>> >
>> > Aside from that we've hardcoded more versions in Craft.git such as the
>> > KF5,
>> > LLVM/Clang, Qt versions so the builds get more reproducible.
>> >
>> > tl;dr: Here's the CI:
>> >   http://kfunk.ddns.net:8080/
>> >
>> > You can download the latest artifacts by visiting one of the jobs:
>> >   http://kfunk.ddns.net:8080/job/kdevelop-appimage-centos/
>> >
>> >     -> "KDevelop-git-x86_64.AppImage"
>> >
>> > Hope that helps!
>> > Kevin
>> >
>> > --
>> > Kevin Funk | kfunk at kde.org | http://kfunk.org
>>
>> Hi Kevin,
>> I think this is really cool! Thanks!
>>
>> Have you contacted sysadmin? it would be ideal if we could run this
>> under KDE servers and hopefully extend the know how for other projects
>> as well.
>
> I think it would make sense to move this system KDE infrastructure and make it
> pop out AppImages/installers for a known set of applications (not just
> KDevelop, indeed).
>
> The Windows CI creating the KDevelop x86 & amd64 installers is currently
> running in a VirtualBox VM and has a size of ~ 100 GB. KF5 is at a fixed
> version, the CI rebuilds all of KDevelop (+ kdev-python, kdev-php) on a
> nightly basis.
>
> The Docker container creating the KDevelop AppImages is relatively simple,
> too. The Dockerfile sets up Qt + LLVM; the CI rebuilds all of KF5, KDevelop (+
> kdev-python, kdev-php) via a custom .sh script written by Sven.
>
> @sysadmins: Is there a place on our servers where I could move the VMs and
> docker container? Or what do you think about the overall idea?

We already have an infrastructure for hosting Docker containers,
although those are temporary ones, not long lived ones.
If this needs to be long lived, it may be better for us to run it in
an LXC container instead.

As for Windows, we already have VMs for that, they're just not in use.
I can probably arrange access to those. Have you connected to the CI
VPN before?

In regards to extending this to other projects, it'll probably need to
be reworked to function more like our existing CI as you'll need to
work across multiple builders and reuse stuff that's already built I
suspect, otherwise it'll become unscalable.

Please note that Sysadmin time is a bit limited at the moment.

>
> Cheers,
> Kevin
>
>> Aleix
>
>
> --
> Kevin Funk | kfunk at kde.org | http://kfunk.org

Cheers,
Ben