measuring software's embodied carbon; was: online meetup Wed. 20 Nov. 18h UTC | Green Coding Solutions: "Eco-CI" and "Powerletrics"

[Ben Cooksley]

On Tue, Dec 3, 2024 at 5:43 AM Volker Krause <vkrause at kde.org> wrote:

> On Samstag, 30. November 2024 11:41:12 Mitteleuropäische Normalzeit Albert
> Astals Cid wrote:
> > El dissabte, 30 de novembre del 2024, a les 4:29:22 (Hora estàndard del
> > Centre
> > d’Europa), Ben Cooksley va escriure:
> > > On Sat, Nov 30, 2024 at 10:48 AM Ingo Klöcker <kloecker at kde.org>
> wrote:
> > > > On Freitag, 29. November 2024 15:06:51 Mitteleuropäische Normalzeit
> Ingo
> > > >
> > > > Klöcker wrote:
> > > > > On Freitag, 29. November 2024 10:31:10 Mitteleuropäische Normalzeit
> > > > > Ben
> > > > >
> > > > > Cooksley wrote:
> > > > > > On Fri, Nov 22, 2024 at 4:51 AM Volker Krause <vkrause at kde.org>
> wrote:
> > > > > > > (2) Optimizing CI operations to reduce energy use
> > > > > > >
> > > > > > > That's IMHO the more interesting goal. However there's probably
> > > >
> > > > bigger
> > > >
> > > > > > > things
> > > > > > > to look into first, such as identifying unnecessary pipeline
> runs
> > > > > > > (a
> > > > > > > work
> > > > > > > branch push/MR creation currently triggers two pipelines which
> are
> > > > > > > identical
> > > > > > > in most cases for example).
> > > > > >
> > > > > > We already have optimisation in place to prevent duplicate jobs
> > > >
> > > > triggering
> > > >
> > > > > > when a merge request already exists, but yes there is potentially
> > > > > > some
> > > > > > work
> > > > > > to be done here to improve efficiency.
> > > > > > If a developer knows they are going to be immediately creating a
> MR
> > > >
> > > > then
> > > >
> > > > > > they can use the Git option ci.skip to prevent CI from running
> (git
> > > >
> > > > push
> > > >
> > > > > > -o ci.skip)
> > > > >
> > > > > There's an even better approach. If one pushes a work branch with
> > > > > `git push -o merge_request.create` then GitLab immediately creates
> an
> > > > > MR,
> > > > > runs an MR pipeline and does NOT run a branch pipeline. (I have
> just
> > > >
> > > > tested
> > > >
> > > > > this.)
> > > >
> > > > Sadly, it seems this does not always work. It worked for
> > > > https://invent.kde.org/pim/libkleo/-/merge_requests/170
> > > > but it didn't work for
> > > > https://invent.kde.org/pim/mimetreeparser/-/merge_requests/62
> > > > where a branch pipeline was started 2 seconds before the MR pipeline.
> > > > Might be
> > > > a timing problem. :-/
> > >
> > > Likely a race condition between the various involved components of
> Gitlab.
> > >
> > > The two options I can see for resolving this are:
> > > a. Making use of the "ci.skip" Git push option when pushing a new work
> > > branch to be used in a merge request
> > > b. We set CI jobs as manual by default on work/ branches (optionally
> > > making
> > > use of a CI environment variable to allow work branches to run by
> default)
> > >
> > > I wouldn't be in favour of making the CI jobs disabled by default on
> work
> > > branches as that is non-intuitive / non-obvious behaviour, especially
> if
> > > you have to pass an environment variable to make them show up.
> >
> > How is it non-obvious? You created a "work" branch so no automatic CI for
> > you, if you want one, create a Merge Request (so you show your work to
> the
> > rest of us) or run the CI manually.
>
> There are valid usecases for CI runs on non-MR work branches. Current
> example
> is the ki18n multi-language lookup work which exposed behavior differences
> on
> FreeBSD and Windows that I could not debug locally. Instead this was done
> in a
> work branch of a fork, with debug output committed and run on the CI. Only
> the
> final result made it back into the MR.
>
> I didn't do this in the MR to not spam you with two dozen extra change
> notifications and to avoid debug commits anywhere near an MR that might
> get
> merged.
>
> And that's not a rare exception, for me at least, as I have little options
> to
> debug on platforms I don't have local access to otherwise.
>
> I wouldn't mind the CI being opt-in in that case, but being able to
> trigger
> this via push options would be nice then, rather than having to go through
> the
> web UI and creating pipelines manually each time.
>

Based on this i'd say it is sounding like we should go for either:
1) Controlling automatic start of CI jobs using an environment variable for
work/ branches; or
2) Using Git aliases to ensure -o ci.skip is passed consistently to Gitlab
(preferred)

Thoughts?


> Regards,
> Volker


Thanks,
Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-community/attachments/20241203/06082dce/attachment.htm>