measuring software's embodied carbon; was: online meetup Wed. 20 Nov. 18h UTC | Green Coding Solutions: "Eco-CI" and "Powerletrics"

Thu Nov 21 15:50:50 GMT 2024

On Mittwoch, 20. November 2024 15:06:18 Mitteleuropäische Normalzeit Joseph P. 
De Veaugh-Geiss wrote:
> Thanks for weighing in, Albert! Anyone else?

Sorry for the late reply, I first wanted to get a better understanding of what 
this actually does, yesterday's presentation certainly helped with that.

There's basically two use-cases you mentioned here:

(1) Embodied carbon of a software release in KDE

My assumption is that measuring a single CI pipeline is just a tiny fraction 
of this. You'd need to sum all CI pipelines run during work on a release, the 
cost of local development and the cost of the dependency chain (or rather a 
certain share of that, for shared components). With the latter practically 
impossible to determine while at the same time presumably the dominating part 
I'm not convinced you'd get to a meaningful result here. And ignoring the 
dependency chain makes it trivial to game the numbers.

(2) Optimizing CI operations to reduce energy use

That's IMHO the more interesting goal. However there's probably bigger things 
to look into first, such as identifying unnecessary pipeline runs (a work 
branch push/MR creation currently triggers two pipelines which are identical 
in most cases for example).

We'd probably also want a KDE-wide monitoring for CI state and CI cost then, 
to identify the projects/pipelines with the biggest impact. And then we get to 
look at optimizing individual pipelines. Any metric would likely do as a first 
approximation there (CPU load, runtime), before looking at energy specifically 
becomes relevant at that level.

Such optimizations would not only benefit energy use but also general CI 
performance/turn around times (and thus contributor efficiency) and potentially 
infrastructure cost.

For energy optimizations specifically there might be other/additional 
approaches as well, as we run our own infrastructure. We could for example 
look at the overall utilization over time and consider moving larger workloads 
caused by automated operations (translation runs, release automation, etc) to 
times or locations with a better energy mix. Assuming there's anything to gain 
there at all, AFAIK we already use data centers that claim to use 100% 
renewable energy anyway.

Regards,
Volker

> On 19.11.24 10:16 PM, Albert Astals Cid wrote:
> > El divendres, 15 de novembre del 2024, a les 13:10:22 (Hora estàndard del
> > 
> > Centre d’Europa), Joseph P. De Veaugh-Geiss va escriure:
> >> Anyone have opinions on this? Good idea? Bad idea?
> > 
> > I think it's a reasonable idea, but who is going to do the work?
> 
> I understand that with sysadmin support Arne from Green Coding Solutions
> would help set up the tool according to KDE's requirements.
> 
> See here for KDE's requirements:
> 
>    https://invent.kde.org/graphics/okular/-/merge_requests/1030#note_995641
> 
> And Arne's reply:
> 
>    https://invent.kde.org/graphics/okular/-/merge_requests/1030#note_1031692
> 
> Perhaps we can discuss it at the meetup tonight.
> 
> Cheers,
> Joseph
> 
> > Cheers,
> > 
> >    Albert
> >> 
> >> On 13.11.24 12:33 PM, Joseph P. De Veaugh-Geiss wrote:
> >>> Hi! I'd like to ask the KDE community to give feedback on a possible eco
> >>> initiative within KDE. The Open Source company Green Coding Solutions
> >>> has a new tool called Eco-CI to estimate the carbon footprint of the CI/
> >>> CD pipeline. Is there support for adopting this tool within the KDE
> >>> community (e.g., for Okular, as proposed)?
> >>> 
> >>> FYI if you are interested in talking with the developers directly, join
> >>> the KDE Eco online meetup next week (see below) where Arne and Didi will
> >>> be presenting.
> >>> 
> >>> As I see it, benefits of integrating Eco-CI into the development
> >>> 
> >>> pipeline include:
> >>>    - Maximal transparency about the embodied carbon of a software
> >>> 
> >>> releases in KDE.
> >>> 
> >>>    - Opportunities for data-driven improvement.
> >>>    - Quantifiable differences for any changes made to the development
> >>> 
> >>> process.
> >>> 
> >>>    - KDE will be a leader in providing data about the carbon footprint
> >>>    of
> >>> 
> >>> our software development.
> >>> 
> >>> Drawbacks could include:
> >>>    - KDE may be the only organization providing such data and it could
> >>> 
> >>> give negative impressions abut our environmental impact (to note: I
> >>> don't think this is necessarily a good argument, but I can see it being
> >>> one).
> >>> 
> >>>    - Discouraging use of the CI/CD pipeline could negatively impact
> >>> 
> >>> software quality.
> >>> 
> >>> Other benefits? Drawbacks? Do you support the idea?
> >>> 
> >>> More info can be found about Green Coding Solutions and the Eco-CI tool
> >>> at this MR in the Okular repository: https://invent.kde.org/graphics/
> >>> okular/-/merge_requests/1030
> >>> 
> >>> """
> >>> [Eco-CI] integrates into the GitLab CI/CD pipeline and estimates the
> >>> energy and CO2 consumption of the pipeline by utilizing a Machine
> >>> Learning model trained on real server energy data from SPECpower
> >>> 
> >>> The tool creates awareness of the energy cost and carbon emissions of
> >>> CI/CD pipelines and empowers developers to create action for more
> >>> sustainability.
> >>> """
> >>> 
> >>> If you want to discuss with the Eco-CI developers directly, join us next
> >>> Wednesday 20 November 19h CET (see below for details).
> >>> 
> >>> Cheers,
> >>> Joseph
> >>> 
> >>> -------- Forwarded Message --------
> >>> Subject: online meetup Wed. 20 Nov. 18h UTC | Green Coding Solutions:
> >>> "Eco-CI" and "Powerletrics"
> >>> Date: Tue, 5 Nov 2024 12:40:14 +0100
> >>> From: Joseph P. De Veaugh-Geiss <joseph at kde.org>
> >>> To: kde-eco-discuss at kde.org
> >>> 
> >>> After a short three-month hiatus, the next KDE Eco online meetup will be
> >>> Wednesday *20 November* 18-19h UTC! I am excited to announce that Green
> >>> 
> >>> Coding Solutions will present two of their new measurement tools:
> >>>    - "Eco-CI" to measure energy consumption of CI/CD pipeline
> >>>    - "Powerletrics" Linux kernel extension for power usage estimations
> >>> 
> >>> Eco-CI has been suggested for integration into the Okular's CI/CD
> >>> pipeline. For discussion, see: https://invent.kde.org/graphics/okular/-/
> >>> merge_requests/1030
> >>> 
> >>> See below for more details.
> >>> 
> >>> Minutes from past meetups can be found here: https://invent.kde.org/
> >>> teams/eco/opt-green/-/tree/master/community-meetups
> >>> 
> >>> _Overview_
> >>> 
> >>> *When*: Wed. 20 November 18-19h UTC
> >>> 
> >>> *Where*: https://meet.kde.org/b/jos-l59-2i1-9yt
> >>> 
> >>> *Topic*: Green Coding Solutions tools "Eco-CI" (CI/CD pipeline) and
> >>> "Powerletrics" (Linux kernel extension)
> >>> 
> >>> *Pad*: Further ideas are collected at this pad, please add ideas of your
> >>> 
> >>> own:
> >>>          https://collaborate.kde.org/s/cactBt4frrfTjbW
> >>> 
> >>> *Details*:
> >>> 
> >>> Eco-CI [0] is an open source GitHub / GitLab Plugin that estimates the
> >>> energy and carbon emissions of a CI/CD workload. It hooks into the
> >>> pipeline and will print a summary directly in the logs or as a
> >>> downloadable artifact. Arne, one of the core developers, will present
> >>> the tool and show some numbers from repos, including their average
> >>> emissions, to get a feel for the importance of the topic. A discussion
> >>> of integrating Eco-CI into Okular's GitLab repository will follow the
> >>> presentation.
> >>> 
> >>> Powerletrics [1] is a kernel extension that brings per process power
> >>> usage estimations to Linux. It is modelled after the MacOS tool
> >>> powermetrics which developers can use to gain insights into their
> >>> environmental impact of their code. With modern ICT infrastructure using
> >>> more and more resources, it is important that there are easy tools for
> >>> monitoring and optimisation available so that we don’t waste precious
> >>> resources.
> >>> 
> >>> [0] https://github.com/green-coding-solutions/eco-ci-energy-estimation
> >>> [1] https://github.com/green-kernel/powerletrics

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-community/attachments/20241121/fe3a1638/attachment.sig>