measuring software's embodied carbon; was: online meetup Wed. 20 Nov. 18h UTC | Green Coding Solutions: "Eco-CI" and "Powerletrics"

Thu Nov 28 18:48:44 GMT 2024

Thank you, Volker! As always, excellent points.

I have replied at the Okular repository MR. For those who are 
interested, here's my response:

   https://invent.kde.org/graphics/okular/-/merge_requests/1030#note_1082785

Cheers,
Joseph

On 21.11.24 4:50 PM, Volker Krause wrote:
> On Mittwoch, 20. November 2024 15:06:18 Mitteleuropäische Normalzeit Joseph P.
> De Veaugh-Geiss wrote:
>> Thanks for weighing in, Albert! Anyone else?
> 
> Sorry for the late reply, I first wanted to get a better understanding of what
> this actually does, yesterday's presentation certainly helped with that.
> 
> There's basically two use-cases you mentioned here:
> 
> (1) Embodied carbon of a software release in KDE
> 
> My assumption is that measuring a single CI pipeline is just a tiny fraction
> of this. You'd need to sum all CI pipelines run during work on a release, the
> cost of local development and the cost of the dependency chain (or rather a
> certain share of that, for shared components). With the latter practically
> impossible to determine while at the same time presumably the dominating part
> I'm not convinced you'd get to a meaningful result here. And ignoring the
> dependency chain makes it trivial to game the numbers.
> 
> (2) Optimizing CI operations to reduce energy use
> 
> That's IMHO the more interesting goal. However there's probably bigger things
> to look into first, such as identifying unnecessary pipeline runs (a work
> branch push/MR creation currently triggers two pipelines which are identical
> in most cases for example).
> 
> We'd probably also want a KDE-wide monitoring for CI state and CI cost then,
> to identify the projects/pipelines with the biggest impact. And then we get to
> look at optimizing individual pipelines. Any metric would likely do as a first
> approximation there (CPU load, runtime), before looking at energy specifically
> becomes relevant at that level.
> 
> Such optimizations would not only benefit energy use but also general CI
> performance/turn around times (and thus contributor efficiency) and potentially
> infrastructure cost.
> 
> For energy optimizations specifically there might be other/additional
> approaches as well, as we run our own infrastructure. We could for example
> look at the overall utilization over time and consider moving larger workloads
> caused by automated operations (translation runs, release automation, etc) to
> times or locations with a better energy mix. Assuming there's anything to gain
> there at all, AFAIK we already use data centers that claim to use 100%
> renewable energy anyway.
> 
> Regards,
> Volker
>   
>> On 19.11.24 10:16 PM, Albert Astals Cid wrote:
>>> El divendres, 15 de novembre del 2024, a les 13:10:22 (Hora estàndard del
>>>
>>> Centre d’Europa), Joseph P. De Veaugh-Geiss va escriure:
>>>> Anyone have opinions on this? Good idea? Bad idea?
>>>
>>> I think it's a reasonable idea, but who is going to do the work?
>>
>> I understand that with sysadmin support Arne from Green Coding Solutions
>> would help set up the tool according to KDE's requirements.
>>
>> See here for KDE's requirements:
>>
>>     https://invent.kde.org/graphics/okular/-/merge_requests/1030#note_995641
>>
>> And Arne's reply:
>>
>>     https://invent.kde.org/graphics/okular/-/merge_requests/1030#note_1031692
>>
>> Perhaps we can discuss it at the meetup tonight.
>>
>> Cheers,
>> Joseph
>>
>>> Cheers,
>>>
>>>     Albert
>>>>
>>>> On 13.11.24 12:33 PM, Joseph P. De Veaugh-Geiss wrote:
>>>>> Hi! I'd like to ask the KDE community to give feedback on a possible eco
>>>>> initiative within KDE. The Open Source company Green Coding Solutions
>>>>> has a new tool called Eco-CI to estimate the carbon footprint of the CI/
>>>>> CD pipeline. Is there support for adopting this tool within the KDE
>>>>> community (e.g., for Okular, as proposed)?
>>>>>
>>>>> FYI if you are interested in talking with the developers directly, join
>>>>> the KDE Eco online meetup next week (see below) where Arne and Didi will
>>>>> be presenting.
>>>>>
>>>>> As I see it, benefits of integrating Eco-CI into the development
>>>>>
>>>>> pipeline include:
>>>>>     - Maximal transparency about the embodied carbon of a software
>>>>>
>>>>> releases in KDE.
>>>>>
>>>>>     - Opportunities for data-driven improvement.
>>>>>     - Quantifiable differences for any changes made to the development
>>>>>
>>>>> process.
>>>>>
>>>>>     - KDE will be a leader in providing data about the carbon footprint
>>>>>     of
>>>>>
>>>>> our software development.
>>>>>
>>>>> Drawbacks could include:
>>>>>     - KDE may be the only organization providing such data and it could
>>>>>
>>>>> give negative impressions abut our environmental impact (to note: I
>>>>> don't think this is necessarily a good argument, but I can see it being
>>>>> one).
>>>>>
>>>>>     - Discouraging use of the CI/CD pipeline could negatively impact
>>>>>
>>>>> software quality.
>>>>>
>>>>> Other benefits? Drawbacks? Do you support the idea?
>>>>>
>>>>> More info can be found about Green Coding Solutions and the Eco-CI tool
>>>>> at this MR in the Okular repository: https://invent.kde.org/graphics/
>>>>> okular/-/merge_requests/1030
>>>>>
>>>>> """
>>>>> [Eco-CI] integrates into the GitLab CI/CD pipeline and estimates the
>>>>> energy and CO2 consumption of the pipeline by utilizing a Machine
>>>>> Learning model trained on real server energy data from SPECpower
>>>>>
>>>>> The tool creates awareness of the energy cost and carbon emissions of
>>>>> CI/CD pipelines and empowers developers to create action for more
>>>>> sustainability.
>>>>> """
>>>>>
>>>>> If you want to discuss with the Eco-CI developers directly, join us next
>>>>> Wednesday 20 November 19h CET (see below for details).
>>>>>
>>>>> Cheers,
>>>>> Joseph
>>>>>
>>>>> -------- Forwarded Message --------
>>>>> Subject: online meetup Wed. 20 Nov. 18h UTC | Green Coding Solutions:
>>>>> "Eco-CI" and "Powerletrics"
>>>>> Date: Tue, 5 Nov 2024 12:40:14 +0100
>>>>> From: Joseph P. De Veaugh-Geiss <joseph at kde.org>
>>>>> To: kde-eco-discuss at kde.org
>>>>>
>>>>> After a short three-month hiatus, the next KDE Eco online meetup will be
>>>>> Wednesday *20 November* 18-19h UTC! I am excited to announce that Green
>>>>>
>>>>> Coding Solutions will present two of their new measurement tools:
>>>>>     - "Eco-CI" to measure energy consumption of CI/CD pipeline
>>>>>     - "Powerletrics" Linux kernel extension for power usage estimations
>>>>>
>>>>> Eco-CI has been suggested for integration into the Okular's CI/CD
>>>>> pipeline. For discussion, see: https://invent.kde.org/graphics/okular/-/
>>>>> merge_requests/1030
>>>>>
>>>>> See below for more details.
>>>>>
>>>>> Minutes from past meetups can be found here: https://invent.kde.org/
>>>>> teams/eco/opt-green/-/tree/master/community-meetups
>>>>>
>>>>> _Overview_
>>>>>
>>>>> *When*: Wed. 20 November 18-19h UTC
>>>>>
>>>>> *Where*: https://meet.kde.org/b/jos-l59-2i1-9yt
>>>>>
>>>>> *Topic*: Green Coding Solutions tools "Eco-CI" (CI/CD pipeline) and
>>>>> "Powerletrics" (Linux kernel extension)
>>>>>
>>>>> *Pad*: Further ideas are collected at this pad, please add ideas of your
>>>>>
>>>>> own:
>>>>>           https://collaborate.kde.org/s/cactBt4frrfTjbW
>>>>>
>>>>> *Details*:
>>>>>
>>>>> Eco-CI [0] is an open source GitHub / GitLab Plugin that estimates the
>>>>> energy and carbon emissions of a CI/CD workload. It hooks into the
>>>>> pipeline and will print a summary directly in the logs or as a
>>>>> downloadable artifact. Arne, one of the core developers, will present
>>>>> the tool and show some numbers from repos, including their average
>>>>> emissions, to get a feel for the importance of the topic. A discussion
>>>>> of integrating Eco-CI into Okular's GitLab repository will follow the
>>>>> presentation.
>>>>>
>>>>> Powerletrics [1] is a kernel extension that brings per process power
>>>>> usage estimations to Linux. It is modelled after the MacOS tool
>>>>> powermetrics which developers can use to gain insights into their
>>>>> environmental impact of their code. With modern ICT infrastructure using
>>>>> more and more resources, it is important that there are easy tools for
>>>>> monitoring and optimisation available so that we don’t waste precious
>>>>> resources.
>>>>>
>>>>> [0] https://github.com/green-coding-solutions/eco-ci-energy-estimation
>>>>> [1] https://github.com/green-kernel/powerletrics
> 

-- 
Joseph P. De Veaugh-Geiss (he/him)
KDE Eco Project & Community Manager
OpenPGP: 8FC5 4178 DC44 AD55 08E7 DF57 453E 5746 59A6 C06F
Matrix: @joseph:kde.org

Generally available Monday-Thursday from 10-16h CET/CEST. Outside of 
these times it may take a little longer for me to respond.

KDE Eco: Building Energy-Efficient Free Software!
Website: https://eco.kde.org
Mastodon: @be4foss at floss.social