Interest in building an LLM frontend for KDE

[Joseph P. De Veaugh-Geiss]

On 12/4/23 22:45, Alexander Semke wrote:
> On Montag, 4. Dezember 2023 12:09:43 CET Joseph P. De Veaugh-Geiss wrote:
>> I agree with the concerns Josh raises about the energy consumption of
>> training LLMs (see, e.g., [1]). A benefit of satisfying the above
>> characteristics is it is then possible for us to measure the energy
>> consumption for training/using the LLMs. This would enable KDE to be
>> transparent about what these tools consume in terms of energy and
>> present this information to users.
> To make this argument more complete, it's not only the training of such models but also
> their usage ("inference") later. For popular generic models the negative impact can quickly
> become bigger than the impact of the training itself:
> https://www.technologyreview.com/2023/12/01/1084189/making-an-image-with-generative-ai-uses-as-much-energy-as-charging-your-phone/[1]
> 
> https://arxiv.org/abs/2311.16863[2]
>

Thank you for the info re power consumption in usage of these models!

> Though, these and similar arguments mostly tend to ignore the fact that the hyperscalers
> have commited to the net-zero imitative and are heavily investing into renewables for
> their data centers and also trying to shift heavy workloads into more sustainable time
> windows.
> 

Yes, the demands of the ICT sector are arguably a big driver in the 
shift to renewable, low-carbon energy sources!

A problem with this, though, is that we have a power consumption 
problem: we consume more than we produce in terms of low-carbon energy, 
and inceasing the consumption of ICT is taking energy away from 
something else. Moreover, there may be limits in natural resources as to 
how much renewable energy can actually be produced. In the same paper 
cited before (see [1]), the authors use the example of silver mining for 
the production of photovoltaic panels. "An average solar panel requires
ca. 20 g of silver [...]. On [the current] trajectory, solar panels 
would use 100% of global silver supplies in 2031 leaving none for 
electric car batteries and other uses" (p. 7). Given this and other 
issues, the authors conclude: "Thus, while a shift to more renewable 
energy is crucial, it does not provide an unlimited supply of energy for 
ICT to expand into without consequences" (p. 8).

Perhaps this is moving too far off topic, though. We can discuss more at 
the energy efficiency Matrix room if you'd like: 
https://matrix.to/#/#energy-efficiency:kde.org

As for integrating LLMs into KDE software, I think measuring and then 
providing transparency to end-users about the energy/CO2 equivalencies 
needed to train and use the models is worth pursing, when possible. 
Moreover, such high-energy consuming tools should be disabled by 
default, or should at least have reasonable default settings to minimize 
power draw.

Another idea that has come up before and is somewhat relevant in this 
context is an "eco" slider integrated into Plasma with sensible default 
settings depending on what the user wants: on one end a green setting 
indicating maximal efficiency, sometimes at the cost of functionality; 
at the other end a red setting indicating maximal functionality, 
sometimes at the cost of efficiency; and a yellow setting in the middle 
compromising between the two. This could be similar to how the Tor 
Browser Bundle has the safe-safer-safest securty levels which enable or 
disable various web features.

Cheers,
Joseph

[1] "The real climate and transformative impact of ICT: A critique of 
estimates, trends, and regulations", 2021. Charlotte Freitag, Mike 
Berners-Lee, Kelly Widdicks, Bran Knowles, Gordon S. Blair, and Adrian 
Friday. https://doi.org/10.1016/j.patter.2021.100340

> 
> --
> Alexander
> 
> --------
> [1] https://www.technologyreview.com/2023/12/01/1084189/making-an-image-with-generative-ai-uses-as-much-energy-as-charging-your-phone/
> [2] https://arxiv.org/abs/2311.16863
> 

-- 
Joseph P. De Veaugh-Geiss
KDE Internal Communications & KDE Eco Community Manager
OpenPGP: 8FC5 4178 DC44 AD55 08E7 DF57 453E 5746 59A6 C06F
Matrix: @joseph:kde.org

Generally available Monday-Thursday from 10-16h CET/CEST. Outside of 
these times it may take a little longer for me to respond.

KDE Eco: Building Energy-Efficient Free Software!
Website: https://eco.kde.org
Mastodon: @be4foss at floss.social