Interest in building an LLM frontend for KDE

[Andre Heinecke]

Hi.

On Friday, 01 December 2023 03:53:22 CET Loren Burkholder wrote:
> they can be quite useful for tasks like programming.

I need it desperately for intelligent spellchecking / grammar fixes 😅

> From a technical standpoint, such an app would be fairly easy to implement.
> It could rely on Ollama[0] (or llama.cpp[1], although llama.cpp isn't
> focused on a server mode) to host the actual LLM; either of those backends
> support a wide variety of hardware (including running on CPU; no fancy GPU
> required), as well as many open-source LLM models like Llama 2.
> Additionally, using Ollama could allow users to easily interact with remote
> Ollama instances, making this an appealing path for users who wished to
> offload LLM work to a home server or even offload from a laptop to a more
> powerful desktop.

I played around with Gpt4all a bit and liked it very much. Especially if I 
could alternatively put in my OpenAI API key in a generalized fronted. Since 
Hardware will only get better some local solutions for some easy tasks might 
also make sense. I can totally  see a use for a generalized KDE fronted or 
even Frameworks API to interact with LLMs 

> From an ideological standpoint, things get a little more nuanced. Does KDE
> condone or condemn the abstract concept of an LLM? What about actual models
> we have available (i.e. are there no models today that were trained in a way
> we view as morally OK)? Should we limit support to open models like Llama 2
> or would we be OK with adding API support for proprietary models like GPT-4?

Please leave ideology out of it, we are doing Free Software. So if you 
"Ideolically" do not want to have something I have the freedom to come with my 
different ideology and just do it anyway. If you want to really work on 
something like that and not just start some Academic discussion keep things as 
generalized and backend agnostic as much as possible IMO.

> Should we be joining the mainstream push to put AI into everything or should
> we stand apart and let Microsoft have its fun focusing on AI instead of
> potentially more useful features? I don't recall seeing any discussion about
> this before (at least not here), so I think those are all questions that
> should be fairly considered before development on a KDE LLM frontend begins.

I don't think so. I have the slight feeling that you want to start an abstract 
discussion here and then magically the "KDE Community" will develop something. 
Just do it or don't. It will always be in the users freedom to use it or not. 
I would love to have an optional KMail plugin that interacts with an LLM. 
Others might not 🤷🏻‍♂

> I fully understand that by sending this email I will likely be setting off a
> firestorm of arguments about the morality of AI, but I'd like to remind
> everyone to (obviously) keep it civil. And for the record, if public opinion
> comes down in favor of building a client, I will happily assume the
> responsibility of kicking off and potentially maintaining development of said
> client.

I don't really see why this should kick of a firestorm of arguments. It's all 
about freedom. Its not like you are proposing to forcefully feed all the users 
data in a remote LLM as a requirement to get Plasma to start.

Start a project on invent, create something useful, and then we see where it 
goes how many users it will find. How well it integrates. I would happily join 
you and I am very interested in this. A simple first useful prototype for me 
would be to have KMail Messagecomposer integration where it could help me 
write mails, just like ELOPe 😀

I am currently working on a prototype to combine  https://invent.kde.org/
schwarzer/klash/ with a local LibreTranslate to at least create fuzzy 
translations for po files and do some trivial translation tasks automatically. 
I think this is slightly related. 


Best Regards,
Andre

-- 
GnuPG.com - a brand of g10 Code, the GnuPG experts.

g10 Code GmbH, Erkrath/Germany, AG Wuppertal HRB14459
GF Werner Koch, USt-Id DE215605608, www.g10code.com.

GnuPG e.V., Rochusstr. 44, D-40479 Düsseldorf.  VR 11482 Düsseldorf
Vorstand: W.Koch, B.Reiter, A.Heinecke        Mail: board at gnupg.org
Finanzamt D-Altstadt, St-Nr: 103/5923/1779.   Tel: +49-211-28010702
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 5655 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-devel/attachments/20231201/8873a2e7/attachment.sig>