Emerge as an open package manager?

Wed Apr 6 02:47:33 CEST 2011

On Thursday 31 March 2011 09:30:11 Pau Garcia i Quiles wrote:
> Hello,
>
> What do you think?
>
> http://www.elpauer.org/?p=687
>
> Copying and pasting for the sake of easier discussion:
>
> --8<----
>
> A while ago I said Koen from Emweb made an interesting proposal at
> FOSDEM about emerge, the KDE Windows build tool.
>
> Yesterday, Jarosław Staniek and I reaffirmed our commitment to
> ‘emerge’. Today, I’d like to go a bit further: let’s bring more
> developers to emerge by opening it up to other projects. Keep reading!
>
> What is emerge, why is it important and what was Koen’s proposal?
>
> Fact: Microsoft Windows is very different to Unix in regards to
> development.
>
> On Unix platforms -that includes Linux and Mac OS X-, software
> isusually installed to /usr: applications in /usr/bin and /usr/sbin,
> libraries in /usr/lib, headers in /usr/include, common resources in
> /usr/share, etc. Also, dependency management is usually something you
> can count on: when you install kdelibs5-dev in Ubuntu, it will
> automatically install libqt4-dev, kdelibs5-data, libfreetype
> (runtime), etc That makes setting up a development environment a very
> easy task: look for shared libraries, header files, etc in the common
> places and you will probably find them.
>
> On Windows there is nothing like that. When you want to compile an
> application, you need to provide (build and install) all its
> dependencies, and you need to tell Visual Studio where to find
> everything. Even CMake usually needs some help in the form of a hint
> for CMAKE_PREFIX_PATH. As you may imagine, building KDE, which has
> more than 200 third party dependencies and tens of modules (and with
> the move + split to git, many more) becomes an almost insurmountable
> task.
>
> ‘Emerge’ to the rescue: inspired by Gentoo‘s emerge, Ralf Habacker,
> Christian Ehrlicher, Patrick Spendrin and others (yours faithfully
> included) developed a tool which downloads the source, configures,
> builds, installs and packages KDE and its dependencies. It makes a
> world of difference when building KDE. Actually, it makes building KDE
> on Windows possible. Once more: thank you very much guys, impressive
> tool.
>
> There are two well-differentiated parts in emerge, the ‘engine’ and
> the ‘recipes’.
>
> The ‘engine‘ takes care of downloading the sources (tarballs,
> checkouts from Subversion and git clones, etc) , configuring and
> building for the usual build systems (QMake, CMake, NMake makefiles,
> etc), installing, bundling packages together and much more. All that
> for four compilers (MSVC2008, MSVC2010, MinGW32 and MinGW-W64). It’s
> the equivalent to Gentoo‘s emerge, Debian‘s dpkg +debhelper/cdbs or
> Homebrew and MacPorts for Macintosh.
>
> The ‘recipes’ are the package-specific “instructions” a particular
> piece of software needs to build: what’s the name of the tarball? URL
> to download from? What versions are available? What build system to
> use to configure and build? Does it need any Windows-specific patch?
> They are written in Python using some helper modules ‘emerge’ provides
> (the debhelper-like part I mentioned above).
>
> Currently, there are about 70 recipes in emerge, organized in our
> ‘portage tree’ (bear in mind the names are taken from Gentoo but the
> internals of the tool are completely different). With those 70
> recipes, we are able to build most KDE modules. Problem is to provide
> all the optional features, we are missing probably 200 to 250 more
> recipes. Given that KDE on Windows is quite short on developers, we
> have to decide: either we fix bugs and improve the integration of KDE
> on Windows, or we keep track of the dependencies. I won’t lie: for KDE
> on Windows, I’d rather focus on development than on packaging.
>
> Given that writing and maintaining recipes does not strictly require
> programming skills (although they help  ), during my talk at FOSDEMI
> kept asking people to join the KDE on Windows team as packagers, even
> if you know little to nothing about software development. If you think
> about it, that’s what we have in Linux distributions: there are
> hundreds (thousands?) of packagers in Debian, Fedora, openSuse, etc
> which take care of the ‘recipes’ to build software developed by
> someone else (‘upstream’, in the parlance). What we have in KDE on
> Windows, where we are the KDE packagers, the KDE developers (upstream)
> and the emerge developers, is actually an anomaly.
>
> Then Koen took the microphone and said: why don’t you open emerge to
> other projects?
>
> And I there I realized he was damn right
>
> We have developed an awesome tool to download, configure, build,
> package and install third party software on Windows, to manage
> dependencies, to update, etc. For four compilers, no less. The
> ‘engine’ of emerge is not really tied to KDE. We already have a lot of
> recipes for software which is not KDE specific: openssl, Qt, bzip2,
> libpng, mysql, etc.
>
> I see no actual reason to disallow Gnome, GStreamer,OpenSceneGraph,
> LibreOffice and many others in (assuming someone wants to take care of
> them, of course – no, do not look at me! I don’t have time!).
> Currently, the only ‘barrier’ preventing those recipes in is ‘emerge’
> is developed in the KDE repository and you need a KDE developer
> account to commit your recipes. That’s about it.
>
> There is also a ‘perception’ problem: “if emerge is developed in KDE,
> it’s because it’s a KDE-thingy, right?”. Well, no. One way to avoid
> that would be to graduate emerge from KDE and make it an independent
> project, one in which the current developers have accounts, and new
> developers get an “emerge project” account instead of a KDE account to
> commit recipes. Please note “graduating” does not mean “expelling”,
> “firing” or anything peyorative. It does not mean that ‘emerge
> developers’ (engine and recipes) are worse than KDE developers, or
> less KDE developers than our god David Faure: in KDE, we consider
> Debian, Fedora, openSuse, etc KDE packagers as equals, they are also
> KDE developers and many of them are in the KDE eV(funny thing is I am
> not).
>
> My wish for today: think of one application or library you’d like to
> see in Windows and become its maintainer in emerge for Windows.
>
> For now, let’s do this in the kde-windows mailing list and if this
> idea succeeds, then we’ll talk about graduating. You could also try to
> add support for a new compiler (OpenWatcom, Intel C++, etc) but that’s
> a lot more more and it’s not a priority now.
>
> PS: Yes, I know about CoApp, Microsoft‘s similar effort, with some
> magic involved. I’ve been watching and trying it since day 1. I’m
> really interested. It’s just not there yet and I do not have spare
> time to helpGarrett (hey Microsoft, hire me and I’ll have all the time
> in the world!  )
>
> --8<----
>
> --
> Pau Garcia i Quiles
> http://www.elpauer.org
> (Due to my workload, I may need 10 days to answer)
> _______________________________________________
> Kde-windows mailing list
> Kde-windows at kde.org
> https://mail.kde.org/mailman/listinfo/kde-windows

I don't know if other people have commented on the blog page instead, but as far as I'm concerned I find the idea to 
make complete sense and be technically doable. 

Indeed a first good step towards opening emerge to non-kde communities would be to move it to an independent repository. 
The question is which one :)

Anyway FWIW I like the idea!
Cheers

-- 
Romain Pokrzywka | romain.pokrzywka at kdab.com | Senior Qt Software Engineer & Trainer
Klarälvdalens Datakonsult AB, a KDAB Group company
Tel. Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322)
KDAB - Qt Experts - Platform-independent software solutions
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2396 bytes
Desc: not available
Url : http://mail.kde.org/pipermail/kde-windows/attachments/20110405/38bf3d5c/attachment.p7s 