Release Script

[Michael Jansen]

Hi

I started to work on some script to help with our release creation and studied 
both dirks and allens scripts for input. But the recent discussion and my 
personal experience in doing releases (working as a software configuration 
manager) made me realize i (or better we) should take a step back and think 
about that process first before scripting it.

This is the chance and perfect time to solve the release problem once and for 
all. There are quite some people aware there is room for improvements and 
there currently is a lot of discussion and action going on.

The biggest tool out there used by thousands of hobby programmes and big 
corporations to build, release, package, test and deploy stuff is maven. While 
it is not a perfect tool in any sence (Don't ever get me started on that) it 
has some nice little features we should strive to copy. When needed i will 
talk about that.

WHICH VERSION TO BUILD?

This is the biggest concern I have with out current release setup. A lot of 
our packages contain no version information apart from the name of the build 
package (kdebase-4.8.3.tgz). I consider that an absolute no go. But the thing 
that strikes me even more wrong is that building kdepim 4.2 against kdelibs 
4.9 leads to libraries named libakregatorprivate.so.4.9.0 .

If we really want to decouple our releases and be more flexible with doing 
them i consider this change a requirement for any decision in that regard.

Each and every module has to have its own version number build in. I guess 
with the frameworks branch much of kdelibs already got that change (no idea 
really, anyone with input?). But we have to do the same with the rest of our 
modules.

The version has to live in one agreed upon place in all modules we have. The 
release script will know where to look for it, how to parse it, increase it. 
That way it is possible to say build the next minor release and the script 
will do the rest.

Maven works like that:

Current Version: 4.7.1-SNAPSHOT (in pom.xml) | Task: Build a minor release

1. Checkout sources and change 4.7.1-SNAPSHOT in pom.xml to 4.7.1
2. Build, run test and other needed stuff.
3. If successfully commit the changed version of pom.xml
4. Make the tag MY_PROJECT-4.7.1 (or whatever)
5. change 4.7.1 in pom.xml to 4.7.2-SNAPSHOT

The last step is done in maven because the support something called snapshot 
release which means their version looks like 4.7.1-20120621_151400. I think it 
could make sense to support that too. Stuff build from master or branch would 
be easily distuinguishable from really released stuff.

The other thing to notice is that there was exactly one version having the 
released version number. No possibility for any kind of confusion there.

I see one problem. As you can see the changed version information is only 
committed AFTER build and test in this setup. That can take quite some time. 
In a project as big as ours that opens the possibility that during that time 
some else commits a change. Which makes it impossible for the script to 
commits its change.

1. Solution: Branch. The Script could create a branch for the release.

2. Solution: Lock the repo (A no go in my opinion)

And a little problem. The feasability of beeing able to build our software 
before packaging. I already have a solution to build the packages with build-
tool as a test. But no idea yet how to combine build-tool with this script 
unless i add this into build-tool. And the admins would like to have python. 
Not ruby. And build-tool would not mash with the idea to use jenkins to 
trigger the releases.

But i will make sure it is possible to do the release without building (for 
emergencies) by calling just one script.

WHERE IS THE RELEASE CONFIG?

We currently have our release configuration outside of our source code. Neatly 
packed away into (currently) the kde-commons subversion module.

Maven uses a xml file located inside the sources to find out how it is 
supposed to build the package. With configuration i mean 
    - the current version number
    - the versioning number scheme,
    - possible hooks into the release process (additional non standard stuff
      to execute during the release process)
    - what to pack
    - what to ignore.
    - How/Where to Tag
While this is not the best solution it has a certain charm. If we don't do 
that people will tend to forget that part of the software development 
livecycle. Since we do not build binary packages ourselves (I consider it a 
bad idea to relabel my sources because i want to change the way my binary 
packages are built) it think the cons aren't that bad.

So the changes i would propose are:

1. Add the version information into all of our modules in a way that a outside 
script can get it. Some kind of file for example that is included by the 
toplevel CMakeLists.txt and only contains the version information.

2. Make the necessary build-system changes to use this version information for 
the .SO names.

3. Make it a rule that there is no other place allowed to contain version 
information. If you need it somewhere else use cmakes configure_file(). Btw. 
the version information should contain a human readable part 4.8.3-beta2 too. 
(For the kdepims out there).

4. Allow the script to maintain the version increase. We have to decide how to 
solve the race condition inherent in this.

5. Add a file into our modules that describe what to pack/ignore for our 
source distributions. This contains the "removestuff" script parts from kde-
common/release. Use a file-format that is extensible (JSON, xml, ...). And add 
possibly more (time will tell)

To summarize. I describe a script that builds exactly one module, needs 
nothing else besides this module, and ends when that package is build and the 
release is tagged.

We will need another script (or jenkins) around it do handle the modules in 
the right order, to checkout the correct branch and so on.

Each and every package we release is its own little project then. If some 
modules belong tighter together we have to release some as one combined 
package.

Mike

-- 
Michael Jansen
http://michael-jansen.biz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-buildsystem/attachments/20120620/e71156bd/attachment.html>