Release Script

[Albert Astals Cid]

El Dimecres, 20 de juny de 2012, a les 21:30:23, Michael Jansen va escriure:
> Hi

Hi

> 
> I started to work on some script to help with our release creation and
> studied both dirks and allens scripts for input. But the recent discussion
> and my personal experience in doing releases (working as a software
> configuration manager) made me realize i (or better we) should take a step
> back and think about that process first before scripting it.
> 
> This is the chance and perfect time to solve the release problem once and
> for all. There are quite some people aware there is room for improvements
> and there currently is a lot of discussion and action going on.
> 
> The biggest tool out there used by thousands of hobby programmes and big
> corporations to build, release, package, test and deploy stuff is maven.
> While it is not a perfect tool in any sence (Don't ever get me started on
> that) it has some nice little features we should strive to copy. When
> needed i will talk about that.
> 
> WHICH VERSION TO BUILD?
> 
> This is the biggest concern I have with out current release setup. A lot of
> our packages contain no version information apart from the name of the build
> package (kdebase-4.8.3.tgz). I consider that an absolute no go. But the
> thing that strikes me even more wrong is that building kdepim 4.2 against
> kdelibs 4.9 leads to libraries named libakregatorprivate.so.4.9.0 .

Yes, that's very bad

> If we really want to decouple our releases and be more flexible with doing
> them i consider this change a requirement for any decision in that regard.
> 
> Each and every module has to have its own version number build in. I guess
> with the frameworks branch much of kdelibs already got that change (no idea
> really, anyone with input?). But we have to do the same with the rest of our
> modules.

No idea about frameworks. David? Kevin?

> 
> The version has to live in one agreed upon place in all modules we have. The
> release script will know where to look for it, how to parse it, increase
> it. That way it is possible to say build the next minor release and the
> script will do the rest.

If we are going to make releases of different stuff at different times as it 
was suggested i guess this is mandatory.

> 
> Maven works like that:
> 
> Current Version: 4.7.1-SNAPSHOT (in pom.xml) | Task: Build a minor release
> 
> 1. Checkout sources and change 4.7.1-SNAPSHOT in pom.xml to 4.7.1
> 2. Build, run test and other needed stuff.
> 3. If successfully commit the changed version of pom.xml
> 4. Make the tag MY_PROJECT-4.7.1 (or whatever)
> 5. change 4.7.1 in pom.xml to 4.7.2-SNAPSHOT
> 
> The last step is done in maven because the support something called snapshot
> release which means their version looks like 4.7.1-20120621_151400. I think
> it could make sense to support that too. Stuff build from master or branch
> would be easily distuinguishable from really released stuff.

That works fine for me, though unfortunately we usually have to re-package 
some tarballs due to fixes that are needed into the release. How do you fit 
this particularity into this way of working?

> 
> The other thing to notice is that there was exactly one version having the
> released version number. No possibility for any kind of confusion there.
> 
> I see one problem. As you can see the changed version information is only
> committed AFTER build and test in this setup. That can take quite some time.
> In a project as big as ours that opens the possibility that during that
> time some else commits a change. Which makes it impossible for the script
> to commits its change.
> 
> 1. Solution: Branch. The Script could create a branch for the release.

Creating a branch for release would also probably "fix" the problem i spoke in 
the previous paragraph

> 
> 2. Solution: Lock the repo (A no go in my opinion)

Yeah, locking kills babies

> And a little problem. The feasability of beeing able to build our software
> before packaging. I already have a solution to build the packages with
> build- tool as a test. But no idea yet how to combine build-tool with this
> script unless i add this into build-tool. And the admins would like to have
> python. Not ruby. And build-tool would not mash with the idea to use
> jenkins to trigger the releases.

"this script" == maven?

> But i will make sure it is possible to do the release without building (for
> emergencies) by calling just one script.
> 
> WHERE IS THE RELEASE CONFIG?
> 
> We currently have our release configuration outside of our source code.
> Neatly packed away into (currently) the kde-commons subversion module.
> 
> Maven uses a xml file located inside the sources to find out how it is
> supposed to build the package. With configuration i mean
>     - the current version number
>     - the versioning number scheme,
>     - possible hooks into the release process (additional non standard stuff
> to execute during the release process)
>     - what to pack
>     - what to ignore.
>     - How/Where to Tag
> While this is not the best solution it has a certain charm. If we don't do
> that people will tend to forget that part of the software development
> livecycle. Since we do not build binary packages ourselves (I consider it a
> bad idea to relabel my sources because i want to change the way my binary
> packages are built) it think the cons aren't that bad.
> 
> So the changes i would propose are:
> 
> 1. Add the version information into all of our modules in a way that a
> outside script can get it. Some kind of file for example that is included
> by the toplevel CMakeLists.txt and only contains the version information.

Makes sense

> 2. Make the necessary build-system changes to use this version information
> for the .SO names.

Makes sense

> 3. Make it a rule that there is no other place allowed to contain version
> information. If you need it somewhere else use cmakes configure_file(). Btw.
> the version information should contain a human readable part 4.8.3-beta2
> too. (For the kdepims out there).

Makes sense

> 4. Allow the script to maintain the version increase. We have to decide how
> to solve the race condition inherent in this.

Don't understand what you mean by this.

> 5. Add a file into our modules that describe what to pack/ignore for our
> source distributions. This contains the "removestuff" script parts from kde-
> common/release. Use a file-format that is extensible (JSON, xml, ...). And
> add possibly more (time will tell)

Make sense

> To summarize. I describe a script that builds exactly one module, needs
> nothing else besides this module, and ends when that package is build and
> the release is tagged.

I see a problem here, the script needs to know which branch to tag. Where 
would you put this information?

> We will need another script (or jenkins) around it do handle the modules in
> the right order, to checkout the correct branch and so on.

Ok, i see :D

> Each and every package we release is its own little project then. If some
> modules belong tighter together we have to release some as one combined
> package.

In general it seems a very well though out proposal, i don't see any obvious 
problem in what you are describing.

One thing that is problematic with the current build scripts is the need of 
having up to date meinproc+kdoctools to build the packages that come after 
kdelibs, do you think that'd be an issue?

Cheers,
  Albert

> 
> Mike