[Kexi-devel] Handling of splitted Calligra repos and API changes

Sun Aug 30 22:21:59 BST 2015

On 28 August 2015 at 19:10, Friedrich W. H. Kossebau <kossebau at kde.org> wrote:
> Hi,
>
> with KDb, KProperty and KReport, a few libs/frameworks have been already split
> off during the port of Calligra to Qt5/KF5.
>
> And they pose a challenge that needs to be solved quickly:
> traditionally we have had no ABI guarantuee in Calligra even in patch releases
> and of course also not during development. Because sourcecode of all libs and
> libconsumers were in a shared repo, and any API changes were done in atomic
> commits both to the libs and libconsumers (because by policy we require any
> commits to the repo to not break it's build).
>
> With libs and libconsumers being split in separate repos the atomicity no
> longer is given.
>
> Which results in problems:
> * people might pull from repos missing part of API change commits
> * current KDE CI triggered by commits will build repos without caring for
>   dependencies between commits, using products from previous lib builds with
>   old API for the build of libconsumer with commit using newer API
> * ?
>
> No idea how to solve the KDE CI one perfectly. But for the problem of pulling
> consistent states, what about the following rules with the split off lib
> repos:
> * no ABI breakage in patch releases for released branches
> * weekday of API breakage: a set day per week development branches can get
>   commits which involve API changes in the libs in separate repos
>   still related commits should be pushed together in an as small timewindow
>   as possible
>
> No ABI breakage in patch releases might be a no-brainer, given that targetted
> 3rd-party consumers would rely on that as well.
> The second one would pick up something which seemed to work well enough in
> times where close kdelibs and kdeapps development was common, and where people
> also did separate svn "pulls" for libs and apps. Pushing related commits to
> all repos in small timewindows might not completely match atomic commits as
> possible with subversion. But in the end we all pull in time ("fetch latest
> commits now") and not by branch state ("fetch commits until xyz id"), so in
> practice this might work as well still.
>
> What do you think? Would this work for you as well?
>
> IMHO we need to have an agreed solution here before Kexi-frameworks is merged
> back, because it will force us into this situation (where Plan currently does
> not depend on KReport, my respective patch waiting since many weeks for
> someone to push this question for handling of the API-break challenge and to
> find an agreed resolution ;) ).
>
> ((While in the discussion this is only about those 3 libs for now... My
> personal desire is to make the Calligra document libs more accessible to other
> apps, which partially do document creating/processing. For that I would see
> some advantage to split off more libs into own repos as well. In the long run
> I would like to see Calligra move out of it's own corner and integrate more
> with the normal KDE application scene.
> But more on that once we have finally passed the port hurdle, are back in
> normal feature development and can start talking post-3.0 (where 3.0 is now
> the first real release) :) ))

Friedrich,
Big thanks for this thread. Sorry for the late response.

I'd be OK with complying with the usual rules of patch releases. I
just expect that major releases can come a lot more often than usual
in case of libs I maintain, to address the challenge with API/ABI
break. I am considering to even have
$PREFIX/include/FooLib/{MAJORVERSIONNUMBER}/ dirs if really needed.
And {MAJORVERSIONNUMBER} in sonames. Side-by-side installs can be
supported this way. If 3rdparties come to use some libs, they are
relatively safe, and Kexi can use fresh dedicated versions. This is
from the POV of actual needs. The same time, packagers get predictable
rules.

API deprecation rules - sure I support that - it's a favour to
co-developers that may port away from older API just a few weeks later
than expected. If possible (from C++ POV) I wouldn't silently drop a
symbol. Unless it was just introduced in a feature branch and has
never landed in a release.

Next, how to check versions. A single properly configured kdesrc-build
can solve issues when people try to fetch individual repos and make
human mistakes.
During development you either use kdesrc-build configured to use the
branch for the repos (e.g. master) or you do that by hand. It works
for Kexi+kdb+kreport+kproperty so far.

For people building largely by hand, I'd introduce a support for
specific case: a
find_package wrapper that checks if the dependency has an exact git
revision. That's like an assertion that tells you that your deps are
too old (or too new, or even from incompatible branch - use git to
find out). (I already offer the git revision information for kdb.git,
etc. and believe providing it even in --help for apps is a good thing
in modern times).

So I don't expect many cases with we have kf5-like state for some of
calligra libs. But even now kdb.git is much easier to reuse than
before. Learned that already. Much easier to hack on, to build it.
Can't wait until 1st release is be announced :)

Re atomicity - I don't feel it is always present in single monolithic
repo when pretty active development happens:
Example follows. If you develop features A and B and both depend on
function C, and everything is developed more or less in parallel; then
if all sits in separate feature branches (happens, this is git, and
code can be nontrivial so we encourage that) - then until you do
regular merges, you're not up to date with function C's code in your A
and B branches.
I discovered how ugly is can be while working on my fixes and
mentoring some GSoC student. In the same time I had to add some helper
APIs for the student. So many checkouts of calligra.git, even with
git-new-workdir it was like wrestling with merging/cherry-picking
code. Error prone.
For a lone programmer, sure all the activities can be atomic.

So in case of separate repos I think I mentioned in some other email:
merging changes in kdb.git with master don't require me to fetch and
merge master changes of calligralibs now. Only with kdb.git master.
Smaller world, not that many conflicts. And my co-developers can
quickly test a branch A from kdb.git, B from kreport.git and C from
kexi.git. The test it all with Calligra Sheets from a specific
calligra.git branch.

Extra: Without separated repos, if you perform multiple full builds of
calligra.git you cannot combine products from separate builds. So you
configure selection of calligra-products for each build. Looks like a
lot of work.

Re atomicity #2: we can have it in releases. I'd like to still see
combined releases of our source code. And share git tags. This way if
people build all these repos for a given tag, everything can work as
in the 'one repo' times.
I'd encourage using tags for development milestones too.

-- 
regards, Jaroslaw Staniek

KDE:
: A world-wide network of software engineers, artists, writers, translators
: and facilitators committed to Free Software development - http://kde.org
Calligra Suite:
: A graphic art and office suite - http://calligra.org
Kexi:
: A visual database apps builder - http://calligra.org/kexi
Qt Certified Specialist:
: http://www.linkedin.com/in/jstaniek