Regressions || Huston we have a problem

[Boudewijn Rempt]

On Friday 15 July 2011 Jul, Sebastian Sauer wrote:
> Aloha,
> 
> yesterday at the ODF plugfest 2011 ( http://www.opendocsociety.org/news/2011-
> berlin-plugfest/ ) we had two interoperability tests and failed on both of 
> them. It seems both where working at some point but cause of regressions 
> didn't any longer at the event. That is a problem.
> 
> Let me add that the problem are NOT the regressions. That can happen. The 
> problem is that we did not discover them for more then a month till that 
> event.
> 
> So, the question is how to improve that situation to make sure we are able to 
> discover regressions faster?
> 
> I see two ways;
> 1. unittests also for saving.

We need those...

> 2. cstester roundtrips.
 
And those as well. Jos' server checks whether we can actually load all the kofficetests document, and even there we have failures.


> The first would be optimal but it would need *lot* of time especially if we 
> try to get a good coverage done.
> 
> The second is the fastest way (I can think of atm). We could first fix 
> cstester so document-roundtrips are proper working and second move that to a 
> server that runs cstester against the large collection of documents located on 
> the KDE-svn server in the kofficetests directory. Maybe we can run that once a 
> day in an automated way and then incoperate that into our IRC unittest-bot? Or 
> maybe we can provide a webpage that shows in which ways what documents changed 
> from one day to the other?


The KO sponsored server that Cyrille is working on should run cstester on the kofficetests repo -- it's easily got the capacity for that. Setting it up takes a lot of effort though.

> 
> What do you think? Is there maybe a better way? Maybe even an easier way? 
> Or....?


-- 
Boudewijn Rempt
http://www.valdyas.org, http://www.krita.org, http://www.boudewijnrempt.nl