Regressions || Huston we have a problem

[Sebastian Sauer]

Aloha,

yesterday at the ODF plugfest 2011 ( http://www.opendocsociety.org/news/2011-
berlin-plugfest/ ) we had two interoperability tests and failed on both of 
them. It seems both where working at some point but cause of regressions 
didn't any longer at the event. That is a problem.

Let me add that the problem are NOT the regressions. That can happen. The 
problem is that we did not discover them for more then a month till that 
event.

So, the question is how to improve that situation to make sure we are able to 
discover regressions faster?

I see two ways;
1. unittests also for saving.
2. cstester roundtrips.

The first would be optimal but it would need *lot* of time especially if we 
try to get a good coverage done.

The second is the fastest way (I can think of atm). We could first fix 
cstester so document-roundtrips are proper working and second move that to a 
server that runs cstester against the large collection of documents located on 
the KDE-svn server in the kofficetests directory. Maybe we can run that once a 
day in an automated way and then incoperate that into our IRC unittest-bot? Or 
maybe we can provide a webpage that shows in which ways what documents changed 
from one day to the other?

What do you think? Is there maybe a better way? Maybe even an easier way? 
Or....?