RC readiness, not

[Thorsten Zachmann]

Hello,

On Saturday 14 July 2012 23:53:03 Friedrich W. H. Kossebau wrote:
> Hi,
> 
> Am Freitag, 13. Juli 2012, 19:07:21 schrieb Thorsten Zachmann:
> > Hello,
> > 
> > On Friday 13 July 2012 16:11:24 Cyrille Berger Skott wrote:
> > > Hum, the question that needs to be answered at this point is: is
> > > calligra
> > > 2.5 worse or better than calligra 2.4.3. If it is better (ie no
> > > regressions, more bug fixes more features) then it should be released to
> > > replace 2.4.3. However if it is worse (ie regressions), then it should
> > > be
> > > delayed.
> > 
> > 2.5 is much better then 2.4.3 and has quite some improvements. And I agree
> > that we should release 2.5 as long as there are no regressions comapred to
> > 2.5.
> 
> Okay, reasonable.
> Butbutbut... :) ... given the few ODF calligratests which are still failing,
> wouldn't it be great if the release would be one where finally no tests are
> failing? You/we are so close :)
>
> What are tests worth if everyone is okay with some of them failing? Why does
> Jos(?) maintain that server at all? Would it not be a good feeling if the
> fail count is at 0? So one knows she/he can rely on the libs used?

I agree we should aim for 0. And it is true that 0 would be much more helpful.

> If everyone could join the efforts, the count of failing ODF calligratests
> (33 for master, says TeamCity server) and the tests (16 for master, says
> TeamCity server) should be pushed to 0 in a few weeks, no?

I see about 133 failures. Ok if I only count the ones in odf docs I come to 
33.  However I don't think that waiting until all is perfect will help us to 
release as then we will never release. It is quite important to release to 
show we are making progress. And our progress is quite impressive. A lot less 
documents fail now then before. So make sure users get that as fast as 
possible instead of using the older stuff that is much worse. 
We should try to put as much fixes in as soon as they are available to improve 
the situation even more. And due to the new release model I think that is 
quite easily possible.
I have to say that I hate that there are bugs in our programs that are not 
fixed but as it is software it will never be bug free.

Ok some comments about the bugs we are facing

o We have about 17 documents showing something like

[01:46:52]: [kofficetests-ods] 
interoperability/spreadsheets/oocalc/oos_inserted_oo_spreadsheet.ods (1m:34s)
[01:48:27]: 
[interoperability/spreadsheets/oocalc/oos_inserted_oo_spreadsheet.ods] 
MISSINGFILE: Object 1/content.xml
MISSINGFILE: Object 1/styles.xml 
MISSINGFILE: Object 1/settings.xml
MISSINGFILE: ObjectReplacements/Object 1

I had a look at the test failures of some documents and I think there is also 
a bug in the script reporting test failures where there is none. The files are 
there. Maybe the test script has a problem with filenames that contain an empty 
space.

So 17 bugs less. Yeah!

o We have about 3 with the following error

[01:51:31]: [kofficetests-ods] 
interoperability/spreadsheets/oocalc/oos_bugs_list.ods
[01:51:31]: [interoperability/spreadsheets/oocalc/oos_bugs_list.ods] Timeout!

This is also not realy a bug as the server has a limit on how long a test can 
run and some big documents this limit seems to be to small. Sure that might 
point also to a bug of being slow to progress these documents. However I don't 
think this is something that can be easily fixed in weeks.

o We have about 6 documents showing the following error

[01:53:20]: [kofficetests-odt] 
interoperability/wordprocessing/oowriter/oow_toc_colored_underline.odt
[01:53:20]: 
[interoperability/wordprocessing/oowriter/oow_toc_colored_underline.odt] 
(unknown file):118: error: element "text:span" not allowed here; expected the 
element end-tag or text 
(unknown file):119: error: element "text:span" not allowed here; expected the 
element end-tag or text
(unknown file):120: error: element "text:span" not allowed here; expected the 
element end-tag or text
(unknown file):121: error: element "text:span" not allowed here; expected the 
element end-tag or text 

o We have about 10 documents with this error:

[01:53:20]: [kofficetests-odt] 
interoperability/wordprocessing/oowriter/oow_toc_colored_underline.odt
[01:53:20]: 
[interoperability/wordprocessing/oowriter/oow_toc_colored_underline.odt] 
INVALIDCONTENTXML: (unknown file):113: error: element "text:index-title" 
missing required attribute "text:name" 

gopalK is looking in the two above.

o 1 ASSERT

[01:53:20]: [kofficetests-odt] odf/odt/OpenDocument-v1.1.odt (6s)
ASSERT failure in QVector<T>::at: "index out of range", file 
../../include/QtCore/../../../../src/qt-everywhere-opensource-
src-4.7.4/src/corelib/tools/qvector.h, line 339

However for me it does not assert. Might be a timing problem

and some other problems

> And blocking the release on this should also help a little bit with
> motivation to switch from hacking exciting new features to making/keeping
> the foundation relyable ;)

Unfortunately that is not what is going to happen. We have tried it before.

> > > As it stands, the two bugs that are tagged as RCs are not regressions,
> > > they
> > > both happens in 2.4 and 2.5, so if there are new regressions they should
> > > be
> > > noted and tagged appropriely, otherwise, I think we should proceed.
> > 
> > Agreed.
> > 
> > > And yes, Words is not considered end-user ready, this why the release
> > > has
> > > the 2.5 version number and not 3.0.
> 
> Hm, is 2.5 a known symbol for not-end-user-ready? ;) This aspect is surely
> also not overemphasized in
> http://www.calligra.org/news/calligra-2-4-released/ ;)
> Which I can understand. See, I was also excited about you finally doing a
> first release with 2.4, showing that the project in progressing. Saying "oh,
> but not really done" would have been working against the original message.
> But now I think the sticker "Technology preview" should be a little bigger
> on those parts which are not too reliable.

the problems that are reported might not affect the user at all

> > And we still can fix those bugs commit them to master and backport them to
> > 2.5 so the next releases will be even better then what we have now.
> 
> But a 2.5 which is not as perfect as it could be (by known failing tests)
> leaves bad annotations with that version. And with the name Calligra. Which
> would be sad. Because besides the few issues IMHO not just Krita, but also
> Words, Stage and Flow are quite usable for end-users by my little
> experience. No idea about Sheets and Karbon and the others.

The same happens if we let users work with the old version that is worse then 
the new version

> Screwing data on saving is really awful. Are you using Calligra programs
> youself for your serious stuff? Do it :) And feel the light fear ;)
> Or why else is the release done? For whom?

the validation failing is one part but that does not mean we screw the data on 
saving (it don't say we don't screw). However this test do not check if we 
actually loose data. We might do but that is not tested by these tests.

Thorsten