cstester - a tool for testing loading and painting of documents

[Inge Wallin]

On Tuesday, January 18, 2011 14:07:21 Thorsten Zachmann wrote:
> Hello all,
> 
> I'm currently working as part of my job for Nokia on cstester a tool to
> test loading and painting for stage, tables and words. The aim of the tool
> is to make sure there are no regressions added when features are modified.
> This is done by loading the document and comparing some pre generated
> images of the content with created images of the content. Also a good code
> coverage should be possible to reach by using this method it uses a lot of
> code when doing that.
> 
> The idea is to create a document set which contains features and test these
> documents on a regularly basis.
> 
> There are still some open questions:
> - Most important one is how to make sure font rendering is always the same.
> One idea here is to create a virtual image that can be used to run the
> tests. - What is the best way to store the test set and images used for
> testing Should we create a new repository where the documents are stored.
> - How to automatically run the tests to find regressions.
> 
> I will shortly commit the branch cstester-zachmann
> 
> How does it work
> 
> Creating the images used for verifying
> 
> cstester --create test.odp
> 
> will create the test images used for comparison. One needs to make sure the
> images are as they are expected to be. The images to check will be created
> in the directory test.odp.check so you need write access to the directory
> the file is in.
> 
> Verifying that still all works fine
> 
> cstester --verify test.odp
> 
> This will load the document and create images of the content and then
> verify them against the images saved in test.odp.check
> 
> It is possible to run the tool on multiple files at once by specifying
> muliple files on the command line
> 
> cstester --create test1.odp test2.odp test3.ods test3.xls
> 
> Currently it has support for documents which are supported by stage and
> tables. words is going to follow. Other document types should be easy to
> add.
> 
> I also have plans for the future to extend the tool by:
> - adding a save / load circle to test that all is saved and loaded
> correctly and we don't have any data loss there.
> - adding odf verification for the saved data

For Words, how about adding a dump of the text layout and compare that?  That 
should be easier (and faster!) than generating and comparing bitmaps. The 
bitmaps themselves would also not give much added information, especially if 
the goal is to find faulty layouts which seems to be the biggest problem right 
now except for crashes. 

> Also we are working inside Nokia on a set of documents that can be used for
> this kind of testing. This documents will then be available for the
> community.
> 
> Thorsten
> _______________________________________________
> calligra-devel mailing list
> calligra-devel at kde.org
> https://mail.kde.org/mailman/listinfo/calligra-devel