[Kde-pim] akonadi testing framework

[Volker Krause]

On Thursday 01 May 2008 05:34:33 Igor Trindade Oliveira wrote:
> my name is Igor and i am participating on GSoC
> mentored by Kevin Krammer , basically my project is to
> do a testing framework that when a developer make a
> agent/resource, the testing framework can test(of
> course :>) this agent/resource automatically.
> And i am right now doing a requirement analysis to see
> what the testing framework need to do.
> So i am asking for you, when you are testing your
> agents/resources what are the steps done by you?

I'm a bit late with this, sorry.

I see two major issues with testing currently:

* Isolation from the production data
Kevin wrote already a bit about that. I see two ways to do that:
(1) start a second Akonadi server instance which is used for testing parallel 
to the production server
(2) use the production server for testing and save and restore its state 
before/after running the tests

I would clearly prefer (1) for safety reasons although that's probably a bit 
harder to implement. The server currently ensures in a few places that there 
is only one instance, mostly implicitly by using a single socket name/D-Bus 
service/etc. These places need to be identified and extended to allow running 
a second instance.

* Performing high-level tests on actual resources
The unit tests we have so far to test the client library operate on fixed 
dummy data and are thus unusable for testing resources. Writing similar tests 
for resources is possible but cumbersome, mostly due to:
- no sharing between resources possible
- hardcoded data for comparison
Therefore, testing resources is mostly done manually, by creating an instance, 
configuring it, syncing it, fetching collections and items and comparing if 
the seen data matches the expectations, manipulate collections/items, do all 
the above again and see if these changes had any effect and finally cleanup 
again.

Parts of this can easily done automatically (and mostly generic), such as the 
setup/cleanup. The tricky part is probably finding a way to easily provide 
comparison data and match Akonadi items to that.

A nice end result would be a ready to use generic resource tester which can be 
configured to define which functions and properties to test (not all 
resources support all changes eg.) and how to set up the resource for that. 
This would also allow to easily test the same resource with different 
settings. 
Not sure if that's possible, but at least needing as less code as possible to 
adapt the resource tester to a specific resource is IMHO the most important 
goal here.

regards
Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20080525/d582a831/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/