Testing KTp

[Daniele E. Domenichelli]

On 19/09/12 13:04, David Edmundson wrote:
> Today I found KTp-send-file doesn't work (fixed now) and simply would
> never show any contacts.
> 
> This has been broken since 14th July 2012, when a change was made in
> AccountsModel over 2 months ago!
> This means we released 0.5.0 with a completely broken component
> without anyone so much as opening it. (or if they did, not bothering
> to open a bug report)

That's not a simple problem.

By changing AccountsModel we introduced a non-backwards-compatible
change that none of us was aware of. That's imho worse than an API/ABI
break, becuase whoever is using it will believe that it is still
working, while it is not.

That's imho a problem in the model. We should not have the user handle
tp-qt features directly when they use just the models, because that's
going to break a lot more when the library will be public.

If it was possible not to set the features by ourself instead of forcing
the user to set them in the "main", all this wouldn't be a problem, and
we could have proper unit tests for the model.

The main problem is that AccountsModel expects the AccountManager from
the user, and if that is not configured with the correct features, then
the problem shows. Moreover, I think that the problem comes out when the
filtering model is used. So it's not a problem of the AccountsModel
itself but also of the AccountsFilterModel

I'm a bit confused about how to handle this.

Can we check if the AccountManager is ready with some features?

It would be better if we could create it inside the AccountsModel, what
was the problem again with doing that? Can we create a minimun
AccountsManager and make it ready at runtime according with the features
required by AccountsFilterModel?

Can we have the model and/or the filter model return the list of
required features?

Can we have our set of features that hide KTp features?

I'm running out of ideas.


> Frankly this is rubbish (not blaming anyone, it's as much my fault as
> anyone else.. possibly more so), but we need to work on improving our
> testing process so this doesn't happen again.
>
> Would it help if we made a checklist of things to run/test before a
> release, which should be run between branching and tag/release?


Agreed, this is rubbish. A checklist could be useful if used properly...
This is my proposal:

- We keep the checklist on the wiki.
- Whenever the hard freeze time comes, we tag an internal 0.X.Y-rc1
  release
- We run the tests using the 0.X.Y-rc1 keeping track of them in the
  wiki (i.e. adding the name of the person who run it next to the test)
- If a test fails, we fix it (same if we fix some bug that is not
  covered by the tests) and we tag an 0.X.Y-rc2 including one or more
  bugfix.
- We clean the wiki and run all the tests on the latest rc until they
  all pass.
- We tag the 0.X.Y release and create the most perfect tarballs ever.

Anyway having automatic unit tests to do the checks whatever can be
checked using them would be a lot better.



Daniele