Testing KTp

[David Edmundson]

On Wed, Sep 19, 2012 at 1:54 PM, Daniele E. Domenichelli
<daniele.domenichelli at gmail.com> wrote:
> On 19/09/12 13:04, David Edmundson wrote:
>> Today I found KTp-send-file doesn't work (fixed now) and simply would
>> never show any contacts.
>>
>> This has been broken since 14th July 2012, when a change was made in
>> AccountsModel over 2 months ago!
>> This means we released 0.5.0 with a completely broken component
>> without anyone so much as opening it. (or if they did, not bothering
>> to open a bug report)
>
> That's not a simple problem.
>
> By changing AccountsModel we introduced a non-backwards-compatible
> change that none of us was aware of. That's imho worse than an API/ABI
> break, becuase whoever is using it will believe that it is still
> working, while it is not.
>
> That's imho a problem in the model. We should not have the user handle
> tp-qt features directly when they use just the models, because that's
> going to break a lot more when the library will be public.
>
> If it was possible not to set the features by ourself instead of forcing
> the user to set them in the "main", all this wouldn't be a problem, and
> we could have proper unit tests for the model.
>
> The main problem is that AccountsModel expects the AccountManager from
> the user, and if that is not configured with the correct features, then
> the problem shows. Moreover, I think that the problem comes out when the
> filtering model is used. So it's not a problem of the AccountsModel
> itself but also of the AccountsFilterModel
>
> I'm a bit confused about how to handle this.
>
> Can we check if the AccountManager is ready with some features?

Yes. Tp::ReadyObject has a "requestedFeatures" object

>
> It would be better if we could create it inside the AccountsModel, what
> was the problem again with doing that? Can we create a minimun
> AccountsManager and make it ready at runtime according with the features
> required by AccountsFilterModel?

The issue of completely internalising the AccountManager into the
model is that lots more things need the account manager. Pretty much
everything, and as I understand it having 2 is rather silly.

There was a design once where the account manager was inside the
model, and there was a method to access it, but this made the
contact-applet always keep an entire model being run just to access
one contact...

What I hadn't realised was that we can call becomeReady() more than
once, so AccountsModel could call it again with the  features it needs
without a problem.  That would also have solved the issue here, also I
want to get rid of the stupid API (that I made) where I call
setAccountManager on everything rather than it being in the ctor where
it belongs.

>
> Can we have the model and/or the filter model return the list of
> required features?
>
As a static method? I've thought of that before thinking "that would
be a really good idea" then never did it.


> Can we have our set of features that hide KTp features?
>
> I'm running out of ideas.
>

I'm not too concerned with this bug getting in, this sort of thing
happens in software. The bad part is not catching it.


>
>> Frankly this is rubbish (not blaming anyone, it's as much my fault as
>> anyone else.. possibly more so), but we need to work on improving our
>> testing process so this doesn't happen again.
>>
>> Would it help if we made a checklist of things to run/test before a
>> release, which should be run between branching and tag/release?
>
>
> Agreed, this is rubbish. A checklist could be useful if used properly...
> This is my proposal:
>
> - We keep the checklist on the wiki.

Ok, I'll start this, though I would like some help.
http://community.kde.org/Real-Time_Communication_and_Collaboration/ReleaseTesting#Release_Testing_for_X.Y-rc1

It's deliberately quite vague and simple.

If we start writing proper test specs with detailed "ensure X happens,
and ensure Y is ... " no-one will write them, and no-one will bother
to run it.

There's tonnes more to add to that list, so get to it :)


> - Whenever the hard freeze time comes, we tag an internal 0.X.Y-rc1
>   release
> - We run the tests using the 0.X.Y-rc1 keeping track of them in the
>   wiki (i.e. adding the name of the person who run it next to the test)
> - If a test fails, we fix it (same if we fix some bug that is not
>   covered by the tests) and we tag an 0.X.Y-rc2 including one or more
>   bugfix.
> - We clean the wiki and run all the tests on the latest rc until they
>   all pass.
> - We tag the 0.X.Y release and create the most perfect tarballs ever.
>
> Anyway having automatic unit tests to do the checks whatever can be
> checked using them would be a lot better.
>
>
>
> Daniele
> _______________________________________________
> KDE-Telepathy mailing list
> KDE-Telepathy at kde.org
> https://mail.kde.org/mailman/listinfo/kde-telepathy