[Kmymoney-devel] Improving the test suite for MyMoneyFile

[Cristian Oneț]

2015-02-20 13:24 GMT+02:00 victor <pedretti_86 at hotmail.com>:
> Hi,
>
> I have tried to become familiar with QSignalSpy. For instance, to comply
> with "After adding a price 'valueChanged()' should be emitted.", I have
> implemented this:
>
> qRegisterMetaType<MyMoneyAccount>("MyMoneyAccount");
>
> QSignalSpy spy(m, SIGNAL(valueChanged(MyMoneyAccount)));
>
>
> m->addPrice(price);
>
> ft.commit();
>
>
> QCOMPARE(spy.count(), 1);
>
>
> However, I do not fully understand what this mechanism contributes in this
> case as far as the test file already checks that situation just after adding
> a price:
>
>
>   QVERIFY(m_balanceChanged.count() == 0);
>   QVERIFY(m_valueChanged.count() == 1);
>   QVERIFY(m_valueChanged.count("A000002") == 1);
>

You are right the signal spy is not needed with these checks present,
although for black-box tests it would be better to check for signal
emission than some internal data structure change.

>
> Thus, I think that what I need is a scenario where a change in the "count"
> of some of those signals arise after inserting the fault/s that I commented.
> This is not happening at the moment with the current mymoneytestfile.cpp.

Then that should be easy, when adding the incorrect code, did you
check that it is actually executed when the testcase is running, we
have two possible scenarios:
1) The incorrect code is executed by the current tests then just extra
checks should be added
2) The incorrect code is not executed by the current tests then add a
test which will trigger that code path

In both cases we can help you with this. Just let us know what you find.

Regards,
Cristian

>
> Regards.
>
>
>
>> Date: Tue, 17 Feb 2015 20:18:23 +0200
>
>> Subject: Re: [Kmymoney-devel] Improving the test suite for MyMoneyFile
>> From: onet.cristian at gmail.com
>> To: pedretti_86 at hotmail.com
>> CC: kmymoney-devel at kde.org
>>
>> To verify signal emission in a test case use QSignalSpy [1] (it's
>> pretty well documented).
>>
>> I'm on the run now but I'll follow up with a specific example and
>> ideas about the balance cache later.
>>
>> Regards,
>> Cristian
>>
>> [1] http://doc.qt.io/qt-5/QSignalSpy.html
>>
>> 2015-02-17 19:24 GMT+02:00 victor <pedretti_86 at hotmail.com>:
>> > Thanks for your answer Cristian. I understand what you mean, but I have
>> > no
>> > idea about how to check that those signals are emitted since I have
>> > never
>> > worked with this mechanism. Could you please show me a specific example?
>> >
>> > Your answer refers to m_valueChangedSet and m_balanceChangedSet. Any
>> > ideas
>> > about the faults involving "m_balanceCache"?
>> >
>> >> Date: Tue, 17 Feb 2015 16:23:42 +0200
>> >> Subject: Re: [Kmymoney-devel] Improving the test suite for MyMoneyFile
>> >> From: onet.cristian at gmail.com
>> >> To: pedretti_86 at hotmail.com
>> >> CC: kmymoney-devel at kde.org
>> >
>> >>
>> >> Hi,
>> >>
>> >> Since those two members are used to implement the emit of the
>> >> 'balanceChanged()' and 'valueChanged()' signals after committing
>> >> changes you could write a test case that asserts that those signals
>> >> are emitted properly.
>> >>
>> >> For example:
>> >> After adding a transaction 'balanceChanged()' should be emitted.
>> >> After adding a price 'valueChanged()' should be emitted.
>> >>
>> >> Regards,
>> >> Cristian
>> >>
>> >> 2015-02-17 16:13 GMT+02:00 victor <pedretti_86 at hotmail.com>:
>> >> > Hello,
>> >> >
>> >> > I am trying to improve some of the test files of KMyMoney for my
>> >> > research
>> >> > studies. This is a really complex task when you are not involved in
>> >> > programming this application, so I need your help to achieve it.
>> >> >
>> >> > All the same, I'm only focusing on some specific parts. What I'm
>> >> > doing
>> >> > is to
>> >> > introduce several concrete faults into the code and then observe if
>> >> > the
>> >> > fault is detected by the test suite (this would be the expected of an
>> >> > adequate test suite, but we all now that this is quite difficult
>> >> > sometimes).
>> >> > In this sense, I have been doing this in "mymoneyfile.cpp" and
>> >> > testing
>> >> > it
>> >> > with its corresponding file test "mymoneyfiletest.cpp". The faults
>> >> > introduced are faults which are common when programming because we
>> >> > sometimes
>> >> > mix the name of variables of the same type, or do not call the
>> >> > appropriate
>> >> > method because of "copy and paste",...
>> >> >
>> >> > Here I show two of these faults that the test suite is not able to
>> >> > detect
>> >> > (the result is the same in the original program and with the inserted
>> >> > fault):
>> >> >
>> >> > 1) The first is in the method "void
>> >> > MyMoneyFile::commitTransaction(void)",
>> >> > where I change:
>> >> >
>> >> > d->m_balanceChangedSet.remove((*it).id());
>> >> >
>> >> > by:
>> >> >
>> >> > d->m_valueChangedSet.remove((*it).id());
>> >> >
>> >> > Exactly the same happens in the method "void
>> >> > MyMoneyFile::rollbackTransaction(void)" if, for instance, I change
>> >> > "d->m_valueChangedSet.clear();" by "d->m_balanceChangedSet.clear();".
>> >> > These
>> >> > changes seem to be meaningful changes, but they are not detected by
>> >> > the
>> >> > test
>> >> > suite at the moment.
>> >> >
>> >> > 2) The other situation that I want to report is in method "void
>> >> > notify(void)" belonging to "MyMoneyFile::Private". If I change,
>> >> >
>> >> > m_balanceCache.clear(i.first, i.second);
>> >> >
>> >> > by:
>> >> >
>> >> > m_balanceCache.clear(i.first);
>> >> >
>> >> > or:
>> >> >
>> >> > m_balanceCache.clear();
>> >> >
>> >> > How could I detect that I are not calling the method "clear" in a
>> >> > correct
>> >> > way?
>> >> >
>> >> > I am aware that this is quite specific and may be difficult to create
>> >> > test
>> >> > cases for these situations, but I think that this is a very good
>> >> > chance
>> >> > to
>> >> > improve the test suite of MyMoneyFile (a very important module in the
>> >> > application).
>> >> >
>> >> > Does somebody know how to reveal the aforementioned faults modifying
>> >> > existing test cases or designing a new one?
>> >> >
>> >> > Thanks in advance.
>> >> >
>> >> > _______________________________________________
>> >> > KMyMoney-devel mailing list
>> >> > KMyMoney-devel at kde.org
>> >> > https://mail.kde.org/mailman/listinfo/kmymoney-devel
>> >> >