[Kmymoney-devel] Improving the test suite for MyMoneyFile
victor
pedretti_86 at hotmail.com
Fri Feb 27 17:47:26 UTC 2015
Ok. I have inserted: std::cout<<"Reached\n"; just before each of the lines were I introduce the faults. Note that I only insert one of these faults in the program, not several of them at the same time.
1) In the first case that I commented, I performed the following four changes:
- In mymoneyfile.cpp, line 385:
a) d->m_balanceChangedSet.remove((*it).id()); --> d->m_valueChangedSet.remove((*it).id());
The line is reached by MyMoneyFileTest::testRemoveInstitution(), MyMoneyFileTest::testRemoveAccount(), MyMoneyFileTest::testRemoveAccountTree() and MyMoneyFileTest::testRemovePayee().
- In mymoneyfile.cpp, line 409:
b) d->m_valueChangedSet.remove(id); --> d->m_balanceChangedSet.remove(id);
In 1b.txt, you can see that many of test cases in MyMoneyFileTest reach this line (Note: search for the word "Reached" in the file. It is written just before the test cases that reach this line.)
- In mymoneyfile.cpp, line 435 and 436:
c) d->m_balanceChangedSet.clear(); --> d->m_valueChangedSet.clear();
d) d->m_valueChangedSet.clear(); --> d->m_balanceChangedSet.clear();
The results are in 1cd.txt.
2) In the second case, I performed the following four changes:
- In mymoneyfile.cpp, line 177:
a) m_balanceCache.clear(i.first, i.second); --> m_balanceCache.clear(i.first);
b) m_balanceCache.clear(i.first, i.second); --> m_balanceCache.clear();
In 2ab.txt, you can see that most of test cases in MyMoneyFileTest and
some in MyMoneyForecast reach the incorrect line in the cases a y b.
- mymoneyfile.cpp, line 179:
c) m_balanceCache.clear(i.first); --> m_balanceCache.clear();
Never reached.
- mymoneyfile.cpp, line 762:
d) d->m_balanceCache.clear(acc.id()); --> d->m_balanceCache.clear();
Reached by MyMoneyFileTest::testRemoveAccount() and MyMoneyFileTest::testRemoveAccountTree()
If you need more data, let me know. Thanks!
> Date: Fri, 27 Feb 2015 17:55:12 +0200
> Subject: Re: [Kmymoney-devel] Improving the test suite for MyMoneyFile
> From: onet.cristian at gmail.com
> To: kmymoney-devel at kde.org
> CC: pedretti_86 at hotmail.com
>
> 2015-02-20 13:24 GMT+02:00 victor <pedretti_86 at hotmail.com>:
> > Hi,
> >
> > I have tried to become familiar with QSignalSpy. For instance, to comply
> > with "After adding a price 'valueChanged()' should be emitted.", I have
> > implemented this:
> >
> > qRegisterMetaType<MyMoneyAccount>("MyMoneyAccount");
> >
> > QSignalSpy spy(m, SIGNAL(valueChanged(MyMoneyAccount)));
> >
> >
> > m->addPrice(price);
> >
> > ft.commit();
> >
> >
> > QCOMPARE(spy.count(), 1);
> >
> >
> > However, I do not fully understand what this mechanism contributes in this
> > case as far as the test file already checks that situation just after adding
> > a price:
> >
> >
> > QVERIFY(m_balanceChanged.count() == 0);
> > QVERIFY(m_valueChanged.count() == 1);
> > QVERIFY(m_valueChanged.count("A000002") == 1);
> >
>
> You are right the signal spy is not needed with these checks present,
> although for black-box tests it would be better to check for signal
> emission than some internal data structure change.
>
> >
> > Thus, I think that what I need is a scenario where a change in the "count"
> > of some of those signals arise after inserting the fault/s that I commented.
> > This is not happening at the moment with the current mymoneytestfile.cpp.
>
> Then that should be easy, when adding the incorrect code, did you
> check that it is actually executed when the testcase is running, we
> have two possible scenarios:
> 1) The incorrect code is executed by the current tests then just extra
> checks should be added
> 2) The incorrect code is not executed by the current tests then add a
> test which will trigger that code path
>
> In both cases we can help you with this. Just let us know what you find.
>
> Regards,
> Cristian
>
> >
> > Regards.
> >
> >
> >
> >> Date: Tue, 17 Feb 2015 20:18:23 +0200
> >
> >> Subject: Re: [Kmymoney-devel] Improving the test suite for MyMoneyFile
> >> From: onet.cristian at gmail.com
> >> To: pedretti_86 at hotmail.com
> >> CC: kmymoney-devel at kde.org
> >>
> >> To verify signal emission in a test case use QSignalSpy [1] (it's
> >> pretty well documented).
> >>
> >> I'm on the run now but I'll follow up with a specific example and
> >> ideas about the balance cache later.
> >>
> >> Regards,
> >> Cristian
> >>
> >> [1] http://doc.qt.io/qt-5/QSignalSpy.html
> >>
> >> 2015-02-17 19:24 GMT+02:00 victor <pedretti_86 at hotmail.com>:
> >> > Thanks for your answer Cristian. I understand what you mean, but I have
> >> > no
> >> > idea about how to check that those signals are emitted since I have
> >> > never
> >> > worked with this mechanism. Could you please show me a specific example?
> >> >
> >> > Your answer refers to m_valueChangedSet and m_balanceChangedSet. Any
> >> > ideas
> >> > about the faults involving "m_balanceCache"?
> >> >
> >> >> Date: Tue, 17 Feb 2015 16:23:42 +0200
> >> >> Subject: Re: [Kmymoney-devel] Improving the test suite for MyMoneyFile
> >> >> From: onet.cristian at gmail.com
> >> >> To: pedretti_86 at hotmail.com
> >> >> CC: kmymoney-devel at kde.org
> >> >
> >> >>
> >> >> Hi,
> >> >>
> >> >> Since those two members are used to implement the emit of the
> >> >> 'balanceChanged()' and 'valueChanged()' signals after committing
> >> >> changes you could write a test case that asserts that those signals
> >> >> are emitted properly.
> >> >>
> >> >> For example:
> >> >> After adding a transaction 'balanceChanged()' should be emitted.
> >> >> After adding a price 'valueChanged()' should be emitted.
> >> >>
> >> >> Regards,
> >> >> Cristian
> >> >>
> >> >> 2015-02-17 16:13 GMT+02:00 victor <pedretti_86 at hotmail.com>:
> >> >> > Hello,
> >> >> >
> >> >> > I am trying to improve some of the test files of KMyMoney for my
> >> >> > research
> >> >> > studies. This is a really complex task when you are not involved in
> >> >> > programming this application, so I need your help to achieve it.
> >> >> >
> >> >> > All the same, I'm only focusing on some specific parts. What I'm
> >> >> > doing
> >> >> > is to
> >> >> > introduce several concrete faults into the code and then observe if
> >> >> > the
> >> >> > fault is detected by the test suite (this would be the expected of an
> >> >> > adequate test suite, but we all now that this is quite difficult
> >> >> > sometimes).
> >> >> > In this sense, I have been doing this in "mymoneyfile.cpp" and
> >> >> > testing
> >> >> > it
> >> >> > with its corresponding file test "mymoneyfiletest.cpp". The faults
> >> >> > introduced are faults which are common when programming because we
> >> >> > sometimes
> >> >> > mix the name of variables of the same type, or do not call the
> >> >> > appropriate
> >> >> > method because of "copy and paste",...
> >> >> >
> >> >> > Here I show two of these faults that the test suite is not able to
> >> >> > detect
> >> >> > (the result is the same in the original program and with the inserted
> >> >> > fault):
> >> >> >
> >> >> > 1) The first is in the method "void
> >> >> > MyMoneyFile::commitTransaction(void)",
> >> >> > where I change:
> >> >> >
> >> >> > d->m_balanceChangedSet.remove((*it).id());
> >> >> >
> >> >> > by:
> >> >> >
> >> >> > d->m_valueChangedSet.remove((*it).id());
> >> >> >
> >> >> > Exactly the same happens in the method "void
> >> >> > MyMoneyFile::rollbackTransaction(void)" if, for instance, I change
> >> >> > "d->m_valueChangedSet.clear();" by "d->m_balanceChangedSet.clear();".
> >> >> > These
> >> >> > changes seem to be meaningful changes, but they are not detected by
> >> >> > the
> >> >> > test
> >> >> > suite at the moment.
> >> >> >
> >> >> > 2) The other situation that I want to report is in method "void
> >> >> > notify(void)" belonging to "MyMoneyFile::Private". If I change,
> >> >> >
> >> >> > m_balanceCache.clear(i.first, i.second);
> >> >> >
> >> >> > by:
> >> >> >
> >> >> > m_balanceCache.clear(i.first);
> >> >> >
> >> >> > or:
> >> >> >
> >> >> > m_balanceCache.clear();
> >> >> >
> >> >> > How could I detect that I are not calling the method "clear" in a
> >> >> > correct
> >> >> > way?
> >> >> >
> >> >> > I am aware that this is quite specific and may be difficult to create
> >> >> > test
> >> >> > cases for these situations, but I think that this is a very good
> >> >> > chance
> >> >> > to
> >> >> > improve the test suite of MyMoneyFile (a very important module in the
> >> >> > application).
> >> >> >
> >> >> > Does somebody know how to reveal the aforementioned faults modifying
> >> >> > existing test cases or designing a new one?
> >> >> >
> >> >> > Thanks in advance.
> >> >> >
> >> >> > _______________________________________________
> >> >> > KMyMoney-devel mailing list
> >> >> > KMyMoney-devel at kde.org
> >> >> > https://mail.kde.org/mailman/listinfo/kmymoney-devel
> >> >> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kmymoney-devel/attachments/20150227/537f4fef/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: 2ab.txt
URL: <http://mail.kde.org/pipermail/kmymoney-devel/attachments/20150227/537f4fef/attachment-0003.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: 1b.txt
URL: <http://mail.kde.org/pipermail/kmymoney-devel/attachments/20150227/537f4fef/attachment-0004.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: 1cd.txt
URL: <http://mail.kde.org/pipermail/kmymoney-devel/attachments/20150227/537f4fef/attachment-0005.txt>
More information about the KMyMoney-devel
mailing list