Need help in getting started with writing the integration test

Sun Jun 3 19:39:31 UTC 2018

Hello Dileep

Am 03.06.2018 08:05 schrieb Dileep Sankhla:
> Hello everyone,

@everyone: Please correct me if I'm wrong or if you have other/better 
ideas.

> I have created a revision[0] for adding the typewriter annotation tool
> to Okular and now I need to write the integration test for the same.
> 
> I'm not getting whether to make a new file
> autotests/typewriterannotationtest.cpp or to append the test in the
> existing autotests/annotationstest.cpp?

I don't think there was an initial master plan for where to implement 
which test. Looks like tests just evolved since 2009. You can walk 
through existing tests and see if there's a good match. I've appended an 
overview of existing tests at the end of the mail. To know where you fit 
best, your next question has to be answered (eventually by yourself).

> And the important thing, what do I need to test in it?

You can break down your feature into isolated functional requirements 
and allocate each functionality to an okular component. Then think of a 
test case how you would prove the specific functionality is actually 
working. You implemented the feature, so I bet you know best what it is 
expected to do in detail. Here are some suggestions from my side:

Functionality in GUI, okularpart.so: The initial toolbar from tools.xml 
contains a button with the T icon for typewriter annotation.
Test case: Load okularpart and check if the QToolBar contains the 
expected ToolBarButton.

Functionality in GUI, okularpart.so: If the typewriter button is 
clicked, a PickPointEngine is started and opens a QInputDialog, waiting 
for user input.
Test case: Click button, and see if PickPointEngine and QInputDialog is 
started.

Functionality in GUI, okularpart.so: If typewriter PickPointEngine is 
active, a left click into the viewport opens a popup window.
Test case: Start PickPointEngine engine, click into viewport and check 
if popup window opens.

Functionality in GUI, okularpart.so: If typewriter was added, the 
annotation shows up in PageView as expected. This shall work if 
typewriter is generator rendered as well as if it's rendered by 
PagePainter.
Test case: Add annotation to a .pdf and a .txt document. Check for the 
expected pixels in PageView widget at correct position (no border, 
transparent background, ...)

Functionality in core libOkular5Core.so: If Okular::TextAnnotation is 
created and setTextType(Okular::TextAnnotation::InPlace) and 
setInplaceIntent(TypeWriter), metadata is generated accordingly.
Test case: Add typewriter to a Okular::Document, and check 
Okular::Annotation::getAnnotationPropertiesDomNode for expected 
attributes.

Functionality in generator okularGenerator_poppler.so: 
PDFGenerator::image provides a QImage that contains typewriter 
annotations.
Test: Add annotation, render image via generator, check the image for 
the expected pixels.

Functionality in generator okularGenerator_poppler.so: 
PDFGenerator::save saves an ISO 32k conforming typewriter into the PDF.
Test case: Load document, add typewriter annotation, save it and check 
the resulting document for object /Type Annot, /Subtype FreeText, /It 
FreeTextTypeWriter etc.

And so on, this was just from top of my head. You easily find dozens of 
them. Because you have limited time, you select those that you find 
important and less likely to change within the next few releases.

Existing autotests target sub-groups of functionality, but they also 
target specific public APIs. Targeted API is also a good criterion to 
figure out to to which autotests/*.cpp you add a specific test (or if to 
create a new file). Choices are:
- Core (libOkular5Core.so): Implemented as ordinary shared lib. Can be 
used by external applications standalone, so it can also be tested 
standalone.
- Desktop UI (okularpart.so): Implemented as KPart. Includes PageView. 
Used by /usr/bin/okular, but it can also be used by other applications 
(e.g. Konqueror). So it can be used by tests.
- Generator (okularGenerator_*.so), implemented as KPlugins. Can be 
loaded via KPluginFactory, so it can be loaded by a test.
- Mobile UI (libokularplugin.so). No specific typewriter functionality.
- Shell (/usr/bin/okular). Parses cmd args and loads okularpart. No 
specific typewriter functionality.
- Each and every single class on its own. If you do this, it's a unit 
test, not integration test.

> Do I need to write a test to create a typewriter annotation
> with some content and then delete it?

This is impossible to answer unless you present well differentiated test 
cases. Here are some common ways how a QTest interacts with a component 
under test.

Always use QVERIFY or QCOMPARE to test for expected outcomes.

The QTest framework provides helpers to interact with your Qt 
applicaiton under test. Some arbitrary, unsorted examples:
QTestEventList::addMouseMove, QTestEventList::addMousePress, 
QTestEventList::simulate, QTest::keyClick, QTest::keyEvent, 
QTest::keyPress, QTest::mouseMove, QTest::mouseClick, 
QTest::qWaitForWindowExposed

Of course there are also the native ways of interaction:
Instantiate a class you want to test, and call methods on it.
Or connect to QObjects you want to and send/receive signals.
Instantiate a parent class/plugin and find a nested object (implicitly 
instantiated) of interest. Interact as above. QObject provides 
introspection features to find QObjects of interest during lifetime, 
like QObject::findChild.

> What will I compare here?

GUI tests compare with autotests/parttest.cpp.
Core tests potentially compare with autotests/*annot*.cpp
Afaikt there are no tests yet that check painting results pixel wise.

> I'm sorry but I'm unable to start before getting these basics right. I
> need a little help in getting started.

Yes, writing tests is as challenging as doing the implementation...

> Thanks and Regards,
> Dileep
> 
> [0] https://phabricator.kde.org/D13203 [1]
> 
> Links:
> ------
> [1] https://phabricator.kde.org/D13203

Regards
Tobias

addremoveannotationtest.cpp
API: core
testAddAnnotations
testAddAnnotationUndoWithRotate_Bug318091
testRemoveAnnotations

annotationstest.cpp
API: core
testDistance

calculatetexttest.cpp
API: core
testSimpleCalculate

documenttest.cpp
API: core
testCloseDuringRotationJob
testDocdataMigration

editannotationcontentstest.cpp
API: core
testConsecutiveCharBackspacesMerged
testConsecutiveNewlineBackspacesNotMerged
testConsecutiveCharInsertionsMerged
testConsecutiveNewlineInsertionsNotMerged
testConsecutiveCharDeletesMerged
testConsecutiveNewlineDeletesNotMerged
testConsecutiveEditsNotMergedAcrossDifferentAnnotations
testInsertWithSelection
testCombinations

editformstest.cpp
API: core
testRadioButtonForm
testCheckBoxForm
testTextLineForm
testTextAreaForm
testFileEditForm
testComboEditForm
testListSingleEdit
testListMultiEdit

generatorstest.cpp
API: generator
testLoadsCorrectly

mainshelltest.cpp
API: shell, part, core
testShell
testFileRemembersPagePosition
test2FilesError
testSessionRestore

modifyannotationpropertiestest.cpp
API: core
testModifyAnnotationProperties
testModifyDefaultAnnotationProperties
testModifyAnnotationPropertiesWithRotation_Bug318828

parttest.cpp
API: okularpart
testReload
testCanceledReload
testTOCReload
testForwardPDF
testGeneratorPreferences
testSelectText
testClickInternalLink
testOpenUrlArguments
test388288
testSaveAs
testSaveAsUndoStackAnnotations
testSaveAsUndoStackForms
testMouseMoveOverLinkWhileInSelectionMode
testClickUrlLinkWhileInSelectionMode
testeTextSelectionOverAndAcrossLinks
testClickUrlLinkWhileLinkTextIsSelected
testRClickWhileLinkTextIsSelected
testRClickOverLinkWhileLinkTextIsSelected
testRClickOnSelectionModeShoulShowFollowTheLinkMenu
testClickAnywhereAfterSelectionShouldUnselect
testeRectSelectionStartingOnLinks
testCheckBoxReadOnly
testCrashTextEditDestroy
testAnnotWindow

searchtest.cpp
API: core
testNextAndPrevious
test311232
test323262
test323263
testDottedI
testHyphenAtEndOfLineWithoutYOverlap
testHyphenWithYOverlap
testHyphenAtEndOfPage
testOneColumn
testTwoColumns

shelltest.cpp
API: shell
testUrlArgs

translateannotationtest.cpp
API: core
testTranslateAnnotation
testSequentialTranslationsMergedIfBeingMovedIsSet
testSequentialTranslationsNotMergedIfBeingMovedIsNotSet
testAlternateTranslationsNotMerged

urldetecttest.cpp
API: url_utils.h (looks like unit test)
testURL

visibilitytest.cpp
testJavaScriptVisibility
testSaveLoad
testActionVisibility