D27678: Restructure tests, add simulator tests, spot expected failures.

[Eric Dejouhanet]

TallFurryMan added a comment.


  Please don't test this on your own setup, because it may change your userdb (it doesn't right now, but will certainly).
  I need a way to use another database, so that options are all default.

INLINE COMMENTS

> kstars_ui_tests.cpp:76
> +    // Initialize our location
> +    GeoLocation * const g = KStars::Instance()->data()->locationNamed("Greenwich");
> +    QVERIFY(g != nullptr);

This was cheating: I used the city database, while I should have used arbitrary latitude and longitude, eventually verifying this was Greenwhich.

> kstars_ui_tests.cpp:96
> +
> +    QEXPECT_FAIL("", "Initial KStars clock is set from system local time, not geolocation, and is untestable for now.", Continue);
> +    QCOMPARE(KStars::Instance()->data()->clock()->utc().toString(), m_InitialConditions.dateTime.toString());

There's apparently no way to *not* use the system properties when initializing KStars, due to KStarsDateTime management of UTC/localtime. This should be tested in a unitary test elsewhere.

> kstars_ui_tests.cpp:99
> +
> +    QEXPECT_FAIL("", "Precision of KStars local time conversion to local time does not allow strict millisecond comparison.", Continue);
> +    QCOMPARE(KStars::Instance()->data()->clock()->utc().toLocalTime(), m_InitialConditions.dateTime);

I always end up with a comparison between xxx.999ms and xxx.000ms. Rounding issues somewhere on the QDateTime/KStarsDataTime chain.

> kstars_ui_tests.cpp:127
> +    QDir writableDir;
> +    writableDir.mkdir(KSPaths::writableLocation(QStandardPaths::GenericDataLocation));
> +    KCrash::initialize();

This is currently useless.

> kstars_ui_tests.cpp:157
> +        }
> +#endif
> +

I had to use this method in order to use multiple test classes that execute in parallel with the GUI.

> kstars_ui_tests.cpp:165
> +    // Limit execution duration
> +    QTimer::singleShot(5*60*1000, &app, &QCoreApplication::quit);
> +

This value hard-codes a limit to the duration of the test. I'll probably remove it and rely on another method to detect the end of the asynchronous tests. Normally QTest::qExec calls should not break out of the singleShot() call above, so this is more a safeguard against lock-up. But that safeguard would be better done with a timeout shell wrapper.

> test_ekos.cpp:67
> +{
> +    /* No-op, we just use init+cleanup */
> +}

This one might seem unrelevant, but it actually reveals that Ekos is resetting and resuming the Simulation Clock of KStars. It results in a relatively long update of all sky objects of KStars. As I have downloaded many Minor Planet elements, this can take around 30 seconds on my Core i7, which is far from negligible.

This has the side effect of breaking all sky object coordinates fixtures, which are computed from initial conditions below in TestEkosSimulator.

> test_ekos_simulator.cpp:63
> +    GeoLocation * const geo = KStarsData::Instance()->geo();
> +    KStarsDateTime const now(KStarsData::Instance()->lt());
> +    KSNumbers const numbers(now.djd());

I would expect this to use the initial conditions in KStarsUiTest, but actually Ekos resets the Simulation Clock.
What is weird is that this test fixture generation is executed *after* the first Ekos startup, so lt() should provide the updated local time.
Still, many sky objects are placed incorrectly in the sky and end below the horizon, even though altitude is tested.
So, this must be bugged. But the test manages that with an expected failure.

> test_ekos_simulator.cpp:111
> +    QTest::qWait(5000);
> +    KTRY_EKOS_CLICK("mountToolBoxB");
> +#else

First, I tried to use the UI to slew to specific RA/DEC.
It appears the Mount Control box is the only one to provide a way to enter numeric values.
However, QML just doesn't provide you a gadget if that gadget miss the "objectName" property.

Also that reveals that apart from the Mount Control box, only the Scheduler is able to slew with numerical values.
The other way is to right-click on the sky map, which is not very numerical.

> test_ekos_simulator.cpp:128
> +        QEXPECT_FAIL(NAME.toStdString().c_str(), QString("Slew target '%1' is expected to be over the horizon during night time.").arg(NAME).toStdString().c_str(), Abort);
> +    QVERIFY(slew_result);
> +#endif

This block is a contrived way to bypass the notification box produced by the telescope handler...

> test_ekos_simulator.cpp:137
> +    QVERIFY(raOut != nullptr);
> +    QTRY_VERIFY_WITH_TIMEOUT(abs(clampRA(RA) - clampRA(raOut->text())) < 2, 15000);
> +    QTest::qWait(100);

I had a certain number of rounding issues with the Simulator telescope, and ended up testing closeness instead of equality.
I'm not sure this is expected from the Simulator Telescope, but I know this happens with real mounts, so...

REPOSITORY
  R321 KStars

REVISION DETAIL
  https://phabricator.kde.org/D27678

To: TallFurryMan, #kstars, mutlaqja
Cc: kde-edu, narvaez, apol
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-edu/attachments/20200226/cdfbac7c/attachment-0001.html>