Fixing tests failing on CI but not on local system

[Stefano Crocco]

On martedì 22 aprile 2025 20:57:53 Ora legale dell’Europa centrale Ben 
Cooksley wrote:
> On Wed, Apr 23, 2025 at 5:24 AM Stefano Crocco <stefano.crocco at alice.it>
> 
> wrote:
> > Hello to everyone,
> 
> Hi Stefano,

Hello Ben,
> 
> > recently, I've being trying to fix failing Konqueror tests, and in doing
> > so I'm
> > facing some issue with the CI tests which I don't know how to solve.
> > 
> > The first issue is that one test passes on my system but fails when run by
> > the
> > CI (it's the konqview test at [1]). I couldn't spot anything in the code
> > which
> > would be likely to be different on the CI than on my system. What do you
> > think
> > is the best way to try and fix this test? Should I try and add some
> > debugging
> > statement in the tests, then run it again on the CI to try to pinpoint the
> > cause of the failure? I wouldn't want to waste CI resources, but I have no
> > better idea.
> 
> This test looks like it might be a KXMLGUI initialisation failure:
> 
> QWARN  : KonqViewTest::textThenHtml() kf.xmlgui: cannot find .rc file
> "konqueror.rc" for component "konqviewtest"
> QWARN  : KonqViewTest::textThenHtml() kf.xmlgui: cannot find .rc file
> "konqueror.rc" for component "konqviewtest"
> 
> It is probably masked on your local system because there will be old files
> / configuration / etc present.
> Might be worth creating a new user account, doing a build of Konqueror
> alone there (without logging in or running Konqueror normally there) and
> seeing if the unit test fails there as that is basically what CI does.

I don't think this can be the difference because I get the same message on my 
system where the test succeeds (because the konqueror.rc file is installed in 
the konqueror directory, while the KXmlGui frameworks expects it in the 
konqviewtest directory since that's the name of the component) . Just to be 
sure, I tried following your suggestion but nothing changed and the test 
succeeded.

> > A similar issue regards tests failing on FreeBSD but not on Linux: how
> > should
> > I proceed, since that I have no FreeBSD machine available and I'm not
> > familiar
> > enough with it to try setting up a virtual machine?
> 
> In the not too distant future once we have VM based CI you'd be able to
> download the VM image the CI system uses.
> This is weeks, not months, away at this point.

This should be very useful. Thanks for the information.

> 
> > Lastly, there's a test (konqviewmgrtest at [2]) which is marked as failed
> > by
> > the CI but, looking at the output it reports, seems successful to me (I
> > can't
> > see any mention of failing tests or errors in the output). I attach a file
> > with
> > the output of the test. My only guess is that the word "failed" which
> > appears
> > in lines 3 and 28 but which is unrelated to the test itself may be
> > confusing
> > the CI. Do you think it is possible? Otherwise, do you have any idea about
> > why
> > the test could be marked as failed?
> 
> Reading the full build log reveals why that happened:
> 
> 4/12 Test #4: konqviewmgrtest ...................***Timeout 60.12 sec
> 
> My guess would be that it is popping up a message box as something didn't
> quite work out when it tries to activate the link:
> 
> QDEBUG : ViewMgrTest::testLinkedViews() ACTIVATING LINK

Thanks for pointing this out. I didn't think of looking at the full output. 
I'll try to find out could be producing a message box, but it won't be easy.

> 
> > Thanks in advance
> > 
> > Stefano
> 
> Hope that gives you some ideas!
> 
> Cheers,
> Ben

Thanks

Stefano