Debugging CI test failures

[Ben Cooksley]

On Sat, Jun 18, 2022 at 11:07 PM Thomas Friedrichsmeier <
thomas.friedrichsmeier at kdemail.net> wrote:

> On Sat, 18 Jun 2022 21:50:33 +1200
> Ben Cooksley <bcooksley at kde.org> wrote:
>
> > On Fri, Jun 17, 2022 at 8:04 AM Thomas Friedrichsmeier <
> > thomas.friedrichsmeier at kdemail.net> wrote:
> >
> > > On Thu, 16 Jun 2022 17:31:16 +0200
> > > Thomas Friedrichsmeier <thomas.friedrichsmeier at kdemail.net> wrote:
> > >
> > > > On Thu, 16 Jun 2022 22:26:31 +1200
> > > > Ben Cooksley <bcooksley at kde.org> wrote:
> > > >
> > > > > On Thu, Jun 16, 2022 at 8:08 AM Thomas Friedrichsmeier <
> > > > > thomas.friedrichsmeier at kdemail.net> wrote:
> > > > >
> > > > > > On Wed, 15 Jun 2022 17:51:52 +0200
> > > > > > Thomas Friedrichsmeier <thomas.friedrichsmeier at kdemail.net>
> > > > > > wrote: [...]
> > > > > > > I suppose, I could try seting up some dummy test binaries,
> > > > > > > linking against the various libraries, hoping to narrow down
> > > > > > > the problem that way?
> > > > > >
> > > > > > Ok, that finally led to something. I have no idea, why this is
> > > > > > happening, or how to fix it, but I have isolated QWebEngine
> > > > > > as the troublemaker.
> > > > > >
> > > > > > relevant portion of CMakeLists.txt:
> > > > > >
> > > > > > add_executable(linktest linktest.cpp)
> > > > > > target_link_libraries(linktest PRIVATE Qt5::Test
> > > > > > Qt5::WebEngineWidgets) add_test(NAME rkward-linktest COMMAND
> > > > > > linktest) ecm_mark_as_test(linktest)
> > > > > >
> > > > > > relevant code:
> > > > > >
> > > > > >
> > >
> https://invent.kde.org/education/rkward/-/blob/work/diagnose_testfailure/rkward/autotests/linktest.cpp
> > >
> > > > > >
> > > > > > result:
> > > > > > https://invent.kde.org/education/rkward/-/jobs/359945
> > > > > >
> > > > > > (2/2 Test #2: rkward-linktest ..................Exit code
> > > > > > 0xc0000135)
> > > > > >
> > > > > > Note that "new QWebEngineView();" does not even have to be
> > > > > > called. It's enough that the line of code is present,
> > > > > > directly or indirectly. In contrast - for instance - creating
> > > > > > a KTextEditor::Document/View does not cause any trouble.
> > > > > >
> > > > >
> > > > > Interesting. When you build RKward on your local system do you
> > > > > use the Craft cache or does your system build everything itself?
> > > > > I wonder if the Craft binaries for WebEngineWidgets are broken
> > > > > - and MSVC doesn't check down the full chain when it does it's
> > > > > linking.
> > > >
> > > > Well, my local craft builds are MinGW-based, and so are using
> > > > QWebKit, instead. So that does not really compare. No problem,
> > > > there, however, despite using the cache.
> > > >
> > > > I did just check that the latest MSVC build from binary-factory is
> > > > working. Of course that doesn't contain the test, but rkward
> > > > itself works fine.
> > > >
> > > > I'm currently trying to get my MSVC craft root back to life, but
> > > > that seems to take another while...
> > >
> > > Ok, so no problem (well not _that_ problem, anyway) on craft MSVC
> > > 2019 with Craft cache. That's a Windows 8.1 VM by the way.
> > >
> >
> > Strange. Just curious - do we see any additional library dependencies
> > in WebEngineWidgets that aren't present in the other Qt libraries?
>
> There may be more elegant ways to find this info (I'm definitely not up
> to speed on Windows tools), but: I started rkward, listed the loaded
> dlls using "tasklist /m", then did the same for kate, and compared the
> lists. Most, but not all, differences should be related to QWebEngine
> ("+" means only in rkward.exe, "-" is only in kate.exe). Surprisingly,
> this also includes some differences in filename case, for whatever
> reason.
>
> + AVRT.dll
> + cfgmgr32.dll
> - CFGMGR32.dll
> + d3d10warp.dll
> + d3dcompiler_47.dll
> + dbghelp.dll
> + DCIMAN32.dll
> + dcomp.dll
> + DDRAW.dll
> + dhcpcsvc6.DLL
> + dhcpcsvc.DLL
> + DWrite.dll
> - dwrite.dll
> + dxva2.dll
> - externaltoolsplugin.dll
> + GLU32.dll
> + HID.DLL
> + iertutil.dll
> - katefiletreeplugin.dll
> + katesnippetsplugin.dll
> + KF5Bookmarks.dll
> + KF5KIOFileWidgets.dll
> - KUserFeedbackCore.dll
> - KUserFeedbackWidgets.dll
> + libEGL.DLL
> + libGLESv2.dll
> + MFCaptureEngine.dll
> + mf.dll
> + mfh264enc.dll
> + mfplat.dll
> + mfreadwrite.dll
> + MMDevApi.dll
> + mscms.dll
> + msmpeg2vdec.dll
> + msvproc.dll
> + ncrypt.dll
> + NLAapi.dll
> + NTASN1.dll
> + ntmarta.dll
> + opengl32.dll
> + Qt5Positioning.dll
> + Qt5QuickWidgets.dll
> + Qt5WebChannel.dll
> + Qt5WebEngineCore.dll
> + Qt5WebEngineWidgets.dll
> + RTWorkQ.DLL
> - tabswitcherplugin.dll
> + urlmon.dll
> + USP10.dll
> + WINHTTP.dll
> + WININET.dll
> + WINSTA.dll
>

Ah bingo! (said like https://youtu.be/QMiubC6LdTA?t=1574)

Looks like DirectX and it's corresponding components are missing:

PS C:\> Get-ChildItem -Path C:\ -Recurse -ErrorAction SilentlyContinue
-Force -Filter ddraw.dll
PS C:\> Get-ChildItem -Path C:\ -Recurse -ErrorAction SilentlyContinue
-Force -Filter glu32.dll
PS C:\> Get-ChildItem -Path C:\ -Recurse -ErrorAction SilentlyContinue
-Force -Filter MFCaptureEngine.dll

Looks like to make QtWebEngine work we would need to make some changes to
our Windows image:
-
https://social.msdn.microsoft.com/Forums/en-US/b646b841-c9fb-4f39-9662-5b59f02279ab/installing-servermediafoundation-in-a-docker-container?forum=windowscontainers
-
https://social.msdn.microsoft.com/Forums/lync/en-US/386adbc4-3e43-4896-8cbb-ba9cc7fc6b72/how-to-install-directx-to-windows-server-core-docker-container?forum=windowscontainers

Cheers,
Ben


>
> > Reason I ask is because the Gitlab CI builds are using Windows Server
> > Core 2022 (LTSC) which is both stripped down compared to a normal
> > Windows desktop system (Server Core is missing most of the UI stuff),
> > and is based on the same foundations as Windows 11.  The Jenkins
> > based Binary Factory however uses full traditional Windows 10 Desktop
> > for its builds.
> >
> > Our Server Core 2022 (LTSC) images do contain .Net 4.8 (including
> > it's SDK) though which I thought would have given us sufficient
> > coverage but perhaps there is some part of Windows missing that
> > WebEngine relies on that is missing here.
>
> Just a thought: Many of the internet search hits on 0xc0000135 are about
> an _older_ version of .NET no longer being enabled, after an update.
> (While, AFAIU, it is still installed, which might account for the
> difference between linking and loading.) Could something along these
> lines be the problem?
>

Potentially, but based on the above it doesn't look like .Net is used by
any parts of Qt (which makes sense, as Qt would prefer the general Win32
APIs to get what it needs to do done)


>
> Regards
> Thomas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-devel/attachments/20220619/ed291cfb/attachment-0001.htm>