digiKam MSVC compilation problems on build.kde.org...

Thu Feb 13 10:44:52 GMT 2020

Right, but the Q is : why with the CI, openCV detection work as expected?

There is a solution to drop definitively the cmake macro in DK to
handle openCV. With 4.x, OpenCV start to deploy the library
configuration through the target cmake scripts which is the modern
way.

I tried with openCV 4.1 to use only the target library scripts
provided by OpenCV and to drop all the macro in DK, but it do not work
well. With 4.2, this problem is certainly solved but i need to check
all case (Linux, MacOS, MinGW, and now MSVC).

The current code work everywhere, even in CI MSVC. So I ask the Q
again: why the binary build infrastructure has this kind of problems ?

Gilles

Le jeu. 13 févr. 2020 à 05:11, Hannah von Reth <vonreth at kde.org> a écrit :
>
> From the logs:
>
> -- OpenCV libs   : opencv_core;opencv_objdetect;opencv_imgproc;opencv_imgcodecs;opencv_dnn;opencv_flann
>
> With modern cmake I'd expect absolute paths here, or at least the full names, -lfoo does not work on Windows.
> Also, have a look on how opencv is installed.
>
> 00:19:58.081  -- Installing: C:/Craft/BinaryFactory/windows-msvc2017_64-cl/build/libs/opencv/image-RelWithDebInfo-4.2.0/Craft/BinaryFactory/windows-msvc2017_64-cl/x64/vc15/lib/opencv_core420.lib
>
> Cheers,
>
> Hannah
>
> On 13.02.20 10:52, Gilles Caulier wrote:
>
> Hannah,
>
> In digiKam, there is a legacy cmake wrapper for OpenCV when we switch
> from 1.x to 2.x, and later 3.x. OpenCV team has badly written the
> cmake code to share information with client application. It's now
> really better with 3.x and now 4.x.
>
> In DK, code is here :
>
> https://invent.kde.org/kde/digikam/blob/master/core/CMakeLists.txt#L238
>
> https://invent.kde.org/kde/digikam/blob/master/core/cmake/modules/MacroOpenCV.cmake
>
> In the past, this wrapper was really very complicated with plenty of
> sub script due to the mess done by OpenCv to deploy the library
> configuration. Now it's really more simpler but, perhaps it still a
> little bug in this macro,
>
> If yes, why CI build with MSVC work as expected when ?
>
> Best
>
> Gilles
>
> Le jeu. 13 févr. 2020 à 04:00, Hannah von Reth <vonreth at kde.org> a écrit :
>
> This doesn't look like a mix of build types, anyhow we only provide
> release builds with symbols.
>
> the big question is why does it try to to use an obj instead of a lib?
>
> I'd check how you locate the lib and how you pass it to the target.
>
> Some find scripts fail with msvc as they expect the wrong name for a
> lib, msvc does not prepend everything with a "lib" by default so you
> might need to add libopencv_core to the possible names.
>
> I'm just guessing here but maybe it helps.
>
>
> Cheers,
>
> Hannah
>
> On 13.02.20 04:57, Gilles Caulier wrote:
>
> digiKAm start to compile now, but :
>
> 18:58:51  [1234/2395] Linking CXX shared library bin\digikamcore.dll
> 18:58:51  FAILED: bin/digikamcore.dll lib/digikamcore.lib
> 18:58:51  cmd.exe /C "cmd.exe /C
> "C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
> -E __create_def
> C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def
> C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def.objs
> && cd C:\_\1ddac771\RelWithDebInfo-master" &&
> C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
> -E vs_link_dll --intdir=core\app\CMakeFiles\digikamcore.dir
> --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
> --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  --
> C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
> /nologo @CMakeFiles\digikamcore.rsp  /out:bin\digikamcore.dll
> /implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
> /machine:x64 /debug /INCREMENTAL
> /DEF:core\app\CMakeFiles\digikamcore.dir\exports.def   && cd ."
> 18:58:51  LINK Pass 1: command
> "C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
> /nologo @CMakeFiles\digikamcore.rsp /out:bin\digikamcore.dll
> /implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
> /machine:x64 /debug /INCREMENTAL
> /DEF:core\app\CMakeFiles\digikamcore.dir\exports.def /MANIFEST
> /MANIFESTFILE:core\app\CMakeFiles\digikamcore.dir/intermediate.manifest
> core\app\CMakeFiles\digikamcore.dir/manifest.res" failed (exit code
> 1104) with the following output:
> 18:58:51  LINK : fatal error LNK1104: cannot open file
> 'opencv_core-NOTFOUND.obj'
>
> This is typically a problem to handle debug version of shared library.
> non debug is found but not debug version.
>
> It due to a mix of build package type. MSVC require a complete package
> build with the same type. I can reproduce this kind of dysfunction in
> my office when i build my working codes...
>
> Best
>
> Gilles
>
> Le mer. 12 févr. 2020 à 15:04, Gilles Caulier
> <caulier.gilles at gmail.com> a écrit :
>
> I pushed this  commit :
>
> https://commits.kde.org/craft-blueprints-kde/51f8cd97d1346364303bd79eaede7ff9b16d7e0e
>
> We will see if build is better tomorrow morning
>
> Gilles
>
> Le mer. 12 févr. 2020 à 07:47, Gilles Caulier
> <caulier.gilles at gmail.com> a écrit :
>
> Le mer. 12 févr. 2020 à 04:11, Ben Cooksley <bcooksley at kde.org> a écrit :
>
> On Sun, Feb 9, 2020 at 10:22 PM Gilles Caulier <caulier.gilles at gmail.com> wrote:
>
> Hi Ben,
>
> Hi Gilles,
>
> By experience as we use openCV in digiKAm since more than 10 years
> now, Using the LTS version is always a good deal to prevent problem.
>
> openCV team always touch a lot of code (too much) when a new devel
> branch is started.
>
> For all bundles that we manage, OpenCV 3.4.x is used for production.
> It well maintained and changes are smaller in time.
>
> https://invent.kde.org/kde/digikam/blob/master/project/bundles/3rdparty/ext_opencv/CMakeLists.txt#L99
>
> OpenCV dependency is very tedious. Updating this library is always a
> source of dysfunctions. Always. So upgrading must be checked safety in
> all conditions.
>
> My 10cts€ tips (:=)))...
>
> Q: It's possible to customize OpenCV dependency only for digiKam or
> it's too much complicated ? I suppose that all these king of shared
> libraries installed on CI are common for all projects...
>
> Please note that CI is very different to the Binary Factory.
> The CI system does not update system level dependencies without manual
> action being taken by a Sysadmin. These builds are currently operating
> fine.
>
> The Binary Factory is driven by the Craft Blueprints, and will update
> as those are updated.
> I can confirm however that a given package (such as OpenCV) can only
> be in the Craft Blueprints once, and the default version is specified
> in that same Blueprint.
>
> I have checked and it appears that frei0r
>
> frei0r python script as the opencv dependency comments actually...
>
> and Digikam are the only
> ones that make use of OpenCV, so the version in use can likely be
> tweaked to keep them both happy.
> As Hannah updated it you will need to check with her why it was
> updated (there may very well be good reason to do so)
>
> The current OpenCV version in Craft is 4.2.0, prior to the update it
> was 4.1.2 (which built fine) - I suspect the failure may have more to
> do with the changes to make it use a system wide Protobuf than the
> updates in OpenCV itself however, as the missing symbols belong to
> Protobuf.
>
> So, we can setup Opencv to compile internal protobuf instead. There is
> a cmake configuration option for that:
>
> -DBUILD_PROTOBUF=OFF
>
> ...to set here :
>
> https://github.com/KDE/craft-blueprints-kde/blob/master/libs/opencv/opencv.py#L18
>
> Gilles