digiKam MSVC compilation problems on build.kde.org...

Sat Feb 15 09:47:52 GMT 2020

Ok, yesterday, i patch the opencv python scrip to revert back to 4.1.2
version to see if DK link better on core object : no more...

03:45:27  -- OpenCV ARCH: x64
03:45:27  -- OpenCV RUNTIME: vc15
03:45:27  -- OpenCV STATIC: OFF
03:45:27  -- Found OpenCV:
C:/Craft/BinaryFactory/windows-msvc2017_64-cl (found version "4.1.2")
found components: core objdetect imgproc imgcodecs dnn flann
03:45:27  -- Found OpenCV 4.1.2 in
C:/Craft/BinaryFactory/windows-msvc2017_64-cl/x64/vc15/lib
03:45:27  -- You might need to add
C:\Craft\BinaryFactory\windows-msvc2017_64-cl\x64\vc15\bin to your
PATH to be able to run your applications.
03:45:27  -- OpenCV Root directory is:
C:/Craft/BinaryFactory/windows-msvc2017_64-cl
03:45:27  -- OpenCV: Found version 4.1.2 (required: 3.3.0)
03:45:27  -- OpenCV headers:
C:/Craft/BinaryFactory/windows-msvc2017_64-cl/include
03:45:27  -- OpenCV libs   :
opencv_core;opencv_objdetect;opencv_imgproc;opencv_imgcodecs;opencv_dnn;opencv_flann

...

04:38:58  FAILED: bin/digikamcore.dll lib/digikamcore.lib
04:38:58  cmd.exe /C "cmd.exe /C
"C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
-E __create_def
C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def
C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def.objs
&& cd C:\_\1ddac771\RelWithDebInfo-master" &&
C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
-E vs_link_dll --intdir=core\app\CMakeFiles\digikamcore.dir
--rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
--mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  --
C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
/nologo @CMakeFiles\digikamcore.rsp  /out:bin\digikamcore.dll
/implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
/machine:x64 /debug /INCREMENTAL
/DEF:core\app\CMakeFiles\digikamcore.dir\exports.def   && cd ."
04:38:58  LINK Pass 1: command
"C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
/nologo @CMakeFiles\digikamcore.rsp /out:bin\digikamcore.dll
/implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
/machine:x64 /debug /INCREMENTAL
/DEF:core\app\CMakeFiles\digikamcore.dir\exports.def /MANIFEST
/MANIFESTFILE:core\app\CMakeFiles\digikamcore.dir/intermediate.manifest
core\app\CMakeFiles\digikamcore.dir/manifest.res" failed (exit code
1104) with the following output:
04:38:58  LINK : fatal error LNK1104: cannot open file
'opencv_core-NOTFOUND.obj'

So it setup back 4.2.0. The problem is in another place...

Gilles

Le jeu. 13 févr. 2020 à 12:07, Ben Cooksley <bcooksley at kde.org> a écrit :
>
> On Thu, Feb 13, 2020 at 11:45 PM Gilles Caulier
> <caulier.gilles at gmail.com> wrote:
> >
> > Right, but the Q is : why with the CI, openCV detection work as expected?
> >
> > There is a solution to drop definitively the cmake macro in DK to
> > handle openCV. With 4.x, OpenCV start to deploy the library
> > configuration through the target cmake scripts which is the modern
> > way.
> >
> > I tried with openCV 4.1 to use only the target library scripts
> > provided by OpenCV and to drop all the macro in DK, but it do not work
> > well. With 4.2, this problem is certainly solved but i need to check
> > all case (Linux, MacOS, MinGW, and now MSVC).
> >
> > The current code work everywhere, even in CI MSVC. So I ask the Q
> > again: why the binary build infrastructure has this kind of problems ?
>
> Likely because the CI system is using the older OpenCV 4.1.x series,
> as it was deployed before the update to OpenCV 4.2.
> The Binary Factory on the other hand is using the current latest state
> of Craft - which is OpenCV 4.2.
>
> Cheers,
> Ben
>
> >
> > Gilles
> >
> > Le jeu. 13 févr. 2020 à 05:11, Hannah von Reth <vonreth at kde.org> a écrit :
> > >
> > > From the logs:
> > >
> > > -- OpenCV libs   : opencv_core;opencv_objdetect;opencv_imgproc;opencv_imgcodecs;opencv_dnn;opencv_flann
> > >
> > > With modern cmake I'd expect absolute paths here, or at least the full names, -lfoo does not work on Windows.
> > > Also, have a look on how opencv is installed.
> > >
> > > 00:19:58.081  -- Installing: C:/Craft/BinaryFactory/windows-msvc2017_64-cl/build/libs/opencv/image-RelWithDebInfo-4.2.0/Craft/BinaryFactory/windows-msvc2017_64-cl/x64/vc15/lib/opencv_core420.lib
> > >
> > > Cheers,
> > >
> > > Hannah
> > >
> > > On 13.02.20 10:52, Gilles Caulier wrote:
> > >
> > > Hannah,
> > >
> > > In digiKam, there is a legacy cmake wrapper for OpenCV when we switch
> > > from 1.x to 2.x, and later 3.x. OpenCV team has badly written the
> > > cmake code to share information with client application. It's now
> > > really better with 3.x and now 4.x.
> > >
> > > In DK, code is here :
> > >
> > > https://invent.kde.org/kde/digikam/blob/master/core/CMakeLists.txt#L238
> > >
> > > https://invent.kde.org/kde/digikam/blob/master/core/cmake/modules/MacroOpenCV.cmake
> > >
> > > In the past, this wrapper was really very complicated with plenty of
> > > sub script due to the mess done by OpenCv to deploy the library
> > > configuration. Now it's really more simpler but, perhaps it still a
> > > little bug in this macro,
> > >
> > > If yes, why CI build with MSVC work as expected when ?
> > >
> > > Best
> > >
> > > Gilles
> > >
> > > Le jeu. 13 févr. 2020 à 04:00, Hannah von Reth <vonreth at kde.org> a écrit :
> > >
> > > This doesn't look like a mix of build types, anyhow we only provide
> > > release builds with symbols.
> > >
> > > the big question is why does it try to to use an obj instead of a lib?
> > >
> > > I'd check how you locate the lib and how you pass it to the target.
> > >
> > > Some find scripts fail with msvc as they expect the wrong name for a
> > > lib, msvc does not prepend everything with a "lib" by default so you
> > > might need to add libopencv_core to the possible names.
> > >
> > > I'm just guessing here but maybe it helps.
> > >
> > >
> > > Cheers,
> > >
> > > Hannah
> > >
> > > On 13.02.20 04:57, Gilles Caulier wrote:
> > >
> > > digiKAm start to compile now, but :
> > >
> > > 18:58:51  [1234/2395] Linking CXX shared library bin\digikamcore.dll
> > > 18:58:51  FAILED: bin/digikamcore.dll lib/digikamcore.lib
> > > 18:58:51  cmd.exe /C "cmd.exe /C
> > > "C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
> > > -E __create_def
> > > C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def
> > > C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def.objs
> > > && cd C:\_\1ddac771\RelWithDebInfo-master" &&
> > > C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
> > > -E vs_link_dll --intdir=core\app\CMakeFiles\digikamcore.dir
> > > --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
> > > --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  --
> > > C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
> > > /nologo @CMakeFiles\digikamcore.rsp  /out:bin\digikamcore.dll
> > > /implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
> > > /machine:x64 /debug /INCREMENTAL
> > > /DEF:core\app\CMakeFiles\digikamcore.dir\exports.def   && cd ."
> > > 18:58:51  LINK Pass 1: command
> > > "C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
> > > /nologo @CMakeFiles\digikamcore.rsp /out:bin\digikamcore.dll
> > > /implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
> > > /machine:x64 /debug /INCREMENTAL
> > > /DEF:core\app\CMakeFiles\digikamcore.dir\exports.def /MANIFEST
> > > /MANIFESTFILE:core\app\CMakeFiles\digikamcore.dir/intermediate.manifest
> > > core\app\CMakeFiles\digikamcore.dir/manifest.res" failed (exit code
> > > 1104) with the following output:
> > > 18:58:51  LINK : fatal error LNK1104: cannot open file
> > > 'opencv_core-NOTFOUND.obj'
> > >
> > > This is typically a problem to handle debug version of shared library.
> > > non debug is found but not debug version.
> > >
> > > It due to a mix of build package type. MSVC require a complete package
> > > build with the same type. I can reproduce this kind of dysfunction in
> > > my office when i build my working codes...
> > >
> > > Best
> > >
> > > Gilles
> > >
> > > Le mer. 12 févr. 2020 à 15:04, Gilles Caulier
> > > <caulier.gilles at gmail.com> a écrit :
> > >
> > > I pushed this  commit :
> > >
> > > https://commits.kde.org/craft-blueprints-kde/51f8cd97d1346364303bd79eaede7ff9b16d7e0e
> > >
> > > We will see if build is better tomorrow morning
> > >
> > > Gilles
> > >
> > > Le mer. 12 févr. 2020 à 07:47, Gilles Caulier
> > > <caulier.gilles at gmail.com> a écrit :
> > >
> > > Le mer. 12 févr. 2020 à 04:11, Ben Cooksley <bcooksley at kde.org> a écrit :
> > >
> > > On Sun, Feb 9, 2020 at 10:22 PM Gilles Caulier <caulier.gilles at gmail.com> wrote:
> > >
> > > Hi Ben,
> > >
> > > Hi Gilles,
> > >
> > > By experience as we use openCV in digiKAm since more than 10 years
> > > now, Using the LTS version is always a good deal to prevent problem.
> > >
> > > openCV team always touch a lot of code (too much) when a new devel
> > > branch is started.
> > >
> > > For all bundles that we manage, OpenCV 3.4.x is used for production.
> > > It well maintained and changes are smaller in time.
> > >
> > > https://invent.kde.org/kde/digikam/blob/master/project/bundles/3rdparty/ext_opencv/CMakeLists.txt#L99
> > >
> > > OpenCV dependency is very tedious. Updating this library is always a
> > > source of dysfunctions. Always. So upgrading must be checked safety in
> > > all conditions.
> > >
> > > My 10cts€ tips (:=)))...
> > >
> > > Q: It's possible to customize OpenCV dependency only for digiKam or
> > > it's too much complicated ? I suppose that all these king of shared
> > > libraries installed on CI are common for all projects...
> > >
> > > Please note that CI is very different to the Binary Factory.
> > > The CI system does not update system level dependencies without manual
> > > action being taken by a Sysadmin. These builds are currently operating
> > > fine.
> > >
> > > The Binary Factory is driven by the Craft Blueprints, and will update
> > > as those are updated.
> > > I can confirm however that a given package (such as OpenCV) can only
> > > be in the Craft Blueprints once, and the default version is specified
> > > in that same Blueprint.
> > >
> > > I have checked and it appears that frei0r
> > >
> > > frei0r python script as the opencv dependency comments actually...
> > >
> > > and Digikam are the only
> > > ones that make use of OpenCV, so the version in use can likely be
> > > tweaked to keep them both happy.
> > > As Hannah updated it you will need to check with her why it was
> > > updated (there may very well be good reason to do so)
> > >
> > > The current OpenCV version in Craft is 4.2.0, prior to the update it
> > > was 4.1.2 (which built fine) - I suspect the failure may have more to
> > > do with the changes to make it use a system wide Protobuf than the
> > > updates in OpenCV itself however, as the missing symbols belong to
> > > Protobuf.
> > >
> > > So, we can setup Opencv to compile internal protobuf instead. There is
> > > a cmake configuration option for that:
> > >
> > > -DBUILD_PROTOBUF=OFF
> > >
> > > ...to set here :
> > >
> > > https://github.com/KDE/craft-blueprints-kde/blob/master/libs/opencv/opencv.py#L18
> > >
> > > Gilles