MimeType issues on CI

[Daniel Vrátil]

On 2016-04-03 15:28, Scarlett Clark wrote:
> On Sun, Apr 3, 2016 at 5:16 AM, Daniel Vrátil <dvratil at kde.org>
> wrote:
> 
>> On Sunday, April 3, 2016 12:58:24 PM CEST David Faure wrote:
>>> On Sunday 03 April 2016 12:43:27 Daniel Vrátil wrote:
>>>> Turns out when you include /usr/share in XDG_DATA_DIRS, all mime
>> types
>>>> that
>>>> might have been provided by other dependencies in
>>>> /srv/jenkins/.../foo/inst/ usr/share/mime are ignored, possibly
>> due to
>>>> update-mime-database not being ran for those directories.
>>> 
>>> I'm not sure I fully understand this sentence. Do you mean the
>> value of
>>> XDG_DATA_DIRS at update-mime-database time or during the running
>> of unit
>>> tests?
>> 
>> XDG_DATA_DIRS value when running the unit tests.
>> 
>>> update-mime-database works on one specific install dir and creates
>> files
>>> there. The value of XDG_DATA_DIRS during the run of
>> update-mime-database
>>> shouldn't matter.
>>> 
>>> If the stuff /srv/jenkins/.../foo/inst/usr/share/mime is ignored,
>> it can be
>>> either
>> 
>>> 1) because update-mime-database wasn't run on that dir
>>> (which is my current hypothesis for the KIO kfileitemtest
>> failure which
>>> needs a mimetype from kcoreaddons)
>> 
>> The thing is that the content of
>> /srv/jenkins/.../foo/inst/usr/share/mime is
>> NOT ignored, when /usr/share is NOT in XDG_DATA_DIRS.
>> 
>> Once you include /usr/share in XDG_DATA_DIRS, the content of the
>> /srv/.../mime
>> folders IS ignored.
>> 
>> Akonadi installs a mime file to share/mime/packages/akonadi-mime.xml
>> which
>> introduces some new mimetypes and are used to find the right
>> serialize plugins
>> for those types. With /usr/share in XDG_DATA_DIRS Akonadi is not
>> able to find
>> serializer plugins for application/x-vnd.akonadi.calendar.todo for
>> instance,
>> because it (using QMimeDatabase) does not know about this mime type.
>> Removing
>> /usr/share from XDG_DATA_DIRS makes Akonadi (QMimeDatabase) find
>> application/
>> x-vnd.akonadi.calendar.todo just find and thus find a respective
>> serializer
>> plugin for the type.
>> 
>>> 2) because that dir isn't in
>>> XDG_DATA_DIRS (this is not the problem) later on.
>> 
>> I verified that it is there.
>> 
>>> 3) because
>>> update-mime-database ran but didn't create the stuff it's supposed
>> to
>>> create -- I understand your email as saying this might be the
>> problem, but
>>> I can't confirm it:
>> 
>> If I compare output of "make install" from CI and running it
>> locally, it
>> indeed appears that update-mime-database is not run on the CI at
>> all:
>> 
>> CI:
>> 16:22:17 Install the project...
>> 16:22:17 -- Install configuration: "Debug"
>> 16:22:17 -- Installing:
>> /home/jenkins/sources/akonadi/kf5-qt5/local-inst/srv/
>> 
> jenkins/install/ubuntu/x86_64/g++/kf5-qt5/kde/pim/akonadi/inst/usr/share/mime/
>> packages/akonadi-mime.xml
>> 16:22:17 -- Installing:
>> /home/jenkins/sources/akonadi/kf5-qt5/local-inst/srv/
>> 
> jenkins/install/ubuntu/x86_64/g++/kf5-qt5/kde/pim/akonadi/inst/usr/lib/x86_64-
>> linux-gnu/cmake/KF5Akonadi/KF5AkonadiConfig.cmake
>> 
>> Locally:
>> -- Install configuration: "Debug"
>> -- Up-to-date: /opt/kde-devel/share/mime/packages/akonadi-mime.xml
>> -- Updating MIME database at /opt/kde-devel/share/mime
>> -- Up-to-date:
>> /opt/kde-devel/lib64/cmake/KF5Akonadi/KF5AkonadiConfig.cmake
>> 
>>> 
>>> on my own system, I have /usr/share/mime/packages/kde.xml defining
>>> application/x-smb-workgroup and /usr/share in XDG_DATA_DIRS, and
>> yet
>>> running update-mime-database on
>> /d/kde/inst/kde_frameworks/share/mime
>>> creates
>>> 
>> 
> /d/kde/inst/kde_frameworks/share/mime/application/x-smb-workgroup.xml
>> as
>>> expected.
>>>> I manually ran update-mime-database on the akonadi and
>> kdepim-runtime
>>>> install dirs on Jenkins and the Zanshin tests magically started
>> passing
>>>> again.
>>> Yes, which only proves that the problem was that it didn't run,
>> not that the
>>> value of XDG_DATA_DIRS is the problem.
>>> 
>>>> Our idea for a quick dirty fix was to simply run
>> update-mime-database once
>>>> CI sets up the environment variables but before the test is
>> executed.
>>>> 
>>>> David, do you have any better suggestions?
>>> 
>>> Yes, I'd like to know why "make install" in e.g. kcoreaddons
>> doesn't seem to
>>> run update-mime-database on CI while it does here (and given that
>>> update-mime-database *is* found in the CI).
>>> With the new CI I'm having trouble debugging this, there are no
>> sources and
>>> build dirs from previous runs anymore :-)
>> 
>> update-mime-database  seems to be run by ECM, the documentation says
>> 
>> # The follow macro is available::
>> #
>> #   update_xdg_mimetypes(<path>)
>> #
>> # Updates the XDG mime database at install time (unless the
>> ``$DESTDIR``
>> # environment variable is set, in which case it is up to
>> package managers
>> to
>> # perform this task).
>> 
>> I suspect that CI uses "make DESTDIR=/srv/jenkins/...../foo/inst
>> install" to
>> install stuff, thus not triggering the update, as documented and
>> it's up to
>> the CI script to run it manually....
> 
>> Dan
> 
> Sorry I have not been more responsive. I am visiting family and have
> been traveling.
> 
> I think I found the issue. The function that checks whether our
> scripts should run it was not looking in /usr/share/mime
> 
> Just commited a fix.
> 
> Scarlett

Looks like update-mime-database is now executed, but there's still some 
issue. I triggerd Akonadi build manually and after "make install" phase 
I can now see

    Directory 'share/mime/packages' does not exist!

and our MimeTypeCheck is still fails. Looks like  
/srv/jenkins/.../akonadi/inst/usr/ is missing in the path that 
update-mime-database is passed?

Dan

> 

-- 
--
Daniel Vrátil
www.dvratil.cz | me at dvratil.cz
facebook.com/danvratil | @danvratil | +DanVrátil