MimeType issues on CI (Was: Re: Delaying KF 5.21)

[Ben Cooksley]

On Mon, Apr 4, 2016 at 12:16 AM, Daniel Vrátil <dvratil at kde.org> wrote:
> On Sunday, April 3, 2016 12:58:24 PM CEST David Faure wrote:
>> On Sunday 03 April 2016 12:43:27 Daniel Vrátil wrote:
>> > Turns out when you include /usr/share in XDG_DATA_DIRS, all mime types
>> > that
>> > might have been provided by other dependencies in
>> > /srv/jenkins/.../foo/inst/ usr/share/mime are ignored, possibly due to
>> > update-mime-database not being ran for those directories.
>>
>> I'm not sure I fully understand this sentence. Do you mean the value of
>> XDG_DATA_DIRS at update-mime-database time or during the running of unit
>> tests?
>
> XDG_DATA_DIRS value when running the unit tests.
>
>> update-mime-database works on one specific install dir and creates files
>> there. The value of XDG_DATA_DIRS during the run of update-mime-database
>> shouldn't matter.
>>
>> If the stuff /srv/jenkins/.../foo/inst/usr/share/mime is ignored, it can be
>> either
>
>> 1) because update-mime-database wasn't run on that dir
>>    (which is my current hypothesis for the KIO kfileitemtest failure which
>> needs a mimetype from kcoreaddons)
>
> The thing is that the content of /srv/jenkins/.../foo/inst/usr/share/mime is
> NOT ignored, when /usr/share is NOT in XDG_DATA_DIRS.
>
> Once you include /usr/share in XDG_DATA_DIRS, the content of the /srv/.../mime
> folders IS ignored.
>
> Akonadi installs a mime file to share/mime/packages/akonadi-mime.xml which
> introduces some new mimetypes and are used to find the right serialize plugins
> for those types. With /usr/share in XDG_DATA_DIRS Akonadi is not able to find
> serializer plugins for application/x-vnd.akonadi.calendar.todo for instance,
> because it (using QMimeDatabase) does not know about this mime type. Removing
> /usr/share from XDG_DATA_DIRS makes Akonadi (QMimeDatabase) find application/
> x-vnd.akonadi.calendar.todo just find and thus find a respective serializer
> plugin for the type.
>
>
>> 2) because that dir isn't in
>> XDG_DATA_DIRS (this is not the problem) later on.
>
> I verified that it is there.
>
>> 3) because
>> update-mime-database ran but didn't create the stuff it's supposed to
>> create -- I understand your email as saying this might be the problem, but
>> I can't confirm it:
>
> If I compare output of "make install" from CI and running it locally, it
> indeed appears that update-mime-database is not run on the CI at all:
>
> CI:
> 16:22:17 Install the project...
> 16:22:17 -- Install configuration: "Debug"
> 16:22:17 -- Installing: /home/jenkins/sources/akonadi/kf5-qt5/local-inst/srv/
> jenkins/install/ubuntu/x86_64/g++/kf5-qt5/kde/pim/akonadi/inst/usr/share/mime/
> packages/akonadi-mime.xml
> 16:22:17 -- Installing: /home/jenkins/sources/akonadi/kf5-qt5/local-inst/srv/
> jenkins/install/ubuntu/x86_64/g++/kf5-qt5/kde/pim/akonadi/inst/usr/lib/x86_64-
> linux-gnu/cmake/KF5Akonadi/KF5AkonadiConfig.cmake
>
>
> Locally:
> -- Install configuration: "Debug"
> -- Up-to-date: /opt/kde-devel/share/mime/packages/akonadi-mime.xml
> -- Updating MIME database at /opt/kde-devel/share/mime
> -- Up-to-date: /opt/kde-devel/lib64/cmake/KF5Akonadi/KF5AkonadiConfig.cmake
>
>>
>> on my own system, I have /usr/share/mime/packages/kde.xml defining
>> application/x-smb-workgroup and /usr/share in XDG_DATA_DIRS, and yet
>> running update-mime-database on /d/kde/inst/kde_frameworks/share/mime
>> creates
>> /d/kde/inst/kde_frameworks/share/mime/application/x-smb-workgroup.xml as
>> expected.
>> > I manually ran update-mime-database on the akonadi and kdepim-runtime
>> > install dirs on Jenkins and the Zanshin tests magically started passing
>> > again.
>> Yes, which only proves that the problem was that it didn't run, not that the
>> value of XDG_DATA_DIRS is the problem.
>>
>> > Our idea for a quick dirty fix was to simply run update-mime-database once
>> > CI sets up the environment variables but before the test is executed.
>> >
>> > David, do you have any better suggestions?
>>
>> Yes, I'd like to know why "make install" in e.g. kcoreaddons doesn't seem to
>> run update-mime-database on CI while it does here (and given that
>> update-mime-database *is* found in the CI).
>> With the new CI I'm having trouble debugging this, there are no sources and
>> build dirs from previous runs anymore :-)
>
> update-mime-database  seems to be run by ECM, the documentation says
>
>         # The follow macro is available::
>         #
>         #   update_xdg_mimetypes(<path>)
>         #
>         # Updates the XDG mime database at install time (unless the ``$DESTDIR``
>         # environment variable is set, in which case it is up to package managers
> to
>         # perform this task).
>
> I suspect that CI uses "make DESTDIR=/srv/jenkins/...../foo/inst install" to
> install stuff, thus not triggering the update, as documented and it's up to
> the CI script to run it manually....

We use DESTDIR, yes.
I believe Scarlett's fix should correct this, although a full rebuild
of all Frameworks and dependencies of affected products will be needed
to clear out MIMEType related issues.

>
> Dan

Cheers,
Ben

>
> --
> Daniel Vrátil
> www.dvratil.cz | dvratil at kde.org
> IRC: dvratil on Freenode (#kde, #kontact, #akonadi, #fedora-kde)