[KDE/Mac] Review Request 124894: [OS X] make it build

Mon Aug 24 15:02:22 UTC 2015

> On Aug. 23, 2015, 10:05 p.m., Kevin Ottens wrote:
> > tests/units/akonadi/CMakeLists.txt, line 9
> > <https://git.reviewboard.kde.org/r/124894/diff/1/?file=397609#file397609line9>
> >
> >     Only difference here is the missing akonadidatasourcequeriestest, any idea why you had to remove it? I'd prefer a proper fix to such a workaround.
> 
> René J.V. Bertin wrote:
>     See the error quoted in the summary. I cannot even make out what statement in the code references the missing symbol. These are only tests, as long as there appears to be no reason to actually run them on OS X I am not very motivated to spend a lot of time hunting an exotic linking error...
> 
> Kevin Ottens wrote:
>     Well, sorry, that's not a valid argument for me. Those tests need to be ran on OS X as well if we want a chance to support it properly.
> 
> René J.V. Bertin wrote:
>     Well, sorry, but in that case you're going to have to step in/up, because it's your code and I don't understand it nor the error it's throwing beyond that it's a case of a missing symbol. I do know that there are subtle differences between gcc and clang and that clang on OS X uses libc++ instead of libstdc++ and that C++11 support has long been incomplete on OS X (and may still be on my version). I also have a strong hunch that doing an uneducated google search on this error will yield either countless irrelevant or obsolete results (cf. the std::accumulate fix), or none at all.
>     
>     Another way of putting my argument would be "you wouldn't insist on testing something that's known not to work on platform X"?
> 
> René J.V. Bertin wrote:
>     BTW, it appears to be rather straightforward nowadays to run OS X under VirtualBox, and apparently the "on Apple hardware" EULA clause is void in countries like Germany.
> 
> Kevin Ottens wrote:
>     Could you pastebin a full compile log somewhere?
>     
>     Also, do you think it's about clang or about OS X? Because I could attempt a build using clang I guess.
> 
> René J.V. Bertin wrote:
>     I just did a Linux build with (almost) the same clang version. It exposed a missing `#include <algorithm>` in `src/domain/task.cpp` (see https://launchpad.net/~rjvbertin/+archive/ubuntu/kdepim/+build/7831190/+files/buildlog_ubuntu-trusty-amd64.zanshin_0.2.1.1163-1-git150821-rjvb-ppa150824b_BUILDING.txt.gz) plus a number of other (cast) errors that I didn't get on OS X. That ought to answer your question: yeah, I fear it's got more to do with OS X (and the fact it uses libc++ instead of libstdc++) than with clang .
>     
>     I'll get you a configure + build log.
>     
>     I'd have to check, but I wouldn't be amazed if FreeBSD (and thus PC-BSD, the "Kubuntu of FreeBSD"; it uses a KDE4 desktop) use libc++ too and show the same build issues if they're caused by libc++. You'd probably feel more at home on that OS, and installing it under VirtualBox is straightforward.
> 
> René J.V. Bertin wrote:
>     BTW: zanshin v0.2.1-1132-g13707d3 built just fine in that same launchpad PPA with the exact same settings.
> 
> René J.V. Bertin wrote:
>     configure + initial full build (using `-k`):
>     https://paste.kde.org/poqsuhugq/ydfbri/raw
>     
>     last (3rd) `make -k` in the same directory, showing only the failing build commands:
>     https://paste.kde.org/pcajg0xn5/ecjszc/raw
> 
> René J.V. Bertin wrote:
>     damn, that last link should be https://paste.kde.org/pkbu3aown/3yr0fm/raw
> 
> Kevin Ottens wrote:
>     OK, this one about akonadidatasourcequeriestest is really obscure. It's not the only one you disabled though, any chance to see the compile or link error you get for the other tests you disabled? I wonder if they're all having the same root cause or if they are several different things.
> 
> René J.V. Bertin wrote:
>     search for `error:` in the log, there's 14 of them ;)
> 
> Kevin Ottens wrote:
>     Ah right, I missed the other ones.
>     
>     OK, they all seem to revolve around mem_fn at least. So I guess switching to a lambda where we use mem_fn could solve it. I'm not a huge fan of that, also it's surprising that it fails for linking some of the tests but not the application itself. Very surprising...
>     
>     Anyway regarding the mem_fn issue, it's likely a long shot but I saw references to a bug with some clang version on OS X, they mention passing the following option at compile time: -D'_LIBCPP_EXTERN_TEMPLATE(...)='
>     
>     Could you try to do the same on your end and see if that helps?
>     
>     I tried building with clang 3.5.0 here but couldn't reproduce the issue so far.
> 
> René J.V. Bertin wrote:
>     OK, I'll try that, after the current build with clang 3.6 (from MacPorts, not Apple) is done.
>     So what is going on, and why would using mem_fn explicitly solve it if that's already a symbol that the linker cannot find?
>     
>     Anyway, it seems my hunch was right that this is related to the use of lambda functions. I always forget if Apple/clang's variant, "blocks", are available outside of ObjC code, but if they are that might possibly be way to circumvent the issue altogether?
>     
>     Are you sure that the application itself includes equivalent expressions to the one that fail in the test cases? If so, that code must be doing something that prevents the linker failure: include a headerfile or pull in the right library/framework explicitly. Have you been able to compare the linker commands for the applications and for the failing tests?
> 
> René J.V. Bertin wrote:
>     Oh, one more thing: clang 3.5.0 had some issues on Linux, which apparently you didn't run into, but it's best to update to at least 3.5.1 .
> 
> Kevin Ottens wrote:
>     Note that it's not an issue with lambdas but with mem_fn, especially the one failing in datasourcequeriestest has no lambda involved at all for that particular symbol... actually swithing to a lambda could help to solve it but I'd rather not go that route if we find a workaround.
>     
>     The thing with mem_fn is that it's mostly template, so I guess there's something fishy in the way it extend them assuming one is extern while it's not the case and then having that instanciation obviously missing at link time. Since I can't seem to reproduce here it is of course wild speculation still.
>     
>     BTW, which Mac OS X and clang versions are you having the issue with?
> 
> René J.V. Bertin wrote:
>     OS X 10.9.5 with the system compiler (Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)) (see "Tested on" above).
>     
>     So, yes, it's got something to do with Apple's clang. When I use clang 3.6.1 from MacPorts (i.e. "stock" clang), the build completes fine. So I compared the preprocessed code from the 2 compiler versions (I'll upload the files here), and there's a difference that could explain why the linker fails but not always : Apple clang 6 defines `__mem_fn::operator() (_ArgTypes&&... __args)` and clang 3.6 `__mem_fn::operator() (_ArgTypes&&... __args) const`.
> 
> Kevin Ottens wrote:
>     Somehow I'd be surprised it's simply caused by the const... But I'm really not sure about what's going on to be honest. I see it's also doing things with visibility so it could be a bug related to that as well in Apple clang.
>     
>     Please try to force the -D'_LIBCPP_EXTERN_TEMPLATE(...)=' flag with Apple clang as I proposed earlier. If that doesn't work, also try -D_LIBCPP_INLINE_VISIBILITY="" and see if that helps.
>     
>     If any combination of those two work I think the workaround will be to add them only in the case of Apple clang, not sure how to detect that with cmake.
> 
> René J.V. Bertin wrote:
>     I tried configuring with `-D'_LIBCPP_EXTERN_TEMPLATE(...)='` ; doesn't make any difference.
>     
>     I did notice that the something inserts a mention of boost::mem_fn into the preprocessed output attached to this ticket. zanshin already depends on boost, so I replaced `std::mem_fn` with `boost::mem_fn` in akonadidatasourcequeriestest.cpp and that appears to solve the issue (and the resulting test runs without errors).

an additional `-D_LIBCPP_INLINE_VISIBILITY` breaks just about everything.

- René J.V.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/124894/#review84235
-----------------------------------------------------------

On Aug. 24, 2015, 4:16 p.m., René J.V. Bertin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://git.reviewboard.kde.org/r/124894/
> -----------------------------------------------------------
> 
> (Updated Aug. 24, 2015, 4:16 p.m.)
> 
> 
> Review request for KDE Software on Mac OS X and Zanshin.
> 
> 
> Repository: zanshin
> 
> 
> Description
> -------
> 
> This is a set of patches that allow zanshin version `0.2.1-1163-gbf5c321` to build on OS X against the git head (or almost head) of kdelibs 4.14 and kdepim* 4.14 .
> I presume for now that a number of changes are self-explanatory (for instance, `zanshinkdepimstatic` calls itself static, isn't installed and doesn't export any symbols explicitly so all points to it being intended to be a static instead of a shared lib).
> The patches that disable a number of tests, and I do not understand myself what exactly goes wrong without them other than that the linker gives errors like
> 
> ```
> # Undefined symbols for architecture x86_64:
> #   "decltype(*(std::__1::forward<QSharedPointer<Domain::QueryResult<QString> > >(fp0)).*fp(std::__1::forward<>(fp1))) std::__1::__invoke<QList<std::__1::function<void (QString, int)> > (Domain::QueryResult<QString>::*&)() const, QSharedPointer<Domain::QueryResult<QString> >, void>(QList<std::__1::function<void (QString, int)> > (Domain::QueryResult<QString>::*&&&)() const, QSharedPointer<Domain::QueryResult<QString> >&&)", referenced from:
> #       std::__1::__function::__func<std::__1::__mem_fn<QList<std::__1::function<void (QString, int)> > (Domain::QueryResult<QString>::*)() const>, std::__1::allocator<std::__1::__mem_fn<QList<std::__1::function<void (QString, int)> > (Domain::QueryResult<QString>::*)() const> >, QList<std::__1::function<void (QString, int)> > (QSharedPointer<Domain::QueryResult<QString> >)>::operator()(QSharedPointer<Domain::QueryResult<QString> >&&) in queryresulttest.o
> # ld: symbol(s) not found for architecture x86_64
> # clang: error: linker command failed with exit code 1 (use -v to see invocation)
> ```
> 
> Possibly a comparable to the reason why I had to define the `std::accumulate` functions.
> 
> 
> Diffs
> -----
> 
>   3rdparty/kdepim/libkdepim/CMakeLists.txt cc8845a 
>   3rdparty/kdepim/libkdepim/tests/CMakeLists.txt 775ac0e 
>   src/akonadi/akonadiserializer.cpp 5116fa5 
>   tests/units/akonadi/CMakeLists.txt 15e887f 
>   tests/units/domain/CMakeLists.txt 3b4d23c 
>   tests/units/presentation/CMakeLists.txt 3ce5a70 
>   tests/units/widgets/CMakeLists.txt ab740a3 
> 
> Diff: https://git.reviewboard.kde.org/r/124894/diff/
> 
> 
> Testing
> -------
> 
> On OS X 10.9.5 with the system compiler (`Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)`)
> 
> My initial reason to install this was to "check it out" and learn how to use it, so I have not done any further testing ...
> 
> 
> File Attachments
> ----------------
> 
> datasourcequeriestest.cpp preprocessed by Apple Clang 6
>   https://git.reviewboard.kde.org/media/uploaded/files/2015/08/24/b2f3e229-ca5a-4e55-b33b-6504b2cdb5c7__datasourcequeriestest-AppleClang6.i
> datasourcequeriestest.cpp preprocessed by clang 3.6.1
>   https://git.reviewboard.kde.org/media/uploaded/files/2015/08/24/807e30a4-aa34-45b4-9e5c-92020ee8c7f5__datasourcequeriestestClang3.6.i
> 
> 
> Thanks,
> 
> René J.V. Bertin
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-mac/attachments/20150824/3b922108/attachment-0001.html>