[KDE/Mac] Review Request 119497: Report crashes of KDE apps in Apple OS X (1) (fix kcrash)

Sat Sep 20 22:32:33 UTC 2014

> On Juli 29, 2014, 3:33 nachm., Aleix Pol Gonzalez wrote:
> > kinit/kinit.cpp, line 1481
> > <https://git.reviewboard.kde.org/r/119497/diff/1/?file=293442#file293442line1481>
> >
> >     do you need $DISPLAY in OS X?
> 
> René J.V. Bertin wrote:
>     Nope. It can be set if the user has XQuartz installed and running, but that shouldn't make a difference.
>     
>     In fact, $DISPLAY shouldn't be used on OS X because we wouldn't want things like socket names change when the user starts or quits XQuartz with KDE apps and/or services running.
> 
> Ian Wadham wrote:
>     Perish the thought ($DISPLAY fluctuating between a value and an empty string). On my system, OS X 10.7.5 (Lion), XQuartz was installed by Apple and $DISPLAY is always set to a value, even if XQuartz (X11.app) is not running (i.e. no X11 server running). I presume installing X11 somehow "doctors" the OS X (Darwin) startup procedures so that DISPLAY is already defined in every user's ~/.profile. I do not know if that would be the case if a FOSS version of XQuartz would be installed (e.g. on OS X 10.9.x Mavericks).
>     
>     The big thing is that $DISPLAY -is- used (non-portably) in KDE to help name a socket in kdeinit4 (kinit.cpp), wrapper.c, kwrapper and KCrash - and FAIK it may be used in this way in several other places. If $DISPLAY is not used consistently in all those places, communication with kdeinit4 can break, as it does already in KCrash/kdeinit4 on Apple OS X. And THAT is what we are trying to fix here. KDE apps on Apple OS X MUST be able to report a crash reliably.
>     
>     This is why I keep asking for help from a KDE core developer. How widespread is this problem in KDE on Apple OS X? What are the implication for KDE apps? Should references to $DISPLAY in KDE be eliminated from the OS X version? What do kdeinit4, kwrapper and klauncher really do? Is whatever they do actually portable to Apple OS X? Or are we all, KDE guys and MacPorts guys alike, just crossing our fingers and hoping for the best?
>     
>     I tried using lxr.kde.org to find further references, but there are thousands: mostly because "display" is a commonly-used English word in programming. And I did tick the case-sensitive box, but it does not seem to work.
> 
> René J.V. Bertin wrote:
>     Hmm, interesting. I should find some time to play with this in my 10.9 VM.
>     Know however that even if $DISPLAY is always set, it will not always have the same value. At least for me, XQuartz has the annoying habit of increasing the display number after a restart.
>     
>     If it's too complicated to remove all references to $DISPLAY from KDE code (which wouldn't surprise me at all) there remains another approach. Identify an appropriate location in the startup/initialisation code that all KDE applications and services share, and set $DISPLAY to something sensible (but preferably NOT a valid X11 display specifier). The only possible undesirable side-effect I can see from here would be that shells in konsole risk to inherit that value.
>     This probably isn't the most elegant solution, but then again that's because KDE apparently never imposed the use of its own internal variable/function (or one from Qt) instead of directly querying $DISPLAY.
> 
> Ian Wadham wrote:
>     Restart of what? My system (Lion) has Apple OS X's XQuartz and $DISPLAY has a random temp-file path prepended every time Apple OS X starts of restarts. And that path is different every time. But so what? $DISPLAY keeps the same value no matter how many times I start XQuartz (X11) by running Gimp or whatever. And that value, determined immediately after boot or restart, should be picked by all KDE components, which come into existence later.
> 
> René J.V. Bertin wrote:
>     I meant restarts of the X server - it does crash sometimes, sometimes I have to restart it for other reasons, like after logging off and back in. I recall that I used to have those weird (socket based?) $DISPLAY values too, but now they're of a perfectly normal host:X.Y form on my systems. Except that X tends to increase each time I start XQuartz. I ignore how common this is, but I think that if you're looking into the use of $DISPLAY on OS X, you could just as well take all possible situations into account. I'd suggest to use the fact that the actual value is irrelevant and without important to clamp it to something appropriate (why not Qt4:Mac) in such a way that all those younger components pick up that value. And not the actual, current value of $DISPLAY, which may or may not have remained unchanged. (I specifically used the term clamp to draw an analogy signal acquisition, where unconnected inputs can carry all kinds of bogus signals.)
> 
> René J.V. Bertin wrote:
>     I've looked into the $DISPLAY value variations mentioned (= described from memory) above. The big lines hold:
>     
>     - When I log in, $DISPLAY is not set. This is probably because I deactivated org.macosforge.xquartz.startx.plist (by moving it from Library/LaunchAgents): I always launch an X11 environment anyways and the autostart feature tends to start another server when I'm least expecting it.
>     - When I start XQuartz and launch (for instance) konsole, $DISPLAY remains unset, unless I launch konsole using `open -a konsole` typed into an xterm (i.e. with $DISPLAY set).
>     - The value $DISPLAY takes (initially) is ":0"
>     - ~/.MacOSX/environment.plist contains <key>DISPLAY</key><string>:0.0</string> but the file does not exist on all accounts I tried (which show the same behaviour, suggesting that the file is irrelevant).
>     - Quitting and restarting XQuartz normally does not modify the $DISPLAY value
>     - A crash (or SIGTERM, kill -9) of XQuartz DOES cause $DISPLAY to become ":1" after restarting the server
>     - This is a system-wide phenomenon that's not linked to a particular login session. In other words, if I log off and back in as a different user, starting XQuartz as that user will still set $DISPLAY=:1 . (If I'd had to guess, I'd say the crash/kill caused something like a shared memory segment to not be released.)
>     - One of the accounts I checked is a guest account in which Xquartz is started in a purely stock way (and had never been started before), without any attempts to set $DISPLAY to something not including a filepath.
>     
>     In short, $DISPLAY doesn't necessarily have a file path (starting with "/tmp") value, and can indeed change during login session.
>     
>     Testing was done on 10.6.8 and 10.9.2, using XQuartz.app maintained by Apple's Jeremy Huddleston. This is not the same as the (deprecated) X11.app that used to be distributed by Apple as an optional component of Mac OS X. It's the same server of which (at least) the local libraries are installed by MacPorts.
> 
> Benjamin Reed wrote:
>     Ultimately, the point of using $DISPLAY in KDE is to be able to associate things happening in KDE with a unique user "session", and on Mac OS X, $DISPLAY is not in any way tied to the user's actual session.  It can get reset or changed for any number of reasons, even though the user is in a single login session to the Mac desktop.
> 
> Ian Wadham wrote:
>     This part of the patch has been discontinued.
> 
> René J.V. Bertin wrote:
>     I'm not sure why you're dropping this. Wouldn't the simplest solution be to make sure, somewhere at a very early stage (and I've seen places where it seems DISPLAY is being set/changed in the environment, from kdelibs!) that the variable is set to something sensible? Something that is a syntactically correct DISPLAY variable?
>     
>     If we presume that KDE-mac never actually connects to an X11 server, it would make perfect sense to set DISPLAY=localhost:0 . Any code using the host information will get the correct host address out of this, the rest doesn't matter. Except that all clients will agree on the host they're displaying on.
> 
> Ian Wadham wrote:
>     The socket name is generated in 3 or 4 places, I am not sure how many. All of these must "line up" if KCrash, kdeinit4, klauncher, kded4 and kwrapper are to run as and when required and interact correctly on Apple OS X. And I am not even sure how much or how many of those background processes are really needed on Apple OS X or what their usual uses are or what their uses should be on Apple OS X or even whether new versions of all the processes are going to be used in KF5.
>     
>     Until I can get some expert help and advice from a core KDE developer I am not going to touch any of that code. That is final.
> 
> Thomas Lübking wrote:
>     http://lxr.kde.org/search?_filestring=&_string=kdeinit4_
>     
>     Only the three sources should be relevant (but I don't speak docbook)
>     This *will* be relevant for wayland at some point, so ideally get mgraesslin on the topic as well.
>     
>     There's also a NO_DISPLAY definition in kinit and kcrash (since around 2006) which actually seems what you want to ensure being defined on OSX.
>     Applying the same approach (from kcrash.cpp & wrapper.c) to kinit.cpp (where it seems forgotten?) should fix socket generation.
> 
> Ian Wadham wrote:
>     Now you are talking! Actually I hit on that kdeinit4_ search-string myself a while back, but could not be certain about exactly where all the edits of socket names should be, in order to make kdeinit4 and friends work on OS X and avoid breaking anything on any other platform. That is one place where I really needed expert help from a KDE core developer who knows kdeinit4 and friends.
>     
>     David Faure has said, re an earlier issue in this RR, that it would be OK to make $DISPLAY an empty string on Apple OS X (the socket name would then end with kdeinit4_) and that would be fine by me ($DISPLAY no longer exists in OS X itself, only in the FOSS X11 called XQuartz).
>     
>     In the course of seeking expert help, I discovered several other serious problems in kdeinit4, such as permanent blocking at several points in its initialisation, including one that might also exist in the Linux version. Another problem was that I once got all three of kdeinit4, klauncher and kded4 to run at once, but then kdeinit4_shutdown failed because it had the wrong socket name! I know why this is, BTW. I raised questions about these problems by emails to the person who had offered to help and advise me, but he never replied.
>     
>     Thomas, could we discuss these problems somewhere else than here: kde-mac, kde-devel or kde-core-devel? I am a member of all three. Do you have the time? Would Martin Graesslin be interested too? IRC is difficult for me because my timezone is UTC + 10 (Australia).

> That is one place where I really needed expert help from a KDE core developer who knows kdeinit4 and friends.

Don't consider me one, sorry :-(
I'm looking at this code the first times myself now.
But lxr.kde.org would find every occasion of "kdeinit4_", so unless there's some supersmart "kdeinit4" + "_" + display, those are the relevant locations. I'f there is, we'll sooner or later hit it.

> such as permanent blocking at several points in its initialisation

Did you open a bug on this (or record the codepoints somehwere)?

>  kdeinit4_shutdown failed because it had the wrong socket name! I know why this is, BTW.

... because the cmake builds it w/o setting the NO_DISPLAY variable, i'd say (unlike kdeinit4_wrapper)

> emails to the person who had offered to help and advise me, but he never replied

You're hopefully not talking about me, since I cannot recall having received such mails (or offered not existing expertise for advisory =)

> could we discuss these problems somewhere else

Sure. kde-core-devel seems most suited to me (and *far* better than private mails) - I can just assume that Martin is very interested in making things work w/o DISPLAY being set (and ultimately we'll all be ;-)

- Thomas

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/119497/#review63447
-----------------------------------------------------------

On Sept. 16, 2014, 11:14 vorm., Ian Wadham wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://git.reviewboard.kde.org/r/119497/
> -----------------------------------------------------------
> 
> (Updated Sept. 16, 2014, 11:14 vorm.)
> 
> 
> Review request for KDE Software on Mac OS X, KDE Runtime, kdelibs, and Michael Pyne.
> 
> 
> Repository: kdelibs
> 
> 
> Description
> -------
> 
> *NOTE: Issues for KCrash have been fixed. Changes to kinit/kinit.cpp (kdeinit4) have been discontinued. For a summary, scroll to the very end of this page.
> 
> When a KDE app crashes in Apple OS X, it just disappears from the screen. At the most, the user is invited to report the crash to Apple. AFAIK this has been a problem in KDE on Apple OS X for years, leading to frustration with KDE among Apple users and MacPorts developers and an attitude among KDE developers of "Why does nobody report the problem(s) on bugs.kde.org?"
> 
> It is my strong belief that the failure to report crashes of KDE apps in Apple OS X also exists in Frameworks.
> 
> So far I have identified a number of portability bugs in KDE on Apple OS X: 1 in KCrash, 1 in kdeinit4 and 5 in Dr Konqi. Patches for the first two are submitted in this review. Patches for three more are submitted in part 2 of this review, against kde-runtime. I am still investigating the other two problems in Dr Konqi - and there could be more than two...
> 
> In this review we have two portability problems:
> 
> 1. KCrash itself crashes (SIGILL) when it tries to close all file descriptors and so Dr Konqi is not started. Some of the FDs belong to the Apple OS X library (COCOA) and it crashes if they are closed prematurely.
> 
> 2. The preferred way to start Dr K is via a socket message to kdeinit4, but that fails in Apple OS X because kdeinit4 is listening with the wrong socket name. The DISPLAY ID is missing from the end of the name. The fallback is for KCrash to use fork() and exec(), which works, but could cause Dr K to be polluted, depending on the nature of the crash.
> 
> This "deafness" of kdeinit4 (and klauncher) could be causing other failures in KDE software in the Apple OS X environment.
> 
> 
> Diffs
> -----
> 
>   kdeui/util/kcrash.cpp 45eb46b 
> 
> Diff: https://git.reviewboard.kde.org/r/119497/diff/
> 
> 
> Testing
> -------
> 
> Using Apple OS X 10.7.5 (Lion) on a MacBook Pro, I have installed KDE libs via MacPorts (at version 4.12.5) and I have adapted kdesrc-build to run in an Apple OS X environment and used it to test against the KDE 4.13 branch.  I have been testing with a KDE app that I can crash at will and using stderr and Apple OS X Console log output to determine the outcome.
> 
> Please note that I am the -only- KDE developer who has this kind of setup, but I am NOT a KDE core developer. My experience before now has been in KDE Games. However I used to be a UNIX and database guru before I retired in 1998.
> 
> I NEED HELP from KDE -core- developers to proceed further. These problems also exist in FRAMEWORKS, but I am as yet unable to build or test Frameworks on Apple OS X. And I am sure there are many more Apple OS X portability problems in KDE software.
> 
> Without my patches, Dr Konqi is not started and, if it does get past its own crash, KCrash reports:
> KCrash: Attempting to start /kdedev/kde4.13/kde4/lib/kde4/libexec/drkonqi.app/Contents/MacOS/drkonqi from kdeinit
> sock_file=/kdedev/kde4.13/home/.kde4.13/socket-Tara.local/kdeinit4__tmp_launch-svPd0L_org.x_0
> Warning: connect() failed: : No such file or directory
> 
> With my patches, Dr Konqi can always be started directly (using fork()) and it -will- start via kdeinit4 and klauncher but it immediately runs into a privilege problem with Apple OS X (a problem which I am still investigating).
> 
> I would not have got this far without help from Michael Pyne, Thomas Lübking and several of the MacPorts developers, as well as the unfailing enthusiasm and encouragement of my friend Marko Käning.
> 
> 
> Thanks,
> 
> Ian Wadham
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-mac/attachments/20140920/6c75c265/attachment-0001.html>