[KDE/Mac] QSP patch/activator

[David Faure]

On Sunday 10 January 2016 21:52:13 René J.V. Bertin wrote:
> You meant inter-INdependent I suppose?

Yes, of course.
 
> Sadly I had completely forgotten I'd ever heard about KDE, 4 years ago (and wasn't where I was ATM, development wise). If I'd have known what I know now I'd have defended KApplication as a transparent wrapper that would be allowed only certain very specific operations to improve (allow!) cohesion among applications on platforms where that cohesion would otherwise suffer or disappear.

We can get cohesion without subclassing QApplication, that's for sure.

> Then it's back to either the current QspXDG activator, or a KStandardPaths framework that would do exactly what I proposed for K*Application vs Q*Application, except limited to QStandardPaths. And since QSP doesn't have a ctor because its never instantiated you'd lose

The idea of static QStringList KStandardPaths::locateAll() (and similar) doesn't need a ctor.

> > But it's so completely contradictory with the actual purpose of KF5, that for sure a different approach has to be found.
> > One which works with the mindset of "Qt plus a bunch of libs".
> 
> Is that mindset allowed to contain concept like cohesion, ecosystem, family or any other concept to indicate that some of those "Qt plus a bunch of libs" applications share requirements that set them apart from "Qt applications that may happens to use a few of those same libs"?
> If not, then it basically means "SOOL, just take it up with Qt" (where Qt will say something similar).

I'm not sure you realize how many things we implemented in Qt, for the benefit of KF5.
Qt _is_ the place where Qt apps get cohesion, obviously.
Having to implement or fix something in Qt does NOT mean you are SOOL.
Yes I know you've been bitten by the XDG* env var patch, and I won't re-iterate why this was doomed in the way it was done,
but trust me, we *can* change Qt, I have a LONG list of commits to prove it.

> > > > > > But I mean "application A can get QSP native while KF5 gets QSP in XDG mode".
> > > > > > So ok not per-call, but not process-global either. Something in between: per module.
> > > 
> > > Do you remember what you meant with "module" here?
> > 
> > Yes: library, plugin or application.
> 
> > If the developer of one library, plugin or application, wants to install its files in directory X and needs
> > to tell QStandardPaths that the files are in directory X, then it needs a way to do that *without*
> > changing the *global* mode of operation of QStandardPaths.
> 
> Adding the activator module only to applications?

Then the libs/plugins used by the app might not find their files anymore. Those libs/plugins that didn't except QSP to suddenly switch to XDG mode.

> I don't think it'd make sense to allow this on a lower level (esp. not plugins), or else you'd need to add a mode selector to each QSP method that can work in different modes. That'd be a much more invasive change than what I've been aiming for, requiring an API change on all platforms.

You can't justify a wrong solution (that would break things for users) with the argument "anything else is too much work".

> > - making it possible for QSP to find files in a custom install prefix 
> >     (the solution with env vars was deemed inappropriate for OSX, IIUC?
> 
> Yes, because you have very little control over who sees what variables.

So why are you trying to push an "activator for QSP that switches it to XDG mode"? That solution would still require users to set XDG_DATA_DIRS
so that it contains /opt/local/share or whichever prefix was used at install time. That's what puzzles me here.
I'm hearing: "Env vars are bad on OSX. Let's use env vars!"

> >     so maybe instead QSP could have a call where a module can register its install prefix - but then it's not relocatable; 
> 
> What is "it" here?

it = the module (i.e. lib/plugin/app). But you say below that it's ok that things can't be moved around, so the addPrefix() solution seems fine in that respect.

> >     another solution is some magic that determines the prefix from the install path of the lib, was discussed long ago on k-f-d, though maybe not for OSX)
> 
> But what lib? 

Any lib. Each module where QSP is being used. That's the solution for fully-relocatable libs but it requires native hackery.
I can't find the thread again. Basically, finding a way for code at runtime to find out where the lib containing that code is installed.

> That idea strikes me as an expensive way of achieving something that could just as well be hardwired at build time...

Dunno about expensive, but hard to implement reliably and portably, yes.

> > > 1) introduce a header-based class (say KStandardPaths in kstandardpaths.h) which wraps all relevant QStandardPaths 
> 
> > > > Well such a wrapper could just do the lookup itself, i.e. no need to toggle any global switch then.
> 
> > KStandardPaths::locateAll(a,b,c) {
> > #ifdef Q_OS_OSX
> >     // look up additional dirs .... which ones btw? $XDG_DATA_DIRS? ok, see below then
> >     if (found in additional dirs)
> >        return result;
> > #endif
> >     return QStandardPaths::locateAll(a,b,c);
> > }
> 
> But in that case you would have to build the KF5 Frameworks with the KStandardPaths flavour of choice

In this idea there is no "flavour". Just a one time search-replace to use an intermediate layer provided by kcoreaddons.
I'm not a big fan of the idea because it creates more inter-dependencies (especially problematic for tier1 frameworks).
And anyway it is based on env vars which you don't want.... (although, well, it could integrate the addPrefix solution,
but still, we should first do that in Qt).

In fact.....

adding /share automatically in Qt for all platforms is problematic (too XDG-oriented).
But instead, we could have
   QStandardPaths::addStandardLocation(GenericDataLocation, "/some/dir")
(useful for unit tests too, etc.)
I like this idea. It should be non-controversial and it solves a good part of the problem already.

> , and you'd still be in a place you probably don't want to be in your Scribus example. I don't think it's a good idea in general if libraries (like the Frameworks) are allowed to have a different idea of where files are stored. That would probably lead to very confusion results for the user.

I don't see a problem with *adding* search paths.
Scribus will still be able to find its files in the fallback OSX standardpaths, while the libs it uses will find their files in the added standard locations.
Only a problem if they install a file with the same name, but that would be a problem on linux too where it all goes into a /usr prefix,
so no risk of that, the distros would catch it much before you hit it.

> > What was refused was $XDG_* because, well, XDG isn't a mac spec.
> > But I think that a review request that adds support for QT_ named env vars,
> > and which has better justification than "KDE needs it" would have good chances.
> 
> It'd still be hard to describe this without referring to XDG at all. TAFKAP comes to mind ;)

I disagree, QT_GENERICDATALOCATION_DIRS would be the dirs returned by GenericDataLocation.
However if you don't want env vars, let's not talk about env vars ;)

It would be a much easier solution (from a developer's point of view) than addStandardLocation
(which requires being called!). But less nice for users.

> > I'll have some objections to that too (not modular, etc.), but at least I won't have the major objection that
> > we've been moving away from subclassing QApp so that one can use a class from framework X without
> > the requirement to suddenly change the main() to use a different type of application. How do you do that
> > when Qt designer loads a plugin that contains widgets from KTextWidgets? Oops, can't change the app
> > class.
> 
> Why would a plugin or a library use a QApp subclass? I'm not getting my head around this at all; what exactly would become impossible if a loadable component that uses a KApplication instance (that doesn't override a single method and adds no member variables or functions) is loaded into a host app that uses QApplication (or vice versa)?

... !?!?!? are you aware that only one instance of QApp can exist in a given process ???

> > The scribus example proves that a toggle is NOT the way to go, please forget about a global toggle.
> 
> I am actually not entirely convinced about this. The fact that a toggle can be misused isn't a sufficient reason to forget about it completely. Besides, the scribus example can be countered. A packager/distributor can decide to build the KF5 frameworks such that they select XDG-compliant mode, and that will indeed change the behaviour of every application that uses even a single KF5 framework. However, in my current implementation the switched is toggled before main is called, so the packager/distributor can undo the effect of his earlier decision by resetting QSP back to native mode.

This seems completely broken to me. App A uses lib B, and app A needs QSP in native mode, lib B needs QSP in XDG mode.
Whichever global mode QSP is in, one of these two modules will be broken. Asking packagers to fix it is just putting the problem on someone else's shoulders.

> I have opted for a link-time activator module until now because I wanted to avoid the maintenance burden of having to patch all applications (including all those little helper applications one could so easily forget about). Requiring all main() functions to set an env. variable so that QSP will respect $XDG_* (or whatever) is no different than calling the QSP toggle from main, in terms of maintenance.

If that env var *toggles* then yes it's just as bad. If it *adds* paths, then it's much better.

But yeah the whole point of env vars is to let the users set them.
If it's about adding code, then let's have methods, not qputenv.

> > You then install application A from macports which depends on frameworks B and C.
> > They all get installed into /opt/local - or worse, a configurable prefix of the user's choice.
> > The user starts the app, it's supposed to magically work, without having to set any env var?
> 
> Basically, yet. For terminal work you have to add ${prefix}/bin to the path of course (and I have reason to believe this is true at least for some KF5 applications too; KDevelop5 fails to find its plugins when this is not the case for instance).

Wrong, plugins aren't located using $PATH at all.
PATH is for executables.
Anyway, out of scope for this discussion.

> > Macports compiles the code on the user's machine, right? So if we remove relocatability from
> 
> Not necessarily. You can benefit from prebuilt packages if you install to the default prefix and use a port's default variants (and the buildbots have built for your system, and there are no redistribution restrictions).

OK, so still no relocatability, good.

> > the equation (the ability to `mv /opt/local /something/else`) then install prefixes built into the
> 
> Relocating is only possible with hacks involving symlinks.

Sure. That's not full relocatability then, good.

> > All "modules" (frameworks and apps) would need to be patched with such an added line
> 
> If you patch frameworks with this mod, you get hit again by the Scribus example, no?

No, not if you just *add* search paths.

> > (or you can start imagining buildsystem hacks to automate it), unless we say that installing
> > every single module into its own prefix isn't supported (it's how our jenkins setup does it though,
> > to ensure that things don't get picked up by accident from libs not marked as dependencies).
> 
> Would it be easier to write a build system hack to disable this particular call? Maybe not use CMAKE_INSTALL_PREFIX but an intermediate variable that can be empty (and QSP::addPrefix would do nothing in that case)?

I don't see why we would ever want to disable the call to (now renamed) 
QSP::addStandardLocation(GenericDataLocation, CMAKE_INSTALL_PREFIX + "/share")
It's good for all users.

> Anyway, MacPorts has trace mode for this kind of situation. That's basically a PRELOAD wrapper library that modifies file open routines so that they fail on things that don't belong to known dependencies. Should be possible to do this on Linux too, if you know exactly what files are provided by the declared dependencies.

Urgh.

> Also, it seems hard to undo the addition of a component to the paths list instead of doing a behaviour toggle (= call a different function that returns the paths). Without such an undo, and if you indeed patch all frameworks, Scribus would be forced to use that new ("XDG-compliant") path or potentially worse, only the KF5 framework(s) it uses would use those paths. 

No, it's about adding search paths. No undo needed, no change of behaviour for existing code.

We do it regularly in QSP, adding new search paths to cover new use cases (e.g. on Windows, searching in the app's own dir).

> > Note that I'm not talking about writableLocation() anywhere, because honestly, who cares if
> > the user's config is in ~/Library/Preferences or ~/.config. I don't see any purpose into adding
> > "XDG-compliant" code on OSX to ever use ~/.config. Let's just use ~/Library/Preferences/.
> > Or am I missing something important ?
> 
> Maybe not for user configuration, but one has to be sure that applications that don't use QSP (i.e. not Qt5-based) can find those configuration files. 2 examples I know of: KDevelop and QtCurve both use ~/.config, ~/.cache and/or ~/.local/share in their KDE4 variants.

And? Why do they have to use the same dirs as Qt5 apps?
Actually I'm pretty sure that KDevelop-kde4 didn't use ~/.config but ~/.kde/share/config.
We've had that migration on Linux too, between qt4/kdelibs4 and qt5/kf5, from ~/.kde/share/config to ~/.config.
Given the state of KF5 on OSX I'm not sure that migration is a big concern?

> To be very honest, other than Apple's App Store rules I really don't see why cross-platform applications couldn't use their usual locations. Recent OS X versions hide ~/Library in the Finder (where all OS X versions already did that with "dot-something" files and folders), so it's not like there is an official argument along the lines "users expect to find this kind of file under ~/Library/<What Ever>". So in fact, yeah ... who cares whether they're in ~/.config or not ... so let's just leave them there? ;)

We agree, except that when I say "let's just leave them there" I mean, where Qt5's QSP puts them on OSX, which is NOT ~/.config.
The path of least effort is to keep ~/Library/Preferences.

> I could live with ~/Library/Preferences/QtApps (instead of ~/.config), ~/Library/Caches/QtApps (instead of ~/.cache) and ~/Library/Application Support/QtApps (instead of ~/.local/share) but only if nothing is done to prevent someone from replacing those locations with a symlink. I really don't want a lot of additional files to start appearing in the root of ~/Library/*

Of course you can symlink them (but then even native apps will follow the symlink, obviously).

> Which reminds me: by default OS X uses a case-insensitive file system. I've seen evidence of aliasing when I still used ~/Library/Preferences/KDE instead of ~/.kde, so it's probably not a bad idea to use a scheme where users can replace the locations suggested above ("QtApps") with a symlink to locations on a case-sensitive partition.

No idea what you mean, and we're getting out of topic *I think*.

-- 
David Faure, faure at kde.org, http://www.davidfaure.fr
Working on KDE Frameworks 5