QSP patch/activator

[René J.V. Bertin]

On Monday January 11 2016 18:58:08 David Faure wrote:

Sorry for the silence, I'm trying to force myself to devote my attention to other things that should have priority right now.

> We can get cohesion without subclassing QApplication, that's for sure.

The question is how. I'm not saying it's impossible, but I'll believe it only when I see it. I cannot think of any way to provide an identified basis for a concept "this application belongs to a particular category/family/suite/collection/whatever of applications set apart by <feature[set]>" that doesn't include a few shared lines of code that allow to activate that feature[set].

> Yes I know you've been bitten by the XDG* env var patch, and I won't re-iterate why this was doomed in the way it was done,
> but trust me, we *can* change Qt, I have a LONG list of commits to prove it.

I don't doubt that. What I do doubt until shown otherwise is whether this particular issue can be solved *completely* by an official change to Qt (i.e. one that doesn't require additional patching to get it just right).
I presume you've seen the reactions I've had to my post about QProcess on Qt's development ML.

> Then the libs/plugins used by the app might not find their files anymore. Those libs/plugins that didn't except QSP to suddenly switch to XDG mode.

For libraries that issue should be completely moot. If they can only support 1 configuration, the decision which one will have to be made at build time, i.e., it becomes a MacPorts (or Fink or ...) problem to address how to build them. That might even include making shared resources (in GenericDataLocation or GenericConfigLocation) available from both locations.
Plugins are different in that one doesn't control at build time in what applications they can be loaded.

I'm having difficulties seeing what exactly it is that you think might break, in part because I'm not seeing any breakage. That may be because most of the plugins concerned that I have installed are provided by KF5 projects, and thus built against my own Frameworks builds. Still, if what you say is true, the simple fact of accessing the platform theme plugin from the frameworkintegration framework should either 1) change the host application's idea of where its files are or 2) cause the platform theme to act up because *it* cannot find its files (if the host application hasn't selected XDG mode).
I'm seeing neither. The platform theme works as one would expect (i.e. kdeglobalrc is found in ~/.config) even when loaded in a pure Qt host app (like Qt Creator or the Charm time tracker) and that host app also shows no signs of dysfunction. Specifically, settings are read correctly, but also written to the correct file after changing them. XDG mode must be selected in the plugin otherwise it shouldn't be finding ~/.config/kdeglobalrc through `KSharedConfig::openConfig(QStringLiteral("kdeglobals"), KConfig::NoGlobals)` (right?).


Maybe we should take a step back and look at how QSP really works and how the XDG compliant mode is implemented. It's never been really clear to me how an application is supposed to choose from the list of options returned by standardLocations(), and how that interplays with the writableLocation(). 
In XDG-compliant mode, I'm getting

  DataLocation: "Application Support" = */Users/bertin/.local/share/QtProject/qtdiag* ; '/Library/Application Support/QtProject/qtdiag' ; '/opt/local/share/QtProject/qtdiag' ; '/opt/local/libexec/qt5/bin/'
  CacheLocation: "Caches" = */Users/bertin/.cache/QtProject/qtdiag* ; '/Users/bertin/.cache' ; '/Library/Caches/QtProject/qtdiag'
  ConfigLocation: "Preferences" = */Users/bertin/.config* ; '/opt/local/etc/xdg'
  GenericDataLocation: "Application Support" = */Users/bertin/.local/share* ; '/opt/local/share' ; '/Library/Application Support'
  GenericCacheLocation: "Caches" = */Users/bertin/.cache* ; '/Users/bertin/.cache' ; '/Library/Caches'
  GenericConfigLocation: "Preferences" = */Users/bertin/.config* ; '/opt/local/etc/xdg'

in native mode:

  DataLocation: "Application Support" = */Users/bertin/Library/Application Support/QtProject/qtdiag* ; '/Library/Application Support/QtProject/qtdiag' ; '/opt/local/libexec/qt5/bin/'
  CacheLocation: "Caches" = */Users/bertin/Library/Caches/QtProject/qtdiag* ; '/Library/Caches/QtProject/qtdiag'
  ConfigLocation: "Preferences" = */Users/bertin/Library/Preferences*
  GenericDataLocation: "Application Support" = */Users/bertin/Library/Application Support* ; '/Library/Application Support'
  GenericCacheLocation: "Caches" = */Users/bertin/Library/Caches* ; '/Library/Caches'
  GenericConfigLocation: "Preferences" = */Users/bertin/Library/Preferences*

in other words, XDG-compliant mode just adds the XDG locations but not even always before the native locations. I could see how an application or plugin would be able to load settings (or another kind of resource) from any location in standardLocations(). I could also see how that flexibility could be extended to writing to such resource files, except that  the existence of a QSP::writableLocation() method suggests that things are not supposed to work that way.


> > That'd be a much more invasive change than what I've been aiming for, requiring an API change on all platforms.
> 
> You can't justify a wrong solution (that would break things for users) with the argument "anything else is too much work".

Please don't confound invasive and "more work" here. Or rather, a more invasive change affecting all platforms would indeed come with more work: justifying the change, and providing sufficient proof that nothing will break.

I agree though that a solution that could induce "breakage" (artefacts) as in your Scribus example should probably not be submitted to Qt ... though ultimately it'd be up to them to judge whether side-effects like that are acceptable.
After all, there are already quite a few AA_* attributes that can be set by any module, and that have the potential to trak things up for the running application nicely and effectively.

> So why are you trying to push an "activator for QSP that switches it to XDG mode"? That solution would still require users to set XDG_DATA_DIRS
> so that it contains /opt/local/share or whichever prefix was used at install time. That's what puzzles me here.
> I'm hearing: "Env vars are bad on OSX. Let's use env vars!"

Heh, we should indeed also take a look at my patch, apparently. While it does introduce support for XDG_DATA_DIRS, it doesn't rely on it exclusively but has a fallback that sets the XDG location from the install prefix -- IIRC the same kind that also exists in the regular Unix QSP:

+    QString xdgDataDirsEnv = QFile::decodeName(qgetenv("XDG_DATA_DIRS"));
+    if (xdgDataDirsEnv.isEmpty()) {
+#ifndef QT_BOOTSTRAPPED
+        dirs.append(QLibraryInfo::location(QLibraryInfo::PrefixPath) + QString::fromLatin1("/share"));
+#endif
+    } else {
+        dirs = xdgDataDirsEnv.split(QLatin1Char(':'), QString::SkipEmptyParts);
+
+        normaliseDirs(dirs);
+    }


> > But in that case you would have to build the KF5 Frameworks with the KStandardPaths flavour of choice
> 
> In this idea there is no "flavour". Just a one time search-replace to use an intermediate layer provided by kcoreaddons.
> I'm not a big fan of the idea because it creates more inter-dependencies (especially problematic for tier1 frameworks).

Yes, additional non-optional dependencies don't seem the way to go, not if it isn't to introduce either a possibility to have a common code-path executed by all application or else some (platform-specific?) dependency that sits between Qt and the KF5 frameworks.

> adding /share automatically in Qt for all platforms is problematic (too XDG-oriented).
> But instead, we could have
>    QStandardPaths::addStandardLocation(GenericDataLocation, "/some/dir")
> (useful for unit tests too, etc.)
> I like this idea. It should be non-controversial and it solves a good part of the problem already.

Yes. You'd probably need append and prepend versions, a solution for the writable location ... and preferably a way to have those elements added for all applications that belong to a certain category.

I've also begun to think about a sort of hybrid working solution that would allow me to test different approaches without having to rebuild *everything* each time I want to test something different. Something that involves an activator module like the one I'm using now, but combined with a KStandardPaths that adds an explicit mode switch to each call to standardLocations() and writableLocations(). The value of the switch argument would be determined at build time. Optional env. variables would allow to disable the activator module and to ignore the per-call mode-switch argument. That should allow to test all possible combinations without having to rebuild every time.

> I don't see a problem with *adding* search paths.
> Scribus will still be able to find its files in the fallback OSX standardpaths, while the libs it uses will find their files in the added standard locations.

Which is the situation we should be in currently

> It would be a much easier solution (from a developer's point of view) than addStandardLocation
> (which requires being called!). But less nice for users.

Not so much easier if you have to resort to setting those env. variables from the code because there is no other viable way to set them in a controlled fashion other than through wrapper scripts. That also works for app bundles, but it'd be somewhat of a nightmare to maintain the packaging adaptations for *all* applications.

Actually, there's a controlled way to do this and more, very flexibly, one that could maybe even be invoked through the proper DYLD_PRELOAD setting :) It would involve calling a script that receives the path to the executable being launched and which can then use any kind of criteria to determine and return a list of env. variables. Sounds a bit like overkill, eh? :)

> 
> > > I'll have some objections to that too (not modular, etc.), but at least I won't have the major objection that
> > > we've been moving away from subclassing QApp so that one can use a class from framework X without
> > > the requirement to suddenly change the main() to use a different type of application. How do you do that
> > > when Qt designer loads a plugin that contains widgets from KTextWidgets? Oops, can't change the app
> > > class.
> > 
> > Why would a plugin or a library use a QApp subclass? I'm not getting my head around this at all; what exactly would become impossible if a loadable component that uses a KApplication instance (that doesn't override a single method and adds no member variables or functions) is loaded into a host app that uses QApplication (or vice versa)?
> 
> ... !?!?!? are you aware that only one instance of QApp can exist in a given process ???

Ah, that, well, yes of course. I suppose that means that plugins won't try to create a QApplication instance, and use qApp or QApplication::instance() instead, if they have to access the class at all. Either should work out fine, no?
I also presume plugins aren't supposed to create the QApplication instance before the host app gets a chance, because that would probably lead to interesting behaviour when the host does its `QApplication app(argc, argv);` ...

> This seems completely broken to me. App A uses lib B, and app A needs QSP in native mode, lib B needs QSP in XDG mode.
> Whichever global mode QSP is in, one of these two modules will be broken. Asking packagers to fix it is just putting the problem on someone else's shoulders.

I don't think any application requires native mode QSP, not in the sense that it wouldn't function otherwise. Apple would require native QSP for admission into the App Store, but in my MacPorts approach the requirement is much weaker and applicable only to pure Qt applications. I'm not shocked by a definition of "pure Qt application" that excludes applications as soon as they start using even a single KF5 framework.

BTW, how are the install locations of KF5 shared resources determined, I double that involves building a utility that links to the framework and uses QSP to obtain the target locations?

> If that env var *toggles* then yes it's just as bad. If it *adds* paths, then it's much better.

I hope it's clear by now that "my" XDG-compliant mode that can be toggled to is nothing but some extra path(s) added to the existing list?
Except for the writableLocation() of course, but I don't see any other choice there.

this is true at least for some KF5 applications too; KDevelop5 fails to find its plugins when this is not the case for instance).
> 
> Wrong, plugins aren't located using $PATH at all.
> PATH is for executables.
> Anyway, out of scope for this discussion.

Out of scope yes, but my observation stands ... however it has to be interpreted ;)


> > Anyway, MacPorts has trace mode for this kind of situation. That's basically a PRELOAD wrapper library that modifies file open routines so that they fail on things that don't belong to known dependencies. Should be possible to do this on Linux too, if you know exactly what files are provided by the declared dependencies.
> 
> Urgh.

Out of scope too, but really a pretty useful tool if you want to ensure repeatable builds (as I assume you would in a CI system).

> And? Why do they have to use the same dirs as Qt5 apps?

Consistency, and not having to search multiple locations for those users who actually have reason to care where those files are installed. And no risk for conflicts with standalone app bundle versions of KF5 applications that one might also use from time to time, for whatever reason.
Pure Qt apps (say Qt Creator) should behave identically if possible, whether they're run from MacPorts or from some official installer (including sharing the Preferences settings) because they're supposed to be different only in where they find their shared read-only resources. Standalone KF5 app bundles will include much more than just libraries and read-only resources and I'm not at all convinced that it'll be possible to deploy all KF5 applications like that without very specific hacks and sacrifices (can you imagine casting KDE PIM into a single app bundle?). I just don't like the idea that such app bundles will use the same settings files as their counterparts built following a more ... familiar approach.

> We've had that migration on Linux too, between qt4/kdelibs4 and qt5/kf5, from ~/.kde/share/config to ~/.config.
> Given the state of KF5 on OSX I'm not sure that migration is a big concern?

No, I don't think migration is a concern. The above is. And there's the fact that MacPorts doesn't discourage the idea of having both KDE4 and KF5 versions of a given application installed (in fact there's demand to allow that).

> The path of least effort is to keep ~/Library/Preferences.

That's for sure.

> Of course you can symlink them (but then even native apps will follow the symlink, obviously).

Yeah, obviously. Boy do I still miss the context-sensitive symlinks from Apollo Unix (DomainOS) :)

> > Which reminds me: by default OS X uses a case-insensitive file system. I've seen evidence of aliasing when I still used ~/Library/Preferences/KDE instead of ~/.kde, so it's probably not a bad idea to use a scheme where users can replace the locations suggested above ("QtApps") with a symlink to locations on a case-sensitive partition.
> 
> No idea what you mean, and we're getting out of topic *I think*.

Maybe, maybe not. It's just something to keep in mind though.

R.