[cmake-developers] [Qt-creator] CMake daemon for user tools

[Brad King]

On 01/12/2016 05:15 AM, Tobias Hunger wrote:
> I read Brad's reply as meaning that your cleanup work is a great
> benefit to cmake, but will probably not be available in the
> foreseeable future (if at all).

Most of Stephen's work completed so far is already in CMake 3.4, and
a bit more is in 'master' and will be in 3.5.  It has no public-facing
changes though so yes more work will be needed to get a solution IDEs
can actually use.

> Alexander goes back to the generator approach we discussed a year ago
> and explicitly says he won't work on that, so nothing will happen
> there.

The generate-json-description approach remains a valid alternative.
Aleix's work on it got pretty far before Stephen proposed the daemon
alternative.  See more below.  IIRC Alexander Neundorf raised concerns
only about how to activate the generation of the json description, and
has now proposed an approach to resolve those concerns.

> So could we please get a *documented* way that an IDE can *rely on to
> be available* that provides basic information on a project/configuration:
> 
> * Source directory, build directory
> * Files that belong to the project (sources, headers, custom cmake
>   modules, CMakeLists.txts, resources, ...)
> * Files that belong to the build system (anything that needs watching
>   to re-generate the IDE-integration information at the right times)
> * Targets that can be built
> * Files that belong to each target
> * Compiler flags
> * Defines in effect
> * Include paths used
> * The compilers used to build those files

Thank you for summarizing the goal so succinctly.  This is different than
I had in mind when writing my previous response.  Before I was thinking
this was about IDEs helping users edit the CMake build system description
itself (adding sources, etc.).  The above is about IDEs helping users
edit their C code with knowledge of how it will be built, thus providing
for completion and such.

These problems are orthogonal, and your requirements certainly need
generation-time information regardless of how the build specification
itself is represented.  Therefore the discussion here should proceed
independent of where the alternative language thread goes.

With that cleared up design discussion can proceed.

-----------------------------------------------------------------------------

IIUC two approaches have been proposed:

1.  CMake generates a JSON description of the above-listed information
    about the entire build tree for IDEs to use as needed.  It can be
    either one giant file or an index and a bunch of per-target files.
    The contract between CMake and IDEs is a JSON schema and file(s) on
    disk.  Updating requires the IDE to do a full re-run of CMake.

    Aleix Pol has implemented a prototype of this approach.

2.  CMake offers a daemon that IDEs can contact (e.g. via named pipes)
    to ask for the information.  The contract between CMake and IDEs is
    a (to-be-designed) protocol (e.g. JSON snippets).  Updating would
    likely be handled internally by the daemon in response to changes
    to the CMake input files between requests from the IDE.

    Stephen Kelly has done refactoring work on CMake internals that
    will be needed to implement such a daemon, but much more work is
    still needed.

Stephen no longer has time to continue his work except to guide others.
Aleix is still interested in working on a solution but that makes sense
only after we all agree on an achievable one.  Is anyone else interested
in working on this?

Stephen Kelly started this discussion thread with the following summary
of the advantages of the daemon approach over the generated JSON files:

On 01/10/2016 06:10 AM, Stephen Kelly wrote:
> This design is independent of any generator, and compared to the
> solution of writing json to a file, this solution doesn't require
> generating anything that is not asked for.
> This also side-steps the problem of 'stale' files remaining
> in the build directory, and the problem of needing to trigger re-generation
> of the file, or watching it for changes.  This solution also allows
> parameters (eg the config) to be specified in requests, side-stepping a
> difficulty we encountered before to represent things like that.  This
> solution also does not require actually generating the buildsystem
> files. Only the configure and compute steps of cmake are executed.

Given the above summaries my reaction is that the daemon approach is
more elegant and desirable in the long run.  However, it will require
much more time and resources to implement.  It is not clear how much
more, or if anyone has such time and resources to offer.

An intermediate solution could be to offer the daemon and protocol
to IDEs but to implement it internally on top of pre-generated JSON
files.  The schema of the JSON files would remain a private contract
between "cmake" and "cmake-daemon".  Then over time the implementation
of the daemon could be improved by computing things internally on
demand instead of updating and reading the JSON files, but the public-
facing protocol would not change.  We just need to make sure that the
protocol is designed to allow such flexibility.  OTOH this approach
requires designing both the daemon protocol and the JSON format.

-----------------------------------------------------------------------------

Note that the generated JSON approach is essentially what we've been
doing with the VS IDE since CMake started, just with a different format.
Many developers use the VS IDE to develop their CMake-built projects,
and they get IntelliSense for completion.  It has worked well for years.
Perhaps the argument for the daemon approach puts too much focus on the
updating case while most developer time is simply spent editing existing
sources in their IDE.

-Brad