CMake daemon for user tools

[Aleix Pol]

On Sun, Jan 10, 2016 at 12:10 PM, Stephen Kelly <steveire at gmail.com> wrote:
>
> Hello,
>
> I've been working on adding a daemon mode for cmake to provide
> information to user tools - such as IDEs - about the buildsystem.
>
> Following the discussion about providing metadata for IDEs to consume
> I proposed creating a long-running process which would provide a protocol
> to access information about the buildsystem, and about the content of the
> cmake files themselves:
>
>  http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=13004
>
> The original post in that thread lists design goals.
>
> This design is independent of any generator, and compared to the solution of
> writing json to a file, this solution doesn't require generating
> anything that is
> not asked for.  This also side-steps the problem of 'stale' files remaining
> in the build directory, and the problem of needing to trigger re-generation
> of the file, or watching it for changes.  This solution also allows
> parameters (eg the config) to be specified in requests, side-stepping a
> difficulty we encountered before to represent things like that.  This
> solution also does not require actually generating the buildsystem
> files. Only the configure and compute steps of cmake are executed.
>
> I am scheduled to give a talk at FOSDEM about the feature and how user
> tools can interact with it:
>
>  https://fosdem.org/2016/schedule/event/enabling_gui_tools_for_cmake_code
>
> with a preview in Berlin:
>
>  http://www.meetup.com/berlincplusplus/events/227896427/
>
> I have now also pushed a branch to my github clone with the start of the
> server mode:
>
>  https://github.com/steveire/CMake/commits/cmake-daemon
>
> Currently the branch only makes targets and backtraces etc available via
> the protocol.  I have also created a plan for extending the protocol to
> make code completion and variable debugging possible, and analysed the
> depedencies of the tasks:
>
>  http://www.steveire.com/cmake-daemon-tasks.png
>
> However, I can't complete those tasks myself: I don't have
> relevant experience building IDEs to know how best to design the
> protocol, what IDE tools really need, how to design a fail-safe parser
> etc.  Additionally, I think a design which incorporates design ideas
> and implementation from more than one person will be better in the end.
>
> So, this needs to be a collaborative effort if it is to go anywhere, with
> more people writing commits in the cmake repo.
>
> The above (copied below) task list would be enough to get a read-only
> browser of a cmake project quite quickly.
>
> Following that, an effort would be needed to handle the difference
> between the dirty state of an editor and the on-disk file content. That's
> a solvable problem and together with filesystem notifications provided
> by libuv, it will lead to features like code completion.
>
> Also necessary eventually would be to make the cmake parser more
> fail-safe, so that information is still available if the user
> had not yet closed a command argument list for example, and make the
> processing more fail-safe in the case where they have written
> add_subdirectory(foo), but have not yet created foo/CMakeLists.txt.
>
> I would be looking to people who work on IDEs for guidance on how to do
> things like that, as some of them are likely already for handling C++
> code in the IDEs.
>
> So, this needs to be a collaborative effort. I can provide guidance to
> anyone who wishes to participate in changing the source code of CMake
> to make it possible.
>
> Is anyone interested enough in this potential cmake feature to
> join the development effort?
>
> Thanks,
>
> Steve.
>
>
> * Provide basic daemon mode with event loop and communication protocol
> -- Written, but not reviewed
> * Provide protocol to query the buildsystem structure
> -- Written, but not complete or reviewed
> * Extend cmListFileParser to record location of start and end of commands
> * Rewrite cmIfCommand to record which branches are executed
> * Store the location of start and end of user-defined functions
> * Provide protocol to query for unexecuted lines
> * Define a new snapshot type to occur at an arbitrary point
> * Create snapshots after branching to other files (after an include() or
> find_package() or add_subdirectory etc)
> * Create snapshots after points with external effects (file(WRITE),
> configure_file(), execute_process etc)
> * Find most recent snapshot occuring before a specified line
> * Implement feature to start at a specific snapshot and execute commands
> up to a particular line
> * Make server find correct snapshot to start at and execute required
> commands up to specified line
> * Provide protocol to dump variable defintions at a particular line
> * Extract implementation of IssueMessage from cmMakefile and
> cmLocalGenerator into cmake
> * Extract computed target properties from cmTarget::GetProperty into
> cmComputedTargetProperties
> * Implement cmGeneratorTarget::GetProperties in terms of
> cmComputedTargetProperties
> * Move implementation of cmTarget::{Get,Set,Append}Property to cmState
> * Extract all IssueMessage parts of cmake into a cmMessenger
> * Extract the cmSystemTools::Message part of cmMessenger into a virtual
> method
> * Publish messages through server protocol
> * Implement snapshot-versioning of target properties
> * Dump target property content
> * Implement snapshot versioning of policies
> * Implement snapshot-versioning of directory properties
> * Implement snapshot-versioning of global properties
> * Dump directory, global properties and policies
> * Extend protocol to provide dirty buffers from editors, instead of only
> executing on-disk commands
> * Provide code completion protocol

Hi Stephen, everyone,
I've already discussed this in private with you. I think it's a good
idea and I'd like to make sure we can benefit from this.

I'm unsure of the feasibility of the project though. Getting any
changes in cmake takes forever (this initiative started almost 2 years
ago already and it hasn't had any impact yet) and we still need to
proof the approach and make sure it has mechanisms to provide
retro-compatibility. At least to some extent.

At the very least, I'd like to see cmake maintainers committing to the
initiative, to make sure I don't waste more of my time.

Regards,
Aleix