RFC: support for building on remote/virtual/etc machines

[David Nolden]

Hi Friedrich,

I think that this usecase is much too complicated to be covered by the
GUI; maintaining a solution which would work for the majority of all
usecases would be close to impossible, because such usecases are very
customized. I think support for something like this should rather be
implemented by means of very simple features in the shell integration
support.

For example, in the shell integration, I already implemented support
for connecting to a remote machine through an ssh tunnel, by using the
"ssh!" command. What you need to do is copy the
kdevplatform_shell_environment.sh script and the
kdev_dbus_socket_transformer tool to the target machine, and ensure
that they are in the path, and that kdev_dbus_socket_transformer is
functional (i.e. its linkage isn't broken). Also the qdbus utility
must be available on the target machine. Then the remote terminal can
communicate with the local KDevelop instance, by calling all the
integrated commands ending with "!". For example, if you're on a
networked file system where files are mounted in the same location,
then the "o!" command will open the file on your local machine,
otherwise it will try to use fish. It would be nice if we would
additionally have some optional path mapping here from remote/local.

Anyway, the question now is, what would be missing to make this really
convenient. In my opinion, two things would be very useful:
1) Being able to get the build logs parsed by KDevelop, so that you
can read the colored log output within KDevelop and directly jump to
errors etc. For this, it would be nice if the output could be piped
into KDevelop in real-time. This would not only be useful for remote
building, but also for local building with build systems that cannot
easily be integrated into KDevelop. Such a call could look like "make
| parse!", where "parse!" would be a KDevelop shell utility that pipes
all the output into KDevelop and tries to do something useful with it,
like highlighting errors and making them browsable etc. For the start,
it might be sufficient if the final build log could by processed by
calling "parse! filename", which would be very similar to "open!
filename", and very easy to implement.
2) Being able to attach the integrated debugger. This should also be
implemented locally. It should somehow pipe the GDB control interface
between KDevelop and the debugger instance.

An even better solution for 1) would be, if KDevelop could monitor all
the output in all its attached sessions even without the "parse!"
command, and would recognize, for example, when you're using gdb
manually to directly jump to the specific source locations etc.,but
this might be quite difficult to achieve because I haven't found any
way to support this yet (except for hacking Konsole directly).

Greetings, David

2016-09-13 2:37 GMT+02:00 Friedrich W. H. Kossebau <kossebau at kde.org>:
> Hi,
>
> first of all: congrats for the 5.0 release of my favourite IDE, dear
> maintainers :) So happy KDevelop exists and keeps improving.
>
> Now, there is one issue that bugs me for years, so I want to try to get
> serious with it myself over the following months:
> being able to build (and/or deploy) on machines other than the one where I run
> KDevelop on (e.g. my laptop).
> Ranging from real remote machines on the other side of the planet to virtual
> local ones (VirtualBox & Co). Or just building as another user on my machine,
> so my normal user account's files are protected from some build tool/script
> going crazy on the home directory accidentally.
>
> So some reasons for doing remote builds:
> * other machine is more powerful (or even a supercomputer)
> * machine has custom setup
> * special build tools only available on that machine
> * protection of local system from buggy installation/deployment
> * saves energy/disk/memory of my laptop, no noise by fan (if real remote)
>
> Looking at current KDevelop (from master to be sure), I found some support for
> remote systems in the Launch Configurations, incl. Remote debugging. So for
> the last part of configure/build/install&deploy/run the concept of non-local
> systems exists as a start. I want to work on extending this to the complete
> set of build actions over the next months.
>
> For this work I would keep the assumption of KDevelop that the complete
> sources are accessible via the local filesystem and shared with the remote
> systems where the build should happen by the usual ways (mounted remote/
> network filesystem or shared directories with virtual machine). So any editing
> of the sources should not be touched in this work.
>
> But any of the steps build/install(&deploy)/run/debug, they all in the end
> should be configurable to be done on other, remote/virtual systems.
>
> Would you welcome such work? Who else would like to work on this with me? Has
> there been some discussion on this already, any links to notes/threads? Any
> previous attempts?
>
> First step would be collecting use-cases, for developing an abstract idea/
> model of the different "build" actions (configure/build/install&deploy/test/
> run/debug/?), the different target systems and the different communication/
> control channels to them.
> What naming pattern would you like below https://community.kde.org/KDevelop
> where I would like to do some wiki pages for organizing this work?
>
> Cheers
> Friedrich