RFC: Live document preview plugin (technical preview (pun-o-pun))

Mon Aug 21 15:52:33 UTC 2017

Hi,

[[cc:kdevelop-devel only for heads-up, please only reply to kwrite-devel]]

who is in for having a live preview when editing some Markdown file? Or Dot 
graph sources, SVG sources, Qt UI sources, you name it...

Seems some people are in, including me. And since last WE I have ended up 
with a simple prototype, which has served already with some real world 
markdown editing :) 

Get an idea and see examples:
* Markdown file: https://share.kde.org/index.php/s/B5imqnzmU3MkI8p
  (KDev, using new Markdown KPart, kde:scratch/kossebau/kmarkdownwebview)
* Qt UI file: https://share.kde.org/index.php/s/BLHvh4fG242SVyd
  (KDev, using kuiviewerpart)
* SVG file: https://share.kde.org/index.php/s/sfK4X8eOLqbh7Dp
  (Kate, using svgpart. could see improvements)

Get the code and try yourself:
	git clone kde:scratch/kossebau/ktexteditorpreviewplugin

See also
https://cgit.kde.org/scratch/kossebau/ktexteditorpreviewplugin.git/about/

And while I myself want this feature for KDevelop, doing the plugin based on 
KTextEditor::Plugin extends the userbase to Kate users. By the price though 
that it is restricted to source files that are KTextEditor::Documents. Which 
works for me (tm) for now.

With that said, now to the section where I want your comments on:
I made a few more architecture decisions which I had to reason about to 
myself, so would like you to do the same and then tell what you think.
Sorry for the braindump, but there is no other place for it :)

Using KParts plugins:

In my perfect world, any rendering of the target format (like SVG) would be 
done from a target document built over the source document, enriched with 
tags with info which parts of the target document are created from which 
parts of the source document, so there could be a mapping. E.g. to highlight 
the counterpart in the source/target when selecting something. And of course 
the target model would also support multiple target views.
Just, all that would need people coding this first for any document type. And 
right now we do not even have simpler plugins/kparts for all interesting 
types (like I had to write a quick'n'dirty kpart for markdown). So for a 
pragmatism-driven initial solution, going for kparts with their single-model-
single-view/widget design and the existing implementations solves quite a few 
use-cases now.
And reusing kparts instead of own preview plugins will result in sharing 
improvements made to the individual kparts with everyone else using them.
But also see below on some kpart-using issues.

Making it a tool-view plugin:

Given the MDI UI for text documents in Kate & KDevelop, together with the 
options for split views, there might use-cases to have previews for more than 
one document at the same time. One option could be to have the preview as 
part of the document view element, like the text-search-bar or the I/O 
message boxes. The other option is to have a tool-view which shows the 
preview of the currently focussed document (not sure if there could be 
multiple ones per tool-view type?).
Being familiar with writing tool-view plugins and a tool-view not needing to 
write additional code for showing/hiding the preview or selecting the 
position, I for now implemented it as tool-view. A lock-to-document toggle 
option allows to cover the use-case of having preview for a given document, 
while switching focus to other document views. Surely could have different 
solutions one day.

Correctly working only for single source document -> single target document:

There are target documents which make use of multiple source documents (e.g. 
HTML page pulling in CSS files). There could be some (k)io-mechanism to 
connect to other loaded documents and get their live data, but that needs 
some infrastructure work first. As there is no metadata in kparts available 
to filter out such multi-source document cases, there is no protection 
against partially broken previews right now, as e.g. can happen with HTML 
files. Not sure what to do here, so done nothing :)

Avoiding to stress the filesystem:

Best preview often is continuous live preview. And when one types a letter 
and wants a preview update, ideally only the one-letter diff would be passed 
to the preview module and that would update the preview accordingly.
As a matter of fact though all kparts currently have no idea about the 
KTextEditor working memory model (i.e. the KTextEditor API). For I/O of 
documents/files they themselves only support reading complete blobs from the 
filesystem (with KParts module transparently handling remote urls via 
temporary files passed to the actual kpart plugin). While the KPart API has 
some alternative streaming methods (ReadOnlyPart::openStream() & Co) as 
option to implement, seems no (interesting) kpart supports it (cmp. https://
lxr.kde.org/ident?_i=doOpenStream).
With the markdown kpart I am already using the streaming API. For any other 
kpart that ideally should be added as well, not sure why there is no 
primitive default implementation instead.
Right now the preview plugin first tries the streaming API (as only the call 
to openStream() would tell if streaming is supported?) and then falls back to 
passing the data blob via a QTemporaryFile.
Not sure how much of an issue that is.

Knowing what preview is wanted:

Right now selection of the kpart to use for the preview is done based on 
"KTextEditor::Document::mimeType()". Which only seems to work for documents 
which already are stored as file in the filesystem.
Any proposal how to fix this for new documents not yet saved? Something based 
on currently selected syntax highlighting?

Selecting KParts plugin to use:

Using the mimetype string, the preview code currently simply queries the 
currently configured preferred service:
"KMimeTypeTrader::self()->preferredService(mimeType, "KParts/ReadOnlyPart");
Which might be an issue in non-Plasma workspaces, given they do not have any 
configuration options for kparts in their system settings UI.
Perhaps also people would like to prefer a different kpart for the live 
preview? Needs more thinking how to enable any configuration.
Next there is also the issue that with the mimetype "text/plain" we get the 
kate part as preview module :) As first primitive approach the code for now 
simply filters out any kparts which list "text/plain" in their supported 
mimetypes. But that runs the chance of blocking also kparts which support 
importing plain text into whatever richer format they are about. Like the 
Okular plugin.
That might need some additional metadata in the kparts, something with the 
semantics of "native" and "imported" mimetypes? No clear idea myself yet.

Where should the preview plugin sources live?

The Preview plugin is similar to the Snippets plugin, usable both by Kate & 
KDevelop (as marked by ServiceTypes=KTextEditor/Plugin,KDevelop/Plugin).
Just, the Snippets plugin is right now part of the Kate sources. Adding the 
Preview plugin there as well is slightly unfortunate, as it both means for 
the developer working on the plugin they have to checkout and build the whole 
of kate (or hack the buildsystem locally) and for the user the chance that 
packagers put the plugin in a package which pulls all of Kate and its deps, 
even if ones only want the plugin for KDevelop.
So having a separate repo for plugins shared by Kate & KDevelop (and perhaps 
other KTextEditor apps?) would be nice to have.
I learned there once was a thread started on this ml discussing such a 
separate repo, on Aug 25, 2015, "Moving KTextEditor plugins out of kate.git?"
http://markmail.org/message/x7e5aq4c4he57pzk#query:+page:
1+mid:s4x3dl6ztbhjujpb+state:results
Time to continue it almost exactly 2 years later :)

Thanks for reading until here.

So far the feedback I have got on irc (#kde-devel/#kdevelop) on the current 
prototype was positive, and my own usage also showed it to be good enough for 
a first version. So besides the open questions I would be fine with the 
current approach to become an official plugin. 
But what do you Katies think about it? :)

Will also propose alternatively as patch for the kate repo, in case the 
separate-kte-plugins-repo discussion will take some more time.

Cheers
Friedrich