[Konsole-devel] shared terminal emulation library

[Lars Doelle]

Hi Jekyll,


> A short introduction: I started contributing to konsole about one year 
> ago.  Most of the fixes and improvements done by me is on the surface, 
> not the core emulation part. So I'm afraid I can't give much technical 
> comment. But still, I would like to say something :)

From my side, i did very first version of the konsole, i think in 1996/7.
I guess i maintained it for 2 years then, but constantly taking care of
90-point bug/wish-list was not possible for me. Thus Jekyll, thank you
for working to keep the konsole up-to-date and kicking.


> I actually from time to time tried to fully understand and document the 
> emulation code in detail , but never really managed to do so. So that 
> part scares and worries me . Although those code is stable as reflected 
> by bug reports on b.k.o, it doesn't feel good when you don't really 
> understand the small core part of the project to which you are 
> contributing .

This might be fixed during the process of extraction, actually. Some good
documentation of this part is overdue, some existing docs checked into
the kde repository might have been deleted...

 
> Second, I'm not that optimistic with the idea of making that library 
> reusable for other projects without losing some konsole features or 
> causing some konsole regressions.
> 
> Konsole code has a long history and I am involved for only one year, but 
> from my experience of fixing a few bugs which need to touch the 
> emulation part, I feel some high level konsole features are strongly 
> coupled with some low level konsole specific emulation features.

I'm not scared about that. Actually, virtual all work on the konsole, and
in particular, Robert Knight almost fully rewrote it, was on the surface,
not in the emulation. As i wrote the emulation, i believe i'm pretty aware
what has been changed and what not.

Additionally, i considered cleaning up the emulation from time to time as
some QT-hacks, i.e. code not respecting the layering, crept in. This is
pretty natural for an open-source project, were many authors try to
improve matters from their particular point of view and interest, clearly
focussing on one particualur extension.

There might be one or two points, i saw, when i looked at the code the
last time, and one is support for asiatic fonts, which uses a particular
input trick supported by QT, which i'm not fully aware of. But i believe
that could be managed by writing to the respective author.

Another point is, that someone (Robert?) "rewrote" the emulation by
renaming the variables and procedures, which makes it a little more
difficult to preserve actual changes. But this should not be such a big
issue, since one can filter the changes out, if any, by a sed-script and
diff. A manual line-by-line review of the code will be necessary, anyway,
too.

 
> One example is the "\e]50;\a" escape sequence use by the konsoleprofie 
> script to change konsole settings in the runtime. If I remember it 
> correctly, that sequence is used for changing font in xterm. If the 
> library is to be reusable for other projects, how to deal with such 
> incompatibility ? Is there some planned spec for that library?

Absolutely. (Sad truth is, that the various xterm projects do not talk with
each other. Thomas Dickey <dickey at his.com>, who maintains xterm
and curses has the habit, to add or change some esc-codes from time
to time without any coordination. So much for "\e]50a".)


> Also by glancing the current code at https://github.com/doelle/libemu, 
> it seems to be based upon some older version. Is that repo just a demo, 
> or will it be the start point of the proposed library ? If the latter 
> case I'm really worried.  Although the emulation part is inactively 
> modified, it still receives a few fixes and improvements in the past year.

Whether this is the starting point or not is not yet decided. What clear is,
is, that for reasons of API/ABI compatibility, it will become a C project.

What the actually of the code w.r.t. to the stuff in konsole concerns, i do
not share your worries, really. I've followed the evolution of this part very
closely, and my summary is, that it is virtually unchanged w.r.t. to the
initial version through all the time.

The stuff in libemu is based on a snap in 2003, from which i removed all
QT material and some stuff needed for an xterm but not for an emulation,
most particular history handling.

As a last point, i'll not start this work unprepared, blindly or unwary. Of
course, factoring out the emulation and replacing it with something new
has the potential to break or miss something.

I'm not in hurry doing this though, i project a possible merge in 3 month
earliest, while previews of a modified konsole would perhaps be available
much earlier.


> This "reusable library" thing reminds me of the big rewriting of konsole 
> in KDE4 (which happens long before I become a KDE user).  That rewriting 
> is mostly about the surface, and I think Robert have done a great job in 
> making those code much better than before. But users just don't care 
> about internal code quality. The will only notice lost features and new 
> bugs. I have had it enough to hear users saying "KDE4 konsole is worse 
> than KDE3 konsole". I really want to be careful to not give users 
> another chance of shouting.

Yes, Robert did a great job! And yes, there were times where the konsole
was so badly broken, that i had to run xterm (2 or 3 month). Guess David
and i (and perhaps you) can do better.


> Finally, I think the general rule is "Those who have done or would do 
> the work make the decision". There is no real/valid objection from me at 
> the moment. If taking the release schedule of KDE SC into account, this 
> library thing is unlikely to catch the deadline of 4.10. So there are a 
> few months before it really makes sense to discuss and decide whether 
> this should happen for konsole in 4.11 or later.

Actually, w.r.t. to freeware projects, i don't care about deadlines. Work is
done when it is done. What i propose here is to rework the emulation after
15 years or so, and i feel, this rework is due.

In particular, factoring this part out, does not only allow to reuse it in other
projects (i did two or three such reuses over the time), but also to focus
clearer on this part, which, as you wrote, is the ugly duckling of konsole.


Additionally, i have another reason to do so. The given state of stuff of 
terminal emulations under Linux is such, that since the overall pipeline is
distributed over many very different projects, e.g. kernel, curses, shell,
libreadline, etc., /major/ improvements on the user experience are not
possible without changes involving all such projects at least a little.

This is very vaguely indicated in the mail "terminal emulation. (various
points)", in which i wrote:

> Terminal-wise, i wonder from to time, if it would not be possible to
> do a step ahead and redefine the (use of the) protocol. My cloudy
> idea is that one could more cleanly separate the fullscreen mode
> from the command line mode and extend the support of the later.
>
> One could then for instance provice features as separting the
> stdout/stderr with different colors, honor the return codes, fold
> long lines in the history (which have arbitrary length) and improve
> support of the Unix job control (i.e. Stevens' Unix  programming
> Chapter 9) extend session management to command line applications,
> etc. pp.

Right now, i'm writing on a paper making this points more explicit. If
successful, it could really allow to go a step ahead and really improve
the user expericence and functionality of the konsole.

Jekyll, as you're focussing on the graphical stuff, you might really be
interested in this topics, a the graphic presentation would be affected
and perhaps some complete new, yet unknown forms of dialogs within
the konsole window would become possible and necessary.

This might actually be something breaking terminal emulation tradition
while maintaining full backward compatibility. This is not yet specified,
but will become clearer as soon as i put out the paper. I hope, you and
others on konsole-devel can contribute with ideas, critiques, counter-
proposals and perhaps graphical code. This all depends a bit, how
things work out, but i'm dedicated to work towards the shared lib, at
least.

As said, i'm not in hurry, but find this work overdue, now.


Kind regards, Lars