[dot] Interview With KDE-PIM Hacker Till Adam

[Dot Stories]

URL: http://dot.kde.org/1116452031/

From: Arjan van Leeuwen <>
Dept: kontact-us-now
Date: Wednesday 18/May/2005, @23:33

Interview With KDE-PIM Hacker Till Adam
=======================================

   Till Adam only started hacking on the KDE mail client, KMail, because
he wanted some features implemented from the command-line client Mutt. 
Now he is one of the main developers of the KDE-PIM project
[http://kdepim.kde.org/], which KMail is part of.   KDE Dot News caught
up with Till to talk about e-mail protocols, groupware and implementing
them in KDE.

     Coming from Laboe, Germany, he is due to visit the NL-PIM meeting
[http://kdepim.kde.org/development/meetings/nlpim1/index.php] scheduled
for May 27-29 in Annahoeve [http://www.wallsteijn.nl/]. During this
meeting, sponsored by NLnet [http://www.nlnet.nl/], Trolltech
[http://www.trolltech.com/], NLUUG [http://www.nluug.nl/] and many more
sponsors [http://www.kde.nl/agenda/2005-kdepim-meeting/sponsors_en.php],
he will work with his fellow developers on improving KDE's PIM
applications and framework. During his stay in the Netherlands for
NL-PIM he has found the time to do a presentation at the NLUUG Spring
Conference [http://www.nluug.nl/events/vj05/] on using IMAP servers as
groupware storage and, together with Kopete developer Will Stephenson,
he will give a lecture at HollandOpen.
 [http://www.hollandopen.nl/]
     There is also a Dutch version
[http://www.kde.nl/agenda/2005-kdepim-meeting/interview-till.php] of
this interview.


     You've done a lot of work on the IMAP protocol in KMail. What are
the main differences between this and POP3 which most people still use
to get their email?
 Till Adam
     POP3 is a protocol for fetching mails from a server to a client.
It's then normally stored on the client and removed from the server.
Like a real-life mailbox, the server holds the mail until the user
fetches it. Whereas with IMAP the mail is stored on the server and
clients only retrieve the mails they need when opening them. The client
only stores caches not the data itself, it queries the server for
everything.

     KMail also has support for disconnected IMAP. What's the difference
in operation between IMAP and disconnected IMAP?

     With disconnected IMAP you have a full copy of the mail stored on
the client. You're able to carry it around with you and access it
without a network connection to the server.

     Could you say that disconnected IMAP is using IMAP as if it were a
POP3 server?

     No, it's very different. Disconnected IMAP synchronises what is on
the server with the client, but the stuff stays on the server. So now
both the client and server have identical copies of the data. Then you
can change stuff locally without a connection to the server such as
deleting mails or moving mails between folders. When you are connected
to the server KMail can synchronise again and all your changes are
uploaded to the server. All changes on the server that happened in the
mean time, such as new mail arriving, are synchronised with the client.
After the synchronisation, both client and server are identical again.
This is unlike POP3, where you get your mail once, and then delete it
from the server.

     How did you get interested in the IMAP protocol?

     I had been using it for a while since I'm running my own mail/web
server at a colo facility with a friend. I wanted to be able to access
my mail from several clients and machines - IMAP nicely provides that,
and it can even do it in a secure manner with SSL/TLS support. Support
for IMAP was a fairly recent addition to KMail at that point and it was
not really mature yet. Carsten Burghardt (who still maintains kio_imap4
and works on KMail as well) had done a great job of getting it into
shape already but there was still much to be done. KMail was not in a
very good shape overall at the time. In particular, it had stability
issues. I'm not claiming I cleaned that up all by myself, but I do think
I brought some new energy into the team and KMail has nicely improved in
the last few years. The attitude towards it, even inside KDE, changed
quite a bit.  It went from "our crashy mail app" to "KMail, it works
well and reliably".

     How important is IMAP for an enterprise mail application - and
especially for a complete groupware solution?

     IMAP is crucial in the enterprise context. IMAP is what is used on
corporate desktops. POP is mostly only used for road warriors, if that.
It makes sense for enterprises to have the data on the server. It can be
centrally backed up, maintained, filtered, etc. So decent IMAP support
and performance is rather important.

     What difficulties did you have in implementing KMail's IMAP
support?

     IMAP is not terribly easy to implement because it's a very powerful
protocol. In KDE we have this idea of encapsulating protocols in KIO
slaves [this makes it possible to access different protocols in a
generic way from many different applications - ed]. This works very well
for protocols like HTTP where all transactions can be described as
"request something by name, wait, get it, do something with it". It's
all nicely serialised and maps nicely to file operations: things like
open, close, read, write and delete. But the IMAP model is different.
You basically just bombard the server with requests, in no particular
order. And then you get a bunch of responses back, as quickly as the
server can give them to you, again in no particular order. This can be
very fast. But to use IMAP as a KIO Slave we had to serialise all
operations. So it's: fire a request, wait for the response, do something
with it, fire the next request, wait for the response, etc. That makes
our IMAP implementation inherently limited, speed wise. Getting it to
perform nicely was therefore somewhat of a challenge.

     It seems to be very fast in KMail's most recent release, 1.8 
included with KDE 3.4.

     Well, we (George Staikos, myself and also Carsten) spent quite a
bit of time profiling and optimising it leading up to 1.8. Disconnected
IMAP is especially a lot faster, but online IMAP is improved as well. I
think there's always room for more improvements.

     Featurewise, KMail is doing quite alright these days. We have some
nice features, such as server side searches, annotation support, ACL
support, etc. Some of that came from the Kolab [http://www.kolab.org]
and Aegypten [http://www.gnupg.org/aegypten/] projects.

     What do you think of the rise of so-called 'rich' web clients,
Google's Gmail comes to mind? Do you think they can ever replace
traditional mail clients? Do you see them as competition for KMail?

     I think it's partly a different use case. For certain workflows,
those interfaces are just fine, even superior to something like KMail in
some ways. Much cleaner, less clutter, fewer options, and a nice
interface. But Gmail has just a very small fraction of the feature set
of KMail, Evolution or other full email clients. So I think there will
always be users who just need or want that power and flexibility. Of
course they are competition. I personally think we can learn a lot from
the web interfaces. For non-powerusers many of our interfaces are way
too complicated. We hope to change that, though, with the help of the
OpenUsability people, for example at the NL-PIM meeting. Ellen and Jan
from OpenUsability [http://www.openusability.org/] will be there to work
on that with us. So I'd say Gmail is a challenge, in some ways, but we
are not afraid of dying out just yet.

     Do you think that a more slick user interface is important for
enterprise usage as well? Would you consider having separate 'modes' in
KMail for simple and advanced usage?

     Yes, enterprise users have different needs, but we could do better
interface wise there as well. As for having a 'guru mode', as I like to
call it, I don't really know. It would feel like giving up to me to have
two user interfaces. Some days I think it's the way to go, but I still
have that lingering hope that we can actually make interfaces that are
straight forward for casual users but still powerful and discoverable so
that people can grow with it and explore things when they need more
features. Let's see if the closer cooperation with some excellent
usability experts helps us achieve that. It's great to have them
involved now.

     Has all the activity around Kontact, KDE's PIM solution which
includes KMail, help to improve KMail?

     Of course. The individual parts of Kontact benefit from each other,
and it also brought us some new users: people who want an integrated
solution rather than standalone applications for email, calendaring,
address book, etc. There are many ways in which the individual
applications interact nowadays. Things like KMail using KOrganizer
functionality for rendering invitations inside emails.

     It's incredible that all these applications work so well together -
Kontact really looks like one application. Were there any difficulties
in getting the applications that make up Kontact to get along?

     Well, there were technical challenges, sure. But since the KDE-PIM
developers have a very good team spirit, and get along great, it was
just a matter of sitting down and agreeing on how to do things. I think
most developers involved with KMail or KOrganizer and the other
applications consider themselves KDE-PIM developers these days. The
applications are just focus areas if you like.

     Has KDE's famously rich development environment, with its existing
libraries, infrastructure, documentation etc., helped you in the
integration of KMail into Kontact?

     Oh yes, majorly. Without our powerfull infrastructure it would not
be possible to get the level of functionality and integration with the
very limited resources we have, in terms of manpower. It's sometimes
astonishingly easy to get things done with the tools we have available.
Of course we've also hit limits of the current technologies, but that's
good, that means we can improve them. This in turn benefits all of KDE.
We are proud of the fact that we've managed to maintain the ability to
run all applications standalone without losing any of the integration.
You can have KAddressbook and KMail running side by side and they will
still interact just as well.

     The well-documented code might also help companies to make local
customisations to Kontact.

     We try to provide extension possibilities, so third parties can do
their own Kontact parts, for example. If you want to plug your third
party application into Kontact and have it interact with the other
parts, you can do that. Or if you want to display calendar info from a
custom database or an application exporting its stuff via SOAP, you can
do that as well, by writing a calendar resource.

     Do you feel that KMail or Kontact is best combined with Kolab
[http://kolab.org/] as a sister project to KDE?

     It's the server project that has worked with the KDE folks the
closest and it's probably the one which works best with Kontact. This is
because Kolab as a project has people with KDE ties involved. Of the
three companies behind Kolab, Intevation, Erfrakon and KDAB, two are run
by KDE people, and KDAB employs several KDE and KDE-PIM developers. So
naturally Kolab is very nicely integrated.

     I also feel that it's important that the newer versions of Kolab
allow users of different platforms (Outlook on Windows) to save their
data in the same format as KMail & friends, making a slow transition to
KDE a lot easier.

     Yes, that's an important aspect of the Kolab concept. If we hope to
offer people ways out of the Outlook trap a server solution will have to
be part of it, and one which allows people to transition step by step.
We can of course never catch up with Outlook when it comes to Exchange
support.

     You mentioned that having some commercial support has helped KMail.
Do you think it would be useful for companies to invest in more fulltime
KDE hackers?

     Of course having people like David Faure hacking fulltime on
KDE-PIM for a while is a great thing. The libraries and applications
have benefited greatly from that. I do feel that companies would do well
to sponsor development of the things they need. My motivation to add
enterprise features I don't personally need in my spare time is rather
low, which is probably understandable. But if I can do it during paid
time, I'll be more than happy to; it's a code base I like to work with,
after all. And I do enjoy seeing KMail improve, of course. I've also
spent quite a bit of time on KMail and KDE-PIM by virtue of working on
Kolab support for KDAB. KDAB will of course gladly implement whatever
people need (shameless plug: at reasonable rates, e-mail info at kdab).
 [http://kdab.net/?page=contact]
     Q: What do you feel are the advantages of using a
KMail/Kontact/Kolab setup above a MS Outlook/Exchange setup?

     Freedom, above all. There are technical benefits as well, of
course, but they are maybe less important. While Exchange is more
feature-rich in some areas, for very many setups and needs, Kolab is
more than sufficient. A lot of development has gone into Exchange of
course, and they do get some things right. But for the vast majority of
users many of the features of Exchange are useless. Kolab will be a
better fit there, I'm sure, and at a price that can't be beat. And
that's not even talking about the vastly smaller hardware and
infrastructure needs.

     In what areas would you like to see improvements to the KDE family?

     Exchange's web interface is pretty good, I think. Horde is a web
interface that can be used with Kolab and is getting there, but it's not
on the same level yet. Of course, Kolab is vastly more platform
independent - Exchange's web interface only works fully in Internet
Explorer. In general, I think we still have to improve when it comes to
user interface polish. Outlook has many little convenient details.

     Sounds like there's still something to do at the NL-PIM meeting. We
wish Till and all the other people that will visit NL-PIM good luck and
a good time in the Netherlands!