[Kde-pim] Re: Moving Trojitá to extragear

Tue Nov 27 18:16:23 GMT 2012

On Tuesday, 27 November 2012 18:05:49 CEST, Christian Mollekopf wrote:
> Nice work with Trojita, it feels indeed very fast and I'm happy 
> to see there 
> is some development going on in this area =)

Hi Christian, thanks for your kind words.

> If the additional thread is necessary or not I don't know, but 
> it looks pretty 
> much like an implementation detail to me that could be changed 
> as we like (it 
> seems to make sense if there is a lot of parsing to-do).

Yes, it's an implementation detail -- I was just commenting on some of the differences, and this one was one of the first things I spotted. I'm in no way suggesting that it's a bad decision to use threads, even though I was bitten by threading issues back then when Trojita's parser was multithreaded :).

Performance wise, I have not seen any trouble running the single-threaded Trojitá on my N950 -- but it's got a 1GHz ARM core inside, after all. That said, there's one kind of response, the THREAD one, which is very time consuming to parse by the current code, and I might have to make Trojitá's parser multithreaded in order to make it perform better (or just come up with a clever algorithm of reversing the list parsing; the current one just sucks). We'll see.

> but I think we could encapsulate that in the session pretty much transparently to 
> the users of KIMAP (the session could just silently pipeline commands which 
> are pipelinable from it's queue), and it would certainly be a very welcome 
> addition for all imap clients.

That would be a layering violation from my point of view. It is actually a good design decision to separate parsing of responses from keeping track of the state -- and you have to keep track of the state at the place which decides what to pipeline and what not.

It looks like KIMAP in its present form combines basic parsing with the command - response cycle of IMAP. Trojita works differently, and I naturally consider Trojita's approach to be a better one. In particular, I don't know how to add pipelining to KIMAP without a redesign to let KIMAP do nothing but turning the stream of bytes into some data structures *and* moving the logic of the jobs to an upper layer. (This is not suggesting you to redesign KIMAP -- again, I'm merely saying what I need from an IMAP library.)

> Yes, KIMAP is all about sending and parsing commands, while 
> that is certainly 
> no everything required it's one of the essential building blocks and if we 
> could shared that codebase I think everyone would profit.

Here's a breakdown of the rough sizes of Trojita's core IMAP components:

- Parser is roughly 5200 SLOC
- Tasks are roughly 5000 SLOC
- Model is roughly 8000 SLOC

...which means that I was wrong, I was estimating the size of the parser to be something like 10% of the IMAP-related code; in reality, it's something like 25%. (Please note that the numbers are very likely skewed, the code for response dispatching and tasks management/lifecycle/... is actually inside the Model etc etc.)

> Especially with you in-depth knowledge of IMAP I think we could improve our 
> codebase in many ways and make it ready for the future. We'll 
> need a fast and 
> reliable imap implementation for the time coming =)
>
> If that means we ditch KIMAP and use something entirely 
> different or improve 
> what's existing with what we need is open for me, but I hope we can avoid 
> having multiple complete implementation of the same (I'm not proposing to 
> ditch KIMAP just yet ;-).

Let's be realistic here -- while I agree with you that it would be great if there was just a single IMAP implementation in KDE (well, technically, it is, Trojitá is not "in" yet and it definitely isn't a reusable library with a reasonable interface), someone would have to do the work.

KDE's implementation works, and is apparently good enough for Akonadi, KMail and other users. Changing its interface (which I assume would be required in order to bring features like pipelining in) would cause trouble to all these applications. I have a limited time and do not intend to work on KIMAP or Akonadi's IMAP code simply because that would not bring any benefits to me or to Trojitá.

On the other hand, Trojitá cannot be built on top of KIMAP at all due to the lack of KIMAP's features (provided I'm correct about the pipelining issue). Even on the second-lowest layer, right on top of the parser, there's a huge difference in the format of data we work with -- compare for example the representation of the FETCH response in Trojitá [1], their parsing [2] and processing [3], to the way how KIMAP handles it at just a single place [4] -- that's the difference between working with structured data vs. dealing with lists of byte arrays. I'm not suggesting that KIMAP's approach is bad, Python's imaplib works in the same way -- but it is not something I want to use in Trojitá.

I quite naturally prefer to stick with code that I'm familiar with (and which is reasonably tested in real world and covered by a test suite). At the same time, the Trojitá's interface is nowhere near ready to offer a stable API as a library, and the whole IMAP code is licensed under the GPL, not the LGPL, which makes it unsuitable as a KDE library. In short, Trojitá's IMAP code cannot really replace KIMAP as-is without quite some effort from someone.

So while in theory it's a perfectly obvious suggestion to have a single IMAP library, unifying KIMAP/Akonadi's IMAP resource/... with Trojitá's IMAP implementation is something which would require months of work.  If someone is willing to do that, I'd be happy to accept patches which make Trojitá's code more modular or easier to use in libraries -- but I don't have the resources to be the driving force behind this work :).

With kind regards,
Jan

[1] https://projects.flaska.net/projects/trojita/repository/revisions/master/entry/src/Imap/Parser/Response.h#L555
[2] https://projects.flaska.net/projects/trojita/repository/revisions/master/entry/src/Imap/Model/MailboxTree.cpp#L333
[3] https://projects.flaska.net/projects/trojita/repository/revisions/master/entry/src/Imap/Parser/Response.cpp#L871
[4] http://code.woboq.org/kde/kdepimlibs/kimap/fetchjob.cpp.html#226
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/