[Kde-pim] Moving Trojitá to extragear

Tue Nov 27 21:52:24 GMT 2012

> > but I think we could encapsulate that in the session pretty much
> > transparently to the users of KIMAP (the session could just silently
> > pipeline commands which are pipelinable from it's queue), and it would
> > certainly be a very welcome addition for all imap clients.
> 
> That would be a layering violation from my point of view. It is actually a
> good design decision to separate parsing of responses from keeping track of
> the state -- and you have to keep track of the state at the place which
> decides what to pipeline and what not.
> 

Sure, on the other hand the session does exactly that (as you noted below) and 
is well separated from the parsing. I'm just wondering why you think it 
wouldn't be possible to implement pipelining (relatively easily) in that 
session, as that might be something I'd want to do eventually (as you 
mentioned you got substantial performance improvements). So I'm just trying to 
learn from your experience here.

AFAIK pipelining is the following:
- You send a bunch of commands (which support pipelining), without waiting for 
a response
- You match the response to the sent command using the tag at the beginning of 
each command.

So in it's most basic form, shouldn't it be possible to start, for example, a 
couple of fetch-jobs, maintain a map for tag-job pairs, and match the results 
again to the corresponding jobs?

As I said, I have no experience with this, but I thought that's the theory and 
would be grateful for clarification.

> It looks like KIMAP in its present form combines basic parsing with the
> command - response cycle of IMAP. Trojita works differently, and I
> naturally consider Trojita's approach to be a better one. In particular, I
> don't know how to add pipelining to KIMAP without a redesign to let KIMAP
> do nothing but turning the stream of bytes into some data structures *and*
> moving the logic of the jobs to an upper layer. (This is not suggesting you
> to redesign KIMAP -- again, I'm merely saying what I need from an IMAP
> library.)
> > Yes, KIMAP is all about sending and parsing commands, while
> > that is certainly
> > no everything required it's one of the essential building blocks and if we
> > could shared that codebase I think everyone would profit.
> 
> Here's a breakdown of the rough sizes of Trojita's core IMAP components:
> 
> - Parser is roughly 5200 SLOC
> - Tasks are roughly 5000 SLOC
> - Model is roughly 8000 SLOC
> 
> ...which means that I was wrong, I was estimating the size of the parser to
> be something like 10% of the IMAP-related code; in reality, it's something
> like 25%. (Please note that the numbers are very likely skewed, the code
> for response dispatching and tasks management/lifecycle/... is actually
> inside the Model etc etc.)
> > Especially with you in-depth knowledge of IMAP I think we could improve
> > our
> > codebase in many ways and make it ready for the future. We'll
> > need a fast and
> > reliable imap implementation for the time coming =)
> > 
> > If that means we ditch KIMAP and use something entirely
> > different or improve
> > what's existing with what we need is open for me, but I hope we can avoid
> > having multiple complete implementation of the same (I'm not proposing to
> > ditch KIMAP just yet ;-).
> 
> Let's be realistic here -- while I agree with you that it would be great if
> there was just a single IMAP implementation in KDE (well, technically, it
> is, Trojitá is not "in" yet and it definitely isn't a reusable library with
> a reasonable interface), someone would have to do the work.
> 
> KDE's implementation works, and is apparently good enough for Akonadi, KMail
> and other users. Changing its interface (which I assume would be required
> in order to bring features like pipelining in) would cause trouble to all
> these applications. I have a limited time and do not intend to work on
> KIMAP or Akonadi's IMAP code simply because that would not bring any
> benefits to me or to Trojitá.
> 
> On the other hand, Trojitá cannot be built on top of KIMAP at all due to the
> lack of KIMAP's features (provided I'm correct about the pipelining issue).
> Even on the second-lowest layer, right on top of the parser, there's a huge
> difference in the format of data we work with -- compare for example the
> representation of the FETCH response in Trojitá [1], their parsing [2] and
> processing [3], to the way how KIMAP handles it at just a single place [4]
> -- that's the difference between working with structured data vs. dealing
> with lists of byte arrays. I'm not suggesting that KIMAP's approach is bad,
> Python's imaplib works in the same way -- but it is not something I want to
> use in Trojitá.
> 
> I quite naturally prefer to stick with code that I'm familiar with (and
> which is reasonably tested in real world and covered by a test suite). At
> the same time, the Trojitá's interface is nowhere near ready to offer a
> stable API as a library, and the whole IMAP code is licensed under the GPL,
> not the LGPL, which makes it unsuitable as a KDE library. In short,
> Trojitá's IMAP code cannot really replace KIMAP as-is without quite some
> effort from someone.
> 
> So while in theory it's a perfectly obvious suggestion to have a single IMAP
> library, unifying KIMAP/Akonadi's IMAP resource/... with Trojitá's IMAP
> implementation is something which would require months of work.  If someone
> is willing to do that, I'd be happy to accept patches which make Trojitá's
> code more modular or easier to use in libraries -- but I don't have the
> resources to be the driving force behind this work :).
> 

Understandable, I suppose that's the same for everyone. And it's also clear 
that it would be a substantial amount of work to start sharing now. 
Nevertheless I hope to learn from your findings to improve the rest of our 
IMAP related software stack =)

Cheers,
Christian

> With kind regards,
> Jan
> 
> [1]
> https://projects.flaska.net/projects/trojita/repository/revisions/master/en
> try/src/Imap/Parser/Response.h#L555 [2]
> https://projects.flaska.net/projects/trojita/repository/revisions/master/en
> try/src/Imap/Model/MailboxTree.cpp#L333 [3]
> https://projects.flaska.net/projects/trojita/repository/revisions/master/en
> try/src/Imap/Parser/Response.cpp#L871 [4]
> http://code.woboq.org/kde/kdepimlibs/kimap/fetchjob.cpp.html#226
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20121127/89a16b3c/attachment.sig>
-------------- next part --------------
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/