KIMAP2

[Christian Mollekopf]

Hi,

I have largely completed the transition to KIMAP2 and will thus retire
KIMAP. Since I know you are probably continuing to use KIMAP at least
for some time, I made KIMAP2 completely coinstallable and namespaced
everything into KIMAP2.
However I will only be able to maintain KIMAP2, and not KIMAP. If you
want to keep KIMAP around (I assume you do), I suggest we release it
from a stable branch "kimap", as I would like to move master to kimap2.
If that causes large problems with the way things are released let's
discuss alternative solutions.

In any case, I suggest to move to KIMAP2 in the long run.

So, here's what's changed with KIMAP2:

* KIMAP2 is now threading free:
There no longer is a dedicated thread for each socket. The reasoning for
this is manyfold:
    * The threading strategy should be up to the application. The
    current design imposes additional threads in any case.
    * The threading resulted in lot's of subtle bugs and code-complexity
    overhead for the synchronization.
    * The whole design revolved around the idea that by running the
    parser in a thread the parser can be blocking and the network will
    always be the limiting factor. That assumption is not necessarily
    true and the parser starts to behave very badly if we get data
    faster than we can process it (The internal buffer will get large
    and a majority of time will be spent on memcopying during the trim()
    call).
    * This design doesn't allow the consumer to regulate the amount of
    data that enters the system. It reads as much into memory as it can,
    which again assumes the network is the slowest part.
    * During benchmarking of Sink the parser actually became the
    bottleneck of the imap resource (because of the pathological
    behaviour with memcopy in trim()).
All of this directly leads to the next point.

* We have now a non-blocking parser:
    * The parser operates with two buffers to emulate a ringbuffer =>
    fixed memory usage (except for some dynamic allocation for large
    literals).
    * The parser never reads more data than it can process => If we
    can't process fast enough the socket buffer will eventually fill up,
    resulting in the server eventually stopping to send more data. Which
    is how the network stack is supposed to work IMO.
    * We open up possibilities for new streaming API to directly stream
    i.e. attachments to disk.
    * This resulted in the parser mostly vanishing from benchmarks and
    memcpy to vanish entirely.

* We no longer depend on ki18n:
    * This is a protocol implementation and not a place for translated
    strings.

* We no longer depend on kio:
    * This was used for ktcpsocket only, and the only remaining benefit
    of that was the KSslCertificateManager integration. This can easily
    be readded externally by implementing the Session::sslErrors
    callback and the Session::ignoreErrors function, as well as
    QSslSOckets default ca certificate functions. (We may want to extend
    the API there a bit).
    * KIO has a whole slew of unrelated dependencies so this was an
    important step to make KIMAP2 more selfcontained.

* I got rid of a lot of the result batching code:
    * It complicated the code and was an optimization at the wrong level
    IMO to save a couple of function calls.

* Less overloaded result signals with lot's of parameters. One signal
with a result struct containing all data.

* The login job has received an overhaul.
    * Removed the slightly confused EncryptionMode names that sometime
    mixed the excryption and the use of starttls (which are two largely
    unrelated things).
    * Cleaned up the login logic which was a very complex statemachine
    scattered accross different classes and is now only a complex
    statemachine scattered over fewer classes =(
    * Fixed a potential race-condition where we would send a CAPABILITY
    request before receiving the greeting, which some servers seem to
    handle anyways but some don't.
    * Removed the encryption negotation which is handled by QSslSocket
    according to the provided settings, and otherwise fails as it
    should.


There are some things left to do left and right, including some
extensions that I'll likely implement (NOTIFY comes to mind), and some
API cleanup,  but in general not much is standing in the way of a first
release.
I have not yet decided on a release date as so far as only Kube depends
on it so far.

You can find all code in the dev/kimap2 branch.

Cheers,
Christian