[Kde-pim] Review Request 120446: [IMAP] Split large STORE request into multiple smaller requests

[Jan Kundrát]

> On Oct. 3, 2014, 10:02 a.m., Jan Kundrát wrote:
> > FYI, this is what the RFCs say:
> > 
> > "A client should limit the length of the command lines it generates to approximately 1000 octets (including all quoted strings but not including literals).  If the client is unable to group things into ranges so that the command line is within that length, it should split the request into multiple commands." -- http://tools.ietf.org/html/rfc2683#section-3.2.1.5
> > 
> > This got recently upgraded on QRESYNC-compliant servers to read: "a client should limit the length of the command lines it generates to approximately 8192 octets (including all quoted strings but not including literals)" https://tools.ietf.org/html/rfc7162#section-4
> > 
> > As a random data point, Trojita doesn't attempt to do these bits. At the same time, we do use the UID ranges pretty heavily, so that the numbers often go out as e.g. `100:200`.
> 
> Dan Vrátil wrote:
>     Thanks, haven't seen that. I'll adapt the patch to reflect it - I'll just reduce the number of UIDs in single request even further.
>     
>     We have support for UID ranges too of course. In theory, UID ranges work nicely but in practice, they only work if you don't delete emails. For example if you use local filters, all your emails go to INBOX first, then are moved to their target folders. UIDs of emails that are then left in the INBOX folder will be very scattered - in the best case you find some ranges of type X:X+1, mostly none though. The situation is of course better in case of the filter destination folders.

> In theory, UID ranges work nicely but in practice, they only work if you don't delete emails.

You can still use them as the server is required to ignore any UIDs not present at the server. I.e. you delete five *adjacent* e-mails with UID 123, 160, 177, 190 and 205, you can do that via 123:205. Yes, doing this requires extra intelligence at a "wrong" layer, so there are not that many benefits in doing it.


- Jan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/120446/#review67848
-----------------------------------------------------------


On Oct. 3, 2014, 9:15 a.m., Dan Vrátil wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://git.reviewboard.kde.org/r/120446/
> -----------------------------------------------------------
> 
> (Updated Oct. 3, 2014, 9:15 a.m.)
> 
> 
> Review request for KDEPIM and Christian Mollekopf.
> 
> 
> Bugs: 339565
>     http://bugs.kde.org/show_bug.cgi?id=339565
> 
> 
> Repository: kdepim-runtime
> 
> 
> Description
> -------
> 
> Although RFC does not say anything about maximum length of an IMAP request, in reality IMAP server implementations have some kind of a limit - sometimes it's maximum length of the command, sometimes it's just the number of UIDs.
> 
> In normal use case we usually don't hit this limits, but when you delete or mark as read a huge amount of items (let's say you try to empty your Trash folder), you can hit this limit. In such case the items just disappear from Akonadi, but reappear on next sync, because server will simply refuse the IMAP command.
> 
> This patch makes ChangeItemsFlagsTask to split a one huge KIMAP::StoreJob into multiple smaller jobs, 2000 UIDs each. 2000 UIDs sounds like a good compromise between performance (you rarely do large operations like this, usually it's much smaller, tens maybe hundreds of items most), and functionality (2000 UIDs sounds like a number that every single IMAP server out there should still handle OK).
> 
> 
> Diffs
> -----
> 
>   resources/imap/changeitemsflagstask.h 9a58652 
>   resources/imap/changeitemsflagstask.cpp 1316b65 
> 
> Diff: https://git.reviewboard.kde.org/r/120446/diff/
> 
> 
> Testing
> -------
> 
> Successfully marked as read few emails, many emails and >2000 emails at once.
> 
> 
> Thanks,
> 
> Dan Vrátil
> 
>

_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/