[Akonadi] [Bug 502560] [Office365][Imap][akonadi] failure to sync/pull all messages from an Inbox with 10GB of e-mails. (infinite loop)

[Boris Shtrasman]

https://bugs.kde.org/show_bug.cgi?id=502560

--- Comment #2 from Boris Shtrasman <boris.sh.1983+kde.bugzilla at gmail.com> ---
(In reply to Daniel Vrátil from comment #1)
> It's not really a loop.  In the past we had problems with GMail having
> really broken support for RFC4551/CONDSTORE extension
> (https://datatracker.ietf.org/doc/html/rfc4551), which allows us to ask the
> email server to give us all messages that have changed since the last time
> we checked. Without CONDSTORE, we are able to easily find new messages, but
> to check a flag change of an existing message (e.g. if you marked some older
> message as read/unread), we need to list *all* messages in the mailbox and
> compare their flags with what we have cached in Akonadi. If a massive
> mailbox, this takes ages....
> 
> There are two steps that we should take:
> 1) Check whether GMail have fixed their CONDSTORE support and re-enable it.
> This would fix the problem instantly for you.
> 2) Get rid of the SEARCH - it was an optimization to ensure that we always
> get a batch of 2000 valid UIDs. Unfortunately on most IMAP servers (incl.
> Gmail), SEARCH takes ages. We can still do batching for the search, if we do
> a regular FETCH instead of UID FETCH, and have UIDs included in the
> response. We can possibly even increase the batch size if we process things
> more efficiently. That should be much much much faster. Even if CONDSTORE is
> still broken on Gmail, this should speed up the sync massively.

But this issue is present on Office365 (not GMail).(In reply to Daniel Vrátil
from comment #1)
> It's not really a loop.  In the past we had problems with GMail having
> really broken support for RFC4551/CONDSTORE extension
> (https://datatracker.ietf.org/doc/html/rfc4551), which allows us to ask the
> email server to give us all messages that have changed since the last time
> we checked. Without CONDSTORE, we are able to easily find new messages, but
> to check a flag change of an existing message (e.g. if you marked some older
> message as read/unread), we need to list *all* messages in the mailbox and
> compare their flags with what we have cached in Akonadi. If a massive
> mailbox, this takes ages....
> 
> There are two steps that we should take:
> 1) Check whether GMail have fixed their CONDSTORE support and re-enable it.
> This would fix the problem instantly for you.
> 2) Get rid of the SEARCH - it was an optimization to ensure that we always
> get a batch of 2000 valid UIDs. Unfortunately on most IMAP servers (incl.
> Gmail), SEARCH takes ages. We can still do batching for the search, if we do
> a regular FETCH instead of UID FETCH, and have UIDs included in the
> response. We can possibly even increase the batch size if we process things
> more efficiently. That should be much much much faster. Even if CONDSTORE is
> still broken on Gmail, this should speed up the sync massively.

Hi. thank you for looking into it.

Just a minor point, that is is not GMail but Office365, and for what I remember
the 2000 limitation is a limitation to prevent Office365 to disconnect akonadi.
(https://bugs.kde.org/show_bug.cgi?id=339393#c3 and
https://bugsfiles.kde.org/attachment.cgi?id=89107  ) .

In the trace I had shown above it start from start after a disconnect.

-- 
You are receiving this mail because:
You are the assignee for the bug.