Akonadi errors with GMail account

[Szőts Ákos]

Hi Daniel,

Thank you very much for your detailed answers.

Daniel Vrátil a következőt írtad ekkor: 2018. december 15., szombat 12:05
> On Friday, 14 December 2018 09:54:23 CET Szőts Ákos wrote:
> > A gentle ping.
>
> Sorry, I only sent the reply to the mailing list without realizing you may
> not be subscribed. Forwarding my original response below, as I cannot seem
> to find November emails in mailing list archives..

Hmm, that’s strange because this is the very reason I subscribed to this 
mailing list and I receive all letters (from CI and other people as well). 
Unfortunately, the archive is indeed broken, so could you please CC me in the 
future as well? Thank you.

>
> On Thursday, 29 November 2018 07:03:09 CET Szőts Ákos wrote:
> > Dear Developers,
>
> Hi Szőts,
>
> > I consider myself as a regular end-user in terms of e-mail client usage. I
> > have a GMail account with IMAP and I would like to send and receive
> > e-mails
> > within KMail/Kontact. I have some folders synchronised besides all of the
> > [GMail] default ones. I started with clean Akonadi profile (deleted
> > everything related beforehand).
>
> Do you have "Download emails for offline use" checked in IMAP account
> settings? From some of the errors below, I suspect you don't but want to
> make sure.

Yes, I do have it.

> > Despite portraying an average user I encountered so many bugs and hiccups
> > (which even made using KMail blatantly difficult) that I decided to
> > collect
> > most of them from a 30-minute long session. These are which an average
> > user
> > using GMail and KMail encounter often in my opinion.
> >
> > I know that some bugs are difficult to reproduce, therefore my idea is to
> > list the errors and if you consider them rather bad for user experience,
> > I'll compile the latest KMail/Akonadi duo and I help you to debug the
> > exact issue and, if you have time to create them, to apply your patches
> > and
> > provide you fast feedback on their usefulness.
>
> Many thanks! I'm currently super-busy with dayjob, but I'll be back to
> hacking on KDE since mid-December, then I'll dive into those issues and
> start providing some patches, hopefully

I think I can debug it a bit this week but after that only from the beginning 
of January.

> <snip>

> > - I can only refresh a folder only once until “akonadictl restart”
>
> Sounds like something gets stuck in the IMAP resource, probably the internal
> task scheduler. When you refresh a folder, open Akonadi Console,
> right-click the IMAP resource and select "Show task list". What tasks does
> it list?
> > - Some mails I cannot move to another folder failing with a dialogue
> > stating
> this ("cannot move item" or "Unable to execute query")
>
> Hmm, weird  Any logs from Akonadi Server that coincide with you triggering
> this event?

I enabled logging on the “Logging” tab but I cannot choose anything from the 
“Programs:” drop-down and it’s not writable. And (perhaps without that) no 
logs appear.The other tabs give so much information that I don’t know how to 
filter them to consider only the useful messages.

>
> > - After some minutes of synchronisation the preview pane stucks in
> > "Retrieving folder contents. Please wait...” message when I try to open a
> > new mail.
>
> This is very likely caused by the resource internal task scheduler getting
> stuck - when it's stuck it can't provide the email content to KMail and the
> whole thing gets stuck

The messages like “Unable update item” popup (and perhaps this one also) 
happens when it synchronises a folder too long. And since it’s resyncing 
GMail’s [All Mail] folder with its ~10 GB of content it takes really long.

>
> > The errors are from console (from about a 30-minute run):
> > - org.kde.pim.akonadicore: Error during ItemSync:  "Multiple merge
> > candidates, aborting"
>
> This one  has been haunting us for a long time. No fix yet, I haven't
> observer a reliable way to get a folder into this inconsistent state.

How can I determine which e-mails are causing this? I suspect there is a 
connection with mails containing multiple labels but I cannot confirm it yet. 
But if I could find out which letter we talking about I could perhaps create a 
test case.

> <snip>

> > - qt.network.ssl: QSslSocket::startClientEncryption: cannot start
> > handshake
> > on non-plain connection
> > -org.kde.pim.ksmtp: Socket error: 2 "Remote server closed connection"
>
> Those two may be related. Maybe something is wrong with your configuration?
> Can you double-check your SMTP account settings, or copy-paste it here
> (without username and password, obviously, just the hostname, port,
> encryption and authentication method)?

Hmm, after a bit of thinking I realised I have a second account which, 
apparently, doesn’t have its password filled out. I think it would be helpful 
to indicate in logs which account are we currently talking about.

Anyway, I’ll set up that account again correctly.

> <snip>

> > - org.kde.pim.akonadiprivate: Error: failed to remove part file “/home/
>
> aki/.local/share/akonadi/file_db_data/79/1051679_r0” [with 100+
>
> > different files, access rights are ok]
>
> Do the files actually exist? I wonder if this is because when the payload is
> supposed to be expired from the cache it's removed the disk, but the SQL
> query to update the database fails, leading to this inconsistency.
>
> Anyway, runing "akonadictl fsck" should fix these problems.

Yes, those files existed. Ran fsck (thank you for the tip) and it moved around 
100 000 files into lost+found. I don’t know where lost+found is located (it’s 
not in ~/.local/share according to find) but anyhow, it fixed those errors.

>
> > - org.kde.pim.akonadiserver: DATABASE ERROR:
> > -- org.kde.pim.akonadiserver:   Error code: "1205"
> > -- org.kde.pim.akonadiserver:   DB error:  "Lock wait timeout exceeded;
> > try
> > restarting transaction"
> > -- org.kde.pim.akonadiserver:   Error text: "Lock wait timeout exceeded;
> > try restarting transaction QMYSQL3: Unable to execute query"
> > -- org.kde.pim.akonadiserver:   Query: "UPDATE PimItemTable
> > SET rev = ?, remoteId = ?, remoteRevision = ?, gid = ?, collectionId = ?,
> > mimeTypeId = ?, datetime = ?, atime = ?, dirty = ?, size = ? WHERE ( id =
> > ?
> > )" org.kde.pim.akonadiserver: Error during updating record with id 351516
> > in table "PimItemTable" "Lock wait timeout exceeded; try restarting
>
> transaction QMYSQL3: Unable to execute query"
>
> I wonder why only MySQL has this kind of issue, haven't seen that on
> Postgres. I don't have an easy reproducer, but I have some ideas on how to
> approach this.

During resyncing it happened again but unfortunately I don’t know how to 
retrieve the actual mail. When I execute the query "Select * from pimItemTable 
where id=353852” it tells me in which folder (label) it resides but I don’t 
know how to get the content of the mail since in “~/.local/share/akonadi/
file_db_data/35/” the file “353852” or “3852” doesn’t exist.

Alternatively, if you tell me what additional information you need, I try to 
extract that.

>
> > - org.kde.pim.akonadiserver: Protocol exception: Failed to write data to
>
> stream
>
> > -org.kde.pim.akonadicontrol: ProcessControl: Application "/usr/bin/
>
> akonadi_imap_resource" stopped unexpectedly ( “The process has crashed” )
>
> > - org.kde.pim.akonadicontrol: Application '/usr/bin/akonadi_imap_resource'
> > crashed. No restart!
>
> The protocol exception is probably caused by the resource crashing and
> suddenly disconnecting from the Akonadi Server. You should see DrKonqi
> popping up when it crashes. Could you get a backtrace?

I happens when I stop Akonadi and probably when a folder sync is in progress. 
I could not reproduce it in idle state.

It turns out it’s hard to get a backtrace from it because after a crash GDB 
tell me “no such process” when I try to print it but once I succeeded. Here is 
the full one from akonadi_imap_resource:
https://paste.opensuse.org/view/simple/1567434

> <snip>

> > - !org.kde.pim.imapresource: Called item() while state holds multiple
> > items!
> I see this occasionally too, it's hard to reproduce and I have no clue where
> the call is happening from. Attaching GDB to the Resource and setting a
> breakpoint to the line that prints this message might help. If you are able
> to do it yourself it would be awesome.

Sure thing. Could you please tell me where can I find this line? I looked in 
https://github.com/KDE but it seems the sentence string is not anywhere in KDE 
hosted repos.

> <snip>
>

> > - QFont::setPixelSize: Pixel size <= 0 (0) [20+ times]
>
> Again probably something from KMail or Kontact. Setting a breakpoint on
> QFont::setPixelSize with condition on size<=0 might help us find where in
> our code this gets called with the wrong argument.
>

Thank you, will do that.

> <snip>
>
> > - org.kde.pim.imapresource: Detected inconsistency in local cache, we're
> > missing some messages. Server:  98582  Local:  98515
> > org.kde.pim.imapresource: Refetching complete mailbox.
>
> Happens with Gmail sometimes, because Gmail's IMAP is crap.

I’ve heard this from many places so I can easily believe it. Is it something a 
client can fix? Downloading 10 GB seems a bit unnecessary for me so if you 
need some information, I can try provide it.

> I don't have a
> solution for this one. It just leads to the resource re-downloading the
> entire folder, which sucks a lot, but at least it's "self-healing"
>

> <snip>

Thank you and all the best,

Ákos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20181217/51726ef3/attachment.sig>