Akonadi errors with GMail account
Daniel Vrátil
dvratil at kde.org
Thu Dec 20 23:29:35 GMT 2018
Hi,
answers inlne.
On Monday, 17 December 2018 22:47:14 CET Szőts Ákos wrote:
> Hi Daniel,
>
> Thank you very much for your detailed answers.
>
> Daniel Vrátil a következőt írtad ekkor: 2018. december 15., szombat 12:05
>
> > On Friday, 14 December 2018 09:54:23 CET Szőts Ákos wrote:
> > > A gentle ping.
> >
> > Sorry, I only sent the reply to the mailing list without realizing you may
> > not be subscribed. Forwarding my original response below, as I cannot seem
> > to find November emails in mailing list archives..
>
> Hmm, that’s strange because this is the very reason I subscribed to this
> mailing list and I receive all letters (from CI and other people as well).
> Unfortunately, the archive is indeed broken, so could you please CC me in
> the future as well? Thank you.
>
> > On Thursday, 29 November 2018 07:03:09 CET Szőts Ákos wrote:
> > > Dear Developers,
> >
> > Hi Szőts,
> >
> > > I consider myself as a regular end-user in terms of e-mail client usage.
> > > I
> > > have a GMail account with IMAP and I would like to send and receive
> > > e-mails
> > > within KMail/Kontact. I have some folders synchronised besides all of
> > > the
> > > [GMail] default ones. I started with clean Akonadi profile (deleted
> > > everything related beforehand).
> >
> > Do you have "Download emails for offline use" checked in IMAP account
> > settings? From some of the errors below, I suspect you don't but want to
> > make sure.
>
> Yes, I do have it.
>
> > > Despite portraying an average user I encountered so many bugs and
> > > hiccups
> > > (which even made using KMail blatantly difficult) that I decided to
> > > collect
> > > most of them from a 30-minute long session. These are which an average
> > > user
> > > using GMail and KMail encounter often in my opinion.
> > >
> > > I know that some bugs are difficult to reproduce, therefore my idea is
> > > to
> > > list the errors and if you consider them rather bad for user experience,
> > > I'll compile the latest KMail/Akonadi duo and I help you to debug the
> > > exact issue and, if you have time to create them, to apply your patches
> > > and
> > > provide you fast feedback on their usefulness.
> >
> > Many thanks! I'm currently super-busy with dayjob, but I'll be back to
> > hacking on KDE since mid-December, then I'll dive into those issues and
> > start providing some patches, hopefully
>
> I think I can debug it a bit this week but after that only from the
> beginning of January.
>
> > <snip>
> >
> > > - I can only refresh a folder only once until “akonadictl restart”
> >
> > Sounds like something gets stuck in the IMAP resource, probably the
> > internal task scheduler. When you refresh a folder, open Akonadi Console,
> > right-click the IMAP resource and select "Show task list". What tasks does
> > it list?
> >
> > > - Some mails I cannot move to another folder failing with a dialogue
> > > stating
> >
> > this ("cannot move item" or "Unable to execute query")
> >
> > Hmm, weird Any logs from Akonadi Server that coincide with you triggering
> > this event?
>
> I enabled logging on the “Logging” tab but I cannot choose anything from the
> “Programs:” drop-down and it’s not writable. And (perhaps without that) no
> logs appear.The other tabs give so much information that I don’t know how
> to filter them to consider only the useful messages.
The comboboxes should populate automatically as processes started producing
log messages. Weird that it doesn't happen for you. Can you make sure that in
kdebusettings utility the Akonadi and other logging categories with "PIM" in
the name are enabled?
>
> > > - After some minutes of synchronisation the preview pane stucks in
> > > "Retrieving folder contents. Please wait...” message when I try to open
> > > a
> > > new mail.
> >
> > This is very likely caused by the resource internal task scheduler getting
> > stuck - when it's stuck it can't provide the email content to KMail and
> > the
> > whole thing gets stuck
>
> The messages like “Unable update item” popup (and perhaps this one also)
> happens when it synchronises a folder too long. And since it’s resyncing
> GMail’s [All Mail] folder with its ~10 GB of content it takes really long.
I can imagine that take a lot of time, the item retrieval probably times out
and gets stuck. I have a grand plan to solve this, I'll finish it over the
holidays :)
>
> > > The errors are from console (from about a 30-minute run):
> > > - org.kde.pim.akonadicore: Error during ItemSync: "Multiple merge
> > > candidates, aborting"
> >
> > This one has been haunting us for a long time. No fix yet, I haven't
> > observer a reliable way to get a folder into this inconsistent state.
>
> How can I determine which e-mails are causing this? I suspect there is a
> connection with mails containing multiple labels but I cannot confirm it
> yet. But if I could find out which letter we talking about I could perhaps
> create a test case.
If you ran Akonadi from terminal (run "akonadictl restart"), when this happens
you should see a message like "Multiple merge candidates:" with a list of the
duplicated entries.
>
> > <snip>
> >
> > > - qt.network.ssl: QSslSocket::startClientEncryption: cannot start
> > > handshake
> > > on non-plain connection
> > > -org.kde.pim.ksmtp: Socket error: 2 "Remote server closed connection"
> >
> > Those two may be related. Maybe something is wrong with your
> > configuration?
> > Can you double-check your SMTP account settings, or copy-paste it here
> > (without username and password, obviously, just the hostname, port,
> > encryption and authentication method)?
>
> Hmm, after a bit of thinking I realised I have a second account which,
> apparently, doesn’t have its password filled out. I think it would be
> helpful to indicate in logs which account are we currently talking about.
Good point :)
>
> Anyway, I’ll set up that account again correctly.
>
> > <snip>
> >
> > > - org.kde.pim.akonadiprivate: Error: failed to remove part file “/home/
> >
> > aki/.local/share/akonadi/file_db_data/79/1051679_r0” [with 100+
> >
> > > different files, access rights are ok]
> >
> > Do the files actually exist? I wonder if this is because when the payload
> > is supposed to be expired from the cache it's removed the disk, but the
> > SQL query to update the database fails, leading to this inconsistency.
> >
> > Anyway, runing "akonadictl fsck" should fix these problems.
>
> Yes, those files existed. Ran fsck (thank you for the tip) and it moved
> around 100 000 files into lost+found. I don’t know where lost+found is
> located (it’s not in ~/.local/share according to find) but anyhow, it fixed
> those errors.
Should be in ~/.local/share/akonadi/file_lost+found.
> > > - org.kde.pim.akonadiserver: DATABASE ERROR:
> > > -- org.kde.pim.akonadiserver: Error code: "1205"
> > > -- org.kde.pim.akonadiserver: DB error: "Lock wait timeout exceeded;
> > > try
> > > restarting transaction"
> > > -- org.kde.pim.akonadiserver: Error text: "Lock wait timeout exceeded;
> > > try restarting transaction QMYSQL3: Unable to execute query"
> > > -- org.kde.pim.akonadiserver: Query: "UPDATE PimItemTable
> > > SET rev = ?, remoteId = ?, remoteRevision = ?, gid = ?, collectionId =
> > > ?,
> > > mimeTypeId = ?, datetime = ?, atime = ?, dirty = ?, size = ? WHERE ( id
> > > =
> > > ?
> > > )" org.kde.pim.akonadiserver: Error during updating record with id
> > > 351516
> > > in table "PimItemTable" "Lock wait timeout exceeded; try restarting
> >
> > transaction QMYSQL3: Unable to execute query"
> >
> > I wonder why only MySQL has this kind of issue, haven't seen that on
> > Postgres. I don't have an easy reproducer, but I have some ideas on how to
> > approach this.
>
> During resyncing it happened again but unfortunately I don’t know how to
> retrieve the actual mail. When I execute the query "Select * from
> pimItemTable where id=353852” it tells me in which folder (label) it
> resides but I don’t know how to get the content of the mail since in
> “~/.local/share/akonadi/ file_db_data/35/” the file “353852” or “3852”
> doesn’t exist.
You have to check the PartTable - each PimItem consist of multiple Parts, so
query the PartTable for parts with pimitem_id=353852 - you will either see the
content of the part in the data column, or the data column will be empty and
then look for a file in ~/.local/share/akonadi/file_db_data that matches the
ID of the Part (no the PimItem!)
>
> Alternatively, if you tell me what additional information you need, I try to
> extract that.
>
> > > - org.kde.pim.akonadiserver: Protocol exception: Failed to write data to
> >
> > stream
> >
> > > -org.kde.pim.akonadicontrol: ProcessControl: Application "/usr/bin/
> >
> > akonadi_imap_resource" stopped unexpectedly ( “The process has crashed” )
> >
> > > - org.kde.pim.akonadicontrol: Application
> > > '/usr/bin/akonadi_imap_resource'
> > > crashed. No restart!
> >
> > The protocol exception is probably caused by the resource crashing and
> > suddenly disconnecting from the Akonadi Server. You should see DrKonqi
> > popping up when it crashes. Could you get a backtrace?
>
> I happens when I stop Akonadi and probably when a folder sync is in
> progress. I could not reproduce it in idle state.
>
> It turns out it’s hard to get a backtrace from it because after a crash GDB
> tell me “no such process” when I try to print it but once I succeeded. Here
> is the full one from akonadi_imap_resource:
> https://paste.opensuse.org/view/simple/1567434
Hmmm, looks like something goes awry during the shutdown. This should also be
solved by my new AgentBase code.
>
> > <snip>
> >
> > > - !org.kde.pim.imapresource: Called item() while state holds multiple
> > > items!
> >
> > I see this occasionally too, it's hard to reproduce and I have no clue
> > where the call is happening from. Attaching GDB to the Resource and
> > setting a breakpoint to the line that prints this message might help. If
> > you are able to do it yourself it would be awesome.
>
> Sure thing. Could you please tell me where can I find this line? I looked in
> https://github.com/KDE but it seems the sentence string is not anywhere in
> KDE hosted repos.
The code lives in kdepim-runtime repository, namely this message is in
ResourceState::item() method in resources/imap/resourcestate.cpp - attaching
GDB to the IMAP resource and setting a breakpoint at the method should do the
trick, although unless you can reproduce this reliably, you will probably get
a tons of false positives. Figuring out on which line the message exactly is
in the file (depends on which exact version you have) and setting the
breakpoint there should do the trick.
>
> > <snip>
> >
> > > - QFont::setPixelSize: Pixel size <= 0 (0) [20+ times]
> >
> > Again probably something from KMail or Kontact. Setting a breakpoint on
> > QFont::setPixelSize with condition on size<=0 might help us find where in
> > our code this gets called with the wrong argument.
>
> Thank you, will do that.
>
> > <snip>
> >
> > > - org.kde.pim.imapresource: Detected inconsistency in local cache, we're
> > > missing some messages. Server: 98582 Local: 98515
> > > org.kde.pim.imapresource: Refetching complete mailbox.
> >
> > Happens with Gmail sometimes, because Gmail's IMAP is crap.
>
> I’ve heard this from many places so I can easily believe it. Is it something
> a client can fix? Downloading 10 GB seems a bit unnecessary for me so if
> you need some information, I can try provide it.
We probably should not rely on UIDNEXT identifier when dealing with Gmail. On
normal IMAP servers UIDNEXT tells us what the UID of the newest message in the
folder is. We can then compare the last UIDNEXT we know locally with the
UIDNEXT reported by the server and we know that all the messages with UID
between those to UIDNEXT values are new and must be downloaded - this allows
us to be more efficient and only download message that are really new. What
happens with Gmail is that Gmail keeps reporting the same UIDNEXT identifier,
but the number of messages changes, which the IMAP resource correctly
recognizes is wrong, but it thinks the fault is on our side and decides to
download everything again just to be sure.
The solution is to do a quick listing (list only UID + FLAGS) from the Gmail
server and compare them with UIDs stored in Akonadi to discover added and
removed emails, and to update flags on all emails.
Cheers,
Dan
>
> > I don't have a
> > solution for this one. It just leads to the resource re-downloading the
> > entire folder, which sucks a lot, but at least it's "self-healing"
> >
> >
> > <snip>
>
> Thank you and all the best,
>
> Ákos
--
Daniel Vrátil
www.dvratil.cz | dvratil at kde.org
IRC: dvratil on Freenode (#kde, #kontact, #akonadi, #fedora-kde)
GPG Key: 0x4D69557AECB13683
Fingerprint: 0ABD FA55 A4E6 BEA9 9A83 EA97 4D69 557A ECB1 3683
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-pim/attachments/20181221/2e678455/attachment.sig>
More information about the kde-pim
mailing list