Missing bodies

[David Faure]

On mardi 21 août 2018 10:05:50 CEST Daniel Vrátil wrote:
> On Monday, 20 August 2018 22:40:27 CEST David Faure wrote:
> > No, this isn't about a murder-related TV series ;)
> > 
> > I just had a bug in akonadi where some old emails are not readable when
> > being offline, despite the "Download messages for offline use" option
> > being
> > checked in the IMAP settings.
> 
> Any chance you turned this option on later on when some emails have already
> been downloaded with the "offline mode" off?

Impossible, this is happening on emails from last week, on an account I 
created years ago.

> > Investigation showed this in the logs:
> >   ItemRetrievalJob::start: processing retrieval request for item (1447066)
> > 
> > parts: ("HEAD", "RFC822")  of resource: "akonadi_imap_resource_2"
> > ItemRetrievalManager::retrievalJobFinished: ItemRetrievalJob finished for
> > request 0x7f13a41b0550 , error: "Unable to retrieve item from resource:
> > Cannot fetch item in offline mode."
> 
> Well, you are offline so Akonadi can't fetch the body. The important part is
> that the attempt for retrieval happened, so the system works.

Yeah yeah, doesn't seem like "working" to the user :)

> It is a perfectly valid state. It means that the payload part has been
> expired from Akonadi and will be re-fetched the next time you try to
> display the email.

Can this expiring ever happen for an account with the "Download messages for 
offline use" setting?

> > MariaDB [akonadi]> select pimItemId,partTypeId,datasize,version,storage
> > from parttable where partTypeId=5 AND datasize=0; shows 3124 rows!!
> > 
> > What is it that might lead to bodies (RFC822) not being stored in the DB?
> > 
> > Could this happen when failing to save to
> > ~/.local/share/akonadi/file_db_data/* ? I had a few "partition full"
> > scenarios lately. Although, hmm, that was the root filesystem, not the
> > home one.
> 
> Full partition would likely fail cause the system to fail more "safely",
> like database failing to commit a transaction.

Unless there was room for the DB transaction but not for the writing to disk.
But anyway, this doesn't seem to be the root cause, see also below.

> > In any case, apart from the root case, I wonder if we could
> > 1) propagate the error to the user when saving fails, and
> 
> Although this wasn't a failure, our error reporting in general indeed is
> insufficient

Yes. Someone else had a similar issue this morning in FatCRM, on Windows:

[13000] org.kde.pim.akonadicore: Failed to open external payload: "C:/Users/
Mike/AppData/Local/akonadi/data/file_db_data/78/17378_r9" "Das System kann die 
angegebene Datei nicht finden."
[13000] item 17378 "" "application/x-vnd.kdab.crm.opportunity" is not a 
SugarOpportunity

It seems to me that both issues are the result of a new and recent bug related 
to the writing to file_db_data/. Did anything change in that area?

I'll replace the assert with a messagebox, but at the app level all we know is 
"this payload is not of the right type", while in fact that underlying error 
is "a file with the contents of the object was not found on disk".
Well, I guess I can also check for empty payload and say "payload disappeared, 
no idea why that happened, try wiping out the DB...".

> > 2) download all missing bodies next time we're online to ensure that those
> > emails will indeed be readable from an air plane...
> 
> This could be done by extending the CollectionScheduler,  but depends on
> what the cost of those queries is to make it viable...

The DB query is trivial, isn't it? I guess you mean the cost of downloading?

-- 
David Faure, faure at kde.org, http://www.davidfaure.fr
Working on KDE Frameworks 5