akonadictl fsck: What messages indicate real issues and what to do about them?
Martin Steigerwald
martin at lichtvoll.de
Mon Jan 29 17:14:08 GMT 2018
Hello Dan, hi everyone.
It has been a while, but I never forgot your detailed answer that shed light
on the issues akonadictl fsck displays, but still did not offer me much in
terms on what I can reasonably do in order to fix them.
Daniel Vrátil - 09.05.17, 09:15:
> On Saturday, May 6, 2017 12:18:00 PM CEST Martin Steigerwald wrote:
> > Hello.
> >
> > After Akonadi was still occupied with stuffing all the 67000+ mails of the
> > maildir folder I just moved inside KMail between different parent folders
> > of the same and only maildir resource, I did a akonadictl stop to stop
> > that madness. On next start of Akonadi it luckily forgot what it was
> > about to do. I used akonadictl fsck to clean out the mess in file_db_data
> > + db_data and that it did.
> >
> > However even now it still reports some other issues I am not sure about.
> > So
> > I ask here: What messages indicicate real issues and what can I do about
> > them?
> >
> > I am first asking here, but I am willing to provide bug reports for any
> > real issues. I store a copy of this fsck run. I don´t provide it in full
> > here due to privacy concerns.
> >
> > # Collections without RID
> >
> > > Looking for dirty objects...
> > > Collection "Search" (id: 1) has no RID.
> > > Collection "OpenInvitations" (id: 395) has no RID.
> > > Collection "DeclinedInvitations" (id: 396) has no RID.
> > > Found 3 collections without RID.
> >
> > I think at least for "Search" this is expected. I am not sure about the
> > other two, but it may be expected for them as well.
>
> The OpenInvitations and DeclinedInvitations collections are virtual search
> Collections too, so they don't have RID.
>
> We should probably not warn about those....
Bug 389606 - akonadictl fsck: do not show missing RID warning for search and
invitation folders
> > # Items without RID
> >
> > > Item "1289323" has no RID.
> > > Item "1289324" has no RID.
> > > Item "1289325" has no RID.
> > > Item "1289326" has no RID.
> > > Item "1289327" has no RID.
> > > [… more than 1800 more …]
> >
> > RID is Remote ID as far as I understood. Yet the remote here is the local
> > maildir. So these appear to be items in the database or file_db_data that
> > are not inside the maildir. How can I look up these items and check
> > whether
> > this is an issue?
>
> Yes, those are items that have not been succesfully replayed into maildir
> and only exist in Akonadi.
>
> Run
>
> SELECT id, storage, data FROM PartTable WHERE pimitemId = 1289323 AND
> parttypeid = (SELECT id FROM parttypetable WHERE ns='PLD' AND name='RFC822')
>
> In Akonadi Console -> DB Console
>
> If storage is 0, then the content of they email will be in the "data" field,
> otherwise the "data" field refers to a file in
> ~/.local/share/akonadi/db_data/ XY/ (where "XY" are the last two digits of
> the part ID).
Okay, but what to do about it? From what I gather I can just check whether
they are mails I want to keep. If so, I can just go with keeping the current
database, cause neither does Akonadi server automatically re-attempt to replay
them into maildir, nor is akonadictl fsck able to do it.
Only thing is I could either copy the mail from the database record or the
file in file_db_data into a file in the correct maildir folder. That is no fun
for:
% grep "Item .* has no RID" akonadictl-fsck-2018-01-29.log | wc -l
3347
items.
How could it happen that Akonadi is not able to replay the mail to the
maildir? It is usually not that the underlaying BTRFS dual SSD RAID 1 runs
away when Akonadi asks it to save a file.
This one and
> > # Items with RID that are dirty
> >
> > > Item "1335889" has RID and is dirty.
> > > Item "1349287" has RID and is dirty.
> > > Item "1349736" has RID and is dirty.
> > > Item "1360764" has RID and is dirty.
> > > Item "1360766" has RID and is dirty.
> > > [… easily more than 2000 …]
> >
> > What about these?
>
> Those are Items that exist in maildir, were somehow changed (marked as read,
> moved, etc.) but the change wasn't succesfully stored in maildir. Currently
> there's no mechanism to force replay the change.
would be
Bug 360834 - no mechanism to reattempt to store items without rid (just in db)
into the resource
then.
> > # RID duplicates not matching the content mime-type of the parent
> > collection
> Means that you have a Collection that contains Items of a wrong time (e.g.
> emails in a calendar collection). Zero clue how that would've happened.
Hmm, no idea either.
> > What?
> >
> > > Looking for rid-duplicates not matching the content mime-type of the
> > > parent
> > > collection
> >
> > Some examples:
> > > Checking akonadi_icaldir_resource_0
> > > Found duplicates
> > > 040000008200E00074C5B7101A82E00800000000201B33D46D12D2010000000000000000
> > > 10
> > > 0
> > > 0000085F61ED021447341AA2A15976A6B8FA7
> > >
> > > Checking aros-dev-ml
> > > Found duplicates 1429563277.R787.merkaba
> > > Found duplicates 1432116953.R995.merkaba
> > > Found duplicates 1436178221.R34.merkaba
> > >
> > > Checking oss-security-ml
> > > Found duplicates 1424451476.R127.merkaba
> > > Found duplicates 1424451476.R255.merkaba
> > > Found duplicates 1424453279.R126.merkaba
> > > […]
> >
> > Lots more.
>
> The same email is stored multiple times in Akonadi in the same Collection.
> This can break sync and causes the "multiple merge candidates" issue.
And how did that happen?
> > Seriously, I am not even sure I want to understand what this is about. But
> > well… what is it about?
> >
> > I think its good for fsck to provide more helpful messages:
> >
> > 1. If its no issue, then don´t report it unless "-v" is given.
> >
> > 2. If its an issue, advice what to do about it.
>
> fsck corrects some of the issues itself, for some of them we don't have any
> fix as of now (like Items without RID or dirty Items)
or those duplicates?
At least this may be easy enough to fix. However in
Checking qt-kde-ml
Found duplicates 1424447870.R861.merkaba
Found duplicates 1424453278.R177.merkaba
Found duplicates 1424453279.R44.merkaba
Found duplicates 1424455081.R806.merkaba
Found duplicates 1424455081.R860.merkaba
Found duplicates 1435497488.R146.merkaba
Found duplicates 1435497488.R303.merkaba:2,S
Found duplicates 1435499290.R326.merkaba:2,S
Found duplicates 1435499291.R292.merkaba
Found duplicates 1435499291.R683.merkaba
Found duplicates 1435499291.R865.merkaba
Found duplicates 1435499291.R886.merkaba:2,S
are all of these the same mail? Does not seem to be the case:
~/.local/share/local-mail/[…]/.Debian.directory/qt-kde-ml/new> ls -l
1424447870.R861.merkaba 1424453278.R177.merkaba 1424453279.R44.merkaba
1424455081.R806.merkaba 1424455081.R806.merkaba 1424455081.R806.merkaba
1424455081.R860.merkaba 1435497488.R146.merkaba 1435497488.R303.merkaba:2,S
1435499290.R326.merkaba:2,S 1435499291.R292.merkaba 1435499291.R683.merkaba
1435499291.R865.merkaba 1435499291.R886.merkaba:2,S
-rw-r--r-- 1 martin martin 6560 Feb 20 2015 1424447870.R861.merkaba
-rw-r--r-- 1 martin martin 6085 Feb 20 2015 1424453278.R177.merkaba
-rw-r--r-- 1 martin martin 4232 Feb 20 2015 1424453279.R44.merkaba
-rw-r--r-- 1 martin martin 4175 Feb 20 2015 1424455081.R806.merkaba
-rw-r--r-- 1 martin martin 4175 Feb 20 2015 1424455081.R806.merkaba
-rw-r--r-- 1 martin martin 4175 Feb 20 2015 1424455081.R806.merkaba
Only these three have the same SHA512 sum, but there is just one file with
that SHA512 sum in the filesystem, so Akonadi seems to be totally confused
about that one.
-rw-r--r-- 1 martin martin 4348 Feb 20 2015 1424455081.R860.merkaba
-rw-r--r-- 1 martin martin 4309 Jun 28 2015 1435497488.R146.merkaba
-rw-r--r-- 1 martin martin 4327 Jun 28 2015 1435497488.R303.merkaba:2,S
-rw-r--r-- 1 martin martin 4351 Jun 28 2015 1435499290.R326.merkaba:2,S
-rw-r--r-- 1 martin martin 4314 Jun 28 2015 1435499291.R292.merkaba
-rw-r--r-- 1 martin martin 8027 Jun 28 2015 1435499291.R683.merkaba
-rw-r--r-- 1 martin martin 4349 Jun 28 2015 1435499291.R865.merkaba
-rw-r--r-- 1 martin martin 7910 Jun 28 2015 1435499291.R886.merkaba:2,S
Would be good if akonadictl fsck could fix these:
Bug 389608 - akonadictl fsck: offer to fix duplicate items
It would be helpful if akonadictl fsck could (optionally, for privacy reasons)
show some meta data about affected items, so that a user can at least get an
idea whether they are items she wants to keep:
Bug 389609 - akonadictl fsck: optional show metadata for affected items to aid
user to fix up things manually
As well as having more helpful messages.
Bug 389607 - akonadictl fsck: Show more helpful messages
Okay, it seems that is about all that I as mostly a user and tester of KDEPIM
can do about the issue without learning to code Akonadi fixes myself. (I still
not fully understand the architecture and the source file layout.)
Of course I could ditch the database once again, for example by switching to
PostgreSQL database (from MariaDB), but I bet the underlying issues are not
yet all fixes, to after some time akonadictl fsck may just display similar
errors.
Thanks,
--
Martin
More information about the kdepim-users
mailing list