Duplicate Emails again

[Colin Close]

 Ok I decided to take matters into my own hands and I have found out some stuff.
I won't bore you with all the details but there are are three things I have discovered so far.
Probably the most important initially is that 'akonadict fsck' simple fscks things worse.
Here's what I have observed and checked out by using queries on the akonadi database.
1: Certain processes (which I won't detail at the moment) create duplicate but ghost database entries in the one of the primary tables.
These 'ghosts' have all the same attributes as the original with the exception of the size. When you check for the original file in .local.share/local-mail/<collection folder> There is only a single file there is no trace of the ghost. Thus it must have been introduced by the database possibly by memory not being cleared properly before re-use.

2: If akonadictl fsck finds one of these duplicates it does not delete it instead it deletes the valid entry. The consequence of this is that when the mail is reloaded after performing the akonadictl fsck the entry for the original mail that exists in the local-mail folder is recreated and suddenly you have a duplicate again.


This is just the first of three problems that I have so far identified. No one  has a chance of getting their kmail installations working properly if the tools for maintaining the database are in themselved broken. 



There seem to be a number of actions that create ghost mails. The first seems to be the building of the db in the first place if a genuine duplicate is found then some kind of action should be taken to alert the user of the problem or to simply remove it to a lost&found folder or similar action.
I haven't yet managed to establish whether ghost duplicates are created while the tables are being filled from the maildir folders though I have a strong suspicion that they are. 

Any actions that involve filters almost always create duplicates in the target folder. 



I have found genuine duplicates where the same file occurs in the maildir subfolders cur, new and tmp in this case they are genuine files I have studied the db tables but I cannot find an entry that would give a clue to the absolute location of an individual mail. The implication of this is that the db has no way of detecting this condition and thus no amount of 'akonadictl fscking'  is going to fix it.


This said this condition may have been created by my efforts to get kmail2 working again with my archived mail which I have done numerous times each time my OS updates the package the db breaks again.



In the early days of kmail2 I switched to postgresql as the preferred db . I intend to do this again to see whether I can observe the same issues as I have with Mariadb the default application shipped with my os.


Best,



Colin Close
itchka at compuserve.com

 

 

-----Original Message-----
From: Pablo Sanchez <pablo at blueoakdb.com>
To: kdepim-users <kdepim-users at kde.org>
Sent: Mon, Sep 25, 2017 2:24 pm
Subject: Re: Duplicate Emails again

[ Comments below, in-line ]


On Mon, 25 Sep 2017 15:21:00 +0200, Martin Steigerwald wrote:
> Peter Humphrey - 25.09.17, 08:59:
>> On Thursday, 21 September 2017 18:58:32 BST Martin Steigerwald wrote:
>>> A users + developers bug squashing party would be awesome. Of course it
>>> needs developers willing to work with users, I think there are quite some
>>> in the KDE project and in the KDEPIM project, and users willing to work
>>> constructively with developers, i.e. really also test things and gather
>>> debug output as instructed.
>>
>> If you wanted it to sound grand, you could call such a team a "task force."
>> I'd be willing to join in, though my coding days are about 30 years behind
>> me.
>>
>> Anyone else?
> 
> I´d join as well, but challenge is… either meet physically, but we
> are spread all over the world I bet. Or via IRC, but then
> reproducability is a thing. One would need to use Teamviewer or
> better a trustable free software variant to allow a dev to actually
> see whats going on and also take over.  There is a free software
> that allows a dev to login via console and do things, which has even
> GPG support and is auditable by the user… but I am not aware of any
> easy to install, just works thing for GUI access.

I don't know whether this is viable or not but if a VM could be set up
with the issues, then it's a self-contained package that a developer
could use.

Cheers,
--
Pablo Sanchez - Blueoak Database Engineering, Inc
Ph:    819.459.1926        iNum:  883.5100.0990.1054


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kdepim-users/attachments/20170929/bda0c805/attachment.html>