[Kde-pim] [FOUND] Re: Possible data loss: one-byte sized files

Sat May 4 12:04:01 BST 2013

Am Freitag, 3. Mai 2013, 07:56:33 schrieb Martin Steigerwald:
> Please, pretty please have a look at this. Thanks. I am starting to feel
> left alone with this, I think, grave data loss issue, which IMHO deserves
> some priority handling. (I know I haven´t bought a support contract, but
> still. And I am willing to co-operate with you on debugging this, already
> invested quite some time myself to find the cause of it). It seems to be
> related to the spam filter rules I had in use (see below).
> 
> Am Donnerstag, 2. Mai 2013, 20:00:01 schrieb Martin Steigerwald:
> > Am Donnerstag, 2. Mai 2013, 17:36:17 schrieb Martin Steigerwald:
> > > Am Donnerstag, 2. Mai 2013, 17:27:11 schrieb Martin Steigerwald:
> > > > Hi!
> > > > 
> > > > Sorry to come back to this, but I think its necessary. Due to previous
> > > > bugs
> > > > I  had with data losses and one-byte sized files, I thought I make a
> > > > sanity
> > > > check with find -size 1c (search for one-byte sized files) and
> > > > unfortunately it failed:
> > > > 
> > > > Please, pretty please have a look at this:
> > > > 
> > > > Bug 319226 - produces 1-byte-sized files on failed move attempts while
> > > > filtering https://bugs.kde.org/319226
> > > > 
> > > > (other related bugs linked from there)
> > > > 
> > > > 
> > > > This now is a pretty simple setup, made from scratch, on Ext4.
> > > > 
> > > > I am afraid, but I think Akonadi looses mail data.
> > > 
> > > I think it does, by comparing my local state from whats on the mailing
> > > list
> > > archive for debian-hurd at lists.debian.org.
> > > 
> > > I will try to prove it with POP3 maildir storage on my Dovecot POP 3
> > > server.
> > > 
> > > KMail activated pipelining for this server automatically. I think I will
> > > disable it now, to see whether it makes a difference.
> > > 
> > > I add all relevant information to bug report.
> > 
> > I got further one-byte sized files. 34 broken files now after I repaired 4
> > mails by comparing message ids on server and locally and copying those 4
> > mail files over into the Akonadi maildir where Akonadi picked them up.
> > 
> > I removed my CRM114 spam filter rules now, so see if mails get lost
> > ocassionaly by piping through crm114. This used to work in KMail 1, I have
> > no single one- byte sized file in KMail-1 maildir.
> 
> I think, it was the CRM114 filter rules.
> 
> I did not have any further one-byte sized files after remove these CRM114
> spam filter rules.

For comparison of rules with O. Sinclair, a another CRM114 user on kdepim-
users, I pasted them. During that I found a grave error in one of those rules:

[Filter #39]
Applicability=0
AutomaticName=false
ConfigureShortcut=true
ConfigureToolbar=true
Enabled=true
Icon=mail-mark-notjunk
StopProcessingHere=true
ToolbarName=crm114: Als Gut einstufen
accounts-set=akonadi_pop3_resource_0,akonadi_maildir_resource_0
action-args-0=H
action-args-1=crm -u ~/.crm114 mailreaver.crm --good
action-args-2=rm -u ~/.crm114 mailreaver.crm

GOSH! That rm -u should be a "crm", a copy and paste error. I think I noticed 
an error like this and fixed it.

That might explain what I am seeing.

action-name-0=unset status
action-name-1=execute
action-name-2=filter app
actions=3
apply-on=manual-filtering
identifier=XOzgjB9dqty8V2Zn
name=crm114: Als Gut einstufen
operator=all
rules=0

This may explain the issues I have experienced, although …

… still that filter was after all mailing list based filters and the one-byte 
sized files where sorted into mailing lists by previous filters which have 
StopProcessingHere.

I will look at newer revisions of my filter rules.

And well, I also had one-byte sized files with imported filter rules from 
KDEPIM-1, which were not having this "rm" in it. Just checked it.

But still:

merkaba:/tmp> echo "test" > test
merkaba:/tmp> LANG=C rm -u test
rm: invalid option -- 'u'
Try 'rm --help' for more information.
merkaba:/tmp#1> cat test
test
merkaba:/tmp>

So if Akonadi pipes the mail through rm -u, it should get back "rm: invalid 
option -- 'u'
Try 'rm --help' for more information."

or something like that.

Well, that could somehow have lead to a one byte sized file.

But, well no: That filter rule is only involved on manual filtering. But these 
corrupted files happened on POP3 retrieval, thus on incoming messages, where 
this filter does not apply.

I am inclined to try again with the "rm -u" fixed to "crm -u", but first I await 
Sinclair´s respone on kdepim-users. Maybe we can compare filter rules and find 
something else.

Well, in case this somehow is the cause of my one-byted sized please accept my 
apology. Cause this clearly is a configuration mistake on my side.

I am still not convinced that it has, tough, as the there are various reasons 
the filter should not have been called.

And I think I might have fixed this later one. Heck, I even think I marked some 
mails as good and these have no been gone. Well I will import just this one 
filter and try it out right now with a test mail instead of speculating.

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/