[kdepim-users] [Probably found] Re: Please check your maildir stores for 1-byte sized files
Martin Steigerwald
Martin at lichtvoll.de
Sat May 4 12:25:25 BST 2013
Am Samstag, 4. Mai 2013, 12:55:45 schrieb Martin Steigerwald:
> Am Freitag, 3. Mai 2013, 09:30:15 schrieb O. Sinclair:
> > On 02/05/2013 20:47, Martin Steigerwald wrote:
> > > Am Donnerstag, 2. Mai 2013, 17:34:05 schrieb Martin Steigerwald:
> > >> Hi!
> > >>
> > >> I stumpled over what I think is a possible data loss again with
> > >> my most recent, I think pretty simple setup with KMail and filter
> > >> configuration from scratch:
> > >>
> > >> Bug 319226 - produces 1-byte-sized files on failed move attempts
> > >> while filtering http://bugs.kde.org/319226
> > >>
> > >> I saw it produce three more of these 1-byte-sized files
> > >
> > > […]
> > >
> > >> I still hope its just happening for me and its somehow related to
> > >> filter rules, but please sanity check your maildir stores.
> > >
> > > I suspect its really something with my filter rules, particularily
> > > the CRM114 spam filtering integration rules I made.
> > >
> > > So it may really just happen to me.
> > >
> > > I removed those CRM114 rules now and will see what happens.
> > >
> > > Still it would be reassuring to hear from some POP3 users that they
> > > do not have one byte sized files in their local maildirs while
> > > using (local) filtering themselves.
> >
> > I find none and I have been using CRM114 though it seems to be botched
> > in Kubuntu 13.04 (not first time, worked nicely in 12.10 though). Have
> > also removed CRM114 as it simply does not seem to work..
>
> Thanks!
>
> Can you recall the exact filter fules you used?
>
> I think it would be helpful to compare them with the ones I have posted to
> the bug report above.
>
> After having switched of my CRM114 spam filter rules I did not see further
> one- byte sized files. So I´d like to find out whats different with your
> rules.
>
> I am quite sure my CRM114 on Debian Sid with still crm114 package version
> 20100106-3 works just fine. It did so a week ago with KMail 1. It never
> produced one-byte sized files or other erratic results as far as I can see.
>
>
> For comparison on list here are my CRM114 filter rules. On looking at the
> filter rules while pasting them here, I found a possible issue with them
> explaining what I was seeing. But still, I´d like to compare with your
> filter rules, before trying again, as its quite tedious to restore crunched
> mails from POP3 server.
>
>
>
> Pipe through crm114 to find if its spam:
>
> [Filter #0]
> Applicability=1
> AutomaticName=false
> ConfigureShortcut=false
> ConfigureToolbar=false
> Enabled=true
> Icon=system-run
> StopProcessingHere=false
> ToolbarName=crm114: Auf Spam prüfen
> accounts-set=akonadi_pop3_resource_0
> action-args-0=crm -u ~/.crm114 mailreaver.crm
> action-name-0=filter app
> actions=1
> apply-on=check-mail,manual-filtering
> contentsA=262144
> fieldA=<size>
> funcA=less-or-equal
> identifier=BfESlQkyajcYUgNg
> name=crm114: Auf Spam prüfen
> operator=or
> rules=1
>
>
> Set status of mails crm114 marked in rule #0 to spam and filter them to spam
> folder:
>
> [Filter #1]
> Applicability=0
> AutomaticName=false
> ConfigureShortcut=true
> ConfigureToolbar=false
> Enabled=true
> Icon=system-run
> StopProcessingHere=true
> ToolbarName=crm114: Spam ausfiltern
> action-args-0=Lokale Ordner/Papierkorb/Spam
> action-args-1=P
> action-name-0=transfer
> action-name-1=unset status
> actions=2
> apply-on=check-mail,manual-filtering
> contentsA=SPAM.*
> contentsB=^[2345]
> fieldA=X-CRM114-Status
> fieldB=X-GMX-Antispam
> funcA=regexp
> funcB=regexp
> identifier=5t65LS2QA9PYTGUe
> name=crm114: Spam ausfiltern
> operator=or
> rules=2
>
>
> Filter mails crm114 marked as unsure to unsure folder:
>
> [Filter #2]
> Applicability=0
> AutomaticName=false
> ConfigureShortcut=false
> ConfigureToolbar=false
> Enabled=true
> Icon=system-run
> StopProcessingHere=true
> ToolbarName=crm114: Unsicher ausfiltern
> action-args-0=Lokale Ordner/Papierkorb/Unsicher
> action-name-0=transfer
> actions=1
> apply-on=check-mail,manual-filtering
> contentsA=UNSURE
> fieldA=X-CRM114-Status
> funcA=contains
> identifier=ui7f6pceYEB2uWgx
> name=crm114: Unsicher ausfiltern
> operator=or
> rules=1
>
>
>
> And add the send manual filters for telling crm114 whether a mail is spam or
> good. I had them at the end cause with apply-on=manual-filtering and
> StopProcessingHere=true any Ctrl+J ended on them and other mailing list
> based filters after it did not work. Now, after having looked at my
> original KMail-1 based filters again, I think I´d just omit uncheck
> "apply-on=manual-filtering" option.
>
> [Filter #39]
> Applicability=0
> AutomaticName=false
> ConfigureShortcut=true
> ConfigureToolbar=true
> Enabled=true
> Icon=mail-mark-notjunk
> StopProcessingHere=true
> ToolbarName=crm114: Als Gut einstufen
> accounts-set=akonadi_pop3_resource_0,akonadi_maildir_resource_0
> action-args-0=H
> action-args-1=crm -u ~/.crm114 mailreaver.crm --good
> action-args-2=rm -u ~/.crm114 mailreaver.crm
>
> GOSH! That rm -u should be a "crm", a copy and paste error. I think I
> noticed an error like this and fixed it.
>
> That might explain what I am seeing.
>
>
> action-name-0=unset status
> action-name-1=execute
> action-name-2=filter app
> actions=3
> apply-on=manual-filtering
> identifier=XOzgjB9dqty8V2Zn
> name=crm114: Als Gut einstufen
> operator=all
> rules=0
>
>
> Still that filter was after all mailing list based filters and the one-byte
> sized files where sorted into mailing lists by previous filters which have
> StopProcessingHere.
>
> I will look at newer revisions of my filter rules.
Seems also to be broken in that way, so I may have not spotted this error.
> And well, I also had one-byte sized files with imported filter rules from
> KDEPIM-1, which were not having this "rm" in it. Just checked it.
>
> But still:
>
> merkaba:/tmp> echo "test" > test
> merkaba:/tmp> LANG=C rm -u test
> rm: invalid option -- 'u'
> Try 'rm --help' for more information.
> merkaba:/tmp#1> cat test
> test
> merkaba:/tmp>
>
>
> So if Akonadi pipes the mail through rm -u, it should get back "rm: invalid
> option -- 'u'
> Try 'rm --help' for more information."
>
> or something like that.
>
> Well, that could somehow have lead to a one byte sized file.
I tried just this misconfigured filter rule a ton of time with Ctrl-J and
triggering via filter actions menu on two test mails. These are still there.
I thought they would be gone. Maybe Akonadi doesn´t accept the pipe result, if
the command returns an error code?
I don´t even see the rm to be called, but may be to quick to notice.
While thats no prove that the rule did not cause any havoc, I have an idea on
how to test crm114 filter rules for one-byte sized files (with crm 114 rules
enabled) safely:
1) create a pop3 test account
2) stuff a lots of mails to this mail account via a for loop and mail command.
3) retrieve mails only from that test account.
I think I will try this way.
I found another mistake with the mark as good rule: It should not be called on
manual filtering, only by being called from filter action menu or icon.
While this should not cause one-byte sized files, it might have contributed to
several filters being called quickly one after another and possible race
conditions relating that.
martin at merkaba:~> while true; do ps aux | grep rm | egrep -v "(grep|btrfs|
dirmngr)" ; sleep 0.01 ; done
martin 6208 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentD29042.tmp
martin 6209 0.0 0.0 89148 1480 ? S 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6263 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentL29042.tmp
martin 6264 0.0 0.0 89144 1468 ? S 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6311 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentF29042.tmp
martin 6312 0.0 0.0 89016 1416 ? R 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6311 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentF29042.tmp
martin 6312 0.0 0.0 89148 1476 ? R 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6361 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentj29042.tmp
martin 6362 0.0 0.0 89152 1480 ? S 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6416 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentg29042.tmp
martin 6417 0.0 0.0 89152 1480 ? S 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6464 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agenta29042.tmp
martin 6465 0.0 0.0 89152 1480 ? S 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6515 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agents29042.tmp
martin 6516 0.0 0.0 89020 1408 ? S 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6515 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agents29042.tmp
martin 6516 0.0 0.0 89152 1472 ? S 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6533 0.0 0.0 89152 648 ? S 13:11 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 6534 0.0 0.0 4180 576 ? S 13:11 0:00 sh -c .
\/mailtrainer.crm --fileprefix= --
good=reaver_cache/known_good/20130504_131157_862193_C8DEACBE --
spam=reaver_cache/empty/
martin 6535 0.0 0.0 93208 1420 ? R 13:11 0:00 /usr/bin/crm
./mailtrainer.crm --fileprefix= --
good=reaver_cache/known_good/20130504_131157_862193_C8DEACBE --
spam=reaver_cache/empty/
martin 7390 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentQ29042.tmp
martin 7392 0.0 0.0 89152 1428 ? S 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7390 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentQ29042.tmp
martin 7392 0.0 0.0 89152 1488 ? R 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7445 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agenta29042.tmp
martin 7447 0.0 0.0 89152 1480 ? S 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7445 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agenta29042.tmp
martin 7447 0.0 0.0 0 0 ? R 13:12 0:00 [crm]
martin 7496 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agente29042.tmp
martin 7498 0.0 0.0 89148 1420 ? S 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7548 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentA29042.tmp
martin 7549 0.0 0.0 89016 1292 ? R 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7548 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentA29042.tmp
martin 7549 0.0 0.0 89148 1472 ? S 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7566 0.0 0.0 89148 652 ? S 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7567 0.0 0.0 4180 580 ? S 13:12 0:00 sh -c .
\/mailtrainer.crm --fileprefix= --
good=reaver_cache/known_good/20130504_131206_383516_C8DEACBE --
spam=reaver_cache/empty/
martin 7568 0.0 0.0 122148 4948 ? R 13:12 0:00 /usr/bin/crm
./mailtrainer.crm --fileprefix= --
good=reaver_cache/known_good/20130504_131206_383516_C8DEACBE --
spam=reaver_cache/empty/
martin 7603 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentb29042.tmp
martin 7604 0.0 0.0 89016 1408 ? S 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7603 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agentb29042.tmp
martin 7604 0.0 0.0 89148 1472 ? S 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7621 0.0 0.0 89148 652 ? S 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
martin 7622 0.0 0.0 4180 576 ? S 13:12 0:00 sh -c .
\/mailtrainer.crm --fileprefix= --
good=reaver_cache/known_good/20130504_131206_586995_C8DEACBE --
spam=reaver_cache/empty/
martin 7623 0.0 0.0 93208 1444 ? R 13:12 0:00 /usr/bin/crm
./mailtrainer.crm --fileprefix= --
good=reaver_cache/known_good/20130504_131206_586995_C8DEACBE --
spam=reaver_cache/empty/
martin 7653 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
(crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
martin/akonadi_mailfilter_agents29042.tmp
martin 7655 0.0 0.0 89148 1472 ? S 13:12 0:00 crm -u
/home/martin/.crm114 mailreaver.crm --good
--
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
_______________________________________________
KDE PIM users mailing list
Subscription management: https://mail.kde.org/mailman/listinfo/kdepim-users
More information about the kdepim-users
mailing list