[kdepim-users] [Probably found] Re: Please check your maildir stores for 1-byte sized files
O. Sinclair
o.sinclair at gmail.com
Sat May 4 12:58:12 BST 2013
On 04/05/2013 13:25, Martin Steigerwald wrote:
> Am Samstag, 4. Mai 2013, 12:55:45 schrieb Martin Steigerwald:
>> Am Freitag, 3. Mai 2013, 09:30:15 schrieb O. Sinclair:
>>> On 02/05/2013 20:47, Martin Steigerwald wrote:
>>>> Am Donnerstag, 2. Mai 2013, 17:34:05 schrieb Martin Steigerwald:
>>>>> Hi!
>>>>>
>>>>> I stumpled over what I think is a possible data loss again with
>>>>> my most recent, I think pretty simple setup with KMail and filter
>>>>> configuration from scratch:
>>>>>
>>>>> Bug 319226 - produces 1-byte-sized files on failed move attempts
>>>>> while filtering http://bugs.kde.org/319226
>>>>>
>>>>> I saw it produce three more of these 1-byte-sized files
>>>>
>>>> […]
>>>>
>>>>> I still hope its just happening for me and its somehow related to
>>>>> filter rules, but please sanity check your maildir stores.
>>>>
>>>> I suspect its really something with my filter rules, particularily
>>>> the CRM114 spam filtering integration rules I made.
>>>>
>>>> So it may really just happen to me.
>>>>
>>>> I removed those CRM114 rules now and will see what happens.
>>>>
>>>> Still it would be reassuring to hear from some POP3 users that they
>>>> do not have one byte sized files in their local maildirs while
>>>> using (local) filtering themselves.
>>>
>>> I find none and I have been using CRM114 though it seems to be botched
>>> in Kubuntu 13.04 (not first time, worked nicely in 12.10 though). Have
>>> also removed CRM114 as it simply does not seem to work..
>>
>> Thanks!
>>
>> Can you recall the exact filter fules you used?
>>
>> I think it would be helpful to compare them with the ones I have posted to
>> the bug report above.
>>
>> After having switched of my CRM114 spam filter rules I did not see further
>> one- byte sized files. So I´d like to find out whats different with your
>> rules.
>>
>> I am quite sure my CRM114 on Debian Sid with still crm114 package version
>> 20100106-3 works just fine. It did so a week ago with KMail 1. It never
>> produced one-byte sized files or other erratic results as far as I can see.
>>
>>
>> For comparison on list here are my CRM114 filter rules. On looking at the
>> filter rules while pasting them here, I found a possible issue with them
>> explaining what I was seeing. But still, I´d like to compare with your
>> filter rules, before trying again, as its quite tedious to restore crunched
>> mails from POP3 server.
>>
>>
>>
>> Pipe through crm114 to find if its spam:
>>
>> [Filter #0]
>> Applicability=1
>> AutomaticName=false
>> ConfigureShortcut=false
>> ConfigureToolbar=false
>> Enabled=true
>> Icon=system-run
>> StopProcessingHere=false
>> ToolbarName=crm114: Auf Spam prüfen
>> accounts-set=akonadi_pop3_resource_0
>> action-args-0=crm -u ~/.crm114 mailreaver.crm
>> action-name-0=filter app
>> actions=1
>> apply-on=check-mail,manual-filtering
>> contentsA=262144
>> fieldA=<size>
>> funcA=less-or-equal
>> identifier=BfESlQkyajcYUgNg
>> name=crm114: Auf Spam prüfen
>> operator=or
>> rules=1
>>
>>
>> Set status of mails crm114 marked in rule #0 to spam and filter them to spam
>> folder:
>>
>> [Filter #1]
>> Applicability=0
>> AutomaticName=false
>> ConfigureShortcut=true
>> ConfigureToolbar=false
>> Enabled=true
>> Icon=system-run
>> StopProcessingHere=true
>> ToolbarName=crm114: Spam ausfiltern
>> action-args-0=Lokale Ordner/Papierkorb/Spam
>> action-args-1=P
>> action-name-0=transfer
>> action-name-1=unset status
>> actions=2
>> apply-on=check-mail,manual-filtering
>> contentsA=SPAM.*
>> contentsB=^[2345]
>> fieldA=X-CRM114-Status
>> fieldB=X-GMX-Antispam
>> funcA=regexp
>> funcB=regexp
>> identifier=5t65LS2QA9PYTGUe
>> name=crm114: Spam ausfiltern
>> operator=or
>> rules=2
>>
>>
>> Filter mails crm114 marked as unsure to unsure folder:
>>
>> [Filter #2]
>> Applicability=0
>> AutomaticName=false
>> ConfigureShortcut=false
>> ConfigureToolbar=false
>> Enabled=true
>> Icon=system-run
>> StopProcessingHere=true
>> ToolbarName=crm114: Unsicher ausfiltern
>> action-args-0=Lokale Ordner/Papierkorb/Unsicher
>> action-name-0=transfer
>> actions=1
>> apply-on=check-mail,manual-filtering
>> contentsA=UNSURE
>> fieldA=X-CRM114-Status
>> funcA=contains
>> identifier=ui7f6pceYEB2uWgx
>> name=crm114: Unsicher ausfiltern
>> operator=or
>> rules=1
>>
>>
>>
>> And add the send manual filters for telling crm114 whether a mail is spam or
>> good. I had them at the end cause with apply-on=manual-filtering and
>> StopProcessingHere=true any Ctrl+J ended on them and other mailing list
>> based filters after it did not work. Now, after having looked at my
>> original KMail-1 based filters again, I think I´d just omit uncheck
>> "apply-on=manual-filtering" option.
>>
>> [Filter #39]
>> Applicability=0
>> AutomaticName=false
>> ConfigureShortcut=true
>> ConfigureToolbar=true
>> Enabled=true
>> Icon=mail-mark-notjunk
>> StopProcessingHere=true
>> ToolbarName=crm114: Als Gut einstufen
>> accounts-set=akonadi_pop3_resource_0,akonadi_maildir_resource_0
>> action-args-0=H
>> action-args-1=crm -u ~/.crm114 mailreaver.crm --good
>> action-args-2=rm -u ~/.crm114 mailreaver.crm
>>
>> GOSH! That rm -u should be a "crm", a copy and paste error. I think I
>> noticed an error like this and fixed it.
>>
>> That might explain what I am seeing.
>>
>>
>> action-name-0=unset status
>> action-name-1=execute
>> action-name-2=filter app
>> actions=3
>> apply-on=manual-filtering
>> identifier=XOzgjB9dqty8V2Zn
>> name=crm114: Als Gut einstufen
>> operator=all
>> rules=0
>>
>>
>> Still that filter was after all mailing list based filters and the one-byte
>> sized files where sorted into mailing lists by previous filters which have
>> StopProcessingHere.
>>
>> I will look at newer revisions of my filter rules.
>
> Seems also to be broken in that way, so I may have not spotted this error.
>
>> And well, I also had one-byte sized files with imported filter rules from
>> KDEPIM-1, which were not having this "rm" in it. Just checked it.
>>
>> But still:
>>
>> merkaba:/tmp> echo "test" > test
>> merkaba:/tmp> LANG=C rm -u test
>> rm: invalid option -- 'u'
>> Try 'rm --help' for more information.
>> merkaba:/tmp#1> cat test
>> test
>> merkaba:/tmp>
>>
>>
>> So if Akonadi pipes the mail through rm -u, it should get back "rm: invalid
>> option -- 'u'
>> Try 'rm --help' for more information."
>>
>> or something like that.
>>
>> Well, that could somehow have lead to a one byte sized file.
>
>
> I tried just this misconfigured filter rule a ton of time with Ctrl-J and
> triggering via filter actions menu on two test mails. These are still there.
>
> I thought they would be gone. Maybe Akonadi doesn´t accept the pipe result, if
> the command returns an error code?
>
> I don´t even see the rm to be called, but may be to quick to notice.
>
> While thats no prove that the rule did not cause any havoc, I have an idea on
> how to test crm114 filter rules for one-byte sized files (with crm 114 rules
> enabled) safely:
>
> 1) create a pop3 test account
>
> 2) stuff a lots of mails to this mail account via a for loop and mail command.
>
> 3) retrieve mails only from that test account.
>
> I think I will try this way.
>
>
> I found another mistake with the mark as good rule: It should not be called on
> manual filtering, only by being called from filter action menu or icon.
>
> While this should not cause one-byte sized files, it might have contributed to
> several filters being called quickly one after another and possible race
> conditions relating that.
>
>
> martin at merkaba:~> while true; do ps aux | grep rm | egrep -v "(grep|btrfs|
> dirmngr)" ; sleep 0.01 ; done
> martin 6208 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentD29042.tmp
> martin 6209 0.0 0.0 89148 1480 ? S 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6263 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentL29042.tmp
> martin 6264 0.0 0.0 89144 1468 ? S 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6311 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentF29042.tmp
> martin 6312 0.0 0.0 89016 1416 ? R 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6311 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentF29042.tmp
> martin 6312 0.0 0.0 89148 1476 ? R 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6361 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentj29042.tmp
> martin 6362 0.0 0.0 89152 1480 ? S 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6416 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentg29042.tmp
> martin 6417 0.0 0.0 89152 1480 ? S 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6464 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agenta29042.tmp
> martin 6465 0.0 0.0 89152 1480 ? S 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6515 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agents29042.tmp
> martin 6516 0.0 0.0 89020 1408 ? S 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6515 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agents29042.tmp
> martin 6516 0.0 0.0 89152 1472 ? S 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6533 0.0 0.0 89152 648 ? S 13:11 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 6534 0.0 0.0 4180 576 ? S 13:11 0:00 sh -c .
> \/mailtrainer.crm --fileprefix= --
> good=reaver_cache/known_good/20130504_131157_862193_C8DEACBE --
> spam=reaver_cache/empty/
> martin 6535 0.0 0.0 93208 1420 ? R 13:11 0:00 /usr/bin/crm
> ./mailtrainer.crm --fileprefix= --
> good=reaver_cache/known_good/20130504_131157_862193_C8DEACBE --
> spam=reaver_cache/empty/
> martin 7390 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentQ29042.tmp
> martin 7392 0.0 0.0 89152 1428 ? S 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7390 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentQ29042.tmp
> martin 7392 0.0 0.0 89152 1488 ? R 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7445 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agenta29042.tmp
> martin 7447 0.0 0.0 89152 1480 ? S 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7445 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agenta29042.tmp
> martin 7447 0.0 0.0 0 0 ? R 13:12 0:00 [crm]
> martin 7496 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agente29042.tmp
> martin 7498 0.0 0.0 89148 1420 ? S 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7548 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentA29042.tmp
> martin 7549 0.0 0.0 89016 1292 ? R 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7548 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentA29042.tmp
> martin 7549 0.0 0.0 89148 1472 ? S 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7566 0.0 0.0 89148 652 ? S 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7567 0.0 0.0 4180 580 ? S 13:12 0:00 sh -c .
> \/mailtrainer.crm --fileprefix= --
> good=reaver_cache/known_good/20130504_131206_383516_C8DEACBE --
> spam=reaver_cache/empty/
> martin 7568 0.0 0.0 122148 4948 ? R 13:12 0:00 /usr/bin/crm
> ./mailtrainer.crm --fileprefix= --
> good=reaver_cache/known_good/20130504_131206_383516_C8DEACBE --
> spam=reaver_cache/empty/
> martin 7603 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentb29042.tmp
> martin 7604 0.0 0.0 89016 1408 ? S 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7603 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agentb29042.tmp
> martin 7604 0.0 0.0 89148 1472 ? S 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7621 0.0 0.0 89148 652 ? S 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
> martin 7622 0.0 0.0 4180 576 ? S 13:12 0:00 sh -c .
> \/mailtrainer.crm --fileprefix= --
> good=reaver_cache/known_good/20130504_131206_586995_C8DEACBE --
> spam=reaver_cache/empty/
> martin 7623 0.0 0.0 93208 1444 ? R 13:12 0:00 /usr/bin/crm
> ./mailtrainer.crm --fileprefix= --
> good=reaver_cache/known_good/20130504_131206_586995_C8DEACBE --
> spam=reaver_cache/empty/
> martin 7653 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c
> (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde-
> martin/akonadi_mailfilter_agents29042.tmp
> martin 7655 0.0 0.0 89148 1472 ? S 13:12 0:00 crm -u
> /home/martin/.crm114 mailreaver.crm --good
>
>
In all honesty I do not remember exactly my crm114 spam filter as what I
did (and have done before) is to use the "create spam filter" wizard,
then do one for CRM114 and one for Spambayes. Then I would just cut and
paste them together, both the "background" and the "spam button" filters.
If memory serves me right (not necessarily) crm114 64-bit went botched
in 11.10 and that was a known bug. Came back working in 12.04
Now when I try to use it I get the same "symptoms" as back then, nothing
really happens. If you switch on filtering log you just see an error
message where crm114 should kick in on something that is spam (not ham).
So I have just assumed that this good, but mostly unknown, spam filter
is again "offline".
And consequently removed the crm114 part, now spambayes kicks in. Good
though not as good as the combined efforts.
And thanks for the idea of putting "Local Folders" to be checked on
application startup, that has really sped up KMail start :-)
Having said that you seem to have a somewhat complicated chain of events
set up to access your mail.
Best
Sinclair
_______________________________________________
KDE PIM users mailing list
Subscription management: https://mail.kde.org/mailman/listinfo/kdepim-users
More information about the kdepim-users
mailing list