[Bug 156619] New: IMAP folder index corruption, messages or contents disappear, crashes on delete of affected messages

Fri Jan 25 01:27:04 GMT 2008

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

http://bugs.kde.org/show_bug.cgi?id=156619         
           Summary: IMAP folder index corruption, messages or contents
                    disappear, crashes on delete of affected messages
           Product: kmail
           Version: unspecified
          Platform: Ubuntu Packages
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: crash
          Priority: NOR
         Component: general
        AssignedTo: kdepim-bugs kde org
        ReportedBy: dev openserve org

Version:            (using KDE 3.5.8)
Installed from:    Ubuntu Packages
OS:                Linux

Symptoms:
- Messages sometimes appear in the folder list, but have no message content when opened. (header is intact but Kmail does not grab the message contents from the server)
- Messages which appear empty as above *always* cause a crash when moving them to a different folder.  [See backtrace for this..]
- Newly received messages sometimes disappear from the list, usually after a crash during a MOVE operation. In one case I had to delete/recreate the account so Kmail would re-download the IMAP headers fresh.

Possibly Related Symptom:
- My /.kde/share/apps/kmail/imap/.[random-number].directory/INDEX had grown to 13MB due to headers from old deleted messages being retained.  When I recreated the account, this dropped to 1.2MB.  It seems these files should be compressed from time to time.

Suspected problem:
- This seems to be a concurrency problem with the IMAP header cache / directory index.  It is most often triggered by moving/deleting a large selection of messages (such as spam) while simultaneously a new message happens to arrive in the INBOX.

Verified:
- IMAP server not the problem.  Other IMAP clients work fine.
- Messages are not actually being lost.  It is Kmail's cache that is getting out of sync with the server. Re-creating the account rebuilds the cache and all messages are still there.

Sidenote to developers: There has been an obvious steady increase in the number of online IMAP (not disconnected) related bugs with subsequent Kmail releases.  Today, Kmail crashes on me at least 10 times a day, with various IMAP operations.  This is only one bug report of many I could file.  It seems prudent at this point that the IMAP code should be audited and overhauled.

Backtrace:
--------------------
Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1".
[Thread debugging using libthread_db enabled]
[New Thread -1243629888 (LWP 6184)]
[New Thread -1291048048 (LWP 6188)]
[New Thread -1282655344 (LWP 6187)]
[New Thread -1274262640 (LWP 6186)]
[New Thread -1265869936 (LWP 6185)]
[KCrash handler]
#6  0xb4beaad9 in KMFolder::open (this=0x0, owner=0xb4ee9fdf "kmmovecommand")
    at /build/buildd/kdepim-3.5.7enterprise20070926/./kmail/kmfolder.cpp:472
#7  0xb4d53949 in KMMoveCommand::execute (this=0x8857918)
    at /build/buildd/kdepim-3.5.7enterprise20070926/./kmail/kmcommands.cpp:2154
#8  0xb4d43439 in KMCommand::slotPostTransfer (this=0x8857918, 
    result=KMCommand::OK)
    at /build/buildd/kdepim-3.5.7enterprise20070926/./kmail/kmcommands.cpp:260
#9  0xb4d4af46 in KMCommand::qt_invoke (this=0x8857918, _id=5, _o=0xbf954fd4)
    at ./kmcommands.moc:148
#10 0xb4d4b26b in KMMenuCommand::qt_invoke (this=0x8857918, _id=5, 
    _o=0xbf954fd4) at ./kmcommands.moc:2801
#11 0xb4d4b2f7 in KMMoveCommand::qt_invoke (this=0x8857918, _id=5, 
    _o=0xbf954fd4) at ./kmcommands.moc:2987
#12 0xb4d4b36b in KMDeleteMsgCommand::qt_invoke (this=0x8857918, _id=5, 
    _o=0xbf954fd4) at ./kmcommands.moc:3062
#13 0xb669c893 in QObject::activate_signal (this=0x8857918, clist=0x88700c8, 
    o=0xbf954fd4) at kernel/qobject.cpp:2356
#14 0xb4d41fae in KMCommand::messagesTransfered (this=0x8857918, 
    t0=KMCommand::OK) at ./kmcommands.moc:126
#15 0xb4d4bf41 in KMCommand::transferSelectedMsgs (this=0x8857918)
    at /build/buildd/kdepim-3.5.7enterprise20070926/./kmail/kmcommands.cpp:355
#16 0xb4d4c0a7 in KMCommand::slotStart (this=0x8857918)
    at /build/buildd/kdepim-3.5.7enterprise20070926/./kmail/kmcommands.cpp:252
#17 0xb4d4af58 in KMCommand::qt_invoke (this=0x8857918, _id=4, _o=0xbf955178)
    at ./kmcommands.moc:147
#18 0xb4d4b26b in KMMenuCommand::qt_invoke (this=0x8857918, _id=4, 
    _o=0xbf955178) at ./kmcommands.moc:2801
#19 0xb4d4b2f7 in KMMoveCommand::qt_invoke (this=0x8857918, _id=4, 
    _o=0xbf955178) at ./kmcommands.moc:2987
#20 0xb4d4b36b in KMDeleteMsgCommand::qt_invoke (this=0x8857918, _id=4, 
    _o=0xbf955178) at ./kmcommands.moc:3062
#21 0xb669c893 in QObject::activate_signal (this=0x81b53e8, clist=0x8869ff0, 
    o=0xbf955178) at kernel/qobject.cpp:2356
#22 0xb6a288ec in QSignal::signal (this=0x81b53e8, t0= 0x81b5410)
    at .moc/debug-shared-mt/moc_qsignal.cpp:100
#23 0xb66bc842 in QSignal::activate (this=0x81b53e8) at kernel/qsignal.cpp:212
#24 0xb66c4258 in QSingleShotTimer::event (this=0x81b53c0)
    at kernel/qtimer.cpp:286
#25 0xb6633af0 in QApplication::internalNotify (this=0xbf9557dc, 
    receiver=0x81b53c0, e=0xbf9554c8) at kernel/qapplication.cpp:2635
#26 0xb663591f in QApplication::notify (this=0xbf9557dc, receiver=0x81b53c0, 
    e=0xbf9554c8) at kernel/qapplication.cpp:2358
#27 0xb6df9ca2 in KApplication::notify (this=0xbf9557dc, receiver=0x81b53c0, 
    event=0xbf9554c8)
    at /build/buildd/kdelibs-3.5.8/./kdecore/kapplication.cpp:550
#28 0xb65c6209 in QApplication::sendEvent (receiver=0x81b53c0, 
    event=0xbf9554c8) at ../include/qapplication.h:520
#29 0xb662653b in QEventLoop::activateTimers (this=0x80af498)
    at kernel/qeventloop_unix.cpp:556
#30 0xb65dad49 in QEventLoop::processEvents (this=0x80af498, flags=4)
    at kernel/qeventloop_x11.cpp:389
#31 0xb664e1ce in QEventLoop::enterLoop (this=0x80af498)
    at kernel/qeventloop.cpp:198
#32 0xb664dfde in QEventLoop::exec (this=0x80af498)
    at kernel/qeventloop.cpp:145
#33 0xb6635699 in QApplication::exec (this=0xbf9557dc)
    at kernel/qapplication.cpp:2758
#34 0x0805bfe5 in main (argc=1, argv=0x88c9c20)
    at /build/buildd/kdepim-3.5.7enterprise20070926/./kontact/src/main.cpp:163
#35 0xb5e43050 in __libc_start_main () from /lib/tls/i686/cmov/libc.so.6
#36 0x0805b701 in _start ()