debugging akonadi_control

Tue Dec 11 23:21:36 GMT 2018

On 11.12.2018 22:13, Milian Wolff wrote:
> On Dienstag, 11. Dezember 2018 12:02:12 CET Jos van den Oever wrote:
>> Hello all,
>> 
>> I've been experiencing daily crashes of akonadi_control. I'd like to 
>> figure
>> out what's going on. I've attached gdb to get stack traces. 
>> Unfortunately,
>> as you can see from the log below, akonadi_control crashes without 
>> giving
>> an opportunity to show the backtraces.
>> 
>> What would be better way of getting to the cause of the crashes?
>> 
>> Best regards,
>> Jos
>> 
>> $ gdb -p 25515
>> GNU gdb (GDB) 8.1.1
>> Copyright (C) 2018 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later
>> <http://gnu.org/licenses/gpl.html> This is free software: you are free 
>> to
>> change and redistribute it. There is NO WARRANTY, to the extent 
>> permitted
>> by law.  Type "show copying" and "show warranty" for details.
>> This GDB was configured as "x86_64-unknown-linux-gnu".
>> Type "show configuration" for configuration details.
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>.
>> Find the GDB manual and other documentation resources online at:
>> <http://www.gnu.org/software/gdb/documentation/>.
>> For help, type "help".
>> Type "apropos word" to search for commands related to "word".
>> Attaching to process 25515
>> [New LWP 25516]
>> [New LWP 25517]
>> [Thread debugging using libthread_db enabled]
>> Using host libthread_db library
>> "/nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-
>> glibc-2.27/lib/libthread_db.so.1".
>> 0x00007f36ec710501 in poll ()
>>    from 
>> /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
>> warning: File
>> "/nix/store/zk5zj2307zxaq7dx585yia3dn5k4qlsl-gcc-7.3.0-lib/lib/
>> libstdc++.so.6.0.24-gdb.py" auto-loading has been declined by your
>> `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
>> To enable execution of this file add
>>         add-auto-load-safe-path 
>> /nix/store/zk5zj2307zxaq7dx585yia3dn5k4qlsl-
>> gcc-7.3.0-lib/lib/libstdc++.so.6.0.24-gdb.py
>> line to your configuration file "/home/oever/.gdbinit".
>> To completely disable this security protection add
>>         set auto-load safe-path /
>> line to your configuration file "/home/oever/.gdbinit".
>> For more information about this security protection see the
>> "Auto-loading safe path" section in the GDB manual.  E.g., run from 
>> the
>> shell: info "(gdb)Auto-loading safe path"
>> (gdb) cont
>> Continuing.
>> [Thread 0x7f36d9fc3700 (LWP 25516) exited]
>> [Thread 0x7f36cc03d700 (LWP 25517) exited]
>> [Inferior 1 (process 25515) exited with code 0377]
>> (gdb) back
>> No stack.
> 
> One cool new tool that you may not be familiar with yet is rr. You 
> could just
> trace akonadi_control until it crashes/exits and then replay to see 
> what
> happens, and why.
> 
> See: https://github.com/mozilla/rr

rr seems a snazzy tool.
Simply running it to output the help of kmail works great:
   rr record kmail --help

When I run kmail in rr without arguments to get the main GUI or with 
--view to open one mail, I'm getting a different error than when running 
kmail in gdb.
Instead of QtScript, this backtrace points to QtWebEngine. A while back, 
I also had crashing kmail. Then it was caused by the web rendering 
crashing and taking down the main application with it.

$ rr record kmail
rr: Saving execution to trace directory 
`/home/oever/.local/share/rr/kmail-0'.

<--- Last few GCs --->

<--- JS stacktrace --->

#
# Fatal process OOM in insufficient memory to create an Isolate
#

Received signal 4 ILL_ILLOPN 7fe4c9bde569
#0 0x7fe4c881819e <unknown>
#1 0x7fe4c719f07c <unknown>
#2 0x7fe4c8818687 <unknown>
#3 0x7fe4c4204f10 <unknown>
#4 0x7fe4c9bde569 <unknown>
#5 0x7fe4c85358a2 <unknown>
#6 0x7fe4c8535bb1 <unknown>
#7 0x7fe4c85b1a96 <unknown>
#8 0x7fe4c8437c08 <unknown>
#9 0x7fe4c8442ece <unknown>
#10 0x7fe4c82e7068 <unknown>
#11 0x7fe4c8445e5d <unknown>
#12 0x7fe4c854fca8 <unknown>
#13 0x7fe4c9df0b2e <unknown>
#14 0x7fe4c9c05b4e <unknown>
#15 0x7fe4c9c06828 <unknown>
#16 0x7fe4cb53ae5a <unknown>
#17 0x7fe4cb509a2f <unknown>
#18 0x7fe4cae1e345 <unknown>
#19 0x7fe4cae21587 <unknown>
#20 0x7fe4cae244fa <unknown>
#21 0x7fe4cae245da <unknown>
#22 0x7fe4cae4f49a <unknown>
#23 0x7fe4c87d9b6a <unknown>
#24 0x7fe4c87d9e8b <unknown>
#25 0x7fe4c9a2bbb6 <unknown>
#26 0x7fe4c87d8ab5 <unknown>
#27 0x7fe4c7212913 QtWebEngine::processMain()
#28 0x000000400f65 <unknown>
#29 0x7fe4c352eb8e __libc_start_main
#30 0x000000400fea <unknown>
   r8: 0000000000000000  r9: 0000000000000046 r10: 0000000000000073 r11: 
0000000000000000
  r12: 0000000000d62020 r13: 00007fe4cbedcb20 r14: 0000000000d62040 r15: 
00007fe4cbedcb20
   di: 00007fffa6f90640  si: 0000000000000046  bp: 00007fffa6f92e40  bx: 
0000000000000000
   dx: 00007fe4c38bd710  ax: 0000000000000046  cx: ffffffffffffffff  sp: 
00007fffa6f92e18
   ip: 00007fe4c9bde569 efl: 0000000000010202 cgf: 002b000000000033 erf: 
0000000000000000
  trp: 0000000000000006 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.
Aborted

The session was recorded and I can replay it.
I put it online: https://www.vandenoever.info/tmp/kmail_rr.7z

Other debugging attempts that did not fix the issue:
  - run kmail as new user -> same crash
  - removing kmail and akonadi configuration -> same crash
  - rolling back generation of user packages or system packages several 
generations -> same crash

So carefully replaying the session seems the best bet at the moment.

Cheers,
Jos