Plasma 5: Akonadi and slow start

[Duncan]

Yaroslav Pronin posted on Sun, 31 Jul 2016 10:41:54 +0300 as excerpted:

> Hello. For a long time (Plasma 5, Fedora and ArchLinux) I have noticed
> that if use akonadi, the startup time is markedly increased (black
> screen 1-1.5 minute after login). I read a lot about this problem, but
> working solution I haven't found. Is there a way to use akonadi without
> effect a slow start?

Well, uninstalling from my machine anything using akonadi, so I could in 
turn remove it, is the solution I ultimately chose, here.

However, I'd suggest the problem isn't akonadi itself, but how it uses 
storage (see below).  As such, attacking the problem at a level lower 
than akonadi itself is likely to ease the problem and shorten startup 
time significantly.  The following suggestions are based on what I found 
helped for a different (non-kde) app that was taking quite a long time to 
startup, that I wanted to keep for $REASONS.  There's a very strong 
likelihood they'll help you with akonadi as well.


First of all, the delay from fragmented IO that I believe is the root of 
the problem will be much more noticeable on conventional spinning rust 
drives than on SSDs.  So I'm assuming you're still on spinning rust as 
the problem would almost certainly be elsewhere (the CPU cycles required, 
thus, the logic of the app itself, which you can't do much about unless 
you're a programmer) if you're seeing that sort of delay on SSDs.  So...

#1 suggestion, if you're not on an SSD, consider upgrading to one, as the 
seek times I believe to be the problem are effectively zero on SSDs, 
giving you the choice to simply eliminate the problem with hardware, tho 
of course that costs money.

And if you are already on an SSD, look elsewhere as the rest of these 
suggestions aren't likely to help much.


As mentioned, I believe the issue is fragmented IO, as either a few large 
and highly fragmented files, or a whole bunch of smaller files that 
individually aren't fragmented as they're small, but that happen to be 
scattered all over your storage device, but either way, that akonadi 
reads into memory as it starts.  Because these files or file fragments 
are physically separated on the disk, the hardware likely spends more 
time moving back and forth between fragments than it does actually 
reading them, so any way you can find to reduce the time spent moving 
between fragments will directly reduce the startup time.  Other than the 
above throw money at it and get an SSD suggestion, the rest of the 
suggestions are various somewhat technical methods you can use to reduce 
fragmentation of those files, either individually for large files, or as 
a group of smaller files, with one exception, which basically just moves 
the problem to a time other than startup, giving everything else time to 
come up first, before starting akonadi.  With that introduction...

#2, consider your disk partitioning.  If everything related is in a 
(relatively) small part of the disk, because it's all on a single 
partition that's only part of the disk instead of using the whole disk as 
a single partition, then it won't be possible for the fragments to be at 
opposite ends of the disk, even if they're at opposite ends of the 
partition, because the partition will be a far smaller portion of the 
disk.  As such, the seeks should be faster as the head won't have as far 
to move to get to the next file or file fragment.  Additionally, files 
that end up highly fragmented on one partition due to usage pattern, 
won't be able to fragment the free-space on a different partition, 
thereby causing new files there to be fragmented as well, even if they'd 
stay in one piece otherwise.

FWIW, I have the main system (what the distro installs) on one 
partition, /home on another, media files on another, and the distro's 
update tree and cache on yet another, so everything's on its own 
partition by how it's used.  I'll leave it at that as there's lots of 
partitioning guides out there.

#3, consider your choice of filesystem.  Linux has a plethora of 
filesystem choices available, and some work better for some usages than 
others.  Also, some have defrag tools available while others don't (see 
the defrag suggestion below).

As an example, btrfs, which I use on my SSDs, is a copy-on-write (COW), 
based filesystem. COW based filesystems don't overwrite files in the same 
place, instead copying unmodified portions of a file block to a new 
location when some part of it is modified.  This helps quite a bit with 
file and filesystem integrity, since writes are atomic such that if a 
crash happens during a write, you should either get the new version or 
the old version, not an unpredictable mix that's partly new and partly 
old, but at the cost of much more extreme fragmentation for any files of 
significant size that have more or less random rewrites.  Database files, 
likely including akonadi's database, and  virtual machine images are 
examples of files that tend to fit this rewrite pattern and be fairly 
large, and thus be quite problematic on btrfs, altho there are 
workarounds possible.  But the point here is that if you're using btrfs 
on spinning rust and haven't applied the anti-fragmentation workarounds 
to your akonadi files, they're likely to be extremely fragmented, thus 
slowing down access for anything attempting to read those files.  Putting 
your database and VM image files on a non-btrfs filesystem can help (here 
again, partitioning can help, since you can then use different 
filesystems on each of the partitions), or simply applying the 
workarounds that help btrfs manage this type of heavy-in-place-rewrite-
pattern file somewhat better than it would by default.

#4, defrag.  As mentioned, some filesystems have a defrag tool of their 
own which you can use, but it's possible to defrag any filesystem 
manually using the method below.

For filesystems that don't, backing up the files on that filesystem 
elsewhere, blowing the filesystem away and recreating it with a fresh 
mkfs, then copying everything back, should effectively defrag the entire 
filesystem along with everything on it.

This is actually how I discovered how much of the slow startup time was 
due to fragmentation on the app I was having problems with, before I 
switched to SSDs for everything but my media partition.  I was using 
reiserfs at the time, and while it does have a defragger, it's a separate 
package on my distro and wasn't installed.  But my backups consist of 
partitions of similar size to the working copies, which I'll blow away 
and recreate from time to time when things seem to be relatively stable, 
then copy everything over from the working copy partition to the backup.  
Of course I then boot to the backup to be sure everything's in order, 
because as any good admin has learned, sometimes the hard way, a backup 
that hasn't been tested can't yet be considered a backup as you don't 
know that it can be used successfully yet.

And occasionally, once I'm booted to the backup and know everything's 
safely there, I'll take the opportunity (while booted to the backup) to 
wipe away the normal working copy, cleanly recreating that filesystem and 
copying everything back from the backup I'm booted to.  Then of course I 
reboot back to my normal working partitions again, testing them in the 
process.

It was one of the times that I did this, that I realized this app that 
normally took several minutes to startup, was now starting much faster.  
At first I thought the copy back had gone haywire and I was missing some 
of my data files, but I double-checked and everything was there.

The difference was simply that as I had accumulated more and more data 
for that app over time, it had gotten fragmented, and the backup, test-by-
booting-to-backup, blow away the working copy with a fresh mkfs, and 
restore from backup, process, had effectively defragged everything since 
each file was copied over serially, one at a time.

From then on, I didn't let the problem get so bad.  Every time things 
started getting too slow, I'd take the opportunity when test-booted to a 
backup to blow away and recreate the working copy on a fresh filesystem, 
so the files this app used weren't so fragmented and it would load 
faster, once again.

Of course eventually I switched to SSD, and while I still have the same 
backup and occasional refresh of the working copy system in place, I no 
longer have to worry about the load time of that app, since SSDs have 
zero seek times. =:^)


And finally...

#5, delay the akonadi load until everything else is up.

If the call to start akonadi is via dbus as it might be, you may have to 
get creative with this, but I've used this trick several times to delay 
something temporarily, until everything else I want loaded is done.

Find the executable, here, the akonadi executable, and whatever starts 
it, and replace the direct call to the executable with a call to a 
script, that has an appropriate delay, before in turn calling the real 
executable.

A script such as the following normally works, adjusting the time and 
path to the executable as necessary.

-----------------------------

#!/bin/bash

# delay this number of seconds
sleep 120

exec /usr/bin/realakonadi

---------------------------

Make it executable of course.

Then either move the akonadi executable to realakonadi and replace it 
with the script, or (if the call uses standard path lookup) put the 
akonadi script in a location earlier in the path so it's found first (but 
because it calls the real executable by full path, it won't just loop-
call itself), or if you know all the places that call the old executable 
(say because you used grep to find them), change them to point to the 
script, instead.

Now when you start plasma, the akonadi start will call the script 
instead, which will sleep for however many seconds first before starting 
the real executable, thus giving everything else time to finish loading 
before akonadi starts hogging all the disk bandwidth trying to load 
itself. =:^)


As I said, save for the first "throw money at it" suggestion, all these 
are somewhat technical, but they should help.  If you aren't technically 
oriented enough to use them and you don't have money for the first one, 
well... you can always do what I did and switch to non-akonadi-based apps 
and get rid of akonadi itself. =:^)  OTOH, the fact that I have that 
defrag results story in the first place means that I too know what it's 
like to have an app I'm not willing to give up that's slow enough I gotta 
work around it, so I understand if that's the case, but it just wasn't 
akonadi, for me -- akonadi I found replacement apps for and blew it away, 
and my system's the faster for it since! =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

___________________________________________________
This message is from the kde mailing list.
Account management:  https://mail.kde.org/mailman/listinfo/kde.
Archives: http://lists.kde.org/.
More info: http://www.kde.org/faq.html.