[kde-linux] Re: Getting rid of irritating CPU-sucking parasites...?

[Duncan]

David J Iannucci posted on Sat, 26 Mar 2011 13:18:51 -0600 as excerpted:

> Actually, it doesn't look like anything past 4.4.10 is in portage at
> all, never mind being masked.  I'm sure there's a good reason for that,
> but... if you're running 4.6, then you must have a different way of
> acquiring it, outside of portage?  Are you a Paludian?

Well, it's rather complicated.  I had started explaining it but was like 
200 lines in with no end in sight, so I scrapped that try and started 
over.  Let's see if I can do better now, just dealing with that bit of it.

First, as you may be aware, kde sources are divided up into several rather 
large tarballs, one for each category, instead of the usual one per
app/library/package.  Each of these kde "modules" contains apps and the 
libs they depend on to fill out a category.  The modules include kdelibs, 
generally required for anything kde, the former kdebase, now split into 
several smaller packages, but generally everything required to run kde as 
a desktop environment, kdegames, kdegraphics, kdeadmin, kdemultimedia, 
kdepim, kdeedu, etc.  Only kdelibs is required for all kde apps, tho 
there's lots of dependencies within a module and a number of dependencies 
between modules.

Gentoo splits each of these modules (with the exception of kdelibs, which 
is as a whole generally required as a dependency for anything else kde 
anyway) into many individual packages.  If you look in a gentoo kde-base/* 
ebuild (gentoo's kde-base category includes everything that forms part of 
kde proper, all the modules above, NOT just the kdebase upstream module), 
you'll see a line telling the kde eclasses what module tarball its in so 
they know what to download, and which specific subdirs within that module 
they need to build, so they can be unpacked, along with other subdirs that 
aren't actually built but still need unpacked so the build can access them.

That's the background.  KDE source tarballs from upstream consists of a 
series of modules, each containing many libraries, apps, and sometimes 
data packages.

The next bit to understand is that all the apps you mentioned, kmail, 
korganizer, kontact, belong to the kdepim module.  This is important 
because kdepim is currently an exception, as I'll explain.

Now, kde4 in general has a release schedule as follows.  Twice a year they 
release a minor version update, kde-4.x -> kde-4.y that's allowed to have 
major feature updates, new text strings for translation purposes, etc.  In 
between those they have monthly bugfix micro version updates with much 
stricter limits on the changes that can be made, 4.x.a -> 4.x.b.  So 
there's normally five such micro/bugfix updates (occasionally 6, in 
theory, could be less if the minor release happens to be unusually stable) 
between minor/feature updates, with the minors happening twice a year, 
February and August,  and the micros happening each month.

That's the release schedule.  Right now, the current kde version is 4.6.1, 
with 4.6.0 released in February, 4.6.1 in March, 4.6.2 should be in April, 
etc, allowing for a bit of slippage here or there.  KDE 4.4 is thus a year 
old, released last February, with 4.4.5 released in July, IIRC.

But, as I mentioned, kdepim and thus all the apps/libs it contains are an 
exception.  Here's the deal with it.

By now, you've likely read/heard about akonadi.  Akonadi is the new 
unified middleware for communications data, contacts, mail, IM, scheduling/
organizer, newsgroups, etc.  Eventually, they will have rewritten all the 
individual apps in kdepim to talk to the new backend instead of using 
their own storage.

With kde 4.4, they switched kdeaddressbook to use akonadi.  The 
akonadified kmail rewrite, kmail2, was originally scheduled to switch with 
kde 4.5.

But the kdepim folks must have learned something from the mistakes made 
with the kde4 rollout, as in what is certainly a major reversal from how 
the kde4 rollout itself was treated, when major bugs remained, they 
decided they were NOT going to simply release it and insist it was ready 
for normal use, as had happened with the kde4 rollout.  Rather, they were 
going to work on it, and it would be released "when it was ready".

NOW we've got most of the pieces and can start to put them together.  
Because what had been planned as kdepim-4.5 wasn't ready, the kdepim folks 
simply shipped the existing 4.4 branch (with kmail still taking care of 
its own mail data as it always had), updated from 4.4.5 to work with the 
changes in the new 4.5 series kdelibs.

Initially they still expected to ship the new version with one of the 4.5 
updates, explaining why they kept the 4.4 series versioning on the 
existing stable code.  But it didn't happen.

By the time the code freeze for 4.6 came around, the new kmail itself was 
considered pretty much ready.  But that wasn't all there was to it, as 
they were using conversion scripts to convert people's existing kmail data 
into the new akonadified format.  Unfortunately, there were still corner 
cases where the conversions were going wrong, especially with larger mail 
folders.

While I've not yet tried the new version, my case is a typical example of 
the cases that were still causing problems, as I imported my mail from MSOE 
into kmail back nearly a decade ago now, in late 2001 or very early 2002, 
with the MSOE mail going back into the mid 1990s.  So I've half a decade's 
worth of early MSOE mail imported into kde nearly a decade ago, PLUS 
nearly a decade's worth of accumulation of mail in kmail itself!  
Obviously, if the conversion were to go haywire and I didn't have a 
backup, I'd be a **VERY** unhappy camper!!

So they've been spending time recently testing, tweaking, and retesting, 
those import scripts, making *SURE* they're not going to be eating 
anybody's existing mail folders for lunch and coming back for more!

As I said, this is certainly in extreme contrast to how the whole kde4 
rollout itself was handled.  Bless them; bless them; bless them, because 
it's *MY* mail data as well at stake! =:^)

Meanwhile, as the main kde versions have continued to roll out, additional 
minor tweaks and bugfixes to the existing kdepim 4.4 branch have been 
necessary to keep it in sync.  Thus, kdepim's current version number of 
4.4.10.

However, remember what I said about kdelibs.  I'm quite sure you have it 
installed as a dependency for kmail, etc, as well.  So when I talk about 
updating to 4.6.1, I'm talking about updating kdelibs and any other 
kde-base/* packages you have merged OTHER than the kdepim based ones, 
which will have the corresponding kdepim 4.4.10 version number.  As it 
happens, all the apps you mentioned are kdepim based, so it would appear 
most of your kde4 apps will be upgraded to 4.4.10, but kdelibs and 
anything NOT kdepim (that's still in kde-base) would be upgraded to 
4.6.1.  (If you have anything like k3b or amarok merged in the kde4 
versions, they'd of course have their independent version numbers, as 
they're independent apps not shipped as part of kde itself.)

OK, NOW the bit about upgrading to either the latest current, 4.6.1, or 
the last bugfix release of the previous minor, 4.5.5, should make more 
sense.  That would be kdelibs and whatever else non-kdepim you have from 
kde-base.  I believe kdepim 4.4.10, thus kmail-4.4.10, kontact-4.4.10, etc 
should work with either one.

As for stability concerns AFTER they release a stable kdepim > 4.4.x, as I 
said, the kdepim folks have demonstrated pretty much the exact REVERSE 
policy of kde4.  As such, I'm actually quite confident that they're doing 
everything within reason, arguably and then some, to ensure a stable 
transition.  Arguably they're going overboard with it.  However, given the 
reputation kde4 has at this point, I'd say that's a GOOD thing, as the 
LAST thing kde4 needs right now is another botched upgrade, particularly 
with something as valuable as people's mail archives!  So I actually 
expect this to be one of the smoothest upgrades ever, when it happens.  
But I'll still make sure I have the current data backed up before I do the 
conversion.  And if you're paranoid, there's nothing keeping you from 
holding off for a few more months after the initial release (and in fact, 
if you normally run gentoo-stable, that's likely what it'll take them to 
stabilize it anyway), to see if there's a vast outcry of people losing 
their data as they try to upgrade.

But meanwhile, if anything, the extended updates to the kdepim 4.4.x 
branch should mean that it's about as stable as it gets, by now.  IMO, 
there's little reason NOT to take advantage of that and get the now ultra-
stable 4.4.10 version while it's there.

That takes care of that angle, but there's another related angle that 
should be considered as well.

One of the difficulties with early akonadi was what it used as its 
backend.  Sqlite wasn't at that point thread-safe.  Mysql was the default, 
but, it turned out, it too had problems, as it's really designed to be 
used by professionals, those who know how and are willing to tweak it to 
keep it running properly.  As such, it was a relatively poor solution for 
use by "ordinary users" as an akonadi database backend, because it really 
requires too much maintenance.  There were many complaints about error 
messages, warnings, akonadi crashing or refusing to start, etc, mostly due 
to relatively minor issues -- or issues that WOULD be relatively minor to 
professionals, but proved NOT so "relatively minor" to the "ordinary 
users" trying to run "this buggy kde4 c***."

Happily, the problems with sqlite were worked thru, and it's rather more 
thread-safe in its current incarnation.  As it's not designed to be a full-
scale database but rather to be used as a library within other apps, it 
"just works" in terms of ordinary users, and that's what has been needed.  
The big problem was the thread-safety and that's now resolved.

As a result, with akonadi-server-1.5 (and IIRC 1.4 as well, but NOT 1.3, 
the gentoo-stable version), the default backend has been switched from the 
earlier mysql default to sqlite.

Unless you consider yourself a mysql guru, and thus able to deal with 
whatever it throws at you, I'd STRONGLY recommend that you upgrade and 
switch to the sqlite backend.  I know I've had FAR less problems since I 
switched to the sqlite backend than I did before, for sure!

There is, however, one caveat.  Gentoo's merges don't mess with user 
config, only the standard system config.  But akonadi stores its backend 
settings per-user.  So after the upgrade, you'll want to edit the user 
akonadi configs to use the sqlite backend as well, because otherwise 
they'll still try to use the mysql backend since that's what they were 
configured to use before.  The akonadi-server emerge logs a message to 
that effect, listing the specific user file that needs changed, but IMO it 
doesn't stress the importance of the issue to the extent that it should, 
and people are likely to blow it off as a result.  If they blow it off and 
they did NOT keep the mysql backend (depends on the USE flags), they'll be 
stuck with a broken akonadi-server until they fix it.  If they kept the 
mysql backend as well, they'll still be using it, which will continue to 
work to the extent that it did, but as I said, that backend has more 
problems that most folks simply aren't equipped to deal with (I had to do 
a lot of googling to fix it here, a couple times), so switching to the 
sqlite backend, now that the sqlite backend actually works, is DEFINITELY 
preferable.

Now I don't use korganizer or kontact myself, only kmail, so I don't know 
to what degree they may be already akonadified.  With kmail, as I said, 
it's only kaddressbook that's akonadified.  So the above akonadi-server 
backend bit will only apply to the addressbook, with current kmail.  If 
you don't rely on the addressbook, it may not be that important to you, 
currently.

However, as we know by now, when the new kdepim hits, with its kmail 
rewrite, it'll be fully akonadi dependent as well.  So the added stability 
of the sqlite backend will certainly help kmail itself, in the future, 
thus even if you choose not to worry about the upgrade now, make a point 
of remembering to do that user config edit when you do finally do it, as 
it should help, possibly dramatically, with akonadi stability, and once 
that kmail rewrite hits, akonadi stability is kmail stability!

That's it.  As I said, rather complicated.  Hopefully I explained it well 
enough so it all makes sense, now. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman