about akonadi

[Duncan]

Kevin Krammer posted on Thu, 25 Mar 2010 21:09:31 +0100 as excerpted:

> On Thursday, 2010-03-25, Duncan wrote:
> 
>> Next question, then. =:^)
>> 
>> Gentoo recently updated from mysql-5.0 to 5.1.  Apparently, mysql
>> doesn't always maintain database compatibility on minor upgrades

> That is a good question.
> I think none of the Akonadi developers were aware that MySQL was so
> unreliable. especially given it is  quite often being used in
> "enterprise" setups.
> MySQL seems to have a very different definition of minor releases, even
> configuration options stop working (or worse start generating errors).

There was a time for second guessing, and I've done my share of that, but 
now's the time to get behind the devs and try to make what we have the 
best we can. =:^)

> If I remember correctly some of the distributors (probably all by now)
> have found some kind of solution to this, but since I haven't run into
> this myself yet I am not sure how it works (just remember reading
> something related once or twice on distributor lists).

Rolling distributions will certainly have to treat this a bit different 
than the release-upgrade distributions, which can simply fold it into all 
the other stuff they do at the upgrade...

The solution for Gentoo (a rolling distribution) is a news item, 
distributed automatically and displayed by the package manager when the 
upgrade comes around, with a short (1 paragraph) explanation and pointing 
to the official MySQL upgrade page.  The pre-release gentoo-dev list 
(which tho I'm not a Gentoo dev, I follow for heads-ups on exactly this 
sort of thing) discussion of the news item is how I became aware of the 
whole issue.

It was the Gentoo MySQL maintainer that said he had his doubts on the 
wisdom of KDE basing their whole mail system, etc, on it.  Unfortunately, 
and this reflects a bit of a Gentoo-internal communications issue, he 
didn't even know anything about KDE/akonadi doing that until just a couple 
days before the discussion of the news release.  Obviously, now that he 
knows, there's going to be a bit more coordination between the MySQL 
maintainer and the Gentoo/KDE project in the future.

> Fortunately Akonadi is designed such that it does not strictly depend on
> a single database so we might see a switch in default some time in the
> future (current codebase sees a lot of improvements for Postgres,
> improvements on the Qt driver for Virtuoso could make that a compelling
> alternative especially for KDE users).

I've been very happy with virtuoso where it's used so far.  The 
complexities of Freedomware Java until a few months ago kept it off my 
system, tho I fortunately have IcedTea now.  But as such, the Sesame2 
backend wasn't workable, here.  And of course the other one (Raptor, was 
it?) was too slow to be practical.  Fortunately, Gentoo had made the whole 
Semantic Desktop thing optional until 4.4, an option which I took 
advantage of to disable it, and I wasn't entirely sure how things were 
going to work out when it was required for 4.4.  But I think the entire 
KDE community is breathing a sigh of relief, at how good virtuoso has 
actually turned out to be.  Of course, that's due to some developers 
putting in some HARD work on it, too, for which I'm sure I'm not the only 
one who's grateful. =:^)

>> Now, what happens with kde 4.5, when kmail is dependent on mysql, at
>> least for caching as we've seen, and these desktop users with little
>> clue they're even running mysql as it's simply a kde dependency, pull
>> in the next mysql upgrade?
> 
> I am not sure this is actually pulled in automatically. I am on Debian
> unstable and MySQL 5.1.x has been available for quite some time now, but
> my installation still says 5.0.y so my guess is that unless something
> explicitly requests the new version the old one is being used and
> satisfies the dependency tree just fine.

Again, that's going to depend to some extent on rolling distribution vs. 
release-upgrade distribution.  And on Gentoo, which is rolling, there's 
the --deep package manager option for --update, which pulls in the latest 
dependencies all the way down the stack (often requiring revdep-rebuilds 
as a result), vs. the non-deep --update behavior of just updating the 
"leaves", until something actually requires the newer version.

But a lot of people, particularly those on ~arch (testing, sort of half-
way between debian testing and unstable, I think) such as myself always 
use --deep, as we like to keep everything on the latest versions -- part 
of the reason we run ~arch in the first place. =:^)

For those sorts of folks, it's thus definitely an issue, even if it could 
be argued to be a self-imposed issue.

FWIW, I just did the upgrade and tried things out the way they were, as 
the MySQL maintainer had said the changes between 5.0 and 5.1 would only 
affect /some/ users, and I figured it was just the address book database 
anyway, and I still had the pre-akonadi backups available if I needed 
them.  Obviously I'd have been more careful if it was all of kmail at risk.

I didn't notice any problems with kaddressbook as a result, but there were 
some minor akonadi errors, mainly of the "the log has errors" type, but 
akonadi started and everything seemed fine, and a couple KDE starts later, 
the errors worked their way thru the logs and I've had no more issues.  Of 
course, those errors might have been due to the occasional X related 
lockups I'm getting, as I'm running pre-release versions of Mesa, the X 
radeon drivers, and the kernel (which I normally run git versions of), in 
ordered to have working OpenGL at all on my radeon hd4650.  So I'm not 
positive it was MySQL upgrade related, after all.

But the one thing I did notice is that while the akonadi test and error 
dialog does link the userbase troubleshooting page, and it does have some 
useful information, it's not as good as it could be.

In particular, the error dialog said there were errors in the log, but 
there's nothing in it OR on userbase telling me where the log actually 
IS!  I found that frustrating, as I'd have liked to actually see what the 
errors were, and as I expected they were just transient, and if I blew 
away the logged errors, it'd "just work".  But the errors worked their way 
thru the system and quit throwing up the error dialog before I had a 
chance to really investigate.

But yes, if the error dialog or at least userbase could say where the logs 
with the errors actually are, so people can actually go read them and see 
what's going on, that'd be very useful. =:^)

>> Is that going to break kmail [...]  Or will akonadi detect the problem
>> and automatically rebuild its cache/indexes/whatever
> 
> Rebuilding the cache can basically be done on-demand, i.e. as mail parts
> (e.g. headers, body) get requested they will be cached. Big problem of
> course for mail not stored locally, e.g. on an IMAP server, so my guess
> would be that this would be a last resort.

That's good, particularly for those of us using POP3 not IMAP.  But even 
for IMAP users, that it's able to detect and rebuild the cache on demand 
as necessary does say good things about the robustness.  That's the thing 
I've seen various folks (me, the doubt the Gentoo MySQL maintainer 
expressed when he learned kmail was going to be using MySQL, tho he 
obviously didn't know to what extent yet, several blogs I've seen...) 
worried the most about.

> Another problem with the rebuild approach is the reverse caching, i.e.
> modifications to data not yet written to the backends (e.g. mails
> originating from an IMAP server being marked as important while the
> system is offline, thus changes being only present in the Akonadi
> cache).

That's not good, but it's understandable.  And it's not the tragedy I 
think a lot of folks are fearing, that they're going to suddenly find a 
two-year (or worse) gap in their decade-old mail archive, due to kmail/
akonadi/mysql simply "eating" them!

If I may suggest it... This thread is clearing up a lot of my /own/ fears 
and misconceptions.  But I know I'm not the only one as I've seen various 
other grumblings.  It'd probably be worthwhile for the akonadi/kmail devs 
to do a kmail akonadi myths debunked article or blog post, preferably high 
visibility (PR/newswire release, get it carried on LWN, the Dot of course, 
KDE front page link, etc.), and have it appear sometime before the 4.5 
release.  Perhaps do a little one now, and another one, probably the big 
one, in the two-week lead-up to 4.5.0.  Because there's a LOT of misplaced 
myths doubts and fears out there!  Replacing them with solid facts can 
only be a good thing.  I know I'm feeling much better about it personally, 
after just this thread.  (Feel free to use some of what I've said as 
starter points for those myths.)

Of course, for all I know, maybe it's already in process.  =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

___________________________________________________
This message is from the kde mailing list.
Account management:  https://mail.kde.org/mailman/listinfo/kde.
Archives: http://lists.kde.org/.
More info: http://www.kde.org/faq.html.