[Kde-pim] PIM Sprint report: Akonadi Next

Wed Nov 26 13:17:41 GMT 2014

On Wednesday 26 November 2014 13:16:49 Daniel Vrátil wrote:
> On Wednesday 26 of November 2014 12:36:56 Milian Wolff wrote:
> > On Wednesday 26 November 2014 10:46:18 Daniel Vrátil wrote:
> > > Hi all,
> > > 
> > > the PIM sprint is over and we have some exciting future ahead of us!
> > > 
> > > Let's start with the largest change: Christian and Aaron came up with a
> > > proposal for redesign of Akonadi using modern reactive design patterns.
> > > We
> > > polished and sorted out a lot of details there, and I'm really excited
> > > about the plan, as it will allow Akonadi to benefit from modern
> > > technologies, making the codebase simpler, faster and more robust (I
> > > guess that's what you guys said about the current Akonadi too ;-)). We
> > > also hope to attract new contributors by using the new "cool" stuff. Big
> > > thanks to Till for his insights into current Akonadi design.
> > 
> > Can you share some more about the ideas you have? This teaser above has me
> > hooked, but it doesn't really say that much about what you guys are up to
> > :)

> Basically the new design absents central server in favor of using
> per-resource storage. Each resource would maintain it's own key-value store
> (allowing each resource to store data in a way that is efficient for them
> to work with) and would provide an implementation of standardized interface
> to access this storage directly. Clients would be allowed direct read-only
> access to these storages, while resources are the only one with write
> access. Internally flatbuffers would be used to provide access to the data
> in super-efficient way (lazy-loading, memory mapping, all the fancy stuff).

OK, cool. That sounds good to me. Actually it sounds extremely similar to what 
I have planned for the future of KDevelop as well. The global cache we use 
there is suboptimal, and now that we work on Clang support, it doesn't really 
fit right. So I'm looking forward to see your work on this in Akonadi, maybe I 
can even reuse parts of this stuff then :)

One question: Why flatbuffers? I'm not arguing against it, I just would like 
to know why that, and not one of the other cool kids on the block, most 
notably Cap'n'Proto. I only read this once: 
http://kentonv.github.io/capnproto/news/2014-06-17-capnproto-flatbuffers-sbe.html

So if anyone did an educated decision, I'd like to know the reasoning. The 
slides below seem to indicate that multiple things where researched.

Also, from a lessons-learned POV, I really urge you all to somehow wrap the 
serialization code such that it can be replaced easily later on. I know that 
I'd love to have that in KDevelop. Now I'll have to refactor tons of code to 
use anything else...

> Resources would implement pipelines allowing some pre-processing of newly
> incoming data before storing them persistently (think mail filtering,
> indexing, new mail notifications etc.). Inter-resource communication (for
> example to perform inter-resource move or copy), and client->resource
> communication would be done through a binary protocol based on what we have
> now in Akonadi. This design also gives us on-demand start/stop of resources
> for free. Something that requires ridiculous amount of work to make work
> with the current design.

Cool. But if you use FlatBuffers or anything else, why not use that instead of 
a custom binary protocol?

> API-wise, while we can't completely get rid of the "imperative" API of
> having jobs, the core method to provide access to data would be through
> models. Making use of storage data versioning, on update the model simply
> requests changes between current and last revision of the stored data. This
> should prevent us from ending up with overcomplicated beast-models like
> ETM.
> 
> This design is rather different from the current one, so we won't probably
> be able to reuse much of the existing code, hence the long timeframe.
> 
> Aaron has also shared some slides [0] that Christian presented on the sprint
> regarding this change
>
> [0] https://plus.google.com/+AaronSeigo/posts/Xt7HNZJV5UJ

Interesting, thanks. Slide 17 has me worried though. How do you intend to make 
this thread/multi-process safe, when you have read/write access from the 
resource and _direct_ read access from the client? That should probably be 
indirected to make each resource be able to use its own storage and centralize 
locking stuff (at least no multi-process issues then).

> > > Implementing these changes and stabilizing the project will take lots of
> > > time. On Akademy I mentioned that my plans I had with Akonadi would take
> > > roughly a year to implement. I expect that this will take at least the
> > > same
> > > amount of time, maybe even more, since I'll have considerably less time
> > > to
> > > work on PIM in the near future. I gave some thoughts to the current
> > > situation and I think that it might be a good idea to implement some of
> > > my
> > > ideas on the current Akonadi master, and release as soon as possible, so
> > > that people can get at least some bug fixes and performance improvements
> > > while we work on the "Akonadi Next". Since KDE PIM is basically ported
> > > to
> > > Qt 5/KF 5 (with kdelibs4support, but that does not really matter) and
> > > seems
> > > to work (more or less), we could try to release from master with next
> > > KDE
> > > Applications (15.04 or whatever the version turns out to be) without any
> > > guarantees and promises of API and ABI compatibility of the Frameworks.
> > > This is necessary, because we can't really wait with release until we
> > > can
> > > give this promise, as there are other projects depending on Akonadi
> > > (Plasma, KTp, ....) and we are effectively blocking them. Once we decide
> > > that the new Akonadi is ready, we should be able to simply switch from
> > > the
> > > old one to the new one and work towards stable API/ABI of the libraries.
> > > Laurent, and other PIM maintainers: opinions?
> > 
> > Releasing a library without any ABI guarantees is like not releasing a
> > library at all, no?
> 
> No. If you don't release a library, you can't have released applications
> using it. If you release it, you can have applications using it. The fact
> that there is not ABI guarantees is not a problem, you can still make sure
> that only the applications you want (you have under control) use it (let's
> say by not installing AkonadiConfig.cmake, but shipping FindAkonadi.cmake
> in the application.

So plasma will be coupled to Akonadi, and a change of ABI in Akonadi will 
trigger a rebuild of Plasma. Not nice, distributions will hate this.

> > What is Plasma, Ktp, ... actually using from Akonadi?
> 
> Plasma needs it for the calendar applet (it's getting a bit annoying that on
> almost every promo screenshot we are showing off our new awesome beautiful
> calendar applet without admitting it actually can't show anything except
> for current date). KTp uses KPeople, which has an Akonadi backend to get
> more info about metacontacts.

> > Could maybe a small shim library be created, specifically for that
> > purpose,
> > which can be shipped with an ABI guarantee?
> 
> Not really.

Why? I'm still not convinced. For the calender e.g. create a small library 
that can be used by the Plasma applet and be done with it.

> > Then the rest can be changed at will and these "external" apps can live
> happily.
> 
> No external applications. Instead we restrict the usage to applications we
> control and where we can make sure that they always require a compatible
> library (through soname bumps, version requirements...)

Plasma is still external from an Akonadi POV.

> > For KDEPIM and the other
> > "heavy lifters", which will definitely require more access to Akonadi API,
> > that is not such a big deal. Since that will probably be released together
> > with Akonadi anyways, no?
> 
> Yep.
> 
> > And it was/is planned to make more stuff internal
> > anyways, right?
> 
> Only Akonadi server is internalized. What is now in kdepimlibs.git/akonadi
> will all remain public.

OK, thanks.

> > > We also discussed some more repositories reshuffling:
> > > 
> > > kdepimlibs:
> > > * merge akonadi-{contacts,calendar,mime,notes,socialutils} to
> > > akonadi-extras * merge gpgme++ and qgpgme
> > > * finish killing kpimutils
> > > 
> > > kdepim-runtime:
> > > * agents -> akonadi-resources
> > > * resources -> akonadi-resources
> > > * plugins -> akonadi-extras
> > > * qml -> akonadi (src/declarative)
> > > * resourcetester -> akonadi-resources
> > > * kioslave -> could go to kio-extras, or akonadi-extras
> > > * migration -> akonadi-migration
> > > 
> > > kdepim:
> > > - no changes atm
> > > 
> > > I'm not mentioning the other topics like VDG and QML API as I wasn't
> > > there
> > > or wasn't paying enough attention :), but I hope others who were there
> > > will
> > > send their summary too.
> > 
> > Sounds like an interesting sprint! Rock on guys

-- 
Milian Wolff
mail at milianw.de
http://milianw.de
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/