[Kde-pim] Making offline/online handling sane

[Andras Mantia]

Hi Kevin,

Kevin Krammer wrote:

> Hi Andras,
> 
> thanks for looking into this,
> 
> On Saturday, 2012-12-01, Andras Mantia wrote:
> 
>> The cause
>> ---------
>>   For the above, the cause is pretty simple:
>>    if ( !isOnline() ) {
>>      const QString errorMsg = i18nc( "@info", "Cannot fetch item in
>>      offline
>> mode." );
>>      emit error( errorMsg );
>>      return errorMsg;
>>    }
>> in ResourceBase::requestItemDeliveryV2.
>> Sounds simple and reasonable at the first read, but here is a question:
>> why do we have it in the ResourceBase?
> 
> It is something all resources should do consistently and ResourceBase is
> code that all of them use, so IMHO the most logical place to put it.
> 
>> The real problem
>> ----------------
>> 
>> The answer to the above question is: because we don't know what
>> "isOnline" means for a resource. There is a weird mix of its meaning in
>> the Akonadi libraries and resources. Does it mean that the resource is
>> turned off? Does it mean that the resource is not allowed to do network
>> operations? Does it mean a mix of the two or something else?
>> Well, the problem is that right now it is a mix.
> Indeed.
> Offline in its wider sense means not operational which is what we are
> currently using it for. However, as you've stated above, the problem is
> that we have many different reasons why the agent reaches that state and
> we can then later no longer determine which reason(s) that were.
> 
>> From the above it means
>> that Akonadi should not even *try* to get anything from a resoruce that
>> is offline, as it will get an error, so it means "offline resource is
>> turned off resource".
> 
> Right, which is intended behavior.
> 
>> But a resoruce is automatically turned offline if there is
>> no network. That means a resource is offline when there is no network.
>> And this is *generic*, the code is in AgentBase that turns them offline.
> 
> Right, which again makes sense to ensure consistent behavior. We could
> split it out in an easy to use helper class though, so agents that
> currently call "needs network" could then instantiate the helper instead.
> 
>> Even more: if a resource is offline, the task scheduler inside the
>> resource
>> does nothing. Task processing is shut down (see  void
>> ResourceScheduler::scheduleNext() ). This means not even a local resource
>> works on offline mode: no task are processed.
> 
> Right. A resource or agent that is offline should not perform any work.

So far we agree how it works, which is good. :) What is not good, is that we 
don't agree this is good or bad. I think it is bad. 
The above feature is "turn off the resource", not make the resource not work 
on the network (what offline really would mean).

>> The suggested solution
>> ----------------------
>> Hereby I suggest to bound the "online" state to the network operations.
>> Online means a resource is *allowed* to perform network operations.
>> Offline means a resource is *not allowed* to perform network operations.
>> The Akonadi libraries (and here I mean AgentBase and ResourceBase) don't
>> deal with online/offline state, the most they do is to signal the network
>> state, although this would just replicate a Solid signal, so no real need
>> for that either. Akonadi *should not* care about what the resources do or
>> allowed to do.
> 
> So if I understand this correctly online/offline would then only reflect
> the state explicitly set by the user (manually or through an application's
> policy).
> Offlines as in "has been take offline".

Offline will mean that the resource is not allowed to do network operations. 
Yes, indeed this will be a request from the user or client application.
If there is a network problem or some other problem, the resource should be 
in an error state (Online, Broken or such).

>> Online/Offline handling will be moved to the resources. If a resource
>> doesn't work on the network, it just ignores it: it cannot be set
>> offline.
> 
> Doesn't that contradict what you've written above? If offline means the
> user does not allow the resource to perform work, why would a user find it
> OK if some resources ignore that wish?

I see no contradiction. Offline will not mean anymore the resource is not 
allowed to perform *any* work, it will mean it won't be allowed to perform 
network operations.
The reason this cannot be in ResourceBase or AgentBase is that we can't 
control what the resource does or not. We can either completely shut it down 
(like we do now) or not. What I propose is to make the offline a flag that 
the resource can use to turn the network operations off, if it needs 
network. If you still want the current offline=turned off behavior, we can 
keep it, but call it the right way.
What I want is to remove the confusion behind the offline term and of course 
to fix the IMAP resource, so I can read my mails also when I have no 
network. That's the whole point of the disconnected mode after all. ;)
And I believe without making a distinction between offline and turned off 
there isn't a clear way to fix the problem.

>> If it wants, it can give back items in another way (think of local cache
>> per resource).
> 
> Akonadi *is* the local cache!
> One of the major reasons to have it in the first place :)

Sure, I know. This doesn't mean it cannot have temporary caches. It was just 
an example. A local resource (maildir) might be a better example: it doesn't 
need network, so it shouldn't have an offline state.

>> The questionmarks
>> -----------------
>> 
>> I couldn't do a complete analysis of the whole code yet, I run out of
>> flight time. :) And there are some questionmarks in my mind:
>> - why can you read offline imap content if you set KMail to offline (but
>> you cannot if you have no network):
> 
> Assuming "KMail offline" means KMail told all/some email resources that
> the user wishes them to remain offline?


KMail Offline means that resources are manually set to offline mode. That is 
what KMail does. (File->Work Offline menu)

>> here I just have a guess: if the IMAP
>> resource cannot connect to the server, it sets itself to offline mode
>> (another confusion, as offline in this context might mean "broken").
> 
> I don't think it means "broken" in the context of no network connection,
> it rather means "backend not reachable due to external constraints".

We can introduce a different state for that, if needed. 

>> If it
>> is in that state, it starts to report errors back to the Akonadi server.
> 
> If course, it is being asked for things it can't do right now.

Sorry, it is not clear for me what you mean here. Right now (with the 
current code), of course cannot report errors, as nothing reaches the 
resource in offline mode.

>> My suspition is that "KMail in Offline mode", but with working connection
>> doesn't set the imap resource completely offline and it performs some
>> operations instead of reporting back errors (like cannot connect to the
>> imap server).
> 
> No idea.While I worked on migration KMail's "offline" meant that it would
> call setOffline on resource for which this was configured.
> 
>> - why does Akonadi asks at all for items on a disconnected IMAP account,
>> when everything should be in cache? Note that my wondering is only for
>> the items. There is anyway the problem I said above, as IMAP sync folders
>> on click, while it can't do it in offline mode. You end up with some sync
>> task scheduled...
> 
> No idea either, maybe some property of the cache policy incorrect or
> missing?

This is something I need to investigate. A quick check at that time shown 
that "setCacheOnly(true)" was used, so it looked correct, but probably I 
missed something.

>> - how many resources would be affected, who relied on the current offline
>> interpration?
> 
> I think the problem is that offline is a combined state, thus there is no
> "current interpretation" that means a single thing.

That is what I try to fix. :)

> As far as I can tell we have three different reasons why an agent is
> offline: 1) user's explicit choice
> 2) backend/service not available, e.g. backend is a remote server, no
> network 3) agent has reached a "broken" state and doesn't want its
> scheduler to dispatch new tasks until the problem is fixed.
> 
> Any other interpretation/reason come to mind?

I agree with the above, and I find it wrong.

> 
>> Anyway, I'd not mind if somebody would try to reproduce what I see to
>> confirm it. How to do it?
>> - set up a maildir and a disconnected IMAP account
>> - sync them, especially the disconnected IMAP so you are sure mail
>> content is downloaded
> 
> What we expect after this stage is that all IMAP emails are fully cached
> by Akonadi and all maildir mails have their envelopes cached, right?

Well, not really. The main problem is that even *cached* items are not 
reachable right now. With disconnected IMAP (when the full mail is cached), 
you cannot read the mails, because the IMAP resource is accessed in offline 
mode, but it cannot react because it is "turned off". 
But what I would expect here is that offline means no network usage. Nothing 
else. 

> 
>> - in akonadiconsole set the maildir resource offline and try to read
>> mails (read those that you haven't read lately, so they are not in the
>> cache)
> 
> Expected result is that folders list their mails (all envelopes cached)
> but mails cannot be displayed due to resource having been "switched off"
> and bodies not having been cached?
> But IMAP mails still working because their bodies have?

Indeed, the expected thing (even with the current implementation) is to be 
able to access everything that is cached. 

>> - unplug any network cable, shut down any wireless acount, if possible
>> disable the wireless completely. Now try to read mail on the IMAP account
>> and switch folders. Alternatively restart Akonadi/KMail after you
>> disabled the network.
> 
> Since the resources are already offline by use intervention, we would
> expect the situation to remain unchanged?


As I wrote somewhere above, turning the resources offline or unplugging the 
cable results in different behavior, even if visually in both cases the IMAP 
resource is just "offline".

>> If you agree with the above approach, I volunteer to come up a solution
>> inside the akonadi libs (which will be easy, like removing code mangling
>> offline and leave there only the interfaces to set a resource
>> offline/online) and check the imap/pop3/mail dispatcher resources that
>> they perform well and do what you should.
>> 
>> The above suggestion removes the feature of turning off a resource. In
>> order to have that, we would need to introduce a new state for resources,
>> but I'm not sure we need it at all.
> 
> I know it is an advanced feature but if possible I'd like advanced users
> to be capable of telling the system not to download emails even if there
> is a network connection.

Sure, that is what "offline" would mean (and what I fixed recently, when 
Akonadi silently turned the resources online, even if they were requested to 
stay offline). My suggestion would be a step forward: separate the "do not 
download my mails" case from "stop working for now completely".

> 
> IIRC, during the migration work Laurent said that his use case is having
> KMail running at home and at the office and switching the one he is
> currently not using to "offline". He will probably not like it if he has
> to remove that feature and quit the KMail instance he's currently not
> using.

Sorry, I think you misunderstood me. I have the same thinking and problems 
like Laurent mentioned. I don't want to remove any feature like that. What I 
would remove *unless there is an opposition* is the way to completely turn 
off a resource.
If you still want that, we can keep, but I will/want to rename the feature 
to "Turned off" instead of "Offline" and make "Offline" do what it should 
do: work, but don't touch the network.

Andras
_______________________________________________
KDE PIM mailing list kde-pim at kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/