[Owncloud] Sync Client needs server help

[Frank Karlitschek]

And here my reply to the list ;-)



On 17.05.2012, at 18:25, Frank Karlitschek <frank at owncloud.org> wrote:

> 
> On 17.05.2012, at 18:23, Tom Needham <tom at owncloud.com> wrote:
> 
>> On 17 May 2012, at 16:43, Frank Karlitschek wrote:
>> 
>>> 
>>> On 17.05.2012, at 11:12, Klaas Freitag <freitag at owncloud.com> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> one of the biggest shortcomings of the sync client currently is that it does a full scan of its the ownCloud directories via webdav to query the last modified times. That causes load and other trouble. It would be great to find out if something has changed server side more cheaply.
>>>> 
>>> 
>>> Completely agreed. The current approach doesn´t work with lot´s of files anymore.
>>> 
>>> 
>>>> We have the file system cache which also has the mod times in the database. My idea is now, instead of querying every single file, I just issue a HEAD request on the top sync directory and get the latest modtime of all files in that dir back. If that is younger than the one I know, I have to do a sync.
>>> 
>>> This should be easy doable.
>>> 
>>> 
>>>> I know that it could be even more cool, ie. delivering the list of files back etc. but lets do small steps. Doing just one HEAD instead of querying the whole tree already will be great.
>>>> 
>>>> The implementation seems easy: Just get all database id's of the fscache table entries below the top directory of the sync dir and do kind of
>>>> SELECT MAX(mtime) FROM fscache WHERE id in ( list-of-all-ids-in dir );
>>>> That should be fast enough.
>>>> 
>>>> My question now is: How do we do that? Should we have another app called /files/sync? Or do we want to enhance the WebDAV server to be able to do the described logic if a HEAD request on a dir comes in?
>>>> 
>>>> I think the latter is more "within the concept" of doing the sync via WebDAV, OTOH a sync app could be useful anyway for other sync related server support.
>>> 
>>> 
>>> Both is fine I think.
>>> We will need some additional REST calls for the sync client anyways if we want to support new features. Examples are: Accessing the history via the sync client, make it possible to share files from the sync client or control the encryption. We need to build a new custom REST API for this anyways. So an additional call to fetch a list of changed files is easy and fit´s into this concept.
>> 
>> Would also be interesting to allow apps to somehow integrate with this api. For example, someone the other day wanted to access bookmarks from an outside app. Saves each app implementing its own API, auth....
> 
> Good point. I think we should integrate this kind of calls into the OCS API in different modules.
> We can use the existing OCS "private data" module for the general stuff that can be mapped to a key value storage.
> 
> Frank
>