[Owncloud] Sync Client needs server help

[Brad McEvoy]

On 19/05/2012 1:38 a.m., Klaas Freitag wrote:
> I understand the concept and indeed its good. It's very near to what I 
> want to implement, with the only difference that instead of the hash 
> sums, I'd like to use the mtimes, as csync does. Why do we think thats 
> a benefit: Well, based on the mtimes its decideable which version is 
> newer. Moreover, the mtime is already a natural meta data in each file 
> system, so we do not have to add something new. That given, csync runs 
> without server support by now.
My experience agrees with Evert, basing sync decisions on the relative 
value of dates can result in files being incorrectly overwritten.
>
> What is missing is the propagation of the mtimes from individual files 
> and directories to their parent directory. If we do that with the 
> ownCloud server support, I think we will have the same benefits that 
> you described above. As we have the data in a database server side we 
> will be able to retrieve the data fast.
If you're updating parent directories anyway calculating directory 
hashes is trivial. Providing that information to the client can make the 
client much more efficient.
>
>>
>> Note that there is a related RFC - http://tools.ietf.org/html/rfc6578 -
>> however I'm not confident that the approach outlined there is quite 
>> right.
> Do you know if its implemented in a WebDAV server already?
This standard seems to have emerged from some corporaty types, so 
there's probably something from ibm or oracle
>
>> Of course finding what files are new or updated is one thing,
>> communicating those changes efficiently is another. Spliffy uses a
>> similar approach to Bup (https://github.com/apenwarr/bup) to split files
>> into blobs which are stable with respect to file changes. Only changed
>> blobs are transmitted.
>>
>> The hashsplitting algorithm is **very** simple, and if you're not doing
>> something like this yet i suggest you take a peek -
>> https://github.com/HashSplit4J/hashsplit-lib
> Thats cool and is a problem we also still have on our list to tackle.
> I stumbled over this already and wonder if there is a C or C++ lib for 
> that.
its really easy. I knocked it up in java in about an hour while watching 
telly, albeit with a couple of major simplifications over Bup. Bup has 
an arbitrary depth of "fanouts" (ie groups of hashes) but its much 
simpler and perfectly practical to have a fixed depth of - root -> 
fanout -> crc.
>
>> Sorry for the long post, and I hope this is of some assistance.
> Great, I really appreciate your input.
You're welcome!