[Owncloud] Thoughts on converting a Chromebook to an Ownbook

Thu May 16 11:03:11 UTC 2013

Hi I looked into the storage problem some time ago, and I think Klass could
solve it with some evolution of the current client (and its local usage of
SQLite), but to justify my thinking a little bit of background analysis is
required:

File usage has some pretty consistent characteristics amongst my near peers:

There is a high frequency usage of small files and low frequency usage of
large files i.e. powerlaw relationship:
1. At the one end of the spectrum I want files that I use frequently on my
local device(s), so I do not have to wait for download.
2. At the other end of the spectrum I have files that I rarely use, but
want access to at some unpredictable point in time within the next two
decades.
3. Files I use often are relatively small, and in contrast, the files I
rarely use are rather large.
4. There are lots of small files and less larger ones.

Behaviour:
5. Users want access to their files from multiple devices of different
specification; a personal example is:
5.1 I have* 2Gb for files on my mobile
5.2 I have* 70Gb for files of 128 Gb SSD on my laptop
5.3 I have* 90Gb for files of 160Gb desktop.
5.4 I have* 3 Tb for files on my OwnCloud server (always online, with fixed
tcp/ip) - this could very well be other scale out services as well.
5.5 Each device will be online at random times, and they can connect
securely together:

The trick that makes it *irrelevant* how much local disk space I have, is
that I permit LOCAL OVERWRITE IFF there is GLOBAL REDUNDANCY (via
OwnCloud).

If a file is updated on the one device I want it up-to-date on the other
devices as soon as possible (via web), but only IF it is used MORE THAN
ONCE on the other devices. This means all files are listed (A lot of
JSON-type symbolic file-links that take up 1-2kb max), but all files are as
JSON-links until used the first time (just like as with google drive
installed on windows).

This requires that the OwnCloud client monitors the file system (iNotify or
similar), and uses statistics of file-usage PER DEVICE PER USER to stay
up-to-date.
First it must assure that 1 copy of local files are up-to-date on the
OwnCloud Server whilst the OwnCloud must have sufficient capacity to "back
the local file up".
Second it must prioritize synchronization as a schedule, and not simply
perform a braindead mirror of N nodes (as it does currently).

For practical purpose of such prioritization a vector that contains
exponential function of "time since last usage" x "frequency of usage"
would suffice.

The point here that the user NEVER runs out of space on a single device,
because the files which are not used after a certain amount of time are
release for overwrite the moment the space is needed for other -  arguably
more relevant files - or they are simply not downloaded to the local device
(because they exist locally as pointer with JSON link).

Please notice that overwritten in this context means that the
"111010101"-blob on your LOCAL device is substituted with a JSON link to
the copy on OwnCloud (if this is more feasible utilisation of the limited
space on the client).
The local pointer to the file X is only change from the pointer named X
that refers to iNode so-and-so-much to a pointer named X that refers to
JSON that points to OwnCloud.
Hereby the local file X is released to be deleted/overwritten (depending on
the HW-driver setting) whilst UI the content underneath the UI simply
changes to point at the OwnCloud. Obviously this overwrite is only
permitted IFF the file exists on the OwnCloud (i.e. has been uploaded).

So the trick in summary: Files which occupy a higher frequency of usage or
generally just are new on the LOCAL devices, everything that does not fit
in the space available locally exists on the OwnCloud, but is visible on
the local device as a JSON-link that from the UI is visually identical the
real file.

For Klaas it means that iNode will change a lot (sorry) and that sha-256
should be packed into the JSON link or the sqlite db. I know Klass is
working on that already.
Also the client could read the sqlite database for updates instead of
visiting the filesystem. To check if two databases are up-to-date a sha-256
of the database itself may be exchanged which drops the required
communication (client-server) significantly at the first connection. The
knowledge alone can permit a less power-consuming startup of connectivity
in stead of the current where bandwidth is consumed naively. However these
details are just the top of the iceberg. But I think OwnCloud client is the
best match "out-there" for taking a step in that direction.

A wider intellectual challenge is to predict behaviour across devices which
are idle are predict which files are to be used so that local devices are
up-to-date using low-power connection BEFORE you pick them up and use them
but I'm still working on that :-)

Kind regards,
Bjorn

On 16 May 2013 07:52, Jan-Christoph Borchardt <hey at jancborchardt.net> wrote:

> As a Chromebook owner who put Ubuntu on it: The main concern or solution
> here seems to be to make ownCloud cope better with being offline. Use more
> client (JS) and less server (PHP) logic, use local browser databases for
> data caching (localStorage, indexedDB) and cache the apps themselves
> (AppCache).
>
> That way there's also less dependence on the server and better performance
> because not everything has to be loaded again so often.
>
> Of course this involves major code forward, but the Appframework and newer
> apps already are on the path to this. Also solves one of my pet peeves: the
> files app reloads every time you click a folder …
>
> This would also make ownCloud better compatible with Firefox OS. Would be
> quite strange to make "thin client apps" for the "fat web frontend" instead
> of just improving it for everyone.
> _______________________________________________
> Owncloud mailing list
> Owncloud at kde.org
> https://mail.kde.org/mailman/listinfo/owncloud
>

-- 
Bjorn Madsen
*Researcher Complex Systems Research*
Ph.: (+44) 0 7792 030 720
bjorn.madsen at operationsresearchgroup.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/owncloud/attachments/20130516/77fa8f60/attachment.html>