[Owncloud] App Data Migration

[Bartek Przybylski]

Hi all!
I talk with tom about possible enhancements about on migration, heres
a part of our talk

<tpn> trying to work out how it could be done simpler :)
<aqu> it was a suggestion to decrease overhead for each app
<tpn> I see the idea :) and it would be great if devs wouldn't have to
write any migration stuff but I just don't see how it's possible :P
<aqu> my idea:
<aqu> app dev declares app.migration in app main dir with some syntax
<aqu> ie: a xml with something like this
<aqu> <migration>
<aqu> <db>
<aqu> <table name="tablename">
<aqu> <field name="name" type="type">
<aqu> ... // more fields
<aqu> </table>
<aqu> </db>
<aqu> <files>
<aqu> <dir path="path"/>
<aqu> <file path="path"/>
<aqu> </files>
<aqu> </migration>
<aqu> then while reading it you can easly determine users which are on
server and export them as needed
<aqu> also on <table> you will need to provide user_id_column for
specified scenario ;)
<aqu> if no user_id_column would be specified you import path as is
<aqu> or similar mechanism
*tpn runs through scenarios in head :P
<tpn> how would you handle things like the bookmark tags then?
<aqu> tag is stored in db right ?
<tpn> each tag has an id relating to a bookmark.
<tpn> yes
<tpn> but the id would change on import
<aqu> if you dont handle it user would need to handle it and user dont
want to do a lot of stuff
<aqu> you might do like this
<aqu> <field name="id" key_field relate_to="tablename.id">
<aqu> you would need to implement some more flexible backend but it
would be easy to use and thats is what we are aming for right ?
<tpn> yes
<tpn> I have a month to get it done :P
<aqu> i think its possible to do it since you already have migration
class in simple format ;)
<tpn> just need to make sure the solution coverss all bases
<tpn> let me think about files_sharing, that will be a tricky one

Tom also ask me to provide xml example for bookmarks app, so here it is :

<migration app_version="0.2">
  <db>
    <table name = "*PREFIX*bookmarks">
      <field name="id"/>
      <field name="url"/>
      <field name="title"/>
      <field name="user_id" user_id/>
      <field name="public"/>
      <field name="added"/>
      <field name="lastmodified"/>
      <field name="clickcount"/>
    </table>
    <table name="*PREFIX*bookmarks_tags">
      <field name="bookmark_id" relates_to="**PREFIX*bookmarks.id" />
      <field name="tag" />
    </table>
  </db>
<!--
   <files>
     <dir path="path_to_export"/>
     <file path="file_to_export"/>
   </files>
-->
</migration>

table tag doesn't have to contain all fiels, but only those to migrate
(they must be null possible)
in case of questions to my idea don't hesitate to comment and ask

bartek

2012/3/3 Tom Needham <tom at owncloud.com>:
> Could you describe this process in more details ? then we could look
> for enhancements ;)
>
>
> OK. Apps will provide an import and export function. See and examples for
> the bookmarks app
> here: http://gitorious.org/owncloud/owncloud/blobs/691103acd5aad2673b6375726ba24fb56e88451b/apps/bookmarks/lib/migrate.php
>
> When ownCloud needs to export a user, it will run
> OC_Migration::export($userid). This will then execute all of the 'export'
> functions for apps that have registered as migration providers. Once it
> receives data back from these functions it wraps it all up into either XML,
> JSON or something along those lines and exports it to a file. When
> importing, its pretty much the reverse of this, except each app will be
> passed its portion of the export file. It can then run through that and add
> items back into the database.
>
> The backend will do the dirty work to make it easier for devs. For example,
> on export it will add in information from the apps info.xml file so this can
> be accessed when importing. The backend will also pass a $uid to the import
> functions because this may have changed from the original install (because
> of a conflict on the new install).
>
> The question is how we should export the data? XML, JSON... ? I tried using
> JSON this afternoon and it saved a lot of time because apps could just pass
> an array structure for their 'export' function and then all the backend has
> to do is merge some arrays and run json_encode(). Also creating arrays is
> much easier than playing with DOMDocument ;)
>
> Also, I found a function that will format the json output so that it is
> still human readable.
>
> Thanks,
> Tom
>
> Tom Needham
> tom at owncloud.com
>
>
>
> On 3 Mar 2012, at 20:37, Bartek Przybylski wrote:
>
> Hi Tom!
> Could you describe this process in more details ? then we could look
> for enhancements ;)
>
> bartek
>
> 2012/3/3 Tom Needham <tom at owncloud.com>:
>
> Hi All,
>
>
> So I'm currently doing some work on a backend for app data migration.
>
> Basically apps can register as migration providers (much like they do with
>
> search) and provide and import and export function.
>
>
> Originally I was planning to use XML as the overall output format but we
>
> just had a chat in IRC and possibly JSON would be easier? Apps could just
>
> provide an array structure on export and be given the same one on import.
>
> The backend will do all the hard work of mashing all the arrays together,
>
> and adding in various other data (app version numbers, owncloud version
>
> number..) and then just run json_encode() and export to a .json file.
>
>
> What does everyone thing? Thought I'd get your feedback before going ahead
>
> with it.
>
>
> Cheers,
>
> Tom
>
>
>
> Tom Needham
>
> tom at owncloud.com
>
>
>
>
>
> _______________________________________________
>
> Owncloud mailing list
>
> Owncloud at kde.org
>
> https://mail.kde.org/mailman/listinfo/owncloud
>
>
>