Java part rework - concept

[Robert Schuster]

Hi all,
finally I found the time to do a write up of the concept of the Java
part rework that Bogdan, Christian and I talked about at QCS in Berlin.
Attached is (hopefully) a drawing that shows the individual components.
This mail is supposed to be a complete explanation of the whole topic.
For a discussion on IRC later this day it would be nice if you've read
it. :)

Base technique: At QCS we found out that it is very impractical to have
that many Java classes in sourcecode form in each application. Simply
because every class becomes part of the public interface. This will
cause a huge maintenance burden. I'd simply say it would not work. We
need to move large parts of the code into places that are independent
from the application. This however strongly required the need to add
classes from a foreign APK or DEX file to the application's classpath.
We were not sure whether Android would allow this but it indeed does. As
a proof of concept we wrote a demo[0] application which can load any (!)
APK or DEX file and create an instance of any class and call its
toString() method. The POC worked in the way we wanted on a rooted but
also on a normal locked device.

With this technique at hand we can do the following:
* Move all code that is not really necessary for app startup into
separate APK files
* Define an interface between App and Loader which can handle updates
(that allows for incompatible changes in the App code over time)
* Profit! Err, yeah ... ;)

Let's explain all the little boxes from the drawing:

App:
----
This is the source code that is part of each Android-Qt application. The
user *may* change little pieces of the code as long as it does not
violate the interface to the Loader or Ministro. What the user can do is
calling any methods of the public interfaces to the loader for example.
We will mark the pieces of code that the user *must* not remove or
modify otherwise.
This whole thing will be as small as possible!
Whenever there is the need to change the App code in an incompatible way
we need to introduce a new Loader interface. E.g. we will have a
LoaderVersion1 interface for now. When it changes the App code will ask
for LoaderVersion2. The Loader's reponsibilities will be explained below.

Loader:
-------
The Loader connects methods from the App with the Android Activity
class. It will implement all available methods in order to gain maximum
control over the App code without letting the App decide by itself. The
Loader is published by us and can as such be updated over time allowing
*us* to fix bugs, introduce new feature to *existing* applications
without changing them.
There might be applications with different versions of the starter code
in it. Each of those version requires a specific interface in the
Loader. The most up to date version of the Loader will understand (=
implement) and offer all interfaces. That way old and new application
start code is always compatible with the Loader.
If a Loader cannot satisfy the requested interface of an application
then this means that the current version of the Loader is too old and it
will update itself.
(The Loader will technically be part of Ministro, when Ministro is
installed. If the user is a developer and chose 'use local libs' then
the loader will also be copied to a well known directory and directly
included into the Apps classpath. As such at runtime of an App the
Loader APK will be part of the App's process space!)

Ministro:
---------
Ministro does what it does right now: It downloads the libs (if
necessary), provides the locations of the libraries and tells the app
that it can start. I'm actually thinking of putting the 'satisfy a
certain Loader interface version' into Ministro, too. Because that seems
like the best place for it).
Unlike now the interface between App and Ministro will become upgrade
compatible. While changes will happen less frequently it is very
important that we allow and can deal with incompatible changes to the
interface between Ministro and the App.

Bridge:
-------
These are all the classes that interact in some way with the C++ code of
the Android-Qt port. Unlike now they will be a private interface.
Meaning that we can do *any* change we want in them. For each supported
Qt version (4.9, 5.0 etc.) there will be an accompanying bridge. The
bridge may even consist of multiple APKs if we want to save space (e.g.
if QtMobility is not used, then the classes for it do not need to be
available as well).

Qt:
---
These are the native Qt libraries. No change needed. Except that any
code can *assume* the existence of certain bridge classes and can
resolve class names at will.

Caveats:
-------
From a development perspective this is quite a task. The Java code
changes are not so difficult (for me at least) but I have no clue how we
can modify the Android-Qt build to also compile Java classes, dexify and
make an APK out of them. Because this is what we need. When you compile
Qt for Android you'll need a Java compiler, the DEX tool and something
that creates an APK for you. IOW this is where I need you help.

The Ministro build also needs to be changed a bit. The Ministro APK
could contain the Loader classes. That would not be a problem. However
somehow QtCreator needs the Loader classes too when using 'use local
libs' flag. I propose that the SDK installer also downloads the Loader
classes (which might be the same as Ministro but as a library not an
application) from a known location.

I plan to use real Java interfaces in order to define the public
interface. An alternative approach would be to rely on reflection only.
However that will make the interface very opaque and the code will be
difficult to read for anyone with only a small knowledge of Java. Having
interfaces however means we will need to share bits of sourcecode
between the projects (namely the interface file). I'm still
contemplating how to do this properly.

For tonight I am mostly interested in how to solve the caveats.

Let me know what you think. If not we'll see each other at 7pm CET/GMT+1. :)

Regards,
Robert

[0] - http://paste.debian.net/122188/

-- 
tarent solutions GmbH
Thiemannstr. 36 a, D-12059 Berlin • http://www.tarent.de/
Tel: +49 30 5682943-30 • Fax: fax +49 228 52675-25

Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 52675-0 • Fax: +49 228 52675-25
HRB AG Bonn 5168 • USt-ID (VAT): DE122264941
Geschäftsführer: Boris Esser, Elmar Geese

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Java part rework.png
Type: image/png
Size: 62551 bytes
Desc: not available
Url : http://mail.kde.org/pipermail/necessitas-devel/attachments/20110707/9838a10c/attachment-0001.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 729 bytes
Desc: OpenPGP digital signature
Url : http://mail.kde.org/pipermail/necessitas-devel/attachments/20110707/9838a10c/attachment-0001.sig