Allowing Java-C++ interaction

Mon Jan 23 05:40:05 UTC 2012

This accidentally didn't get sent to the list; forwarding. Sorry about that.

---------- Forwarded message ----------
From: Tyler Mandry <tmandry at gmail.com>
Date: Tue, Jan 17, 2012 at 12:19 AM
Subject: Re: Allowing Java-C++ interaction
To: BogDan <bog_dan_ro at yahoo.com>

Hi BogDan,

That definitely could work. But it does add to the complexity, and possibly
start-up time, of the app. An alternative solution would be: simply to load
the bundled libs in the main classloader, UNLESS they are bundled Qt libs
that are used by the .jar, in which case they are loaded in the child CL.
This could be accomplished by separating these two kinds of libs into
different categories, or even auto-detecting them by their names at
runtime. What do you think?

(One thing I've found: native libs loaded in different classloaders can
always talk to each other, since they're part of the same process. In other
words, you could load core Qt libs in the child CL and QtWebKit in the
parent CL and it will all work. I don't know if this is useful but thought
it might be.)

Thanks,
Tyler

On Mon, Jan 16, 2012 at 2:22 AM, BogDan <bog_dan_ro at yahoo.com> wrote:

> Hi Tyler,
>
>
>    Thanks you for sharing this with us, it helped me to understand better
> what is the problem.
> I also check your patch [1] but it doesn't seem to fix all the problems,
> bundle libs will have the same problem, so I tried to find another solution
> which hopefully will handle all this situations. The solution is create a
> separate package with your own java files, and this package to be loaded by
> the same class loaded which loads the plugin .jar files.
>
> Please let me know if this approach is ok for you.
>
>
> Cheers,
> BogDan.
>
> [1] http://sourceforge.net/p/necessitas/tickets/140/
>
>
>   ------------------------------
> *From:* Tyler Mandry <tmandry at gmail.com>
> *To:* BogDan <bog_dan_ro at yahoo.com>
> *Sent:* Friday, January 13, 2012 6:43 AM
> *Subject:* Re: Allowing Java-C++ interaction
>
> Hi BogDan,
>
> What currently happens is the native library is loaded in the child class
> loader (by the code in the .jar, in fact,) so application code (loaded by
> the parent class loader) can't see any native functions loaded by the
> native library. And no, I don't try to access anything in the .jar in my
> application.
>
> I did make a simple testcase that fails before the patch and works after.
> Since the patch involves changing QtActivity.java, it's difficult to send a
> complete project and see the before and after. Instead, do the following:
>
>    1. Create a new Qt Quick Application project in Qt Creator, with an
>    Android target
>    2. (i) Replace main.cpp with the attached main.cpp, and (ii) add in
>    the Java source to the tree (copy the
>    android/src/org/kde/necessitas/example folder in attached archive to
>    android/src/org/kde/necessitas in the project)
>    3. Go to QtActivity.java and add (i) "import
>    org.kde.necessitas.example.simpleapp.MyNative;" at the top, and (ii) the
>    following at the END of loadApplication():
>
>         //do stuff after loading
>
>         MyNative nat = new MyNative();
>
>         nat.doSomething();
>
>
> [Modifying QtActivity.java is currently the only way I know to run your
> own Java code as the application loads.]
>
> After that, run the project and you should see "No implementation found
> for native Lorg/kde/necessitas/example/simpleapp/MyNative;.doSomething ()V"
> along with an UnsatisfiedLinkError in the log.
>
> Then point Qt Creator to a build of Qt with the patch and do the process
> again with that version of Qt. (**Of course you'll need to deploy and use
> local Qt libs!**) You should see "This is a native function, doing
> something!!" in the logs, and no crash.
>
> I hope this is clear enough and that it sheds some light on what I'm
> trying to accomplish :). Let me know if anything is unclear.
>
> Thanks for all your hard work!!
> Tyler
>
>
> On Thu, Jan 12, 2012 at 8:46 AM, BogDan <bog_dan_ro at yahoo.com> wrote:
>
> Hi Tyler,
>
>     I reviewed my code and seems to be ok, when I create m_classLoader
> (check QtActivity.java) I pass its parent correctly, so, even if your
> library is loaded by a class which was loaded by m_classLoader and there
> your package is not visible, m_classLoader should search also into its
> parent for that class.
>
>     I hope your package don't try to access classes from .jar file,
> because those classes are private, and they should be used *ONLY* by
> platform plugin.
>
>
> Would you please create a simple example which fails, it will help me to
> understand the problem better ?
>
>
> Thank you !
>
>
> Cheers,
> BogDan.
>
>
> + CC: necessitas-devel
>
>
> >
> >
> >Hi BogDan,
> >
> >
> >No worries, I just didn't want it to be overlooked :)
> >
> >
> >In a nutshell, any (user) Java code the application had couldn't see the
> user's native JNI code, because their native library was loaded in a class
> loader that was different from the (primary) class loader that their Java
> code ran in. This upset a few people that have custom Java and JNI code,
> like me :). The only way to make calls between different class loaders is
> to use reflection, so instead I made a way for the application native
> library to be loaded in the primary class loader.
> >
> >
> >That was a mouthful, but I hope it helps. Let me know if you have further
> questions!
> >
> >
> >Thanks,
> >Tyler
> >
> >
> >On Tue, Jan 10, 2012 at 1:49 PM, BogDan <bog_dan_ro at yahoo.com> wrote:
> >
> >Hi Tyler,
> >>
> >>Please accept my apologize for slow reply.
> >>
> >>I need a few days to understand what is wrong with current design and to
> check your patch, then I'll come back with feedback !
> >>
> >>Thank you very much for your patience !
> >>
> >>Cheers,
> >>BogDan.
> >>
> >>
> >>
> >>
> >>
> >>>
> >>>
> >>>BogDan,
> >>>
> >>>
> >>>Do you think this patch can be included in the next alpha? If not, what
> needs to be done?
> >>>
> >>>
> >>>Thanks,
> >>>Tyler
> >>>
> >>>
> >>>
> >>>On Thu, Dec 29, 2011 at 12:48 AM, Tyler Mandry <tmandry at gmail.com>
> wrote:
> >>>
> >>>Here is the patch (against alpha4 head 2469081) to make everything
> work. I've tested and it works. The basic approach is to compile
> qtmain_android.cpp into its own .so during Qt compilation, and load it in
> the application. It uses dynamic linker calls to find the main() function
> at runtime. So there is no awkward back-and-forth library loading needed
> after all.
> >>>>
> >>>>
> >>>>Backward compatibility: I made sure my changes to the JAR allowed it
> to still work with existing built applications as-is, and light testing
> confirms that it is. This required a few extra lines that are marked as
> such in case you want to remove them.
> >>>>
> >>>>
> >>>>I didn't touch Ministro, however, and I suspect that my addition to
> lib/rules.xml will cause new versions of Ministro to try to load
> libqtmain.so, which won't exist in old/current applications. I'll leave
> that issue for someone more familiar with Ministro to look at, if it's a
> concern.
> >>>>
> >>>>
> >>>>Finally, I mentioned creating a new category of user-defined bundled
> libs in case the user wants to use JNI with those libs, but did not
> implement that.
> >>>>
> >>>>
> >>>>I hope you like it and that it can be included in the next alpha. Let
> me know if you have any questions or comments :)
> >>>>Tyler
> >>>>
> >>>>
> >>>>On Sat, Dec 24, 2011 at 11:28 PM, Tyler Mandry <tmandry at gmail.com>
> wrote:
> >>>>
> >>>>BogDan,
> >>>>>
> >>>>>
> >>>>>Merry Christmas again! When you have time:
> >>>>>
> >>>>>
> >>>>>Separating out qtmain_android.cpp into its own shared lib allows us
> to build it during the Qt build rather than with the application projects.
> (We have to remove the --no-undefined linker option, but it works.) My
> question is, where should that library go in the Qt install? It isn't
> technically a Qt plugin, so I don't know about putting it with plugins, and
> lib/ is all official Qt libs. It fits in best with jar/, but then the
> directory name doesn't make sense :). We could also make another directory,
> though it seems unnecessary. I'll probably stick it in jar/ unless I hear
> from you.
> >>>>>
> >>>>>
> >>>>>I've proved that this method works, and now I'm just working on
> integrating all the changes in code.
> >>>>>
> >>>>>
> >>>>>Cheers,
> >>>>>Tyler
> >>>>>
> >>>>>
> >>>>>
> >>>>>On Fri, Dec 23, 2011 at 2:14 AM, Tyler Mandry <tmandry at gmail.com>
> wrote:
> >>>>>
> >>>>>One thing I've encountered: If I'm not mistaken, people can/will use
> bundled libraries to bundle Qt libraries along with their application (to
> bypass Ministro.) Others will use bundled libs of their own for other
> purposes. We will want to load the Qt bundled libs inside the child class
> loader, so our JAR can talk to them, and the other bundled libs inside the
> main Java class loader, in case they want to interact with those native
> bundled libs via JNI. This will require us to make a new distinction on the
> type of bundled library.
> >>>>>>
> >>>>>>Other thoughts: I've been looking and it looks like this isn't a
> difficult change to make. It's very minimal in the way it changes how
> things are done right now. It does require a change to the JAR files to
> accomodate for that back-and-forth lib loading that I described. Anyway, I
> plan to implement all this tomorrow and see how it goes.
> >>>>>>
> >>>>>>
> >>>>>>On Thu, Dec 22, 2011 at 8:51 PM, Tyler Mandry <tmandry at gmail.com>
> wrote:
> >>>>>>
> >>>>>>Hi all, and Merry Christmas.
> >>>>>>>
> >>>>>>>
> >>>>>>>We were discussing the importance of allowing Java and C++ to
> interact for many people's apps in android-qt (thread: C++ and Java
> interaction in Alpha3.) With the new separation of the JAR files, and use
> of a different class loader for the app code, it has become very difficult
> to allow this. Traditional native interface classes can no longer see the
> native methods that are loaded by the separate class loader.
> >>>>>>>
> >>>>>>>
> >>>>>>>I did some playing around, and I think I found a simple solution
> that works. If we separate out the code in qtmain_android.cpp into its own
> (very small!) shared library, and allow the user's application code to be
> in one library all by itself, we can load them in separate class loaders.
> While you can't do Java-Java or Java-native interaction easily across the
> class loader barrier, native-native interaction works fine since it's all
> the same process. Simply load the Qt library in the /child/ class loader,
> followed by the application code library in the /main/ class loader,
> followed by the qtmain_android code in the /child/ class loader. Then the
> user's Java code can see the user's C++ code and vice versa, because they
> are all loaded under the same class loader.
> >>>>>>>
> >>>>>>>
> >>>>>>>I did a testcase of this (with libraries of my own, not Qt) and
> confirmed that it works in a simple case. If no one sees any problems with
> this let's implement it.
> >>>>>>>--
> >>>>>>>Tyler Mandry
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>--
> >>>>>>Tyler Mandry
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>--
> >>>>Tyler Mandry
> >>>>
> >>>
> >>>
> >>>
> >>>--
> >>>Tyler Mandry
> >>>
> >>>
> >>>
> >>
> >
> >
> >
> >--
> >Tyler Mandry
> >
> >
> >
>
>
>
>
>

-- 
Tyler Mandry

-- 
Tyler Mandry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/necessitas-devel/attachments/20120122/85ee4fe8/attachment-0001.html>