Allowing Java-C++ interaction

Sun Jan 29 03:59:10 UTC 2012

Done! https://groups.google.com/forum/#!topic/android-ndk/Z8GDn8JuopI

On Fri, Jan 27, 2012 at 2:42 AM, BogDan <bog_dan_ro at yahoo.com> wrote:

> Hi Tyler,
>
>
> >
> >Hi BogDan,
> >
> >
> >Good point. I assume you are referring to native libraries being able to
> access other native libraries loaded by different class loaders in Java?
>
>
> Yes !
>
>
> > If so, we should ask about it, but I really don't think it would break.
> It's pretty low-level stuff that allows dynamically-linked code to be
> found, used, and reused within a process. So breaking this would mean
> changing the OS/kernel itself in a non-standards-compliant way (unlikely.)
> Or the JVM somehow splitting our single application/activity into more than
> one process, which is also unlikely and would contradict some of the
> documentation for Activity (thus breaking other apps.) If there are other
> ways it could break, I don't see them.
> >
>
> I also have doubts they can do it, but doesn't hurt to ask :)
>
>
> >
> >That said, it's always a good idea to ask. Let me know if we're thinking
> of the same thing or not. I'm willing to post to android-ndk, but you're
> welcome to do it instead.
> >
>
> Please post the question (and share with us the link :) ).
>
>
> >
> >Have a great day!
> >Tyler
> >
>
> Cheers,
> BogDan.
>
>
> >
> >On Thu, Jan 26, 2012 at 2:48 AM, BogDan <bog_dan_ro at yahoo.com> wrote:
> >
> >Hi Tyler,
> >>
> >>
> >>  The big question is if Google can change android in order to hide the
> symbols if they are loaded by different classloaders ?
> >>
> >>If they can't do it, I think your solution is ok, but if it is possible
> to do it, then we risk to break everyones applications !
> >>
> >>
> >>  Before we make a decision I think we should ask google if they are
> planning to change current behavior or not. Do you want to open a topic to
> android-ndk[1] or should I do ?
> >>
> >>
> >>Cheers,
> >>BogDan.
> >>
> >>
> >>[1] http://groups.google.com/group/android-ndk
> >>
> >>
> >>>________________________________
> >>> From: Tyler Mandry <tmandry at gmail.com>
> >>>To: necessitas-devel at kde.org
> >>>Sent: Monday, January 23, 2012 7:40 AM
> >>>Subject: Allowing Java-C++ interaction
> >>
> >>>
> >>>
> >>>This accidentally didn't get sent to the list; forwarding. Sorry about
> that.
> >>>
> >>>
> >>>---------- Forwarded message ----------
> >>>From: Tyler Mandry <tmandry at gmail.com>
> >>>Date: Tue, Jan 17, 2012 at 12:19 AM
> >>>Subject: Re: Allowing Java-C++ interaction
> >>>To: BogDan <bog_dan_ro at yahoo.com>
> >>>
> >>>
> >>>Hi BogDan,
> >>>
> >>>
> >>>That definitely could work. But it does add to the complexity, and
> possibly start-up time, of the app. An alternative solution would be:
> simply to load the bundled libs in the main classloader, UNLESS they are
> bundled Qt libs that are used by the .jar, in which case they are loaded in
> the child CL. This could be accomplished by separating these two kinds of
> libs into different categories, or even auto-detecting them by their names
> at runtime. What do you think?
> >>>
> >>>
> >>>(One thing I've found: native libs loaded in different classloaders can
> always talk to each other, since they're part of the same process. In other
> words, you could load core Qt libs in the child CL and QtWebKit in the
> parent CL and it will all work. I don't know if this is useful but thought
> it might be.)
> >>>
> >>>
> >>>Thanks,
> >>>Tyler
> >>>
> >>>
> >>>
> >>>On Mon, Jan 16, 2012 at 2:22 AM, BogDan <bog_dan_ro at yahoo.com> wrote:
> >>>
> >>>Hi Tyler,
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>   Thanks you for sharing this with us, it helped me to understand
> better what is the problem.
> >>>>I also check your patch [1] but it doesn't seem to fix all the
> problems, bundle libs will have the same problem, so I tried to find
> another solution which hopefully will handle all this situations. The
> solution is create a separate package with your own java files, and this
> package to be loaded by the same class loaded which loads the plugin .jar
> files.
> >>>>
> >>>>
> >>>>
> >>>>Please let me know if this approach is ok for you.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>Cheers,
> >>>>BogDan.
> >>>>
> >>>>
> >>>>
> >>>>[1] http://sourceforge.net/p/necessitas/tickets/140/
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>________________________________
> >>>>> From: Tyler Mandry <tmandry at gmail.com>
> >>>>>To: BogDan <bog_dan_ro at yahoo.com>
> >>>>>Sent: Friday, January 13, 2012 6:43 AM
> >>>>>Subject: Re: Allowing Java-C++ interaction
> >>>>>
> >>>>>
> >>>>>
> >>>>>Hi BogDan,
> >>>>>
> >>>>>
> >>>>>What currently happens is the native library is loaded in the child
> class loader (by the code in the .jar, in fact,) so application code
> (loaded by the parent class loader) can't see any native functions loaded
> by the native library. And no, I don't try to access anything in the .jar
> in my application.
> >>>>>
> >>>>>
> >>>>>I did make a simple testcase that fails before the patch and works
> after. Since the patch involves changing QtActivity.java, it's difficult to
> send a complete project and see the before and after. Instead, do the
> following:
> >>>>>    1. Create a new Qt Quick Application project in Qt Creator, with
> an Android target
> >>>>>    2. (i) Replace main.cpp with the attached main.cpp, and (ii) add
> in the Java source to the tree (copy the
> android/src/org/kde/necessitas/example folder in attached archive to
> android/src/org/kde/necessitas in the project)
> >>>>>    3. Go to QtActivity.java and add (i) "import
> org.kde.necessitas.example.simpleapp.MyNative;" at the top, and (ii) the
> following at the END of loadApplication():
> >>
> >>>>>//dostuffafter loading
> >>>>>MyNativenat=newMyNative();
> >>>>>nat.doSomething();
> >>>>>
> >>>>>
> >>>>>[Modifying QtActivity.java is currently the only way I know to run
> your own Java code as the application loads.]
> >>>>>
> >>>>>
> >>>>>After that, run the project and you should see "No implementation
> found for native
> Lorg/kde/necessitas/example/simpleapp/MyNative;.doSomething ()V" along with
> an UnsatisfiedLinkError in the log.
> >>>>>
> >>>>>
> >>>>>Then point Qt Creator to a build of Qt with the patch and do the
> process again with that version of Qt. (**Of course you'll need to deploy
> and use local Qt libs!**) You should see "This is a native function, doing
> something!!" in the logs, and no crash.
> >>>>>
> >>>>>
> >>>>>I hope this is clear enough and that it sheds some light on what I'm
> trying to accomplish :). Let me know if anything is unclear.
> >>>>>
> >>>>>
> >>>>>Thanks for all your hard work!!
> >>>>>Tyler
> >>>>>
> >>>>>
> >>>>>
> >>>>>On Thu, Jan 12, 2012 at 8:46 AM, BogDan <bog_dan_ro at yahoo.com> wrote:
> >>>>>
> >>>>>Hi Tyler,
> >>>>>>
> >>>>>>    I reviewed my code and seems to be ok, when I create
> m_classLoader (check QtActivity.java) I pass its parent correctly, so, even
> if your library is loaded by a class which was loaded by m_classLoader and
> there your package is not visible, m_classLoader should search also into
> its parent for that class.
> >>>>>>
> >>>>>>    I hope your package don't try to access classes from .jar file,
> because those classes are private, and they should be used *ONLY* by
> platform plugin.
> >>>>>>
> >>>>>>
> >>>>>>Would you please create a simple example which fails, it will help
> me to understand the problem better ?
> >>>>>>
> >>>>>>
> >>>>>>Thank you !
> >>>>>>
> >>>>>>
> >>>>>>Cheers,
> >>>>>>BogDan.
> >>>>>>
> >>>>>>
> >>>>>>+ CC: necessitas-devel
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>Hi BogDan,
> >>>>>>>
> >>>>>>>
> >>>>>>>No worries, I just didn't want it to be overlooked :)
> >>>>>>>
> >>>>>>>
> >>>>>>>In a nutshell, any (user) Java code the application had couldn't
> see the user's native JNI code, because their native library was loaded in
> a class loader that was different from the (primary) class loader that
> their Java code ran in. This upset a few people that have custom Java and
> JNI code, like me :). The only way to make calls between different class
> loaders is to use reflection, so instead I made a way for the application
> native library to be loaded in the primary class loader.
> >>>>>>>
> >>>>>>>
> >>>>>>>That was a mouthful, but I hope it helps. Let me know if you have
> further questions!
> >>>>>>>
> >>>>>>>
> >>>>>>>Thanks,
> >>>>>>>Tyler
> >>>>>>>
> >>>>>>>
> >>>>>>>On Tue, Jan 10, 2012 at 1:49 PM, BogDan <bog_dan_ro at yahoo.com>
> wrote:
> >>>>>>>
> >>>>>>>Hi Tyler,
> >>>>>>>>
> >>>>>>>>Please accept my apologize for slow reply.
> >>>>>>>>
> >>>>>>>>I need a few days to understand what is wrong with current design
> and to check your patch, then I'll come back with feedback !
> >>>>>>>>
> >>>>>>>>Thank you very much for your patience !
> >>>>>>>>
> >>>>>>>>Cheers,
> >>>>>>>>BogDan.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>BogDan,
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>Do you think this patch can be included in the next alpha? If
> not, what needs to be done?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>Thanks,
> >>>>>>>>>Tyler
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>On Thu, Dec 29, 2011 at 12:48 AM, Tyler Mandry <tmandry at gmail.com>
> wrote:
> >>>>>>>>>
> >>>>>>>>>Here is the patch (against alpha4 head 2469081) to make
> everything work. I've tested and it works. The basic approach is to compile
> qtmain_android.cpp into its own .so during Qt compilation, and load it in
> the application. It uses dynamic linker calls to find the main() function
> at runtime. So there is no awkward back-and-forth library loading needed
> after all.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>Backward compatibility: I made sure my changes to the JAR
> allowed it to still work with existing built applications as-is, and light
> testing confirms that it is. This required a few extra lines that are
> marked as such in case you want to remove them.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>I didn't touch Ministro, however, and I suspect that my addition
> to lib/rules.xml will cause new versions of Ministro to try to load
> libqtmain.so, which won't exist in old/current applications. I'll leave
> that issue for someone more familiar with Ministro to look at, if it's a
> concern.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>Finally, I mentioned creating a new category of user-defined
> bundled libs in case the user wants to use JNI with those libs, but did not
> implement that.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>I hope you like it and that it can be included in the next
> alpha. Let me know if you have any questions or comments :)
> >>>>>>>>>>Tyler
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>On Sat, Dec 24, 2011 at 11:28 PM, Tyler Mandry <
> tmandry at gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>BogDan,
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>Merry Christmas again! When you have time:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>Separating out qtmain_android.cpp into its own shared lib
> allows us to build it during the Qt build rather than with the application
> projects. (We have to remove the --no-undefined linker option, but it
> works.) My question is, where should that library go in the Qt install? It
> isn't technically a Qt plugin, so I don't know about putting it with
> plugins, and lib/ is all official Qt libs. It fits in best with jar/, but
> then the directory name doesn't make sense :). We could also make another
> directory, though it seems unnecessary. I'll probably stick it in jar/
> unless I hear from you.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>I've proved that this method works, and now I'm just working on
> integrating all the changes in code.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>Cheers,
> >>>>>>>>>>>Tyler
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>On Fri, Dec 23, 2011 at 2:14 AM, Tyler Mandry <
> tmandry at gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>One thing I've encountered: If I'm not mistaken, people
> can/will use bundled libraries to bundle Qt libraries along with their
> application (to bypass Ministro.) Others will use bundled libs of their own
> for other purposes. We will want to load the Qt bundled libs inside the
> child class loader, so our JAR can talk to them, and the other bundled libs
> inside the main Java class loader, in case they want to interact with those
> native bundled libs via JNI. This will require us to make a new distinction
> on the type of bundled library.
> >>>>>>>>>>>>
> >>>>>>>>>>>>Other thoughts: I've been looking and it looks like this isn't
> a difficult change to make. It's very minimal in the way it changes how
> things are done right now. It does require a change to the JAR files to
> accomodate for that back-and-forth lib loading that I described. Anyway, I
> plan to implement all this tomorrow and see how it goes.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>On Thu, Dec 22, 2011 at 8:51 PM, Tyler Mandry <
> tmandry at gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>Hi all, and Merry Christmas.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>We were discussing the importance of allowing Java and C++ to
> interact for many people's apps in android-qt (thread: C++ and Java
> interaction in Alpha3.) With the new separation of the JAR files, and use
> of a different class loader for the app code, it has become very difficult
> to allow this. Traditional native interface classes can no longer see the
> native methods that are loaded by the separate class loader.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>I did some playing around, and I think I found a simple
> solution that works. If we separate out the code in qtmain_android.cpp into
> its own (very small!) shared library, and allow the user's application code
> to be in one library all by itself, we can load them in separate class
> loaders. While you can't do Java-Java or Java-native interaction easily
> across the class loader barrier, native-native interaction works fine since
> it's all the same process. Simply load the Qt library in the /child/ class
> loader, followed by the application code library in the /main/ class
> loader, followed by the qtmain_android code in the /child/ class loader.
> Then the user's Java code can see the user's C++ code and vice versa,
> because they are all loaded under the same class loader.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>I did a testcase of this (with libraries of my own, not Qt)
> and confirmed that it works in a simple case. If no one sees any problems
> with this let's implement it.
> >>>>>>>>>>>>>--
> >>>>>>>>>>>>>Tyler Mandry
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>--
> >>>>>>>>>>>>Tyler Mandry
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>--
> >>>>>>>>>>Tyler Mandry
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>--
> >>>>>>>>>Tyler Mandry
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>--
> >>>>>>>Tyler Mandry
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>>
> >>>--
> >>>Tyler Mandry
> >>>
> >>>
> >>>
> >>>
> >>>--
> >>>Tyler Mandry
> >>>
> >>>_______________________________________________
> >>>Necessitas-devel mailing list
> >>>Necessitas-devel at kde.org
> >>>https://mail.kde.org/mailman/listinfo/necessitas-devel
> >>>
> >>>
> >>>
> >>
> >
> >
> >
> >--
> >Tyler Mandry
> >
> >
> >
>

-- 
Tyler Mandry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/necessitas-devel/attachments/20120128/377587b2/attachment-0001.html>