[Kde-bindings] Progress report on Py*5 binding generation

Thu Apr 21 16:54:50 UTC 2016

Hi Steve,

On 20 April 2016 at 22:27, Stephen Kelly <steveire at gmail.com> wrote:
> Shaheed Haque wrote:
>
>> Specifically, the core logic in sip_generator.py has almost no
>> references to anything Qt or KDE specific (there are a few, though I
>> expect them to be generally non-intrusive, and they could be factored
>> out if anybody cared). The obvious exception is that the CLI defaults
>> are Qt and KDE centric, but they are just defaults. For normal sized
>> projects with a sane number of files to process, this may be all you
>> need, along with your custom rules_XXX.py.
>
> Yes, the defaults got in my way already :).
>
>>> I tried running it on just one header so I can learn some fundamentals
of
>>> what is happening here. I got an error when I tried to generate:
>>>
>>> $ ./sip_generator.py /usr/include/KF5/KItemModels/kselectionproxymodel.h
>>> > sip/kselectionproxymodel.sip
>>> ERROR: Parse error /usr/include/x86_64-linux-
>>> gnu/qt5/QtCore/qglobal.h:38[10] 'stddef.h' file not found
>>
>> Yes; the Clang LLVM compiler front end works like a real compiler :-).
>
> I could get past the above error by specifying on the command line include
> directories which are usually built-in when using the clang driver.
>
> However, after that I get more errors:
>
>  /usr/include/x86_64-linux-gnu/c++/4.9.3/bits/os_defines.h:44:5: error:
> function-like macro '__GLIBC_PREREQ' is not defined
>
> Is there no way to tell libclang to use the built-in includes and defines?

You are treading a well-work path :-) in that I did exactly the same thing
before getting to the point where I gave up on the basis that the failure
to find stddef.h *appears* not to cause any other issues. I tried several
things, including giving it a local dummy stddef.h (which later on grew to
include a few macros and suchlike), along with a few other sibling empty
system header files before I decided I was on a hiding to nothing. So far,
apart from that scary looking error, it seems to all work.

> Anyway, we should be able to pass the correct includes to the tool with
some
> cmake macro.

In due course, avoiding the error message would be nice indeed.

>> However, the specific reason for the failure in your case is that the
>> individual SIP files generated from .h files are not intended to be
>> compiled standalone (note the bit at the end where it thinks you are
>> compiling a C module?). Instead, in each directory, there is a
>> <whatever>mod.sip file which collects together all the other files
>> into a single compilable unit. Note how the file <whatever>mod.sip
>> file is the one that is compiled by default:
>
> Note that I am trying to avoid the built-in KF5 stuff in the tools, so
that
>
> 1) I can understand the tool and help with the cmake stuff
> 2) I can know how to use this with non-KF5 libraries

Makes sense.

> To that end, in addition to sip/kselectionproxymodel.sip I now have:
>
>  $ cat sip/kitemmodelsmod.sip
>
>  %Module PyKDE5.kitemmodels
>
>  %ModuleHeaderCode
>  #pragma GCC visibility push(default)
>  %End
>
>  %Import QtCore/QtCoremod.sip
>
>  %Include kselectionproxymodel.sip
>
>
> Is this what I am supposed to do? Here is the output I get when attempting
> to use it:

Yeeessss...it is a bit tricky...

First, if you use the "-v" flag to the sip_compiler.py, it will show you
the exact command like for SIP and later the C++ compiler/linker. That
might help you see what is going on.

Next, if you look at the XXXmod.sip (and the XXXmod.sip.tmp) that I
generate, the latter includes two things ine "individual" .sip files do not:

   - A set of %Feature declarations.
   - The funny "includes" extract thing you get an error on below.

The %Feature thing you may be able to ignore. It is used along with the SIP
compiler's -x (lower case X) option to try to work around the problem where
in KF5, you often get AAAmod.sip and BBBmod.sip which need to import each
other: SIP does not like this, and I use the features (and a
"modules.features" output file which you might have spotted) to deal with
issue.

For your problem, the important thing to get is that my code auto-generates
the %Imports based on analysing the first-level #includes of the original
header files (as reported to me by Clang) and crucially this analysis is
also used to track the #includes which the C++ compile steps needs for
those same dependencies. Now, in order to pass this information from the
sip_bulk_generator.py stage to the sip_compiler stage via processing
through the SIP compiler, I include the needed information in the
XXXmod.sip file in an %Extract section named "includes". Then, in order to
unpack this information for use by the C++ compiler...

> $ ./sip_compiler.py --select @kitemmodelsmod.sip sip cxx
> INFO: Creating cxx/
> ERROR: sip: There is no extract defined with the identifier "includes"
while
> processing sip/kitemmodelsmod.sip
> Traceback (most recent call last):
>   File "./sip_compiler.py", line 211, in main
>     d.process_one_module(args.select)
>   File "./sip_compiler.py", line 145, in process_one_module
>     self._run_command(cmd)
>   File "./sip_compiler.py", line 171, in _run_command
>     raise RuntimeError(stdout)
> RuntimeError: sip: There is no extract defined with the identifier
> "includes"
>
> The error seems to come from sip. Do you know what it means?

...I used the SIP compiler's -X (upper case X) option to extract the
%Extract data, and generate the "-I foo -I bar" paths for the C++ compiler.
So, you can just put a %Extract section like this into your file:

%Extract(id=includes)
/usr/include/KF5/KDBusAddons
/usr/include/x86_64-linux-gnu/qt5/QtCore
/usr/include/x86_64-linux-gnu/qt5/QtDBus
%End

Possibly easier would be to just let sip_bulk_generator.py do its thing. If
you want to only process part of KF5, you could just use the --selector
option to say something like:

./sip_bulk_generator --select KItemModels <sip-output-dir>
./sip_compiler --select KItemModels <sip-input-dir-from-above>
<cxx-output-dir>

Equally, you are welcome to just run stuff by hand (as noted, use the -v
option to see what the compiler stage is doing).

>> I believe this is related to needing to link against the underlying
>> KF5 library, but I have not automated the hookup needed to test this
>> theory:
>
> I can help with the cmake/linking side, but I'm still struggling to get
the
> tool to work for a trivial case above. Can you help with that?

Hope the above helps, if needed we can work together interactively
timezones etc. permitting. Send me a PM if you think that is needed (I'm in
the UK).

Thanks, Shaheed

> Thanks,
>
> Steve.
>
> _______________________________________________
> Kde-bindings mailing list
> Kde-bindings at kde.org
> https://mail.kde.org/mailman/listinfo/kde-bindings
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-bindings/attachments/20160421/cd28f41d/attachment.html>