[Kde-bindings] Re: Documenting PyKDE4's bindings generation
simon at simonzone.com
Tue Apr 26 19:43:39 UTC 2011
On 04/18/2011 09:02 AM, Luca Beltrame wrote:
> after fixing a rather un-obvious bug in PyKDE4 on Saturday, I thought
> would be good to kick-start an effort to document the binding
> PyKDE4. I had this idea in mind for a while, in fact.
> The pros are:
> - Higher bus factor ;)
> - Being able to fix build issues faster
> - Being able to adjust some obscure issues faster
> - Possibility for packagers to apply fixes
> - Possibility of attracting new contributors
> - Possibility of adding bindings for non-kdelibs projects (on the
> In order to do so, I have a few questions for you. If you are able to
> them I'll try to start a specific page on KDE's Techbase to present
> to people interested:
> - Recent bindings use twine2. Is this tool present somewhere? If not,
> it be made public?
> - How much work does usually one go through to transform automated
> from twine2 to perfectly working bindings?
> - Is there a specific philosophy to follow when designing bindings?
> - Any other general remarks?
> If you want, I can also bring this forward to the kde-bindings ML, in
> make the whole process public.
> Thanks for your time.
Thanks for writing Luca. I agree with the points you've made and welcome
spreading the PyKDE bindings knowledge around and hopefully getting more
people involved. I think I've done a reasonably good job of handling the
base load of making sure that PyKDE is updated for each new major KDE
release which comes out. But above that level of support it'll take
extra help. Making it easier for and packagers and other people to
contribute is certainly a step in the right direction.
I'll try to answer some of your questions and explain how PyKDE
I only update PyKDE towards the end of KDE's development cycle, usually
after the soft API and/or hard API freeze stages. This means that the
version of PyKDE in git master doesn't actively track API changes in
kdelibs etc, and also if it breaks for some reason it doesn't get
immediately fixed. This is not necessarily a bad thing. It does save a
lot of time and few people use PyKDE from master.
I also bump the PyQt and SIP dependencies to match the latest stable
versions of each. Most distros like to keep thier version of PyQt and
SIP updated too and their isn't the manpower to test new PyKDEs on too
many different combinations of Qt, PyQt and SIP versions.
Approaching release I keep a good eye on any changes in git/svn and
constantly keep checking that nothing gets broken. (If an API change is
needed at this late stage, it is policy for devs to CC the kde-bindings
twine2 is indeed present somewhere. It can be found here:
It is also the first hit off google. ;-) It is based off a tool Jim
Bublitz wrote for maintaining PyKDE, called twine. I've done a heap of
rewriting and refactoring of the parsing and internal representation of
the C++ headers and sip files etc to get it where it is now. I also
ripped out support for handling multiple KDE versions in one sip file.
This wasn't needed any more since PyKDE is shipped with KDE, and we just
have different source versions in svn/git.
The simple version of what twine2 does is read in and parse a C++ .h
files, read in and parse the .sip files, translates the C++ headers to
sip, and then merges the new sip code into the existing sip files.
The entry point for updating PyKDE is the kdelibs.py file. It is a
script which calls the twine2 code with a ton of parameters to update
the PyKDE bindings and spit out the updated code. It also calls routines
to generate the kdelibs documentation for PyKDE including indexes etc.
There are quite a few details and configuration settings/parameters
needed to make it all work. Here are some of the main ones which you can
see in kdelibs.py:
* name of the Python module being generated. (module=)
* where to write the sip files to. (outputDirectory=)
* where to write the docs to. (docsOutputDirectory=)
* where to find the first page of the docs. (mainDocs=)
* where the CMakeLists.txt file(s) are so that the list of header files
to parse can be determined. (cmakelists=... Yes, there is a simple cmake
parser in here too.)
* list of headers files to ignore. (ignoreHeaders=)
* list of sip files to ignore or not update. (noUpdateSip=)
* list of C++ macros to substitute. (preprocessSubstitutionMacros=)
* list of C++ macros to parse. (macros=)
* list of C++ macros that don't have args which should be parsed.
* where to find PyQt's sip files. (sipImportDirs=)
* list of PyQt modules needed by this module. (sipImports=)
* copyright notice text. (copyrightNotice=)
* list of macro names which indicate that an API is public and exported.
i.e. should be wrapped (exportMacros=)
* list of rules which annotate the bindings with extra directives for
When updating PyKDE, I usually update, fix and compile each module (e.g.
plasma, kdeui, kdecore etc) one at a time. There are a number of cases
which twine2 can't handle and they have to be fixed by hand. Enums and
default args which are enums tend to have the wrong 'namespace' /
prefix. Sometimes overloaded methods which are distinct in C++ conflict
in Python. Sometimes method implementations are just missing etc etc.
Lines which start with //ig indicate methods which should be ignored by
twine2. (If twine2 sees the method in the C++ header, it is not added
and enabled in the sip file.)
It is pretty common that the C++ parser breaks on some obscure C++
syntax at least once for each KDE major release. These days an update
for KDE takes me maybe a week of evenings assuming nothing really bad
re: Bindings design philosophy
PyKDE follows the C++ API in a fairly straight forward manner and
follows PyQt lead. This is more pragmatism than design. It is easiest to
follow PyQt since PyKDE uses SIP and should look like an extension of
PyQt with regards to style etc. Copying the C++ API directly makes it
easier to translate C++ code, examples and solutions to Python. PyKDE
and the community around it isn't big enough to design and maintain a
Python specific API along with documentation etc etc. That said, if
someone wants to make an API designed for Python then the most sensible
implementation strategy is to use Python code to build on top of the
'plain' C++ API.
That should help demystify PyKDE somewhat. There are a lot of other
little details but in total it is not rocket science. If there are other
questions then just ask.
Simon Edwards | KDE-NL, Guidance tools, Guarddog Firewall
simon at simonzone.com | http://www.simonzone.com/software/
Nijmegen, The Netherlands | "ZooTV? You made the right choice."
More information about the Kde-bindings