[Kde-bindings] Progress report on Py*5 binding generation
Shaheed Haque
srhaque at theiet.org
Mon Apr 18 22:56:51 UTC 2016
Hi Steve,
On 18 April 2016 at 22:21, Stephen Kelly <steveire at gmail.com> wrote:
> Shaheed Haque wrote:
>
>> The next steps are to pursue the crash, the rest of the fanout, also
>> automate the hand edits and...eventually get some bindings to
>> compile/load/work.
>
> Hi Shaheed,
>
> Thanks for your work on this.
>
> I am vaguely aware of several bindings-related technologies (smoke, swig,
> sip, boost-python), but have never tried to use any of them.
>
> Nevertheless, I'm interested in your work and I read through all of the
> commits in your branch. As far as I can tell, the only other tool similar to
> the sip_generator is smoke, which I think also works by parsing C++ headers.
>
> As far as I can tell, your tool should be usable with any C++ library, not
> only KF5 ones, and not only Qt-based libraries, right?
Yes, Yes and Yes. The key principles are:
1. Use LLVM as that they will likely be able to keep up as C++ evolves.
2. Use easily defined rules to tailor the generated code.
3. (In progress) Allow manual code to be cleanly injected into the
generated code.
4. (Todo) Figure out if something smart/automatic can be done for some
usages of templates (e.g. functions taking QPair<>s).
> I am most interested in how the tools work for the KF5 KItemModels library
> and the non-KF5 Grantlee libraries.
Specifically, the core logic in sip_generator.py has almost no
references to anything Qt or KDE specific (there are a few, though I
expect them to be generally non-intrusive, and they could be factored
out if anybody cared). The obvious exception is that the CLI defaults
are Qt and KDE centric, but they are just defaults. For normal sized
projects with a sane number of files to process, this may be all you
need, along with your custom rules_XXX.py.
Similarly, the next layer on the generator side is
sip_bulk_generator.py, which is again not intended to be closely tied
to Qt/KDE. Again, the intent is that the CLI defaults are there to be
overridden.
(It does consciously exploit/support some patterns that are common in
KDE, such as broken includes, but those could happen elsewhere too).
In both cases, sane patches considered.
The sip_compiler.py side should be fully Qt/KDE agnostic. The code
here should become a lot more flexible as I work out what patterns are
needed to really make SIP automated. I have in mind to use the
%Include(optional=True) capability to provide the ability to inject
fragments of had written code.
> I tried running it on just one header so I can learn some fundamentals of
> what is happening here. I got an error when I tried to generate:
>
> $ ./sip_generator.py /usr/include/KF5/KItemModels/kselectionproxymodel.h >
> sip/kselectionproxymodel.sip
> ERROR: Parse error /usr/include/x86_64-linux-
> gnu/qt5/QtCore/qglobal.h:38[10] 'stddef.h' file not found
Yes; the Clang LLVM compiler front end works like a real compiler :-).
I had code which downgraded the errors more aggressively, but
concluded that was misleading. I settled instead for suppressing
duplicates of errors from Clang to avoid overwhelming the user for the
many files case. Feedback on this welcome.
> But the file was actually generated:
>
> $ cat sip/kselectionproxymodel.sip
> class KSelectionProxyModel: QAbstractProxyModel
> {
> %TypeHeaderCode
> #include </usr/include/KF5/KItemModels/kselectionproxymodel.h>
> %End
> public:
> // Discarded FUNCTION_TEMPLATE on line 87 'qt_check_for_QOBJECT_macro'
> // Discarded VAR_DECL on line 87 'staticMetaObject'
> // Discarded CXX_METHOD on line 87 'metaObject'
> // Discarded CXX_METHOD on line 87 'qt_metacast'
> // Discarded CXX_METHOD on line 87 'tr'
> // Discarded CXX_METHOD on line 87 'trUtf8'
> // Discarded CXX_METHOD on line 87 'qt_metacall'
> public:
> KSelectionProxyModel(QItemSelectionModel *selectionModel, QObject
> *parent /TransferThis/ = 0);
> KSelectionProxyModel();
> virtual ~KSelectionProxyModel();
> virtual void setSourceModel(QAbstractItemModel *sourceModel);
> QItemSelectionModel *selectionModel() const;
> void setSelectionModel(QItemSelectionModel *selectionModel);
> enum FilterBehavior {
> SubTrees,
> SubTreeRoots,
> SubTreesWithoutRoots,
> ExactSelection,
> ChildrenOfExactSelection,
> InvalidBehavior
> };
> void setFilterBehavior(KSelectionProxyModel::FilterBehavior behavior);
> KSelectionProxyModel::FilterBehavior filterBehavior() const;
> virtual QModelIndex mapFromSource(const QModelIndex &sourceIndex) const;
> virtual QModelIndex mapToSource(const QModelIndex &proxyIndex) const;
> virtual QItemSelection mapSelectionFromSource(const QItemSelection
> &selection) const;
> virtual QItemSelection mapSelectionToSource(const QItemSelection
> &selection) const;
> virtual Qt::ItemFlags flags(const QModelIndex &index) const;
> virtual QVariant data(const QModelIndex &index, int role =
> Qt::DisplayRole) const;
> virtual int rowCount(const QModelIndex &parent = QModelIndex()) const;
> virtual QVariant headerData(int section, Qt::Orientation orientation,
> int role = Qt::DisplayRole) const;
> virtual QMimeData *mimeData(const QModelIndexList &indexes) const;
> virtual QStringList mimeTypes() const;
> virtual Qt::DropActions supportedDropActions() const;
> virtual bool dropMimeData(const QMimeData *data, Qt::DropAction action,
> int row, int column, const QModelIndex &parent);
> virtual bool hasChildren(const QModelIndex &parent = QModelIndex())
> const;
> virtual QModelIndex index(int __0, int __1, const QModelIndex &__2 =
> QModelIndex()) const;
> virtual QModelIndex parent(const QModelIndex &__0) const;
> virtual int columnCount(const QModelIndex &__0 = QModelIndex()) const;
> virtual QModelIndexList match(const QModelIndex &start, int role, const
> QVariant &value, int hits = 1, Qt::MatchFlags flags =
> Qt::MatchFlags(Qt::MatchStartsWith|Qt::MatchWrap)) const;
> signals:
> protected:
> // Discarded CXX_METHOD on line 311 'sourceRootIndexes'
> };
Indeed.
> However, the sip_compiler was not able to process it:
>
> $ ./sip_compiler.py --select @kselectionproxymodel.sip sip cxx
> INFO: Creating cxx/ from sip/kselectionproxymodel.sip
> ERROR: sip: cxx/kselectionproxymodel.sip.tmp:1: Class definition not allowed
> in a C module while processing sip/kselectionproxymodel.sip
> Traceback (most recent call last):
> File "./sip_compiler.py", line 212, in main
> d.process_one_module(args.select)
> File "./sip_compiler.py", line 145, in process_one_module
> self._run_command(cmd)
> File "./sip_compiler.py", line 172, in _run_command
> raise RuntimeError(stdout)
> RuntimeError: sip: cxx/kselectionproxymodel.sip.tmp:1: Class definition not
> allowed in a C module
>
>
> Is that expected at this point in development?
Yes. It is precisely this class of error that I am now working
through. In many cases, this will be due to bogosity on my part, and
those I will fix in the core code. In a few cases, especially around
typedefs, I'm pushing Clang well beyond its current capabilities (at
least the Python binding) or SIP beyond its. In these areas, the rules
are there to allow you to tailor the output, or discard it.
However, the specific reason for the failure in your case is that the
individual SIP files generated from .h files are not intended to be
compiled standalone (note the bit at the end where it thinks you are
compiling a C module?). Instead, in each directory, there is a
<whatever>mod.sip file which collects together all the other files
into a single compilable unit. Note how the file <whatever>mod.sip
file is the one that is compiled by default:
$ ./sip_compiler.py --select KItemModels -v sip cxx
2016-04-18 23:28:55,446 __main__ INFO: Creating cxx/KItemModels
2016-04-18 23:28:55,446 __main__ DEBUG: cxx/KItemModels/KItemModelsmod.sip.tmp
...
As it happens, as of tonight, this particular compile is clean, but it
won't load:
$ ./sip_compiler.py --select KItemModels sip cxx
INFO: Creating cxx/KItemModels
$ cd cxx/KItemModels
$ python
...
>>> import KItemModels
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: ./KItemModels.so: undefined symbol: _ZTI20KSelectionProxyModel
I believe this is related to needing to link against the underlying
KF5 library, but I have not automated the hookup needed to test this
theory:
$ nm /usr/lib/debug/.build-id/0f/fca3dbbf65ef0e7bda9a7bfc7feca6dd882b53.debug
| grep ZTI20KSelectionProxyModel
000000000023c810 V _ZTI20KSelectionProxyModel
Thanks, Shaheed
> Steve.
> _______________________________________________
> Kde-bindings mailing list
> Kde-bindings at kde.org
> https://mail.kde.org/mailman/listinfo/kde-bindings
More information about the Kde-bindings
mailing list