End of 2016 update on PyKF5 bindings

[Shaheed Haque]

Hi,

On 18 January 2017 at 21:44, Stephen Kelly <steveire at gmail.com> wrote:
> Shaheed Haque wrote:
>
>> HI Steve,
>>
>> I closed two of the reviews based on testing KDE/master. That leaves
>> only https://git.reviewboard.kde.org/r/129763/: this should be
>> non-controversial as it removes a functional no-op which just gets ion
>> the way of my stuff, so please take a look at it when you can.
>
> This doesn't look like a no-op to me. I'm also generally not in favor of
> adding things to ECM which do not directly help the goal of getting bindings
> into frameworks repos.

If you look at how the CMake constructs ${hdr_filename} and
${hdr_file}, you'll see that they are redundant. Further, the changes
you made to sip_generator.py to use these redundant values is also
redundant; the proposed change can get rid of both set of redundant
code, and at the same time thus restore the APi to create_sip in a way
which works for me.

I understand you don't want the bulk stuff. But this change does not
introduce that: it simply restores the API to one I can easily drive,
while changing the internal variable names inside sip_generator.py
back into the original names making the merging easier for me. Please
reconsider the '763 change.

> I know you are going to continue with running your tool on your /usr/include
> anyway, but that is not going to get bindings packaged and into the hands of
> users.
>
> One of the lessons of cleaning up the buildsystem in transitioning from KDE4
> to KF5/ECM is that having things in the repo which are not used, or which no
> one remembers why they are there is a bad thing. I don't think your bulk
> generator will ever need to be in ECM (it is not needed to build bindings in
> repos), and so this change shouldn't be either. Am I missing something?

I know we disagree, but I want PyKF5 to succeed and your work on ECM
is critical to that, and so I'm not going to argue with you any more
about this. (At some point, I *may* come beg for a subdirectory...not
sure about that).

> I would rather make use of your work by considering headers you have already
> processed with the bulk approach (and perhaps you have already created rules
> for), determining which repo they belong to, and getting the bindings into
> the repo (and getting whatever additional sip_generator.py features are
> needed into ECM and unit tested there). That will result in them getting
> packaged and into users hands.
>
>> In the meantime, I will rebase onto ECM KDE/master (I see you have
>> been busy!) and hopefully we can get stuff merged once I have
>> something for you to look at.
>
> Cool. I'm mostly interested in features that will either improve bindings in
> repos which already have bindings at all (eg kconfig), or features which
> will allow bindings to be added to repos which do not already have them.

I plan to do the rebasing in two phases, mostly to cleanly separate
the bulk stuff from your view. I'm making decent progress on the first
phase. So far a couple things have come up, and I will discuss them
with you in a separate thread so we can explore the way forward as
they come up.

> If you are processing other headers in your /usr/include, you might have
> some idea of what frameworks should be next on the list to get bindings. I
> have found
>
>  http://agateau.com/2013/kf5-diagrams/kf5.png
>
> useful, but I don't know if there is a more-updated version.

Remember that that the focus of my work is to make sure we have the
rule support to handle any issues. I typically stop working on a
framework once I have convinced myself that any remaining errors are
due to the semantics of the API, and I don't try to get something
actually working. So, my commits you may have noticed saying
such-and-such a framework is "fixed" is only "fixed" for this very
limited definition...and the rules I develop for the various
frameworks might be a handy reference/backstop but might not help much
beyond that.

Put another way, right now, (subject to the rebasing etc), I can
usefully run sip_generator.py on these frameworks which seem to be a
pretty decent overlap/superset for Tiers 1, 2 and 3 in the diagram:

Akonadi
AkonadiAgentBase
AkonadiCore
AkonadiSearch
AkonadiWidgets
AkonadiXml
Attica
BalooWidgets
BluezQt
CalendarSupport
ComposerEditorNG
EventViews
FollowupReminder
gpgme++
GrantleeTheme
Gravatar
IncidenceEditor
KActivities
KActivitiesStats
KaddressbookGrantlee
KAlarmCal
KArchive
KAuth
KBlog
KBookmarks
KCalCore
KCalUtils
KCMUtils
KCodecs
KCompletion
KConfigCore
KConfigGui
KConfigWidgets
KContacts
KCoreAddons
KCrash
KDBusAddons
KDCRAW
KDeclarative
KdepimDBusInterfaces
KDESu
KDEWebKit
KDGantt2
KDNSSD
KEmoticons
KExiv2
KF5KDEGames
KF5KMahjongg
KFace
KFileMetaData
KGAPI
KGeoMap
KGlobalAccel
KGuiAddons
KHolidays
KHtml
KI18n
KIconThemes
KIdentityManagement
KIdleTime
KIMAP
kimaptest
kio
KIOCore
KIOFileWidgets
KIOGui
KIOWidgets
KIPI
KItemModels
KItemViews
KJobWidgets
kjs
KJsEmbed
KLDAP
KManageSieve
KMbox
KMediaPlayer
KMime
KNewStuff3
KNotifications
KNotifyConfig
KontactInterface
KPackage
KParts
KPeople
KPIMTextEdit
KPlotting
KPty
KrossCore
KrossUi
KRunner
KSane
KScreen
KService
KSieveUi
KStyle
KTextEditor
KTextWidgets
KTNEF
KUnitConversion
KWallet
KWidgetsAddons
KWindowSystem
KXmlGui
KXmlRpcClient
Libkdepim
Libkleo
MailCommon
MailImporter
MailTransport
MessageComposer
MessageCore
MessageList
MessageViewer
NetworkManagerQt
PimCommon
Plasma
PlasmaQuick
PRISON
purpose
purposewidgets
qgpgme
SendLater
Solid
SonnetCore
SonnetUi
Syndication
TemplateParser
ThreadWeaver
wtf
XsltKde

Some of these comprise multiple modules, for a grand total of 156
xxxmod.sip files. I can then run the sip_compiler.py on all 156, with
only about 10 remaining syntax errors flagged by the actual SIP
compiler. In each case, these are due to the kind of semantics I
mentioned above, not bugs in the code of sip_generator.py. Of course,
many of the binding fail to pass through the SIP compiler itself, but
my expectation (and analysis of a sample of the issues) suggests these
are the things that each framework needs to handle at the semantic
level. For example, most of the Akonadi stuff fails with complaints
about the lack of a definition for std::exception. Right now, from the
156, I have 25 modules which actually result in a .so file, and of
those, I routinely test that 13 can actually be "import"ed.

Of course it is possible that some automation may be possible for some
of the remaining issues, but the whole point is that I am pretty sure
the rules engine is now proven to be sufficiently flexible to deal
with pretty much anything.

Thanks, Shaheed

> Thanks,
>
> Steve.