F-Droid meta data generation

[Ben Cooksley]

On Mon, Oct 26, 2020 at 4:07 AM Aleix Pol <aleixpol at kde.org> wrote:

> On Sun, Oct 25, 2020 at 2:28 PM Volker Krause <vkrause at kde.org> wrote:
> >
> > Hi,
> >
> > I've been looking into the F-Droid meta data generation a bit, now that
> I am
> > able to run this locally and make changes to it.
> >
> > The current system with generating the meta data from the final APK on
> the
> > signing system is nice in that it is entirely transparent for the app,
> but it
> > does have a few limitations:
> > - it's happening late in the pipeline, which makes some information hard
> to
> > access (such as the source code repository URL)
>
> You might be right about that. The repository URL is something fiddly
> in any case because there's many approaches that don't build software
> from git (e.g. tarballs).
>
> I would like to include the git repository url in the appstream
> metadata for many reasons (de-duplication, development tools
> integration, etc.). This is something I could push for.
> That said, it might not solve the problem entirely.
>
> > - it's running on the signing system, which requires deploying updates
> > manually, and presumably doing things like screenshot downloads on there
> isn't
> > ideal either from a security POV either
>
> Yes, this was something that changed recently because Ben preferred to
> do it this way. I agree it's not ideal and again it's something we can
> probably improve upon by doing it closer to the developer setup.
>

Please note that historically the metadata extraction process was always
part of the F-Droid repository management tooling.
I only continued that approach - my change was to split the signing and
repository management off from the Android builders to a dedicated isolated
system to improve security.

I agree that it makes much more sense to have the build process produce a
dedicated metadata file we can make use of, rather than us inferring it
from a bunch of information contained within the APK.


>
> > - it's not able to make use of manually provided parts for the meta data
> (e.g.
> > the banner image which has no equivalent in appstream).
>
> True.
> We can have custom fields in appstream too, not sure if that is useful
> elsewhere.
>
> > - it's not accessible for F-Droid release builds, ie. builds running on
> their
> > infrastructure
>
> We can split the code outside, wherever makes sense.
>
> > A possible way to address this could be to move meta data generation
> from repo
> > generation to the build process, implemented as part of ECM. The
> resulting
> > meta data would then need to be transferred alongside the APKs to the
> signing
> > system for integration into the F-Droid repo, presumably this can be
> done by
> > the same mechanism as the APKs are transferred.
> >
> > I think this can be done in a similarly transparent way, while now also
> being
> > able to implement merging existing meta data fragments from the source
> dir. By
> > moving to ECM this also becomes independent of our CI tooling, which
> should
> > help F-Droid release builds.
> >
> > Thoughts?
>
> Overall I think it's good to make this process more reliable. I wonder
> what's the right way, the line between powerful and overengineered can
> be thin.
> ECM would be a fine place to put it. Also the AndroidExtras repository
> we keep talking about could be a place. Our toolchain already depends
> on Python so it shouldn't be entirely out of place.
>
> When you say "the same mechanism" you mean passing the metadata side
> by side or including the metadata within the apk?
>
> > @Nico: would that cover what F-Droid builds need?
> > @Aleix: could you add a license/copyright to the generation script (or
> provide
> > me with one) please, preferably with an ECM-compatible license?
>
> Sure, no problem.
> https://invent.kde.org/sysadmin/binary-factory-tooling/-/merge_requests/120


Cheers,
Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-android/attachments/20201026/e3a35283/attachment.htm>