<div dir="ltr"><div dir="ltr">On Tue, Jul 8, 2025 at 7:04 AM Soumyadeep Ghosh <<a href="mailto:soumyadeepghosh2004@zohomail.in">soumyadeepghosh2004@zohomail.in</a>> wrote:</div><div class="gmail_quote gmail_quote_container"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div style="padding-bottom:1px">
<p>Hello again,</p></div></blockquote><div><br></div><div>Hi Soumyadeep, </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="padding-bottom:1px">
<p>So, the last time I checked the notary service, what it does is
basically sends the artifact to a remote server, where the
artifact is signed with a valid key that's recognisable as KDE's
to validate the artifact as KDE's artifact in various stores. Am I
explaining it correctly?<br></p></div></blockquote><div><br></div><div>That is only part of what it does. </div><div><br></div><div>The Notary Service exists to provide a secure and protected way to do privileged operations on the CI system. The Notary Service exists not only to handle using signing keys (some of which will in the future be in a HSM due to mandated requirements for Windows signing keys), but also anything else which involves sensitive materials on CI - like API keys. This is how the Google Play and Microsoft Store publishers work. </div><div><br></div><div>It allows us to enforce validation checks (to ensure that an application only publishes under names it is allowed to for instance) as part of it's operation which includes ensuring certain authentication and authorization checks are met as a default position.</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="padding-bottom:1px"><p>
<br>
If I am right, then, I don't think that's needed for snaps. To
upload a snap to the store, one basically needs the auth token of
the KDE account in the snap store and pass it when uploading the
snap. IIRC, till now, some of the vms were having this auth token
embedded. Can't we replicate something like this?</p></div></blockquote><div><br></div><div>That is an inherently insecure approach and is why publishing only worked for certain repositories within <a href="http://invent.kde.org/neon/">invent.kde.org/neon/</a>.</div><div><br></div><div>It may be the path of least resistance, but it offers no validation or other checks and also exposes the sensitive credential (the API key) to the CI job - allowing for it to be used to do things other than publishing applications.</div><div>A Notary Service prevents this from happening and helps secure the build pipeline process and is why a Notary is required to implement publishing in production.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="padding-bottom:1px">
<p>Thanks,</p>
<p>Soumyadeep<br></p></div></blockquote><div><br></div><div>Regards,</div><div>Ben </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="padding-bottom:1px"><p>
</p>
<div>On 08/07/25 00:20, Ben Cooksley wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">On Mon, Jul 7, 2025 at 11:55 PM Soumyadeep Ghosh
<<a href="mailto:soumyadeepghosh2004@zohomail.in" target="_blank">soumyadeepghosh2004@zohomail.in</a>>
wrote:</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div dir="auto" style="font-size:10pt;font-family:Verdana,Arial,Helvetica,sans-serif">
<div id="m_9189247605577897767m_-4962481818660731400message" dir="auto">
<div><font size="2">Hi Ben,</font></div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Hi Soumyadeep,</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div dir="auto" style="font-size:10pt;font-family:Verdana,Arial,Helvetica,sans-serif">
<div id="m_9189247605577897767m_-4962481818660731400message" dir="auto">
<div><font size="2"><br>
</font></div>
<div><font size="2">This is an awesome news! I just
wanted to know what would be the problem in
publishing those snaps built inside the new VM
based CI.</font></div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>There is currently no Notary built that supports handling
the publishing process at this time, which is why we're
unable to publish.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div dir="auto" style="font-size:10pt;font-family:Verdana,Arial,Helvetica,sans-serif">
<div id="m_9189247605577897767m_-4962481818660731400message" dir="auto">
<div><font size="2"><br>
</font></div>
<div>Thanks,</div>
<div dir="auto">Soumyadeep Ghosh</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Cheers,</div>
<div>Ben</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div dir="auto" style="font-size:10pt;font-family:Verdana,Arial,Helvetica,sans-serif">
<div id="m_9189247605577897767m_-4962481818660731400message" dir="auto">
<div><br id="m_9189247605577897767m_-4962481818660731400br3">
<br id="m_9189247605577897767m_-4962481818660731400br3">
</div>
</div>
<div id="m_9189247605577897767m_-4962481818660731400content" dir="auto"><br>
<br>
---- On Mon, 07 Jul 2025 17:19:14 +0530 <b> <a href="mailto:bcooksley@kde.org" target="_blank">bcooksley@kde.org</a>
</b> wrote ----<br>
<br>
<blockquote style="border:0px;padding:0px;margin:0px">
<div>
<div dir="ltr">Hi all,
<div><br>
</div>
<div>I'm happy to announce that VM based CI is
now in a state where it is ready to begin
making the transition into general
availability. As part of this i'm happy to
advise that builds for Snaps (but not their
publishing) are also becoming generally
available, and FreeBSD will be updating to Qt
6.9 as well.</div>
<div><br>
</div>
<div>This transition will also mark the general
retirement of Docker based CI, with Docker
functionality only being retained for website
builds and linter jobs (such as xml-lint,
cppcheck, etc). </div>
<div><br>
</div>
<div>As part of this support for Qt 5 (except
for general Linux) is also being retired. This
includes all FreeBSD and Windows support, as
well as all binary support (Appimages, Windows
exes, etc)</div>
<div><br>
</div>
<div><u>Steps you need to take:</u></div>
<div><br>
</div>
<div>If you only inherit the templates from
sysadmin/ci-utilities, and do not have custom
jobs, then no action is needed on your part.</div>
<div><br>
</div>
<div>If you have custom jobs which are based on
the templates currently in
sysadmin/ci-utilities then you should examine
the diff in <a href="https://invent.kde.org/sysadmin/ci-utilities/-/merge_requests/507" target="_blank">https://invent.kde.org/sysadmin/ci-utilities/-/merge_requests/507</a>
to ensure that no changes are needed to your
job.</div>
<div><br>
</div>
<div>Custom jobs not based on the sysadmin
provided templates likely do not need to be
changed.</div>
<div><br>
</div>
<div>Projects that are Qt 5 based should remove
includes for everything except Linux general
(linux.yml) as those templates will be removed
as part of this roll out.</div>
<div><br>
</div>
<div><u>Timeline for the transition:</u></div>
<div><br>
</div>
<div>Over the next week the first node (<a href="http://node6.ci.kde.org" target="_blank">node6.ci.kde.org</a>)
will be taken out of rotation for running
existing container based jobs to facilitate
testing of the automated deployment. Once this
has been validated, the conversion of the
remaining nodes (node2, node3, node4, node5)
will be scheduled - converting them all to be
VM based CI runners.</div>
<div><br>
</div>
<div>To allow for final legacy jobs to be
completed, node1 will be left to run remaining
legacy jobs for a period of time (to be
determined). Once that finishes up, that node
will also be converted.</div>
<div><br>
</div>
<div>As part of this the dedicated VM runners
currently supplied for building KDE Linux and
Snaps will also be retired. Support for Yocto
builds is out of scope for this transition at
this time but that may be re-evaluated in the
future.</div>
<div><br>
</div>
<div><u>What specs will the VMs be provided
with?</u></div>
<div><br>
</div>
<div>By default builds will be provided with 8
vCPU cores, 16GB RAM and 200GB of disk space,
although some of that disk space will be
occupied by the VM OS, development tools,
etc. </div>
<div>Builds requiring additional resources
should file a Sysadmin ticket.</div>
<div><br>
</div>
<div>VMs will have a shared mount provided from
the host at /mnt that allows for artifacts to
be cached between builds on that node. This is
mainly intended to be used for caching
dependencies and other downloaded artifacts
and should not be relied on to transfer
results between jobs (as there is no guarantee
contents won't go away, or that jobs will be
allocated to the same build node).</div>
<div><br>
</div>
<div><u>What benefits will we get as part of
this change?</u></div>
<div><br>
</div>
<div>For Linux based builds, as these are now
running in a fully fledged system, anything
that depends on system level services (like
CUPS or systemd/logind) or which needs to
undertake actions normally restricted in a
Docker/Podman container (like manipulating
FUSE mounts or interacting with hardware like
a TPM) should now be able to be unit tested.</div>
<div><br>
</div>
<div>For FreeBSD based builds, these images no
longer have to be built manually by hand, and
are now easily available to be run locally on
developer systems for local testing.
Reliability of FreeBSD builds should also
improve with the elimination of the dependency
on the Podman daemon being running on the
build node.</div>
<div><br>
</div>
<div>For Windows based builds, these should run
faster due to the elimination of the overhead
of the Docker overlay file system, which on
Windows introduces significant overhead to IO
operations. For Kate, regular Windows CI build
times were reduced from 6 minutes (under
Docker) to 3 minutes 40 seconds (in a VM based
CI setup). This includes the overhead of
provisioning the VM and waiting for Windows to
start. We should also see improvements to
builder availability, as the out of memory
issues that have caused Windows build nodes to
be killed by the OOM-killer are not possible
in a VM based CI environment.</div>
<div><br>
</div>
<div>For binary builds, Linux support is
enhanced by it now being possible to build
Snaps, as well as Flatpaks no longer requiring
workarounds to be built, while Appimages are
now able to be run if needed. These are
actions that have all previously been
restricted by operating in a container
environment.</div>
<div><br>
</div>
<div>The system will also generally benefit from
being able to scale build capacity for
different OSes on an as needed basis within
our available hardware (so long build queues
for Windows post major release events should
become less of an issue).</div>
<div><br>
</div>
<div>As an additional benefit, the system will
require significantly less work to maintain.
Currently each build node, along with the
FreeBSD and Windows VM thereon, have to be
maintained by hand and disk space allocated
between them in a fixed fashion. This means
that any cleanup from stale disk images,
over-filled caches, etc. has to be done 3
times on each build node (being the Linux host
as well as the FreeBSD and Windows guest VMs).
Currently provisioning new nodes is
significantly labour intensive as well (see <a href="https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/gitlab-templates/README.md" target="_blank">https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/gitlab-templates/README.md</a>
for the instructions). </div>
<div><br>
</div>
<div>This is essentially completely eliminated
with the transition to VM based CI, with the
majority of the deployment now being possible
using Ansible with the only manual step being
the registration with Gitlab - which is a
fairly quick process taking less than 20
minutes per node. Maintenance is significantly
reduced as each node only needs one set of
cleanup - not three.</div>
<div><br>
</div>
<div>Should there be any questions on the above
please let me know.</div>
<div><br>
</div>
<div>Many thanks,</div>
<div>Ben</div>
</div>
</div>
</blockquote>
</div>
</div>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<u></u><u></u>
</blockquote></div></div>