[neon/snap-packaging/snap-helpers] /: Initial commit with snapcraft-inject

[Kevin Ottens]

Git commit 187c50fe02c38f1399bec4f81010f7f43cf0683f by Kevin Ottens.
Committed on 18/04/2024 at 15:19.
Pushed by ervin into branch 'master'.

Initial commit with snapcraft-inject

A  +51   -0    README.md
A  +119  -0    snapcraft-inject

https://invent.kde.org/neon/snap-packaging/snap-helpers/-/commit/187c50fe02c38f1399bec4f81010f7f43cf0683f

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..4141377
--- /dev/null
+++ b/README.md
@@ -0,0 +1,51 @@
+# Snap Helpers
+
+This repository contains scripts to help building snaps using snapcraft. In particular it tries to address the need for developers to build things locally testing dependencies without having to publish on the store at each step.
+
+## snapcraft-inject
+
+### Setup
+
+For easier use, `snapcraft-inject` needs to be in your path. It assumes the `SNAPCRAFT_INJECT_ROOT` environment variable is set and points to a directory containing the snaps to inject.
+
+`snapcraft-inject` will look into `SNAPCRAFT_INJECT_ROOT` for snaps to inject, but it will do so recursively. This is especially convenient if you have several repositories coming from [snap-packaging](https://invent.kde.org/neon/snap-packaging) in a single directory. You just need to point the environment to said directory.
+
+For instance, let's assume a `$HOME/snap-packaging` directory containing clones of `kf6-snap` and `kf6-snap-runtime`. Then setting `SNAPCRAFT_INJECT_ROOT` to `$HOME/snap-packaging` is probably a good choice.
+
+Also this script will assume you're using the lxd build provider with snapcraft. So if not already done, make sure you installed lxd properly and run `snap set snapcraft provider=lxd`. For more details about build providers and how to set them up, check out the [snapcraft documentation about build providers](https://snapcraft.io/docs/build-providers).
+
+### Building snaps using locally built dependencies
+
+`snapcraft-inject` should be seen as an alternative to the `snapcraft pull` lifecycle command of `snapcraft`. In fact, under the hood `snapcraft-inject` will run `snapcraft pull` to do its job. For more information, check out the documentation about [snapcraft parts lifecycle](https://snapcraft.io/docs/parts-lifecycle).
+
+Once the setup is done, building a snap using your locally built snaps as dependencies should be as simple as running:
+
+* `snapcraft-inject && snapcraft`
+
+### How it works
+
+`snapcraft-inject` does the following:
+
+* runs a `snapcraft pull` in trace mode to find the lxd container used to build the snap
+* then it gets into the container to look for stage-snaps in each parts and for installed build-snaps
+* for each snap found, it looks for a replacement in `SNAPCRAFT_INJECT_ROOT`
+  * when a replacement matches, the original is removed and the replacement is put in its place
+
+When `snapcraft` is run after `snapcraft-inject` the following will happen:
+
+* the `pull` step will be skipped since `snapcraft` knows it already happened
+* all the dependencies replaced by `snapcraft-inject` will thus stay in place
+* `snapcraft` will carry on with the other steps all the way to packaging
+
+### Dependencies
+
+This is the usual you would need to use snapcraft productively so:
+
+* snap
+* snapcraft
+* lxd
+
+This is also script so to run it needs:
+
+* python3
+
diff --git a/snapcraft-inject b/snapcraft-inject
new file mode 100755
index 0000000..1d02661
--- /dev/null
+++ b/snapcraft-inject
@@ -0,0 +1,119 @@
+#! /usr/bin/env python3
+
+import glob
+import os
+import subprocess
+
+
+SNAPCRAFT_INJECT_ROOT = os.environ["SNAPCRAFT_INJECT_ROOT"]
+
+
+class Lxc:
+    def __init__(self, container_name):
+        self.container_name = container_name
+
+    def command(self, params):
+        return subprocess.check_output(
+            f"lxc --project snapcraft {params}", shell=True, text=True
+        ).splitlines()
+
+    def exec(self, params):
+        return self.command(f"exec {self.container_name} -- {params}")
+
+    def start(self):
+        self.command(f"start {self.container_name}")
+
+    def stop(self):
+        self.command(f"stop {self.container_name}")
+
+    def list_parts(self):
+        return self.exec("ls parts")
+
+    def list_stage_snaps(self, part):
+        if "stage_snaps" in self.exec(f"ls parts/{part}"):
+            return [
+                f
+                for f in self.exec(f"ls parts/{part}/stage_snaps")
+                if f.endswith(".snap")
+            ]
+        else:
+            return []
+
+    def replace_stage_snap(self, part, stage_snap, local_snap):
+        print(f"Replacing {part}'s stage snap {stage_snap} with {local_snap}")
+        base_name = os.path.basename(local_snap)
+        stage_assert = os.path.splitext(stage_snap)[0] + ".assert"
+
+        self.command(f"file push {local_snap} {self.container_name}/tmp/{base_name}")
+        self.exec(f"rm parts/{part}/stage_snaps/{stage_snap}")
+        self.exec(f"rm parts/{part}/stage_snaps/{stage_assert}")
+        self.exec(f"mv /tmp/{base_name} parts/{part}/stage_snaps")
+
+    def list_installed_snaps(self):
+        return self.exec("snap list | grep -v Version | awk '{print $1;}'")
+
+    def replace_installed_snap(self, snap_name, local_snap):
+        print(f"Replacing installed snap: {snap_name} with {local_snap}")
+        self.command(
+            f"file push {local_snap} {self.container_name}/tmp/{snap_name}.snap"
+        )
+        self.exec(f"snap remove {snap_name}")
+        self.exec(f"snap install --devmode /tmp/{snap_name}.snap")
+        self.exec(f"rm /tmp/{snap_name}.snap")
+
+
+def pull_and_get_container_name():
+    print("Please wait, executing snapcraft pull and trying to find container name")
+    command = """
+        snapcraft pull --verbosity=trace 2>&1 | 
+        grep 'Executing on host: lxc --project snapcraft stop' | 
+        awk '{print $NF;}'
+    """
+    container_name = subprocess.check_output(command, shell=True, text=True).strip()
+    return container_name
+
+
+def find_local_snaps():
+    return glob.glob(SNAPCRAFT_INJECT_ROOT + "/**/*.snap", recursive=True)
+
+
+def snap_base_name(snap_name):
+    base_name = os.path.basename(snap_name)
+    base_name = os.path.splitext(base_name)[0]
+    return base_name.split("_")[0]
+
+
+def main():
+    local_snaps = find_local_snaps()
+    local_snaps = {snap_base_name(snap): snap for snap in local_snaps}
+
+    lxc = Lxc(pull_and_get_container_name())
+
+    lxc.start()
+
+    installed_snaps = lxc.list_installed_snaps()
+
+    parts = lxc.list_parts()
+    stage_snaps_by_part = {part: lxc.list_stage_snaps(part) for part in parts}
+    stage_snaps_by_part = {
+        part: snaps for part, snaps in stage_snaps_by_part.items() if snaps
+    }
+
+    # Replace installed snaps
+    for snap_name, local_snap in local_snaps.items():
+        if snap_name in installed_snaps:
+            lxc.replace_installed_snap(snap_name, local_snap)
+
+    # Replace stage snaps in each part
+    for part, stage_snaps in stage_snaps_by_part.items():
+        for stage_snap in stage_snaps:
+            stage_snap_name = snap_base_name(stage_snap)
+            if stage_snap_name in local_snaps:
+                local_snap = local_snaps[stage_snap_name]
+                lxc.replace_stage_snap(part, stage_snap, local_snap)
+
+    lxc.stop()
+
+
+if __name__ == "__main__":
+    main()