Kdenlive quality projects

[Eric Jiang]

Hi all,
I continue to believe that the general quality of Kdenlive releases is
one of the most important areas for improvement. I have some ideas
that I think could help, but only limited time to implement them. I'd
like to hear from you all what would be most valuable and should be
done first.

* UI Automation Testing - We can create some UI tests (click menu
item, click button, etc.) for some common tasks.

Pros: Can run tests without source code, so we can test packaged
releases and check for packaging issues like missing dependencies.
Possible to run tests across platforms. Can test UI that the current
unit tests cannot cover.
Cons: UI tests tend to be flakey, slow, and/or hard to write. No code
coverage data.

* Project test suite - We can create a suite of Kdenlive projects and
test that they render pixel-for-pixel in each Kdenlive version.

Pros: Catches when we accidentally break/remove effects. Catches
regressions when we change timeline behavior and clips are shifted.
Helps ensure that old projects will continue to work in future
versions of Kdenlive. Could catch some rendering crashes (e.g. GPU
accel).
Cons: Slow to run. Requires a lot of sample projects + audio/video
files as test cases. Might require some code changes to be able to
script properly.

* Better crash logging - Many of the bug reports do not have a
backtrace, and it's not currently possible to get backtraces for some
release types (AppImage?).

Krita provides a UI dialog to view its log, which also includes
backtraces. The krita log file is also easy to get on Windows.
https://docs.krita.org/en/reference_manual/sharing_krita_logs.html#getting-backtrace
Other areas for improvement include logging what frame a render job
crashes at along with what effects are used during that time. I'd also
like to know how other KDE projects improve the bug reporting flow.

Pros: backtraces make it easier to fix bugs.
Cons: Users need to be educated on how to find and upload logs. May be
difficult to get backtraces in AppImage, Flatpak, Snap, or for the
renderer. Doesn't help catch bugs before they're released.

I think all are feasible to implement, although all would require some
additional ongoing work to be useful. Any suggestions or comments? Any
other ideas that I should look into?

Thanks,
Eric