Scene redesign

[David Edmundson]

> On Wayland, no window quads are generated for sub-surfaces. This
> basically means that effects like Magic Lamp and Wobbly Windows won't
> work as expected for clients that utilize sub-surfaces, e.g. video
> players. Another problem is that sub-surfaces cannot be clipped since
> they don't have quads. See https://bugs.kde.org/387313
>
I think we'll need clipping regardless for non-transformed case with
window occlusion.

> Some of those issues were discussed in
> https://phabricator.kde.org/T10530. In general, I don't see how we can
> fix texture bleeding and introduce window quads for sub-surfaces without
> breaking API. Anyway,
> let me go quickly through the proposed solution.
>
> Sub-surfaces form a tree, thus our scene data structures must reflect
> that. Though, we could go a bit further and include the server-side
> drop-shadow as well the server-side decoration in that tree, e.g.
>
>     ShadowNode
>     └── DecorationNode
>         └── SurfaceNode (A)
>             ├── SurfaceNode (B)
>             │   └── SurfaceNode (C)
>             └── SurfaceNode (D)
>
> - ShadowNode represents the server-side drop-shadow
> - DecorationNode represents the server-side decoration
> - SurfaceNode (A) represents the main surface (or pixmap)
> - SurfaceNode (B) (C) (D) represent sub-surfaces
>
> Neither ShadowNode nor DecorationNode nor SurfaceNode solve the texture
> bleeding problem. Their purpose is to model surface hierarchy and act as
> the source of window quads that can be used **internally** by Scene.
>
> In order to fix the texture bleeding, we need to render the client into
> an offscreen texture, after that map the offscreen texture on the screen
> with all transformations applied. However, we can't implement that since
> WindowQuadContents, WindowQuadDecoration, and WindowQuadShadow will have
> texture coordinates that are not suitable for the offscreen texture.

I don't understand why we can't implement that.

Both scene and a transforming effect would see 1 WindowQuadContents
which map to the whole offscreen texture, with the others empty.  An
effect still has access to the various window geometry methods of the
root window to acquire this information.

It breaks some hypothetical cases of anyone relying on
WindowQuad::type / WindowQuad::decoration, but that appears to only be
screenshot in a way that's easily fixable. Not worth over-engineering
anything else for. It also breaks the case of two effects performing a
transform at once..but that effectively breaks anyway.

> One
> could say that we could adjust the texture coordinates late in the
> rendering process, however it is not possible due to the way we render
> server-side drop-shadows. Also, adjusting texture coordinates will make
> the code much more difficult.
>
> Which brings us to sub-surface window quads. I think the best option is
> to not introduce them to libkwineffects. In fact, it will be better not
> to expose any window quad types to effects.
>
> Q: What does it mean for us, KWin developers?
> A: Two things. First, we'll be able to add proper support for
> sub-surfaces that won't suck (I hope). Second, we will have some space
> for potential future scene redesigns that won't require changes in third
> party effects.
>
> Q: What does it mean for third-party binary effects?
> A: Yeah... Effects like shapecorners will stop working.
>
> So, the proposed rendering algorithm looks something like this
>
> 1. Generate a WindowQuad suitable for the offscreen texture and pass it
> to WindowPrePaintData
> 2. Call prePaintWindow method. If an effect transforms the window quad
> in **any** way, it must set PAINT_WINDOW_TRANSFORMED
> 3. Call paintWindow method
> 4. When performPaintWindow() of the scene window has been reached, two
> things might happen
> 4.1. (the window is transformed) Render the window into an offscreen
> texture, then map the offscreen texture on the screen. The scene could
> check whether the client has been damaged in order to skip the first
> step and move straight to the second step, i.e. mapping the offscreen
> texture on the screen
> 4.2. (the window is not transformed) Render the window onto the screen
> with optimizations and all that good stuff
>
> Redesign of Scene can be accomplished in two steps
> - First, we need to introduce the offscreen rendering mechanism in the
> OpenGL scene (BREAKING CHANGE)
> - Second, refactor scene.cpp and introduce ShadowNode, SurfaceNode, and
> DecorationNode classes (perhaps, split/re-brand WindowPixmap class)
>

So to clarify we would still have sometimes a node referring to a
texture that's part of the decoration atlas / padded window pixmap
texture, sometimes referring to a portion of the composited offscreen
pixmap, just we hide things behind a wrapper so it's abstracted from
effects.

> Unfortunately, my proposed solution breaks API compatibility. However,
> as I said a bit earlier in this email, I don't see any other way. If you
> know a way to do that, I'm all ears. Thoughts? Objections? Suggestions?
>

David

> Cheers,
> Vlad