My guess would be your GPU is running out of resources. On a composited desktop, each window is a texture map. The textures then get composited onto an off-screen surface. All of this requires a lot of memory. - Bartosz