[KRdp] [Bug 507949] New: RDP sessions that are not closed client side but stop receiving data cause OOM condition

[Cody Wilson]

https://bugs.kde.org/show_bug.cgi?id=507949

            Bug ID: 507949
           Summary: RDP sessions that are not closed client side but stop
                    receiving data cause OOM condition
    Classification: Plasma
           Product: KRdp
      Version First 6.4.3
       Reported In:
          Platform: CachyOS
                OS: Linux
            Status: REPORTED
          Severity: normal
          Priority: NOR
         Component: general
          Assignee: unassigned-bugs at kde.org
          Reporter: cody at codywilson.co
                CC: ahiemstra at heimr.nl
  Target Milestone: ---

SUMMARY
RDP sessions that are not closed client side but stop receiving data cause OOM
condition 

STEPS TO REPRODUCE
1. Connect to the kRDP server as normal 
2. Allow the client's operating system to sleep without closing the RDP client
session

OBSERVED RESULT
RAM utilization on the system will slowly increase until the system becomes
unresponsive, requiring either manual intervention or being killed by oomkiller
and the like. 

EXPECTED RESULT
A timeout or buffer size monitor threshold is triggered when the server can not
send data to the client after an appropriate length, closing the connection
server-side. 

SOFTWARE/OS VERSIONS
Client Side: 
Thincast Remote Desktop Client
1.1.634 stable (tcc-stable-1.1.634)
OpenSSL
OpenSSL 1.1.1w 11 Sep 2023
OрenH264 2.6.0
CJSON: 1.7.16
FreeRDP: 3.16.1-devO (3.16.1)
WinPR: 3.16.1-devO (3.16.1)
Qt: 5.15.15

macOS: 15.6

Server Side:
Operating System: CachyOS Linux
KDE Plasma Version: 6.4.3
KDE Frameworks Version: 6.16.0
Qt Version: 6.9.1
Kernel Version: 6.16.0-5-cachyos (64-bit)
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 7 9800X3D 8-Core Processor
Memory: 64 GiB of RAM (60.4 GiB usable)
Graphics Processor 1: AMD Radeon RX 6800 XT
Graphics Processor 2: AMD Radeon Graphics

ADDITIONAL INFORMATION
I believe this to be caused by either krdpserver buffering video, or the
network socket buffering the data to be sent to the client. I'm in the process
of collecting additional data about this should it be valuable. 

I have not been able to test this with other clients as this particular client
is the only one I currently have that supports h.264 enabled clients
appropriately. I have however been able to reproduce the issue several times
successfully. It is easier to reproduce this when the server side is producing
complex video data to send over the wire. In my case, this is usually triggered
when I forget to close an RDP client session before switching to the RDP server
and playing a game. I imagine watching a video would also produce sufficiently
complex data to be encoded and trigger the condition.

-- 
You are receiving this mail because:
You are the assignee for the bug.