accumulating projects in kate lead to excessive kate startup time due to git

[Thiago Macieira]

On Monday, 20 June 2022 13:17:18 PDT Milian Wolff wrote:
> Which branch is this made for? It does not apply to the current HEAD of the
> kde patch collection (i.e. git at invent.kde.org:qt/qt/qtbase.git branch
> kde-5.15).

dev, of course. That's the only branch I work on. This patch would be a 
performance improvement, so it wouldn't be a candidate for backporting to 
older versions of Qt anyway.

forkfd itself should be nearly identical between 5.15 and dev. 
QProcessPrivate::startProcess() is very different because it's missing the 
clean ups I made for 6.2, but you should be able to insert in the right 
places. The logic is still the same.

> On Monday, 20 June 2022 17:53:42 CEST Thiago Macieira wrote:
> > The most important of which is that posix_spawn is implemented using
> > fork()
> > +execve() inside glibc, so it doesn't improve anything.
> 
> This is not true anymore, I believe. See e.g. [1] which stats:
> > glibc's posix_spawn was rewritten in 2016 to use clone() with CLONE_VM

On Linux, with glibc, yes. That doesn't solve the other problems, other 
platforms, and the older glibcs.

> > It just moves the
> > code doing fork() down into a large function whose implementation
> > duplicates what we have, poorly.
> 
> Can you revisit this assessment please in light of more modern
> implementations?

Yes.

It's still incompatible with forkfd's needs, namely that the child process 
does not exit prior to our installing its PID in the array of children we're 
waiting on. Without the ability to either use a pidfd or block the child 
before execve(), we race the child's exit and may receive its SIGCHLD before 
letting the SIGCHLD handler know to wait for this child. So it wouldn't and 
we'd end up with a zombie.

> > The other visible reason is the fact that QProcess allows you to do
> > something in the child process before execve() as part of its API. So we
> > need to be able to control the forking.
> 
> What exactly does this mean - which part of the API of QProcess is not
> portable to posix_spawn?

https://doc.qt.io/qt-6/qprocess.html#setChildProcessModifier
or the Qt 5 equivalent:
https://doc.qt.io/qt-5/qprocess.html#setupChildProcess

As you can see from the (stale) warning in Qt 5, it didn't use to do anything 
on QNX which is what spawnfd was created for. But as I pointed out above, 
there's a race condition without proper fork() in the absence of pidfds.

> > I thought of using vfork() instead, but that has a lot of extra
> > restrictions too. QProcess would probably have to revert back to regular
> > fork if the user set a child process modifier function, the same way it
> > reverts to regular fork() instead of clone() on Linux. But it might help
> > because it does not mark the child process's page as COW to the parent.
> 
> What is a "process modifier function"? I can find [2] but that seems windows
> specific to me?

See link above. Obviously I'm talking about the Qt 6 API. The Qt 5 equivalent 
was broken and replaced in Qt 6 *because* it prevented us from using pidfds 
properly.

-- 
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
   Software Architect - Intel DPG Cloud Engineering