accumulating projects in kate lead to excessive kate startup time due to git

[Christoph Cullmann (cullmann.io)]

On 2022-06-19 21:25, Christoph Cullmann (cullmann.io) wrote:
> On 2022-06-19 21:10, Milian Wolff wrote:
>> On Samstag, 18. Juni 2022 14:15:42 CEST Milian Wolff wrote:
>>> Hey all,
>>> 
>>> Kate took ~4s to show its main window on my beefy workstation with 
>>> lots of
>>> RAM, CPUs and speedy NVME disks. I found this quite odd and wondered 
>>> about
>>> the reason so I sat down and profiled it. Perf shows a lot of 
>>> external git
>>> processes running sequentially, which I could also replicate with 
>>> strace:
>> 
>> <snip>
>> 
>>> b) Can we query the git status in parallel for all projects, instead 
>>> of
>>> serially? My machine has 12 cores and 24 threads, and the NVME disk 
>>> and ram
>>> should also allow this.
>> 
>> Sorry, hit sent too early...
>> 
>> You can download the perfparser file here:
>> 
>> https://milianw.de/files/kate.slow.startup.perfparser
>> 
>> You can open that in hotspot and then go to the off-CPU time flame 
>> graph.
>> Basically all of that comes from _really_ slow memory allocations, 
>> which is a
>> first for me. It seems like my system is suffering from some extreme 
>> slowdowns
>> in `int_malloc` - but only in kate. Other applications don't show this
>> behavior, and I'm unsure where this comes from... See the excessively 
>> slow
>> calls to rwsem_down_read_slowpath from _int_malloc, even in the main 
>> thread.
>> If you look at the main thread e.g. there we see ~1s off cpu time from
>> _int_realloc by _FcConfigParse::FcStrBufData alone!
>> 
>> I'll try to continue to figure this out
> 
> Hi,
> 
> from top of my head what could be an issue is that we do use stuff like
> 
> QtConcurrent::blockingMap
> 
> inside runnables.
> 
> We have one runnable per project, we have one thread pool for them.
> 
> But I would assume the QtConcurrent again will span as many threads
> as cores per default.
> 
> That might lead to suboptimal performance.
> 
> But I could be wrong.

Could you try if you pass the pool to the worker and use the same pool
there the behavior is better?

Greetings
Christoph

> 
> Greetings
> Christoph

-- 
Ignorance is bliss...
https://cullmann.io | https://kate-editor.org