Review Request: Implemented multithreading in KFileItemModelSortAlgorithm

[Emmanuel Pescosta]

> On Oct. 25, 2012, 5:37 a.m., Frank Reininghaus wrote:
> > Thanks for working on this!
> > 
> > How many CPU cores do you have? I have only 2, so I can't really test the ">2 cores" case. However, I have an idea what could be the reason for the performance drop that you see when trying to use more than 2 threads: you actually try to create more threads than QtConcurrent lets you.
> > 
> > Example: Suppose you have 4 cores. Your parallelSort() function will:
> > 
> > * sort the first half using QtConcurrent::run(), which creates a thread 1,
> > * sort the second half using QtConcurrent::run(), which creates a thread 2.
> > 
> > Thread 1 will now split the first half into two quarters and:
> > 
> > * create a thread 3 for the first quarter,
> > * create a thread 4 for the second quarter.
> > 
> > At the same time, thread 2 tries to sort the third and fourth quarters concurrently, but Qt:Concurrent doesn't let it because the number of parallel threads is already equal to the number of cores -> it has to wait until the first half is finished. But note that threads 1 and 2 are actually idle in this scenario!
> > 
> > Suggested solution: in parallelSort(), only sort the first half using QtConcurrent::run(), and sort the second one synchronously using a plain recursive call of parallelSort(). Could you try if that works?
> >

> How many CPU cores do you have?
I have 4 cores. (Every core runs with 100% while sorting)

> you actually try to create more threads than QtConcurrent lets you
Nope, I already thought about that ;) I implemented it without recursive function calls.

> Suggested solution: in parallelSort(), only sort the first half using QtConcurrent::run(), and sort the second one synchronously using a plain recursive call of 
> parallelSort(). Could you try if that works?
Hmm ... 97 seconds for 500.000 files (With recursive parallelSort and with your suggestions)

I will upload my other patch, which uses as many threads as possible (according to idealThreadCount())


- Emmanuel


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/107025/#review20832
-----------------------------------------------------------


On Oct. 24, 2012, 5:20 p.m., Emmanuel Pescosta wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/107025/
> -----------------------------------------------------------
> 
> (Updated Oct. 24, 2012, 5:20 p.m.)
> 
> 
> Review request for Dolphin and Frank Reininghaus.
> 
> 
> Description
> -------
> 
> Implemented multithreading in KFileItemModelSortAlgorithm.
> 
> If more than 100 items to sort and ideal thread count is greater than 1 -> sort them with parallelSort (2 Threads)
> 
> Use maximal 2 Threads, because more than 2 Threads are "slower" (more overhead than speed up). (I also have a patch which uses n Threads for sorting, if you want test it ;)
> 
> 
> Diffs
> -----
> 
>   dolphin/src/kitemviews/private/kfileitemmodelsortalgorithm.h 3a596df 
>   dolphin/src/kitemviews/private/kfileitemmodelsortalgorithm.cpp e0aac13 
> 
> Diff: http://git.reviewboard.kde.org/r/107025/diff/
> 
> 
> Testing
> -------
> 
> About 2 seconds faster with sorting 500.000 files.
> About 5 seconds faster with sorting 1.000.000 files.
> 
> 
> Thanks,
> 
> Emmanuel Pescosta
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.kde.org/mailman/private/kfm-devel/attachments/20121025/cb98783c/attachment.htm>