Review Request: Implemented multithreading in KFileItemModelSortAlgorithm

[Frank Reininghaus]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/107025/#review20832
-----------------------------------------------------------


Thanks for working on this!

How many CPU cores do you have? I have only 2, so I can't really test the ">2 cores" case. However, I have an idea what could be the reason for the performance drop that you see when trying to use more than 2 threads: you actually try to create more threads than QtConcurrent lets you.

Example: Suppose you have 4 cores. Your parallelSort() function will:

* sort the first half using QtConcurrent::run(), which creates a thread 1,
* sort the second half using QtConcurrent::run(), which creates a thread 2.

Thread 1 will now split the first half into two quarters and:

* create a thread 3 for the first quarter,
* create a thread 4 for the second quarter.

At the same time, thread 2 tries to sort the third and fourth quarters concurrently, but Qt:Concurrent doesn't let it because the number of parallel threads is already equal to the number of cores -> it has to wait until the first half is finished. But note that threads 1 and 2 are actually idle in this scenario!

Suggested solution: in parallelSort(), only sort the first half using QtConcurrent::run(), and sort the second one synchronously using a plain recursive call of parallelSort(). Could you try if that works?



dolphin/src/kitemviews/private/kfileitemmodelsortalgorithm.cpp
<http://git.reviewboard.kde.org/r/107025/#comment16445>

    This change is unrelated to multithreading, could you revert it? It makes the code harder to read, and if the compiler is clever, it will generate the same code in both cases anyway.
    
    (the same applies to other uses of ">> 1" vs. "/ 2").


- Frank Reininghaus


On Oct. 24, 2012, 5:20 p.m., Emmanuel Pescosta wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/107025/
> -----------------------------------------------------------
> 
> (Updated Oct. 24, 2012, 5:20 p.m.)
> 
> 
> Review request for Dolphin and Frank Reininghaus.
> 
> 
> Description
> -------
> 
> Implemented multithreading in KFileItemModelSortAlgorithm.
> 
> If more than 100 items to sort and ideal thread count is greater than 1 -> sort them with parallelSort (2 Threads)
> 
> Use maximal 2 Threads, because more than 2 Threads are "slower" (more overhead than speed up). (I also have a patch which uses n Threads for sorting, if you want test it ;)
> 
> 
> Diffs
> -----
> 
>   dolphin/src/kitemviews/private/kfileitemmodelsortalgorithm.h 3a596df 
>   dolphin/src/kitemviews/private/kfileitemmodelsortalgorithm.cpp e0aac13 
> 
> Diff: http://git.reviewboard.kde.org/r/107025/diff/
> 
> 
> Testing
> -------
> 
> About 2 seconds faster with sorting 500.000 files.
> About 5 seconds faster with sorting 1.000.000 files.
> 
> 
> Thanks,
> 
> Emmanuel Pescosta
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.kde.org/mailman/private/kfm-devel/attachments/20121025/e4d3702b/attachment.htm>