D10085: Faster drag&drop in directories with thousands of files

[Jaime Torres Amate]

jtamate added a comment.


  In https://phabricator.kde.org/D10085#199702, @markg wrote:
  
  > @jtamate, you are trying to make this faster, which is awesome! But i think there is another issue. I just tried to benchmark this as well with a folder of 5000 empty files.
  >  Within DragAndDropHelper::urlListMatchesUrl i see 40.007 calls to QUrl::matches. That smells weird.
  >  Sure, there are probably some extra urls's (don't know where they come from), but that could explain the last 7. But the remaining 40.000 is 8x as much as the number of items that are being dragged. That's wrong!
  >  Note that this is merely with one drag/drop.
  >
  > Further investigation shows DragAndDropHelper::urlListMatchesUrl is being called:
  >  6x from KItemListController::dragMoveEvent (the function itself is called 6x)
  >  1x from DragAndDropHelper::dropUrls (the function itself is called 1x)
  
  
  One more call with the same list and there you are the 40.000 matches calls.
  
  > That's a room for optimization. You could call DragAndDropHelper::urlListMatchesUrl when you start a drag event and simply use those results in subsequent calls till the drag is canceled. That would reduce the number of calls to DragAndDropHelper::urlListMatchesUrl from 7 to 1 (and in effect make it 7x faster :)).
  
  I tried something similar. Take a look at @elvisangelaccio first comment.
  
  I tried to insert some data in the QClipboard to mark that it has been already done, but it is constant. :-(
  
  If someone knows a way to keep an status about the last call, please implement it, I'm out of ideas here.
  
  > But there is more! Deeper in KIO KFileItemListProperties is filled for this drag/drop event (to determine the common actions).
  
  And also there is another similar copy of the same method (That I've found today).
  
  > That object is created 13 times. Each time it's calling setitems (KFileItemListPropertiesPrivate::setItems), that function could use some optimizations. The most costly one in there is the call to KFileItem::isWritable call.
  >  Just reducing the amount of times KFileItemListProperties is created gives an instant performance win as well and might be easier to optimize.
  
  I'll try that also, but the way I do it is looking for the most expensive calls in callgrind or perf and try to reduce its time or the number of calls it receives, and then do the same with the next one...
  
  > To conclude, please don't optimize this by running it through QtConcurrent. It gives the "appearance" of being faster but you only mask the real issues. As you can see with what i just said, there are quite a few places where you can optimize it which would benefit everyone, not just Dolphin.
  
  I just go step by step, I've been lucky up to now, but it is getting much harder.
  For example I've spent a week to try to figure out why plasma is blocked when pasting so many files. I think the problem has something to do with the clipboard trying to refill the complete list thousands of times,but the QXcbConnection errors make this hard.
  
  > I am therefore putting this on "Request Changes".

REPOSITORY
  R318 Dolphin

REVISION DETAIL
  https://phabricator.kde.org/D10085

To: jtamate, #dolphin, elvisangelaccio, markg
Cc: markg, anthonyfieroni, michaelh, elvisangelaccio, ngraham
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.kde.org/mailman/private/kfm-devel/attachments/20180202/6846b854/attachment.htm>