[Nepomuk] Review Request 108950: FileIndexer: Do not use QDirIterators in the queues

[Vishesh Handa]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/108950/
-----------------------------------------------------------

Review request for Nepomuk and Simeon Bird.


Description
-------

    FileIndexer: Do not use QDirIterators in the queues
    
    A DirIterator works by opening the directory (as a file) and reading its
    contents, which are the list of files in that directory. As long as the
    dirIterator is open, it consumes one file descriptor.
    
    When iterating over all the folders, by using QDirIterator, we open a
    new file descriptor per directory. Since we traverse the file system via
    breadth first, the number of valid QDirIterators can get quite high.
    This sometimes results in us exceeding the number of file descriptors
    required which results in a crash, cause QProcess doesn't handle running
    out of file descriptors.
    
    Instead we now just iterate over all the directories and store them as
    strings. This may consume more memory, but it is better than a crash.
    
    BUG: 310777


This addresses bug 310777.
    http://bugs.kde.org/show_bug.cgi?id=310777


Diffs
-----

  services/fileindexer/basicindexingqueue.h 5d1c190 
  services/fileindexer/basicindexingqueue.cpp b581786 

Diff: http://git.reviewboard.kde.org/r/108950/diff/


Testing
-------

Checked the number of open file descriptors via `ls -l /proc/PID/fd/`. It remains constant with this patch, earlier it used to increase significantly based on the directory structure. 


Thanks,

Vishesh Handa

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/nepomuk/attachments/20130214/cb35c36d/attachment-0001.html>