[Nepomuk] Review Request 108950: FileIndexer: Do not use QDirIterators in the queues

Simeon Bird bladud at gmail.com
Thu Feb 14 16:59:05 UTC 2013


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/108950/#review27458
-----------------------------------------------------------

Ship it!


This is indeed the source of the bug, and the patch fixes it - I wrote something similar independently last night.

- Simeon Bird


On Feb. 14, 2013, 7:57 a.m., Vishesh Handa wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/108950/
> -----------------------------------------------------------
> 
> (Updated Feb. 14, 2013, 7:57 a.m.)
> 
> 
> Review request for Nepomuk and Simeon Bird.
> 
> 
> Description
> -------
> 
>     FileIndexer: Do not use QDirIterators in the queues
>     
>     A DirIterator works by opening the directory (as a file) and reading its
>     contents, which are the list of files in that directory. As long as the
>     dirIterator is open, it consumes one file descriptor.
>     
>     When iterating over all the folders, by using QDirIterator, we open a
>     new file descriptor per directory. Since we traverse the file system via
>     breadth first, the number of valid QDirIterators can get quite high.
>     This sometimes results in us exceeding the number of file descriptors
>     required which results in a crash, cause QProcess doesn't handle running
>     out of file descriptors.
>     
>     Instead we now just iterate over all the directories and store them as
>     strings. This may consume more memory, but it is better than a crash.
>     
>     BUG: 310777
> 
> 
> This addresses bug 310777.
>     http://bugs.kde.org/show_bug.cgi?id=310777
> 
> 
> Diffs
> -----
> 
>   services/fileindexer/basicindexingqueue.h 5d1c190 
>   services/fileindexer/basicindexingqueue.cpp b581786 
> 
> Diff: http://git.reviewboard.kde.org/r/108950/diff/
> 
> 
> Testing
> -------
> 
> Checked the number of open file descriptors via `ls -l /proc/PID/fd/`. It remains constant with this patch, earlier it used to increase significantly based on the directory structure. 
> 
> 
> Thanks,
> 
> Vishesh Handa
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/nepomuk/attachments/20130214/49b240eb/attachment.html>


More information about the Nepomuk mailing list