<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="http://git.reviewboard.kde.org/r/110794/">http://git.reviewboard.kde.org/r/110794/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On June 3rd, 2013, 11:01 a.m. UTC, <b>Vishesh Handa</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">No. I do not think this is the correct way of going about this.
The main problem is that if some 10 files are failing, they will fail each time the FileIndexer will run. We do not want to try and reindex those files on startup each time. Also, what's the point of showing this to the user? They cannot do anything about it. It will just annoy them.
Instead, I think it would be better to modify the kext:indexingLevel to -1 or some other value. This way, those files will not be returned in the fillQueue query. Maybe in 4.12 we could have some code which tries to re-index all the files which have not been successfully indexed.</pre>
</blockquote>
<p>On June 3rd, 2013, 2 p.m. UTC, <b>Simeon Bird</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Ok - hmm. My thought is that files which do not index right
are always a bug (and it seems quite rare bugs since 4.10).
So what we really want, I think, is for them to be re-indexed on every upgrade
(we might have fixed the bug between 4.10 and 4.10.1). Is there a way of getting
when the nepomuk version has changed?
Also, as to showing this to the users: since these are bugs, I wanted to
have some way to get the information about the failed files to us. I guess actually what would
be ideal would be a nice little dialog saying "Nepomuk failed to index file XXX.
Would you like to send this file to the developers?" and then a button that does that.
Is that a good idea?</pre>
</blockquote>
<p>On June 3rd, 2013, 2:14 p.m. UTC, <b>Vishesh Handa</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Yes. You're right. It is a bug. But it's not a very serious bug. The user can still search based on file name and mimetype.
I rather not annoy the user about something as trivial as a file not being indexed. There is a debug mode which can be enabled to get which files have failed to index. [1]
I'm not too keen on indexing the files on each upgrades. Or maybe it is. My ideal approach would be for each plugin to have a version number, which can be used to detect if that particular indexer has been updated, and accordingly reindex the files. We could even theoretically do major.minor version numbers, and when a minor update is done, only the files that failed to index the last time.
Or maybe I'm thinking too much.
[1] http://userbase.kde.org/Nepomuk/FileIndexer#File_Indexing_Errors</pre>
</blockquote>
<p>On June 4th, 2013, 1:35 p.m. UTC, <b>Simeon Bird</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Ok - agreed on not notifying on a files not being indexed.
As for re-indexing: versioned plugins make sense to me and could work well, if we remember to update the versions. I guess your idea is that major versions would signal new indexing functionality?
I do think though that files failing to index are very rare right now, empirically, as deduced from the fact that we have had only about 4 bugs in the last 6 months complaining about nepomuk using all the cpu re-indexing something (and its always only one file). So I think it is acceptable to just try to re-index these files on upgrade as well. Minor updates would I think be too much effort for this rare case.
What do you think?</pre>
</blockquote>
<p>On June 4th, 2013, 1:41 p.m. UTC, <b>Vishesh Handa</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Sure.
Do you want to write a patch so that it doesn't keep reindexing the same files?</pre>
</blockquote>
</blockquote>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Yup, will do.</pre>
<br />
<p>- Simeon</p>
<br />
<p>On June 3rd, 2013, 3:57 a.m. UTC, Simeon Bird wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('http://git.reviewboard.kde.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Nepomuk, Jörg Ehrichs and Vishesh Handa.</div>
<div>By Simeon Bird.</div>
<p style="color: grey;"><i>Updated June 3, 2013, 3:57 a.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">FileIndexer: report if indexing certain files failed
At the moment, if the fileindeer chokes on a file, it will be repeatedly
requeued, and the indexer will consume all the CPU forever.
This patch stores a list of any files that did not index internally and
does not add them to the queue. This list does not persist across
restarts, because indexing failures may well be transient.
We are limited to 100 failed files. Once we have this many, just stop
indexing.
Furthermore add dbus methods to the indexer service to report the
number of failed files as well as their filenames and error message.
BUG: 315817
FIXED-IN: 4.11
There should be some sort of ui component for this, so that user can easily see and
copy-paste the names of files that didn't index properly. I didn't write this yet,
as I was hoping that by waiting I can just add it to the new qml controller.
I'm not completely convinced by the showFailedFiles function either - a newline
separated list seems like it could be improved upon.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Compiled, ran, called the dbus methods. No files fail to index for me now though</pre>
</td>
</tr>
</table>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="http://bugs.kde.org/show_bug.cgi?id=315817">315817</a>
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>interfaces/org.kde.nepomuk.FileIndexer.xml <span style="color: grey">(9e56cb139f05e4a4267adf575d0894c5a69935a1)</span></li>
<li>services/fileindexer/fileindexer.h <span style="color: grey">(3d87a04bcf6a7d8c9402e8085daeb7c630bc4e91)</span></li>
<li>services/fileindexer/fileindexer.cpp <span style="color: grey">(8c712dfc43af06c55f5503295e8460da89e5fcb5)</span></li>
<li>services/fileindexer/fileindexingqueue.h <span style="color: grey">(b017f226b0a209d57bcac7b9fb949d5b17e79cba)</span></li>
<li>services/fileindexer/fileindexingqueue.cpp <span style="color: grey">(2b119255e39fc873db08148291e1638e1a8c510a)</span></li>
<li>services/fileindexer/indexscheduler.h <span style="color: grey">(c6a9e35bed7ce885a8948aec37eccd5d23f7eb5d)</span></li>
<li>services/fileindexer/indexscheduler.cpp <span style="color: grey">(2835eb21274c80738b47f4526c47028d0d9dd06e)</span></li>
</ul>
<p><a href="http://git.reviewboard.kde.org/r/110794/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>