<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="http://git.reviewboard.kde.org/r/102356/">http://git.reviewboard.kde.org/r/102356/</a>
</td>
</tr>
</table>
<br />
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('http://git.reviewboard.kde.org/media/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Nepomuk and Strigi.</div>
<div>By Sebastian Trueg.</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">The problem is simple: when indexing the text from the cells in ods documents the analyser currently simply calls addText for each cell. This results in the backend (indexer) to concatenate all those strings which in turn means invalid tokenization for full-text-search.
xmlindexer and rdfindexer work around this by adding a newline after each block of text added via addText. This, however, is clearly wrong since 1. the API does not suggest that, 2. all other plugins - most prominently the text analyser - do not strip away any line feeds, and 3. it would significantly lower the power of the API to provide a line-based interface.
Thus, the only correct approach is to take care of proper text handling in the analysers. In this case the simplest way is to add a space after each token.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Indexing an ods results in proper tokenization for cell content. Indexing an odt results in the last word of a line not being concatenated with the first word of the next line.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>lib/helperanalyzers/odfcontenthelperanalyzer.h <span style="color: grey">(4fbfd45)</span></li>
<li>lib/helperanalyzers/odfcontenthelperanalyzer.cpp <span style="color: grey">(d2a0a72)</span></li>
</ul>
<p><a href="http://git.reviewboard.kde.org/r/102356/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>