<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="http://git.reviewboard.kde.org/r/102391/">http://git.reviewboard.kde.org/r/102391/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<p style="margin-top: 0;">On August 21st, 2011, 10:07 a.m., <b>David Faure</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Thanks Peter and Miroslav. The analysis looks correct, the pre-read part of the patch looks good. I'm just wondering about using Unbuffered. If someone installs a mimetype definition with multiple rules trying to match some bytes after the 2K limit, then all this seeking-and-reading back and forth will be very slow, in unbuffered mode (since neither cache will be used).</pre>
</blockquote>
<p>On August 22nd, 2011, 8:12 p.m., <b>Miroslav Ľos</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I find Unbuffered causing slowness improbable. From what I've seen in uses of KMimeType::findBy\w*Content in kdelibs, they all (e.g. ftp* and http kioslaves) rather provide their own buffer in a QByteArray rather than a QIODevice; most just provide the path (which is only opened using QFile if it is_local_file).
All QFile's buffering is implemented in the QIODevice superclass, it adds Unbuffered in open() to its openMode for its fileEngine() backend. Thus, no buffering is propagated down. The unnecessary 16K read did hit several more EIO's on the broken CD I have, but it is just cosmetic I guess.
Nonetheless, I find most uses of these functions ever will be through QFile or QBuffer and any client passing another QIODevice may open it itself. Maybe a note in the documentation could help them do it if necessary.
Finally, I wonder if buffering makes a difference as we are only using a few small block reads, not many getChar()'s. Then again, the documentation to QTcpSocket says they cannot be opened (i.e. ignore I guess) Unbuffered, which is what any other QIODevices may do as well.
* ftp passes a 1K-capped buffer (kioslave/ftp/ftp.cpp:2471). That may be insufficient for the single rule on my system that needed 1029 bytes.</pre>
</blockquote>
<p>On September 2nd, 2011, 7:33 a.m., <b>David Faure</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: 10px;">
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">(Pasting here, after realizing you probably didn't see my reply on kde-core-devel)
> I find Unbuffered causing slowness improbable. From what I've seen in uses
> of KMimeType::findBy\w*Content in kdelibs, they all (e.g. ftp* and http
> kioslaves) rather provide their own buffer in a QByteArray rather than a
> QIODevice; most just provide the path (which is only opened using QFile if
> it is_local_file).
The most common case for mimetype determination is from the file manager
listing local files, in which case findByUrl will use a QFile for content-
determination.
Ah, and BTW I have just found a magic rule that needs 4K of data:
vmware-player.xml says:
<match type="string" value='config.version = "' offset="0:4096"/>
> All QFile's buffering is implemented in the QIODevice superclass, it adds
> Unbuffered in open() to its openMode for its fileEngine() backend. Thus, no
> buffering is propagated down.
I'm not sure what you mean there.
> The unnecessary 16K read did hit several more
> EIO's on the broken CD I have, but it is just cosmetic I guess.
Yes I'm not sure it's worth optimizing for this special case.
> Nonetheless, I find most uses of these functions ever will be through QFile
> or QBuffer
Sure. It's QFile that I have in mind here, when I say that seeking back and
forth will be slow, in unbuffered mode.
> Finally, I wonder if buffering makes a difference as we are only using a few
> small block reads, not many getChar()'s.
It's about seeking and reading, vs just having the data in memory.
> * ftp passes a 1K-capped buffer (kioslave/ftp/ftp.cpp:2471). That may be
> insufficient for the single rule on my system that needed 1029 bytes.
Right. Small bug, but a corner case (only matters if extension unknown).
</pre>
</blockquote>
</blockquote>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Miroslav, can you clarify why Unbuffered is beneficial? I did not understand it from the ongoing discussion, and I have the feeling David did not either :)</pre>
<br />
<p>- Christoph</p>
<br />
<p>On August 20th, 2011, 5:21 p.m., Peter Penz wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('http://git.reviewboard.kde.org/media/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for kdelibs and David Faure.</div>
<div>By Peter Penz.</div>
<p style="color: grey;"><i>Updated Aug. 20, 2011, 5:21 p.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">If KMimeTypeRepository::findFromContent() tries to determine MIME from a file that cannot be read, such as on a corrupted optical disc, a read attempt is made in KMimeMagicMatch::match() for every available rule, resulting in UI hangs (e.g. file dialogs, dolphin) for tens of minutes (see https://bugs.kde.org/show_bug.cgi?id=280446 for more details).
I've submitted this patch here on behalf of Miroslav ?os, who has submitted the bug-report and also has written the patch.
</pre>
</td>
</tr>
</table>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Bugs: </b>
<a href="http://bugs.kde.org/show_bug.cgi?id=280446">280446</a>
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>kdecore/services/kmimetype.cpp <span style="color: grey">(955bf62)</span></li>
<li>kdecore/services/kmimetyperepository.cpp <span style="color: grey">(6ff3d16)</span></li>
</ul>
<p><a href="http://git.reviewboard.kde.org/r/102391/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>