Great work. Huge speedup!<div><div><br><div class="gmail_quote">On Mon, Jan 24, 2011 at 2:08 PM, Peter Kuemmel <span dir="ltr"><<a href="mailto:syntheticpp@gmx.net">syntheticpp@gmx.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Attached the benchmark results with values for Linux.<br>
It was a 310 MB gyrodata file, and I always have loaded<br>
column three only.<br></blockquote><div><br></div><div><meta http-equiv="content-type" content="text/html; charset=utf-8"><div>Can you explain what the various cases in the table are? Are these all fixed width columns, or are they variable width? Or did you show both cases?</div>
<div> </div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
I found that the atof function which we already use on Windows<br>
by default is also faster on Linux. Therefore I think we should<br>
also use it on Linux, especially our numbers aren't that<br>
complicate to parse.<br></blockquote><div><br></div><div>Good. Can it parse scientific notation? What does it do about '.' vs ',' (I haven't looked...) </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Additionally we should change the default comment delimiter as<br>
Nicolas already suggested. Then a normal user who often uses the<br>
defaults settings would see a speed on Linux by factor 5 by<br>
simply updating to Kst 2.0.3 on windows it is about factor 2-3.<br></blockquote><div><br></div><div>Is the proposal to have '#' as the default comment delimiter? The speedup is from only having one?</div><div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
But speedup is only for the pure data loading.<br>
The internalDataSourceUpdate is still very slow, counting the<br>
rows and looking for comments is now slower than reading the data!<br>
This makes no sense so we should also optimize internalDataSourceUpdate<br>
before we release 2.0.3.<br></blockquote><div><br></div><div>Yes. It should be far faster.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Do we support comments which are anywhere in the data or is it<br>
enough to only support complete lines as comments, lines which<br>
starts with the comment delimiter?<br></blockquote><div><br></div><div>Well... we should probably support white space before a comment at the begining of a line, but a correctly formed ascii file will have the same number of columns for every line, so if there is a comment later in a line other than the first line it will either be after the last column, or will be a syntax error in the file.</div>
<div><br></div><div>So: check for comments characters anywhere in the first line when chosing the number of columns.</div><div>After that, only check at the begining of the line (up to the first non-white space character).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Peter<br>
<font color="#888888">--<br>
Neu: GMX De-Mail - Einfach wie E-Mail, sicher wie ein Brief!<br>
Jetzt De-Mail-Adresse reservieren: <a href="http://portal.gmx.net/de/go/demail" target="_blank">http://portal.gmx.net/de/go/demail</a><br>
</font><br>_______________________________________________<br>
Kst mailing list<br>
<a href="mailto:Kst@kde.org">Kst@kde.org</a><br>
<a href="https://mail.kde.org/mailman/listinfo/kst" target="_blank">https://mail.kde.org/mailman/listinfo/kst</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br><span style="font-family:arial, sans-serif;font-size:13px;border-collapse:collapse;color:rgb(136, 136, 136)">C. Barth Netterfield<br>University of Toronto<br>416-845-0946</span><div>
<span style="font-family:arial, sans-serif;font-size:13px;border-collapse:collapse;color:rgb(136, 136, 136)"><br></span></div><br>
</div></div>