[Kst] AsciiSource: new defaults, Kst's atof
Peter Kuemmel
syntheticpp at gmx.net
Mon Jan 24 20:08:53 CET 2011
Attached the benchmark results with values for Linux.
It was a 310 MB gyrodata file, and I always have loaded
column three only.
I found that the atof function which we already use on Windows
by default is also faster on Linux. Therefore I think we should
also use it on Linux, especially our numbers aren't that
complicate to parse.
Additionally we should change the default comment delimiter as
Nicolas already suggested. Then a normal user who often uses the
defaults settings would see a speed on Linux by factor 5 by
simply updating to Kst 2.0.3 on windows it is about factor 2-3.
But speedup is only for the pure data loading.
The internalDataSourceUpdate is still very slow, counting the
rows and looking for comments is now slower than reading the data!
This makes no sense so we should also optimize internalDataSourceUpdate
before we release 2.0.3.
Do we support comments which are anywhere in the data or is it
enough to only support complete lines as comments, lines which
starts with the comment delimiter?
Peter
--
Neu: GMX De-Mail - Einfach wie E-Mail, sicher wie ein Brief!
Jetzt De-Mail-Adresse reservieren: http://portal.gmx.net/de/go/demail
-------------- next part --------------
Measured times on Windows:
Delimiters : time in seconds
msvc10 (mingw 4.4/Linux/L&atof)
comment , column func ptr inlined 2.0.2
-----------------------------------------------------------------------------
No comment, custom: one space : 4.8 3.7 (3.6/5.4/2.2) 14.4 (14.4/14.5)
# comment, custom: one space : 5.7 4.2 (3.8/5.7/2.4) 14.7 (14.9/15.4)
No comment, whitespace : 6.2 4.0 (3.6/5.6/2.3) 10.0 (8.8/10.1)
# comment, whitespace : 6.9 4.2 (4.3/5.6/2.4) 10.5 (9.3/10.8)
Defaults: Linux Win
2.0.2 #/c!; white sys-atof : 12.1 ~10
2.0.3 # white kst-atof : 2.4 4.3
Factor ~5.0 ~2.3
More information about the Kst
mailing list