[Kst] Wide Line ASCII Performance

[Barth Netterfield]

I created an ASCII file with 6 character wide fixed width columns.
In all cases, there was 1E6 samples in the file.

The Channel selector in the data wizard showed n^2 behavior.
With 10000 columns, it took 30s to select all but one field using
wildcards, and with 3333 columns it took 3s - ie, n^2 behavior.

This is because selecting or unselecting a single item in the string
selection widget goes as the length of the list.  However selecting
or unselecting all of them is very fast.  So I did the optimization
of selecting them all, then unselecting the unwanted ones for the case
where you are selecting most of the fields.

Reading all of the channels into one plot in white-space separated mode
was unbearably slow with 1000 x 1000.  This makes sense: the algorithm
for reading all channels for white-space separated mode is n^2 in the
number of channels.

Fixed width mode was faster, but still n^2.  Valgrind claims that
this is because, even in fixed with mode, the entire row is read for
each sample. Since, with these really long lines, a single line is longer
than a disk block, reading just what is needed could be faster.


-- 
C. Barth Netterfield
University of Toronto
416-845-0946
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kst/attachments/20120615/b35efe89/attachment.html>