[Kst] Custom datasource plugin readField() weirdness

bug.zilla.vynce at neverbox.com bug.zilla.vynce at neverbox.com
Tue Jun 2 19:02:27 CEST 2009


After much digging through the Kst source, I've determined that this
problem is caused by a bug in Kst 1.7.0. It is unrelated to my custom
data source plugin and definitely affects the ASCII data source plugin
-- and possibly others.

*Root cause:*
In KstRVector::doUpdate(), _numSamples wasn't being updated correctly.
This caused the field to be marked as dirty because _numSamples !=
_size. The data source would be reset and the whole file would be read
from scratch again. This problem would occur right after reading an
incremental number of samples from the end of the file. This bug
causes a major performance hit when reading a large dataset that is
being actively updated. After fixing this bug, large ASCII data files
become a somewhat viable option.

*Description of changes:*
Currently, _numberOfFrames is decremented so that the last frame is
re-read to catch any incomplete data. _numSamples also needs to be
recalculated at this point to keep it in sync.

*How to verify the change:*
1. Add a debug statement to the beginning of
KstDataSource::readField() that prints out the value of s and n:
printf("readField: %30s\tstart: %4i\tnum: %4i\n", field.ascii(), s, n)
2. Get Kst running with a data source that is actively updating.
3. Watch the debug output and make sure that data is only read
incrementally and that the whole file isn't re-read from scratch every
second update.

*Patch:*
Here's a patch against Kst 1.7.0. This fixes the problem for data
sources that only have 1 sample per frame. I didn't consider data
sources that have multiple samples per frame in depth, so it may or
may not work correctly in other cases. It would be worthwhile for
someone that is more familiar with this code to take a closer look.

diff a/kst/src/libkst/kstrvector.cpp b/kst/src/libkst/kstrvector.cpp
--- a/kst/src/libkst/kstrvector.cpp
+++ b/kst/src/libkst/kstrvector.cpp
@@ -625,16 +625,22 @@ KstObject::UpdateType KstRVector::doUpda
       if (!rc) {
         // FIXME: handle failed resize
         abort();
       }
     }

     if (_numberOfFrames > 0) {
       _numberOfFrames--; /* last frame read was only partially read... */
+
+      if (_doSkip) {
+        _numSamples = _numberOfFrames/_skip;
+      } else {
+        _numSamples = (_numberOfFrames-1)*_samplesPerFrame+1;
+      }
     }

     // read the new data from file
     if (startPastEOF) {
       _v[0] = KST::NOPOINT;
       nRead = 1;
     } else if (_file->samplesPerFrame(_field) > 1) {
       assert(newNumberOfFrames - _numberOfFrames - 1 > 0 ||
newNumberOfFrames - _numberOfFrames - 1 == -1 || force);

Michael


More information about the Kst mailing list