[Kst] Custom datasource plugin readField() weirdness

[Andrew Walker]

Hi Michael,

I modified the code in the 1.8.0 branch 
(http://websvn.kde.org/branches/extragear/kde3/graphics/kst/).

The change is a one-line fix shown below:

if (_numberOfFrames > 0) {
          _numberOfFrames--;
}

becomes:

if (_numberOfFrames > 0 && _file->samplesPerFrame(_field) > 1) {
          _numberOfFrames--;
}

Many thanks for drawing our attention to this problem,

Andrew

----- Original Message ----- 
From: <bug.zilla.vynce at neverbox.com>
To: <kst at kde.org>
Sent: Tuesday, June 02, 2009 10:02 AM
Subject: Re: [Kst] Custom datasource plugin readField() weirdness


> After much digging through the Kst source, I've determined that this
> problem is caused by a bug in Kst 1.7.0. It is unrelated to my custom
> data source plugin and definitely affects the ASCII data source plugin
> -- and possibly others.
>
> *Root cause:*
> In KstRVector::doUpdate(), _numSamples wasn't being updated correctly.
> This caused the field to be marked as dirty because _numSamples !=
> _size. The data source would be reset and the whole file would be read
> from scratch again. This problem would occur right after reading an
> incremental number of samples from the end of the file. This bug
> causes a major performance hit when reading a large dataset that is
> being actively updated. After fixing this bug, large ASCII data files
> become a somewhat viable option.
>
> *Description of changes:*
> Currently, _numberOfFrames is decremented so that the last frame is
> re-read to catch any incomplete data. _numSamples also needs to be
> recalculated at this point to keep it in sync.
>
> *How to verify the change:*
> 1. Add a debug statement to the beginning of
> KstDataSource::readField() that prints out the value of s and n:
> printf("readField: %30s\tstart: %4i\tnum: %4i\n", field.ascii(), s, n)
> 2. Get Kst running with a data source that is actively updating.
> 3. Watch the debug output and make sure that data is only read
> incrementally and that the whole file isn't re-read from scratch every
> second update.
>
> *Patch:*
> Here's a patch against Kst 1.7.0. This fixes the problem for data
> sources that only have 1 sample per frame. I didn't consider data
> sources that have multiple samples per frame in depth, so it may or
> may not work correctly in other cases. It would be worthwhile for
> someone that is more familiar with this code to take a closer look.
>
> diff a/kst/src/libkst/kstrvector.cpp b/kst/src/libkst/kstrvector.cpp
> --- a/kst/src/libkst/kstrvector.cpp
> +++ b/kst/src/libkst/kstrvector.cpp
> @@ -625,16 +625,22 @@ KstObject::UpdateType KstRVector::doUpda
>       if (!rc) {
>         // FIXME: handle failed resize
>         abort();
>       }
>     }
>
>     if (_numberOfFrames > 0) {
>       _numberOfFrames--; /* last frame read was only partially read... */
> +
> +      if (_doSkip) {
> +        _numSamples = _numberOfFrames/_skip;
> +      } else {
> +        _numSamples = (_numberOfFrames-1)*_samplesPerFrame+1;
> +      }
>     }
>
>     // read the new data from file
>     if (startPastEOF) {
>       _v[0] = KST::NOPOINT;
>       nRead = 1;
>     } else if (_file->samplesPerFrame(_field) > 1) {
>       assert(newNumberOfFrames - _numberOfFrames - 1 > 0 ||
> newNumberOfFrames - _numberOfFrames - 1 == -1 || force);
>
> Michael
> _______________________________________________
> Kst mailing list
> Kst at kde.org
> https://mail.kde.org/mailman/listinfo/kst
>
> __________________________________________________
> D O T E A S Y - "Join the web hosting revolution!"
>             http://www.doteasy.com