[Kst] Plotting large datasets in real time

[Theodore Kisner]

Hi Barth,

On Tuesday 05 May 2009 10:12:55 am C. Barth Netterfield wrote:
> > It would seem that just watching the data file for changes and
> > only reading the new content at the end of the file would work in most
> > cases and would be much more efficient.
>
> That is what kst does.

This is a timely topic for me.  I have built a plugin (inheriting from 
KstBasicPlugin) which has some computationally expensive operations.  I was 
attempting to make use of the numShift() and numNew() member functions of 
KstVector to know which samples needed to be recomputed.

In tests (with a dirfile datasource and all fields being appended), I found that 
these functions produced unexpected numbers.

If I set up my KstRVector to "count from end", then once the number of samples 
on disk exceeds the size of the vector, I expect numShift() to return the 
number of samples shifted off the start of the fixed-length vector (which also 
equals the number of samples at the end of the vector which have modified 
values).

Basically, numShift() was always returning zero.  Looking in the source for 
KstRVector::doUpdate, I see this:

-------- code ----------
// shift vector if necessary
  if (newStartingFrame < _startingFrame || newStartingFrame >= _startingFrame 
+ _numberOfFrames) { // No useful data around.
    reset();
  } else { // shift stuff rather than re-read
    if (_doSkip) {
      shift = (newStartingFrame - _startingFrame)/_skip;
      _numberOfFrames -= (newStartingFrame - _startingFrame);
      _numSamples = _numberOfFrames/_skip;
    } else {
      shift = _samplesPerFrame*(newStartingFrame - _startingFrame);
      _numberOfFrames -= (newStartingFrame - _startingFrame);
      _numSamples = (_numberOfFrames-1)*_samplesPerFrame+1;
    }
    // FIXME: use memmove()
    for (i = 0; i < _numSamples; i++) {
      _v[i] = _v[i+shift];
    }
  }
-------- code ----------

So in this bit of code, shouldn't the private member "_numShifted" be set to 
the value of "shift"?

Perhaps the bigger question I would love to answer is:

"Under what conditions should the _numNew and _numShifted members be modified, 
and what do these values represent in terms of changes since the last update?"

-Ted