[Kst] Does kst have any support for data that changes, but doesn't grow?

[Barth Netterfield]

Hey Joseph,

Handling this would at least require writing a new data source, since doing 
something like this requires handling locking of the file to avoid race 
conditions.

However, I'm not sure that this will be enough - kst itself, as far as I know, 
optimizes by not reading old data.

On the other hand, I think this would be a cool feature.  Let me think some 
more.

cbn

On December 11, 2013 08:15:42 PM Fowler, Joseph W. wrote:

> Dear kst team:
> 
> I am working on an application where the data set is generated by x-ray
> sensors called TES microcalorimeters. The raw data are uniformly sampled
> time series, like in a CMB experiment, but with one key difference: most of
> our interest is in the random Poisson-distributed "events" that correspond
> to an x-ray pulse arriving.
> 
> I want to convince our team to shift our real-time plotting functions to
> kst. It will be better in a hundred ways, plus it would remove a serious
> code maintenance burden if we no longer had to roll our own plots.
> 
> One big obstacle is that I can't figure out how to use kst for the data
> acquisition activity we currently use the most: plotting the most recent
> fixed-length pulse record from a given calorimeter. (Think: oscilloscope
> with triggering logic.) We store our data in a non-standard format, so my
> plan was to start generating an auxiliary, temporary fixed-size dirfile
> field holding only the latest pulse record. At each trigger, this file gets
> overwritten. I've tried this concept. The result is that I can plot the
> dirfile once, but GetData is smart enough not to re-read the dirfile when
> its contents change. (It does update if--but only if--I click the Reload
> All Data Sources button.)
> 
> My question #1 for you all: Is there any way to convince kst2 (or GetData)
> that it needs to update a certain data source whose size has not grown?
> Maybe a clever use of cur-files?
> 
> I can see a couple of workarounds, but they have their own problems.  I
> could just copy all the pulse records to a dirfile and let it grow in the
> normal way, but this would be a complete duplication of our already large
> dataset.
> 
> Or I could write a kst datasource plugin for our proprietary data files.
> (The website says to mail this list for help getting started on a plugin.
> Question #2: Can you help? Is it not so hard? I have plenty of C++
> experience.) Our data file format is not dissimilar to a one-field dirfile,
> but with timestamps interspersed and with a horrible ASCII header tacked on
> the head.
> 
> Or I could just switch our system to writing dirfiles. This might be the
> best solution, but you can probably imagine there's some institutional
> inertia resisting this change.
> 
> Thanks, gang.
> Joe Fowler
> NIST Boulder Labs
> 
> 
> System details: I am using Mac OS 10.8 with the latest kst2 binary from
> sourceforge (2.0.7, though I also tried 2.0.6), and the data are being
> generated in Python through GetData 0.8.5. In our lab environment, we will
> be running on an Ubuntu Linux system instead.

-- 
C. Barth Netterfield
416-845-0946