[Kst] Getdata library

[Adam D. Hincks]

Dear KSTers,

This email is about the dirfile datasource used by KST; specifically, the 
C library called getdata.

I work on the ACT project which uses dirfiles for its data storage.  We 
use KST for viewing data, and also are starting to use getdata for our 
analysis pipeline.  We prefer to use getdata (rather than write our 
own) because it is "tested and true" and compatible with KST.

There are a some features not currently in the library that we have 
decided we need, however, which I outline below.  I am willing and able to 
make these additions but of course would need to coordinate with whomever 
is in charge of the source.

1.  Byte-swapping.  I have examined the source and it appears that there 
are no provisions to do byte-swapping on differently-ended machines. 
Right now we use all little-endian machines so this has not been an issue, 
but we might do analysis in the future on architectures with different 
byte orders.  (Our files will all be recorded little-endian.)  The 
most convenient thing would be if getdata did byte-swapping somewhere in 
the DoIfRaw() function.

2.  A "truthful" version of GetNFrames().  GetNFrames() computes the 
length of one of the files in the dirfile and uses that to compute the 
number of frames.  Then it subtracts 2.  This is good when reading live 
data as it avoids race conditions.  But with finished data sets we will 
always be losing 2 seconds of data (our frames are ~1 second long).  We 
would like a GetTrueNFrames().

3.  "Ragged ends".  This is a bit more complicated to explain, but here 
goes:

Because of the nature of the ACT's data acquisition, we are unable to 
guarantee that dirfiles will have a round number of frames.  On our 
faster channels there will inevitably be a spill-over after the last frame 
of a number of samples.  This number of samples will be the same for all 
data at the same rate.  So, for example, if we have fields at 10 
samples per frame (SPF), 5 SPF and 1 SPF, we could have after the last frame:

    10 SPF fields:  6 extra samples
    5 SPF fields:   3 extra samples
    1 SPF fields:   0 extra samples

The following would also be possible:

    10 SPF fields:  3 extra samples
    5 SPF fields:   2 extra samples
    1 SPF fields:   1 extra sample

So we need a function to supplement GetNFrames().  This function would 
return the number of extra samples after the last frame for a given field. 
We can guarantee that data with the same SPF have the same number of extra 
samples.


As I said before, I am able to make these changes.  But we are reluctant 
to have our own version of getdata and would prefer that the changes be 
incorporated into the KST repository.  Any help that you can provide would 
be greatly appreciated and I can cooperate in any way to make the process 
as expeditious as possible.

Thanks,

Adam