[Kst] Getdata library

[Adam D. Hincks]

> On Wed, Aug 22, 2007 at 11:39:25AM -0400, Adam D. Hincks wrote:
>> 1.  Byte-swapping.  I have examined the source and it appears that there
>> are no provisions to do byte-swapping on differently-ended machines.
>> Right now we use all little-endian machines so this has not been an issue,
>> but we might do analysis in the future on architectures with different
>> byte orders.  (Our files will all be recorded little-endian.)  The
>> most convenient thing would be if getdata did byte-swapping somewhere in
>> the DoIfRaw() function.
>
> I'm somewhat surprised it doesn't already do this.  AFAIK, kst is already
> run on big endian systems.  Dirfiles are explicitly supposed to be stored
> little endian.  How does kst currently deal with dirfiles in on big-endian
> machines?
>
>> 2.  A "truthful" version of GetNFrames().  GetNFrames() computes the
>> length of one of the files in the dirfile and uses that to compute the
>> number of frames.  Then it subtracts 2.  This is good when reading live
>> data as it avoids race conditions.  But with finished data sets we will
>> always be losing 2 seconds of data (our frames are ~1 second long).  We
>> would like a GetTrueNFrames().
>
> This is a bug.  If applications using getdata (viz. kst) want to subtract
> two from the number of available frames, they need to do that themselves.
> It shouldn't happen in the library.

It is a pretty deliberate bug, it looks like.  Here's the source:

   nf -= 2;
   if (nf < 0)
     nf = 0;
   return(nf);

Who is in charge of the library -- shall I consider this bug "submitted"?

>> 3.  "Ragged ends".  This is a bit more complicated to explain, but here
>> goes:
>>
>> Because of the nature of the ACT's data acquisition, we are unable to
>> guarantee that dirfiles will have a round number of frames.  On our
>> faster channels there will inevitably be a spill-over after the last frame
>> of a number of samples.  This number of samples will be the same for all
>> data at the same rate.  So, for example, if we have fields at 10
>> samples per frame (SPF), 5 SPF and 1 SPF, we could have after the last
>> frame:
>>
>>    10 SPF fields:  6 extra samples
>>    5 SPF fields:   3 extra samples
>>    1 SPF fields:   0 extra samples
>>
>> The following would also be possible:
>>
>>    10 SPF fields:  3 extra samples
>>    5 SPF fields:   2 extra samples
>>    1 SPF fields:   1 extra sample
>>
>> So we need a function to supplement GetNFrames().  This function would
>> return the number of extra samples after the last frame for a given field.
>> We can guarantee that data with the same SPF have the same number of extra
>> samples.
>
> Are you asking for a GetNSamples, which would take a field name?

Essentially yes.  Something like:

int GetNSamples(const char *filename_in, const char *field, int frame_num,
                 int *error_code);

But recall, these are _extra_ samples after the last frame.  So it would 
have to be able handle something like this:

n_extra = GetNSamples("/data/dirfile",
                       "My_Field",
                       GetNFrames("/data/dirfile", &err, "My_Field"),
                       &err2);

An alternative would be something that always checks past the last frame 
and doesn't have frame_num as an input:

int GetExtraSamples(const char *filename_in, const char *field, int
                     *error_code);

Thanks,

Adam


> Cheers,
> -dvw
> -- 
> Don Wiebe                                       dwiebe at physics.utoronto.ca
> Department of Physics
> University of Toronto                           Office: ES4150
> 60 St. George St.                               Tele: +1-416-946-0946
> Toronto ON  M5S 1A7                             Lab:  +1-416-946-0516
>