[Kst] Re: CDF datasource debugging ?

[Barth Netterfield]

On July 12, 2004 12:19 pm, Nicolas.Brisset at eurocopter.com wrote:
> On Friday 09 July 2004 21:05, you wrote:
> > > 1) why is seems to be called more than once even though in the first
> > > call the arguments say from the 0th value, for -1 values (which means
> > > all values as you said in an earlier mail)
> >
> >   Oh dear, I think I gave you misinformation.  I'm really sorry about
> > that. The ascii source actually returns 1 sample for n < 0.  Hrm we
> > really need to document this better.
>
> I changed the behavior or readField() to be that of ASCII (if (n < 0) n =
> 1), and it does work better :-) However, at first only 1 point was read,
> which I found out to be linked to the fact that frameCount() always
> returned 1. I started changing that, which exposed another problem: in my
> cdf files, the variables do NOT all have the same frame count. This does
> not seem to have been planned, though... As a result, I am now using the
> maximum value across all variables, and I have to handle this in the
> readField method otherwise I get a lot of extra "0"s under some
> circumstances. Not really great !

The concept of 'frames' is this.
In a given data source is is possible that not all variables have been sampled 
at the same rate.  But it is often the case (and we assume) that:
	i) all variables/vectors in the data source start at the same time and cover 
the same period.
	ii) all sample rates are an integer multiple of some lower rate.  We will 
call this lower rate the 'frame rate'.

eg: BOOMERANG files have 3 sample rates: 
	1) detectors sampled at 60Hz, 
	2) pointing sensors sampled at 10Hz, 
	3) housekeeping sampled at 5Hz.  
So for BOOMERANG, we declare the frame rate to be 5Hz.  The detector fields 
have 12 samples per frame, pointing fields have 2 samples per frame, and 
housekeeping fields have 1 sample per frame.

The datasource needs to figure these things out from the actual file for each 
field (eg, I assume this info will be encoded in CDF files).

so,  the relavent subroutines defined in the datasource (eg, dirfile.h):
	virtual int readField(double *v, const QString &field, int s, int n);
		Read 'n' frames of field 'field' starting at frame 's'.  
		If 'field' refers to a detector, there will be 12*n 
		samples read starting at sample s*12.
		if (nf<1) then read the first sample in frame 's'.
		Since all fields start at the same time, frame 's'
		starts at the same time for all fields.

	virtual int samplesPerFrame(const QString &field);
		for BOOMERANG, returns 12 for detector fields, 
		2 for pointing fields, and 1 for housekeeping fields.

	virtual int frameCount() const;
		same for all fields in the data source.  Number of complete 
		frames in the file.

(*)Note on end time: complete frames are assumed.  So for BOOMERANG one will 
have 12*nf  samples per detector field, 2*nf samples per pointing field and 
nf samples per housekeeping field.  The starting times are assumed to be the 
same for all vectors, and the samples are assumed to be evenly distributed 
through the frame.  If a given field has one sample per frame, it is assumed 
the sample is at the begining of the frame.

> > > 2) how the v table of doubles can be allocated to the right size when
> > > the caller obviously has not way of knowing how many items will be read
> > > (which is why I realloc it
> >
> >    You cannot reallocate it.  That would cause the crashes you're seeing.
>
> I no longer do it :-) But I still produce nice crashes !
> I suspect there is a problem with threading. From the traces in the code,
> frameCount() seems to be called very often, even though update always
> returns ::NoChange. Strange ?
> Furthermore, I originally chose to open the cdf file only in the
> constructor and close it in the destructor because that operation can
> become quite long (especially with large compressed cdf files). But it does
> not seem to be thread-safe to leave the file open when multiple threads are
> trying to access it concurrently !!! I now open and close the file in each
> method and things work better. Some problems remain, though...
>

for dirfiles, I cache the file pointers with no problems.  And only the update 
thread should be accessing the data source anyway, so there shouldn't be 
concurrency problems.

Barth