[Kst] [Bug 123699] fails to create INDEX from .fits file binary extention

[Barth Netterfield]

The reason to have a single length per field is to handle asyronously arriving 
real time data in count from end mode (ie, the #1 requirement for kst), to 
make sure that the X vectors and the Y vectors stay synced.  Because of this, 
in fact, currently, fields within a data source are required to have the same 
frame count.  NF is a property of a datasource.

If we can simultaneously solve syncing asyncronous fields and handling 
variable length fields, then that is great.  But the former has to take 
priority over the latter.

The fundamental problem is this:
Imagine that we are in read-from-end mode (reading 20 frames), and go to read 
from a potentially asynchronous data source (eg, a dirfile), and find that 
field X has 1001 frames, and field Y has 1005 frames.  Kst could conclude 
that:
  -X is just behind, and so read 20 frames ending with 1001.
  -There are different length frames, and so read 20 frames from X ending at 
1001, and 20 frames from Y, ending with 1005.

The existence of this discussion indicates that there will be times when you 
want one, and other times when you want the other.

Perhaps the best solution is to let the data source decide - it can know which 
is appropriate.

So: 
-kst learns to read NF on a per field basis, not a per data source basis.

-Data sources that need to are responsible to make sure that NF is the same 
for all fields.  Data sources that don't need to don't (eg: dirfiles can have 
a bool in the format file which tells getdata what to do here)  We keep 
datasource->update() in order for syncronized data sources to determine their 
one and only NF.  non-sycronized data sources don't have to do anything here.  

-INDEX always returns what you ask for, without wondering if the data exists 
for any other fields.


Seem sensible?

The other issue we have to address is marshaled reads, like for NAD or for 
ascii.  This doesn't solve that, but may be related.

cbn

On Saturday 17 June 2006 00:25, Theodore Kisner wrote:
> I sent this to the bug tracking system, but it hasn't shown up for a couple
> hours, so I have revised it a bit and sent it to the list...
>
> On Friday 16 June 2006 14:22, netterfield at astro.utoronto.ca wrote:
> | The problem is significant: A fits file has no concept of frames.  But
> | kst wants to know how many frames a data source has so it knows what it
> | can ask for.  It must be that now the data source generates NF from the
> | size of the first data object in the file, so it would be expected that
> | it would also generate INDEX from the first data object in the file.
> |  Anything else, and we have bigger fish to fry (like over or under
> | reading fields).
>
> Isn't this is the *whole point* of specifying the desired field in the call
> to KstDataSource::frameCount(field)?  In general, fields within a
> datasource are not required to have the same frame count.  If a specific
> datasource wants to require that (e.g. dirfile), then fine, but it is not a
> general requirement.
>
> | I suggest that for the short term, we generate INDEX and NF from the
> | first data object in the file.
>
> why?  why can't the datasource return as many index samples as requested? 
> The datasource should query each "real" field to determine its length.  If
> we cannot support this scenario, then (in my opinion) our datasource model
> is fundamentally broken.  We should never use the length of the INDEX field
> to determine the number of samples/frames.
>
> The index field is just a dummy placeholder when we want to plot data
> vectors versus sample number.  At some level, the INDEX functionality
> should be built into the basecurve so we don't have to deal with it over
> and over again in the datasources.
>
> | For the long term, I think we need to some how break a fits file into
> | sub-data sources each with equal sized fields as Piolib apparently does.
>
> This is not the method I'm using in libfitstools and the fitsgeneral
> datasource.  I think it is better to have the fitsgeneral datasource
> present a fieldlist/matrixlist of all data in a given FITS file.
>
> -Ted
>
>
>
> _______________________________________________
> Kst mailing list
> Kst at kde.org
> https://mail.kde.org/mailman/listinfo/kst