[Kst] [Bug 123699] fails to create INDEX from .fits file binary extention
netterfield at astro.utoronto.ca
Sat Jun 17 06:52:59 CEST 2006
The reason to have a single length per field is to handle asyronously arriving
real time data in count from end mode (ie, the #1 requirement for kst), to
make sure that the X vectors and the Y vectors stay synced. Because of this,
in fact, currently, fields within a data source are required to have the same
frame count. NF is a property of a datasource.
If we can simultaneously solve syncing asyncronous fields and handling
variable length fields, then that is great. But the former has to take
priority over the latter.
The fundamental problem is this:
Imagine that we are in read-from-end mode (reading 20 frames), and go to read
from a potentially asynchronous data source (eg, a dirfile), and find that
field X has 1001 frames, and field Y has 1005 frames. Kst could conclude
-X is just behind, and so read 20 frames ending with 1001.
-There are different length frames, and so read 20 frames from X ending at
1001, and 20 frames from Y, ending with 1005.
The existence of this discussion indicates that there will be times when you
want one, and other times when you want the other.
Perhaps the best solution is to let the data source decide - it can know which
-kst learns to read NF on a per field basis, not a per data source basis.
-Data sources that need to are responsible to make sure that NF is the same
for all fields. Data sources that don't need to don't (eg: dirfiles can have
a bool in the format file which tells getdata what to do here) We keep
datasource->update() in order for syncronized data sources to determine their
one and only NF. non-sycronized data sources don't have to do anything here.
-INDEX always returns what you ask for, without wondering if the data exists
for any other fields.
The other issue we have to address is marshaled reads, like for NAD or for
ascii. This doesn't solve that, but may be related.
On Saturday 17 June 2006 00:25, Theodore Kisner wrote:
> I sent this to the bug tracking system, but it hasn't shown up for a couple
> hours, so I have revised it a bit and sent it to the list...
> On Friday 16 June 2006 14:22, netterfield at astro.utoronto.ca wrote:
> | The problem is significant: A fits file has no concept of frames. But
> | kst wants to know how many frames a data source has so it knows what it
> | can ask for. It must be that now the data source generates NF from the
> | size of the first data object in the file, so it would be expected that
> | it would also generate INDEX from the first data object in the file.
> | Anything else, and we have bigger fish to fry (like over or under
> | reading fields).
> Isn't this is the *whole point* of specifying the desired field in the call
> to KstDataSource::frameCount(field)? In general, fields within a
> datasource are not required to have the same frame count. If a specific
> datasource wants to require that (e.g. dirfile), then fine, but it is not a
> general requirement.
> | I suggest that for the short term, we generate INDEX and NF from the
> | first data object in the file.
> why? why can't the datasource return as many index samples as requested?
> The datasource should query each "real" field to determine its length. If
> we cannot support this scenario, then (in my opinion) our datasource model
> is fundamentally broken. We should never use the length of the INDEX field
> to determine the number of samples/frames.
> The index field is just a dummy placeholder when we want to plot data
> vectors versus sample number. At some level, the INDEX functionality
> should be built into the basecurve so we don't have to deal with it over
> and over again in the datasources.
> | For the long term, I think we need to some how break a fits file into
> | sub-data sources each with equal sized fields as Piolib apparently does.
> This is not the method I'm using in libfitstools and the fitsgeneral
> datasource. I think it is better to have the fitsgeneral datasource
> present a fieldlist/matrixlist of all data in a given FITS file.
> Kst mailing list
> Kst at kde.org
More information about the Kst