[Kst] Re: CDF datasource debugging ?
Nicolas.Brisset at eurocopter.com
Nicolas.Brisset at eurocopter.com
Fri Jul 16 12:16:00 CEST 2004
On Thursday 15 July 2004 22:21, George Staikos wrote:
> I have the CDF plugin working here now, with a sample database. So far it
> looks to be ok - it just needs some bug fixes.
Reading from CDF is not so complicated that talented and experimented
programmers like you could not handle this within a reasonable time :-)
> I don't fully understand how CDF works though, so I can't comment too much
> on how it fits into Kst yet.
CDF is hard to understand at the beginning due to the specific terminology
they use (and if you look at the source code of their curses-based tools, be
prepared : this is not nice well structured C++ !! By the way, I found kscope
very handy to examine this code: this piece of code is really a great tool
for such cases where you need to get to grips with some external code).
Here is a CDF quickstart guide (the docs included in the distribution explain
that fairly well):
- in CDF, variables can have different dimensionalities, which means that for
each sample (called "record") you may have a scalar (dimension 0), a vector
(dimension 1), a matrix (dimension 2), etc... In my code I deliberately chose
to use only 0-dimensional variables. I suppose this is sensible for a 2D
- the number of records may differ from one variable to the next
- the storage uses a very compact binary platform-independent (no
big/little-endian problems) format
- there can be "attributes" attached globally to the file or to each variable
with arbitrary metadata in them
- for historical reasons, there can be 2 types of variables in a cdf file,
called rVars and zVars (zVars superseding rVars). In the files I have, there
are only rVars but I tried to use the same macros they use in the official
tools to handle both transparently (rVars may be lagging behind right now,
- to look at the contents of a cdf file, the handiest way is to use the
"cdfexport" tool. A bit rough for people used to nice Qt/KDE widgets :-) but
still pretty efficient when you know the basics:
1) load the cdf file with "cdfexport file" when the file is called file.cdf,
which is not really intuitive !
2) you will get a list of variables and type/dimension information. From
there, you can look at the options in the "popup" brought by Ctrl+A,
especially 'list to file...' which will create an ASCII file. You can select
which variables to export first, which you should do in case you have vectors
or matrices (i.e. vars with dimensions > 0) otherwise the ASCII file will be
hard to use and get very big. Beware, the sample I sent includes some such
vars towards the end of the list !
Now, one *IMPORTANT POINT* regarding the organization in "blocks" I mentioned
in my post yesterday, which Barth suggested to encode into kst: this is NOT a
CDF standard. It just happens that the files I work with use this
_convention_ between the people who create them and us. For that reason I
would not recommend to hard-code the notion of groups into the kst
datasource, which in my view should be generic and work with any cdf file out
there. That's why I prefer to indicate a reading problem when asked for a
non-existing value. It is however not clear to me what the best way to
achieve this is: return a lot of NaNs in the v buffer (the CDFlib will return
0 when a record beyond the last available for a variable is queried, but we
can store NaNs in v), or stop at the last available data and use the returned
int to indicate the number of values read (possibly lower than the argument
n, or 0/-1 if none) ? The second looks nicer to me, but I do not know the
internals of kst.
> In piolib we have a gross hack that needs to be fixed. Actually this
> is high on my TODO list. Please don't look at that code. :-)
You are tempting me :-)
Thanks to all for your kind help. Feel free to ask for more information in
case something is still unclear.
More information about the Kst