[Kst] Support for NetCDF4 files: yes or no?

Fowler, Joseph W. joe.fowler at nist.gov
Mon Jan 28 21:51:35 UTC 2013


On Jan 24, 2013, at 1:20 PM, Nicolas Brisset <nicolas.brisset at free.fr<mailto:nicolas.brisset at free.fr>> wrote:

Hi Joseph,

----- Mail original -----
De: "Joseph W. Fowler" <joe.fowler at nist.gov<mailto:joe.fowler at nist.gov>>
À: kst at kde.org<mailto:kst at kde.org>
Envoyé: Jeudi 24 Janvier 2013 02:40:47
Objet: [Kst] Support for NetCDF4 files: yes or no?
The python script below makes simple NetCDF files in each of 4
flavors (provided that you have py-netCDF4 installed, of course).
My kst2 correctly recognizes the two NetCDF3 files as NetCDF, but
it recognizes the two NetCDF4 files as ASCII (which is technically
true, but unhelpful).

I am using the compiled Mac OSX disk image downloaded yesterday from
source forge, kst version 2.0.6 (Revision 1307223), on an OS 10.8
(Mountain Lion) Mac Pro.

My questions are:

1. It seems like standard kst2 doesn't read NetCDF4 files.  Is that
true?
See above, provided that you use a recent and properly built netcdf library
all versions should be recognized.


Hi Nicolas,

Thanks for the answer, and to Peter for confirming how the Mac image was built.

Unfortunately, I think all the example NetCDF files at http://www.unidata.ucar.edu/software/netcdf/examples/files.html are NetCDF-3 files.  (I downloaded 12 of the the 25 examples, and "for F in *.nc; do ncdump -k $F; done" reports that they are all "classic" format, meaning version 3.)

I converted one example file (sgpsondewnpnC1.nc) from version 3 to 4.  I used first a Python script (nc3tonc4-2.7, which is part of the Mac Ports py27-netcdf4 port) and then the example C program nccopy4.c found at UCAR http://www.unidata.ucar.edu/software/netcdf/examples/programs/  In both cases, the ncdump program reports that the resulting file was a valid NetCDF-4 file, but kst identified both as being just ASCII files.

Could the issue be as simple as a test within kst on the file magic, before the NetCDF plugin is even invoked?  Here you can see that the file magic numbers are not the same for the example NetCDF-3 file and the converted-to-version4 file:

$ od -tc sgpsondewnpnC1.nc | head -1
0000000    C   D   F 001  \0  \0  \v 207  \0  \0  \0  \n  \0  \0  \0 001
$ od -tc sgpsondewnpnC1_netcdf4.nc | head -1
0000000  211   H   D   F  \r  \n 032  \n 002  \b  \b  \0  \0  \0  \0  \0

It's just my uneducated guess, but a test on file magic could explain how Peter could link against a modern netcdf4 library, yet the version 4 file format is not recognized by kst.  I have no idea how kst chooses the file reader, but the netcdf plugin source does not seem to test file magic; it just asks the library to open the file with NcFile.  So…probably not a helpful guess, but I thought I'd mention it.


I have plenty of experience with the getdata dirfiles, and I think my best approach now is to figure out how to make them work for me and just forget about NetCDF.  But for the kst maintainers, it might be useful to know that the UCAR example files don't seem to exercise kst's ability to read NetCDF-4.  Not as far as I can tell.


On a separate issue, I wrote:

3. Another thread on this mailing list seemed to state that NetCDF files cannot be used in live (streaming) mode.  True?

False!  I don't want to leave this malicious rumor on your mailing list archives without debunking it.  I'm not sure why this seemed true to me last week, but today I had no trouble reading NetCDF-3 files in streaming mode.  That is, it works provided (1) that I had the NetCDF file writer call the sync() from the NetCDF API to flush data to disk regularly, and (2) that I told the kst data wizard to use Change Detection as the update type.  (I must have missed one of these requirements in my earlier test.)


Anyway, great work, team kst.  It is even more sweet now than it was when I last used it 4 years ago.

Best wishes,
Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kst/attachments/20130128/1f674ed8/attachment.html>


More information about the Kst mailing list