MAN page from openSUSE Tumbleweed netcdf-openmpi-


Updated: $Date: 2010/04/29 16:38:55 $


ncgen - From a CDL file generate a netCDF-3 file, a netCDF-4 file or a C program 


ncgen[-b][-c][-f][-k format_name][-format_code][-l output language][-n][-o netcdf_filename][-x][input_file]


ncgen generates either a netCDF-3 (i.e. classic) binary .nc file,a netCDF-4 (i.e. enhanced) binary .nc fileor a file in some source language that when executed willconstruct the corresponding binary .nc file.The input to ncgen is a description of a netCDFfile in a small language known as CDL (network Common Data form Language),described below. Input is read from standard input if no input_fileis specified.If no options are specified in invoking ncgen, it merely checks thesyntax of the input CDL file, producing error messages forany violations of CDL syntax. Other options can be used, for example,to create the corresponding netCDF file,or to generate a C program that uses the netCDF Cinterface to create the netCDF file.

Note that this version of ncgen was originally called ncgen4.The older ncgen program has been renamed to ncgen3.

ncgen may be used with the companion program ncdump to performsome simple operations on netCDF files. For example, to rename a dimensionin a netCDF file, use ncdump to get a CDL version of the netCDF file,edit the CDL file to change the name of the dimensions, and use ncgento generate the corresponding netCDF file from the edited CDL file. 


Create a (binary) netCDF file. If the -o option is absent, adefault file name will be constructed from the basename of the CDLfile, with any suffix replaced by the `.nc' extension. If afile already exists with the specified name, it will be overwritten.
GenerateCsource code that will create a netCDF filematching the netCDF specification. The C source code is written tostandard output; equivalent to -lc.
GenerateFORTRAN 77source code that will create a netCDF filematching the netCDF specification.The source code is written tostandard output; equivalent to -lf77.
-o netcdf_file
Name of the file to pass to calls to "nc_create()".If this option is specified it implies(in the absence of any explicit -l flag) the "-b" option.This option is necessary because netCDF filescannot be written directly to standard output, since standard output is notseekable.
-k format_name
The -k flag specifies the format of the file to be created and, by inference,the data model accepted by ncgen (i.e. netcdf-3 (classic) versusnetcdf-4 vs netcdf-5). As a shortcut, a numeric format_code may be specified instead.The possible format_name values for the -k option are:
'classic' or 'nc3' => netCDF classic format
'64-bit offset' or 'nc6' => netCDF 64-bit format
'64-bit data or 'nc5' => netCDF-5 (64-bit data) format
'netCDF-4' 0r 'nc4' => netCDF-4 format (enhanced data model)
'netCDF-4 classic model' or 'nc7' => netCDF-4 classic model format
Accepted format_number arguments, just shortcuts for format_names, are:
3 => netcdf classic format
5 => netcdf 5 format
6 => netCDF 64-bit format
4 => netCDF-4 format (enhanced data model)
7 => netCDF-4 classic model format
The numeric code "7" is used because "7=3+4", a mnemonic for the formatthat uses the netCDF-3 data model for compatibility with the netCDF-4storage format for performance. Credit is due to NCO for use of thesenumeric codes instead of the old and confusing format numbers.

Note: The old version format numbers '1', '2', '3', '4', equivalentto the format names 'nc3', 'nc6', 'nc4', or 'nc7' respectively, arealso still accepted but deprecated, due to easy confusion betweenformat numbers and format names. Various old format name aliases arealso accepted but deprecated, e.g. 'hdf5', 'enhanced-nc3', etc.Also, note that -v is accepted to mean the same thing as-k for backward compatibility.

Don't initialize data with fill values. This can speed up creation oflarge netCDF files greatly, but later attempts to read unwritten datafrom the generated file will not be easily detectable.
-l output_language
The -l flag specifies the output language to usewhen generating source code that will create or define a netCDF filematching the netCDF specification.The output is written to standard output.The currently supported languages have the following flags.
c|C' => C language output.
f77|fortran77' => FORTRAN 77 language output
; note that currently only the classic model is supported.
j|java' => (experimental) Java language output
; targets the existing Unidata Java interface, which means thatonly the classic model is supported.

Choosing the output format

The choice of output format is determined by three flags.
-k flag.
_Format attribute (see below).
Occurrence of CDF-5 (64-bit data) or
netcdf-4 constructs in the input CDL."The term "netCDF-4 constructs" meansconstructs from the enhanced data model,not just special performance-related attributes such as
 _ChunkSizes, _DeflateLevel, _Endianness, etc.The term "CDF-5 constructs" meansextended unsigned integer types allowed in the 64-bit data model.

Note that there is an ambiguity between the netCDF-4 caseand the CDF-5 case is only an unsigned type is seen in the input.

The rules are as follows, in order of application.

If either Fortran or Java output is specified,then -k flag value of 1 (classic model) will be used.Conflicts with the use of enhanced constructsin the CDL will report an error.
If both the -k flag and _Format attribute are specified,the _Format flag will be ignored.If no -k flag is specified, and a _Format attribute valueis specified, then the -k flag valuewill be set to that of the _Format attribute.Otherwise the -k flag is undefined.
If the -k option is defined and is consistent with the CDL,ncgen will output a file in the requested form,else an error will be reported.
If the -k flag is undefined,and if there are CDF-5 constructs, only, in the CDL,a -k flag value of 5 (64-bit data model) will be used.If there are true netCDF-4 constructs in the CDL,a -k flag value of 3 (enhanced model) will be used.
If special performance-related attributes are specified in the CDL,a -k flag value of 4 (netCDF-4 classic model) will be used.
Otherwise ncgen will set the -k flag to 1 (classic model).


Check the syntax of the CDL file `foo.cdl':

ncgen foo.cdl

From the CDL file `foo.cdl', generate an equivalent binary netCDF filenamed `':

ncgen -o foo.cdl

From the CDL file `foo.cdl', generate a C program containing thenetCDF function invocations necessary to create an equivalent binary netCDFfile named `':

ncgen -lc foo.cdl >x.c




CDL Syntax Overview

Below is an example of CDL syntax, describing a netCDF file with severalnamed dimensions (lat, lon, and time), variables (Z, t, p, rh, lat, lon,time), variable attributes (units, long_name, valid_range, _FillValue),and some data. CDL keywords are in boldface. (This example is intended toillustrate the syntax; a real CDL file would have a more complete set ofattributes so that the data would be more completely self-describing.)

netcdf foo {  // an example netCDF specification in CDLtypes:    ubyte enum enum_t {Clear = 0, Cumulonimbus = 1, Stratus = 2};    opaque(11) opaque_t;    int(*) vlen_t;dimensions:        lat = 10, lon = 5, time = unlimited ;variables:        long    lat(lat), lon(lon), time(time);        float   Z(time,lat,lon), t(time,lat,lon);        double  p(time,lat,lon);        long    rh(time,lat,lon);        string  country(time,lat,lon);        ubyte   tag;        // variable attributes        lat:long_name = "latitude";        lat:units = "degrees_north";        lon:long_name = "longitude";        lon:units = "degrees_east";        time:units = "seconds since 1992-1-1 00:00:00";        // typed variable attributes        string Z:units = "geopotential meters";        float Z:valid_range = 0., 5000.;        double p:_FillValue = -9999.;        long rh:_FillValue = -1;        vlen_t :globalatt = {17, 18, 19};data:        lat   = 0, 10, 20, 30, 40, 50, 60, 70, 80, 90;        lon   = -140, -118, -96, -84, -52;group: g {types:    compound cmpd_t { vlen_t f1; enum_t f2;};} // group ggroup: h {variables:        /g/cmpd_t  compoundvar;data:        compoundvar = { {3,4,5}, enum_t.Stratus } ;} // group h}

All CDL statements are terminated by a semicolon. Spaces, tabs,and newlines can be used freely for readability.Comments may follow the characters `//' on any line.

A CDL description consists of five optional parts:types,dimensions,variables,data,beginning with the keyword`types:',`dimensions:',`variables:',and`data:',respectively.Note several things:(1) the keyword includes the trailing colon, so there must not be any space before the colon character,and (2) the keywords are required to be lower case.

The variables: section may contain variable declarationsand attribute assignments.All sections may contain global attribute assignments.

In addition, after the data: section, the usermay define a series of groups (see the example above).Groups themselves can contain types, dimensions, variables,data, and other (nested) groups.

The netCDF types: section declares the user defined types.These may be constructed using any of the following types:enum, vlen, opaque, or compound.

A netCDF dimension is used to define the shape of one or more of themultidimensional variables contained in the netCDF file. A netCDFdimension has a name and a size. A dimensioncan have the unlimited size, which means a variable using thisdimension can grow to any length in that dimension.

A variable represents a multidimensional array of values of thesame type. A variable has a name, a data type, and a shape describedby its list of dimensions. Each variable may also have associatedattributes (see below) as well as data values. The name, datatype, and shape of a variable are specified by its declaration in thevariable section of a CDL description. A variable may have the samename as a dimension; by convention such a variable is one-dimensionaland contains coordinates of the dimension it names. Dimensions neednot have corresponding variables.

A netCDF attribute contains information about a netCDF variable orabout the whole netCDF dataset. Attributes are usedto specify such properties as units, special values, maximum andminimum valid values, scaling factors, offsets, and parameters. Attributeinformation is represented by single values or arrays of values. Forexample, "units" is an attribute represented by a character array suchas "celsius". An attribute has an associated variable, a name,a data type, a length, and a value. In contrast to variables that areintended for data, attributes are intended for metadata (data aboutdata).Unlike netCDF-3, attribute types can be any user defined typeas well as the usual built-in types.

In CDL, an attribute is designated by aa type, a variable, a ':', and then an attribute name.The type is optional and if missing, it will be inferred from the valuesassigned to the attribute.It is possible to assign global attributesnot associated with any variable to the netCDF as a whole by omittingthe variable name in the attribute declaration.Notice that there is a potential ambiguity in a specification such as

x : a = ...
In this situation, x could be either a type for a global attribute,or the variable name for an attribute. Since there could both be a type namedx and a variable named x, there is an ambiguity.The rule is that in this situation, x will be interpreted as atype if possible, and otherwise as a variable.

If not specified, the data type of an attribute in CDLis derived from the type of the value(s) assigned to it. The length ofan attribute is the number of data values assigned to it, or thenumber of characters in the character string assigned to it. Multiplevalues are assigned to non-character attributes by separating thevalues with commas. All values assigned to an attribute must be ofthe same type.

The names for CDL dimensions, variables, attributes, types, and groupsmay contain any non-control utf-8 characterexcept the forward slash character (`/').However, certain characters must escaped if they are used in a name,where the escape character is the backward slash `\'.In particular, if the leading character off the name is a digit (0-9),then it must be preceded by the escape character.In addition, the characters ` !"#$%&()*,:;<=>?[]^`'{}|~\'must be escaped if they occur anywhere in a name.Note also that attribute names that begin with an underscore (`_')are reserved for the use of Unidata and should not be used in userdefined attributes.

Note also that the words`variable',`dimension',`data',`group',and `types'are legal CDL names, but be careful that there is a spacebetween them and any following colon character when used as a variable name.This is mostly an issue with attribute declarations.For example, consider this.

 netcdf ... { ... variables:    int dimensions;        dimensions: attribute=0 ; // this will cause an error        dimensions : attribute=0 ; // this is ok.        ... }

The optional data: section of a CDL specification is wherenetCDF variables may be initialized. The syntax of an initializationis simple: a variable name, an equals sign, and acomma-delimited list of constants (possibly separated by spaces, tabsand newlines) terminated with a semicolon. For multi-dimensionalarrays, the last dimension varies fastest. Thus row-order rather thancolumn order is used for matrices. If fewer values are supplied thanare needed to fill a variable, it is extended with a type-dependent`fill value', which can be overridden by supplying a value for adistinguished variable attribute named `_FillValue'. Thetypes of constants need not match the type declared for a variable;coercions are done to convert integers to floating point, for example.The constant `_' can be used to designate the fill value for a variable.If the type of the variable is explicitly `string', then the specialconstant `NIL` can be used to represent a nil string, which is not thesame as a zero length string. 

Primitive Data Types

char    charactersbyte    8-bit datashort   16-bit signed integersint     32-bit signed integerslong    (synonymous with int)int64   64-bit signed integersfloat   IEEE single precision floating point (32 bits)real    (synonymous with float)double  IEEE double precision floating point (64 bits)ubyte   unsigned 8-bit dataushort  16-bit unsigned integersuint    32-bit unsigned integersuint64  64-bit unsigned integersstring  arbitrary length strings

CDL supports a superset of the primitive data types of C.The names for the primitive data types are reserved words in CDL,so the names of variables, dimensions, and attributes must not beprimitive type names. In declarations, type names may be specifiedin either upper or lower case.

Bytes are intended to hold a full eightbits of data, and the zero byte has no special significance, as itmays for character data.ncgen converts byte declarations to chardeclarations in the output C code and to the nonstandard BYTEdeclaration in output Fortran code.

Shorts can hold values between -32768 and 32767.ncgen converts short declarations to shortdeclarations in the output C code and to the nonstandard INTEGER*2declaration in output Fortran code.

Ints can hold values between -2147483648 and 2147483647.ncgen converts int declarations to intdeclarations in the output C code and to INTEGERdeclarations in output Fortran code. longis accepted as a synonym for int in CDL declarations, but isdeprecated since there are now platforms with 64-bit representationsfor C longs.

Int64 can hold values between -9223372036854775808and 9223372036854775807.ncgen converts int64 declarations to longlongdeclarations in the output C code.

Floats can hold values between about -3.4+38 and 3.4+38. Theirexternal representation is as 32-bit IEEE normalized single-precisionfloating point numbers. ncgen converts floatdeclarations to float declarations in the output C code and toREAL declarations in output Fortran code. real is acceptedas a synonym for float in CDL declarations.

Doubles can hold values between about -1.7+308 and 1.7+308. Theirexternal representation is as 64-bit IEEE standard normalizeddouble-precision floating point numbers. ncgen convertsdouble declarations to double declarations in the output Ccode and to DOUBLE PRECISION declarations in output Fortrancode.

The unsigned counterparts of the above integer typesare mapped to the corresponding unsigned C types.Their ranges are suitably modified to start at zero.

The technical interpretation of the char type is that it is an unsigned8-bit value. The encoding of the 256 possible valuesis unspecified by default. A variable of char typemay be marked with an "_Encoding" attribute to indicatethe character set to be used: US-ASCII, ISO-8859-1, etc.Note that specifying the encoding of UTF-8 is equivalent tospecifying US-ASCIIThis is because multi-byte UTF-8 characters cannotbe stored in an 8-bit character. The only legalsingle byte UTF-8 values are by definitionthe 7-bit US-ASCII encoding with the top bit set to zero.

Strings are assumed by default to be encoded using UTF-8.Note that this means that multi-byte UTF-8 encodings maybe present in the string, so it is possible that the numberof distinct UTF-8 characters in a string is smaller thanthe number of 8-bit bytes used to store the string.


CDL Constants

Constants assigned to attributes or variables may be of any of thebasic netCDF types. The syntax for constants is similar to C syntax,except that type suffixes must be appended to shorts and floats todistinguish them from longs and doubles.

A byte constant is represented byan integer constant with a `b' (or`B') appended. In the old netCDF-2 API, byte constants could also berepresented using single characters or standard C character escapesequences such as `a' or `. This is still supported for backwardcompatibility, but deprecated to make the distinction clear betweenthe numeric byte type and the textual char type. Example byteconstants include:

 0b             // a zero byte -1b            // -1 as an 8-bit byte 255b           // also -1 as a signed 8-bit byte

short integer constants are intended for representing 16-bitsigned quantities. The form of a short constant is an integerconstant with an `s' or `S' appended. If a short constantbegins with `0', it is interpreted as octal, except that if it begins with`0x', it is interpreted as a hexadecimal constant. For example:

-2s     // a short -20123s   // octal0x7ffs  //hexadecimal

int integer constants are intended for representing 32-bit signedquantities. The form of an int constant is an ordinary integerconstant, although it is acceptable to optionally append a single `l' or`L' (again, deprecated). Be careful, though, the L suffix is interpretedas a 32 bit integer, and never as a 64 bit integer. This can be confusingsince the C long type can ambigously be either 32 bit or 64 bit.

If an int constant begins with `0', it is interpreted asoctal, except that if it begins with `0x', it is interpreted as a hexadecimalconstant (but see opaque constants below).Examples of valid int constants include:

-21234567890L0123            // octal0x7ff           // hexadecimal

int64 integer constants are intended for representing 64-bitsigned quantities. The form of an int64 constant is an integerconstant with an `ll' or `LL' appended. If an int64 constantbegins with `0', it is interpreted as octal, except that if it begins with`0x', it is interpreted as a hexadecimal constant. For example:

-2ll    // an unsigned -20123LL  // octal0x7ffLL  //hexadecimal

Floating point constants of type float are appropriate for representingfloating point data with about seven significant digits of precision.The form of a float constant is the same as a C floating pointconstant with an `f' or `F' appended. For example the followingare all acceptable float constants:

-2.0f3.14159265358979f       // will be truncated to less precision1.f

Floating point constants of type double are appropriate forrepresenting floating point data with about sixteen significant digitsof precision. The form of a double constant is the same as a Cfloating point constant. An optional `d' or `D' may be appended.For example the following are all acceptable double constants:


Unsigned integer constants can be created by appendingthe character 'U' or 'u' between the constant and any trailingsize specifier, or immediately at the end of the size specifier.Thus one could say10U, 100su, 100000ul, or 1000000llu, for example.

Single character constants may be enclosed in single quotes.If a sequence of one or more characters is enclosedin double quotes, then its interpretation must be inferredfrom the context. If the dataset is created using the netCDFclassic model, then all such constants are interpretedas a character array, so each character in the constantis interpreted as if it were a single character.If the dataset is netCDF extended, then the constant maybe interpreted as for the classic model or as a true string(see below) depending on the type of the attribute or variableinto which the string is contained.

The interpretation of char constants is that thosethat are in the printable ASCII range (' '..'~') are assumed tobe encoded as the 1-byte subset ofUTF-8, which is equivalent to US-ASCII.In all cases, the usual C string escape conventions are honoredfor values from 0 thru 127. Values greater than 127 are allowed,but their encoding is undefined.For netCDF extended, the use of the char type is deprecatedin favor of the string type.

Some character constant examples are as follows.

 'a'            // ASCII `a' "a"            // equivalent to 'a' "Two\nlines\n" // a 10-character string with two embedded newlines "a bell:\007"  // a string containing an ASCII bell
Note that the netCDF character array "a" would fit in a one-elementvariable, since no terminating NULL character is assumed. However, a zerobyte in a character array is interpreted as the end of the significantcharacters by the ncdump program, following the C convention.Therefore, a NULL byte should not be embedded in a character string unlessat the end: use the byte data type instead for byte arrays thatcontain the zero byte.

String constants are, like character constants,represented using double quotes. This represents a potentialambiguity since a multi-character string may also indicatea dimensioned character value. Disambiguation usually occursby context, but care should be taken to specify thestringtype to ensure the proper choice.String constants are assumed to always be UTF-8 encoded. Thisspecifically means that the string constant may actuallycontain multi-byte UTF-8 characters.The special constant `NIL` can be used to represent a nil string, which is not thesame as a zero length string.

Opaque constants are represented assequences of hexadecimal digits preceded by 0X or 0x: 0xaa34ffff,for example.These constants can still be used as integer constantsand will be either truncated or extended as necessary. 

Compound Constant Expressions

In order to assign values to variables (or attributes)whose type is user-defined type, the constant notation has beenextended to include sequences of constants enclosed in curlybrackets (e.g. "{"..."}").Such a constant is called a compound constant, and compound constantscan be nested.

Given a type "T(*) vlen_t", where T is some other arbitrary base type,constants for this should be specified as follows.

    vlen_t var[2] = {t11,t12,...t1N}, {t21,t22,...t2m};
The values tij, are assumed to be constants of type T.

Given a type "compound cmpd_t {T1 f1; T2 f2...Tn fn}",where the Ti are other arbitrary base types,constants for this should be specified as follows.

    cmpd_t var[2] = {t11,t12,...t1N}, {t21,t22,...t2n};
The values tij, are assumed to be constants of type Ti.If the fields are missing, then they will be set using anyspecified or default fill value for the field's base type.

The general set of rules for using braces are defined in theSpecifyingDatalistssection below.


Scoping Rules

With the addition of groups, the name space for defined objectsis no longer flat. References (names)of any type, dimension, or variable may be prefixedwith the absolute path specifying a specific declaration.Thus one might say

    variables:        /g1/g2/t1 v1;
The type being referenced (t1) is the one within group g2, which inturn is nested in group g1.The similarity of this notation to Unix file paths is deliberate,and one can consider groups as a form of directory structure.

When name is not prefixed, then scope rules are applied to locate thespecified declaration. Currently, there are three rules: one for dimensions,one for types and enumeration constants, and one for all others.

When an unprefixed name of a dimension is used (as in avariable declaration), ncgen first looks in the immediatelyenclosing group for the dimension. If it is not foundthere, then it looks in the group enclosing this group.This continues up the group hierarchy until the dimension isfound, or there are no more groups to search.
2. When an unprefixed name of a type or an enumeration constantis used, ncgen searches the group tree using a pre-order depth-firstsearch. This essentially means that it will find the matching declarationthat precedes the reference textually in the cdl file and thatis "highest" in the group hierarchy.
3. For all other names, only the immediately enclosing group is searched.

One final note. Forward references are not allowed.This means that specifying, for example,/g1/g2/t1 will fail if this reference occurs before g1 and/or g2 are defined. 

Specifying Enumeration Constants

References to Enumeration constants (in data lists) can be ambiguoussince the same enumeration constant name can be defined in more thanone enumeration. If a cdl file specified an ambiguous constant,then ncgen will signal an error. Such constants can be disambiguatedin two ways.

Prefix the enumeration constant with the name of the enumeration separatedby a dot: enum.econst, for example.
If case one is not sufficient to disambiguate the enumeration constant,then one must specify the precise enumeration type usinga group path: /g1/g2/enum.econst, for example.

Special Attributes

Special, virtual, attributes can be specified to provideperformance-related information about the file format andabout variable properties.The file must be a netCDF-4 file for these to take effect.

These special virtual attributes are not actually part of the file,they are merely a convenient way to set miscellaneousproperties of the data in CDL

The special attributes currently supported are as follows:`_Format',`_Fletcher32,`_ChunkSizes',`_Endianness',`_DeflateLevel',`_Shuffle', and`_Storage'.

`_Format' is a global attribute specifying the netCDF formatvariant. Its value must be a single stringmatching one of `classic', `64-bit offset', `64-bit data', `netCDF-4', or`netCDF-4 classic model'.

The rest of the special attributes are all variable attributes.Essentially all of then map to some corresponding `nc_def_var_XXX'function as defined in the netCDF-4 API.For the attributes that are essentially boolean (_Fletcher32, _Shuffle,and _NOFILL), the value true can be specified by using the strings`true' or `1', or by using the integer 1.The value false expects either `false', `0', or the integer 0.The actions associated with these attributes are as follows.

`_Fletcher32 sets the `fletcher32' property for a variable.
`_Endianness' is either `little' or `big', depending onhow the variable is stored when first written.
`_DeflateLevel' is aninteger between 0 and 9 inclusive if compression has been specifiedfor the variable.
`_Shuffle' specifies if the the shuffle filter should be used.
`_Storage' is `contiguous' or `chunked'.
`_ChunkSizes' is a list of chunk sizes for each dimension ofthe variable

Note that attributes such as "add_offset" or "scale_factor"have no special meaning to ncgen. These attributes arecurrently conventions, handled above the library layer byother utility packages, for example NCO.


Specifying Datalists

Specifying datalists for variables in the `data:` section can be somewhatcomplicated. There are some rules that must be followedto ensure that datalists are parsed correctly by ncgen.

First, the top level is automatically assumed to be a list of items, so it should not be inside {...}.That means that if the variable is a scalar, there will be a single top-level elementand if the variable is an array, there will be N top-level elements.For each element of the top level list, the following rules should be applied.

Instances of UNLIMITED dimensions (other than the first dimension) must be surrounded by {...} in order to specify the size.
Compound instances must be embedded in {...}
Non-scalar fields of compound instances must be embedded in {...}.
Instances of vlens must be surrounded by {...} in order to specify the size.

Datalists associated with attributes are implicitly a vector (i.e., a list) of values of the type of the attribute and the above rules must apply with that in mind.

No other use of braces is allowed.

Note that one consequence of these rules is thatarrays of values cannot have subarrays within braces.Consider, for example, int var(d1)(d2)...(dn),where none of d2...dn are unlimited.A datalist for this variable must be a single list of integers,where the number of integers is no more than D=d1*d2*...dn values;note that the list can be less than D, in which case fill valueswill be used to pad the list.

Rule 6 about attribute datalist has the following consequence.If the type of the attribute is a compound (or vlen) type, and ifthe number of entries in the list is one, then the compound instancesmust be enclosed in braces.


Specifying Character Datalists

Specifying datalists for variables of type char also has somecomplications. consider, for example

dimensions: u=UNLIMITED; d1=1; d2=2; d3=3;            d4=4; d5=5; u2=UNLIMITED;variables: char var(d4,d5);datalist: var="1", "two", "three";

We have twenty elements of var to fill (d5 X d4)and we have three strings of length 1, 3, 5.How do we assign the characters in the strings to thetwenty elements?

This is challenging because it is desirable to mimicthe original ncgen (ncgen3).The core algorithm is notionally as follows.

Assume we have a set of dimensions D1..Dn,where D1 may optionally be an Unlimited dimension.It is assumed that the sizes of the Di are all known(including unlimited dimensions).
Given a sequence of string or character constantsC1..Cm, our goal is to construct a single stringwhose length is the cross product of D1 thru Dn.Note that for purposes of this algorithm, character constantsare treated as strings of size 1.
Construct Dx = cross product of D1 thru D(n-1).
For each constant Ci, add fill characters as neededso that its length is a multiple of Dn.
Concatenate the modified C1..Cm to produce string S.
Add fill characters to S to make its length be a multiple of Dn.
If S is longer than the Dx * Dn, then truncateand generate a warning.

There are three other cases of note.

If there is only a single, unlimited dimension,then all of the constants are concatenatedand fill characters are added to theend of the resulting string to make itslength be that of the unlimited dimension.If the length is larger thanthe unlimited dimension, then it is truncatedwith a warning.
For the case of character typed vlen, "char(*) vlen_t" for example.we simply concatenate all the constants with no filling at all.
For the case of a character typed attribute,we simply concatenate all the constants.

In netcdf-4, dimensions other than the first can be unlimited.Of course by the rules above, the interior unlimited instancesmust be delimited by {...}. For example.

variables: char var(u,u2);datalist: var={"1", "two"}, {"three"};
In this case u will have the effective length of two.Within each instance of u2, the rules above will apply, leadingto this.datalist: var={"1","t","w","o"}, {"t","h","r","e","e"};The effective size of u2 will be the max of the two instance lengths(five in this case)and the shorter will be padded to produce this.datalist: var={"1","t","w","o","\0"}, {"t","h","r","e","e"};

Consider an even more complicated case.

variables: char var(u,u2,u3);datalist: var={{"1", "two"}}, {{"three"},{"four","xy"}};
In this case u again will have the effective length of two.The u2 dimensions will have a size = max(1,2) = 2;Within each instance of u2, the rules above will apply, leading to this.
datalist: var={{"1","t","w","o"}}, {{"t","h","r","e","e"},{"f","o","u","r","x","y"}};
The effective size of u3 will be the max of the two instance lengths(six in this case) and the shorter ones will be padded to produce this.
datalist: var={{"1","t","w","o"," "," "}}, {{"t","h","r","e","e"," "},{"f","o","u","r","x","y"}};
Note however that the first instance of u2 is less than the max lengthof u2, so we need to add a filler for another instance of u2, producing this.
datalist: var={{"1","t","w","o"," "," "},{" "," "," "," "," "," "}}, {{"t","h","r","e","e"," "},{"f","o","u","r","x","y"}};



The programs generated by ncgen when using the -c flaguse initialization statements to store data in variables, and will fail toproduce compilable programs if you try to use them for large datasets, sincethe resulting statements may exceed the line length or number ofcontinuation statements permitted by the compiler.

The CDL syntax makes it easy to assign what looks like an array ofvariable-length strings to a netCDF variable, but the strings may simply beconcatenated into a single array of characters.Specific use of the string type specifier may solve the problem


CDL Grammar

The file ncgen.y is the definitive grammar for CDL, but a strippeddown version is included here for completeness.

ncdesc: NETCDF        datasetid        rootgroup        ;datasetid: DATASETIDrootgroup: '{'           groupbody           subgrouplist           '}';groupbody:                attrdecllist                typesection                dimsection                vasection                datasection                ;subgrouplist:          /*empty*/        | subgrouplist namedgroup        ;namedgroup: GROUP ident '{'            groupbody            subgrouplist            '}'            attrdecllist            ;typesection:    /* empty */                | TYPES                | TYPES typedecls                ;typedecls:          type_or_attr_decl        | typedecls type_or_attr_decl        ;typename: ident ;type_or_attr_decl:          typedecl        | attrdecl ';'        ;typedecl:          enumdecl optsemicolon        | compounddecl optsemicolon        | vlendecl optsemicolon        | opaquedecl optsemicolon        ;optsemicolon:          /*empty*/        | ';'        ;enumdecl: primtype ENUM typename ;enumidlist:   enumid            | enumidlist ',' enumid            ;enumid: ident '=' constint ;opaquedecl: OPAQUE '(' INT_CONST ')' typename ;vlendecl: typeref '(' '*' ')' typename ;compounddecl: COMPOUND typename '{' fields '}' ;fields:   field ';'        | fields field ';'        ;field: typeref fieldlist ;primtype:         CHAR_K                | BYTE_K                | SHORT_K                | INT_K                | FLOAT_K                | DOUBLE_K                | UBYTE_K                | USHORT_K                | UINT_K                | INT64_K                | UINT64_K                ;dimsection:     /* empty */                | DIMENSIONS                | DIMENSIONS dimdecls                ;dimdecls:       dim_or_attr_decl ';'                | dimdecls dim_or_attr_decl ';'                ;dim_or_attr_decl: dimdeclist  | attrdecl  ;dimdeclist:     dimdecl                | dimdeclist ',' dimdecl                ;dimdecl:          dimd '=' UINT_CONST        | dimd '=' INT_CONST        | dimd '=' DOUBLE_CONST        | dimd '=' NC_UNLIMITED_K        ;dimd:           ident ;vasection:      /* empty */                | VARIABLES                | VARIABLES vadecls                ;vadecls:        vadecl_or_attr ';'                | vadecls vadecl_or_attr ';'                ;vadecl_or_attr: vardecl  | attrdecl  ;vardecl:        typeref varlist ;varlist:      varspec            | varlist ',' varspec            ;varspec:        ident dimspec ;dimspec:        /* empty */                | '(' dimlist ')'                ;dimlist:        dimref                | dimlist ',' dimref                ;dimref: path ;fieldlist:          fieldspec        | fieldlist ',' fieldspec        ;fieldspec: ident fielddimspec ;fielddimspec:     /* empty */                | '(' fielddimlist ')'                ;fielddimlist:          fielddim        | fielddimlist ',' fielddim        ;fielddim:          UINT_CONST        | INT_CONST        ;/* Use this when referencing defined objects */varref: type_var_ref ;typeref: type_var_ref      ;type_var_ref:          path        | primtype        ;/* Use this for all attribute decls *//* Watch out; this is left recursive */attrdecllist: /*empty*/  | attrdecl ';' attrdecllist  ;attrdecl:          ':' ident '=' datalist        | typeref type_var_ref ':' ident '=' datalist        | type_var_ref ':' ident '=' datalist        | type_var_ref ':' _FILLVALUE '=' datalist        | typeref type_var_ref ':' _FILLVALUE '=' datalist        | type_var_ref ':' _STORAGE '=' conststring        | type_var_ref ':' _CHUNKSIZES '=' intlist        | type_var_ref ':' _FLETCHER32 '=' constbool        | type_var_ref ':' _DEFLATELEVEL '=' constint        | type_var_ref ':' _SHUFFLE '=' constbool        | type_var_ref ':' _ENDIANNESS '=' conststring        | type_var_ref ':' _NOFILL '=' constbool        | ':' _FORMAT '=' conststring        ;path:          ident        | PATH        ;datasection:    /* empty */                | DATA                | DATA datadecls                ;datadecls:          datadecl ';'        | datadecls datadecl ';'        ;datadecl: varref '=' datalist ;datalist:          datalist0        | datalist1        ;datalist0:        /*empty*/        ;/* Must have at least 1 element */datalist1:          dataitem        | datalist ',' dataitem        ;dataitem:          constdata        | '{' datalist '}'        ;constdata:          simpleconstant        | OPAQUESTRING        | FILLMARKER        | NIL        | econstref        | function        ;econstref: path ;function: ident '(' arglist ')' ;arglist:          simpleconstant        | arglist ',' simpleconstant        ;simpleconstant:          CHAR_CONST /* never used apparently*/        | BYTE_CONST        | SHORT_CONST        | INT_CONST        | INT64_CONST        | UBYTE_CONST        | USHORT_CONST        | UINT_CONST        | UINT64_CONST        | FLOAT_CONST        | DOUBLE_CONST        | TERMSTRING        ;intlist:          constint        | intlist ',' constint        ;constint:          INT_CONST        | UINT_CONST        | INT64_CONST        | UINT64_CONST        ;conststring: TERMSTRING ;constbool:          conststring        | constint        ;/* Push all idents thru here for tracking */ident: IDENT ;



Choosing the output format
CDL Syntax Overview
Primitive Data Types
CDL Constants
Compound Constant Expressions
Scoping Rules
Specifying Enumeration Constants
Special Attributes
Specifying Datalists
Specifying Character Datalists
CDL Grammar

This document was created byman2html,using the manual pages.