Difference between revisions of "Filling Out the Table Character Data Structure"

From The SBN Wiki
Jump to navigation Jump to search
(→‎<Field_Character>: Update for IM 1.14.0.0)
 
(30 intermediate revisions by 2 users not shown)
Line 18: Line 18:
  
 
If you want to reference this ''Table_Character'' from somewhere else in this label, give it a formal label here.  Use an identifier that would make a valid variable name in a typical programming language, and you should be OK syntactically.
 
If you want to reference this ''Table_Character'' from somewhere else in this label, give it a formal label here.  Use an identifier that would make a valid variable name in a typical programming language, and you should be OK syntactically.
 +
 +
== &lt;md5_checksum&gt; ==
 +
 +
''OPTIONAL''
 +
 +
Use this attribute to provide the MD5 checksum of the object ''only''.  If the object occupies the entire file, then the checksum should be given as an attribute of the ''&lt;File&gt;'' object.  This checksum should be calculated using ''only'' the bytes defined as being part of this table.
  
 
== &lt;offset&gt; ==
 
== &lt;offset&gt; ==
Line 32: Line 38:
 
''REQUIRED''
 
''REQUIRED''
  
This is the total number of records in the table.  Note that in a ''Table_Character'' table, each record must have carriage-return/linefeed record delimiter (including the last record).
+
This is the total number of records in the table.  Note that in a ''Table_Character'' table, each record including the last must have a carriage-return/linefeed record delimiter at the end.
  
 
== &lt;description&gt; ==
 
== &lt;description&gt; ==
Line 44: Line 50:
 
''REQUIRED''
 
''REQUIRED''
  
This attribute must have the value '''carriage-return line-feed'''.  The data file must also have carriage-return/linefeed record delimiters, of course.
+
This attribute must have the value '''Carriage-Return Line-Feed'''.  The data file must also have carriage-return/linefeed record delimiters, of course.
  
 
== &lt;Uniformly_Sampled&gt; ==
 
== &lt;Uniformly_Sampled&gt; ==
Line 51: Line 57:
  
 
If this ''Table_Character'' contains records which are uniformly spaced in some dimension (time, wavelength, distance, etc.), you can use this class to define that dimension and interval rather than including an additional field in each row to hold the value explicitly. The details are on the [[Filling Out the Uniformly Sampled Class]] page.
 
If this ''Table_Character'' contains records which are uniformly spaced in some dimension (time, wavelength, distance, etc.), you can use this class to define that dimension and interval rather than including an additional field in each row to hold the value explicitly. The details are on the [[Filling Out the Uniformly Sampled Class]] page.
 
Here's a quick summary of what's in that class.  All but ''sampling_parameter_scale'' are required:
 
* sampling_parameter_name
 
* sampling_parameter_interval
 
* sampling_parameter_unit
 
* first_sampling_parameter_value
 
* last_sampling_parameter_value
 
* sampling_parameter_scale
 
  
 
== &lt;Record_Character&gt; ==
 
== &lt;Record_Character&gt; ==
Line 70: Line 68:
 
''REQUIRED''
 
''REQUIRED''
  
The number of ''Field'' classes directly under (that is, in the first  nestinglevel of) the ''Record'' class.  Do '''''not''''' count ''Field'' classes nested under ''Group_Field'' classes.  
+
The number of ''Field_Character'' classes directly under (that is, in the first  nesting level of) the ''Record_Character'' class.  Do '''''not''''' count ''Field'' classes nested under ''Group_Field'' classes.  
  
If your ''Record'' contains only one or more ''Group_Field'' classes, this will have a value of zero.
+
If your ''Record_Character'' contains only one or more ''Group_Field_Character'' classes, this will have a value of zero.
  
 
=== &lt;groups&gt; ===
 
=== &lt;groups&gt; ===
Line 78: Line 76:
 
''REQUIRED''  
 
''REQUIRED''  
  
The number of ''Group_Field'' classes directly under (that is, in the first nesting level of) the ''Record'' class.  Do '''''not''''' count ''Group_Field'' classes nested under other ''Group_Field'' classes.
+
The number of ''Group_Field_Character'' classes directly under (that is, in the first nesting level of) the ''Record_Character'' class.  Do '''''not''''' count ''Group_Field_Character'' classes nested under other ''Group_Field_Character'' classes.
  
If your ''Record'' contains only one or more ''Field'' classes, this will have a value of zero.
+
If your ''Record_Character'' contains only one or more ''Field_Character'' classes, this will have a value of zero.
  
 
=== &lt;record_length&gt; ===
 
=== &lt;record_length&gt; ===
Line 93: Line 91:
 
----
 
----
  
=== A Note about '''Fields''' and '''Group_Fields''' ===
+
=== A Note about '''Fields''' and '''Group Fields''' ===
  
Records are composed of ''Fields'' and ''Group Fields''.  A ''Record'' must have at least one of those, and can have an arbitrary number of them, in any order (that is, you can have ''Fields'' and ''Group Fields'' interspersed). Note, however, that ''Group Fields'' are '''''never''''' necessary - they are a notational convenience to save writing out large numbers of essentially identical ''Field'' definitions.
+
Records are composed of ''Fields'' and ''Group Fields''.  A ''Record'' must have at least one of those (either will do), and can have an arbitrary number of them, in any order (that is, you can have ''Fields'' and ''Group Fields'' interspersed). Note, however, that ''Group Fields'' are '''''never''''' necessary - they are a notational convenience to save writing out large numbers of essentially identical ''Field'' definitions.
  
{| class="wikitable" style="background-color: yellow"
+
{| class="wikitable" style="background-color: lightcyan"
 
|  
 
|  
: SBN data preparers should '''''never''''' use a ''Group_Field'' where a reasonable set of ''Fields'' will do.  In particular, no ''Group_Field'' should have a ''&lt;repetitions&gt;'' count of less than 2 in any SBN data product.  If you have data that seem like they should be an exception, please contact your SBN data consultant with the details.
+
: SBN data preparers should '''''never''''' use a ''Group Field'' where a reasonable set of ''Fields'' will do.  In particular, no ''Group Field'' should have a ''&lt;repetitions&gt;'' count of less than 2 in any SBN data product.  If you have data that seem like they should be an exception, please contact your SBN data consultant with the details.
 
|}
 
|}
  
Line 105: Line 103:
  
 
=== &lt;Field_Character&gt; ===
 
=== &lt;Field_Character&gt; ===
 +
 +
''OPTIONAL''
  
 
This class defines a single, non-repeating scalar field.
 
This class defines a single, non-repeating scalar field.
Line 118: Line 118:
 
''OPTIONAL''
 
''OPTIONAL''
  
This is the sequential number of the ''Field'' definition. For SBN data products, the ''field_number'' is intended to be a help to human readers trying to map field definitions to columns in a print-out of the ''Table''.
+
This is the sequential number of the ''Field_Character'' definition. For SBN data products, the ''field_number'' is intended to be a help to human readers trying to map field definitions to columns in a print-out of the ''Table''.
  
{| class="wikitable" style="background-color: yellow"
+
{| class="wikitable" style="background-color: lightcyan"
 
|  
 
|  
: SBN will generally require ''&lt;field_number&gt;'' be used in all flat tables (i.e., those without ''Group_Fields'').
+
: SBN will generally require ''&lt;field_number&gt;'' be used in all flat tables (i.e., those without ''Group Fields'').
  
: The ''Standards Reference'' lays out rules for using the ''field_number'' in cases where there are ''Group_Fields'' present which can be useful in programmatic contexts, but not so much in the visual-inspection case. For these tables SBN considers ''field_number'' and ''group_number'' to be optional.
+
: The ''Standards Reference'' lays out rules for using the ''field_number'' in cases where there are ''Group_Field_Character''s present which can be useful in programmatic contexts, but not so much in the visual-inspection case. For these tables SBN considers ''field_number'' and ''group_number'' to be optional.
 
|}
 
|}
  
Line 131: Line 131:
 
''REQUIRED''
 
''REQUIRED''
  
This is the location, in bytes, of the first character in the field relative to the start of the enclosing ''Record'' or ''Group_Field''.  (Note that locations begin with one, rather than zero.)  You must indicate a unit of bytes for this field:
+
This is the location, in bytes, of the first character in the field relative to the start of the enclosing ''Record_Character'' or ''Group_Field_Character''.  (Note that locations begin with one, rather than zero.)  You must indicate a unit of bytes for this field:
 
<pre>
 
<pre>
 
     <field_location unit="byte">1</offset>
 
     <field_location unit="byte">1</offset>
Line 138: Line 138:
 
This must be the location of the first byte that actually constitutes the field data.  It must  
 
This must be the location of the first byte that actually constitutes the field data.  It must  
 
''not'' correspond to any field delimiter or gutter space included in the data table.
 
''not'' correspond to any field delimiter or gutter space included in the data table.
 +
{| class="wikitable" style="background-color: yellow"
 +
|
 +
: ''As a rule and as good practice, ''table fields should not overlap''. That sort of thing can hide subtle errors from users, reviewers, and software.  If you have a ''really'' good reason to consider defining overlapping fields, please make sure you get that OKed by your consulting PDS node ''very'' early on in the development process.''
 +
|}
  
 
==== &lt;data_type&gt; ====
 
==== &lt;data_type&gt; ====
Line 160: Line 164:
 
''OPTIONAL''
 
''OPTIONAL''
  
The value of this attribute is a string representing the read/print format for the data in the field, using a subset of the POSIX print conventions defined in the ''Standards Reference'', and also described on the [[PDS4_field_format_Conventions]] page.
+
The value of this attribute is a string representing the read/print format for the data in the field, using a subset of the POSIX print conventions defined in the ''Standards Reference'', and also described on the [[PDS4 field format Conventions]] page.
 +
 
 +
==== &lt;validation_format&gt; ====
 +
 
 +
''OPTIONAL''
  
 
{| class="wikitable" style="background-color: yellow"
 
{| class="wikitable" style="background-color: yellow"
| SBN will require that this attribute be present in '''all''' ''Field'' definitionsIt is used for validation of the ''Table'' contents.
+
|  
 +
: The intention of this attribute seems to be to provide a format for validation purposes - to enable the validator to detect output field overflow in a programmatically-produced table, for example. But if you use it, you are also required to include ''&lt;field_format&gt;'', and both attributes are required to have exactly the same valueThis is a strange implementation.  Use with caution.
 
|}
 
|}
  
Line 192: Line 201:
 
Free-format text describing the content of the field.
 
Free-format text describing the content of the field.
  
{| class="wikitable" style="background-color: yellow"
+
{| class="wikitable" style="background-color: lightcyan"
 
| '''''Note:''''' While not required, SBN expects to see a useful definition for every ''Field'', as do both reviewers and users.  Omit this field at your peril.
 
| '''''Note:''''' While not required, SBN expects to see a useful definition for every ''Field'', as do both reviewers and users.  Omit this field at your peril.
 
|}
 
|}
Line 200: Line 209:
 
''OPTIONAL''
 
''OPTIONAL''
  
This class defines flag values used to indicate that a particular field value is unknown for one reason or another.  It is identical to the ''&lt;Special_Constants&gt;'' class used in the ''Array'' classes.  For details, check the [[Filling_Out_the_Array_2D_Data_Structure#.3CSpecial_Constants.3E|Filling Out the Array 2D Data Structure - &lt;Special_Constants&gt;]] page.  Here is a quick list of the special constants available in this class:
+
This class defines flag values used in the data table to indicate why a particular field value is not available, and to document limiting values for the field.  It is identical to the ''&lt;Special_Constants&gt;'' class used in the ''Array'' classes.  For details, check the [[Filling_Out_the_Array_2D_Data_Structure#.3CSpecial_Constants.3E|Filling Out the Array 2D Data Structure - &lt;Special_Constants&gt;]] page.  Here is a quick list of the special constants included in this class:
 
* ''saturated_constant''
 
* ''saturated_constant''
 
* ''missing_constant''
 
* ''missing_constant''
Line 214: Line 223:
 
* ''low_representation_saturation''
 
* ''low_representation_saturation''
  
 +
You may use as many of these within the ''Special_Constants'' class as are applicable to the field. Say, for example, you have a table of compiled properties, and in one particular integer field there are cases where either the value is unknown, or in some cases the field itself is just not applicable.  If you wanted to document that the valid range of values is 100-500, and you are using "-999" to indicate "Unknown", and "-888" to indicate "Not Applicable", you would adding this to your ''&lt;Field_Character&gt;'' class:
 +
<pre>
 +
  <Special_Constants>
 +
    <unknown_constant>-999</unknown_constant>
 +
    <not_applicable_constant>-888</not_applicable_constant>
 +
    <valid_maximum>500</valid_maximum>
 +
    <valid_minimum>100</valid_minimum>
 +
  </Special_Constants>
 +
</pre>
  
 
==== &lt;Field_Statistics&gt; ====
 
==== &lt;Field_Statistics&gt; ====
Line 220: Line 238:
  
 
If you want to include things like extrema, mean value, and such for all the values that occur in this field through all the records in the table, this is the place to do it.  This class is identical for all ''Field'' types.  For details, see [[Filling Out the Field Statistics Class]].  Here is a quick list of the field statistics available in this class:
 
If you want to include things like extrema, mean value, and such for all the values that occur in this field through all the records in the table, this is the place to do it.  This class is identical for all ''Field'' types.  For details, see [[Filling Out the Field Statistics Class]].  Here is a quick list of the field statistics available in this class:
 +
* ''local_identifier''
 
* ''maximum''
 
* ''maximum''
 
* ''minimum''
 
* ''minimum''
Line 225: Line 244:
 
* ''standard_deviation''
 
* ''standard_deviation''
 
* ''median''
 
* ''median''
 +
* ''description''
  
 +
=== &lt;Group_Field_Character&gt; ===
  
=== &lt;Group_Field_Character&gt; ===
+
''OPTIONAL''
  
This class defines a set of ''Fields'' and ''Group Fields'' that repeats a given number of times in each record.  ''Group Fields'' may be nested.
+
This class defines a set of ''Field_Character''s and ''Group_Field_Character''s that repeats a given number of times in each record.  ''Group_Field_Character''s may be nested.
  
{| class="wikitable" style="background-color: yellow"
+
{| class="wikitable" style="background-color: lightcyan"
| '''''Note:''''' Unless you have three good reasons, don't use ''Group Fields'' in SBN data.
+
| '''''Note:''''' Unless you have three good reasons, don't use ''Group_Field_Character'' in SBN data.
 
|}
 
|}
 +
 +
==== &lt;name&gt; ====
 +
 +
''OPTIONAL''
 +
 +
You can use this field to specify a name for the group, if you like.
  
 
==== &lt;group_number&gt; ====
 
==== &lt;group_number&gt; ====
  
Analogous to ''field_number'' for scalar fields, this is a sequential number useful for referencing ''Group_Field'' classes at a single nesting level of a complex ''Record'' definition.
+
''OPTIONAL''
 +
 
 +
Analogous to ''field_number'' for scalar fields, this is a sequential number useful for referencing ''Group_Field_Character'' classes at a single nesting level of a complex ''Record_Character'' definition.
  
 
==== &lt;repetitions&gt; ====
 
==== &lt;repetitions&gt; ====
Line 243: Line 272:
 
''REQUIRED''
 
''REQUIRED''
  
The number of times the complete set of ''Fields'' and ''Group Fields'' comprising this ''&lt;Group_Field_Character&gt;'' repeats.
+
The number of times the complete set of ''Field_Character''s and ''Group_Field_Character''s comprising this ''&lt;Group_Field_Character&gt;'' repeats.
  
{| class="wikitable" style="background-color: yellow"
+
{| class="wikitable" style="background-color: lightcyan"
| '''''Note:''''' ''The minimum value for this field listed in the data dictionary is one, but as a rule no product will pass SBN review unless this value is at least two.''
+
| '''''Note:''''' ''The minimum value for this field listed in the data dictionary is one, but it is unlikely a product will pass SBN review unless this value is at least two.''
 
|}
 
|}
  
Line 253: Line 282:
 
''REQUIRED''
 
''REQUIRED''
  
The count of ''Field'' classes directly under (i.e., at the first nesting level below) the ''Group_Field'' definition.  This will be zero if the group contains no scalar ''Fields''.
+
The count of ''Field_Character'' classes directly under (i.e., at the first nesting level below) the ''Group_Field_Character'' definition.  This will be zero if the group contains no ''Field_Character'' classes.
  
 
==== &lt;groups&gt; ====
 
==== &lt;groups&gt; ====
Line 259: Line 288:
 
''REQUIRED''
 
''REQUIRED''
  
The count of ''Group_Field'' classes directly under (i.e., at the first nesting level below( the present ''Group_Field'' definition.  This will be zero of the group contains no nested ''Group_Field'' classes.
+
The count of ''Group_Field_Character'' classes directly under (i.e., at the first nesting level below) the present ''Group_Field_Character'' definition.  This will be zero if the group contains no nested ''Group_Field_Character'' classes.
 +
 
 +
==== &lt;description&gt; ====
 +
 
 +
''OPTIONAL''
 +
 
 +
Here's a place to describe what this grouping of fields represents.
  
 
==== &lt;group_location&gt; ====
 
==== &lt;group_location&gt; ====
Line 265: Line 300:
 
''REQUIRED''
 
''REQUIRED''
  
This is the location of the first byte of the first field of the first repetition of this group relative to the containing ''Record'' or ''Group_Field'' location.  If the group starts at the beginning of the containing ''Record'' or ''Group'', this has a value of one.  You must specify a unit of "byte" for this value.  For example:
+
This is the location of the first byte of the first field of the first repetition of this group relative to the containing ''Record_Character'' or ''Group_Field_Character'' location.  If the group starts at the beginning of the containing ''Record_Character'' or ''Group_Field_Character'', this has a value of one.  You must specify a unit of "byte" for this value.  For example:
 
<pre>
 
<pre>
 
     <group_location unit="byte">1</group_location>
 
     <group_location unit="byte">1</group_location>
 
</pre>
 
</pre>
  
This value should be chosen bearing in  mind that it will be used as the base offset for locating each repetition of the group.  In other words, the location of the start of the first repetition of the group is ''group_location'' + 0*''group_length''; the location of the start of the second repetition of the group is ''group_location'' + 1*''group_length'', and so on.  
+
This value should be set bearing in  mind that it will be used as the base offset for locating each repetition of the group.  In other words, the location of the start of the first repetition of the group is ''group_location'' + 0*(''group_length''/''repetitions''); the location of the start of the second repetition of the group is ''group_location'' + 1*(''group_length''/''repetitions''), and so on.
 +
 
 +
==== &lt;group_length&gt; ====
 +
 
 +
''REQUIRED''
 +
 
 +
This is the length of the entire group - that is, '''''the length of one repetition of the group, multiplied by the value in the ''&lt;repetitions&gt;'' attribute''''', above.  You must indicate a ''unit'' of "bytes" for this length.  It must be evenly divisible by the value of ''repetitions''.
 +
 
 +
{| class="wikitable" style="background-color: yellow"
 +
|
 +
: '''Note:''' ''This value is '''not''' validated.  Calculate carefully.''
 +
|}
  
==== &lt;Fields&gt; and Nested &lt;Group_Fields&gt; ====
+
==== Fields and Nested Groups ====
  
As in the ''Record'', the ''Group'' may contain either ''Fields'', or ''Group_Fields'', or both intermixed.  ''Group_Fields'' may be nested arbitrarily deeply.  The requirement for these data structure classes inside a ''&lt;Group_Field_Character&gt;'' are identical to those above.
+
As in the ''Record_Character'', the ''Group_Field_Character'' may contain either ''Field_Character'' classes, or ''Group_Field_Character'' classes, or both intermixed.  ''Group_Field_Character'' classes may be nested arbitrarily deeply.  The requirements for these data structure classes inside a ''&lt;Group_Field_Character&gt;'' are identical to those above.

Latest revision as of 18:05, 3 August 2020

The <Table_Character> class contains the data structure definition for a character table. Each row in the table has the same structure, defined in a Record_Character class. Records are themselves composed of scalar fields, or sub-records called "grouped fields".

For additional explanation, see the PDS4 Standards Reference, or contact your PDS node consultant.

Following are the attributes and subclasses you'll find in <Table_Character>, in label order.

Note that in the PDS4 master schema, all classes have capitalized names; attributes never do.

<name>

OPTIONAL

If you'd like to give this table a descriptive name, here's the place to do it.

<local_identifier>

OPTIONAL

If you want to reference this Table_Character from somewhere else in this label, give it a formal label here. Use an identifier that would make a valid variable name in a typical programming language, and you should be OK syntactically.

<md5_checksum>

OPTIONAL

Use this attribute to provide the MD5 checksum of the object only. If the object occupies the entire file, then the checksum should be given as an attribute of the <File> object. This checksum should be calculated using only the bytes defined as being part of this table.

<offset>

REQUIRED

The offset, in bytes, from the beginning of the file holding the table data to the first character in the table. You must specify the unit for this keyword, like this:

    <offset unit="byte">0</offset>

<records>

REQUIRED

This is the total number of records in the table. Note that in a Table_Character table, each record including the last must have a carriage-return/linefeed record delimiter at the end.

<description>

OPTIONAL

This attribute provides a place for additional free-format text comments.

<record_delimiter>

REQUIRED

This attribute must have the value Carriage-Return Line-Feed. The data file must also have carriage-return/linefeed record delimiters, of course.

<Uniformly_Sampled>

OPTIONAL

If this Table_Character contains records which are uniformly spaced in some dimension (time, wavelength, distance, etc.), you can use this class to define that dimension and interval rather than including an additional field in each row to hold the value explicitly. The details are on the Filling Out the Uniformly Sampled Class page.

<Record_Character>

REQUIRED

This class describes the structure of one complete record in the table.

<fields>

REQUIRED

The number of Field_Character classes directly under (that is, in the first nesting level of) the Record_Character class. Do not count Field classes nested under Group_Field classes.

If your Record_Character contains only one or more Group_Field_Character classes, this will have a value of zero.

<groups>

REQUIRED

The number of Group_Field_Character classes directly under (that is, in the first nesting level of) the Record_Character class. Do not count Group_Field_Character classes nested under other Group_Field_Character classes.

If your Record_Character contains only one or more Field_Character classes, this will have a value of zero.

<record_length>

REQUIRED

The total length of the record, including all fields, all repetitions of group fields, any space between fields, and the record delimiters. You must specify a unit of bytes for this value:

    <record_length unit="byte">1234</record_length>

A Note about Fields and Group Fields

Records are composed of Fields and Group Fields. A Record must have at least one of those (either will do), and can have an arbitrary number of them, in any order (that is, you can have Fields and Group Fields interspersed). Note, however, that Group Fields are never necessary - they are a notational convenience to save writing out large numbers of essentially identical Field definitions.

SBN data preparers should never use a Group Field where a reasonable set of Fields will do. In particular, no Group Field should have a <repetitions> count of less than 2 in any SBN data product. If you have data that seem like they should be an exception, please contact your SBN data consultant with the details.

<Field_Character>

OPTIONAL

This class defines a single, non-repeating scalar field.

<name>

REQUIRED

The name of the field. SBN recommends that this be something fairly human-readable that can be easily turned into a variable name for use in applications, or displayed as a meaningful column heading.

<field_number>

OPTIONAL

This is the sequential number of the Field_Character definition. For SBN data products, the field_number is intended to be a help to human readers trying to map field definitions to columns in a print-out of the Table.

SBN will generally require <field_number> be used in all flat tables (i.e., those without Group Fields).
The Standards Reference lays out rules for using the field_number in cases where there are Group_Field_Characters present which can be useful in programmatic contexts, but not so much in the visual-inspection case. For these tables SBN considers field_number and group_number to be optional.

<field_location>

REQUIRED

This is the location, in bytes, of the first character in the field relative to the start of the enclosing Record_Character or Group_Field_Character. (Note that locations begin with one, rather than zero.) You must indicate a unit of bytes for this field:

    <field_location unit="byte">1</offset>

This must be the location of the first byte that actually constitutes the field data. It must not correspond to any field delimiter or gutter space included in the data table.

As a rule and as good practice, table fields should not overlap. That sort of thing can hide subtle errors from users, reviewers, and software. If you have a really good reason to consider defining overlapping fields, please make sure you get that OKed by your consulting PDS node very early on in the development process.

<data_type>

REQUIRED

The type of the values in the field. This must be one of the values listed in the Standard Values Quick Reference.

<field_length>

REQUIRED

The length of the field, in bytes. You must specify the unit:

    <field_length unit="byte">12</field_length>

As with field_location, this must include only the bytes containing field data. Do not include any delimiters or gutter space that might be included in the data table.

<field_format>

OPTIONAL

The value of this attribute is a string representing the read/print format for the data in the field, using a subset of the POSIX print conventions defined in the Standards Reference, and also described on the PDS4 field format Conventions page.

<validation_format>

OPTIONAL

The intention of this attribute seems to be to provide a format for validation purposes - to enable the validator to detect output field overflow in a programmatically-produced table, for example. But if you use it, you are also required to include <field_format>, and both attributes are required to have exactly the same value. This is a strange implementation. Use with caution.

<unit>

OPTIONAL

If the value in this field has an associated unit, this is where it goes. This value is case sensitive, and you may use characters from the UTF-8 character set (like the Angstrom symbol) where appropriate.

Note: If a field contains a unitless value, then there should be no <unit> attribute. NEVER include a null unit value, or even worse, this: <unit>N/A</unit>.

<scaling_factor>

OPTIONAL

If the observational values of this field were scaled prior to writing, this attribute should contain the value the data must be multiplied by to get back to the original value. Scaling factors are applied prior to adding any offset.

<value_offset>

OPTIONAL

If the observational values of the field were shifted by an offset prior to writing, this attribute should contain the value that must be added to each field value to get back to the original value. Offsets and added after the scaling factor, if any, is applied.

<description>

OPTIONAL

Free-format text describing the content of the field.

Note: While not required, SBN expects to see a useful definition for every Field, as do both reviewers and users. Omit this field at your peril.

<Special_Constants>

OPTIONAL

This class defines flag values used in the data table to indicate why a particular field value is not available, and to document limiting values for the field. It is identical to the <Special_Constants> class used in the Array classes. For details, check the Filling Out the Array 2D Data Structure - <Special_Constants> page. Here is a quick list of the special constants included in this class:

  • saturated_constant
  • missing_constant
  • error_constant
  • invalid_constant
  • unknown_constant
  • not_applicable_constant
  • valid_maximum
  • high_instrument_saturation
  • high_representation_saturation
  • valid_minimum
  • low_instrument_saturation
  • low_representation_saturation

You may use as many of these within the Special_Constants class as are applicable to the field. Say, for example, you have a table of compiled properties, and in one particular integer field there are cases where either the value is unknown, or in some cases the field itself is just not applicable. If you wanted to document that the valid range of values is 100-500, and you are using "-999" to indicate "Unknown", and "-888" to indicate "Not Applicable", you would adding this to your <Field_Character> class:

   <Special_Constants>
     <unknown_constant>-999</unknown_constant>
     <not_applicable_constant>-888</not_applicable_constant>
     <valid_maximum>500</valid_maximum>
     <valid_minimum>100</valid_minimum>
   </Special_Constants>

<Field_Statistics>

OPTIONAL

If you want to include things like extrema, mean value, and such for all the values that occur in this field through all the records in the table, this is the place to do it. This class is identical for all Field types. For details, see Filling Out the Field Statistics Class. Here is a quick list of the field statistics available in this class:

  • local_identifier
  • maximum
  • minimum
  • mean
  • standard_deviation
  • median
  • description

<Group_Field_Character>

OPTIONAL

This class defines a set of Field_Characters and Group_Field_Characters that repeats a given number of times in each record. Group_Field_Characters may be nested.

Note: Unless you have three good reasons, don't use Group_Field_Character in SBN data.

<name>

OPTIONAL

You can use this field to specify a name for the group, if you like.

<group_number>

OPTIONAL

Analogous to field_number for scalar fields, this is a sequential number useful for referencing Group_Field_Character classes at a single nesting level of a complex Record_Character definition.

<repetitions>

REQUIRED

The number of times the complete set of Field_Characters and Group_Field_Characters comprising this <Group_Field_Character> repeats.

Note: The minimum value for this field listed in the data dictionary is one, but it is unlikely a product will pass SBN review unless this value is at least two.

<fields>

REQUIRED

The count of Field_Character classes directly under (i.e., at the first nesting level below) the Group_Field_Character definition. This will be zero if the group contains no Field_Character classes.

<groups>

REQUIRED

The count of Group_Field_Character classes directly under (i.e., at the first nesting level below) the present Group_Field_Character definition. This will be zero if the group contains no nested Group_Field_Character classes.

<description>

OPTIONAL

Here's a place to describe what this grouping of fields represents.

<group_location>

REQUIRED

This is the location of the first byte of the first field of the first repetition of this group relative to the containing Record_Character or Group_Field_Character location. If the group starts at the beginning of the containing Record_Character or Group_Field_Character, this has a value of one. You must specify a unit of "byte" for this value. For example:

    <group_location unit="byte">1</group_location>

This value should be set bearing in mind that it will be used as the base offset for locating each repetition of the group. In other words, the location of the start of the first repetition of the group is group_location + 0*(group_length/repetitions); the location of the start of the second repetition of the group is group_location + 1*(group_length/repetitions), and so on.

<group_length>

REQUIRED

This is the length of the entire group - that is, the length of one repetition of the group, multiplied by the value in the <repetitions> attribute, above. You must indicate a unit of "bytes" for this length. It must be evenly divisible by the value of repetitions.

Note: This value is not validated. Calculate carefully.

Fields and Nested Groups

As in the Record_Character, the Group_Field_Character may contain either Field_Character classes, or Group_Field_Character classes, or both intermixed. Group_Field_Character classes may be nested arbitrarily deeply. The requirements for these data structure classes inside a <Group_Field_Character> are identical to those above.