Difference between revisions of "PDS3 to PDS4 Migration Workshop"

From The SBN Wiki
Jump to navigation Jump to search
m (categories)
 
(7 intermediate revisions by one other user not shown)
Line 1: Line 1:
Links to various bits and pieces needed for the September 2012 PDS3 to PDS4 migration workshop:
+
This page contains files needed prior to the introductory label-writing workshop held 2012-09-04 at UMCP, as well as some follow-up.  The workshop lasted 2 hours. We covered basic, getting-started type tasks to gain familiarity with tools and jargon.
 +
 
 +
== Workshop Goals ==
 +
 
 +
 
 +
#Giving the participants a chance to work with a validating, schema-aware XML editor (in this case, ''eclipse'');
 +
#Developing a broad, general sense of where the complexities are in migrating PDS3 labels (largely in the meta-data, rather than the data structures); and what things need to be addressed before migration of any particular data set (or data sets generally) can begin.
 +
#Providing a superficial introduction to the PDS core document set, in particular the master XSD schema and the Data Dictionary document, as reference documents
 +
 
 +
 
 +
== Pre-workshop Downloads ==
 +
 
 +
Here's a link to a zipped package containing all the downloadable files listed below:
 +
 
 +
:::http://pdssbn.astro.umd.edu/PDS4/WikiSupport/workshop01.zip
 +
 
 +
 
 +
Here are individual links to various bits and pieces needed for creating PDS4-like versions of the PDS3 sample labels:
  
 
:* Data Dictionary Mock-ups - These XML Schema files can be used to try out validation across multiple namespaces:
 
:* Data Dictionary Mock-ups - These XML Schema files can be used to try out validation across multiple namespaces:
:** [http://pdssbn.astro.umd.edu/PDS4test/WikiSupport/sbnDD.xsd SBN Discipline Dictionary Mock-Up] (use ''Save As'' to download the file)
+
:** [http://pdssbn.astro.umd.edu/PDS4/WikiSupport/sbnDD.xsd SBN Discipline Dictionary Mock-Up] (use ''Save As'' to download the file)
:** [http://pdssbn.astro.umd.edu/PDS4test/WikiSupport/imgDD.xsd Imaging Discipline Dictionary Mock-Up] (use ''Save As'' to download the file)
+
:** [http://pdssbn.astro.umd.edu/PDS4/WikiSupport/imgDD.xsd Imaging Discipline Dictionary Mock-Up] (use ''Save As'' to download the file)
 
:* PDS Schema Files - these appear to be the latest versions available:
 
:* PDS Schema Files - these appear to be the latest versions available:
 
:** [http://pds.jpl.nasa.gov/pds4/schema/develop/pds/PDS4_PDS_0910B.xsd PDS4_PDS_0910B.xsd] - XML Schema, used for validation (use ''Save As'' to download the file)
 
:** [http://pds.jpl.nasa.gov/pds4/schema/develop/pds/PDS4_PDS_0910B.xsd PDS4_PDS_0910B.xsd] - XML Schema, used for validation (use ''Save As'' to download the file)
Line 9: Line 26:
 
:** [http://pds.jpl.nasa.gov/pds4/doc/dd/v09/pds4datadictionary_abridged.pdf PDS4 Data Dictionary] - a PDF file that you'll probably want to keep open for searching while you're editing to look up attribute definitions
 
:** [http://pds.jpl.nasa.gov/pds4/doc/dd/v09/pds4datadictionary_abridged.pdf PDS4 Data Dictionary] - a PDF file that you'll probably want to keep open for searching while you're editing to look up attribute definitions
 
:* [http://pds.jpl.nasa.gov/pds4/doc/index.shtml Other PDS4 Documentation] maintained at PDS/EN
 
:* [http://pds.jpl.nasa.gov/pds4/doc/index.shtml Other PDS4 Documentation] maintained at PDS/EN
 +
 +
 +
Here's the data set we'll be using as our source:
 +
 +
:* [http://pdssbn.astro.umd.edu/holdings/ear-c-i0039-2-sbn0007_keckiiesi-v1.0/ ear-c-i0039-2-sbn0007_keckiiesi-v1.0]
 +
 +
We'll start with this table label:
 +
 +
:* [http://pdssbn.astro.umd.edu/holdings/ear-c-i0039-2-sbn0007_keckiiesi-v1.0/data/wild2astrom.lbl data/wild2astrom.lbl]
 +
 +
And, time permitting, move on to this image label:
 +
 +
:* [http://pdssbn.astro.umd.edu/holdings/ear-c-i0039-2-sbn0007_keckiiesi-v1.0/data/2003dec19/processed/esi0012.lbl data/2003dec19/processed/esi0012.lbl]
 +
 +
You'll probably also want to reference these catalog files:
 +
 +
:* [http://pdssbn.astro.umd.edu/holdings/ear-c-i0039-2-sbn0007_keckiiesi-v1.0/catalog/dataset.cat catalog/dataset.cat]
 +
:* [http://pdssbn.astro.umd.edu/holdings/ear-c-i0039-2-sbn0007_keckiiesi-v1.0/catalog/obs378t2.cat catalog/obs378t2.cat]
 +
:* [http://pdssbn.astro.umd.edu/holdings/ear-c-i0039-2-sbn0007_keckiiesi-v1.0/catalog/i0039.cat catalog/i0039.cat]
 +
 +
 +
== Post-workshop Results ==
 +
 +
We worked through the first sample label (the ASCII table label) to the point of beginning the process of documenting the individual columns.  Some of the details in the ''Observation_Area'' class were ellided, and the ''Discipline_Area'' was covered ''very'' quickly, mainly with reference to what is going to be needed to create a discipline dictionary.
 +
 +
Here is the completed label based on the the schemas we were using in the workshop: [http://pdssbn.astro.umd.edu/PDS4/WikiSupport/wild2astrom.xml wild2astrom.xml]
 +
 +
Few things to note:
 +
* If you load this up and validate it, you will see one validation error remaining in the <code>reference_type</code> of the observatory definition (look for the string "???" in line 56).  The "???" are standing in for values that should exist but don't.
 +
* I finished this label off in a different XML editor, so to get the Schematron validation working you'll need to change the <code><nowiki><?xml-model?></nowiki></code> processing instruction '''href''' to point to "Schema/PDS4_PDS_0910B.sch", as in the original examples.  (Quit complaining - it's good practice!)
 +
* The <code><nowiki><Primary_Result_Summary></nowiki></code> class has been added at line 90 to provide broad classification information for the data product.  This class is actually in the wrong place - it should occur much higher in the label.  That problem will be fixed in the next release of the test schema.
 +
 +
The label is not quite complete, but the structures needed to complete it are missing.  There should be an ''sbn:Primary_Result_Details'' class in the discipline area (around line 86), in particular, to provide a more detailed description of the result type as a set attributes still TBD.  This is the sort of thing I need scientific input on, not just on a data set by data set basis, but to develop a system to be used across the SBN.  For example, this is where we would note that the images were taken through a Cousins R filter, and that the astrometry is given as a function of aperture radius (if that's a significant discriminator for selecting data and what I just said makes any sense).
 +
 +
 +
 +
[[Category:Workshop Support]]
 +
[[Category:Migration]]

Latest revision as of 15:51, 3 August 2017

This page contains files needed prior to the introductory label-writing workshop held 2012-09-04 at UMCP, as well as some follow-up. The workshop lasted 2 hours. We covered basic, getting-started type tasks to gain familiarity with tools and jargon.

Workshop Goals

  1. Giving the participants a chance to work with a validating, schema-aware XML editor (in this case, eclipse);
  2. Developing a broad, general sense of where the complexities are in migrating PDS3 labels (largely in the meta-data, rather than the data structures); and what things need to be addressed before migration of any particular data set (or data sets generally) can begin.
  3. Providing a superficial introduction to the PDS core document set, in particular the master XSD schema and the Data Dictionary document, as reference documents


Pre-workshop Downloads

Here's a link to a zipped package containing all the downloadable files listed below:

http://pdssbn.astro.umd.edu/PDS4/WikiSupport/workshop01.zip


Here are individual links to various bits and pieces needed for creating PDS4-like versions of the PDS3 sample labels:


Here's the data set we'll be using as our source:

We'll start with this table label:

And, time permitting, move on to this image label:

You'll probably also want to reference these catalog files:


Post-workshop Results

We worked through the first sample label (the ASCII table label) to the point of beginning the process of documenting the individual columns. Some of the details in the Observation_Area class were ellided, and the Discipline_Area was covered very quickly, mainly with reference to what is going to be needed to create a discipline dictionary.

Here is the completed label based on the the schemas we were using in the workshop: wild2astrom.xml

Few things to note:

  • If you load this up and validate it, you will see one validation error remaining in the reference_type of the observatory definition (look for the string "???" in line 56). The "???" are standing in for values that should exist but don't.
  • I finished this label off in a different XML editor, so to get the Schematron validation working you'll need to change the <?xml-model?> processing instruction href to point to "Schema/PDS4_PDS_0910B.sch", as in the original examples. (Quit complaining - it's good practice!)
  • The <Primary_Result_Summary> class has been added at line 90 to provide broad classification information for the data product. This class is actually in the wrong place - it should occur much higher in the label. That problem will be fixed in the next release of the test schema.

The label is not quite complete, but the structures needed to complete it are missing. There should be an sbn:Primary_Result_Details class in the discipline area (around line 86), in particular, to provide a more detailed description of the result type as a set attributes still TBD. This is the sort of thing I need scientific input on, not just on a data set by data set basis, but to develop a system to be used across the SBN. For example, this is where we would note that the images were taken through a Cousins R filter, and that the astrometry is given as a function of aperture radius (if that's a significant discriminator for selecting data and what I just said makes any sense).