Difference between revisions of "Python PDS4 Tools"
m (Clarified table reading requirements)
|Line 77:||Line 77:|
pds4_viewer: [http://www.numpy.org/ NumPy]
pds4_viewer: [http://www.numpy.org/ NumPy]
You may use <tt>pds4_read</tt> to read-in data without any extra packages, only
You may use <tt>pds4_read</tt> to read-in data without any extra packages, only <tt>pds4_viewer</tt> requires additional packages.
==== Optional ====
==== Optional ====
Revision as of 08:19, 31 July 2015
- 1 Introduction
- 2 Reading PDS4 Images
- 3 Reading PDS4 Tables
This document describes the current status and usage of Python tools developed at PDS-SBN to read and visualize PDS4 data in Python. Please note that a more feature-complete PDS4 reader and visualizer for IDL is also available.
Reading PDS4 Images
This section describes an example Python module that can read and display an image from a BOPPS/BIRC PDS4 data product. The code will read the data based on the label keywords, but does not otherwise validate the label. If the user wants to display the image, the code will consider the label's Display_Settings, and provide a copy of the image in the correct orientation for drawing with the origin in the lower left corner (which can then be displayed with Matplotlib).
The code below is specifically designed for reading BOPPS/BIRC images, but can be used as an example for other limited problems. A more general solution may use a different approach, and will be developed by SBN in the future.
Contact Mike Kelley with questions or comments regarding this code or its description.
Goal and Method
The goal is to read in an image from a BOPPS/BIRC data product into a Numpy array, providing the correct orientation for display. We will provide a function with the name of the label, the function will then
- Open the label.
- Find the data product file name.
- Determine the Array_2D_Image data type and shape.
- Read in the data array.
- Return the array and meta data in a single object.
The object will have two attributes that allow access to the data
- the data with the axis order and orientation as provided in the file, and
- the data with the axis order and orientation reconfigured according to the label's Display_Settings class, so that it will have the correct orientation if drawn with the origin in the lower left corner.
For this basic example, we designed the reader as a function in a module named birc_example_reader. The user calls a single function, read_image(), passing the name of the label as the first argument. The function will load the label using the ElementTree module and find the first Array_2D_Image element to read in. A second function, read_pds4_array(), determines the correct data type and shape, then reads the data from the file. A class specifically designed for PDS4 Array_2D_Image objects, aptly named PDS4_Array_2D_Image, is initialized with the data, the label describing the data, and the local_identifier of the array. The local_identifier is not normally required in PDS4 array objects, but it must be present when the image display orientation is provided via Display_Settings. Since these are present in the BIRC labels, our class assumes local_identifier is included. The class then determines the image orientation. The image is stored as a class attribute data. The class attribute display_data is also provided, which can be used for displaying with Matplotlib.
To read and display a BOPPS/BIRC image:
import birc_example_reader as birc import matplotlib.pyplot as plt # array is im.data # array for displaying is im.display_data im = birc.read_image('cerh2_1_010000_rb_n169_n011.xml') plt.clf() plt.imshow(im.display_data, origin='lower') plt.draw()
Minimal Working Example
Below we provide a minimal working example with the same basic functionality. The example is a flat script with extensive comments, which may more clearly illustrate some of the methods for working with PDS4 image labels.
Reading PDS4 Tables
This section describes a Python package that can read and display PDS4 table data. In the future this tool is expected to support all PDS4 objects. The package expects labels that pass PDS4 Schema and Schematron validation.
Contact Lev Nagdimunov with questions or comments regarding this code or its description.
Python 2.6 or 2.7.
You may use pds4_read to read-in data without any extra packages, only pds4_viewer requires additional packages.
The following is a list of supported PDS4 data objects:
Table_Binary: Full, except BitFields, SignedBitString and UnsignedBitString
No other PDS4 data objects are currently supported.
Download the ZIP file File:PDS4 tools-0.2.zip
Extract the downloaded file to a directory Python can find. To use it follow the instructions in Example Usage except with the following lines first,
import sys sys.path.extend(['/path/to/your/extraction/directory']) # On a windows machine use backslashes (/) instead of windows' normal forward slashes to specify paths
Use "pip install PDS4_tools-0.2.zip" or "easy_install PDS4_tools-0.2.zip". You can also extract the ZIP file and use "python /path/to/extracted/setup.py install". Note that there is no uninstall script provided (although "pip uninstall pds4_tools" should work), and that this tool currently has many missing features and will be updated in the future.
You may call pds4_read from command line or from your own script. Typing the shell command python pds4_read.py -h prints out the help text:
usage: pds4_read.py [-h] [--quiet] [--use_numpy] [--object_num OBJECT_NUM] [--object_name OBJECT_NAME] [--object_lid OBJECT_LID] filename positional arguments: filename Filename, including full path, of the label optional arguments: -h, --help show this help message and exit --quiet Suppresses all info/warnings --use_numpy Returned data will be a numpy array and use numpy data types --object_num OBJECT_NUM Only reads the data object specified by zero-based order (integer) --object_name OBJECT_NAME Only reads the data object specified by name --object_lid OBJECT_LID Only reads the data object specified by local identifier
If called from another module or a script, all of the above optional arguments would be available as optional named parameters of the function pds4_read(). Basic example usage is as follows:
""" Basic pds4_read example """ from pds4_tools import pds4_read obj_list = pds4_read('/path/to/label.xml') table = obj_list['table_name'] # or table = obj_list # Dictionary-like access column = table.data['field_name'] row_1_to_100 = column[0:100] # List-like access column = table.data row_1_to_100 = column[0:100] # Meta-data access column_meta = table.data.meta_data('field_name') column_meta = table.data.meta_data(0) print column_meta['description'] print column_meta['unit'] # Label access, provides ElementTree object label = obj_list.label # Full label label = table.label # Label section describing the table object
To display the objects in a label you may call pds4_viewer from the command line:
usage: pds4_viewer.py [-h] [--quiet] [--object_num OBJECT_NUM] [--object_name OBJECT_NAME] [--object_lid OBJECT_LID] [filename] positional arguments: filename Filename, including full path, of the label optional arguments: -h, --help show this help message and exit --quiet Suppresses all info/warnings --object_num OBJECT_NUM Only reads the data object specified by zero-based order (integer) --object_name OBJECT_NAME Only reads the data object specified by name --object_lid OBJECT_LID Only reads the data object specified by local identifier
It is not necessary to include the filename parameter for pds4_viewer, you may simplify call it without any options or arguments and a GUI will open from which you can open labels.
You may also call pds4_viewer from another module or script. All the above optional arguments are available as optional named parameters. A basic example usage is as follows:
""" Basic pds4_viewer example """ from pds4_tools import pds4_read, pds4_viewer pds4_viewer() # or pds4_viewer('label.xml') # or obj_list = pds4_read('label.xml') pds4_viewer('label.xml', from_existing_objects=obj_list) # Won't re-read the data