There are four basic ways or methods of working with (including reading and writing) HDF files. These include two levels of programming interfaces within the HDF library, a set of command line utilities also contained in the HDF library, and a wide range of browsing and visualization software provided by both commercial vendors and non-profit organizations (NCSA, for example). Further detail on each method is given below:

• Low-level interface
• High-level interface (APIs)
• Command line utilities
• HDF browsing and visualization tools/software

Both the command line utilities and the browsing and visualization tools provide easy-to-use methods for HDF non-experts to work with HDF files. As shown above, the use of the command line utilities is rather straight forward. However, neither the command line utilities nor tools provide the user with the flexibility and means of working with the HDF files in such an encompassing fashion as permitted in the High-level APIs. For this reason, as well as the fact that information and directions regarding the use of the HDF tools are better provided by the Internet sites linked above, the following sections of the tutorial will mainly concentrate on using the APIs to work with HDF files.

Return to Main Topics

The low-level interface is mainly reserved for expert HDF programmers and software developers who are interested in not only reading and writing HDF files, but also such features as error handling, memory management, and storage. A lot of the features in this interface are unnecessary for the novice HDF user. Another drawback is that routines/operation callable through this interface are only available in C and not FORTRAN.

Return to top

In this interface, Application Programming Interfaces (APIs) are specifically tailored for each type of data (Images, Scientific Data arrays, etc.) supported by the HDF library. These APIs are callable routines which will allow the user to access, read and write HDF files specifically for the type of data they are interested in. Although it is necessary for the call of these APIs and associated routines to occur in either a C or FORTRAN program, the programming is usually limited to a set of call statements that access, open, operate (read, write, etc.), and terminate. All of the rest is taken care of by the interface itself. With its' availability in both C and FORTRAN, the minimal amount of programming necessary, and the independent APIs and routines for each data type, the High-level interface provides a relatively simple way for the average programmer and novice HDF users to work with HDF files.

Available APIs

Rather then providing material on all of the high-level interface APIs, we have chosen in this tutorial to use the Multi-dimensional array, multi-file interface (SD API) as an example to teach the novice how to use HDF. For detailed information on the other APIs, all of which are used in similar fashion to the SD API, the reader of the tutorial is directed to the documentation identified in Section 2 - Where can I get additional and detailed information on HDF?

Return to top

One method of working with HDF files is through command line utilities during a terminal session. Entered just like other terminal commands (such as UNIX), command line utilities allow the user to call up HDF application programs outside of formal C and FORTRAN programs and view, manipulate and convert HDF files.

List and description of command line utilities

Although the command line utilities permit the user to perform common operations on HDF using a simple one-line command, the main drawback to this method is the limited number of operations supported.

Return to top

A complete and current listing of the browsing and visualization tools that can work with HDF files is provided on the NCSA HDF Home Page. There are both publicly available (free) and commercial software packages that can be used to work with HDF files. A summary of some of the more useful and commonly-used software, including the address of the internet site/home page where the software may be accessed, is provided below:

• Publicly Available Software
• Commercial Software
• Contributed Software

Return to top

Through the High-level interface, the HDF library provides APIs and associated routines for all the data types supported by HDF; 8- and 24-bit raster images, palettes, scientific data arrays, metadata (Annotation), multivariate data stored as tables (Vdatas), and EOS Scientific Data (point, swath, and grid APIs contained in the HDF-EOS sub-library). In addition, there are separate APIs and routines for multi-file data sets of the various types. Each API is independent of the others and is identified by a certain prefix (different for both the FORTRAN and C program version) which is assigned to all the function calls employed by the user in his/her program for that specific data type.

The following is a list and short description of the various APIs with the C and FORTRAN prefaces for each interface given in parenthesis:

• SD API (SD/sf): For scientific data sets (multi-dimensional arrays together with a record of dimension and number type). The SD API is used to store, manage and retrieve multi-dimensional arrays (integer or floating point decimal), including their dimensions and attributes in more than one file.
• GR API (GR/mg): The GR API is used to store, manage and retrieve general raster image data sets, including their dimensions and palettes. However, unlike the DF24 and DF8 APIs, this information can be in more than one file. In addition, the GR API can also manage unattached palettes.
• VS API (VS/vsf): The VS API is used for reading and writing customized tables which are stored in fixed length fields (Vdata).
• V API (V/vf): The V API is used to create, group, and manipulate primary HDF objects in a file (Vgroup).
• VSQ API (VSQ/vsq): The VSQ API is used for querying or obtaining information on vdatas. This includes the number of records, names, and number types.
• VF API (VF/not available): The VF API can be used for obtaining information on the fields in an existing vdata.
• AN API (AN/af): The AN API is used to store, manage, and retrieve text strings used as metadata to describe the data file itself or any of the data elements inside the file.

• DFSD API (DFSD/ds): The DFSD API is similar to the SD API, but only operates on one single file.
• DFR8 API (DFR8/d8): The DFR8 API is used to store, manage and retrieve 8-bit raster images, along with their dimensions and color palettes. This information is all included in one file.
• DF24 API (DF24/d24): The DF24 API is used to store, manage and retrieve 24-bit raster images, including the dimensions of the image. This information is also included in one file.
• DFP API (DFP/dp): The DFP API is used to store and retrieve 8 bit palettes in one file.
• DFAN API (DFAN/da): The DFAN API is used for reading and writing text string (metadata) assigned to HDF files or objects.

• PT API (PT/pt): The PT API is used for storing, retrieving, and manipulating data in point data sets. These data have associated geolocation information, but are not organized in a spatial or temporal fashion. The PT API is part of the HDF-EOS sub-library.
• GD API (GD/gd): The GD API is used for storing, retrieving, and manipulating data that has been stored in a rectilinear array based on a defined map projection. The GD API is part of the HDF-EOS sub-library.
• SW API (SW/sw): The SW API is used for storing, retrieving, and manipulating time-ordered data sets such as satellite swath data. The SW API is part of the HDF-EOS sub-library.

Return

HDF Command line utilities can be executed at the command level (prompt) similar to UNIX. The following is a list of some of the command-line utilities available in the HDF library:

1. hdp - displays contents and data objects within an HDF file
2. hdf24to8 - converts 24-bit raster images to HDF 8-bit images
3. hdf8to24 - converts 8-bit raster images to HDF 24-bit images
4. hdfcomp - re-compresses an 8-bit raster HDF file
5. hdfls - lists basic information about an HDF file
6. hdfpack - compacts an HDF file
7. hdfunpac - unpacks an HDF file
8. hdftopal - extracts a pallete from an HDF file
9. hdftor8 - extracts 8-bit raster images and palettes from an HDF file
10. hdfed - HDF file editor
11. paltohdf - converts a raw palette to HDF
12. r8tohdf - converts 8-bit raster image to HDF
13. ristosds - converts a series of raster image HDF files into an HDF file
14. vshow - dumps out vsets from an HDF file
15. jpeg2hdf - converts jpeg images to HDF raster images
16. hdf2jpeg - converts HDF raster images to jpeg images
17. fp2hdf - converts floating point data to HDF floating point format and to HDF 8-bit raster image format
18. vmake - create Vset structures from ASCII text

The hdp command line utility is a very helpful operator, especially for the average HDF user. HDP can list the contents of HDF files at various levels and with different details. It can also dump the data of one or more specific objects in the file.

Return

Freely available software for viewing and browsing HDF files have been developed by both NCSA and various other institutes, science or data centers, and businesses. We have broken these tools down into three categories:

• Current NCSA Tools

• Older NCSA Tools (not updated to run with latest version of HDF)

• Non-NCSA Tools

Return

1. The NCSA Java-based HDF Viewer (JHV) - Java based tool that allows the user to view the contents of an HDF file.
2. The HDF WWW Scientific Data Browser - an interface program that reads HDF files by accessing the HDF library and can visualize or format the data (in HTML) on the web.
3. The Java HDF Server (JHS) - The java based program that calls the HDF library through the Java interface and can access remote HDF files.

1. NCSA Collage - Collaborative visualization program
2. NCSA Polyview - Visualization and analysis of HDF files
3. NCSA Reformat - Converts to and from HDF files
4. NCSA X DataSlice - Manipulates 3-D HDF images

The following are just a few of the tools that have been developed independently from NCSA, but are still available in the public domain. A more comprehensive list of available software can be found at NCSA's HDF Software page.

1. LinkWinds - A visual data analysis and exploration system designed to rapidly and interactively investigate large multivariate data sets (including HDF and HDF-EOS format).
2. The Data and Dimensions Interface (DDI) - Can extract, read, write and visualize large data sets in HDF format.
3. hdfv - An HDF read-only interface that is an HDF viewer with a GUI. Only supports vgroup/Vdata data types.
4. Data Explorer - General purpose software package for data visualization and analysis. The data may be imported from HDF format.
5. SHARP - A viewer for MODIS Airborne Simulator (MAS) HDF data
6. SciAN - Scientific visualization and animation package.
7. VCS - Facilitates the selection, manipulation and display of scientific data. Supports the HDF format for both reading and writing.
8. EOSView - An HDF file verification tool that allows the display of most HDF and HDF-EOS data types.
9. The Data and Information Access Link (DIAL) - A server which provides tools for the searching, browsing, and visualizing of HDF and HDF-EOS files through the WWW.
10. HDFLook - A viewer used to access and view HDF and HDF-EOS files, particularly raster images and scientific data sets.
11. IRI/LDEO - A climate data library that helps in the writing of HDF files and the management of data sets.
12. Webwinds - A platform independent system written in java that acts as an interactive visualization tool for data in HDF and HDF-EOS format.
13. view_hdf - A visualization tool written in IDL that was developed by NASA LARC that provides for the accessing, viewing, plotting, and manipulation of HDF datasets.
14. Rocketeer - Visualizes 3D scientific data sets on many different grids and can read data stored in HDF.
15. VisAD - A Java library that allows for the interactive visualization and analysis of HDF and HDF-EOS files.

Return

Below is a partial list of some of the more powerful and more commonly used commercial software packages for working with HDF files. Once again, a more complete list can be seen at NCSA's HDF Software page.

1. AVS5/AVSExpress - Can read and write files in HDF format. Also includes a suite of data visualization and analysis techniques/tools (3-D visualization, plots, etc...).
2. IDL - A software package for the analysis and visualization of data. Includes advanced image processing, interactive 2-d and 3-D graphics, and flexible date input/output.
3. Noesys - A desktop software program specifically designed to easily access, view, analyze and archive data in the HDF format.
4. Plot - A package that can read, analyze and plot HDF data sets of column data using Windows, Macintosh and UNIX. Plot is included as part of the Noesys package, as are T3D and Transform, two other tools used to visualize, plot and transform HDF and HDF-EOS files.
5. HDF Explorer - A visualization program that reads and views data sets in HDF format
6. MATLAB - Allows for the visualization and analysis of data in HDF format.
7. Tecplot - Interactive data analysis and visualization software that can read and write HDF data and files.

Return

In addition to the above-mentioned software, also available from the NCSA anonymous ftp server is a collection of software routines and utilities developed by HDF users who wish to share their knowledge and work with the HDF community. These software can be found in many of the subdirectories of the main pub/hdf/contrib/ directory of the NCSA ftp server. Most of these "contributed" routines were developed with specific platforms and operating systems in mind.

Below are a few of the many examples:

• readDF - reads HDF files into IRIS Explorer
• fits2hdf - converts FITS files (another format) into HDF
• iristohdf - converts SGI image format to HDF format
• hdfxdis - directly displays HDF image on an X-server
• hdiff - compares attributes of two different hdf files
• AVStoHDF - converts AVS images to HDF format
• hdfinfo.c - describes the organization of an HDF file
• InspectHDF - Displays information on HDF files
• iristohdf - converts SGI image format to HDF format

These routines together with the name and address of the developer are free and publicly available to all interested users of HDF.

Return