spacer link to MAST page spacer logo image spacer
 
link to STScI page


STScI's Internet connections will be UNAVAILABLE Saturday, 28 October 2017 from 7:00 AM - 1:00 PM. STScI web servers, SSH and similar services will be unreachable from the Internet.

MAST HLSP Models/Simulations Data Delivery Standards and FITS Keyword Information

Summer 2013

Theoretical data deliveries must be closely related to MAST data and/or generally not available through another permanent archive. This page describes FITS models/simulations data standards for HLSP deliveries. Also please read the general HLSP guidelines page. more | less

We prefer all models/simulations to be delivered in FITS format whenever possible. We have tried to generalize the FITS keywords requirements across any number of models/simulations data types, therefore some of these keywords may not be applicable to your data. We suggest you consult a MAST staff member prior to delivery, to be sure your data abides by the FITS and MAST delivery standards.

MAST has put together the following information because it is essential for data deliveries. If the archive cannot extract the information they need from dataset headers, MAST will attempt to calculate it based on the information provided (in the data and README). If MAST is not able to extract or calculate what it needs based on the delivered dataset headers, then some of the data will not be searchable and will not be displayed across multi-mission search results.

[Overview]   [General Header Information]   [FITS Standards]   [File Names]   [File Types]  

[Keyword Nomenclature]   [Units]   [README File]   [Superseded Results]   [Examples]  

Overview

HLSP models/simulations data deliveries are preferred to be in FITS format (N-dimensional image or FITS BINARY table) whenever possible (tabular data can be delivered in ASCII format). The following types of data are considered models/simulations; the links provide information on observational data requirements per data type; we suggest reading the requirements and following the rule set as close as possible when applicable:
  • Simulated Images - 2D, 3D, N-D images, movies and animations
  • Simulated Spectra - 1D (flat) or 2D (images of spectra), Spectral Energy Diagrams (SED) Models
  • Simulated Catalogs - Tabular data (e.g. a source lists, surveys, mock catalogs or color-magnitude data)
  • Simulated Time-Series - Tabular data where one axis is TIME
  • Simulated Spectral Line Lists - Compilation of identified spectral lines and their wavelength properties.
Observational data requirements can be quite different than specifications for theoretical data. We outline some of these differences below.

General Header Information

MAST requires that FITS headers contain the following information:
  • Relevant ID information (program ID, proposal ID, PI name, etc) if appropriate
  • Time coverage (start and duration) if appropriate
  • List of exposures (origin of datasets, only needed for MAST Missions) if appropriate
  • Pointing location and World Coordinate System (WCS) if appropriate
  • Wavelength coverage if appropriate

FITS Standards

The above general requirements translate into FITS header keywords. These keywords and files themselves must abide by the FITS standard, and therefore cannot have errors when being checked by tools like 'fverify' or 'fitsverify'. more | less

File Names

The names of the files should follow MAST's HLSP file name definitions when possible. These definitions are in place to make your contributed HLSP able to be integrated with the rest of MAST's holdings, including the Hubble Legacy Archive. For full details, visit the Filenames section in the HLSP main page, and expand for full details of how each field in the file name should be defined. Since many of the field-names are not applicable for models/simulations, we only require the use of underscores to separate fields and at minimum the three fields: hlsp_project_*_version_*.extension. more | less

We realize that difference projects have very different data, we suggest using the "mission" or "product" naming slot to indicate that these data are simulations/models by using text such as "modelmap", "theory", "sim", "model", "map" and "SED" within the file name.

All delivered files must have unique names (no duplicate file names). The naming convention used should be documented in the README file. As an example, see the DIGGSS project naming scheme by following any link under the 'Name' column.

File Types

Due to the variety of astronomical simulated data, the types of products can be vast. MAST will accept 1-dimensional through N-D data "cubes", as well as tabular data. The FITS file format is preferred for all data types, but ASCII is accepted for tabular data. We will accept multi-extension FITS files, N-D data "cubes" and FITS BINARY (or ASCII) tables. more | less

Data need to be correctly described - tabular axes have to abide by the FITS standards; data should have either a physical, logical or a World Coordinate System (WCS) using header keywords (see IRAF help pages for a description of coordinate systems). Metadata needs to be provided as header keywords. A listing of required, recommended and optional header keywords are provided for observation data types (see: images, spectra, catalogs, time-series and spectral line lists). Please use appropriate keywords for models/simulations that best match your data type. For questions, please contact MAST.

Please provide data previews whenever possible in JPEG, GIF or PDF format. Animations are also accepted and can be delivered in any format as long as they can be played on a standard player such as realPlayer, Quicktime or VLC player.

Header Keyword Nomenclature

For single images, the keyword will contain a value; if the image is a composite, that keyword can contain the value of "MULTI", followed by a list of N similar keywords which encompass all the values needed to explain the composite data using a series of related keywords. more | less

For example, the single image can have FITS keyword 'FILTER' = "g". For a composite image made of UBV exposures, the keys would be as follows:

  • FILTER = "MULTI"
  • FILTER01 = "U"
  • FILTER02 = "B"
  • FILTER03 = "V"

To designate multiple keywords, we denote this as [nn].

Keyword Value Units

Some keywords have standard units and do not need to be explicitly specified in the headers. In cases where you need to specify units for header keywords, this can be done in one of two ways: more | less

  1. KEYWORD + KEYWORD_UNIT pair
  2. You can specify the keyword units using a second, similar keyword where the keyword name contains the string "_UNIT" or "UNIT". Please remember to stay within the FITS 8-character keyword name limit. E.g.:

    WAVECENT = 1688 / central wavelength
    WAVEUNIT = angstrom / central wavelength units

  3. KEYWORD = value / [unit] in brackets within the FITS comment
  4. Comments can contain the keyword unit within brackets; this should be the first text following the standard fits comment delimiter "/", the single slash. Please remember to stay within the FITS 80-character line length limit, which includes the comments. E.g.:

    RA_TARG = 2.0952854139 / [deg] right ascension of target

README File Contents

It is vital to have a comprehensive README file when delivering HLSP models/simulations. A reference to the "in press", submitted or published peer-refereed journal should accompany the data. Please provide information such as who created the data, what type of data are being delivered (images, spectra, catalogs, etc.) and what type of objects or science is encapsulated in these files (single galaxy, mock deep field, lensing map, etc.). We also need step-by-step instructions/description of how to read the files. The documentation needs to have information on how to validate, use and view the data. more | less

MAST will not store the software which was used to create the models or simulations; this may be done by the data provider using services like the Astrophysics Source Code Library. The README should contain instructions on how to reproduce the data based on a model (pointer to the software when possible/available, the version used and parameters/settings/configuration). Software to create previews may be accepted in the event that MAST can use it on-the-fly to create dynamic data previews; but, such software will not be maintained long-term by MAST. Suggestions on data visualization software tools is highly valuable information in the README.

If the data are associated with existing observational data already part of the MAST archive, please provide a file listing all the associated datasets/exposures. If MAST does not have the related datasets, we will accept these associated data for ingest; these should be described within the README file. Observational data deliveries need to follow more rigorous MAST HLSP guidelines per data type, please see requirements for: images, spectra, catalogs, time-series and spectral line lists.

Superseded Results (when models become out of date)

Models and simulations may become out of date. It is up to the data provider to contact MAST and provide improved/updated results. If data are not superseded by the data provider, they will be subject to review nominally every 3-5 years. MAST will contact the PI to inquire whether better models exist since the previous delivery; models can be replaced with more up to date results; the file version number must be incremented when delivering updates to archived data.

Back to top


HLSP Models/Simulations Examples

Expand for Examples | Retract/Close Examples

MAST has been ingesting and distributing HLSP data products for over 10 years. During this period, the requirements for HLSP data deliveries have expanded in order to help unify all datasets housed at MAST for ease of multi-mission searching. The example HLSP headers may not abide by all the requirements listed above because they were delivered prior to some requirements being written. We encourage the data delivery teams to provide data sample so that all header and data issues can be worked out prior to the actual delivery for ingestion into the archive.

The following project contains example simulation datasets:

  • FITS Image, 2D Simulated Image Data
    From the DIGGSS Project:
    • DIGGSS Image Header (plain text hlsp_diggss_theory_g3g3Pt-cam0-t401_g_v1_seg.fits header)
    • DIGGSS FITS File (hlsp_diggss_theory_g3g3Pt-cam0-t401_g_v1_seg.fits from plain text header example)

Back to top


Back to general HLSP guidelines