NOTE: Most of the High Level Science Products are unavailable while
unscheduled maintenance is being performed. They will be incrementally
restored over the course of this week. We apologize for any inconvenience.
NASA Data Center Annual Program Plan
Multi-mission Archive at Space Telescope (MAST)
(Optical/UV Science Archive Research Center)
Space Telescope Science Institute
3700 San Martin Drive
Baltimore, MD 21218
MAST supports active and legacy mission datasets and related catalogs and surveys,
focusing primarily on data in the ultraviolet, optical, and near-IR spectral
Support includes curation of the data, providing expert support to users of the data,
providing access to data-specific calibration and analysis software, providing user support
for this software, and maintaining public access interfaces to the data. MAST works with new
mission teams in the supported wavelength regions to assist in the development of data management
plans, especially in the areas of data formats, descriptive metadata and standardization of keywords,
in the development of data access and delivery plans, and assuring data quality control.
Total MAST Holdings by volume
MAST holdings without HST/GSC/DSS
MAST Data Holdings
Number of Observations
Active Mission Duration
8.25 GB (compressed)
Sept. 1993; Nov. 1996
Sept. 1993; Nov. 1996
Dec. 1990; March 1995
Dec. 1990; March 1995
Dec. 1990; March 1995
IUE Final Archive
CATALOGS & SURVEYS
183.98 GB (compressed)
Digitized Sky Surveys
GSC I, II
Services Provided: MAST provides support for
users seeking to understand the properties and instrumental signatures
of all archived datasets and assistance with the interfaces
to browse and retrieve these data. Access to non-HST mission
and instrument specific calibration and analysis software and assistance
in its use continues on a time-available basis. Full support
for HST related software is provided by the MAST Help Desk and
Non-HST Data Analysis Software Provided: IUE "RDAF"
package (IDL-based), IUE Final Archive processing software (IRAF port),
EUVE analysis software package (IRAF-based), Copernicus data
analysis software (IDL-based), UIT data reduction/analysis software
(FORTRAN, C, and IDL routines), WUPPE data analysis software
(FORTRAN routines requiring the FITSIO library), and HUT data
reduction software (IRAF-based) are available through MAST.
Last fiscal year the contract between MAST and FUSE ended, limiting
MAST's role to continuation of data ingest. Nonetheless, the new DADS v10.3
build of June 2004 permits the delivery of CALFUSE data (with a new
intermediate file to be ingested in late July 2004) with negligible extra
effort from the STScI or MAST groups. MAST continues to respond to ad hoc
requests from the Project, such as providing statistics and viewgraphs for
the FUSE Users' Group, and to participate in periodic planning meetings
to assess requirements pertaining to smooth ingest of FUSE data into DADS
and to respond to requests from the Project for improved interface display
and search tools.
The GALEX archive represents the largest mission dataset supported
by the MAST contract (HST archiving is supported by the primary HST
contract). In preparation for the first (Early Release) Observations,
MAST has scheduled weekly meetings with Caltech and has exchanged
personnel for training sessions and consultations. MAST's ERO made its
debut in early 2004 and staged these data for Cycle 1 of GALEX's GI
program. GALEX data usage (see figure) has a sustained usage
of over 15000 hits/month since January.
In addition to this release, MAST received an "Internal
Release 0.1" dataset and is about to receive data for an IR0.9. These
internal releases have permitted MAST to continue development
of its database in preparation for the first large public release, DR1,
scheduled for October 2004. During the period an Interface Control
document (version 2.1) was written to document data interface protocols
and arrangements between Caltech and MAST.
Number of nonrobotic hits during 2004 (July incomplete)
of MAST's GALEX website. (The darker red indicates the number of page errors.)
Over the last two years MAST has developed a GALEX database using
Microsoft SQLServer and .NET technology. This technology permits the
automated ingest of data, self-documentation, rapid browsing of the
entire database structure, including a listing of tables, keywords and
procedures used for qualified searches of object classes, and listings
of all observations. An active helpdesk is maintained for user questions
pertinent to the database.
The VLA-FIRST archive has been significantly enhanced,
in part through the addition of coadded images. The FIRST database table
and cross-correlation search tool was updated to include the new data.
MAST continues to interface with the CHIPS Project concerning
their archiving plans. This project plans to transfer the CHIPS dataset
and auxiliary tools approximately one year after mission completion, or
about the middle of 2006. (The total data volume will be only 10GB.)
As part of a continuing effort to enhance archive mission interoperability,
MAST held several discussions with the IRSA center to discuss integration
of some of the IRSA datasets into the Scrapbook.
Committee Participation within the STScI:
Levay chaired an ad hoc STScI-wide committee to investigate
solutions to the computer bandwidth problems associated with difficulties
users had in the summer and fall of 2003 in downloading data in a reasonable
timescale. Initially, the need for increased bandwidth was documented by capturing
network usage and summing all network activities for major subgroups at STScI.
This documentation was used to justify to the HST project and to NISN the ISP,
the increase in bandwidth from 11 Mbits/sec to 26 Mbits/sec.
in addition, data going to current Internet 2 subscribers goes through
an Internet 2 router at NISN. While this bandwidth is adequate
currently, the projected growth in OPO services and deliveries of HST ACS data and High-Level
Science Products (mostly related to the HST Treasury programs) Is expected to saturate
the bandwidth during peak use times in about 18 months. Therefore, Levay is participating in
a subsequent committee to investigate the use of another ISP either as a replacement or a supplemental service.
Shiao and Conti participated in an ad hoc Sybase
Alternatives Committee to explore more economical and robust
options to the Sybase database system. A report with recommendations
was sent to Rodger Doxsey, and the announcement of an Institute-wide
database technology is expected soon.
ACTIVITIES AND MAJOR ACCOMPLISHMENTS OF THE LAST YEAR
MAST Data Ingest & Retrieval Activity
The combined active mission (HST, FUSE, GALEX) annual data ingest rate
has maintained the steep increase from last year,
raising the total holdings to 19.74 GB. The continued high rate
from last year was due mainly to the addition of ACS and NICMOS
instruments data from the HST. A jump in the number of retrieved datasets
was due to the download of all HUT and Copernicus datasets in June by
the Korean Astronomical Institute in preparation for the SPEAR mission
Ingest Volume (GB) - Active Missions
Retrieval Volume (GB) - Active Missions
Retrieval Volume (GB) - Legacy Missions
Datasets Retrieved - Active Missions
Datasets Retrieved - Legacy Missions
As MAST does not maintain retrieval statistics for DSS ,
only the number of searches is displayed in order to show the general
interest level in these data.
Data Discovery and Search Tools:
Scrapbook and Pointings Table: FUSE was added to the Spectral/Imaging Scrapbook. New datasets are added to the scrapbook each month from the HST and FUSE observations. ACS and WFPC2 Associations Pointings tables were constructed and added to the pointings interface tool. Data for all the active HST instrument pointings tables were updated weekly.
High Level Science Products:
Several sets of High Level Science Products (HLSP) were delivered and
implemented this year:
GOODS Version 1.0 data were delivered in late August 2003. The associated catalogs were delivered in December 2003.
The stacked, composite image obtained during a search for Kuiper Belt objects was delivered in December 2003.
The Hubble Ultra Deep Field data were released in early March 2004. Anticipating high demand for these data, CADC and ST-ECF set up mirror sites for the UDF data. The UDF data is comprised of 4 distinct sets of data: ACS-WFC, NICMOS, ACS-HRC (parallels to the ACS-WFC), and NICMOS parallels to the ACS-WFC data.
An atlas of FOS spectra of AGN and quasars was delivered in March 2004.
Version 0.5 of the data from the Archival Pure Parallel program was delivered in May 2004.
The page describing the High-Level Science Products holdings was rewritten
to include all HLSP in a single page and to include a new type of search
to help users locate sets of interest. After a couple of years of experience
with HLSP, the Guidelines page for contributing HLSP was simplified and made
more "user friendly".
A complete listing of HLSP hosted at MAST is below. Although MAST
provides an interface to the WFPC2 Associations, the data are
held at CADC. MAST distributes the data via a proxy.
High-Level Science Product Holdings
High Level Science Product Set
Number of Files
10 Lac Spectral Atlas (HST/GHRS)
AGN and Quasar Spectral Atlas
Copernicus Atlas of 6 Selected Stars
EUVE Spectral Atlas of Stars (EUVE)
GOODS: The Great Observatories Origins Deep Survey
Grayscale of Time Variation of gamma Cas Near SiIV Doublet
Hubble Deep Field
Hubble Deep Field South
Hubble Helix Observations
Magellanic Cloud Planetary Nebulae
OB Stars (Galactic): FUSE Spectral Atlas
OB Stars (Magellanic): FUSE Spectral Atlas
Pre-Main Sequence Stars: IUE Spectral Atlas
Procyon (FV-IV) Spectral Atlas
Quasar Spectrum HST/FOS
Search Field from a Search for Kuiper Belt Objects
The Medium Deep Survey
Ultra Deep Field
Ultraviolet Images of Nearby Galaxies
WFPC2 Archival Parallels (Version 0.5)
alpha Ori Spectral Atlas
chi Lupi Spectral Atlas
New plotting and graphical display tools:
Working with the author, Ivo Busko, MAST staff have beta tested and
participated in further development of Specview (a spectral data analysis
tool). This tool is now mature enough for use on MAST pages, but for most
efficient use should (and will) be developed as a web service.
Another new tool provides the ability to plot results from any
MAST mission or VizieR catalog search with a new JAVA-based graphical
display tool called VOPlot (http://vo.iucaa.ernet.in/~voi/voplot.htm).
This new tool allows users to plot the search results from any numerical
field as a histogram, or as a scatter plot against
any other field, as well as to display statistics on the results. (VOPlot
was developed as a part of the Virtual Observatory - India initiative.)
New protocols and IVOA-related services:
During the past year MAST increased activity in the Virtual Observatory
effort. All mission searches are now available as a web service
utilizing the VO Simple Cone Search Protocol. HST previews, UIT data
and previews, VLA-FIRST coadded images and selected High-Level Science Products
(GOODS and UDF) are available as a web service utilizing the VO
Simple Image Access Protocol (SIAP).
A search form has been written to allow MAST users to access SIAP
web services available through the IVOA.
The search form operates on either target name or coordinates.
MAST began work on a web-based image cutout service in conjunction with
the GOODS team. This service will have ``spin-off" applications to other
The GALEX archive website has been implemented as a SkyNode in the
Skyquery portal system developed by JHU as part of the IVOA project.
Several HST catalogs (HDF-N, HDF-S, and GOODS) have also been registered on
Skyquery. In addition, MAST has entered the GALEX cone search and
Simple Image Access Protocol in the US-NVO (IVOA) registry.
New interface pages and search tools:
A new page entitled ``MAST and the Virtual Observatory" was developed
to document ways in which MAST is adopting IVOA protocols and services
to aid users in data discovery and to integrate IVOA services and tools
into its website and applications.
A new VizieR Catalog Search and Cross-correlation tool allows MAST
users to search the entire set of 4,000 VizieR catalogs and cross correlate
the results with any MAST mission. This web-based search tool accesses one
of the several VizieR catalog servers (e.g. at CfA) as a web service and
retrieves the search results in a machine-readable VOTable XML format.
A search interface was added for the AGN catalog of Padovani, the
High Level Science Products, and the WFPC2 Associations.
The user interface to the GALEX
database is a web site that provides the capability of querying the data
by object name, sky position, or physical property by a "MAST-style" query
form or by detailed queries written in SQL. Results pages (see figure) are
"active." This means that objects anywhere in the field of view can be "clicked,"
permitting image and spectra to be displayed with identifying information.
This interface was cited in NASA's Senior Review report as representing the
"state of the art" and an example of how other missions and archives should
represent and deliver data to the community. A tutorial was written for
ease of navigation. A syndicated (automated) What's New page has been
developed on the home page.
Enhancements of User Interface Pages and Tools:
The MAST search interface has been rewritten in PHP to permit a
common set of functions to be shared in searches across all MAST missions.
This enhancement improves the maintenance of interfaces for existing
missions, facilitates the construction of new mission search pages, as well
as existing search page "tools" such as the Cross Correlation Pointings
Table, and Scrapbook. The PHP rewrite also incorporates increased functionality,
for example, the inclusion of lists of target names as well as the storage
of input parameters for later use in assembling statistics on usage.
ACS and WFPC2 Associations have been added to the Pointings Tables
search. This addition allows users to determine what regions of the sky
these instruments have observed and to do sophisticated searches based
on the observational parameters.
FUSE mission data was added to the MAST Scrapbook.
The MAST Spectral Coplotter has been enhanced to allow users to
provide averaged spectra of "like" constituent datasets, that is of the
same mission, and spectroscopic configuration.
Outreach to the user community:
MAST conducted a comprehensive survey of user preferences,
attitudes, and search practices (315 users responded).
The results were presented at the October 2003 MUG
meeting and have since been used as a guide to prioritize development
of web pages and data search/retrieval tools.
MAST wrote and distributed biannual newsletters to its user community.
The February 2004 newsletter was the first to be "published" on-line.
MAST staff participated in several conferences (AAS, ADASS, and
SpaceObs) during the year by presenting posters on new MAST
services and the new GALEX website.
The Astrophysical Data Centers Executive Council (ADEC):
ADEC representatives Rachel Somerville and Paolo Padovani participated
in ADEC meeting in September 2003. The principal topic, the standardization
of dataset identifiers, has been taken as an action item during the
reporting period. MAST has completed its part by creation of a web service that
verifies that a data ID is a valid MAST dataset. This in turn is part of
a coordinated service to be hosted at ADS.
ADEC members are working with various publishers of astronomical journals to encourage
or possibly require authors to submit a complete list of the data IDs used in the paper.
MAST representatives also participated in several
discussions to coordinate the ADEC role in the Senior Review. A
common appendix was added to the proposal of each data center.
MAST Literature Links:
The publications database and the links between scientific papers
and the referenced MAST datasets are regularly updated
as new citations become available through the ADS. The journal database
tables are now updated by an automated monthly procedure.
Senior Review Proposal:
With its funding continued only through September 2004, MAST participated
in NASA's Senior Review proposal process. This consisted of a 4-year
proposal outlining the organization's program and goals for this period
with a comprehensive budget covering the slated staffing, software, and
hardware needs. This proposal was defended by Marc Postman and Rick White
in April. The reviewers of the proposal gave MAST very positive grades,
ranking it first among NASA's data archiving centers.
STAFFING CHANGES AT MAST
Rick White became the new Chief of the Archive Branch (in October 2003)
and MAST, (in May 2004). During the year Mike
Corbin, Megan Donahue, Paolo Padovani, and Antonio
Volpicelli left MAST, while Inga Kamp and Bernard Shiao
joined the staff. (Dr. Dorothy Fraquelli will start work with
MAST on August 2, 2004.) Of the staff listed, only Shiao
and Volpicelli are, or have been, salaried from the MAST contract.
Press Releases from MAST Data (HST-archival only):
PR03-17 "Brighter Neptune Suggests a Planetary Change of Season" - L. Stromovsky
PR03-20 "Celestial Fireworks" - Hubble Heritage Team
PR03-21 "Too Close for Comfort" - Hubble Heritage Team
PR03-29 "Hubble Photographs Turbulent Neighborhood Near Eruptive Star" -
Hubble Heritage Team
PR03-30 "Firestorm of Star Birth Seen in a Local Galaxy" - Hubble Heritage
PR04-13 "Hubble Sees Stars as Numerous as Grains of Sand in Nearby Galaxy"
- Hubble Heritage Team
PLANS FOR COMING YEAR
MAST will ingest GALEX Data Release 1 (DR1) in late fall of 2004.
MAST will begin hosting GALEX/DR1 data (400-500 GB) for archival Guest
Investigators and other interested users in the fall of 2004. In early
2005 MAST will serve Cycle 1 proprietary data to Guest Investigators
via password-protected accounts.
MAST will continue to archive new sets of HLSP. Expected during
the coming year are data from the GEMS and COSMOS projects, and additional
data from the GOODS and Archive Parallels projects.
MAST has recently learned of a second diffuse UV spectrum project,
SPEAR, ("Spectroscopy of Plasma Evolution from Astrophysical Radiation"),
which wishes to archive its data with MAST. MAST initiated contacts
with J. Edelstein to establish a rough timeline (2006-08) and data
volume (500 GB) for this mission. In late 2004 a timeline for data
delivery will be determined, pending an identification of funding for
archiving of this dataset.
Following NASA's acceptance of the Preliminary Design review of the
Kepler Project, MAST's role in the Data Management System will begin
ramping up in FY2005. The roles of each of the data pipeline (STScI)
and archive teams (DADS, MAST) will be defined during this period.
New services for archival research:
Coincident with the anticipated surge in usage following the DR1
release, MAST will enhance its GALEX website services with new functionality
and tools. These services will include: new registered web services
(footprint and SSAP services) entered in the IVOA registry, a fully
functional GMAX cross-correlator that links GALEX data to data from
other missions over requested regions of the sky, cutout services
or various data types (including fits files), upgrades to the web
interface forms (including several of the project's "20 Questions"
written in SQL), and a prototype of "IDL on the Network" (ION) for
quick evaluation and analysis of data on our website.
MAST will continue work
with GOODS team members to adapt a cutout service developed for team usage,
to a more general web based application for use with GOODS, UDF, and other
MAST will continue to work with IRSA to integrate MAST and IRSA data into a joint (archive center-wide) Scrapbook.
Work continues on delivering recalibrated data products from CADC on the GHRS and FOC instruments.
Specview will be ported for use on appropriate MAST applications such as the Scrapbook.
Continued Services for Archival Researchers:
MAST continues to acquire and make available High-Level
Science Products, such as spectral and image atlases, composite-data
spectra and images, and products from the HST Treasury and Large programs.
MAST will continue to provide assistance for archival researchers
with questions about searches, retrievals, data characteristics, analysis
software, and other issues related to MAST data.
MAST will continue to send out biannual newsletters to its user
MAST/GALEX will conduct a poll of its GI users to determine their
preferences for enhancements of its website pages and data mining tools.