spacer link to MAST page spacer logo image spacer
 
link to STScI page


Extended downtime notice

Due to a planned outage on Friday, April 12th starting at noon through Sunday, April 14th, access to this site will be unavailable during this time. We apologize for any inconvenience.

NASA Data Center Annual Program Plan

Program Year: FY 2004 
Data Center/Service: Multi-mission Archive at Space Telescope (MAST)
(Optical/UV Science Archive Research Center) 
Supporting Organization: Space Telescope Science Institute
3700 San Martin Drive
Baltimore, MD 21218

Overall Mission:  MAST supports active and legacy mission data sets and related catalogs and surveys, focusing primarily on data in the ultraviolet, optical, and near-IR parts of the spectrum. Support includes curation of the data, providing expert support to users of the data, providing access to data-specific calibration and analysis software, providing user support for this software, and maintaining public access interfaces to the data. MAST works with new mission teams in the supported wavelength regions to assist in the development of data management plans, especially in the areas of data formats, descriptive metadata and standardization of keywords, in the development of data access and delivery plans, and assuring data quality control.

MAST Data Holdings

Name Size Number of Observations Active Mission Duration
ACTIVE MISSIONS
FUSE 313.492 GB 2799 1999-
HST 13.956 TB 426,688 1990-
LEGACY MISSIONS
ORFEUS: BEFS 4.1 GB 332 Sept. 1993; Nov. 1996
ORFEUS: IMAPS 0.3 GB 643 Sept. 1993; Nov. 1996
ORFEUS: TUES 0.2 GB 229 Nov. 1996
EUVE 96 GB 1377 1992-Jan. 2001
ASTRO: UIT 56 GB 1,442 Dec. 1990; March 1995
ASTRO: HUT 0.6 GB 516 Dec. 1990; March 1995
ASTRO: WUPPE 0.1 GB 238 Dec. 1990; March 1995
IUE Final Archive 475 GB 103,552 1978-1996
IUE SIPS 125 GB 104,296 1978-1996
Copernicus 0.8 GB 551 1972-1981
CATALOGS & SURVEYS
SDSS: Early Data Release 1 TB n/a 1998-
VLA-FIRST 109 GB 14,940 1993-
Digitized Sky Surveys 5 TB n/a 1950-58, 1975-99
GSC I, II 2 TB n/a 1950-58, 1975-99

Services Provided: MAST provides support for users seeking to understand the properties and instrumental signatures of all archived data sets and assistance with the interfaces to browse and retrieve these data. Access to non-HST mission and instrument specific calibration and analysis software and assistance in its use continues is on a time-available basis. Full support for HST related software is provided by the MAST Help Desk and staff.

Non-HST Data Analysis Software Provided: IUE "RDAF" package (IDL-based), IUE Final Archive processing software (IRAF port), EUVE analysis software package (IRAF-based), Copernicus data analysis software (IDL-based), UIT data reduction/analysis software (FORTRAN, C, and IDL routines), WUPPE data analysis software (FORTRAN routines requiring the FITSIO library), and HUT data reduction software (IRAF-based) are available through MAST.

Mission Interfaces:

  • In September, 2002 FUSE Project began reprocessing all science data and new data with a new pipeline system. The reprocessed version contain new data products, such as files which can alert the PI to image wander during an exposure. The products also contain several types of browse products, both coadded for each "channel" (wavelength range) of an observation and also for constituent exposures that constitute a FUSE "observation." Use can browse these images to search the pre-extracted spectral images for flaws, for image draft, and for trends in count rate. As the reprocessed data are ingested, they supplant the older data version. The MAST contract with FUSE ended on March 31, 2003, limiting MAST's role to continuation of data ingest.
  • On April 29, 2003 the GALEX satellite was launched. (This satellite will survey images and spectra in the UV, including an all-sky image survey, to faint light limits.) To prepare for the data archiving, MAST and CalTech co-wrote an Interface Control Document with the GALEX Project. In anticipation of a small small data release in late 2003, MAST staff have created a database management system based on Microsoft SqlServer. Using simulated data products from the GALEX Project, MAST wrote and tested ingest procedures on its GALEX webserver. This website is now open to the public but at the moment serves simulated data.
  • Ultraviolet spectropolarimetric previews have been received from the WUPPE Project and ingested into the MAST database.

Interoperability Activities: MAST is working with the Astrophysics Data Center (ADEC) on a project to build a simple interoperability framework among the NASA astronomical data centers referring service in a web-based search in a Web-based search, pointing MAST users to relevant data at other data centers and alerting users at these other centers of MAST data that may be of interest. This project has the immediate goal of improving the interaction between data centers and of laying the framework for the National Virtual Observatory.



ACTIVITIES AND MAJOR ACCOMPLISHMENTS OF THE LAST YEAR

Data Ingests and Retrievals: The combined active mission (HST, FUSE) annual data ingest rate increased markedly to 5.8 TB, raising the total to 14.0 TB. This upturn was due mainly to the addition of data from the ACS and NICMOS instruments, which started operations late in the previous year. For all major missions (active and archival) the number of requested datasets has increased over the past year. Over the last year a new data storage array was delivered to the Institute. Staff have completed the the migration of DADS data (HST, FUSE), as well as data previously stored in other media.

MAST Data Ingest & Retrieval Activity

Date Ingest Volume (GB) - Active Missions Retrieval Volume (GB) - Active Missions Retrieval Volume (GB) - Legacy Missions Datasets Retrieved - Active Missions Datasets Retrieved - Legacy Missions
Jun 2002 382.78 679.93 1.99 32699 7493
Jul 2002 401.14 970.65 4.43 45948 4087
Aug 2002 397.98 1122.59 62.75 46824 8995
Sep 2002 477.64 1148.02 3.11 40626 1855
Oct 2002 383.54 1176.66 2.28 47416 2267
Nov 2002 472.18 936.35 2.91 32675 1949
Dec 2002 480.44 1134.36 5.07 33559 2090
Jan 2003 459.73 1260.76 5.16 51208 5136
Mar 2003 484.29 1550.37 6.65 39247 2174
Apr 2003 436.17 1152.21 8.53 31791 10855
May 2003 487.73 1096.89 4.29 37218 4420
Jun 2003 446.17 1909.7 0.98 39898 1415
Feb 2003 524.00 1500.41 6.32 37260 6174
TOTALS5833.79 15638.90 114.44 516369 58910

*As MAST does not maintain retrieval statistics for DSS and SDSS, the number of searches are displayed to show the interest level in these datasets.

"FastAccess Service": The large size of ACS files poses logistical problems for transfer of these files to users. Staff members created a FastAccess" area on the MAST website from which users can retrieve popular and non-proprietary ACS files via ftp, including ACS Early Release Observations (ERO) and the Great Observatories Origins Deep Survey (GOODS) observations. This service also reduces the number of requests made to the sometimes overloaded DADS retrieval system. All the GOODS observations were completed May 27, 2003. The FastAccess service was a great success with over 900 unique computer domains (users) retrieving over 1.2 TB of data (25,000+ files) over the past year. Near the end of the year, the ACS/ERO and some of the GOODS data were temporarily removed from the FastAccess area due to lack of space on the new RAIDTEC staging array. The data are now available from the new EMC storage area.

New Storage Media: A 32 Terrabyte RAID array purchased from the EMC Corporation will serve as the new primary storage/backup medium for all DADS, MAST, and GALEX datasets (as well as a staging area for OTF and OPUS operations). A RAIDTEC array purchased the previous year is now being used for FastAccess data (see below) and as a staging area for DADS retrievals.

Improved MAST Services:

  • document "How MAST Works" has been posted for users of MAST data to and data flow within the organization.
  • A survey of MAST users was conducted to provide feedback on the efficiency and usefulness of various MAST services. A major concern expressed in the responses was the lack of a secure FTP site for uploading of proprietary data. A request for a compilation of most popular HST proposals was approved and implemented. The resulting list of programs shows that "SNAP" proposals" are those most frequently utilized, thus providing feedback to the TAC on reviews of new HST proposals.

Data Discovery/Search Tools: Scrapbook and Pointings Table: STIS and NICMOS images were added to the Spectral/Imaging Scrapbook. New datasets are added to the Scrapbook table each month from HST and FUSE observations. Data for active instrument missions, such as for STIS, WFPC2, and NICMOS, are added weekly the Pointings Table.

This chart shows the number of times each tool was used for a search or display over the past year.

High Level Science Products (HLSP):

  • The first set of HLSP from the GOODS (Great Observatories Origins Deep Survey) HST Treasury program have been delivered to MAST. In addition to these HLSP data being visible through the HST and HLSP search interfaces, they may be accessed via anonymous ftp

  • A HLSP search form has been created similar to those for actual MAST missions. This form allows searches based on project name, object coordinates, target name. It also permits searches based on instrument, product type (image, spectrum, or catalog), and file format. The results pages for all missions now include a column pointing to any HLSP dataset. The plots below show time histories of data volume (GB) of HLSP data downloaded each month and the number of new users (computer domains) added. The roughly constant positive slope in the latter plot shows that many new astronomers are still discovering this data resource.

  • WFPC2 Associations: The Canadian Astronomical Data Center (CADC) , the Space Telescope - European Coordinating Facility ST-ECF , and MAST collaborated in creating and providing access to a set of ~16,000 combined "WFPC2 association" image datasets from the current archive of ~50,000 individual WFPC2 observations recorded in the MAST/HST archives. These associations are coadded datasets for sky regions observed several times and facilitate usage with the highest quality WFPC2 images. The grayscale figure shows an example of a WFPC2 association image (right) and one of its constituent observations (left).

  • Atlases: spectral atlases (FUSE data of OB stars, and IUE data of pre-main sequence stars). One of these atlases includes computed spectra of H2 absorptions for an array of column densities, while another includes identification of metallic interstellar features. These presentations are of special value to all FUSE proposer interested in distant objects near the Galactic plane. An image atlas of the Helix nebula has also been added to the HLSP collection. Newly observed data continue to be added to the existing Magellanic Cloud Planetary Nebula HLSP site.

User Interface Enhancements:

  • Proposal abstracts searches: Abstracts for the IUE mission have been optically scanned and entered into a database, permitting searches of them by author A similar text-search program has been implemented for HST, FUSE, and EUVE proposals, thus enhancing the search capability for programs in these missions.
  • A new search interface form has been written in the PHP scripting language for the MAST missions. This interface includes expanded functionality and more standardization in queried parameters among different MAST missions. Search results can be listed in multiple formats, including the newly adopted VOTable format of the NVO.
  • A dataset verifying tool has been written that allows a user to determine the essential attributes of an input name of a MAST dataset. This tool will be used for verification of dataset names by authors submitting lists of these names to the Astrophysical Data System (ADS) catalog.

The Astrophysical Data Centers Executive Council: MAST has played an active role within the ADEC to coordinate and provide common standards for the data services of NASA's astronomical data archive centers. Acting on request from NASA/OSS's Science Archive Working Group, the ADEC wrote and submitted a white paper on "NASA Science-Archive Interoperabililty" Acting on Headquarters request, the ADEC wrote and submitted a white paper on "NASA Science-Archive Interoperabililty" to NASA Headquarters (no formal response, as of this writing).

Literature Links: The publications database and the links between scientific papers and the MAST datasets they link to were regularly updated as new citations became available through the ADS. About 1/8 of all papers utilizing MAST datasets referred to more than one mission.

Press Releases from MAST Data (HST-archival only):

  • PR02-15 "Colorful Fireworks Finale Caps a Star's Life" - R. Fesen
  • PR02-23 "Odd Couple Widely Separated by Time and Space" - R. Knacke
  • PR03-05 "Scientists Find Faint Objects with Hubble that May Have Ended the Universe's 'Dark Ages'" - H. Yan
  • PR03-06 "Close up of M27, the Dubbell Nebula" - C. O'Dell
  • PR03-07 "Hubble Resolves a Blaze of Stars in a Galaxy's Core" - M. Tosi
  • PR03-09 "Rainbow Image of a Dusty Star" - R. Sahai
  • PR03-13 "Hubble Captures a Perfect Storm of Turbulent Gases," - J. Nester
  • JHU Press release: "Uncovering a Hidden Supernova Remnant" - C. Danforth
  • JHU Press release: "Cooling off the hottest stars," L. Bianchi and M. Garcia



PLANS FOR COMING YEAR

New datasets:

  • MAST will ingest of GALEX Data release "0" in the fall of 2003. This will mark the inauguration of the MAST public website (see figure) many new features such as "active images" and the ability to plot returned data on line.
  • MAST will respond to the Kepler mission's request for a level of support of technical developers in fall, 2003 and coordinate with the Kepler Program manager and OPUS (both are STScI entities). Kepler will be launched in 2007, will observe obtain brightnesses of 100,000 Sun-like stars every 15 minutes, and ultimately archive its data products (some 0.5 TB) within MAST,
  • MAST will ingest of UVSTAR (Shuttle mission) spectral data products,
  • MAST will design help and data-retrieval web pages for the CHIPS mission. Ingest of CHIPS data is expected in the late-2004.

New services for archival research:

  • Interfaces will be constructed to allow HST/FUSE PIs to authorize other users and oneself to access proprietary data rather than having to set up an account,
  • Interface scripts formerly written in CGI will be rewritten in PHP to permit more complicated queries and to make them adaptable as templates for interfaces to other missions,
  • The Spectrum Coplotting tool will provide the capability of displaying a mean-averaged spectrum for a set of input of spectra from the same instrument and grating/dispersion configuration,
  • An enhanced website for GALEX researchers will be designed for on-line queries, browsing and data-plotting (see plot),
  • The reprocessed FUSE dataset will be added to the Spectral Scrapbook,
  • WFPC2 associations and ACS data will be included in the Pointings table,
  • Using the HTM sky-meshing scheme, MAST will construct a field-of-view "footprint" service, first for GALEX and then other missions. This service will return a list of all observations from MAST missions that encompass a pair of coordinates specified by a user of tne service,
  • Adding search interface for press releases for STScI's Office of Public Outreach.


Thumbnail of GALEX data "return page" from querying on an observed object (data are simulated). Click on thumbnail for larger version or go to the site.

Coordination activities:

  • MAST will continue to vigorously support activities of the Astrophysical Data Centers Executive Council, and the International Virtual Observatory Alliance (NVO). MAST will assist in the development of spectral format protocols analogous to the Simple Image Access Protocol as well as new applications written with this protocol.
  • MAST will coordinate with other archiving centers to implement common interfaces for coordinated observing programs, such as the HST/Chandra observing program,
  • With SIRTF and IRSA, MAST will participate in setting up standards for the access of data from multiple arctive centers to center-wide tools, such as an interarchive-center Scrapbook,
  • The Pointings table will be enhanced to permit cross-instrument searches.

Continued Services to Archival Researchers:

  • CADC and MAST will create and stage a second group of WFPC2 association images and associated catalogs of all objects in them will be generated. Similar work will begin on ACS associations, previews, and catalogs,
  • MAST continues to acquire and make available High-Level Data Science Products, such as spectral and image atlases,
  • MAST will continue to provide assistance for archival researchers with questions about searches, retrievals, data characteristics, analysis software, and other issues related to MAST data.
Drafting of next senior review proposal to NASA.
This document will detail MAST activities over the last four years and describe plans for the future.