spacer link to MAST page spacer logo image spacer

The Multimission Archive at STScI Newsletter


June 29, 2006 Space Telescope Science Institute Volume 17


The Multimission Archive at STScI (MAST) Newsletter disseminates information to users of the HST, FUSE, GALEX, IUE, Copernicus, EUVE, HUT, UIT, WUPPE, IMAPS, BEFS, TUES, VLA-FIRST, and XMM-OM data archives supported by MAST. Inquiries should be sent to archive@stsci.edu.


Table of Contents

white rabbit picture


GALEX Release (GR2) Data (Images) at MAST

MAST is happy to report two portions of the GR2 were released to the public in early May and early June. The current release subsumes the previous GR1 release made 1-1/2 years ago. The data volume and sky coverage is now increased by a factor of 3-4. The primary component of the release, the AIS (All Sky Imaging Survey) now covers 7077 sky regions called tiles (generally circular areas of 1.2 square degrees). The Project reports that the AIS is now about 21% complete (note that when completed the AIS will cover about 3/4 of the full sky). The GR2 deliveries so far also include the Medium and Deep Imaging Survey data as well as the Nearby Galaxy Survey.
GALEX plot of galaxy
GALEX (Redshifted) Spectrum
of the galaxy [ALF2001]
MAST's grism data consists primarily of data from 7 tiles from the GR1 Spectroscopic Survey. MAST will augment this with its GR2 release of spectroscopic data, perhaps in late July.

Over the past year, MAST has received 4 imaging and 1 spectroscopic deliveries of Guest Investigator programs. Note that MAST learns which programs are coming only upon their delivery. The MAST GI website informs potential archival users of GI program information, including data, abstracts, and in the case of still-proprietary programs, release dates to the public.

Our website (
http://galex.stsci.edu) is evolving to accommodate the larger data volume and to provide redundancy for data search and retrieval. For example, the GR2 data can be queried at the object or tile level on either the familiar MAST form or SQL form. Survey data can be browsed all together, by survey (AIS, DIS, MIS, and NGS), or at coordinates of interest, at the tile level.

MAST expects to archive the first GALEX High Level Products Data this summer. Gil de Paz et al. plan to provide an image atlas of nearly 1000 galaxies in both near and far UV bands. Watch the "What's New" box for more updates!


XMM Optical Monitor Data at MAST

XMMOM Preview examples The X-ray Multi-Mirror (XMM) Telescope, launched by ESA in December 1999, carries a 30-cm optical/UV telescope with a microchannel-plate pre-amplified CCD detector in its focal plane. The Optical Monitor (OM) spatial pixel size in normal operation is 1 arcsecond with a 17 arcminute field of view, and the limiting sensitivity is B=24 for a star viewed with the detector in unfiltered light. Filters cover the 160-600 nm wavelength range. The OM telescope is coaligned with the X-ray telescope, and most XMM X-ray observations have accompanying OM images. MAST now distributes the XMM-OM images (http://archive.stsci.edu/xmm-om/). These will be of interest to many of our users, since they are typically more sensitive and higher resolution than comparable GALEX images. Most fields have images through multiple UV and optical filters (see the figure for examples.)

Community Contributed High-Level Science Products

High-Level Science Products (HLSPs) are fully processed (reduced, co-added, cosmic-ray cleaned etc.) community provided images and spectra that are ready for scientific analysis. HLSPs also include object catalogs and spectral atlases. The Multimission Archive at the Space Telescope Science Institute (MAST) archives HLSPs, which have MAST missions as an integral part of the products. The data are accessible through the MAST web pages and available via ftp download. We have made most HLSP images available through the Virtual Observatory (VO) and its common tools such as Aladin. See a complete list of currently held HLSP at
http://archive.stsci.edu/hlsp/index.html.

Five sets of HLSP have been archived since the last archive newsletter, summarized as follows:

ACS imaging of HUDF NICMOS parallel field (PI:Stiavelli)

Diagram of UDF05 fields
This diagram shows the layout of the UDF05 fields. Click on image to see details.
PI Massimo Stiavelli (STScI) and colleagues obtained an HST cycle 14 program 10632 titled Searching for galaxies at z>6.5 in the Hubble Ultra Deep Field (also known as UDF05). The program was conceived as a follow-on program to the Hubble Ultra Deep Field (HUDF) and was designed to obtain deep ACS (F606W, F775W, F850LP) imaging in the area of the original HUDF NICMOS parallel fields. In addition, deep NICMOS (F110W, F160W) images within the original HUDF field were made as simultaneous parallel observations. The HLSPs that are part of the initial release are of one of the two principal fields NICP12 R.A. = 03:33:03.60; Dec. = -27:41:01.80 (J2000). Subsequent releases are planned for the remaining data. For more information about the HLSP associated with this project see the website http://archive.stsci.edu/prepds/udf05.

The Catalog and Atlas of Cataclysmic Variables (PI:Downes)

sample CVS catalog page
A CV catalog object description page
Click on thumbnail to see a full size image.
MAST began to host Ron Downes' (STScI) popular Catalog and Atlas of Cataclysmic Variables in March 2006. The Catalog is a web-based version of the published catalogs: Downes et.al. 2001, PASP 113, 764; Downes and Shara 1993, PASP 105, 127; Downes, Webbink, and Shara 1997, PASP 109, 345. One of the goals of the catalog was to have the basic information on the objects (i.e., coordinates, type, magnitude range, and finding charts) in one central location, thus making it easy for observers to obtain data on the objects. The Web site also allows for easy searching of the catalog and for generation of basic statistics (e.g., how many dwarf novae, how many CVs have Hubble Space Telescope [HST] data, etc.).

COSMOS (PI:Scoville)

cosmos field
COSMOS Field
Click on image to see the ACS mosaic page.
COSMOS is an HST Treasury Program (Scoville-CalTech) to survey a 2 square degree equatorial field, centered on RA=10:00:28.6 and DEC=+02:12:21.0 with the ACS in the I band of the VIMOS equatorial field. Parallel observations with WFPC2 and NICMOS were also obtained. MAST has archived the ACS, WFPC2, NICMOS and GALEX science products from the COSMOS project. In addition to the FITS files, team member Anton Koekemoer produced a mosaic of the ACS data. The mosaic as a single file is a 100,800 × 100,800 pixel FITS file 81GB in size. The mosaic has been divided into a set of 49 tile FITS images.

You may download the images through your browser by clicking on the filename or via wget. Most data is also available via anonymous ftp (ftp archive.stsci.ed cd /pub/hlsp/cosmos).

Eta Carinae (PI:Davidson)

Eta Car picture

Kris Davidson (U. Minnesota) is the PI for an HST Treasury Program that observed Eta Carinae. The program was designed to obtain data during an anticipated spectroscopic event in mid-2003. Observations were obtained using the HST STIS/CCD, STIS/MAMA, WFPC2 and ACS images. The initial release of HLSP contains only the STIS/CCD observations. The team has an extensive website at http://etacar.umn.edu/. They provide in addition to the actual data also the teams detailed view of the observation including a preview and a display showing the slit placement. The MAST archive web pages for the Eta Car HLSP are at http://archive.stsci.edu/prepds/etacar.

Archive Pure Parallel Program (PI:Casertano)

PU581GS01 from the Four Filter Data (FFD) collection
Parallel at 17 50 13.26 -37 08 16.8
The Archival Pure Parallel Program (Casertano - STScI) processed and combined with state-of-the-art tools about 2,000 WFPC2 images, primarily in the wide UBVI filters, obtained in parallel with other HST instruments. The team produced combined, drizzled, cosmic-ray cleaned images and object catalogs, for each pointing. In June, 2006 the APPP team delivered release 2 of these data tripling the amount of data in the previous release. See the MAST website http://archive.stsci.edu/prepds/appp.

These data can be used to address a wide range of science topics: measuring the cosmic shear on scales from 20" to 2'; discovering ~ 50 starforming galaxies at z ~ 4; finding optical counterparts to AGNs in wide-area radio and X-ray catalogs; improving the determination of the scale length of the Galactic disk; and studying stellar populations down to 1 solar mass for about 50 separate lines of sight in the Magellanic Clouds.

We are interested in archiving your High-Level Science Products!

HLSPs are provided by experienced users and legacy teams, who make their calibrated data available to the community after a careful and sophisticated data reduction process. Our archive retrieval statistics over the past year reveal that the HLSPs get downloaded ~10 times more often than the average HST pipeline processed dataset. This will certainly reflect in the citation index of associated papers.

If, as part of your research, you have produced such science ready data products based on data from any of the MAST missions and you are interested in contributing them as HLSPs to MAST, please see our page of guidelines and contact us at archive@stsci.edu.


Recent Changes to the Digitized Sky Survey

Color composite image of Horsehead from 3 bandpasses of DSS POSS-II The Digitized Sky Survey (from scans of the Palomar Schmidt and AAO UK Schmidt all-sky photographic surveys) is one of the most heavily used resources available from the MAST archive, with an average of 5000 downloads per day (http://archive.stsci.edu/cgi-bin/dss_form). Even though these data were created and made available to help support HST operations, they are valuable for many other uses and have become part of the the underlying infrastructure of modern astronomy. (See http://www-gsss.stsci.edu/SkySurveys/SkySurveys.htm for more detailed information on the available surveys). A color composite image derived from the 3 passbands of the POSS-II is shown as an example of the usefulness of the DSS data.

One of the limitations of the data has been the precision of its absolute astrometry. All of the astrometric information in the headers was tied to the HST Guide Star reference frame which was constructed back in the 1980's when there was no single unified all-sky reference frame. This resulted in potential astrometric errors of 1-2 arcseconds when compared to observations in other coordinate systems. In addition, when initially conceived, there was no standard FITS world coordinate system, so the astrometric keywords contained in the FITS headers are not standard and require one to either convert the headers or use a tool that understood this particular implementation.

The Catalogs and Surveys group at STScI have created an updated version of the Guide Star Catalog (GSC-II) that beginning with HST cycle 15 replaced the GSC-I as the guide star reference frame (see http://www-gsss.stsci.edu/Catalogs/GSC/GSC2/GSC2.htm for additional details). This is based on the IAU recommended International Celestial Reference System (ICRS) and is the reference frame for many other catalogs (e.g. SDSS, 2MASS etc).

In coordination with this change, the astrometry of the DSS has also been updated to use the GSC-II reference frame in order that HST observers can measure coordinates for observation planning. We also took this opportunity to update the headers to reflect the current FITS standard. In addition to preserving the GSC-I astrometric solutions in the FITS header, we have added the more complex GSC-II astrometry (including a position-dependent correction as an extension table) for completeness. However, since this is not supported by most display tools, the code that returns the subset of the image from the selected plate will dynamically compute a simple FITS WCS for the extracted image and inserts these keywords into the returned FITS header that is useable by most image analysis tools.


EUVE Extracted Files

EUVE preview plot of Procyon

EUVE Preview of Procyon
EUVE spectra extracted with "euvextract" from the euv.euvred IRAF package are now available from MAST. Previously, only the spectral image files were distributed, although extracted preview data could be displayed from the EUVE preview pages. The euvextract program extracts fluxes from the sw, mw and lw spectra and merges them into a single spectrum binary table FITS file. Damian Christian, former EUVE and MAST staff member, produced the files for us. Links to the files can be found on the EUVE search results page and on the EUVE preview pages.


NED Search Interface at MAST

screenshot of NED/MAST interface page
Screenshot of the NED MAST interface

During a lively discussion at the April MAST Users Group, a new search interface was conceived. This interface permits a user to search the NASA/IPAC Extragalactic Database (NED) and cross correlate the results of the NED search with observations from the MAST archive. The interface is now available at http://archive.stsci.edu/ned.php. Like the VizieR search tool (http://archive.stsci.edu/vizier.php) the NED interface uses Virtual Observatory (VO) web services to submit parameters from the MAST search form to NED. The returned results can then be displayed in a standard MAST search results page. The interface utilizes NED's "Near Position" and "Advanced All-sky" search scripts, which are both accessible at http://nedwww.ipac.caltech.edu. Current search parameters include target name, coordinates, search radius, redshift constraints, and "included" and "excluded" object classifications. Alternate output formats are also available including VOTable, Excel spreadsheet, and comma-separated values (CSV).

Additional parameters are planned so if you have suggestions for enhancements or changes, please let us know by sending mail to archive@stsci.edu or dropping us a suggestion.



HST Preview Improvements

HST preview images and spectra have undergone some improvements.

The CADC, which creates the previews, has increased the sizes of preview images. WFPC2 images, formerly mosaicked, binned and converted to integer, are now actual-size mosaics. (The datatype of the pixels is still integer.) The new preview images include a full set of header keywords and are cosmic-ray rejected. Older previews have not been redone in the larger size.

We've added some new stretches (which for ACS will be automatically selected) which eliminates a percentage of the brightest and darkest pixels. These improve ACS preview GIF images in particular. You may wish to view the preview with a different pixel correction. To do this, click on the link "More preview format options" just under the initial preview image. You may then choose from a number of pixel correction options. You may also choose to see the full size version of the gif file.


A User/MAST Collaboration - HST Pointings Interface and Table Improvements

For a number of years, MAST has maintained a set of tables that summarize available data for regions of the sky as well as an interface that allows users to query these tables. We called this the "HST Pointings Interface". Early last winter, Phil Marshall, who wanted to use the pointings interface to select data to be retrieved for the HAGGLeS archival program wrote to us. The pointings interface and summaries we had developed were almost what he needed! He had some specific needs and suggestions on how to make the pointings interface more useful in general. The team was looking for sky regions that had more than 2000 seconds of total exposure time and that had data in two or more bands (not filters). The team was doing reiterative searches and needed a constant reference (pointing number). He wanted some additional meta-data returned in the search results and he wanted to be able to search via pointing but have all the exposures displayed in a single return. He also noted that it seemed that the algorithm we were using to define regions was perhaps too conservative and thus defined as separate regions, sky areas that could actually be combined. We reviewed his suggestions and decided to implement several of them.

The algorithm for selecting the observations to be grouped together was made more selective, resulting in fewer "regions". The selection processes was redefined so that once a sky region has been defined, it maintains the arbitrary "pointing number" assigned to it. As new observations are acquired they are assigned to an existing sky region when possible or used to define a new region if appropriate. Now separate pointings are maintained for different instrument modes (e.g. ACS-WFC and ACS-HRC) while previously they were combined.

The interface was ported to our standard format. You may now search for pointings, or for the exposures assigned to those pointings. The new interface allows you to select the instrument mode and the output format (html, comma separated value (CSV), VOTable or Excel).

Check out the new HST Pointings Interface and answer such questions as How many high galactic latitude observations exist for RA between 9 and 18 hours with dec > -20 where there have been at least two observations and where at least one of those observations was in the I-band, and at least one of those observations was in any other filter?

Set these parameters in the interface:
Galactic Latitude Above & below plane +/- > 20.0 degrees
Band I >0
Number of Unique Bands >1
Total Number of Exposures >1
RA 09..18
Dec >-20.0

You may also access the pointings interface as a web service running the query from a GET request. For more information about running MAST queries as a web service see the web page http://archive.stsci.edu/vo/mast_services.html. You may also send mail to archive@stsci.edu.

For example, you may want to search the ACS Pointings table for the target "M 81", output format = "CSV", and more than 11 exposures in the U band: http://archive.stsci.edu/pointings/search.php?pnt_ucount=%3C11&target=m+81&
primary=ACS&outputformat=CSV&action=Search+Pointings
(Try it)

If you search the pointings for exposures, you will be able to make quite large requests. PLEASE be careful not to exceed the DADS operational capacity. Please read the information on constraints for making large requests from DADS.


MAST Website Navigation Changes

This month we modified some of the links in the top navigation bar to improve site navigation. These changes are in response to some suggestions received from the MAST User Survey asking that better navigation to the HST search form be provided from the main MAST page. The "missions" pulldown menu has in the past been used to provide links to the main mission pages; although these were also provided in the mission links in the lower right navigation section of the top MAST page. The "missions" pulldown is now labeled "Mission Search" and contains links to the main mission search forms. This allows you to access search forms directly from the main MAST page. Since this navigation bar also appears on all search forms, users can now move from one search form to another with one click. In a parallel change, the "Searches-Tools" link is now labeled "Tools", and contains links to programs and search forms which are not specific to any one mission. For example, the link to the new NED search form is now here. A new link called "Abstracts" was added to help users find proposal abstract search forms. Since the links in the pulldown menus contain only the main MAST search forms, both "Tools" and "Mission Search" now have a "more" link which links to
a page that displays all the available MAST search forms. This page contains links to all seven HST-related search forms, and all seven GALEX search forms. To suggest further ways to improve access to the MAST web site, please send comments to archive@stsci.edu.


MAST Virtual Observatory (VO) Activities

Besides offering new interfaces using
Virtual Observatory (VO) services (see article on new NED interface), work at MAST in support of the VO has progressed in mainly three areas: the spectral container project, documentation, and validation of existing services. Much of this work is transparent to the user.

Screenshot of Specview session
Screenshot of Specview session
Coplotted STIS, FUSE, IUE, EUVE, and HUT data
Spectral Container: The goal of the spectral container project is to create a common FITS file format for the storage and distribution of MAST spectral data that is compatible with the Spectral Data Model being developed for the VO. The files will be accessible to VO users via the Simple Spectral Access Protocol (SSAP). The container files are not a replacement for the original project-delivered data sets which will continue to be available to users. The project is a joint effort between HST, CADC, and ESO with occasional input from the various VO developers. The spectral container project is based on draft versions of the SSAP and Data Model papers. Some of the data have been registered and the data are already in use by some spectral browsers such as Specview and VOSpec.

Unlike image data, projects have historically archived spectral data in a variety of FITS file formats. Wavelengths, fluxes, and sigmas have been stored in separate files, as non-homogeneous primary arrays and in a variety of binary table formats using scalar, vector, or variable length array fields. In addition, the metadata stored in the FITS headers varies considerably between missions with common parameters stored using a variety of keyword names. VO application developers (for example the people developing the spectral browser tools) would have to write mission-specific code to extract data from each type of file.

As a result of these problems and following the FITS serializations recommended in the VO Spectral Data Model paper, files were created using a FITS binary table format containing fixed length vector fields with a core set of standard keywords defined in the binary table extension header (the original mission-defined primary header keywords were preserved). This format has been used to generate new files for the following missions (special modifications, if any, are described on a separate page. Work is also progressing on defining a set of standard keywords that will use consistent names and definitions for all missions.

Documentation: We are in the process of updating our web page describing available MAST web services. The page has been expanded to include examples of how scripts can be written to access web services using different programming languages, and to provide more information on the parameters that may be specified in an HTTP GET request. See http://archive.stsci.edu/vo/mast_services.html

Validation of Existing Services: A web site is now available from the National Center for Supercomputing Applications (NCSA) for VO developers to validate their registered Cone Search and Simple Image Access Protocol (SIAP) VO services. The web site runs a series of tests using both valid and invalid search parameters, then evaluates the returned VOTable files for errors. Before this tool was available, SIAP and cone search services were based solely on each data provider's interpretation of the VO standards. As a result of running the validator on the registered MAST services, changes were made to several MAST services to:

The testing also resulted in correcting some errors discovered in the validator itself. Similar tests are planned when the SSAP validator becomes available.


MAST Archive Status and Activities

plot showing downloads vs holdings With delivery of data for the second release from the GALEX project, the new addition of XMM Optical Monitor data, the new community contributed High-Level Science Products, and the continued accumulation of HST and FUSE data, MAST continues to grow in volume. As of June 1, 2006, MAST holds over 44 TB of data. In the past 6 months MAST has distributed over 11 TB of data to archive users, while ingesting about 3 TB of data. GALEX Release 2 data accounts for about a third of the ingested data.

The plot to the left contrasts the holdings as of April 2006 with the total downloads from April 2005 through March 2006. For almost every group of data, a value equal to the of the total volume of data was distributed during that past year, with the community contributed HLSP being the most popular group of data.
plot showing median retrievals times

On June 13, 2006, DADS serviced its 100,000th request! Jay Anderson of Rice University submitted the request for HST/ACS data. Over the past year, users retrieved a total of about 29 Tbytes, an average of nearly 80 Gbytes/day. The typical retrieval time for HST data is usually under 1 hour, with on-the-fly-reprocessing (OTFR) requests taking a median time of about 50 minutes and non-OTFR requests taking about 10 minutes.

The MAST Users Group (MUG) met in April 2006. This year's members were Ann Zabludoff (Chair), Anton Koekemoer, Leonidas Moustakas, Jeff Newman, Sandyha Rao, Aki Roberge, and Richard Wade. The MUG provides us with important feedback from the user's perspective, helps to set priorities and suggests directions for future enhancements to the archive. You can see the presentation made at this year's meeting and at previous years at http://archive.stsci.edu/mug/.

The 2006 MAST User Survey drew a record 364 responses. Thanks to all who participated for taking the time to give us feedback! We have already implemented some changes in response to your comments. For example, the algorithm used to create the display of ACS previews has been greatly improved so that the browser images are much more useful. More changes will appear in the coming months. The results are summarized at http://archive.stsci.edu/surveyresults/2006/survey_mar2006.html. We welcome additional suggestions at any time using the MAST Suggestion Box or through email to archive@stsci.edu.


How Can MAST Improve and Expand to Serve You Better?

suggestion box picture MAST is continually looking for ways to help you find the data you want and we would really like to hear your ideas. We have provided a "suggestion box" at
http://archive.stsci.edu/suggestions.html and encourage you to post suggestions, comments and concerns through this interface. A link to the suggestion box is found on the second line of the top menu on most MAST web pages.

Some past users have allowed us to post their suggestions. Those suggestions and the MAST responses can be found at http://archive.stsci.edu/suggestion_response.html.

Let us hear from you!


To Subscribe or Unsubscribe to the Newsletter

If you would like remove yourself from the mailing list send mail to archive_news-request@stsci.edu and put the single word unsubscribe in the BODY of the message. Information on the subject line will be ignored.

If you would like to subscribe to the mailing list to receive notification of future Archive Newsletters, send mail to archive_news-request@stsci.edu and put the single word subscribe in the BODY of the message. Information on the subject line will be ignored. You will be asked to confirm the subscription.