Hubble Advanced Spectral Products (HASP)
Revolutionizing Spectral Data Access: HASP's Automated Coaddition Approach
The HASP program, developed by the Space Telescope Science Institute, revolutionizes the utilization of the Mikulski Archive for Space Telescopes (MAST), providing high-quality one-dimensional spectra for Hubble Space Telescope (HST) spectroscopic data. HASP offers robust coadded and combined products for nearly every Cosmic Origins Spectrograph (COS) and Space Telescope Imaging Spectrograph (STIS) spectrum, covering over 3200 programs and 64000 input datasets in MAST.
HASP is your gateway to visit and program-level coadds for COS and STIS, enhancing the legacy value of archival spectra and lowering the barrier to scientific access for new HST observers
The ultimate goal of HASP is to simplify access to HST spectroscopic data by providing high-quality 1-D spectra that are both robust and flexible. By default, combinations of different gratings, central wavelengths (CENWAVEs), or apertures for individual programs are not provided, necessitating users to manually coadd spectra for scientific analyses. HASP not only automatically coadds most datasets but also provides tools for users to create custom coadds, offering further flexibility and control over the coaddition process. Explore custom coadd options in our Jupyter Notebooks.
Building on the success of previous efforts, such as the Hubble Spectroscopic Legacy Archive (HSLA) and the UV Legacy Library of Young Stars as Essential Standards (ULLYSES) program, HASP extends automated coaddition to nearly every COS and STIS spectrum in the archive. This project includes a publicly available Python script and Jupyter Notebooks, enabling custom coaddition by the community. HASP lays the foundation for automated multi-program coaddition, creating an updated HSLA.
Spectra
HASP coadd spectra for select targets are shown in black, overlaid on top of constituent x1d spectra.
Slideshow
Co-added and abutted COS G130M/G160M product from HST Program 16196 , which monitored the active galaxy Mrk 817, in black. 391 G130M and 259 G160M constituent spectra that form the coadded product are plotted in pink and blue, respectively.
Coadded and abutted STIS G230L/G430L/G750L products from HST Program 16656 for supernova 2022wsp in black at +10 days (top) and +20 days (bottom). G230L, G430L, and G750L constituent spectra that form the coadded products are plotted in blue, green, and orange, respectively. Product fluxes are offset from observed values.
COS and STIS HASP data products from HST Program ID 14076 for a selection of the complete sample of white dwarves within 13 pc are plotted in black. Constituent G130M, G160M, E140M, and E230M spectra that form the coadded products are plotted in purple, blue, red, and pink, respectively. Product fluxes are offset from observed values.
Coadded COS G185M products from HST Program 12520 for several early-B stars in the open cluster, NGC 3293, in black. 391 Colored spectra represent the constituent G185M observations for each of the nine stars.
Coadded and abutted COS FUV G130M/G160M and NUV G225M/G285M product from HST Program ID 11561 that studied the accretion of rocky material from a circumstellar disk around a white dwarf, plotted in black. Constituent G130M, G160M, G225M, and G285M spectra that form the coadded product are plotted in purple, blue, green, and orange, respectively. Note that the black coadded data is offset.
Data Products
The coadd code implemented in the HASP project generates two distinct data products: coadds and abutments, created at both the visit and program levels. Co-added spectra are the result of combining spectra from a common grating, while abutments involve the combination of spectra from different gratings and/or instruments.
Naming Convention
Program-level co-add files follow the format:
hst_<PID>_<instrument>_<target>_<opt_elem>_<ippp>_cspec.fits
Visit-level co-add files follow a similar, if slightly different format:
hst_<PID>_<instrument>_<target>_<opt_elem>_<ipppss>_cspec.fits
where:
- <PID> is the program ID, e.g. 11068
- <instrument> is the instrument used (one of COS, STIS, OR COS-STIS)
- <target> is the target of the observation, e.g. "NGC5457"
- <opt_elem> is the optical element (i.e. filter or grating), e.g. "G140L"
- <ippp> is the instrument and program ID of the program level coadd, e.g. ld5z (where l = COS and d5z = Proposal ID 14772)
- <ipppss> is the instrument, program ID, and observation ID of the visit-level coadd, e.g. Id5z20 (visit 20 of the above example)
FITS Structure
The FITS file structure for these co-added products consists of two BINTABLE extensions: a Science extension containing specific information about the combined product and a metadata extension recording attributes of each spectrum contributing to the combination.
Science Extension Table
The Science extension table stores data elements of a single spectrum:
| Keyword | Units | Type |
|---|---|---|
| WAVELENGTH | Angstrom | single-precision float |
| FLUX | erg/cm2/s/Angstrom | single-precision float |
| ERROR | erg/cm2/s/Angstrom | single-precision float |
| SNR | --- | single-precision float |
| EFF_EXPTIME | s | single-precision float |
Provenance Table
The provenance table contains metadata from the headers of contributing spectra.
| Keyword | Units | Type | Description |
|---|---|---|---|
| FILENAME | -- | string | Input spectrum filename |
| EXPNAME | -- | string | Exposure name, if multiple spectra per file |
| PROPOSID | -- | string | Proposal ID |
| TELESCOPE | -- | string | Observatory |
| INSTRUMENT | -- | string | Instrument |
| DETECTOR | -- | string | Instrument detector |
| DISPERSER | -- | string | Grating |
| CENWAVE | -- | string | Central wavelength of grating |
| APERTURE | -- | string | Aperture selected |
| SPECRES | -- | double-precision float | Estimated spectral resolution |
| CAL_VER | -- | string | Calibration version |
| MJD_BEG | d | double-precision float | Exposure start time (MJD) |
| MJD_MID | d | double-precision float | Exposure mid-point (MJD) |
| MJD_END | d | double-precision float | Exposure end time (MJD) |
| XPOSURE | s | double-precision float | Exposure time |
| MINWAVE | Angstrom | double-precision float | Minimum wavelength |
| MAXWAVE | Angstrom | double-precision float | Maximum wavelength |
Data Access
-
The HASP Search Form in MAST
HASP products are now available in the HST search form. As part of the default products available for download, you'll now see new, HASP-generated cspec.fits files.
-
astroquery.mast
A streamlined interface for Python users to access data in MAST. As HASP data is now included in MAST, these files are available through the astroquery API. This API ntegrates easily with Astropy functionality and other Python analysis tools, streamlining the process to access and analyze your data.
Notebooks
The HASP project release is accompanied by a set of user-friendly Jupyter Notebooks, enabling users to perform custom coadditions for specific use cases not supported in automatic coadds. These Notebooks were developed to aid in setting up and running the coadd script and creating custom coadd products.
Performance Evaluation
The project's target requirement was established with the aim of achieving over 75% accuracy for compact, non-variable targets. This involves ensuring that default 1-D spectral products meet baseline absolute flux and wavelength accuracies for each spectral mode and instrument. To assess flux accuracy, we conducted tests on residuals between binned coadd products and their respective x1d observation files. The objective was to have the mean and standard deviation of flux residuals both under 5%.
Middle: Fluxes are combined into 20 bins for comparison.
Bottom: Residuals for x1d - coadd flux difference. The dark blue line provides the mean flux difference, light blue lines show the standard deviation in residuals.
Similarly, to assess wavelength accuracy, we measured the cross-correlation lag between unbinned coadd products and their constituent observation files. The goal was to meet the defined wavelength accuracy for each spectral mode and instrument, as outlined in their respective instrument handbooks.
Bottom: Cross-correlation coefficient distribution from the comparison of _x1d and coadded spectra.
HASP coadd product flux and wavelength accuracies meet requirements in more than 90% of archive datasets in a majority of observing modes. All observing modes meet and exceed the testing success rate of 75%.
Moreover, coadd products underwent a comparison with CALSPEC stellar templates, a scientific validation that quantifies how well the choices of flux filtering, keyword filtering, and abutment rules preserve the integrity of an individual observing mode.
These achievements highlight the effectiveness of the coadd code in delivering accurate and reliable results for a wide range of spectroscopic data, contributing significantly to the success of the Hubble Advanced Spectral Products project.
Caveats
Datasets with the following caveats are not included in HASP coadd data products:
Observing Issues
- Guide star acquisition failures
- Observatory or detector failure events
- EXPFLAG (exposure data quality flag) header keyword set to anything other than 'NORMAL'
- EXPTIME (exptime) is zero seconds
- Actual exposure time is less than 80% of the planned exposure time
- FGSLOCK (fine guidance system lock) is not 'FINE', i.e., guide star tracking was not locked
- The shutter was closed, denoted by the Take Data Flag (TDF)
- Data flagged with bad archive data quality (resulted from filed alertobs)
Observation Parameters
-
POSTARG1 ≠ 0.0, i.e., there is a pointing offset
-
POSTARG2 ≠ 0.0 AND P!_PURPS ≠ 'DITHER', i.e., there is a pointing offset not in the dispersion direction
-
PATTERN1 = STIS-PERP-TO-SLIT and P1_FRAME = POS-TARG, i.e. there is a cross-dispersion direction pointing pattern (STIS only)
-
P1_PURPS = MOSAIC, i.e., there is a pointing mosaic offset pattern (STIS only)
-
OPT_ELEM (grating) = PRISM (STIS only)
-
APERTURE = BOA (Bright Object Aperture, COS only)
Target Parameters
- Moving targets
- Variable (These are not rejected by default, but some exposures may be removed by the code's flux-checking routine)
Observations of moving targets with the same name are co-added at the visit level but not at the program level.
Coadd data products are derived from standard pipeline .x1d observations. As such, additional reduction steps (defringing, cosmic ray removal, echelle blaze shifts, etc.) are not performed. Users must generate custom coadds to address these issues in default pipeline products.
Citations
If you are using HASP data in a published work, please cite the following paper:
Instrument Science Report COS 2024-01
Also, please include the following acknowledgment in all publications that make use of HASP data:
Based on observations obtained with the NASA/ESA Hubble Space Telescope, retrieved from the Mikulski Archive for Space Telescopes (MAST) at the Space Telescope Science Institute (STScI). STScI is operated by the Association of Universities for Research in Astronomy, Inc. under NASA contract NAS 5-26555.
References
Questions about HASP? For the fastest response, please visit the HST Help Desk Website.