Hubble Advanced Spectral Products (HASP)

Revolutionizing Spectral Data Access: HASP's Automated Coaddition Approach

The HASP program, developed by the Space Telescope Science Institute, revolutionizes the utilization of the Mikulski Archive for Space Telescopes (MAST), providing high-quality one-dimensional spectra for Hubble Space Telescope (HST) spectroscopic data. HASP offers robust coadded and combined products for nearly every Cosmic Origins Spectrograph (COS) and Space Telescope Imaging Spectrograph (STIS) spectrum, covering over 3200 programs and 64000 input datasets in MAST.

HASP is your gateway to visit and program-level coadds for COS and STIS, enhancing the legacy value of archival spectra and lowering the barrier to scientific access for new HST observers

The ultimate goal of HASP is to simplify access to HST spectroscopic data by providing high-quality 1-D spectra that are both robust and flexible. By default, combinations of different gratings, central wavelengths (CENWAVEs), or apertures for individual programs are not provided, necessitating users to manually coadd spectra for scientific analyses. HASP not only automatically coadds most datasets but also provides tools for users to create custom coadds, offering further flexibility and control over the coaddition process. Explore custom coadd options in our Jupyter Notebooks.

Building on the success of previous efforts, such as the Hubble Spectroscopic Legacy Archive (HSLA) and the UV Legacy Library of Young Stars as Essential Standards (ULLYSES) program, HASP extends automated coaddition to nearly every COS and STIS spectrum in the archive. This project includes a publicly available Python script and Jupyter Notebooks, enabling custom coaddition by the community. HASP lays the foundation for automated multi-program coaddition, creating an updated HSLA.

Spectra

HASP coadd spectra for select targets are shown in black, overlaid on top of constituent x1d spectra.

Slideshow

Mrk817 Coadd

Co-added and abutted COS G130M/G160M product from HST Program 16196 , which monitored the active galaxy Mrk 817, in black. 391 G130M and 259 G160M constituent spectra that form the coadded product are plotted in pink and blue, respectively.

SN 2022wsp

Coadded and abutted STIS G230L/G430L/G750L products from HST Program 16656 for supernova 2022wsp in black at +10 days (top) and +20 days (bottom). G230L, G430L, and G750L constituent spectra that form the coadded products are plotted in blue, green, and orange, respectively. Product fluxes are offset from observed values.

White Dwarfs within 13pc

COS and STIS HASP data products from HST Program ID 14076 for a selection of the complete sample of white dwarves within 13 pc are plotted in black. Constituent G130M, G160M, E140M, and E230M spectra that form the coadded products are plotted in purple, blue, red, and pink, respectively. Product fluxes are offset from observed values.

B Stars in NGC3293

Coadded COS G185M products from HST Program 12520 for several early-B stars in the open cluster, NGC 3293, in black. 391 Colored spectra represent the constituent G185M observations for each of the nine stars.

White Dwarf with a Planetary Debris Disk

Coadded and abutted COS FUV G130M/G160M and NUV G225M/G285M product from HST Program ID 11561 that studied the accretion of rocky material from a circumstellar disk around a white dwarf, plotted in black. Constituent G130M, G160M, G225M, and G285M spectra that form the coadded product are plotted in purple, blue, green, and orange, respectively. Note that the black coadded data is offset.

Data Products

The coadd code implemented in the HASP project generates two distinct data products: coadds and abutments, created at both the visit and program levels. Co-added spectra are the result of combining spectra from a common grating, while abutments involve the combination of spectra from different gratings and/or instruments.

Naming Convention

Program-level co-add files follow the format:

hst_<PID>_<instrument>_<target>_<opt_elem>_<ippp>_cspec.fits

Visit-level co-add files follow a similar, if slightly different format:

hst_<PID>_<instrument>_<target>_<opt_elem>_<ipppss>_cspec.fits

where:

  • <PID> is the program ID, e.g. 11068
  • <instrument> is the instrument used (one of COS, STIS, OR COS-STIS)
  • <target> is the target of the observation, e.g. "NGC5457"
  • <opt_elem> is the optical element (i.e. filter or grating), e.g. "G140L"
  • <ippp> is the instrument and program ID of the program level coadd, e.g. ld5z (where l = COS and d5z = Proposal ID 14772)
  • <ipppss> is the instrument, program ID, and observation ID of the visit-level coadd, e.g.  Id5z20 (visit 20 of the above example)

FITS Structure

The FITS file structure for these co-added products consists of two BINTABLE extensions: a Science extension containing specific information about the combined product and a metadata extension recording attributes of each spectrum contributing to the combination.

Science Extension Table

The Science extension table stores data elements of a single spectrum:

Keyword Units Type
WAVELENGTH Angstrom single-precision float
FLUX erg/cm2/s/Angstrom single-precision float
ERROR erg/cm2/s/Angstrom single-precision float
SNR --- single-precision float
EFF_EXPTIME s single-precision float

Provenance Table

The provenance table contains metadata from the headers of contributing spectra.

Keyword Units Type Description
FILENAME -- string Input spectrum filename
EXPNAME -- string Exposure name, if multiple spectra per file
PROPOSID -- string Proposal ID
TELESCOPE -- string Observatory
INSTRUMENT -- string Instrument
DETECTOR -- string Instrument detector
DISPERSER -- string Grating
CENWAVE -- string Central wavelength of grating
APERTURE -- string Aperture selected
SPECRES -- double-precision float Estimated spectral resolution
CAL_VER -- string Calibration version
MJD_BEG d double-precision float Exposure start time (MJD)
MJD_MID d double-precision float Exposure mid-point (MJD)
MJD_END d double-precision float Exposure end time (MJD)
XPOSURE s double-precision float Exposure time
MINWAVE Angstrom double-precision float Minimum wavelength
MAXWAVE Angstrom double-precision float Maximum wavelength

 

Data Access

  • The HASP Search Form in MAST

    The HASP Search Form in MAST

    HASP products are now available in the HST search form. As part of the default products available for download, you'll now see new, HASP-generated cspec.fits files.

  • astroquery.mast

    astroquery.mast

    A streamlined interface for Python users to access data in MAST. As HASP data is now included in MAST, these files are available through the astroquery API. This API ntegrates easily with Astropy functionality and other Python analysis tools, streamlining the process to access and analyze your data.

Notebooks

The HASP project release is accompanied by a set of user-friendly Jupyter Notebooks, enabling users to perform custom coadditions for specific use cases not supported in automatic coadds. These Notebooks were developed to aid in setting up and running the coadd script and creating custom coadd products.

Performance Evaluation

The project's target requirement was established with the aim of achieving over 75% accuracy for compact, non-variable targets. This involves ensuring that default 1-D spectral products meet baseline absolute flux and wavelength accuracies for each spectral mode and instrument. To assess flux accuracy, we conducted tests on residuals between binned coadd products and their respective x1d observation files. The objective was to have the mean and standard deviation of flux residuals both under 5%.

perf-eval.png
Top: A coadded G130M data product (orange) compared to an individual _x1d spectrum (blue) for the AGN NGC 5548 observed in HST Program 13330.
Middle: Fluxes are combined into 20 bins for comparison.
Bottom: Residuals for x1d - coadd flux difference. The dark blue line provides the mean flux difference, light blue lines show the standard deviation in residuals.

Similarly, to assess wavelength accuracy, we measured the cross-correlation lag between unbinned coadd products and their constituent observation files. The goal was to meet the defined wavelength accuracy for each spectral mode and instrument, as outlined in their respective instrument handbooks.

cross-correlation coefficient distribution
Top: A coadded G185M data product (orange) compared to an individual _x1d spectrum (blue) for a B star in the open cluster NGC 3293, observed in HST Program 12520.
Bottom: Cross-correlation coefficient distribution from the comparison of _x1d and coadded spectra.

HASP coadd product flux and wavelength accuracies meet requirements in more than 90% of archive datasets in a majority of observing modes. All observing modes meet and exceed the testing success rate of 75%.

results of flux and wavelength tests. all gratings exceed the 75% success criterion
Success rate for flux and wavelength accuracies.

Moreover, coadd products underwent a comparison with CALSPEC stellar templates, a scientific validation that quantifies how well the choices of flux filtering, keyword filtering, and abutment rules preserve the integrity of an individual observing mode.

calspec stellar templates compared to the HASP coadded fluxes
Coadd products across both instruments, employing all well-utilized gratings and cenwaves (colors), compared to CALSPEC models of GD71, G191B2B, and WD 0308-565 (black)

These achievements highlight the effectiveness of the coadd code in delivering accurate and reliable results for a wide range of spectroscopic data, contributing significantly to the success of the Hubble Advanced Spectral Products project.

Caveats

Datasets with the following caveats are not included in HASP coadd data products:

Observing Issues

  • Guide star acquisition failures
  • Observatory or detector failure events
  • EXPFLAG (exposure data quality flag) header keyword set to anything other than 'NORMAL'
  • EXPTIME (exptime) is zero seconds
  • Actual exposure time is less than 80% of the planned exposure time
  • FGSLOCK (fine guidance system lock) is not 'FINE', i.e., guide star tracking was not locked
  • The shutter was closed, denoted by the Take Data Flag (TDF)
  • Data flagged with bad archive data quality (resulted from filed alertobs)

Observation Parameters

  • POSTARG1 ≠ 0.0, i.e., there is a pointing offset

  • POSTARG2 ≠ 0.0 AND P!_PURPS ≠ 'DITHER', i.e., there is a pointing offset not in the dispersion direction

  • PATTERN1 = STIS-PERP-TO-SLIT and P1_FRAME = POS-TARG, i.e. there is a cross-dispersion direction pointing pattern (STIS only)

  • P1_PURPS = MOSAIC, i.e., there is a pointing mosaic offset pattern (STIS only)

  • OPT_ELEM (grating) = PRISM (STIS only)

  • APERTURE = BOA (Bright Object Aperture, COS only)

Target Parameters

  • Moving targets
  • Variable (These are not rejected by default, but some exposures may be removed by the code's flux-checking routine)

 

 

Observations of moving targets with the same name are co-added at the visit level but not at the program level.

Coadd data products are derived from standard pipeline .x1d observations. As such, additional reduction steps (defringing, cosmic ray removal, echelle blaze shifts, etc.) are not performed. Users must generate custom coadds to address these issues in default pipeline products.

Citations

If you are using HASP data in a published work, please cite the following paper: 

Instrument Science Report COS 2024-01

Also, please include the following acknowledgment in all publications that make use of HASP data: 

Based on observations obtained with the NASA/ESA Hubble Space Telescope, retrieved from the Mikulski Archive for Space Telescopes (MAST) at the Space Telescope Science Institute (STScI). STScI is operated by the Association of Universities for Research in Astronomy, Inc. under NASA contract NAS 5-26555.

References

Questions about HASP? For the fastest response, please visit the HST Help Desk Website.