There are two components to the EUVE Permanent Archive: the EUVE Telemetry Archive and the EUVE Science Archive. The Telemetry Archive contains a complete copy of the raw telemetry from the entire EUVE mission, and is intended as a permanent record of the mission. It is not primarily designed to support scientific archival research and access to the data will be difficult because of the unprocessed nature of the telemetry. The EUVE Science Archive, however, is intended for scientific research and is what most people probably have in mind when they refer to the "EUVE archive".

The EUVE Permanent Archive was created at the Center for EUV Astrophysics (CEA). The data products in the archive have been delivered from CEA to the NSSDC for distribution to the astronomical community. EUVE Science Archive pointed data products are being made available through archive sites at STScI and the HEASARC. At the time of this writing, EUVE continues to perform Guest Observer (GO) observations of new targets. The data from these observations is periodically delivered by CEA to the NSSDC and becomes available to the public through the archive sites.

This document details several aspects of the Science Archive and its use. It is intended as an introduction to the Archive for users who are already familiar with the EUVE GO data products that have been delivered since the start of the GO program. A more complete set of documentation will eventually become available when the EUVE Guest Observer Software User's Guide and EUVE Guest Observer Data Products Guide are rewritten to describe the Science Archive. Until those documents are revised, they are still useful for their in-depth discussions of data reduction and instrumental issues, but with regard to details about data formats or analysis steps, the descriptions in this introduction should be considered more up-to-date. Archive data users unfamiliar with the previous GO data products might want to consult the earlier documents to understand terms mentioned but not fully described here.

Note that the EUVE Science Archive User's Guide that is mentioned in the comments in the archive data headers refers to the rewritten EUVE Guest Observer Data Products Guide and therefore is not yet available.

The EUVE Science Archive contains multiple types of data. During the first six months of the EUVE mission in 1992-1993, a survey of the EUV sky was performed. The maps and catalogs from this survey have been either published in the astronomical literature and/or made available from CEA for several years. These products are part of the Science Archive but are not discussed further in this introduction. Since the end of the survey, EUVE has been performing pointed GO observations. These observations are primarily done with the Deep Survey/Spectrometer (DS/S) telescope; a very small number of pointed observations have been done with the Scanner telescopes. The Science Archive contains event lists and images from all these pointed observations.

Data from pointed observations is delivered at the time of the observation to the GO to whom it belongs. In the past, after the expiration of the proprietary period of the data, CEA also made exact copies of the data sets available to archival researchers. All of the data products delivered from CEA were in a standard format (a mixture of FITS tables, FITS images, and IRAF QPOE files), now called the GO format. Between March 1997 and February 1998, all existing EUVE pointed observations were reprocessed at CEA and data sets were produced in a new format, the Science Archive pointed format. The reprocessing produced a uniformly processed set of observations for the Science Archive.

A side effect of the reprocessing is that there is not a one-to-one correspondence between the observations in the Science Archive and those delivered in the past to GOs and archival users. Some observations were split in pieces, while others were merged together. Where these differences exist, the new Science Archive definitions of the observations are to be considered more accurate (or at least more consistent).

Occasionally, there is an anomaly during the observation or processing of a target that causes the resulting data to have unusual characteristics. When EUVE personnel noticed such an anomaly during the observation or its processing, written notes were appended to the FITS files in an additional binary table extension. We cannot ensure that the notes are comprehensive or always useful to researchers, but at least some information will be available if needed.

Next, we describe the contents of the archive FITS files in the nominal case. We use the abbreviations DS, SW, MW, and LW for the Deep-Survey imager and the Short-, Medium-, and Long-Wavelength spectrometers, respectively, and ScA, ScB, ScC for the Scanner A, Scanner B, and Scanner C telescopes.

• Extension 1, BINTABLE, extname = "valid_times"

• Extension 2, BINTABLE, extname = "ds_events"

• Extension 3, BINTABLE, extname = "sw_events"

• Extension 4, BINTABLE, extname = "mw_events"

• Extension 5, BINTABLE, extname = "lw_events"

• Extension 6, BINTABLE, extname = "quadrant"

• Extension 7, BINTABLE, extname = "adcnts"

• Extension 8, BINTABLE, extname = "orientation"

• Extension 9, BINTABLE, extname = "corrected_aspect"

• Extension 10, BINTABLE, extname = "ephemeris"

• Extension 11, BINTABLE, extname = "anomaly"

The extension numbers are for the nominal case. The final three extensions are optional, and if data for any detector is, for some reason, not present for an observation, the corresponding extensions are simply omitted from the FITS files. Therefore, extension numbers may vary. However, the order of the extensions is always the same when they are present. Always check the contents of your FITS files before using them.

In daytime DS/S data, the format is the same as for night data except that only the SW spectrometer data is present. This means that only one event list extension (sw_events) is included and the quadrant and adcnts extensions contain only columns relevant to the SW spectrometer.

In scanner data, the format is the same as for night DS/S data except that the appropriate scanner detectors replace the DS and spectrometer detectors. There is never daytime scanner data.

• Extension 1, IMAGE, extname = "ds"

• Extension 2, IMAGE, extname = "sw_night"

• Extension 3, IMAGE, extname = "mw"

• Extension 4, IMAGE, extname = "lw"

• Extension 5, BINTABLE, extname = "ds_limits"

• Extension 6, BINTABLE, extname = "sw_night_limits"

• Extension 7, BINTABLE, extname = "mw_limits"

• Extension 8, BINTABLE, extname = "lw_limits"

• Extension 9, BINTABLE, extname = "anomaly"

In daytime data, the format is the same as the night data except that only the SW spectrometer data is present; this means that only one image (named sw_day) and limits table (named sw_day_limits) are included.

In scanner data, the format is the same as for night DS/S data except that the appropriate scanner detectors replace the DS and spectrometer detectors. There is never daytime scanner data.

This is a brief description of the header keywords that appear in the archive FITS files.

The following are standard FITS keywords with a standard meaning and are not described here: SIMPLE, BITPIX, NAXIS, NAXISn, EXTEND, DATE, ORIGIN, CREATOR, and END.

These are standard FITS keywords used to describe an extension and its structure (some are BINTABLE specific): XTENSION, PCOUNT, GCOUNT, TFIELDS, EXTNAME, TTYPEn, FORMn, TUNITn, TDISPn, TLMINn, TLMAXn, TDMINn, TDMAXn, TNULLn, and INHERIT.

These are standard FITS keywords used in images for scaling and world coordinate system information, or identification: BSCALE, BZERO, CTYPEn, CRVALn, CRPIXn, CDELTn, CUNITn, and FILENAME.

• INSTTYPE

• INSTRUME

• DETNAM

• PHOTOND

• TIMEDEL

• OBJECT

• RA_OBJ, DEC_OBJ

• RA_PNT, DEC_PNT

• RA_PROC, DEC_PROC

• OBSERVER

• DATE-OBS, TIME-OBS

• DATE-END, TIME-END

• OBS_MODE

• DITHER

• DETMODE

• OFF-AXIS

• MOVING

• DAYNIGHT

• VALIDTIM

• RA_UNIT, DEC_UNIT

• EQUINOX

• RADECSYS

• TIMESYS

• TIMEZERO

• TIMEUNIT

• CLOCKCOR

• TIMEREF

• TASSIGN

• TSTART, TSTOP

• TIERRABS

• MJDREF

• EGOCSVER

• REFVERS

• EXPTIME

• RAWEXP

This is a description of the structure and meaning of all of the binary table and image extensions which can appear in Science Archive FITS files. No single FITS file will contain all these extensions. A table describing the columns of each binary table extension is given. The "Type" field refers to the FITS standard data types.

There are several categories of data in the binary table extensions. Event lists contain a row for every event detected by the EUVE detectors. Auxiliary values are reported in the telemetry at regular intervals during the observation, or derived from other auxiliary values or the event data. These values describe the state of the spacecraft or instruments. Other possibilities are time intervals, filter limits, or an ephemeris.

The archive format has been designed primarily as a FITS format. That means that considerable care has been taken to make the FITS files well organized and easily understood, and to provide complete meta-data in the headers. This means that the data is now much easier to read and manipulate using ordinary FITS tools. This is especially helpful to users who wish to bypass the IRAF file formats used in the EUV package and do their analysis in a non-IRAF environment. The description of the headers and tables given earlier should be sufficient for such users.

On the other hand, those users who wish to use the EUV package, or simply return to more familiar products, will have to reconstruct those products from the archive FITS files. Here are step-by-step instructions to unpack your data and reproduce the basic analysis steps, using IRAF and the EUV software package. The resulting products are very similar to the products that were delivered in the old GO format.

Note that the new procedure is rather different from the one you may have used in the past to unpack GO format data. In particular, you no longer need to use the EUV package task qprst.

At the time of this writing, the most recent publicly available version of the EUV software package is version 1.7. That version of the EUV package is compatible with IRAF 2.10.4, but not with the most recent IRAF release, 2.11. Release 1.8 of the EUV package will work with IRAF 2.11.

This document is applicable to either version of the EUV package. In the instructions below, where parameter lists are shown for IRAF tasks, they will be the IRAF 2.11 versions (any differences are minor). The same instructions can be used with IRAF 2.10.4/EUV 1.7, but you need to refer to unpacked ST tables and IRAF images, as described in the next section, rather than FITS extensions.

If you are using IRAF 2.11/EUV 1.8, you have several options available regarding the method you use to access the tables in your data set. Under IRAF 2.11, image and table processing tasks can directly read FITS image and binary table extensions. Several software tasks described below require tables from the data set as input. You can either unpack the FITS files into a one or more standalone ST tables, which you then specify as input where needed, or you can specify individual FITS binary table extensions directly as input to the software.

We will always use the last of these methods of specifying a table in the code examples below. You can choose another if it is more convenient. Also, keep in mind that the files don?t have to be in the current directory; you can specify the path to the file if needed.

There is one additional complication: the IRAF CL will try to interpret certain characters specially if you do not prevent it. In particular, it won?t like the "=" (equals sign) character in the above examples. So when typing an expression on the command line, you either can quote the entire expression (using single or double quotes), or escape the equals sign, as in

or

Of course, you can also avoid this problem by using one of the table references that doesn?t contain an equals sign, or by entering your parameters in the IRAF epar utility, where they are not interpreted by the CL.

Important IRAF 2.10.4 note: If you are using IRAF 2.10.4, directly reading from a FITS file is not possible. You must always unpack your FITS files as described below and then refer to the ST tables using one of the first two methods in the above table.

Convert the FITS extensions into ST tables and IRAF images. This step is optional when using IRAF 2.11/EUV 1.8, but required under IRAF 2.10.4. The conversion can be done using the strfits task in the TABLES package. Set up your parameters like this:

and/or

where you replace the pathname of the FITS files with whatever is correct for the observation in your system. The FITS files need not be in the current directory; simply supply the full path (they can even be on a CDROM). Obviously, if you don't require both the event lists and images you need only unpack one of the FITS files.

To unpack only a single extension, specify the desired extension by number, such as

Alternatively, you can simply copy the extension out using the tcopy task for a table, or the imcopy task for an image, which also conveniently allows you to specify the extension by name:

Users familiar with the format previously used to deliver EUVE Guest Observer and Archive data (referred to as the "GO format") will initially find the new format to be rather different. The new format has been designed from the start as a FITS format for EUVE data. That means that all design decisions have been made with the goal of producing FITS files that are portable and compatible with other standards or conventions for delivering event data as FITS.

Within those restrictions, however, the format has been designed to be as backwardly compatible as possible with the GO format. This means that the FITS files can be unpacked to ST Tables and QPOE files can be rebuilt. The resulting set of files will resemble the equivalent files in the GO format. In addition, all tasks in the EUV/IRAF analysis package have been modified (as of version 1.7) to work with both GO & archive data formats with little change from the user's point of view.

More subtle than the data format changes are changes to the science content of the data. These have been made to try to accommodate the majority of scientific uses of the data while minimizing the size of the archive. The changes also attempt to maximize the utility of more reduced products such as images. This lessens the likelihood that a researcher will need to work with the less reduced event list products, since the latter are larger and more complex.

These are the most significant changes:

• The archive data products do not contain the complete raw telemetry for the observation. In particular, they do not contain all events that were detected during the observation. For those events which are delivered, we still provide both remapped and detector coordinates, so that the data may still be reprocessed. However, in the GO format, it was possible to go to the raw data in the ST Tables to get access to events that were stripped out of the QPOE files. This is no longer possible. For example, there is no access to the stimpins in the spectrometers.
• In the spectrometers, we still deliver only events in a strip in remapped coordinates. The strip has now been narrowed from Y=720:1299 to Y=875:1174 (about 22 arcmin). For extended sources for which the strip would render the data less useful, we make an exception and open the strip to Y=1:2048 (the whole field of view).
• In the DS data, while we still process only events in the Lexan quadrant, we no longer deliver only a horizontal strip of events in the range Y=720:1299 in remapped coordinates. All events in the Lexan quadrant are now included in the event list and image.
• The vast majority of engineering monitors-the columns in tables 1-8 in the GO format-have been dropped. Only those monitors which are believed to be in common use for data analysis will be in the archive: aspect, position, quadrant counts, and A-D counts (along with primbsch/deadtime information).
• The monitors have been reorganized. The quadrant extension contains quadrant counts previously in table2 and primbsch and deadtime info previously in the primbsch table. The adcnts extension contains A-D counts previously in table1. The orientation extension contains aspect and position information previously in table1.
• The event list FITS extensions are sorted by time, whereas the QPOE files in the GO format were sorted by position.
• In event lists containing sky coordinates, we have changed the RA values to be in the range 0 to 360 degrees. In previous data, it was in the range -180 to 180 degrees.
• All event lists now contain pulse height information for each event for which it was available (meaning all events taken in WSZ mode).
• The header keywords in the tables and images have been completely reworked to meet FITS standards or guidelines from the HEASARC for event list data. In particular, the cards JDREF, OBSERVAT, and PI_NAME have been deleted. The cards INSTRUME, TELESCOP, TIMESYS, and ORIGIN now have different values. And many, many new cards have been added.
• The data types of some table columns have changed from floating point to integers: primarily all the counts in the quadrant extension.
• The DS image we produce has now been filtered with an upper limit on the quadrant counts in quadrant 1 (the Lexan filter). The upper limit is 50 counts per two frames for all but a handful of the brightest DS sources; for those, we increase it to 150. In GO format data, there was no upper limit in the DS count rate filter.
• For moving targets, we now use an ephemeris to correct the aspect for the motion of the target on the sky. The ephemeris and the corrected aspect values are included with the observation data set.
• For some moving targets, in addition to the strips of events mentioned above, we also remove events in certain portions of the detector specified in detector coordinates, such as the stimpins and unused quadrants. This is to prevent these detector features from being smeared all over the image of the source due to the large source motion on the detector.

Where is the DISPAXIS header keyword that euvextract complains is missing from my image?

How do I see the ephemeris used in my moving target analysis before parallax corrections were applied?

How can I rebuild QPOE files for all of the event lists at once?

Where are the extracted spectra?

Mark Abbott

Center for EUV Astrophysics

26 June, 1998