EUVE Guest Observer Software User’s Guide

Extreme Ultraviolet Explorer

Guest Observer Center Documentation

A. Miller and M. Abbott

Version 1.5

September 13, 1995

All software and documentation Copyright \begin{matrix}c \\ ◯ \end{matrix}1991, 1992, 1993, 1994, & 1995, The Regents of the University of California. All Rights Reserved. Some software tasks described in this document make use of routines which are Copyright \begin{matrix}c \\ ◯ \end{matrix} 1986 Numerical Recipes Software.

5/97

2 5/97

TABLE OF CONTENTS

PREFACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i

1 Using Telemetry in ST Tables 1-1

1.1 Data Quality Selection Using EUVE Telemetry . . . . . . . . . . . . . 1-1 1.1.1 DQSELECT creates and installs QPOE filters . . . . . . . . . 1-1 1.1.2 Count rate limits and signal-to-noise . . . . . . . . . . . . . . . 1-12

1.2 Specialized Data Quality Selection Tasks . . . . . . . . . . . . . . . . . 1-14 1.2.1 MKPRIMBSCH makes deadtime correction tables . . . . . . 1-15 1.2.2 BACKMON makes tables of geographic orbit parameters . 1-18

1.3 Aspect and Pointing Problems . . . . . . . . . . . . . . . . . . . . . . . . 1-22 1.3.1 MKASPECT creates coordinate tables from aspect records 1-23 1.3.2 ASPCORR creates new aspect tables from source positions 1-29 1.3.3 QCALC calculates quaternions . . . . . . . . . . . . . . . . . . . 1-37

2 Selecting Events From QPOE Files 2-1

2.1 QPOE I/O Interface Expressions . . . . . . . . . . . . . . . . . . . . . . 2-2 2.1.1 General syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2 2.1.2 Attribute filters have attributes as keywords . . . . . . . . . . 2-3 2.1.3 Useful function keywords . . . . . . . . . . . . . . . . . . . . . . . 2-5 2.1.4 Some QPOE expression rules . . . . . . . . . . . . . . . . . . . . 2-8

2.2 QPOE Macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9 2.2.1 Reading and writing macro definitions . . . . . . . . . . . . . . 2-9 2.2.2 QPMEDIT displays and edits QPOE macros . . . . . . . . . . 2-10 2.2.3 TFILTER creates QPOE time filters from ST tables . . . . . 2-12

2.3 Making Spectral Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-15 2.3.1 QPMKIM makes corrected EUVE spectral images . . . . . . 2-15 2.3.2 Making images with IMCOPY . . . . . . . . . . . . . . . . . . . 2-18

2.4 Attribute Binning and Timing Analysis . . . . . . . . . . . . . . . . . . 2-19 2.4.1 QPBIN rebins on any attribute . . . . . . . . . . . . . . . . . . . 2-19 2.4.2 PERIODINT calculates times for different phase points … 2-23

3 Extracting and Fluxing EUVE Spectra 3-1

3.1 Extracting EUVE Spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2 3.1.1 EUVEXTRACT creates and documents EUVE spectra … 3-2 3.1.2 Extracting EUVE spectra with APALL . . . . . . . . . . . . . 3-6 3.1.3 MKPROFILE fits profiles at specified intervals . . . . . . . . 3-18

3.2 Spectral Corrections and Flux Calibration . . . . . . . . . . . . . . . . 3-19 3.2.1 DIVEA divides by effective area . . . . . . . . . . . . . . . . . . 3-20 3.2.2 SUBORDER removes higher order counts . . . . . . . . . . . . 3-21 3.2.3 EUVHCLEAN strips obsolete header cards from spectra . . 3-28

5/97

3.2.4 SPECPRINT prints spectrum and error band . . . . . . . . . 3-29

4 Simulations and Model Comparisons 4-1

4.1 Making Simulated EUVE Spectra . . . . . . . . . . . . . . . . . . . . . . 4-1 4.1.1 ISM calculates transmission of the interstellar medium … 4-1 4.1.2 SPECMOD makes simulated EUVE spectra . . . . . . . . . . 4-3

4.2 Statistical Comparisons Between Spectra . . . . . . . . . . . . . . . . . 4-7 4.2.1 COMPSPEC makes statistical comparisons . . . . . . . . . . . 4-7

5 Doing Your Own Data Reduction 5-1

5.1 Running The EUVE Event Pipeline, CEP . . . . . . . . . . . . . . . . 5-1 5.1.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2 5.1.2 Processing scripts control cep . . . . . . . . . . . . . . . . . . . . 5-4 5.1.3 Other processing options with cep . . . . . . . . . . . . . . . . . 5-8

5.2 Rerunning CEP to Correct Aspect Problems . . . . . . . . . . . . . . 5-10 5.2.1 Correcting constant aspect error . . . . . . . . . . . . . . . . . . 5-11 5.2.2 Finding Deep Survey image centroids . . . . . . . . . . . . . . 5-13

6 Converting EUVE Data Products to FITS Format 6-1

6.1 Creating FITS Files with STWFITS . . . . . . . . . . . . . . . . . . . . 6-1 6.2 QPWST converts QPOE files to sets of ST tables . . . . . . . . . . . 6-2

A Using the QPOE I/O Interface A-1

A.1 QPOE File Headers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 A.1.1 Types of QPOE header parameters . . . . . . . . . . . . . . . . A-1 A.1.2 Reading the imhead display . . . . . . . . . . . . . . . . . . . . . A-3 A.1.3 Editing QPOE header parameters . . . . . . . . . . . . . . . . . A-5 A.1.4 Processing EUVE QPOE files with the XRAY package … A-6

A.2 Additional Notes on QPOE Expression Syntax . . . . . . . . . . . . . A-7 A.2.1 Multiple intervals and filters . . . . . . . . . . . . . . . . . . . . . A-7 A.2.2 Special punctuation for attribute filters . . . . . . . . . . . . . A-9

A.3 Macro Types and Hierarchies . . . . . . . . . . . . . . . . . . . . . . . . . A-10

A.3.1 Global macros and the QPDEFS file . . . . . . . . . . . . . . . . A-12 A.4 QPOE Interface Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . A-13

A.4.1 Accommodating long macros . . . . . . . . . . . . . . . . . . . . A-14 A.5 Trouble-shooting your QPOE interface expressions . . . . . . . . . . A-17

B Plotting ST Table Columns with TBPLOT.IGI B-1

C Obsolete IRAF/EUV tasks C-1 C.1 LISTSPEC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 C.2 EUVCOMBINE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 C.3 MKEUVSPEC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-3

5/97 D Other Resources D-1

D.1 Other Useful IRAF Packages . . . . . . . . . . . . . . . . . . . . . . . . . D-1 D.1.1 Basic image operations with generic IRAF tasks . . . . . . . D-1 D.1.2 Operating on EUVE spectra with the onedspec package . . D-1 D.1.3 Manipulating QPOES with PROS XRAY tasks . . . . . . . . D-2 D.1.4 The TABLES layered packages . . . . . . . . . . . . . . . . . . . D-3

D.2 Electronic Resources for Guest Observers . . . . . . . . . . . . . . . . . D-5 D.3 Documents You Can Get From IRAF . . . . . . . . . . . . . . . . . . . D-6 D.4 Writing IRAF-Compatible Software . . . . . . . . . . . . . . . . . . . . D-7 D.5 General Information on FITS . . . . . . . . . . . . . . . . . . . . . . . . . D-8

5/97 6 TABLE OF CONTENTS 5/97

LIST OF EXAMPLES

1-1 Creating the standard exposure filters . . . . . . . . . . . . . . . . . . . 1-8 1-2 Creating a new earth-blockage filter . . . . . . . . . . . . . . . . . . . . 1-20 1-3 Making a new aspect table for an observation of the moon . . . . . 1-34

2-1 Making a phase-limited spectrum . . . . . . . . . . . . . . . . . . . . . . 2-13 2-2 Making a light curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-21

3-1 Extracting a spectrum with euvextract . . . . . . . . . . . . . . . . . . . 3-5 3-2 Extracting a continuum spectrum with apall . . . . . . . . . . . . . . . 3-14 3-3 Subtracting higher-order contributions with suborder . . . . . . . . . 3-26

4-1 Creating spectra with specmod . . . . . . . . . . . . . . . . . . . . . . . . 4-6 4-2 Compare specmod outputs with compspec . . . . . . . . . . . . . . . . 4-11

5-1 Rerunning the pipeline with new reference data . . . . . . . . . . . . 5-7 5-2 Reprocessing an observation of the Moon . . . . . . . . . . . . . . . . . 5-11 5-3 Reprocessing an observation to correct for flexure . . . . . . . . . . . 5-15

A-1 Working around a deffilt . . . . . . . . . . . . . . . . . . . . . . . . . . . A-9 A-2 Adapting the QPOE interface for large macros . . . . . . . . . . . . . A-16

5/97 8 LIST OF EXAMPLES 5/97

LIST OF FIGURES

1-1 dqselect parameter file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2 1-2 Detector count rates displayed in dqselect. . . . . . . . . . . . . . . . . 1-9 1-3 Spectrometer detector count rate displayed in dqselect . . . . . . . . 1-12 1-4 a) Detector count rate enlarged in dqselect . . . . . . . . . . . . . . . . 1-13 1-4 b) Display of detector count rate in igi . . . . . . . . . . . . . . . . . . . 1-14 1-5 Parameter file for mkprimbsch . . . . . . . . . . . . . . . . . . . . . . . . 1-16 1-6 Deadtime/primbsch correction column displayed in dqselect …. 1-17 1-7 backmon parameter file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-19 1-8 Parameter file for mkaspect . . . . . . . . . . . . . . . . . . . . . . . . . . 1-24

1-9 RA vs. TableTime from mkaspectds.tab . . . . . . . . . . . . . . . . . . . . . . 1-25 1-10 RA vs. DEC from mkaspecds.tab, plotted with igi . . . . . . . . . . . . . . . 1-26

1-11 Coordinates from mkaspect for a dithered observation . . . . . . . . 1-27 1-12 Orientation of detector coordinates in the DS/S . . . . . . . . . . . . 1-28 1-13 Using mkaspect output to locate sources. . . . . . . . . . . . . . . . . . 1-30 1-14 Vector geometry for correcting changing aspect error. . . . . . . . . . 1-31 1-15 Sample parameter file for aspcorr . . . . . . . . . . . . . . . . . . . . . . 1-32 1-16 Moon in LW after standard processing . . . . . . . . . . . . . . . . . . . 1-34 1-17 ST table input to aspcorr . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-35 1-18 Columns of an output table from aspcorr. . . . . . . . . . . . . . . . . . 1-35 1-19 Moon after processing with corrected aspect . . . . . . . . . . . . . . . 1-36

2-1 Parameter file for qpmedit . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11 2-2 Parameter file for tfilter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12 2-3 Parameter file for qpmkim . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-16 2-4 Parameter file for qpbin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-19 2-5 Light curve constructed with qpbin . . . . . . . . . . . . . . . . . . . . . 2-22 2-6 Light curve with error bars . . . . . . . . . . . . . . . . . . . . . . . . . . 2-24 2-7 Parameter file for periodint . . . . . . . . . . . . . . . . . . . . . . . . . . 2-25

3-1 Parameter file for euvextract . . . . . . . . . . . . . . . . . . . . . . . . . 3-3 3-2 Averaged and smoothed backgrounds from euvextract . . . . . . . . 3-4 3-3 Spectrum profiles in implot . . . . . . . . . . . . . . . . . . . . . . . . . . 3-6 3-4 Spectrum extracted with euvextract . . . . . . . . . . . . . . . . . . . . 3-7 3-5 Parameter file for apall. a) Global parameters . . . . . . . . . . . . . . 3-8 3-5 b) aperture editing parameters . . . . . . . . . . . . . . . . . . . . . . . . 3-9 3-5 c) Spectrum tracing parameters . . . . . . . . . . . . . . . . . . . . . . . 3-10 3-5 d) Extraction and summing parameters . . . . . . . . . . . . . . . . . . 3-11 3-6 Display in apedit with a user-defined aperture. . . . . . . . . . . . . . 3-15 3-7 Spectrum trace displayed in aptrace. . . . . . . . . . . . . . . . . . . . . 3-16 3-8 Plot of fit point residuals in aptrace. . . . . . . . . . . . . . . . . . . . . 3-17 3-9 SW spectrum extracted with apall . . . . . . . . . . . . . . . . . . . . . . 3-18 3-10 Parameter file for mkprofile . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19

5/97

3-11 Parameter file for divea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20 3-12 Parameter file for suborder . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-23 3-13 Display of spectra in suborder . . . . . . . . . . . . . . . . . . . . . . . . 3-24 3-14 Second order MW in LW from suborder . . . . . . . . . . . . . . . . . . 3-27 3-15 Enlarged section of order-subtracted spectrum . . . . . . . . . . . . . 3-28 3-16 Final output from suborder . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29 3-17 Parameter file for specprint . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29

4-1 Sample parameters for ism . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2 4-2 Effect of ism on a hot DA white dwarf model . . . . . . . . . . . . . . 4-2 4-3 specmod parameters for making simulated EUVE spectra . . . . . . 4-3 4-4 Example SW Detector pset for use with specmod . . . . . . . . . . . . 4-5 4-5 Output spectra from specmod . . . . . . . . . . . . . . . . . . . . . . . . . 4-8 4-6 Sample parameters for compspec . . . . . . . . . . . . . . . . . . . . . . . 4-8 4-7 Logic to determine variance of input to compspec . . . . . . . . . . . 4-10

5-1 Schematic of directories and files for running cep. . . . . . . . . . . . 5-3 5-2 cep parameter file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-4 5-3 Incorrect imaging due to coordinate/pointing offset. . . . . . . . . . . 5-13 5-4 Sample parameter file for imcntr . . . . . . . . . . . . . . . . . . . . . . . 5-14

6-1 Sample parameter file for stwfits . . . . . . . . . . . . . . . . . . . . . . . 6-2 6-2 Sample parameter file for qpwst . . . . . . . . . . . . . . . . . . . . . . . 6-3

B-1 ROLL vs. time in igi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-2 B-2 RA vs. Dec for an interrupted observation . . . . . . . . . . . . . . . . B-3 B-3 RA vs. Dec for a dithered observation . . . . . . . . . . . . . . . . . . . B-4

C-1 Sample parameters for euvcombine . . . . . . . . . . . . . . . . . . . . . C-2 C-2 Sample parameters for mkeuvspec . . . . . . . . . . . . . . . . . . . . . . C-4

5/97

LIST OF TABLES

2-1 QPOE interface keyword expressions. . . . . . . . . . . . . . . . . . . . 2-4 2-2 Summary of PLIO region specifications . . . . . . . . . . . . . . . . . . 2-7

4-1 Which input variance is used in χ^{2}? . . . . . . . . . . . . . . . . . . . . . 4-11

A-1 EUVE QPOE file header parameters . . . . . . . . . . . . . . . . . . . . A-4 A-2 Form of QPOE file macro references . . . . . . . . . . . . . . . . . . . . A-11

5/97 12 LIST OF TABLES 5/97

PREFACE

Purpose

This User’s Guide describes the IRAF/EUV layered software packages (also called the euv package)^{1} furnished to all EUVE Guest Observers, and the use of the euv layered packages in IRAF. The functioning of each task is described in general, with a sample parameter file, and notes on running many tasks with EUVE data are provided. The most important and frequently used applications to reduction of EUVE Guest Observer data are described, and some are illustrated with examples. Please refer to the IRAF help pages on each task for more detailed descriptions of the tasks’ algorithms, their parameters, output, and command line formats. Information about the EUVE Guest Observer data products, their format, and the EGO Center nominal reduction of GO data are now included in the EGO Center Data Products Guide, which is also delivered with every data set.

The euv software package is integrated with IRAF, the Imaging Reduction and Analysis Facility (IRAF) software, using version 2.10.3 or later. This User’s Guide is intended for use with all versions of the IRAF/EUV software numbered 1.5, and the EUVE reference data, version 1.9 or later. The addition of the tables layered packages, version 1.3.1 or later, is also used at many stages. The IRAF system is readily available, and is widely used by the astronomical community. See the resources in Appendix D for information on obtaining IRAF, additional copies or updates of the IRAF/EUV layered packages, and the EUVE reference data set.

All the data products on the standard EUVE Guest Observer data tape are furnished as FITS format tables and images. GO’s who wish to work in another software system should obtain and read the EGO Center Data Products Guide, to become familiar with the various data files, their production, and important instrumental effects. They may occasionally want to refer to this Software User’s Guide also, as it describes the IRAF/EUV tasks used to produce the data products.

What This Guide Assumes

This User’s Guide is not intended as an introduction to IRAF, and it is assumed that the user is familiar with starting IRAF and setting its environment variables, package structure, editing parameter files, and running tasks and the use of IRAF’s graphics cursor mode. New IRAF users should obtain a copy of A Beginner’s Guide to Using IRAF, by Jeanette Barnes, available from the NOAO ftp site. Detailed instructions for obtaining this and other IRAF documentation are in Appendix D.

We hope this Guide will be equally useful to you in reducing your data at the EGO

5/97

1Copyright \begin{matrix}c \\ ◯ \end{matrix}1991, 1992, 1993, 1994, & 1995, The Regents of the University of California. All Rights Reserved.

Center in Berkeley and as a guide to using the IRAF/EUV software at your home institution. This document assumes that you have access to your data on disk at the EGO Center, or that you are working on a system with IRAF version 2.10.4 or later, installed. The IRAF/EUV software packages should be installed in IRAF, along with the EUVE reference data, according to the installation instructions on the data tape. It is assumed that your data have been read from tape to disk, and that all FITS files have been converted to their internal IRAF formats. Specific instructions for unpacking the data and the standard arrangement of data directories and files on disk are given in Chapter 0 of the EGO Center Data Products Guide. If you are working at the EGO center, you need only obtain a password for the appropriate guest account, log in, and start IRAF before beginning.

Many of the euv tasks are written to be applicable to other types of event-based data, and some tasks have parameters that may not seem strictly necessary for processing EUVE data. In each case, the hierarchy of parameters and the default values assigned to them are arranged to facilitate processing EUVE data. So when in doubt, using the default value is usually the best course. Some exceptions have to do with running IRAF in a VMS environment, and are specified in the notes on the relevant tasks.

Style Conventions

Throughout this guide, the names of IRAF tasks are set off in boldface type. The standard IRAF notation package.task is used for the first instance of a task name, to show which package must be loaded for access to the task. Typed commands, IRAF parameter names, and commands typed in interactive mode are displayed in typewriter font. Typewriter font is also used for terms internal to IRAF files, such as the column names of ST tables or QPOE attribute names. What the user types is set off with an IRAF style prompt, e.g.

          eu> taskname arg1 arg2 ...

for running a task in the euv layered packages. Other file names, directory names, program names, titles, and non-IRAF commands are set in italic type.

Task names in subsection titles are set off in CAPITALs to make them easy to spot when scanning the table of contents. They are NOT generally capitalized in IRAF. Tasks that are part of the IRAF/EUV layered packages usually appear at the beginning of a subsection title. Task names elsewhere in a subsection title indicate other IRAF packages.

Examples and processing recipes in this Guide show two methods for running most IRAF tasks. The user who wishes to edit parameter settings before running a task can enter eparam and use the sample parameter files as a model. Then typing a “:go” while in the editor will start the task running. Users who wish to avoid using eparam will find that example command lines with the necessary parameter settings have been provided for many common applications.

5/97

text in boxes deserves your special attention. It is used to indicate items of special interest, and of un-interest. Boxes with titles like “Do I need to run < task >?” are meant to help you avoid reading about tasks that you may not need to use; others are meant to inform you about important points or potential problems.

Organization of This Guide

The chapters approach the tasks and their application to data products in roughly the order that most GO’s use them. The order is based on the assumptions that many GO’s will want to perform some data quality selection on the basis of their science criteria, and that most will want to re-extract their one-dimensional spectra. The introduction to each chapter, and the list below provide an overview to help GO’s pick their starting point, if the front-to-back approach is not appropriate. In addition to the usual table of contents and list of figures, this User’s Guide includes a list of examples, to help users quickly locate the example material that most closely matches a particular task.

1. Using Telemetry in ST Tables – focuses on using the telemetry tables to make time filters for data selection. Commonly used types of filters are constructed in the examples. Tasks that create new ST tables by reprocessing the telemetry information are introduced, including a specialized task for use in correcting aspect errors.

2. Selecting Events From QPOE Files – describes tasks and operations performed on the QPOE files. We include an introduction to QPOE interface expressions, tasks for installing filter macros in the files, and for making spectral images, and light curves.

3. Extracting and Fluxing EUVE Spectra – treats the extraction of 1-d spectra using two tasks, the IRAF/EUV task euvextract and the standard NOAO task apextract.apall. A task to dump EUVE-format spectral images to ASCII files is provided for those who wish to continue their analysis outside IRAF.

4. Simulations and Model Comparisons – describes tasks for simulating the output of the EUVE Spectrometer from an input spectrum, determining flux at the telescope from a target spectrum, and for making statistical comparisons between spectra.

5. Doing Your Own Data Reduction – gives instructions for reprocessing the EUVE event table, table0.tab, with the euv event pipeline task euvred.cep to produce new QPOE files. GO’s may need to reprocess their telemetry if an improved reference data set becomes available, or to correct problems with aspect reporting.

6. Converting EUVE Data Products to FITS Format – describes the process of

converting EUVE data and reduction products to FITS files for transport on tape or via ftp. The process of packing QPOE files into FITS files has two stages, but allows QPOE’s to be transported between machines with different types of binary encoding.

5/97

Appendices

A Using the QPOE I/O Interface – gives more general and detailed information about QPOE interface expressions and QPOE macros, for those who wish to explore the possibilities provided by the QPOE I/O interface.

B Plotting ST Table Columns with tbplot.igi – describes an alternative to dqselect

for plotting the columns of the telemetry tables. While tables.tbplot.igi does not have the capability of dqselect for creating data selection filters, it can show a plot of every entry in a table column, rather than sampling the table.

C Obsolete IRAF/EUV Tasks – Beginning with this version (1.5), a number of tasks from previous euv versions have been transferred to the package euvobsolete. While it is hoped that these tasks won’t be used in future because their functionality has been replaced by better tasks that are more user-friendly, information about them is preserved in Appendix C.

D Other Resources – A short guide to obtaining more information on IRAF packages, IRAF user manuals and programming guides, and FITS documentation.

Questions and Feedback

Please send comments, suggestions, and questions to:

egoinfo@cea.berkeley.edu

Your questions will be answered as quickly as possible by the appropriate EUVE GO personnel. Your comments and suggestions for improving this Software User’s Guide are very important to us, and we appreciate your feedback.

Thanks,

Anne Miller EUVE Guest Observer Program technical writer

Mark Abbott EUVE Guest Observer support scientist

5/97

Chapter 1

USING TELEMETRY IN ST TABLES

1.1 Data Quality Selection Using EUVE Telemetry

The first part of this chapter is about using the information in your telemetry tables, (mostly table1.tab) to create data quality selection filters. This method of data selection is based on the event times recorded in both the QPOE files and the telemetry tables. When a data quality selection filter is applied to the QPOE file, the time tag on each event is compared to a list of “good time” intervals, made from the telemetry tables, to determine if that event should be included in subsequent processing. The 2-d spectral images on each GO data tape are the result of this type of filtering; each event’s time was compared to a time filter to determine whether it would be in the binned image. The criteria used in nominal processing are not especially stringent; every event from periods when the detector was operating at a normal voltage and the source was visible are included. This is a filtering known as choosing all events from “valid times”–the most basic criterion for usable data.

Some Guest Observers want to further restrict this choice of events, to eliminate periods of high background, at orbit dawn or dusk, or when the satellite crosses the South Atlantic Anomaly (SAA). Each GO’s priorities may be different; where one investigator might accept a small loss of source data to omit periods of high background, another might wish to retain source counts at some cost to S/N.

In these cases, one must reselect the events from the QPOE file, using a new selection rule that passes only the events which meet the GO’s criteria. This chapter should help GO’s create filters in the form of tables of “good times” that select the best data set for their science. We will see how to pass those filters to the QPOE interface and create new images from the QPOE file in Chapter 2.

GO’s who wish to use the 2-d spectral images created by the EGO Center nominal data reduction process without further filtering can proceed to Chapter 3, to determine whether they wish to re-extract their one-dimensional spectra.

1.1.1 DQSELECT creates and installs QPOE filters

The task euvred.dqselect is an interactive tool which creates plots of the data in ST table columns vs. time, constructs filter strings defining good time intervals based on the data, and writes the string in a QPOE file or table. The time-plots are like strip charts of each column, on which the user can set and mark limits on the values. Internal task commands create tables of valid times based on when the data transit those limits, and install the start and end times in a QPOE file as a data selection

5/97

1-2 CHAPTER 1. USING TELEMETRY IN ST TABLES

filter. We will describe using dqselect to set limits on the various columns of EUVE telemetry tables, which correspond to different monitors of the instruments and spacecraft, and constructing time filters which delimit intervals when the monitor values were “good” ^{1}.

Please read the dqselect help page for a detailed description of the operation of the task. Refer to Appendix A of the Data Products Guide for definitions of the data in the telemetry tables and indications of their usefulness to GO’s.

Do I need to use dqselect?

dqselect is made to facilitate data quality filtering, which actually takes place in the process of creating an IRAF spectral image from the QPOE file. All EUVE QPOE’s have had a minimal data quality filter, the valid times (exposure) filter, applied when they are binned to make the 2dimensional images, sw.imh, mw.imh, lw.imh, and ds.imh. GO’s who wish to add criteria of their own to the definition of usable data will probably want to use dqselect to translate those criteria into good time intervals, or some other sort of event attribute filter.

Since the exposure filters are also installed in the QPOE files as macros (a process explained below), there is usually no need to duplicate creation of the default exposure filter. Even if the GO has re-run the event pipeline task cep to produce new QPOE files, the same exposure filters can often be reinstalled, from tables supplied on the tape. New exposure filters must be made only if the reprocessing changes the good time intervals, e.g. by subdividing or summing observations.

A sample parameter file for running dqselect is shown in figure 1-1.

         eu> lpar dqselect

     tables = "observation/table1.tab" Monitor Tables
    (limits = "")             Limits Tables
       (gap = 1)              Time Gap
     (imode = "line")         Input mode
    (format = "13.3f")        Interval format control
    (device = "stdgraph")     Graphics device
      (mode = "al")

Figure 1-1: dqselect parameters with default values for processing EUVE data.

1The term “good times” will always be used with the to imply a process of selecting the best data for the particular purpose at hand, while the word “valid” is used for the minimal data quality selection performed in the EGO Center’s nominal reduction.

5/97

Data Quality Selection Using EUVE Telemetry 1-3

Preparing to run dqselect

□ tables: the tables to be loaded into the task’s buffer. The value in the example will load only table1.tab, which has all the most frequently used EUVE monitors.

□ imode: the task can process input in either “line” or “cursor” mode. In line mode (the default), users enter commands to a command line cursor, choosing from a displayed menu. The alternative is the normal IRAF cursor mode; all line mode commands are still accessible as colon commands, as well as many commands that use cursor reads as input.

□ limits: parameter used to load the name of a table of pre-defined limits, such as limits saved from a previous session with dqselect.

□ gap: The parameter gap defines the minimum number of consecutive readings (lines) that must be missing from the table before dqselect considers the data stream interrupted. Gaps are shown in the display as breaks in the curve of displayed values. Make gap equal to 1 on the first viewing, so that dqselect notices all the gaps, major or minor. See the subsection “Gaps in the data” below, for an explanation of gaps and their consequences.

The next few sections describe the main functions of dqselect, in the order they are normally used. Most dqselect sessions cycle through four steps: display the monitors of interest, edit each monitor to set the desired upper and lower limits, run gtgen to generate the start and end times for the resulting good time intervals, and install the good times as a filter in the QPOE file. See Example 1.1 for a more detailed example, showing the dqselect display.

To begin running dqselect from the command line, type:

          eu> dq observation/table1.tab

Monitor display

Once dqselect has been started with a table loaded in memory, the command:

          Command> display monitor

produces a plot of the named monitor against time on the STDGRAPH device. The argument is case insensitive; you don’t have to know whether the column name has any capitalization. The plot’s time axis is numbered relative to the first time-tag in the loaded tables. Up to four monitors can be displayed on STDGRAPH at once. After that, one must remove a monitor from the display by using the command hide before a new monitor can be plotted. Hidden monitors can be resummoned to STDGRAPH later in the session by reissuing the display command with three or fewer monitors in the window.

5/97

Display sampling of ST tables

Due to the limitations on most STDGRAPH devices, dqselect must sample any table longer than about 500 lines. This means that most EUVE telemetry tables, which typically span tens of thousands of seconds, will be heavily sampled for display in dqselect. Sampled display means that high-frequency variations in the data may not be visible in dqselect, but this not usually problematic unless one is setting very narrow limits on the detector count rates (see section 1.1.2 below). See Appendix B if monitor plots at full resolution are required.

Since all EUVE observation data are selected from only nighttime telemetry, when the satellite is in the earth’s shadow, the daytime parts of each orbit also appear as large gaps in the monitor displays. This is not a result of the the sampling, and is completely normal.

Setting limits

Limits can be set on each monitor displayed, establishing the upper and lower bounds of an acceptable range of monitor values. The capability to set limits on several monitors allows the user to apply several criteria at once. Use the command:

        Command> edit monname

to set or edit the limits on the monitor monname. Issuing the command causes dqselect to enter cursor mode, and print a prompt:

          Please pick the first limit value ...

Limit values can then be entered as cursor reads by placing the cursor and entering any keystroke^{2}, or typed in as colon commands with the syntax:

         :y limit.decimal
         and again...

         :y limit.decimal

The limits can also be entered as a combination of colon commands and cursor reads. The order does not matter; the lower number will be treated as the lower limit, regardless of whether it was entered first or second.

Once limits are set, they appear on the monitor display as horizontal bars on the vertical axis. All the current limits can be saved as an ST table at any time, by using the command:

2or by reading the cursor position from mouse clicks.

5/97

          Command> limsave mylimits

to create the table mylimits.tab in the current working directory. Tables of saved limits can be reloaded into dqselect for a later session.

Defining good time intervals

Once appropriate limits are set, a template for valid times can be created from the times when each monitor transits the limit value. Valid times are computed by the dqselect command:

          Command> gtgen

This command causes dqselect to find all the times when each monitor transits one of its limits. Each time when a monitor are set enters the range prescribed by its limits becomes a “start time,” and each time it goes above or below the range becomes an “end time.” Each start time/end time pair makes a good time interval. All the sets of good times for all monitors with limits are intersected, producing the set of intervals when all the monitors are within their limits. If the tables are long, or many monitors are involved, gtgen may take a while to run.

The sampling that is implemented to display monitors does not affect the generation of good times. This means the list of intervals made by gtgen may show breaks in the valid time intervals not visible in the graphic display. In these cases it is the good time intervals that are accurate, not the display.

Saving the results of gtgen

The start and end times calculated with gtgen are stored in dqselect’s memory, but if they are to be used for selecting events from a QPOE file, they must be saved somehow before quitting from dqselect. dqselect can write the start and end times to an ST table, and can also format them into a filter string and write the string as a macro in a QPOE file.

The command filter is used to convert the list of start and end times into a standard QPOE I/O interface format, write this filter string in a named QPOE file, and give the filter string a name, making it a macro:

          Command> filter QPOE name macro name

The start and end times can also be saved in a table in the current working directory

5/97

called mygtimes.tab using the command:

          Command> gtsave mygtimes

Saving the good times to a table is always a good idea; they don’t usually take up much space, and it’s often handy to have them around later for variations or fixes to your reduction. For example, if the user wants to intersect separate sets of good times using the task euvtools.tfilter, as in section 2.2.3, the good times must be present in ST tables like those made by gtsave.

Repeated use of gtgen

An important point to remember is that whenever gtgen is run in dqselect, any set

of good times that already exists in the internal buffer will be intersected with the newly computed

good times. dqselect always starts up with an empty good times buffer, but if you issue the gtgen command more than once before quitting, the good times buffer should usually be cleared each time by issuing the command:

          Command> gtnew

Do not use the gtnew command until you have saved any useful good times as a table or a QPOE macro.

Likewise, limits which are not meant to be re-used when gtgen is re-issued must be removed with the command delete, prior to re-running gtgen. This must be done even though the monitors may not be visible on the screen, having been displaced by those displayed more recently. The limits still exist, and will still be used to generate start and end times. For this reason, it is a good idea to keep a written log of each session with dqselect, recording all major commands so that you can keep track of what limits and good times exist in memory.

The gtgen command merges time intervals by default, because this feature can be very useful. If the user wants to add a new criterion for good data to those used for a previous set of good times, they need not recreate the entire process. The command gtload can be used to load a previously saved table of the old good times into the buffer, after dqselect has been started with the data table containing the monitor to be “added.” The user then displays and edits limits on the new monitor, and uses gtgen to intersect the good times for the new monitor with the old ones.

Gaps in the data

The frequency of readings in EUVE telemetry is predetermined, but inevitably there are some gaps, or periods when there is no data, for each monitor’s history. Lines may be missing from the table altogether, or there may be INDEF values in some of the columns. During these gaps, the values of the monitors are unknown, but gaps do not necessarily indicate any malfunction or change in the monitor; they are

5/97

just lapses in reporting. Gaps between the time stamps of successive lines in the table occur for two main reasons:

1. First of all, the data included in EUVE GO data sets is selected from nighttime telemetry only, to obtain the best signal-to-noise. This means the data tables are composed of continuous records for each orbit night ≈ 1800 − 2000 seconds), when the satellite is in the earth’s shadow, interspersed with gaps the length of the orbit day (≈ 3600 seconds).

2. In addition, one or several values of any monitor may be missing because of telemetry transmission problems. The problem may be a single even upset (SEU) in the instrument or satellite electronics, or corruption of the telemetry at one of several stages in transmission to the ESOC at Berkeley.

The user must tell dqselect how long a data gap can be (in units of the reporting interval) before it is considered significant, by setting the gap parameter in the parameter file or using the command gap after starting dqselect. When dqselect is constructing the good times for a monitor, it compares the interval between the time tags of each pair of monitor values to gap. When the interval between successive times is greater than (gap × report interval), a good time interval is ended, or not begun, depending on what state dqselect is in at that time. This means that a small value of gap can affect the average length of good time intervals, even though the gaps may not appear as breaks in the display, because of sampling by the graphics display device.

As an example of how the value of gap interacts with gtgen, lets assume limits of 3–60 have been placed on the monitor Det4ADct (the ADC counts for the SW spectrometer) and that gap is set to 5. This monitor is in table1.tab, for which the reporting interval is one major frame, or every 1.024 seconds. As it is creating good times, dqselect encounters a line where the monitor jumps from a value of 80 to 55. Since no good time interval has been started, dqselect considers the time of the row containing the value 55 as the start of a good time interval. The monitor now stays within limits for many rows. At some point, two consecutive rows are found to be separated by 3.072 seconds, rather than the expected 1.024. Since this is a gap of 3 reporting intervals, and gap is set to 5, this is not a major gap, so dqselect ignores the gap and continues.

At a later point, a gap of ≈ 3600 seconds is encountered (the spacecraft entered daytime). This exceeds the value of gap, so dqselect terminates the current good time interval with the time of the last reported value before the gap.

Related topics

◇ The process of creating the nominal time filter used in EGO Center processing and applying it to the QPOE is described in Chapter 4 of the EGO Center Data Products Guide, “EGO Center Data Processing.”

5/97

Example 1.1 : Creating the standard exposure filters

Each EUVE QPOE file is delivered with standard time filters, representing the basic “valid” exposure time for the observation^{3}. The filters select times when the detector was turned on, the count rate was within operational limits, and when the earth was not in the field of view. The exposure filter is slightly different for each detector. This example describes the generation of the “detector on” part of the exposure filter, based on the ADC count rates for each of the DS/S detectors. The steps of making the filters are presented here to illustrate the use of dqselect and to give the user a basic recipe that can be modified to create other filters.

The detector-on filters are based on the monitor Det#ADct in table1. The filter for earth blockage is based on another monitor, DSSZen in backmon. Example 1.2 illustrates creation of an earth blockage filter. After starting IRAF, the GO first moves to the target directory, directly above the observation directory with the telemetry tables:

          eu> chdir theobject

She then starts an interactive session with dqselect, loading table1.tab into memory, and setting the output format to print three digits after the decimal point:

          eu> dqselect observation/table1 gap=1 format=13.3f

She displays plots of each Spectrometer detector count rate monitor on STDGRAPH:

          Command> display det4adct
          Command> display det5adct
          Command> display det6adct

Figure 1-2 shows a display of the three Spectrometer count rate monitors for an average-length observation.

Limits representing valid operating conditions must be set on each count rate, and a separate set of good times generated for each detector. The value of each ADC count rate should be above 2 counts/sec, to exclude turned-off intervals with 1–2 spurious counts, and below about 5000, to include all times when the detector was turned on. The GO sets the detector count rate limits for the SW

3For a description of these filters, see the EGO Center Data Products Guide, version 1.5, section 4.3.

5/97

Figure 1-2: Spectrometer detector count rates for a 125 ksec observation, displayed in dqselect. The display can show up to four monitors from the loaded table(s) at once. The horizontal bars near the Y-axes of the bottom two plots show user-defined limits.

Spectrometer with the edit command:

          Command> edit det4adct
           Please pick the first limit value ...

           :y 3
           and again:
           :y 5000

Horizontal limit markers should appear on the vertical axis of the Det4ADct plot^{4}. Now the GO generates the set of good time intervals associated with the short

5/97

4The markers will not be visible if limits are off the plot scale.

1-10 CHAPTER 1. USING TELEMETRY IN ST TABLES

wavelength count rate:

          Command> gtgen
           Generating good times for monitor, det4adct.

She places a filter string with the good time intervals in the QPOE file, sw.qp:

          Command> filter sw sw adct gt
           Creating the filter string.

dqselect writes the filter string in the QPOE file header as the definition of the macro sw adct gt. The GO also saves the ADC count good times in a table, for possible future use:

          Command> gtsave sw adct gt
           Good times saved to file, sw adct gt.

The GO repeats this process for the medium, long, and DS detectors. Before setting the same count rate limits on the medium wavelength ADC count rate, she removes the limits from the Det4ADct monitor:

          Command> delete det4adct

and clears the SW detector good times from memory with the command:

          Command> gtnew
           Ready for generation of new good times.

Now she is ready to edit the limits on Det5ADct, generate the MW count rate filter, and install it in mw.qp. The processing messages from dqselect always tells the user which monitors are being used to create the intervals.

          Command> gtgen
           Generating good times for monitor, det5adct.

5/97

          Command> filter mw mw adct gt
           Creating the filter string.

          Command> gtsave mw adct gt
           Good times saved to file, mw adct gt.

And so forth, for the LW and DS detectors as well, saving each set of good times in an appropriately named ST table. After finishing the LW macro and quitting from dqselect, the GO uses the task euvtools.qpmedit (discussed in section 2.2.2) to list the macros in the QPOE files. An abbreviated version of the display generated from the SW QPOE file looks like this:

sw,time = d0
sw,x = s8
sw,y = s10
sw,lambda = r12
sw,theta = r16
sw,dx = s20
sw,dy = s22
sw,sw_adct_gt = 780800001.460:780802014.644,78080552.939:780807708.084,78081121
9.380:780813402.548,780816913.844:780819097.012,780822607.283:780824790.451,7808
28301.747:780829478.323,780829480.371:780830484.915...:780924686.475

with each pair of colon-separated numbers representing one orbit night. Each time in the macro sw adct gt is rounded to three decimal places. The accuracy of times in the telemetry is about 1 – 8 milliseconds, so the other digits in the table1 times are not significant. dqselect truncated the times according to the parameter format, which was set to 13.3f. Also, the typical interval between two colon-separated times is roughly 2000 seconds. This is a normal length for a nighttime pass, so the only times that are being excluded are orbit days.

This example shows only the first stage of making the valid time filters. During EGO Center batch processing, both table1.tab and the table backmon.tab are loaded into dqselect. After the ADC valid times are found, the times free from earth blockage are computed from data in backmon.tab and merged with the ADC times to create the exposure filters called <det> gt in each QPOE.

5/97

1.1.2 Count rate limits and signal-to-noise: the long table of very short time intervals

Some GO’s may wish to use a more stringent definition of valid times for their observations. For example, a GO who notices a marked increase or fluctuation in the count rate of a non-varying source might infer that the background due to airglow was changing during the observation. If one wishes to remove those periods with the highest background levels, one can reformulate the valid times with a decreased upper limit on the ADC count rate. Any new spectral image created with the “pickier” filter will contain a new corrected exposure time. This strategy can increase signal-to-noise, at the cost of losing a few source counts, and can decrease or eliminate the need for primbsching/deadtime corrections in observations that are affected by primbsching for only brief intervals. However, over-tightening the count rate limits can have undesirable effects, whose origins may not be clear in dqselect.

Figure 1-3 shows a dqselect display of an SW detector count rate for a short observation, including two orbit nights and one day. The good times produced for this observation by the standard limits have only two intervals: one for each nighttime pass. Now in the figure, limits have been set at 20 and 40, which appear to exclude periods with higher count rates at the beginning of each nighttime pass. By contrast, the list of good times generated from these limits has 316 intervals, varying in length from a few to a few tens of seconds.

Figure 1-3: Spectrometer detector count rate for a short observation, displayed in dqselect.

If this result is multiplied over the 25 – 40 orbits of an average EUVE observation, the number of intervals could be somewhere between 3000 and 7000, and the

5/97

number of characters in the macro could be from 84,000 to nearly 200,000. This is cumbersome at best, and doesn’t really improve the signal to noise. Also, there are limits both on the length of QPOE macros and on the number of intervals they can specify. With the euv package installed the limit on the length of a single QPOE macro is 65536 characters^{5}.

The GO may be puzzled at the number of good times generated with the new limits, until recalling that the display of most monitors in dqselect is heavily sampled. This prevents the user from seeing just how often the ADC count rate transits the limits they set, or indeed from seeing whether fluctuations visible in the display are really significant within the actual range of data values.

Users who wish to see every line of the table in their plot can use the igi (for

interactive graphics interface) in the package tables.tbplot. Its use is described briefly

in Appendix B; please note the instructions on dealing with the double precision time stamps in EUVE tables igi can be very helpful for setting count rate limits and finding the exact times of outbursts and other transient effects.

Figure 1-4a) shows the 2-orbit dqselect display of the SW ADC count rate monitor, zoomed in X to show the spacing of the sampled monitor readings for about 200 seconds. The vertical scale is from 16.5 to 55.3 Figure 1-4b) shows the same SW ADC count monitor displayed in igi with about the same vertical scale. One can see from the igi display that transits of the value 40 cps are quite frequent, and that

Figure 1-4: a) Detector count rate enlarged in dqselect. Plot spans approximately 200 seconds with a vertical range of 16.5 – 55.3.

5/97

5Otherwise, the limit is usually 8192 characters.

the range of values present is larger than in the sampled dqselect display. The moral of the story is that placing very tight limits on detector count rates in dqselect may NOT significantly increase signal-to-noise, may make for unwieldy exposure filters, and may exclude a large portion of the valid exposure time.

Figure 1-4: b) Display of detector count rate in igi. igi reads every line of the selected table column, without sampling. The window has been “Zoomed” to show the fluctuations due to Poisson noise.

Related topics

◇ If for some other reason you want to create a longer QPOE macro than is allowed by your system, there are ways to work around the size limitations. See Appendix A, section A.4.1 for more information on setting up your IRAF environment to deal with long QPOE macros.

1.2 Specialized Data Quality Selection Tasks

This section is about several specialized tasks that address particular problems that may occur in some data sets. All these tasks create new ST tables from existing ones, usually with the purpose of rendering the telemetry data into a more usable form. The tables are used for data selection, and correction of exposure times for effects called “primbsching” and TIF (Telescope InterFace) deadtime. GO’s can refer to Chapters 4 and 5 of the EGO Center Data Products Guide for more background

5/97

Specialized Data Quality Selection Tasks 1-15

on the problems these tasks address. Here, we’ve tried to include material to help users determine whether and when they need to use these tasks.

1.2.1 MKPRIMBSCH makes deadtime correction tables

The task euvtools.mkprimbsch computes corrections for primbsching and TIF deadtime which can be applied to the data in order to get the most accurate count rates. The origins of data loss due to primbsching and deadtime are discussed in more detail in section 5.1 of the EGO Center Data Products Guide. mkprimbsch supplies the capability for investigating the severity of the problem, and applying corrections in the process of rebinning a QPOE file into a two-dimensional spectral image.

mkprimbsch reads information from the raw telemetry tables and from a reference table of TIF deadtime calibrations, and computes the amount of deadtime and primbsching which occurred during each pair of telemetry frames (2.048 seconds). The computations are based on count rates reported by the detectors for each filter quadrant before TIF processing, and the numbers of events actually recorded in table0.tab. From these, it produces a table containing columns of reference data, and multiplicative correction factors for each of the detectors during each two-frame interval. Once the primbsching/deadtime correction table is produced, one can use the results to determine the magnitude of the correction required, and to selectively exclude frames with corrections above some limit, or create spectral images with corrected exposure times, using the task qpmkim (section 2.3).

Do I need to run mkprimbsch?

mkprimbsch is run for each observation as part of the routine EGO Center data reduction, starting with version 1.4 of the IRAF/EUV software package. The output table primbsch.tab is supplied on the standard data tape, and it should not be necessary for most GO’s to run mkprimbsch. Beginning with data processed under version 1.5 of the IRAF/EUV package, the standard two-dimensional spectral images supplied on each tape have recorded exposure times that have been corrected using the table primbsch.tab. If new images with specialized filters are desired, they can be produced with primbsch/deadtime corrected exposures from any QPOE file for which there is a primbsch table, by using qpmkim. Even if the QPOE files are remade by rerunning the pipeline, the event times, and thus primbsch.tab, are not changed. Primbsching/deadtime correction can be applied to the new QPOE’s in the process of creating new spectral images with qpmkim. See section 2.3 for more detail on making corrected spectral images.

The parameters for mkprimbsch are shown in figure 1-5. The task requires information from several sources; the parameters given here will produce a correction table for all three Spectrometer detectors and the Deep Survey detector, assuming the nominal arrangement of the data on disk (shown in figure 0-3 of the Data Products Guide). See the help page for a detailed description of columns in the output table.

5/97

      eu> lpar mkprimbsch

  output = "primbsch"      output table name
(detectors = "sw,mw,lw,ds")  list of detectors to process
(refdata = "reference/detector.tab") detector reference table
(infomsgs = yes)            give informational messages
(warnmsgs = yes)            give warning messages
(diagmsgs = no)             give diagnostic messages
(negprimb = no)             report negative primbsch values
(readper = 100)            percent of input file to read
(obsdata = "observation")  path to observation directory
 (events = "table0")       event table name
(gaptimes = "table1")       gap times table
(quadcts = "table2")       quadrant counts table
(ctsoffset = 1.024)          quadrant counts reporting offset in seconds
   (mode = "ql")

Figure 1-5: Sample parameter file for the task mkprimbsch.

Notes on running mkprimbsch

□ The parameters diagmsgs and readper are mostly for diagnostic purposes, and should not normally be changed from the defaults.

□ When the parameter negprimb is set, the program reports the number of “negative primbsching” events which occur in the data. These are intervals for which the telemetered counts exceed the quadrant counts. The detector quadrant counts are not changed.

□ The parameters events and quadcts point to the tables used for the calculations. table0 is scanned to find the number of telemetered events in a detector quadrant during each interval, and table2 is scanned for the detector quadrant counts

□ table1 should be used for gaptimes, to find gaps where a major frame is missing from the telemetry.

□ Users should not change the value of ctsoffset.

To run mkprimbsch from the command line, producing a table of correction values for all of the DS/S detectors, simply type:

      eu> mkprim primbsch sw,mw,lw,ds

mkprimbsch can print various messages on STDOUT to inform the user of its progress. The flags infomsgs, warnmsgs, diagmsgs and negprim to report various conditions. In particular, the warning: “values not found in deadtime table”, indicates that some values of the detector count rate were not in the calibrated range of the TIF deadtime table. This happens only for extremely bright sources, such as the moon.

5/97

Out-of-range detector count rates cause mkprimbsch to use the largest fractional correction in the calibration table tifdeadtime.tab.

Each row of the output table includes the time stamp for an interval of two major frames (the 2.048 seconds which end at the time stamp), the telemetered and quadrant counts for each Spectrometer detector and the Deep Survey detector, correction factors, and errors. The tables task ttools.tlcol can be used to list the columns^{6}.

The most interesting columns are the ones named Det4Q0dpc and Det[5-7]Q1dpc. These are the combined deadtime and primbsching corrections for the spectrum quadrants and the central Lexan filter of the Deep Survey. For example, if some row contains the value 1.05 for Det4Q0dpc, then the number of counts detected in the SW in that interval should be multiplied by 1.05 to get the true value. Figure 1-6 shows a typical deadtime/primbsch correction column for a bright source in the LW detector, as plotted by dqselect. Remember that because of the table’s length and the limitations of the graphic display, the plot shows a sampling of the data.

Figure 1-6: Deadtime/primbsch column displayed in dqselect with limits.

Gaps in the data affect the results in the primbsch table, and their occurrence is identified at run time if the task is run with diagmsgs on. The presence of gaps should be noted and understood by the user, but the messages are not signals of catastrophic failure. The resulting output table will still be useful; the image creation task qpmkim will apply a unit correction factor (no correction) for intervals

5/97

6Some of the columns are trivial or uninteresting. Quadrants 2,3 for detectors 5 and 6; 1,2,3 for detector 4, and quadrant 3 for detector 7 do not exist, and quadrant 0 in detectors 4-6 does not have the spectrum.

where the correction factors are missing or INDEF.

While the ADC counts are recorded every major frame, the filter quadrant counts are only recorded every other frame. If, for any pair of major frames, the major frame with the quadrant counts is missing, then the correction table will have a 2-frame gap. If a major frame is missing between the quadrant count reports, only the primbsch correction will be affected. This correction will be INDEF, and this will make the combined deadtime+primbsch correction column INDEF for that detector in both those frames. These two types of data loss normally occur in about one two-frame interval in every 5000; if they occur more frequently, something may be wrong with the input tables or data paths. Also, if for any interval the telemetered detector counts actually exceed the quadrant counts (negative primbsching), a primbsch correction factor of 1.0 is entered.

Related topics

◇ Correction for deadtime and primbsching is applied when a spectral image is created from the QPOE file. When you are ready to make a new corrected spectral image, go on to section 2.3.

◇ The task qpmkim has two different correction modes. Correction can be applied by adjusting the exposure time (the default), or by adjusting the number of counts in each pixel. See section the items in “Notes on running qpmkim” in section 2.3.1.

1.2.2 BACKMON makes tables of geographic orbit parameters

The EUVE telemetry contains quite a bit of geophysical data, but the format of the data may be unfamiliar, or difficult to integrate with the GO’s analysis plan. For example, the aspect and ephemeris of the Spectrometer and satellite are recorded as quaternion components referenced to a celestial coordinate system, while investigators who wish to use data on the satellite’s physical orientation may find this format inconvenient. The task euvtools.backmon (so named because its output can be used to monitor sky background) was written to help GO’s create data quality filters based on geophysical parameters. GO’s who wish to convert only the aspect quaternions into celestial coordinates can skip to section 1.3.1.

Given the location of the ST table containing the aspect and ephemeris reports (e.g. table1.tab) and a set of reference data, backmon calculates a list of selected orbit parameters for each major frame and puts them out in a new ST table. It calculates instrument look direction, satellite position and velocity vectors, and sun position in geocentric coordinates. A selection of reference angles can also be produced, as well as the satellite magnetic latitude, which often correlates with the local density of ions. The data can also be sampled at some user-specified interval. The time of each sampled line is recorded, and all latitudes and longitudes can be converted to RA and Dec using the Greenwich Meridian location. See the IRAF help page for a complete listing of the output columns available and their units.

5/97 Do I need to run backmon?

The task backmon has already been run on table1.tab for all standard EUVE data sets, starting with IRAF/EUV version 1.4. The output is in the table backmon.tab in the tarfile labeled Processing on the data tape listing. Most GO’s will never need to rerun backmon. However, it is advisable to have some understanding of the quantities backmon produces and their role in the EGO Center nominal data reduction, so we suggest that all GO’s inspect backmon.tab and read example 1.2, on creating the earth blockage filter.

A sample parameter file is shown in figure 1-7.

      eu> lpar backmon

  output = "backmon"       Name of output ST table
   (skip = 10)             Input skip factor
(maxtimegap = 600)            Max gap in time (secs) for Sat. Vel. Vectors
(dodsspos = "yes")          Make cols for DSS Look Direction Long & Lat
(dosatpos = "yes")          Make cols for Satellite Long, Lat & Height
(dosunpos = "yes")          Make cols for Sun Long & Lat
(doangles = "yes")          Make cols for ASunZen, DSSZen, DSSASun angles
(dogmpos = "yes")          Make col for Greenwich meridian position
(domaglat = "yes")          Make col for Magnetic Latitude
 (dommjd = "yes")          Make col for Modified Modified Julian Date
(dosatvel = "yes")          Make cols for Satellite Velocity Vectors
(obsdata = "observation")  directory of ST observation tables
 (aspect = "table1")       Name of input aspect/ephemeris ST table
(refdata = "reference/detector.tab") Name of ST table for reference data
(detectors = "sw")           detector (for boresight quaternions)
   (mode = "al")

Figure 1-7: Sample parameter file for backmon.

Notes on running backmon

□ backmon uses the parameter skip to determine how often it should sample the aspect table. For example, if run with the parameters above it will calculate all the selected quantities for the first line of the aspect/ephemeris table, and every 10 lines thereafter. This can save space, at the expense of resolution.

□ The parameter maxtimegap is the longest time in seconds that is allowed between successive sampled lines when computing the satellite velocity vector and the velocity view angle (angle in degrees from the DS/S look direction to the spacecraft ram direction). If successive sampled lines are separated by an interval larger than maxtimegap, these columns have the value INDEF. It is normal for the daytime gaps in telemetry to produce INDEFs.

5/97

□ Specify only one detector as a boresight reference; all DS/S detectors are equivalent, as they share the same telescope.

Like mkprimbsch, backmon is usually run with standard parameters, and can easily be run from the command line:

          eu> backmon backmon skip=10

Related topics

◇ One of the uses of backmon.tab is to test for earth blockage during a pointed observation. See section 4.3.2 in the EGO Center Data Products Guide and the example below.

Example 1.2 : Creating a new earth-blockage filter

The conditions used to make the earth blockage filter used in EGO Center processing are described in the Data Products Guide, section 4.4.2. It is installed by a process like the one described in example 1.1 above. The standard filter is conservative, in that it filters out events from paths up to a relatively high altitude above the earth’s surface: 450 km. This was done to be sure that none of the data passed are from lines of sight with significant atmospheric absorption, which would skew any flux values that were derived. GO’s whose images show that a significant number of events have been removed by the earth blockage filter may wish to use a lower altitude to define earth blockage, and to make a new filter that lets more events through.

The condition for earth blockage can be stated in terms of one of the angles calculated by backmon: DSSZen, which is the angle between the DS/S look direction and the spacecraft zenith vector.

DSSZen > 180 − arcsin \frac{(R_{e} + A)}{(R_{e} + H)}

where R_{e} is the earth’s radius, H (km) is the average height of EUVE’s orbit above the earth’s surface, and the signal is considered blocked for purposes of EUVE science when it must penetrate the atmosphere at an altitude ≤ A (km).

The GO must first determine, by some physical argument (and possibly considering in what part of the bandpass he expects the source flux), the altitude in the earth’s atmosphere at which the source EUV photons are subject to enough absorption to decrease the detected count rate. Suppose he arrives at an altitude of 200 km above the earth’s surface.

The GO next looks at the column SatHt of table backmon.tab, which gives the height of the satellite. The GO gets the min, max, and mean of this column

5/97 using ttools.tstat ^{7}.

          tt> tstat backmon SatHt
           # backmon SatHt

 # nrows  mean         stddev   median   min      max
    4099  6883.719085  2.49885  6883.01  6880.83  6890.19

This shows a negligible variation in height over the observation, and supplies a value for the height H. Since SatHt gives the height from the center of the earth, the relation to the height in the earth blockage condition is H = \overline{SatHt} − R_{e} = 6883.01 − 6378 ≈ 505 km. Using this value and 200 km for A, the GO calculates the limiting DS/S-Zenith angle above which the line of sight is blocked:

Blockage ≡ DSSZen > 180 − arcsin \frac{(R_{e} + 200km)}{(6883km)}
DSSZen ≤ 107.12

Notice the sense of the condition is changed– the GO is getting ready to make a QPOE filter, which requires a range of acceptable values for the attribute in question (time), so the limiting values of DSSZen should the times without earth blockage. To skip over earth-blocked data, he must select data with values of DSSZen less than 107.12 degrees.

Having determined this limit, the GO can easily make the appropriate filter string with dqselect. With the table backmon.tab loaded, dqselect displays a time

plot of DSSZen on STDGRAPH:
          Command> display dsszen

He sets limits on DSSZen at 0 and 107.12:

          Command> edit dsszen
           Please pick the first limit value ...

           :y 0
           and again:
           :y 107.12

5/97

7The values shown are examples, and do not correspond to any particular GO data set.

The GO then generates the earth blockage good times from the TableTime column with the gtgen command, and saves them in a table:

          Command> gtgen
           Generating good times for monitor, dsszen.

          Command> gtsave myear gt
           Good times saved to file, myear gt.

The new earth blockage filter should be merged with the existing ADC count rate filter in each QPOE file to make a new exposure filter, so the GO quits dqselectand uses euvtools.tfilter to create and install the new filter macros:

          eu> tfilter sw adct gt,myear gt output=sw.qp name=losw gt cmdline-
          format=13.3f
          eu> tfilter mw adct gt,myear gt output=mw.qp name=lomw gt
          eu> ^{∧}mw^{∧}lw^{∧}g
           tfilter lw adct gt,myear gt output=lw.qp name=lolw gt

Once the tfilter parameters cmdline and format are set, they need not be reset each time the task is used.

1.3 Aspect and Pointing Problems

The pointing of the DS/S telescope is constantly subject to small drifts. While the pointing of the satellite is updated periodically, measurements of the satellite orientation, or aspect, are recorded in the telemetry more often: every 1.024 seconds. The readings are based on readings of the on-board gyros, star trackers, and the digital fine sun sensor.

The aspect correction algorithm in the EGO Center processing pipeline cep (Chapter 5) uses these satellite aspect reports to determine the true instrument pointing direction at the time of each detector event. The aspect is compared to the source position and correction factors are calculated that should restore each event to the position on the detector associated with a perfect, on-boresight pointing. If a stationary target’s position is known with enough accuracy, the main limit on the

5/97

Aspect and Pointing Problems 1-23

spectrum focus is the quality of the aspect reports in telemetry.

However, targets for EUV spectroscopy include some objects that move too rapidly for EUVE’s multiple attitude control system (MACS) to track continuously. These include planets, such as Mars and Jupiter, comets, and the moon. Observations of such objects have the source moving through the field of view, so that the relationship of the satellite pointing to the target position changes with time. The resulting spectra are smeared out in both the imaging and spectral directions on each detector, even after cep has performed its aspect correction, because cep assumes a constant source position on the sky. Occasionally, stationary targets are also subject to problems which cause inaccurate aspect reporting, with similar results.

The next two sections consider how to determine whether an observation is affected by any of these problems. We introduce tasks which GO’s can use to evaluate the quality of pointing and aspect reports for their observation, and to reprocess the data from observations of sources with changing parallax.

1.3.1 MKASPECT creates coordinate tables from aspect records

The task euvtools.mkaspect creates a table of time-tagged celestial coordinates from time-tagged aspect quaternions in an ST table (e.g. table1.tab). This can be useful for checking the accuracy and stability of the spacecraft pointing during an observation, and for determining the positions of objects in the field of view. The output table has columns of times, Right Ascension, Declination, and roll angles. Because roll angles for the various detectors differ by multiples of 60 and 120 degrees, a different table is generated for each DS/S detector specified in the input. In addition, a column with the angular distance between the reported aspect and a user-supplied position on the sky (usually the target coordinates), can also be calculated.

Do I need to run mkaspect?

The task mkaspect is run for the Deep Survey detector as part of standard EGO Center processing. The resulting aspect coordinate table is called mkaspectds.tab,and is in the fourth tarfile on the data tape. This table contains the reported DS/S pointing direction in RA and Dec for all the DS/S detectors, with roll angles for the DS detector. All GO’s should inspect the mkaspect table, but most will not need to rerun mkaspect unless they have some specific need, such as identifying the sources of multiple spectra.

See the subsection titled Source orientation in Spectrometer images in section

1.3.1. If the DS detector was turned off because of high count rates, the task is run for the LW spectrometer, producing mkaspectlw.tab.

An example parameter file for running mkaspect is shown in figure 1-8. The target is Beta CMa, and the task is to be run from the source directory in a standard EUVE reduction setup. Separate output tables will be produced for the SW and MW Spectrometers. Angles from the DS/S optical axis to the source coordinates will also be computed for every line of the aspect table table1.tab.

5/97

      eu> lpar mkaspect

   input = "observation/table1"Name of input aspect ST table
  output = "aspcoords"     Prefix for output aspect ST tables
   (skip = 10)             Number of rows to skip over
(refdata = "reference/detector.tab") Name of ST table for reference data
(detectors = "sw,mw")        Detector list
(distance = yes)            Compute distance to boresight?
     (ra = 6.3783)         Central RA in hours
    (dec = -17.956)        Central DEC in degrees
   (mode = "al")

Figure 1-8: Parameter file setup for the task mkaspect. The parameters ra and dec are set to the target coordinates to produce the off-axis angles in the DISTANCE column.

Notes on running mkaspect

□ The values of the parameters ra and dec should be set to the target coordinates desired if distance angles (see below) are to be computed.

□ Although the aspect quaternion is reported every major frame, the calculations can be done less frequently by setting the parameter skip, which causes the task to pass over the given number of records after each calculation. This can shorten the resulting table if the aspect is known to be sufficiently stable over intervals of skip ×1.024 seconds.

□ If distance is “yes”, the angle from the target coordinates ra and dec to the telescope boresight is calculated as a positive angle. Azimuthal information is not provided directly, but the distance angle can be combined with the roll information to determine the exact orientation of the telescope boresight to the source position given by ra and dec.

To run mkaspect in IRAf command line mode for the same target, and with distance on, one would type:

      eu> mkasp observation/table1 aspcoord detectors=sw,ds distance+ ra=6.3783
      dec=-17.956

The output tables will be called aspcoordsw.tab and aspcoordmw.tab, and each will have five columns: TableTime, RA and DEC of the detector boresight, detector ROLL angle, and the DISTANCE in degrees between boresight and the specified RA and Dec. Roll angles are given in degrees from the direction south to the positive X-axis of the unremapped detector image (dx). Positive angles are counter-clockwise (ccw) from south to dx, while east is the direction 90 degrees clockwise from north^{8}.

8Roll_{sw} ≈ Roll_{ds} − 180, Roll_{mw} ≈ Roll_{ds} − 60, Roll_{lw} ≈ Roll_{ds} + 60 5/97

Using mkaspect’s output to verify pointing

All GO’s should inspect the contents of mkaspectds.tab in dqselect or tbplot.igi, along with the DS detector image, to familiarize themselves with the behavior of the satellite pointing during their observation. Since mkaspect produces the reported pointing directions in celestial coordinates, this table may be more helpful than the raw quaternions in table1.tab. Plotting columns of mkaspectds.tab in igi is most effective, as the full range of values can be inspected at any resolution. See Appendix B for a tutorial. A plot of RA vs. DEC, such as that in figure 1-10, shows literally where the telescope was pointed on the sky.

Some of the irregularities that may affect pointing and aspect reporting are listed below; most pointing errors are accurately reported in the aspect and corrected for in the standard data reduction, but GO’s should be aware of them. If the aspect report is in error, this must be evaluated and corrected by the GO.

• Looking at any coordinate column plotted against time will show any unexpected data gaps, such as a slew to another target. Figure 1-9 shows an igi plot of RA vs. TableTime for a 300 ksec observation that was interrupted for about two orbits to perform a pointing test^{9}. The GO may want to check the gap times and exposure for the target.

Figure 1-9: RA vs. TableTime from mkaspectds.tab, showing a data gap of 10,000 seconds. The observation was interrupted by a test, which lasted for 2 orbits.

5/97

9The telemetry for slews and time observing other targets is not included in the GO data set. This time is not counted against the integration time in scheduling or processing.

• The boresight RA and Dec in mkaspectds.tab correspond to the un-remapped coordinates of the telescope boresight, which may not coincide to the target coordinates. Most observations are pointed off-axis by 1–2 arc minutes to avoid the DS detector dead spot. The event positions are corrected using the aspect report during processing, and do not affect spectrum focus.

• Observations that have been dithered will show a series of systematic variations in RA and Dec, (and distance angle, if calculated), as shown in both figures 1-10 and 1-11. These deviations are also removed during processing by using the aspect report to remap the event positions.

Figure 1-10: RA vs. DEC from mkaspecds.tab, plotted with tbplot.igi. The pointings of a dithered observation are small dots within 2 arc minutes of the of the target. The observation was interrupted twice.

• If the source image in the remapped DS detector image ds.imh is not at the detector center, then an unreported pointing error may have occurred. The aspect reports may be inaccurate, and the GO should take steps to find and correct the problem. Unreported pointing errors may be caused by things that “fool” the MACS, like telescope flexure or guide star confusion in the star trackers. Aspect errors will not be confirmed by mkaspect’s distance output. See section 1.3.2, on correcting for aspect errors.

• Inspecting a plot of roll vs. time can help to identify occasions when the roll angle was changed in mid-observation. This usually occurs when the available guide stars change, and is part of normal operations. However, there is always a possibility that one of the two positions in such an observation is subject

5/97

to telescope support flexure, which alters the pointing although the change is not reported in the aspect. The DS image and spectra may be blurred in such cases. Constant pointing offsets are addressed in section 1.3.2.

Figure 1-11: Coordinates from mkaspect for a dithered observation, plotted with dqselect. This plot shows RA, Dec, and roll for the DS detector.

Source orientation in Spectrometer images

Another use for the data from mkaspect is to determine the locations and relative orientations of multiple sources, or of different parts of an extended source like a supernova remnant. The DS QPOE file and the remapped DS image are produced so that north is always along the positive Y axis, with east increasing to the right. This is not practical for the Spectrometer QPOE’s, as half of the position information for each event is embedded in the wavelength solution. In Spectrometer QPOE’s, only the Y axis has position information, and the imaging angle has a different orientation on the sky for each channel.

The imaging angle of each detector event corresponds to the angle from boresight in a plane tangential to the circumference of the telescope aperture plane. This can be seen from figure 1-12, which shows that each detector Y axis is oriented tangential to its respective aperture section, while the X axes are arranged radially. This picture can be compared directly to the raw detector images, with appropriate rotations. The Z-axis points out of the page.

5/97

Figure 1-12: Orientation of detector coordinates in the DS/S. In the larger diagram, the observer is facing the detectors, with the optical axis (Z) pointing out of the page. The smaller diagram illustrates the definition of the imaging angle, Θ_{i}.

The radial geometry of the instrument means the vector displacement of a point off the boresight is resolved with a different fractional component along the on the imaging axis in each detector. The orientation of north to the Y axis depends on to the telescope roll angle and the angular separations of the Spectrometer detectors’ Y axes (120 deg). To find the orientation of multiple sources in one of the Spectrometer images, it is recommended that the GO run mkaspect for that particular Spectrometer channel. Although the relation between the roll angles for the different DS/S detectors is simple, this saves the trouble of worrying about sign

5/97

changes with an odd number of reflections, and whether offsets are to be added or subtracted from the values in mkaspectds.tab.

The roll angle in mkaspect*w.tab is unique to the observation. When applied as stated above, the roll angle tells how the Spectrometer Y axis is related to the directions north, south, east and west for that particular detector during that observation. The difference in orientation for detector images (* det.imh) and remapped spectral images (*.imh), is small enough to be ignored in a first approximation. Once the relation of the Y axis to north is established, this defines the direction on the sky associated with the imaging angle. But one must bear in mind that the imaging angle is only one component of the vector displacement between the sources.

For example, suppose two bright spots in a supernova remnant produce spectra separated by an angle of .05 degrees in the SW image, and the roll angle produced by mkaspect for the DS/S is 57.5 degrees. Since roll is given in degrees ccw from south to dx, this means that dy points 23.5 degrees ccw (west) of due north. The two sources must be separated by .05 degrees on a line 23.5 degrees ccw from north, with the target nearest boresight to the south and east (see figure 1-13). A check on the relative shift in X between the two spectra might supply 0th order information about their separation along the perpendicular direction.

Related topics

◇ Sections 3.3 and 4.3.4 of the EGO Center Data Products Guide describe the table

mkaspectds.tab that is delivered on the GO data tape. ◇ When two spectra are present in an image, the EGO Center reduces the data

for only the requested pointing. If the second source position can be found, e.g. from a DS image, the GO may wish to rerun the event pipeline on the data to create QPOE files with the second spectrum, as the target. See section 5.2.1 on correcting constant pointing offsets and aspect errors.

1.3.2 ASPCORR creates new aspect tables from source positions

euvtools.aspcorr is a task designed to help GO’s process observations with moving targets or aspect reporting errors which change with time. While telescope pointing cannot be corrected after the fact, unreported pointing errors can be addressed in the EGO Center reduction pipeline using the task aspcorr if some reliable source of information on the target’s apparent position in the field of view (based on the aspect report) is available.

aspcorr requires the satellite aspect reports in table1.tab, a fixed target position, and an ST table with the apparent celestial coordinates of the target at regular intervals as input. aspcorr produces a new table of aspect readings which can be used to rerun the event pipeline, cep. The new table will have one line for each line in table1.tab; if necessary, the apparent positions are interpolated between the times given in the input to produce aspect reports with the same times as the original aspect table.

5/97 Figure 1-13: Using mkaspect output to locate off-boresight sources in the Spectrometers. Cartoon compares the roll angle data from mkaspect with the orientation of the detector imaging axis on the sky. The position of the source along the line perpendicular to dy, labeled ΔΘ(λ), is related to the shift in X position of a spectral feature which appears in both spectra, as indicated by the ∧, ∨’s on the spectral image.

aspcorr uses the apparent positions to calculate a compensating factor for each of the original aspect reports. Figure 1-14 illustrates the calculation for a source moving through the DS/S field of view. For the location t⃗ of the target at some moment, t⃗′ is the direction of the actual pointing, which is offset by some factor Δt⃗. The aspect is reported as a⃗; the true aspect is a⃗′. The dashed circle shows the radius of normal fluctuations in the aspect between pointing updates. It may or may not include the target location, depending on Δt⃗.

The difference Δa⃗ is the correction which cep calculates from the aspect report and catalog.tab, and does not include the offset Δt⃗. cep assumes it to have the relation a⃗ = t⃗ + Δa⃗ to the target. When aspcorr is run, the apparent locations in the input table correspond to the target position t⃗ at different times. For each aspect report, aspcorr calculates the factor δ⃗ = Δa⃗ − Δt⃗, which includes the offset between the pointing and the target direction for that time. If t⃗ and Δt⃗ are rapidly changing functions of time, e.g. for the moon, the compensation must be calculated for every aspect report (≈1/second).

In other words, each line in the aspect table created by aspcorr has a compensating factor added to the original satellite aspect quaternion, which is equal and opposite

5/97 Figure 1-14: Vector geometry for correcting changing aspect error. The dashed circle represents the range of normal aspect variation – about 30′′ ⌢. If cep uses the apparent location t⃗, it will calculate the compensating factor δ⃗.

to the difference between the fixed target position and the apparent target coordinates. When the new aspect table and the same fixed target position are input to cep the pipeline produces a corrected QPOE file. The new aspect table “fools” cep into compensating for the aspect errors, with the result that each event position is corrected to the current target position for that time, and the resolution of the spectrum is improved. However, this process depends on the precision with which the user knows the apparent coordinates of the source in the DS/S field of view.

Calculating the apparent coordinates of the source is done in one of two ways, depending on the type of error to be corrected:

1. For moving objects where the aspect reports are accurate, but the satellite could not track the target accurately, the target’s apparent RA and Dec can be calculated from known target positions and the satellite aspect and ephemeris in table1. The output table from backmon produces several vectors, referenced to common geocentric coordinate systems, which may be useful.

2. In cases where the aspect reports for a distant target are not accurate, centroids can be obtained from the Deep Survey data if the source has sufficient flux in the Lexan bandpass. EUVE GO data tapes processed under version 1.5 include all events from the DS Lexan filter in table0.tab and in a DS QPOE file. The frequency with which centroids can be produced depends on the count rate.

5/97 Do I need to run aspcorr?

This task is for solving a particular kind of problem: when the actual pointing offset is not equal to the difference between the target coordinates and the reported aspect, AND the difference is changing in time. As indicated above, the two main reasons this occurs are that the target was actually moving in the instrument field of view, or that the star trackers were not properly locked onto guide stars, causing unreported aspect drift. Unless you know that your observation is subject to one of these two conditions, you probably don’t need to run aspcorr. Other aspect problems, like flexing of the telescope support structure or use of inaccurate target coordinates in the reduction pipeline, produce constant aspect offsets which are simpler to correct. These are discussed in section 5.2.1, on rerunning cep for constant offsets.

In situations where the apparent position or aspect error are changing rapidly, it is best to supply positions at a frequency of about 1 row per second. Once the apparent coordinates are obtained, the procedure is the same: pack the coordinates into a standard format ST table using a task like ttools.tcreate, run aspcorr to create the new aspect table, substitute it for table1, and rerun cep with the corrected aspect table to produce the corrected QPOE file(s).

A typical parameter file for running aspcorr with a table of positions for a moon observation is shown in figure 1-15.

        eu> lpar aspcorr

   position = "moon.tab"      apparent positions table
   asptable = "table1"        satellite aspect table
     output = "new_table1"    output table
      alpha = 0.              fixed position ra(hours)
      delta = 0.              fixed position dec(degrees)
        (ra = "RA")           ra column name
       (dec = "DEC")          dec column name
      (time = "Time")         time column name
    (aspect = "AspectT,AspectW,AspectX,AspectY,AspectZ") Time,W,X,Y,Z
      (mode = "al")

Figure 1-15: Sample parameter file for running aspcorr on a lunar observation. The fixed source coordinates, alpha and delta, are arbitrary when dealing with a moving source like the moon.

Notes on running aspcorr

□ The parameter position should point to a three-column ST table with the apparent celestial coordinates of the object at intervals of 1 second or more. The table should have the format shown in figure 1-17. The frequency of the positions may determine the accuracy of the correction for fast-moving sources, such as the moon.

5/97

□ If target coordinates were obtained from an ephemeris, they may need to corrected for parallax. The position must reflect the RA and Dec of the target as seen from the position of the EUVE satellite. The satellite ephemeris can be obtained directly from the columns PositionX/Y/Z in table1.tab (in meters), or from the output of backmon(in Lat/Long/Ht).

□ The times in the position table must span (i.e. include times both before and after) the reported times Table1Time in table1.

□ The name of the original satellite aspect table is supplied in asptable, and will generally be table1 for EUVE data.

□ Fixed source coordinates in the field of view are supplied in alpha (RA, Hrs:min:sec) and delta (Dec, degrees.dec). The same position will be used to run cep with the corrected aspect table. For moving targets, the choice of coordinates is

arbitrary, as long as it is correct for the coordinates in the position table.

□ The three parameters ra, dec and time are used to specify the column names used in the position table, which in this example, happen to be called RA, DEC, and Time. Column names are case sensitive.

□ The parameter aspect supplies the column names of the aspect quaternion in asptable. The default values are those for EUVE telemetry tables, and need not be changed.

Note that the command line parameters are position, asptable, output, alpha, and delta. Thus, the task could be run using the position table moon.tab in the observation directory using the default hidden parameter values as follows:

      eu> aspcorr moon.tab table1 new table1 0.0 0.0

This produces a new aspect table, with the same column names for time and the four components of the aspect quaternion as those in table1.tab.

Since aspect is the only information from table1 that is used by cep, the new aspect table can be substituted directly for table1.tab. The next step is to run cep with the new aspect table to produce QPOE files with more accurate event locations.

Related topics

◇ See Chapter 5 and especially section 5.2 to correct aspect problems by rerun-

ning the event pipeline. ◇ Events in an area around the boresight can be selected for various intervals to

create a record of the pointing offset in time, as described in section 5.2.2. ◇ Section 5.3 of the Data Products Guide also describes the effects of some aspect

reporting errors, and the procedure called “dithering.”

5/97

Example 1.3 : Making a new aspect table for an observation of the moon

A 1600-second test observation in which the full moon scanned through the DS/S field of view has been processed using aspcorr, using apparent coordinates for the moon, as calculated by the GO. The detector image in the LW spectrometer, after uncorrected processing with cep, is shown in figure 1-16. The accumulated image of the uncorrected data looks like a number of stripes, nearly perpendicular to the spectral direction.

The spectrum is actually composed of overlapping images of the full lunar disk at a number of wavelengths. Each image is a single spectral feature, its width determined by the 20′ ⌢ FWHM of the collimator transmission. The image shapes are elongated in Y and shortened in X because the optical speeds along the dispersion and imaging axes are different. The moon is obviously very bright (remember this is only 1600 seconds of data), but the spectrum is badly spread out in both imaging and spectral directions. The GO wrote a program which

Figure 1-16: The moon in the LW Spectrometer, after standard processing with cep and the satellite aspect. Streaks show the paths of spectral features across the detector during the observation.

uses the satellite ephemeris from table1.tab and the satellite look direction in backmon.tab to calculate the apparent RA and Dec of the lunar disk every 2.048 seconds, and created a table with columns of times, RA, and Dec. The first few lines of moon.tab, containing the apparent celestial coordinates of the moon, are shown in figure 1-17.

The GO then ran aspcorr using the parameter settings shown in figure 1-15 to produce the ST table new table1.tab. The new table contains the “corrected”

5/97

Column   1               2                    3               4
Label  Index __________Time_________________RA_____________DEC______
1 0     774678132.7360001         81.0991821      23.7006283
2 1     774678134.7840004         81.0980301      23.7021084
3 2     774678136.8320002         81.0968246      23.7035847
4 3     774678138.8800002         81.0956726      23.7050686
5 4     774678140.9280001         81.0944824      23.7065468
6 5     774678142.9760003         81.0933304      23.7079926
7 6     774678145.0240002         81.0921249      23.7094669
8 7     774678147.072             81.0909271      23.7109451
9 8     774678149.1200001         81.0897369      23.712431
10 9     774678151.1680003         81.0885468      23.7138958
11 10    774678153.2160001         81.0873489      23.7153378

Figure 1-17: Part of an ST table input to aspcorr, with apparent coordinates of the moon for alternate major frames.

aspect quaternions, each with an offset equal and opposite to the difference between the apparent coordinates and the reported aspect. A few lines of new table1.tab are shown in figure 1-18.

Column             1                         2                         3
Label  _________AspectT__________________AspectW__________________AspectX_________
1 774678133.1390079         0.6644102135026706        -0.3740989780195041
2 774678134.163008          0.6644057080561128        -0.3741017126098966
3 774678134.7840004         0.6644029725503054        -0.3741033700186317
4 774678135.187008          0.6644011547295496        -0.3741044181743463
5 774678136.2110082         0.6643962913513004        -0.3741070834119812
6 774678136.8320002         0.6643933395028272        -0.3741088446477844
7 774678137.2350078         0.6643914734961072        -0.3741100191197195
8 774678138.2590079         0.664386962646536         -0.3741125250967082
              4                         5
Label  _________AspectY__________________AspectZ_________
1 -0.5232151820760572       0.3805978845892577
2 -0.5232145074152104       0.3806041354529378
3 -0.5232139558862321       0.3806080750180024
4 -0.5232136479903312       0.3806106641431434
5 -0.5232131065275788       0.3806170105491646
6 -0.5232129243487749       0.3806210076622806
7 -0.5232127482948769       0.3806235635017167
8 -0.523212066339116        0.380629823501779

Figure 1-18: Columns of an output table from aspcorr.

5/97

Note that the first time in figure 1-17 is slightly before the first time AspectT in the aspect table; aspcorr requires that times in the input position table span those in the aspect table. The new aspect table was then substituted for table1 in re-running cep (see directions and example 5.2 in section 5.1).

The resulting spectral image is shown in figure 1-19. Each oblong image of the lunar disk is a spectral line whose width is determined by the spectrometer optics. The height of the spectrum shows the size of the moon in the Spectrometer’s 2-degree field of view.

Figure 1-19: Lunar observation after processing with the standard pipeline and corrected aspect table created by aspcorr.

The aspect correction for a distant source with an inaccurate aspect table is improved by essentially the same method. Any record of the object position can be converted to apparent coordinates in an ST table, and used to calculate the corrected aspect table with aspcorr, before rerunning the pipeline. If positions from DS data are to be used, they may be interpolated to achieve more frequent positions, if another source of information, such as a source ephemeris, is available.

5/97

1.3.3 QCALC calculates quaternions

The task euvtools.qcalc is a quaternion calculator that recognizes some common forms of astronomical coordinates. As many as 26 quaternions can be loaded into registers named a, b, c..., and manipulated through a number of vector operations. Numbers enclosed in angle brackets (<>) are interpreted as celestial coordinates in decimal degrees, and those in curly brackets ({ }) as coordinates in hours, and are converted to quaternions automatically. The user can enter the calculator in interactive mode by typing the task name on the command line:

          eu> qcalc

and begin loading vectors and typing commands after the prompt:

          qcalc>

Exit the task when finished, with a “control-d”, as in the following example:

          qcalc> a = .92 0 .73 0
          qcalc> b = 0 1 0 0
          qcalc> c = a * b
          qcalc> c

0. 0.9200000000000001 0. -0.73

          qcalc> d = 12 54
          qcalc> &d

-0.5877852522924731 7.198055546404513E-17 0.8090169943749474

          qcalc> !d

180. 54.

          qcalc> ^D

12:00:00.0 54:00:00.0

Typing “&” or “!” before a register name signals the task to print the contents in vector or degree form, respectively, with vector as the default.

5/97

1-38 CHAPTER 1. USING TELEMETRY IN ST TABLES 5/97

SELECTING EVENTS FROM QPOE FILES

In this chapter we introduce operations on EUVE QPOE files, which are mostly aimed at producing an IRAF image, like a two-dimensional spectral image or a light curve, from your selected data set. This is the step when data quality selection actually takes place. The creation of new spectral images is where GO’s can apply the data selection filters constructed from the telemetry in Chapter 1, to send only those events that meet the selection criteria to a task that creates an image. While QPOE’s can be used for more extensive processing, it is usually more efficient for IRAF to operate on an image with the selected data set.

Section 2.1 introduces the basic concept of the QPOE interface expression: the language used to tell the QPOE interface what events you want from the QPOE file and how they should be binned when the file is opened for processing. We give some examples of expressions for selecting events based on time, position, the most common types of rules.

Section 2.2 describes the use of macros, which define substitution rules for use in QPOE interface expressions. The use of macros can expedite data selection and image creation, especially when one makes use of the specialized IRAF/EUV tasks for reading, editing, and creating QPOE I/O macro definitions. You may have already used the task dqselect, which is one of these. This section introduces tasks which make it possible to construct a macro from data in an ST table, to merge parts of several expressions together, and to edit the macros in a QPOE file.

Section 2.3 gives instructions for creating spectral images from EUVE QPOE file by using QPOE interface expressions and the specialized task euvred.qpmkim. This task also implements correction for primbsching and TIF deadtime.

Section 2.4 shows how to make light curves or images based on any QPOE attribute using the task euvtools.qpbin. We give an example of making a spectral light curve, and introduce a task for generating tables of good time intervals based on the phase of a periodic source.

Related topics

◇ This chapter does not describe QPOE files in general, or EUVE QPOE file format; please read Chapter 2 and section 4.2 of the EGO Center Data Products Guide for this information.

◇ The causes of primbsching and TIF deadtime losses are described in section

5.1 of the Data Products Guide.

5/97

2-2 CHAPTER 2. SELECTING EVENTS FROM QPOE FILES

2.1 QPOE I/O Interface Expressions

In this section, we will introduce the syntax of QPOE/IO interface expressions and common applications with EUVE QPOE files. These methods are a means to improve signal-to-noise, save considerable processing time, and even to produce very different types of data products. When a QPOE file is sent to a task as input, it is first processed by software known as the “QPOE I/O interface,” or more simply, the “QPOE interface.” This is because QPOE files have a structure different from IRAF images, with many pieces of information, called attributes, for each event. A single operation may use only a subset of the attributes for some subset of the events. For example, one might want to select events by their positions to make an image array, or by times and wavelengths to make a light curve.

The pre-processing of QPOE files by the QPOE interface is how the concept of data quality selection, as introduced in Chapter 1, is implemented. The QPOE interface parses the command line to determine how to bin the QPOE event list for input to a task. This information must be packaged in a specific format, called a QPOE I/O interface expression, or a QPOE expression, for short. QPOE expressions can be typed on the command line, in a way analogous to giving IRAF an image section. Parts of the expression may be supplied by macro definitions inside the QPOE file itself. One can perform many useful data selection tasks, “on the fly”, but as selections become more sophisticated, the expressions can also become quite complex. The strings of start and end times created in examples 1.1 and 1.2 of Chapter 1 are parts of one type of expression, often called a filter.

2.1.1 General syntax

The most basic use of the QPOE I/O interface is to put the entire QPOE expression on the command line, as the input to some task. The syntax of a QPOE interface expression has the general form:

           'filename.qp[keyword1=string1,keyword2=(string2)...]'

The part inside the [ ]’s is what we will call the QPOE expression; it is understood that a QPOE expression must be preceded by a QPOE file name to perform any action. Also note the quote marks: single quotes must surround QPOE interface expressions used on the IRAF command line. The quotes tell the cl to pass the expression to the QPOE interface to be parsed. Quotes need not be used when entering QPOE expressions into the parameter file using epar.

A fairly complete description of the expression syntax can also be found in the xray.qpoe and xray.filter help pages of the xray package. The attribute names and much of the example material are specific to ROSAT and EINSTEIN data.

QPOE I/O Interface Expressions 2-3

5/97 Keyword expressions

The QPOE expression in its most general form is a comma-separated list of ‘keyword expressions’ with the form, keyword=string. The string will be called a ‘keyword string.’ There are two important categories of keyword expressions: attribute filters, and function keyword expressions. The keywords that are most useful with EUVE QPOE’s and their syntax are described briefly in the sections below and the keyword expressions are summarized in table 2-1.

The entire QPOE expression is parsed before any action is taken. Most of the function keywords do not affect one another, with one exception noted below. Filter expressions, on the other hand, are applied to the event list in the order they appear in the QPOE expression. Some care must therefore be taken when using more than one filter, to be sure that their combined effects produce the desired selection. Filter string syntax is discussed in the next section.

Note on nomenclature:

While we attempt to use a consistent nomenclature in this User’s Guide, various parts of the QPOE expression syntax are sometimes referred to elsewhere as ‘QPOE expressions,’ ‘filter expressions’, ‘event attribute expressions,’ ‘QPOE filters,’ and other things too fierce to mention. In particular, the xray QPOE documentation refers to any type of keyword expression as a filter, while this document reserves the term for expressions with an attribute as a keyword. The moral is that one must be careful to make clear exactly what part of the syntax is being referred to in conversations, e-mail, and other correspondence.

2.1.2 Attribute filters have attributes as keywords

The first type of keyword expression uses any attribute in the QPOE as the keyword. The string is called a ‘filter string,’ and is a logical expression for allowed values of the attribute, so that the keyword expression has the function of a filter. We will often refer to this type of keyword expression, and will call it a filter expression or filter for short. The string is bracketed by parentheses whenever the it is a complex series which contains commas (often). The interface allows any combination of the following possibilities as a (comma-separated) list:

            a               equality test                 dx=4
            :a              open range (<= a)             dx=(:4)
            b:              open range (>=b)              dx=(4:)
            a:b             range (a to b inclusive)      dx=(4:16)
            ! expr          not this expression           dx=!(4:16)
            @file           get filters from file         dx=(@foo.filter)

Note that although any QPOE attribute specification, like d0 or s8, can be used as a filter keyword, in EUVE QPOE files the names for the attributes – time, x,

5/97

y,lambda, theta, dx, dy – can also be substituted. For example, the filter:

theta=(-.5:-.4916,.4916:.5)

could appear inside the [ ]’s of a QPOE expression for an EUVE QPOE file, since every EUVE QPOE contains the macro theta for the imaging angle attribute, r12. This filter would select two sections of imaging angles in Y, from above and below image center for processing. The ( )’s are necessary any time a filter string is a list of expressions with commas. As a general rule, a comma between keyword expressions expressions functions as a logical “AND”, and a comma between range expressions inside a filter string functions as a logical “OR”^{1}.

Table 2-1: QPOE interface keyword expressions.

keyword  keyword string  function  examples

rect pair of colon-separated values for key attributes select rectangular section [rect=[57:230,500:524]]

[key=(dx,dy),rect=[40:89,100:140]]

block  single integer  define bin size used to sort

QPOE event list

[block=4]

filter attr=(comma-separated range expressions):

a (attr=a), :a (attr ≤ a);

a:b (a ≤ attr ≤ b); b: (attr ≥ b); !expr (not expr) select allowed values of attribute

{[time=(1234:4567)] \atop [s10=(100:144,160:)]}

mask  mask=maskfile.pl  use PLIO to include or ex-

clude regions [mask=backgr.pl] [mask=b(1024,964,1620,100)]

key comma-separated pair of integer attributes define attributes to be used as binning indices [key=(s8,s10)] [key=(dx,dy)]

5/97 QPOE I/O interface bug.

1See also the discussion in section A.2.1.

There is a bug in all versions of the IRAF/QPOE interface through 2.10.4, that makes it dangerous to specify multiple filters on a single attribute, using any variation of the form: filename.qp[attr1=(string1)...attr1=(string2)]. It turns out that the interface does not merge same-attribute filters reliably. Tasks will run on such expressions without apparent errors, but the number of events processed is not consistent. Unfortunately, there is no simple rule to determine which filters will be merged properly and which will not, so one must always avoid letting the QPOE interface merge filters. A single filter can safely be used to specify a large number of intervals, with a complex logical

expression, as long as the attribute name is only used once.

2.1.3 Useful function keywords

The second type of keyword expression uses a specialized keyword to perform a specific function, and requires a specific syntax in the string. Three of the most useful keywords for EUVE GO’s are: key, block, and rect. The function and syntax of each is described below.

rect=[x1:x2,y1:y2]

The rect keyword allows one to specify a rectangular subregion of the QPOE file which is to be extracted. The keyword string is a region specification using almost the same syntax as an IRAF image section. rect operates rather like an attribute filter, except that it involves two attributes instead of one: it selects the specified ranges of the default key (x,y for EUVE QPOE’s) unless some other key has been specified with key. For example, the expression:

           'test.qp[rect=[100:199,1015:1034]]'

will select a region of 100 columns (x) × 20 rows (y) from an EUVE QPOE file, picking out a short section of the spectrum. Note: the rect string has its own set of [ ]’s, inside the square braces that surround the entire QPOE expression.

The wild card character ‘∗’ can be used to select all possible values. One difference between rect and the IRAF image section syntax is that the rect syntax does not allow a single row or column to be specified by a single number. A range must be given, so rect=[*,5] won’t work; use rect=[*,5:5] to select all events in the fifth row.

Another important difference between rect and filters on position attributes x and y is that rect produces an image with the selected dimensions, and a WCS with

5/97

logical coordinates. The filter:

           [y=(a:b),x=(c:d)]

produces an image with the same size as the QPOE, with 0’s outside the selected area. For this reason, position filters are rarely used. A combination of rect and key expressions can be used to select almost any section of the QPOE.

Since the rect expression is used to select a subset of the event list before it is input to the operating task, it is applicable with tasks that do not bin the event list, such as qppage. i This enables the user to list the events of a small section of a QPOE file, if desired. The command:

          eu> qppage 'sw.qp[rect=[140:1800,970:1054]]'

will cause qppage to display only those events whose (x,y) values match the string, since (x,y) is the default key. Events are displayed in y-order, from 970 to 1054, including only those with x-values in the range 140–1800, which is about the width of the detector in the 2048 × 2048 pixel space.

Related topics

◇ The task qppage is used for scrolling the event list of a QPOE file, and is described in section 2.3.2 of the EGO Center Data Products Guide.

block=pixsize

For expressions using the keyword block, the keyword string is a positive integer which tells the QPOE interface the number of pixels to sum together when binning the event list. Like rect, it operates on the current key attributes. Its single argument is applied to both the column key and the row key; if [block=4] is specified, a 2048 × 2048 EUVE QPOE file would be binned into a 512 × 512 image. To display an EUVE QPOE file as a blocked image in your image tool, type:

          tv> display 'sw.qp[block=4]' 1

Like key, the block keyword is usually used when the QPOE file is being binned for handling by an image task, such as tv.display, qppage, or qpmkim.

mask=(<plio expr> or file.pl)

There is also a keyword that allows the creation of more complex image sections than rect, using what are called PLIO (pixel list input/output) region specifications. The QPOE interface uses the keyword mask to refer to this kind of keyword string, which may be included literally, or stored in a PLIO region mask file (hereafter

5/97

simply ‘mask file’). The mask file may contains a list of many region specifications that select areas with various geometric shapes from the QPOE pixel space. Neither parentheses or the “@” symbol are needed, but the file must be a PLIO mask file and its name must end with .pl.

PLIO region mask files are made with the task xray.ximage.plcreate; please read the help page for instructions on using the task. The PLIO syntax and rules for specifying regions —the input to plcreate— are described in the xray documentation. Type ‘help xray.regions’, ‘help xray.reggeometry’, ‘help xray.regalgebra’, and ‘help xray.regcoords’ for an overview and details. The information needed for simple applications is summarized below.

Table 2-2 gives an incomplete list of shape definitions, from the help page xray.regions. These can be input to plcreate as a command line list separated by semicolons (;) to make a plio file. Arguments are (converted to) real numbers, and angles are in degrees counter-clockwise from the positive Y-axis. The region names are caseinsensitive, and arguments may be listed with or without parentheses, separated by commas and/or spaces. Regions are inclusive by default, with the exception that preceding a region by a minus sign (e.g. −c xcen ycen radius) excludes that region from any other included region in the mask. If two regions overlap, the pixels in the overlap region are included or excluded according to the last expression to the right, so order can be important.

Table 2-2: Summary of PLIO region specifications. The PLIO interface permits parameters to be entered as comma- and/or space-separated lists, with or without parentheses.

Region type abbreviation arguments  examples
ANNULUS ANN,A,a,ann xcen ycen rad1 rad2  a 1029 1019 15 35
[n=# OR rad3 rad4...]  ANN(1029,1019,15,35)
BOX b,box xcen ycen xwidth yheight [angle] box 1020 1030 1800 12

b(1020,1030,12,1800,90)

CIRCLE CIR,C,c,cir xcen ycen radius cir 917 1012 10

c(917,1012,10)

ELLIPSE  ELL,E,e,ell xcen ycen xwidth yheight angle E 34 129 40 60 30
PIE  pie  xcenter ycenter angle1 angle2... pie(1200,1400,15,135)
POLYGON  POL,pol  x1 y1 x2 y2 x3 y3... xn yn  pol 40,120,40,220,65,170

Region expressions may use logical, physical, or world coordinates; logical coordinates, which may be changed by filters or other keyword expressions that precede the mask, are assumed as the default. This means that if a mask is preceded by a block expression, the region arguments must be given in the blocked coordinates.

More complex region expressions can be built up using shapes, the arithmetic operator –, and the logical operators ! (not), & (and), ∧ (exclusive or), and | (inclusive or). Type “help xray.regalgebra” for specifics and some examples.

5/97

key=(sn,sn+2)

The key expression is used implicitly or explicitly whenever the QPOE event list is binned into an image array. This happens when the QPOE is used as input to tasks that expect an image, like display or imcopy, and tasks that create an image as output, like qpmkim. Each QPOE has a default key: the positional attribute used to sort the event list when the file was created. If the QPOE is accessed with no explicit key specification, this default key tells the interface to bin the QPOE using the sort-attributes (usually row and column) as the indices. Therefore, using the default key is the most efficient way to bin a QPOE file.

When this keyword is used explicitly, the keyword string consists of a commaseparated pair of short integer attributes in parentheses. Only short integers can be used for the key, as they will become the IRAF image indices. The most common application is to specify alternate position attributes to use in binning the events when sending a QPOE to a task like tv.display. Sorting by a non-standard key can produce a different kind of image. For example, to create the detector images on the data tape, (s,m,l)w det.FITS and ds det.FITS, the assignment

           [key=(dx,dy)]

is used. This is equivalent to the expression:

           [key=(s20,s22)]

without the special names dx and dy. This tells the interface to bin the events using the raw detector coordinates for the column and row indices, rather than the default key coordinates, the remapped x and y.

2.1.4 Some QPOE expression rules

• The function keywords rect, block, and mask, normally operate on the default key. Specifying an alternate key causes such keywords to operate on the selected key attributes instead, regardless of the order in which the keywords appear in an expression.

• Any keyword string that contains a comma must be surrounded by parentheses, or QPOE will misinterpret the expression. Parenthesized expressions may be nested within a keyword string. See section A.2.1 for more information about how commas and parentheses interact.

• Blocking is performed after rect and attribute filter operations, so rect and filter expressions are written in unblocked coordinates. Later references to sections of displays or images made by blocking a QPOE file must be written in the blocked coordinates.

• Using a key other than the default key can cause a large increase in processing time, since the event list is already sorted according to the default key.

5/97

QPOE Macros 2-9

• One can make a region filter equivalent to a rect expression by using an attribute filter directly on the current sort key (see the item above). However, this will create a full-sized image (2048 × 2048 and 8 Mgb for EUVE QPOE’s) with empty pixels outside the desired section, instead of a smaller image.

Related topics

◇ Appendix A of this User’s Guide has more detailed descriptions of several aspects

of the QPOE interface. ◇ Section A.2 in Appendix A has more details about QPOE expression syntax,

including notes on constructing efficient filters to reduce processing time. ◇ Use the IRAF/EUV task tfilter, described in section 2.2.3 below, to merge

several filters on a single attribute. ◇ Section A.1.3 introduces QPOE header parameters which can be used for stor-

ing default attribute filters and region masks.

2.2 QPOE Macros

QPOE expressions can become long, complex, and tedious to type. Because EUVE time tags have some 12 digits of accuracy, many filter strings are much too long to even attempt typing them on the command line. In Chapter 1, we saw that the task dqselect can install filter strings in QPOE files as macro definitions. The IRAF/EUV packages contain other tasks which can create filter strings from one or more ST tables, merge existing filter strings, install a string as a macro in a QPOE file, and edit existing QPOE macros. Before we introduce these tasks, we will introduce the concept of macros on a more general level, and discuss their role in EUVE data reduction. More detailed descriptions of the use of macros with the QPOE interface have been deferred to Appendix A.

2.2.1 Reading and writing macro definitions

Most QPOE macros are substitution rules with the form NAME=string. For example the definitions in EUVE QPOE files which allow you to use more mnemonic names for the attributes, x, y, time, etc., are simple substitution macros. Macros appear in the output from imhead as lines like:

’TIME = ’ macro[002] macro definition’.

but imhead is not capable of showing the entire macro definition. only its length in characters ([002]) is given, and the macro name has been capitalized, to make it conform to FITS-card format. Most of this section is about data selection macros that contain part of a QPOE I/O interface expression, like a filter string, and are somewhat more complex.

Although we have seen how filter strings can be installed as macro definitions in a QPOE file, placing a macro in the file doesn’t do any data selection. Macros have no

5/97 effect until the macro name is used in a QPOE interface expression which causes the interface

to select the events the filter specifies for some processing. For example, the twodimensional spectral images on each data tape are made using a QPOE time-filter macro, called (det) gt^{2}. However, the events that go into the 2-d spectral images are not selected until the images are made, using the task qpmkimwith macro name in a QPOE expression replacing the filter string. The QPOE interface performs the appropriate substitution, and selects the subset of QPOE events selected by the filter for binning. The macro can be reused any number of times to make new images from the QPOE, and provides a permanent record of the selection filter that travels along with the event list.

Related topics

◇ The construction and installation of the filter macros is described in section

4.3 of the EGO Center Data Products Guide, and in example 1.1 of this User’s Guide.

◇ Macros can be defined in at least two places in IRAF, allowing for a hierarchy of precedence; see the discussion of global and local macros in section A.3.

2.2.2 QPMEDIT displays and edits QPOE macros

Suppose one wanted to inspect or change the macros that are installed by the EGO Center in the QPOE files. Users frequently find it convenient to use the task imhead to look at the global information in a QPOE file, but the information that imhead can supply on macros is minimal. You may recall from section 2.1 of the EGO Center Data Products Guide that this is because QPOE files are not images–the listing one gets by running imhead on a QPOE file is merely an attempt to represent the contents of the QPOE file in the FITS-like format of an IRAF image header.

The task euvtools.qpmedit operates on QPOES in much the same way that images.hedit operates on image header files. It provides capabilities for examining, adding, deleting, and changing the macro definitions in any QPOE file. Other parts of the QPOE file, such as the header parameters and the event list, cannot be changed with qpmedit. The qpmedit parameters also supply options for safeguarding existing macros from incorrect and inadvertent changes.

Figure 2-1 shows a parameter file setup to display the definition of the exposure filter macro sw gt, using the special character “.” for value, which means “the current value.” One can view the complete names and definitions of all the macros in a QPOE file by using the wild card character (∗) for the macro name in qpmedit:

          eu> qpmedit sw.qp * .  update-

Explicitly setting the hidden parameter update on the command line insures that none of the macros will be changed, regardless of the settings of the parameters add

2where det is a substitute for sw, mw, lw, or ds, which is installed into each QPOE file during the EGO Center’s standard batch processing.

5/97

          eu> lpar qpmedit

        files = "sw.qp"         QPOE files to be edited
       macros = "sw_gt"         macros to be edited
        value = "."             value expression
         (add = no)             add rather than edit fields
      (delete = no)             delete rather than edit fields
      (verify = yes)            verify each edit operation
        (show = yes)            print record of each edit operation
      (update = no)             enable updating of the image header
        (mode = "al")

Figure 2-1: Parameter file for qpmedit. Typing “:go” in epar will start the task, displaying the macro “sw gt” in full, gory detail on STDOUT.

or delete. To add or delete macros, either add or delete and the parameter update must be “yes.” The command:

          eu> qpmedit lw spec "y=1000:1040" add+ update+

will create a macro in the QPOE file lw.qp that substitutes the string y=1000:1040 for the name spec in a QPOE expression, which will select a strip from the center of the detector when invoked in a QPOE expression. Note that another macro, the attribute name y, is used in the macro value definition. Existing macros can be edited by setting add and delete to “no”, and giving the macro name and the new value. The following will redefine an existing exposure macro called exposure in an MW QPOE to be the contents of the file mytimes:

          eu> qpmedit mw.qp exposure "@mytimes" add- delete- update+

The file mytimes should contain part or all of a correct QPOE expression. It is the user’s job to keep track of which parts, and whether punctuation like ( )’s need to be used when the macro is referenced. If you looked at the macros in sw.qp, you may have noticed that the definition of “sw gt” did NOT include the attribute name time. When the QPOE is sent to the task that creates the spectral images, the macro supplies only the filter string in the QPOE expression. The [ ]’s, filter keyword, and parentheses are also needed to make the complete expression:

           `sw.qp[time=(sw gt)]'

Whenever a macro is referenced, the string on the right of the “=” sign, must form a complete QPOE I/O expression with correct syntax when substituted into the command line. The person creating the macro decides what parts of the desired filter to include in the definition. All QPOE macros installed in EGO Center processing follow the convention of including only the filter string.

5/97

2.2.3 TFILTER creates QPOE time filters from ST tables

The type of data selection that has been implemented in the euv package uses filtering of the recorded times of QPOE events. This means that when a GO wishes to apply more than one criterion to data selection, for example selecting both for valid exposure times and for certain values of a periodic phase, two or more separate sets of time intervals may be required, one set to satisfy each of the criteria. Simultaneous application of all the criteria amounts to taking the intersection (logical and) of all the sets of time intervals to create a single specifically tailored “good times” filter. This is what the task euvtools.tfilter is designed to do.

Why should I use tfilter?

The main reason for having tasks that intersect sets of time intervals (e.g.

tfilter and dqselect) is to work around a QPOE interface bug, which causes multiple filters on the same attribute to produce unreliable results. Tests have shown that the results of this type of expression depend in an unspecified way on the order of the filters in the QPOE expression.

The main reason for EUVE GO’s to want to use multiple criteria is to merge the nominal exposure filter (det) gt with other time filters customized for their science. So GO’s who wish to add their own time filters to the QPOE files should produce an ST table of start and end times for each criterion, and use tfilter to create a single time filter macro, rather than trying to AND several separate keyword expressions or filter strings together “on the fly.” The gtsave command in the task dqselect creates ST tables of start and end times in the format required by tfilter.

tfilter converts table columns with the start and end times for good time intervals into a QPOE filter string, which can then be written to one of several possible output locations. Here we describe the option of inserting the string into a QPOE file as a new macro. A sample parameter file for tfilter, set up to create a new time filter string from two existing ST tables, is shown in figure 2-2.

          eu> lpar tfilter
        input = "det7filt,ds_gt" Input table name
      (output = eds.qp")        Output file
        (name = "bestime")      Name of output macro
       (start = "stime")        Name of starting column
         (end = "etime")        Name of ending column
     (cmdline = no)             Command line output flag
      (format = "13.3f")          Interval format control
        (mode = "al")

Figure 2-2: Parameter file for tfilter to merge two lists of time intervals. The filter string will be installed as a macro in the DS QPOE file.

5/97

Notes on running tfilter

□ The value of input may be a comma-separated list of several table names; do not include the tab extensions.

□ Each table in input should have columns of start and end times in the same order, named start and end. Tables that were not created by IRAF/EUV tasks may need to be edited, using an editing task such as ttools.tchcol, or ttools.tedit.

□ output names the destination of the finished time filter string. If output has the .qp suffix, then the string is written into the QPOE file output as the macro name. Alternatively, output can be a text filename, to which the complete definition of the macro name will be written or appended.

□ The cmdline parameter takes precedence over output. If it is “yes” (the default), the task output is directed to STDOUT. This option allows the user to make a

“dry run” first, and check the output. cmdline must be “no” to write the output to the destination specified by output.

□ The value of format specifies the format for the start and end times output in the filter string, using C language formatting conventions. The time stamps supplied in the EUVE QPOE files and good times tables have a nine-digit mantissa and numerous insignificant decimal places. Considerable space can be saved by truncating the times to 3 digits; this implies no loss of accuracy, as the times in the telemetry are known only to within ≈ 1 ms in WSZ mode data, or ≈ 8 ms for XY mode.

To create the QPOE time filter bestime from two existing tables and install it in the Deep Survey QPOE file:

          eu> tfilter det7filt,ds gt output=ds.qp name=bestime

tfilter may be used to define either a local macro in the QPOE file itself, as in the examples above, or a global QPOE macro in a macro definition file. See section A.3.1 in Appendix A for a short discussion of global QPOE macros. If the output file is a text file of global macros, the definition of the macro name is appended to the file. The cmdline+ option also provides a way of applying a filter to the QPOE in a command substitution macro reference. Command substitution macros are described earlier in section A.3 of Appendix A.

Example 2.1 : Making a phase-limited spectrum

Suppose that a GO has determined the start and end times of some interesting event, perhaps a stellar flare during an observation, and wants to extract the short wavelength spectrum for that particular interval. The interval may span several orbits, including daytime gaps, and may contain interference from the SAA or another source of background. The GO should therefore combine the interval of interest with the valid times filter already installed in the QPOE file. A new spectral image must then be created using the new filter.

5/97

Specifically, if the GO wishes to extract a spectrum from the SW spectrometer taking only photons from the time interval JD=(762023822:762028792), he can use the task tfilter to combine this desired time interval with the standard exposure macro called sw gt in the SW QPOE file. In order for tfilter to be able to read this interval it must be contained in a time intervals table, with the same format as the tables produced by the command gtsave in dqselect.

The easiest way to make the table is to modify another table of the same form, so the GO uses one of the tables of SW detector good times supplied on the data tape. The table for the SW spectrometer is called sw gt.tab, so the GO uses the task ttools.tselect to make a new table caled mytimes.tab, containing a copy of the first row of the existing table:

          ta> tselect sw gt mytimes "row()=1"

The GO then uses the task tedit to change the start and end times in mytimes.tab to the times bracketing his flare event. If several non-overlapping time intervals were desired, rather than just one, more lines from sw gt.tab could be included in mytimes.tab, and edited in the same way. See the help page for tedit for details on its use, or start the task and type “control-d help” on the prompt line to list tedit commands.

Next, the new table is intersected with the EGO Center valid exposure times in tfilter, and the resulting filter string is used to create a QPOE macro named

mytimes in sw.qp:

          eu> tfilter mytimes,sw gt output=sw.qp name=mytimes cmdline-
          format=13.3f

The parameter cmdline is turned off, so that the output is written to the QPOE file, instead of displayed on STDOUT. The format parameter tells tfilter how many significant digits of each timetag to use.

Once the appropriate filter string is installed in the QPOE file, the GO can create a new 2-d spectral image from the flare with the QPOE expression:

           'sw.qp[time=(mytimes)]'

as input to the task qpmkim (see section 2.3.1).

Related topics

◇ Section A.3 in Appendix A introduces some additional ways to create, store,

and use macros. ◇ See section 4.3.1 – 4.3.3 of the EGO Center Data Products Guide for a description

of macros installed in EUVE QPOE files during EGO Center processing.

5/97

Making Spectral Images 2-15

◇ If you have problems with a macro that is very long (> 64k characters), read section A.4.1, on configuring the QPOE interface to allow longer macros.

2.3 Making Spectral Images

Carefully made time filters select an optimal subset of event data for spectral extraction or other processing. However, the filtering of the QPOE file that must be done by the QPOE I/O interface to access the selected data set is time-consuming, especially if several filters are involved, and can take quite a long time with EUVE QPOE files. It is more generally more efficient to filter the QPOE through the interface only once, creating a standard IRAF spectral image that is used from then on. Creating a light curve is essentially the same task; the only significant difference being that the QPOE file is binned with time as one of the key attributes. Once a data subset is selected and placed in an image, it is not necessary to work with the entire QPOE to extract spectra and determine fluxes (Chapter 3), or for model comparisons (Chapter 4). The focus of this section is helping GO’s create the best two-dimensional spectral image for their science, using the specialized time-filters created with dqselect and tfilter, and the euv task qpmkim for making corrected spectral images.

2.3.1 QPMKIM makes corrected EUVE spectral images

The task euvred.qpmkim is written specifically for making two-dimensional spectral images from EUVE QPOE files, with corrections for TIF deadtime and primbsching effects. Although it is possible to make images from QPOE files using imcopy, qpmkim adds the capability of correcting the exposure time or the resulting pixel values, for TIF deadtime and primbsching effects that might otherwise cause low count rates. In the default mode, corrections are based on the information in the processing table primbsch.tab, which is supplied on the standard data tape. See the help page for the task mkprimbsch and section 1.2.1 for details on how the corrections are calculated.

Do I need to run qpmkim?

If you have revised or remade the data selection filters for your QPOE file, using dqselect, tfilter, or qpmedit, you will need to re-create your twodimensional spectral images to carry out the actual filtering on the data. For those GO’s who find the EGO Center exposure filter adequate to their needs, this work is already done; they can move on to re-extracting the spectra from the 2-d spectral images, or reducing the spectra supplied on the tape.

A sample parameter file for qpmkim is shown in figure 2-3, set up to produce an exposure-corrected image like those on a GO data tape. We give some notes on running qpmkim, but read the help page for more complete information.

5/97

      eu> lpar qpmkim

   input = "sw.qp[time=(mytimes),rect=[140:1779,950:1089]]" QPOE file
  output = "swimg_myti"    Image file
(corrtab = "primbsch")     correction table name
(refdata = "reference/detector") reference directory
(applycorr = yes)            apply correction
(adjexptime = yes)            adjust exposure time
(errband = no)             create error band
(deadtime = yes)            do deadtime correction
(primbsch = yes)            do primbsch correction
(attribute = "time")         attribute used for exposure time
(timecol = "")             optional alternate time column name
(corrcol = "")             optional alternate correction column name
(tepsilon = INDEF)          alternate epsilon value used for time comparison
(deltime = INDEF)          alternate delta of correction times
(infomsgs = yes)            give processing information messages
(fixmwcs = yes)            apply QPOE mwcs workaround fix
(timefield = "time")         QPOE event time field
(dxfield = "dx")           QPOE event dx field
(dyfield = "dy")           QPOE event dy field
   (mode = "al")

Figure 2-3: Parameters for creating an exposure-corrected image file with qpmkim, using the standard correction table primbsch.tab.

Notes on running qpmkim

□ The value of input should be a QPOE interface expression, including any filters that should be applied to the event list. For EUVE QPOE’s, the QPOE expression should specify at least an exposure filter, to eliminate the daytime gaps from the exposure time recorded in the image header. Read the next text on page 2-18 if you have processed your QPOE file with the xray package.

□ The value of refdata should point to the table of detector data, supplied in the reference data on the tape. The default value should be correct in the standard EUVE data directory configuration (see figure 0-3 in the Data Products Guide).

□ The default value of corrtab is the name for the table produced by the task mkprimbsch in the standard EGO Center data processing.

□ If applycorr = “yes” qpmkim gives the user a choice of two different correction modes, depending on the value of adjexptime:

1. When applycorr and adjexptime are both “yes,” the task runs in adjusted exposure time mode. This means each two-frame interval of the selected exposure time is divided by the appropriate correction factor, from corrtab. Factors to correct for TIF deadtime and/or primbsching effects can be

5/97

specified using the boolean parameters deadtime and primbsch. Adjusted exposure time mode is used during EGO Center data processing, because it makes the most efficient use of the QPOE interface. It is recommended that GO’s use this mode unless there is some compelling reason to use the adjusted pixel mode.

2. If applycorr is “yes” and adjexptime is “no,” the task runs in adjusted pixel mode. Then each event is given a real value of 1.0 and multiplied by the correction factor before being added to the value of the pixel at its location. The result is an image with real-valued pixels, which requires twice as much memory as the usual short-integer pixels.

□ In adjusted pixel mode, Poisson errors are calculated for each pixel, and can be saved in a separate image band by setting errband to “yes.” Users should be aware that this effectively doubles the image size again over what is produced in exposure correction mode, as the error band is also real-valued. A 2048×2048 pixel-corrected image with error band requires 32 Mgb.

□ qpmkim reads the INSTRUMENT header parameter from the input QPOE file to determine which detector’s correction factors to use. If deadtime and primbsch are both “yes”, it looks for a column called Det#Q#dpc. If only one is set, either column Det#Q#dc or Det#Q#pc will be used^{3}.

□ If an alternate correction table is to be used instead of primbsch.tab, the name for the column of time stamps for each correction factor is loaded into the parameter timecol. Similarly, the parameter corrcol can be used to specify the name of the correction column in corrtab.

□ The fixmwcs option enables or disables a fix to a QPOE mini-world coordinate system bug. The bug causes the values of the pixels in adjusted pixel mode to be shifted slightly. Until it is confirmed that the bug has been fixed, leave this parameter set to ”yes”.

□ EUVE GO’s should not normally need to change the values of attribute,

dxfield, dufield, timefield, dxfield, or dyfield.

To create an image from one of the EUVE QPOE files, first determine the type of filtering needed and the necessary criteria for “good” data. Use the criteria to create filter strings from the telemetry monitors and save them in tables or in the QPOE header using dqselect, tfilter, and qpmedit. Determine what correction, if any, is needed, and make sure you have a suitable table of correction factors, such as primbsch.tab. You are then ready to run qpmkim.

5/97

3The Q#’s in the column names indicate filter quadrants. However, at this time mkprimbsch can only calculate correction factors for the quadrants which normally have spectra or the DS image.

Does the QPOE file have a deffilt card?

If the QPOE file has been processed with tasks in the xray package, it should have had the header parameter deffilt inserted. It is important to be aware of the action of this filter, which is always parsed by default in the

QPOE interface and merged with any other filters on the same attributes. Because

of the QPOE interface bug mentioned on page 2-5, some attempts by the interface to merge same-attribute filters may produce erroneous results. You may also wish to read section A.1.3 of Appendix A for guidelines on how to prevent the QPOE interface from merging multiple filters.

Simple calls to qpmkim that use the default table and correction type are just as easily made on the command line. For example, to create a spectral image of the entire detector area for the SW:

          eu> qpmkim 'sw.qp[time=(mytimes)]' bigsw myti

qpmkim receives the events, plus the spectral WCS from the QPOE, packed in an appropriate set of image header cards. If the task is run in exposure time correction mode, the corrected exposure is written into the header card EXPTIME, and the time calculated from the exposure filter is saved in the RAWEXP card.

Finally, do not attempt to use qpmkim to make a pixel corrected image from any QPOE file that has been resorted in other than y,x order (sorted first by row, then by column). If qpmkim is used on such a QPOE file in pixel correction mode, it will produce an incorrect number of events, because of the way the algorithm is optimized. This is most likely to come up for GO’s who have created time-sorted versions of the QPOE file for timing analysis. qpmkim will not fail or produce any error message under these circumstances. The amount of error increases with the amount of primbsch correction needed in the data set. Use qpbin on the y,x-sorted QPOE file (section 2.4.1) if an image binned by event time is required.

2.3.2 Making images with IMCOPY

Making a spectral image from a QPOE file with the standard images task imcopy is equivalent to running qpmkim with the parameter applycorr set to “no.” Data selection capabilities are still provided by the QPOE I/O interface in either case.

          im> imcopy `sw.qp[time=(mytimes),rect=[140:1779,954:1089]]' sw spimg

is equivalent to:

          eu> qpmkim `sw.qp[time=(mytimes),rect=[140:1779,954:1089]]' sw spimg ap
          plycorr-

Attribute Binning and Timing Analysis 2-19

5/97 Related topics

◇ If you are using a QPOE that has been processed with the xray package, please

see sections A.1.4 and A.1.3, and example A.1, on working with default filters. ◇ The subsection “Examples of specialized QPOE expressions” in section A.2.1

has some examples of QPOE expressions for creating different images.

2.4 Attribute Binning and Timing Analysis

Some targets have interesting periodic behavior in the EUV. Having the data in the form of time-tagged event list (QPOE) files makes it relatively easy to create light curves from EUVE data. Making a light curve corresponds simply to sorting the data with the attribute time as one of the keys. This section shows how to create light curves for all or part of a spectrum, or from the Deep Survey Lexan image.

2.4.1 QPBIN rebins on any attribute

The task euvtools.qpbin is used to rebin or “project” a QPOE file with respect to one of its attributes. Since rebinning along the coordinate axes is already easy to do using QPOE interface expressions, qpbin is most helpful for time binning. It sorts events from QPOE’s into a one-dimensional distribution of counts vs. attribute value with an bin width specified by the user, and writes the array as an IRAF image. A typical parameter file for qpbin is shown in figure 2-4. The use of qpbin to make a background-subtracted light curve is illustrated in example 2.2.

          eu> lpar qpbin

        input = "swdet.qp[rect=(300:350,720:1299)]" QPOE file
       output = "sw_83A"        Image file
   (attribute = time)           QPOE field name
          (x1 = INDEF)          Lowest value
          (x2 = INDEF)          Highest value
          (dx = 100.)           Delta value
          (nx = 64.)            Number of output bins
       (units = "seconds")      QPOE field units
   (integrate = yes)            Provide range information
        (mode = "al")

Figure 2-4: Parameters for qpbin to make a light curve with 100-second bins. The value of nx is superseded by that of dx.

5/97

Notes on running qpbin

□ Many users want to make a light curve for only part of the spectrum, or even a single feature. Select the events from the interesting spectral region by using a QPOE interface expression such as a rect expression in input. Expressions may also be made using the attributes λ (wavelength) and Θ (imaging angle) as the sort key; this obviates having to locate the desired area in the (x,y) pixel space. The size of a spectral resolution element in each of the Spectrometer channels is about seven pixels.

□ Any attribute in the QPOE file may be used to do the binning, and can be selected by setting the value of attribute to a QPOE attribute expression, like d8, or by using a macro already defined in the file, such as time.

□ If the range parameters x1 and x2 are both “INDEF,” qpbin will use the entire available range of attribute. This will override any setting of nx.

□ At least one of the parameters dx and nx must always be defined (i.e. not INDEF). If the values of dx and nx conflict, dx, the bin size, takes precedence, and qpbin creates as many pixels as necessary to span the interval.

□ The units of attribute can be specified for inclusion in the image header.

□ Set the parameter integrate to create a second line with valid time information.

Data gaps and valid time integration

Gaps in the data cause problems in creation of equal-width time bins. Usually, qpbin divides the interval x1-x2 by dx or nx to create time bins and sums together the events it finds within each bin. However, because there are always some gaps in the telemetry, at least for orbit daytimes, and additional time filters are often used, one cannot predict in advance how many seconds of any time bin include good, or even valid, data. For this reason, the parameter integrate should be turned on when any time filter, which may incorporate the standard exposure filter and user-defined filters, is applied to the input to qpbin, i.e., just about always.

When an exposure filter is used, and integrate = “yes”, qpbin computes the number of seconds in each bin that passed the filter, and this information appears in the output image as the value of a corresponding pixel in a second line. The GO can then normalize the pixel values, by using the task images.imarith to divide the first line of the output image by the second^{4}. This produces a light curve of the count rate versus time. Appropriate error bars can be computed using imarith and the information in the two lines of the output image. If the bins have very small effective exposures, each one will have few counts and the data will be noisy.

4Be sure to set divzero in imarith to“0” in case some bins correspond to no valid times.

5/97

Background subtraction for light curves

Background subtraction poses another complication. The background count rate can vary substantially during an EUVE observation, and qpbin makes no background correction. The user can perform this task by making a light curve of an appropriate background region with qpbin, and then using imarith to subtract the two light curves.

Example 2.2 : Making a light curve

This example demonstrates the use of qpbin in making a light curve of a selected spectral region. Suppose the GO wants to monitor the brightness of a group of spectral features between 106 and 128 Å. The correct QPOE expression must be input to the task qpbin to extract the light curve.

First, the GO displays the QPOE file sw.qp in an image window, the lines are identified “by eye”, and the area they occupy is marked with the cursor. This part of the spectrum is pixels 633 − −956 in x and pixels 1015 − −1034 in y. This leads to the following QPOE expression, to be used as input to qpbin:

           sw.qp[rect=[633:956,1015:1034],time=(sw gt)]

The exposure filter string from the nominal EGO Center’s reduction is accessed as the macro sw gt stored in the file.

The GO then uses a single command to create the light curve and write it into an image called scurve:

          eu> qpbin 'sw.qp[time=(sw gt),rect=[633:956,1015:1034]]' scurve attri=
          time dx=30 units=seconds integrate+

The same method is used to select a background region with the same width as the source region, but above the spectrum and 100 lines high. The GO extracts a background light curve with a similar call to qpbin:

          eu> qpbin 'sw.qp[time=(sw gt),rect=[633:956,1040:1139]]' bcurve attri=
          time dx=30 units=seconds integrate-

Note that a line of normalization factors is not made for the background curve (integrate-). It is not necessary to normalize the curves independently; since they are both created with the same time filter, their norms would be the same:

\frac{scurve}{norm} − \frac{bcurve}{norm} = \frac{[scurve − bcurve]}{norm}

The task images.imarith is then used to normalize the background light curve

5/97

by the ratio of the background sample area to the spectrum, and subtract the background curve from the source curve:

          im> imarith bcurve / 5.  bcurve n
          im> imarith scurve[*,1] - bcurve n[*,1] scurve sub

where 5 if the ratio of heights of the background and the section used for the light curve. This puts the background-subtracted light curve into the image scurve sub. Then the exposure is corrected, using the normalization values in line

2 of scurve:

          eu> imarith scurve sub / scurve[*,2] final curve

This final step turns counts to count rate in each time bin, correcting for any exposure variations between the bins. The subtracted and normalized curve is shown in figure 2-5.

Figure 2-5: Light curve constructed with qpbin for a section of the SW spectrum, in background-subtracted normalized counts per second.

The count rate shown is from the entire spectrum area chosen with the rect QPOE filter. The bins with the very highest rates are actually derived from low-exposure low-count times bordering some rate shutdowns in the SAA, and so will have large error bars. The GO constructs error bars for the light curve from the signal and background counts in the light curves. Recall that the

5/97

standard deviation of events accumulated as Poisson statistics in a bin with signal S and average background B̄ is given by:

σ_{S} = \sqrt{S + B̄ (1 + 1/n)}

where n is the ratio of the background height to the spectrum height, Since scurve contains both source and background counts (S + B), and the GO has already formed the average of the background curve in bcurve n, an image containing the deviation for each pixel of final curve can be created by the commands:

          im> imarith bcurve n[*,1] / 5.  sigmaB
          im> imarith scurve[*,1] + sigmaB variance
          im> imfunction variance deviation sqrt
          im> imarith deviation / scurve[*,2] error

The resulting image, error, contains the properly scaled Poisson errors for final curve. The error bars are plotted in figure 2-6.

The smaller deviations in the plot show variations of up to 5%, due mainly to unremoved Primbsching/deadtime effects, so the curve shows no obvious variations at this time resolution.

A broad band light curve can be made from the Deep Survey QPOE file, using similar commands, and region masks which select the source area and a background region. For example:

           ds.qp[time=(ds gt),mask=image.pl]
           ds.qp[time=(ds gt),mask=backgr.pl]

where the PLIO mask files image.pl and backgr.pl contain the regions cir(1024, 1024,

15) and ann(1024,1024,20,35) respectively. See the help pages xray.region and xray.reggeometry

for a detailed description of region masks.

2.4.2 PERIODINT calculates times for different phase points

The euv package supplies the task euvtools.periodint for generating tables of phase intervals for periodic sources. It creates a table of start and end times for a selected section of the object’s phase cycle, over the period of the observation. The inputs

5/97

Figure 2-6: Light curve of a portion of a SW spectrum; same data as in figure 2-5, with errors.

required are the observation start and end times, a time associated with a phase of 0, and the start and end phases of an interval to be selected or excluded. The output, which can be ASCII or an ST table, has the same standard column names as good times tables output from dqselect, and is suitable for creating a QPOE time filter string with tfilter.

A sample parameter file for periodint is shown in figure 2-7, with values set to produce the a table of start and end times for the brightest 30% of a source period whose length is 132 minutes.

Notes on running periodint

□ The range of startphase and endphase are defined to be 0 − 1.0, where maximum brightness occurs at a phase of 0.

□ The value of epoch must be ≤ the value of startobs, although they need not be stated in the same units. If you wish to use a time within your observation for epoch, you must make startobs equal to this time, and discard the portion of the observation prior to it.

□ The boolean parameter include is used to determine whether the interval between startphase and endphase should be included in the time intervals generated

5/97

          eu> lpar periodint

       output = "bright"        Name of output file/table
       (epoch = 2448921.0)      time of prior maximum point of period
  (epochunits = "jd")           epoch units [jd|mjd|euvesecs]
   (periodlen = 132.)           length of period
 (periodunits = "minutes")      period units [seconds|minutes|hours|days]
  (startphase = 0.85)           start phase of interval
    (endphase = 0.15)           end phase of interval
     (include = yes)            include [yes] or exclude [no] phase interval
    (startobs = 781740329.41216) observation start time
      (endobs = 781794000.46016) observation end time
    (obsunits = "euvesecs")     observation units [jd|mjd|euvesecs]
       (table = yes)            make an ST table ([no] for ASCII)
        (mode = "al")

Figure 2-7: Sample parameters for the task periodint, to calculate start and end times of the brightest 30% of each phase cycle. The julian date given for epoch is a known peak in brightness from a previous observation.

or excluded from them. The task produces a list of start and end times for the “good” times selected, regardless of the value of include. For example, the following command line will produce a table equivalent to that produced with the parameters in figure 2-7:

               eu> periodint notdim startph=0.15 endphas=0.85 include-

since excluding the phase from .15 to .85 is equivalent to including that from .85 to .15.

□ The definition of the units “Modified Julian Date” and “euvesecs” is given in the help page. The task backmon can be used to produce times which are equivalent to “euvesecs”, in that they need only be converted from days to seconds. This is the output in the backmon column labeled MMJD.

Related topics

◇ The xray.xtiming package also has tasks that may be useful. See the IRAF help pages for general description, and section A.1.4 for tips on using these tasks with EUVE QPOE files.

5/97

2-26 CHAPTER 2. SELECTING EVENTS FROM QPOE FILES 5/97

EXTRACTING AND FLUXING EUVE SPECTRA

All EUVE spectra delivered to Guest Observers have been extracted using the IRAF/EUV task euvred.euvextract. The same set of spectral and background apertures is used for all GO extractions. Extraction and background subtraction are done on a rectangular aperture in each 2-d spectral image, whose dimensions have been chosen to ensure that as much of the source flux as possible falls within the spectral aperture, assuming that the spectrum is perfectly centered^{1}. A detailed description of EGO Center extractions may be found in section 4.5 of the EGO Center Data Products Guide. No effort is made to optimize the extractions on a case-by-case basis. It is therefore recommended that all GO’s perform a re-extraction of each spectrum, to correct any possible omission of source counts.

The aperture used (25 rows) may also seem quite wide for faint spectra, as the source counts in the tails of the distribution are often overwhelmed by noise. The signalto-noise of such spectra may be improved by “tailoring” a narrower aperture, but if a narrower aperture is used, some source flux will be eliminated. Any absolute flux measurements based on the extraction will then need to be corrected (if possible) or accompanied by some estimate of the systematic errors introduced. Relative fluxes from different parts of the spectrum will have different levels of systematic error, as the spectral profile changes slightly with wavelength.

IRAF spectral terminology:

It will be helpful in reading this chapter to have the following definitions firmly in mind. In IRAF the term “dispersion axis” means the image dimension along which wavelengths are dispersed. Each spectral image contains the header card DISPAXIS, which is assigned a value of 1 for dispersion along lines or rows (X) of the image, and 2 for dispersion along columns (Y). The dispersion direction must also be indicated in the parameter files for many

of the tasks in twodspec, such as apextract.apall.

The term “dispersion line”, on the other hand, means pixels along the dimension perpendicular to the dispersion axis, whether that be along rows or columns. Therefore, since for EUVE images the dispersion axis is along X (rows, DISPAXIS=1), dispersion lines correspond to image columns along Y. “Dispersion line” is used in a number of help pages and parameter files, usually in reference to producing a spectrum profile for aperture centering.

5/97

1Empirical tests indicate that the 25-row aperture includes roughly 98% of the source flux in a centered spectrum.

3-2 CHAPTER 3. EXTRACTING AND FLUXING EUVE SPECTRA

This chapter describes the use of the IRAF/EUV extraction task euvextract, as well as extraction with the standard IRAF aperture-editing task apextract.apall. Tasks are introduced which can produce estimated fluxes and subtract the effects of multiple spectral orders.

3.1 Extracting EUVE Spectra

It is assumed throughout that spectra are being extracted from a two-dimensional spectral image created by filtering the event list of an EUVE QPOE file. Twodimensional exposure-corrected spectral images are supplied on the Guest Observer data tape, and can be used without accessing the QPOE files if no other data filters are needed for GO science. The spectral images have been trimmed to the physical coordinate section [1:2048,720:1299], and have logical coordinates of [1:2048,1:580]^{2}. All locations specified on images with the names sw.imh, mw.imh, or lw.imh are in these coordinates. If one is extracting spectra from 2-d spectral images made by some other means, such as using qpmkim with a custom data selection filter, aperture centers and dimensions may differ from the examples below.

Related topics

◇ See section 4.5.1 of the Data Products Guide for a diagram illustrating the image sections used to create the 2-d spectral images and the extracted spectra.

3.1.1 EUVEXTRACT creates and documents EUVE spectra

The task euvred.euvextract performs a spectral extraction with subtraction of averaged background, and produces a [2048 × 1 × 3] dimensional spectrum image with the background-subtracted spectrum, standard deviation, and average background in three separate bands. The user may define the aperture center and size, may apply a simple centering algorithm, or let the center value default to the center of the image. The centering algorithm sums a specified set of dispersion lines (columns) to find the row corresponding to the highest resulting pixel value. Spectral and background apertures are defined in terms of image rows above and below the center of the spectral aperture [-low:+high], following standard practice in IRAF. The sides of the spectral aperture but need not be symmetric about the center. Multiple background apertures may be specified as a comma-separated list of sections in Y, with the same syntax and using the aperture center as the reference.

2Logical coordinates are the default for most plotting tasks in IRAF.

5/97

Extracting EUVE Spectra 3-3

Do I need to run euvextract?

The EGO Center advises GO’s to re-extract their spectra, and most GO’s will find the task euvextract, which uses a straight rectangular aperture and averaged background, is adequate for this purpose. Since EUVE spectra are typically faint and difficult to trace, optimal extraction is not usually practical. Those with bright spectra, especially continua, or long exposures may wish to use the more flexible and complete task apall, which is described in section 3.1.2.

A sample parameter file for euvextract is shown in figure 3-1.

      eu> lpar euvextract

   input = "lw"            Input image
  output = "alcen_lwsp"    Output spectrum
 (center = 294)            Spectral pixel position
(spec_ap = "-7:7")         Spectrum aperture
(seekspec = yes)            Find the spectral position?
(cntr_ap = "168:179")      Centering aperture
 (radius = 100)            Radius from center to seek
  (thres = 4)              Threshold, sigma above the noise
(background = "smooth")       Background computation method
 (bkg_ap = "-84:-10,10:84") Background sample region
(bkg_fltr_sz = 7)              Size in pixels of bkg smoothing filters
(verbose = no)             Print informational messages?
   (mode = "al")

Figure 3-1: Parameter file for extracting a spectrum of Alpha Centauri with euvextract, using automatic centering and background smoothing.

Notes on running euvextract

□ Use euvextract after examining the spectral image or QPOE using display and a 2-d image tool, or by plotting profiles with splot or plot.implot. Note the apparent location of the spectrum, and determine the areas you want to use for background subtraction, making sure they are free of hotspots, deadspots or other local variations.

□ The value of center may be specified, or left as “INDEF”, which defaults to the center line of the input.

□ Specify the spectrum and background apertures as ± limits with respect to center. You may use multiple background apertures, such as ranges of lines above the spectrum (+a:+b,+c:+d) and one below (-e:-f,-g:-h).

□ If the centering feature of euvextract is used, seekspec must be “yes” and cntr ap should be a section of columns where the spectrum is bright. Use columns that

5/97

span one or more visible features A comma-separated list of sections can be used to select several features.

□ The value of radius must be large enough to include both the spectrum and a representative sample of the background above and below for the centering algorithm to be effective. It is recommended that the default value of 100 not be changed.

□ The background computation method may be “average”, “smooth”, or “none”. For the first two, the background is averaged in Y and scaled by the width of the spectral aperture. If “smooth” is chosen, boxcar and median filters of width bkg fltr sz are also applied before subtraction, and the background subtraction is assumed not to add additional noise. Samples of the averaged and smoothed backgrounds for an SW spectrum are shown in figure 3-2.

Figure 3-2: Averaged (bottom) and smoothed (top) backgrounds computed with euvextract for a spectrum in the SW Spectrometer.

When the task is run, the spectrum pixels are summed along each column. All the background pixels at the same column are averaged and scaled by the spectrum aperture width. The background spectrum may be smoothed by filters of user-specified width before being subtracted. The spectrum, minus the average background (B̄) at each column, is placed in the first image band. The 1-σ standard deviation from counting statistics is calculated from the spectrum and background counts, using the definition:

5/97

σ_{S} = \sqrt{S + B̄(1 + 1/n)}

where S is the subtracted spectrum, and n is the ratio of the background to spectrum aperture heights. The errors are placed in the second band. The averaged background spectrum is placed in the third band.

The error calculation assumes that background subtraction adds no noise (n → ∞) when background is “smooth.” Subtracting a smoothed background is therefore advantageous for relatively bright spectra with substantial backgrounds (average pixel > 1 count), as it produces a higher S/N spectrum. However, smoothing may cause under- or over-subtraction near sharp spectral features and edges, so users must think carefully about what background regions they should use.

When seekspec is “yes,” the parameters seekspec, cntr ap, radius, and thresh govern the spectrum-finding algorithm. The algorithm sums together the columns specified by cntr ap. Then the highest valued pixel within radius of center (in the Y direction) is chosen as the peak position, subject to the requirement that its value is more than thresh standard deviations above the mean of the nearby background.

The background mean and standard deviation for the area within radius of center are computed by an iterative process. First they are computed for the entire distribution, and any points more than thresh standard deviations from the mean (e.g. the bright columns with the spectrum) are eliminated. Then the process is repeated until no more eliminations occur, and the last mean is assumed to represent the background. The reason why the peak finding radius should include mostly background area is that the background on either side of the spectrum should dominate calculations of the mean after a small number of iterations. If there are no bins within radius that exceed the mean by threshσ, the task cannot distinguish any peak from the background, and the aperture center defaults to center.

Example 3.1 : Extracting a spectrum with euvextract

Suppose that a display of the short wavelength spectral image for an observation of the target EQ Pegassi shows a faint but visible emission spectrum with a height in Y of 15 rows at its brightest. The GO may be able to improve the signal-to-noise over the automated extraction, without losing an appreciable quantity of signal, by re-extracting the spectrum with a narrower aperture than was used in the nominal extraction.

To get a more precise estimate of the aperture width, the GO notes the approximate locations of some features at different points along the spectrum and plots several sections of columns in implot. Figure 3-3 shows three profiles, centered at columns 1025, 795, and 446, overplotted in the graphics window. Each profile is averaged over 10–12 columns. The GO uses cursor reads to determine that an aperture from 298 to 313 will include the vast majority of source counts.

The GO also notes the locations of two sections that are well separated from the spectrum, and each 100 lines in height, to give an overall background-to-spectrum ratio of 12.5. Since this is an emission spectrum, the section used for the profile-finding algorithm in euvextract must be on at least one of these features; the GO uses all three lines for peak-finding.

5/97

Figure 3-3: Spectrum profiles for several lines, plotted with implot. Each profile is an average of several columns from a different part of the spectrum.

The GO enters the following on the command line to perform the extraction:

          eu> euvextract lw eqpeg swsp center=306 spec ap=-8:+8 seekspec+ 440:453,
          789:800,1020:1030 bkg ap=-114:-15,15:114 bkg fltr sz=7

The extracted spectrum, error band, and background are plotted in figure 3-4.

The size of the background smoothing filter used corresponds roughly to the size (in pixels) of the resolution element for each of the spectrometer detectors. The background begins outside the width of the default spectrum aperture used in nominal processing (±12 pixels), so that source counts in the tails of the spectrum distribution do not add to the background subtraction. The profile finding algorithm located the peak of the profile at row 306.

3.1.2 Extracting EUVE spectra with APALL

Some GO’s may wish to use the tasks in the twodspec.apextract package to extract their spectra, to make use of the aperture editing, tracing and background fitting functions available in these tasks. Using apextract.apall, one may define an aperture, set background sampling regions and subtraction methods, and attempt tracing the spectrum interactively before extraction. If these tools are applied carefully, an

5/97 Figure 3-4: Background-subtracted spectrum of EQ Pegassi (top), the standard deviation (center), and the smoothed background (bottom). Extracted with euvextract.

increase in signal-to-noise can sometimes be achieved. Users who are not already familiar with apall should read the help pages for the individual tasks apedit, aptrace, and apsum. More detailed information on the algorithms used can be found in the IRAF User’s Guides on apextract tasks, listed in Appendix D.

Since the apextract package was written for CCD data, the algorithms are not well suited to the low signal and high noise levels often present in EUVE data. In particular it is very difficult to trace most spectra. Most EUVE spectra have noise and small amounts of residual curvature near the ends, caused by varying detector field effects and poor distortion correction for these areas. Even with a bright continuum, some pixels may be lost to background subtraction noise at the ends of any spectrum. The overlapping spectral bandpasses of the channels should allow extraction of every wavelength in at least one detector. Bright continuum spectra are the best prospects; they may produce an accurate spectrum trace, which will follow small deviations, but the tracing algorithm may still need help.

Parameter setup for apall

An example of the parameter file for the interactive task apall is shown in figure 3-5, parts (a-d). It is partitioned, and shown in the format of epar, rather than lpar, because it is quite long. However, most users will probably want to edit the parameter file at least once. It actually includes (as the name suggests) the

5/97

parameters for nearly all of the other tasks in the apextract package, which are called from within apall.

The user sets several boolean parameters in the first section to call each of these tasks or not, and subsequent parts of the parameter list become useful or not, depending on the choices. The parameters are not used if the task is not called, and can then be ignored. The parts of figure 3-5 associated with functions that EUVE GO’s will not use may be ignored. Instructions for each of the sections EUVE GO’s might use are included in the notes after each part of the figure.

          ap> epar apall
        PACKAGE = apextract
        TASK = apall
     input   =  sw_hz43[*,720:1299]  List of input images
     (output =            hz43_swsp) List of output spectra
     (format =            multispec) Extracted spectra format
     (referen=                     ) List of aperture reference images
     (profile=                     ) List of aperture profile images
     (interac=                  yes) Run task interactively?
     (find   =                   no) Find apertures?
     (recente=                   no) Recenter apertures?
     (resize =                   no) Resize apertures?
     (edit   =                  yes) Edit apertures?
     (trace  =                  yes) Trace apertures?
     (fittrac=                  yes) Fit the traced points interactively?
     (extract=                  yes) Extract spectra?
     (extras =                  yes) Extract sky, sigma, etc.?
     (review =                  yes) Review extractions?
     (line   =                  745) Dispersion line
     (nsum   =                  100) Number of dispersion lines to sum

Figure 3-5: a) Global parameters for the task apall. These govern the calling of other apextract tasks, and the image section used to show the spectrum profile.

Notes on global apall parameters

□ If you are not using one of the two-dimensional spectral images supplied on your data tape, it can be helpful to use an image trimmed to similar dimensions. This omits the bright stim pin events near the detector edges, which tend to divert spectrum traces.

□ The name for the output file is a hidden parameter, so it must either be entered in the parameter file, or given on the command line with the pname=value syntax.

□ The format of the output should be “multispec”; which allows the subtracted background spectrum to be stored in a second image band.

5/97

□ Aperture reference and profile images are not generally available for EUVE data. Leave these values as null strings.

□ Run the task interactively. The IRAF graphics capital letter commands are always available when you are interacting with a display in STDPLOT.

□ Turn off the calls to tasks for automatic aperture finding, centering, and resizing, as these generally fail for EUVE spectra.

□ Turn on the call to apedit, the aperture editing task. Use aptrace in interactive mode if at least part of the spectrum is bright enough to be seen in display. Turn on extraction, background extraction and the ability to review the spectra.

□ Set line to a dispersion line (column) in a bright area of the spectrum, away from any airglow features. For an emission spectrum, choose a line in the brightest feature. Set the number of columns to be summed, nsum, to span the feature, or to at least 20 for continuum sources.

                                # DEFAULT APERTURE PARAMETERS
(dispaxi=                    1) Dispersion axis (1=along lines, 2=along columns)
(lower  =                  -7.) Lower aperture limit relative to center
(upper  =                   7.) Upper aperture limit relative to center
(apidtab=                     ) Aperture ID table (optional)
                                # DEFAULT BACKGROUND PARAMETERS
(b_funct=              spline1) Background function
(b_order=                    1) Background function order
(b_sampl=      -114:-15,15:114) Background sample regions
(b_naver=                   20) Background average or median
(b_niter=                    3) Background rejection iterations
(b_low_r=                   3.) Background lower rejection sigma
(b_high_=                   3.) Background upper rejection sigma
(b_grow =                   3.) Background rejection growing radius

Figure 3-5: b) Parameters for editing apertures using apedit in the interactive mode of apall.

Notes on apedit parameters

□ The dispaxis parameter determines the definition of dispersion lines (see box on IRAF spectral terms on page 3-1). Since dispersion is along image lines, dispaxis should be set to 1.

□ The values of upper and lower for spectral and b sample for background apertures are specified relative to aperture center, which is NOT set anywhere in the parameter file. The upper and lower limits of the aperture may be changed at will in the interactive mode, so these values can be guesses.

□ apidtable is not used in EUVE applications.

5/97

□ Always set the background sampling region. Whether or not the other BACKGROUND parameters are used depends on whether the EXTRACTION parameter background is set to “fit” (figure 3-5 d). The sample file sets b sample to two 100-line regions. The b naverage has been set to average (+)20 points together (- produces the median) for each point in the fit. Background parameters can also be changed interactively.

□ Skip over parameters for APERTURE CENTERING, FINDING AND ORDERING, RECENTERING,
and RESIZING.

The task which uses the next parameter set, aptrace, locates a spectrum profile at a number of places along the dispersion axis, and fits a function through the peak locations. All the parameters can be changed while running in interactive mode.

                                # TRACING PARAMETERS
(t_nsum =                   50) Number of dispersion lines to sum
(t_step =                   10) Tracing step
(t_nlost=                   10) Number of consecutive times profile is lost befo
(t_funct=             legendre) Trace fitting function
(t_order=                    2) Trace fitting function order
(t_sampl=             300:1750) Trace sample regions
(t_naver=                   -5) Trace average or median
(t_niter=                    3) Trace rejection iterations
(t_low_r=                   3.) Trace lower rejection sigma
(t_high_=                   3.) Trace upper rejection sigma
(t_grow =                   0.) Trace rejection growing radius

Figure 3-5: c) Parameters for tracing the spectrum: using aptrace in the interactive mode of apall.

Notes on aptrace paramters

□ EUVE spectra are fairly straight, but noisy, so averaging more peaks per point can smooth out the trace. Set t naverage, the number of columns averaged together for each profile, to 50, or at least as wide as your widest emission feature. The step size between fit points, set by t step should be fairly small.

□ The number of trace points with no profile that are allowed before the trace gives up, t nlost, should be generous, as this happens quite often.

□ The polynomial form is of little importance, as most traces don’t involve terms higher than second order.

□ The trace sampling region, t sample, should include only the detector. The example parameters assume a bright continuum, in keeping with our HZ43 example. The areas outside and just inside the detector edges (about 120-140 and 1750-1780 in X), should generally be excluded.

The next set of parameters defines the conditions of the spectrum extraction, including the method used to estimate the background.

5/97

                                     # EXTRACTION PARAMETERS
     (backgro=              average) Background to subtract
     (skybox =                    7) Box car smoothing length for sky
     (weights=                 none) Extraction weights (none|variance)
     (pfit   =                fit1d) Profile fitting type (fit1d|fit2d)
     (clean  =                   no) Detect and replace bad pixels?
     (saturat=                INDEF) Saturation level
     (readnoi=                   0.) Read out noise sigma (photons)
     (gain   =                   1.) Photon gain (photons/data number)
     (lsigma =                   4.) Lower rejection threshold
     (usigma =                   4.) Upper rejection threshold
     (nsubaps=                    1) Number of subapertures per aperture
     (mode   =                   ql)

Figure 3-5: d) Parameters for extraction and summing with apsum in apall.

Notes on apsum parameters

□ The type of background subtraction may be “none,” “average,” “median,” “minimum,” or “fit.” This example illustrates the use of averaged background subtraction, so that the sampling region is averaged and scaled to the aperture size as in euvextract, and all other BACKGROUND parameters (figure 3-5 b, above) are ignored. Background fitting can also be used, and may be better for long exposures, if there is some appreciable variation in the Y-direction.

□ Set weights and clean to “no,” as optimized extraction is usually impractical with EUVE spectra. Other parameters can be safely ignored for EUVE data.

Running apall

The task apall combines the steps of finding, editing, and tracing an aperture and extracting a spectrum. It is made mostly of ordered calls to the other tasks in the apextract package, any of which can be activated or skipped over either in the parameter file setup or at runtime. A lot of the business of running apall is keeping your orientation. While running in interactive mode, apall and its sub-tasks issue a number of queries at prospective branch points in the process, making it possible to bypass any single function during the execution. The queries signal that the task is ready move on, and when the user responds, the current subtask, and usually the display, will change.

Besides the different task calls, there are various modes within some tasks. For example, both apedit and aptrace make use of an IRAF routine called icfit, for “interactive curve fitting”. Each interactive session with one of the tasks or icfit ends when the user types q to return to the calling task. Keeping track of the number of nested task calls is important; both for avoiding mistakes and for being sure you know what processing path created the output. After some practice, you will learn to associate the different displays with the various “layers” of the task.

5/97

There are four basic steps to extracting most spectra, as enumerated below. You may decide to skip the tracing step if your spectrum is very faint, but it isn’t a bad idea to try it once as it illustrates nicely where the spectral profile is and is not obvious. The task can be started from the parameter file by typing :go, or typing:

      ap> apall input imagename

and answering “yes” to the edit query. If you need help on the command options at any stage, type ? to scroll a help page in the text window. Remember the help page you get depends on (and will tell you) what subtask you are in. Return to interactive mode by typing q and then q or <cr>.

1. Define the aperture. apedit has no parameter for the aperture center in apall. The task forms the sum of nsum columns about the column line (“line” is short for “dispersion line”, which in the case of EUVE images, is a column). The sum is displayed on STDGRAPH as a plot of counts vs. row number. Make a new aperture by placing the cursor on the peak and typing m. As soon an aperture is made, the limits are set using the values of upper and lower. The display should look something like figure 3-6. Typing :parameter in apall causes the current parameter value to be displayed. Typing :parameter value sets the parameter to “value”. For example, you can adjust the center, upper, and lower limits of the aperture with :center, :upper, and :lower. Or position the cursor and type c, u, or l, and the cursor position is read as the new value.

2. Set background parameters. If you have set the parameter background to “av-

erage,” skip this step and type “q” to exit from apedit. Otherwise, you are fitting the background (background = fit), so use the b key command to enter the background fitting mode of apedit. This calls the IRAF-internal task, icfit, which performs the fitting specified by the BACKGROUND parameters, and displays a plot of the background sampling region, summed over all rows in the background aperture(s), with the fit points and the resulting function. The command :vshow lists all current icfit parameter values. The fit points are averages of naverage rows (perpendicular to spectrum). All fit parameters can be changed with colon commands, as in step 1 above. Type f after changing parameters to redo the fit and see the new results. When satisfied with the appearance of the background function, type q to return to apedit, which called icfit. Now the aperture and background subtraction are set up, so type q again to return to apall. You will be queried about the next step, which in our example is aptrace. Type y if you want to trace the spectrum, or n to skip on to the extraction in step 4.

3. Trace the spectrum. Enter the interactive fitting mode of aptrace by answering “y” to the next query (or two). An initial trace and fit are performed immediately using the aptrace parameter values (figure 3-5 c). aptrace sums each t nsum columns and searches for a peak in the resulting profile. If for any particular profile, no peak is found, the task reports the failure and moves on. It can fail t nlost times in succession before it “gives up”. Every t naverage

5/97

peak locations are averaged together, and a fit is again calculated by icfit to the averaged points. Since fittrace is set to “yes” in figure 3-5 c), the display will go back to icfit’s interactive mode, and wait for you to adjust the fit. You will see a plot spanning all the columns of the image, but only a few rows near where the profile peaks were found. A sample display of the first trace for a bright continuum spectrum is shown in figure 3-7 of example 3.2 below. Small crosses mark each of the peak locations, and larger crosses represent the averaged points used in the fit. A dotted line represents the function fitted to the averaged points. Tracing the spectrum across the entire detector is difficult with EUVE data, as most emission spectra have gaps, and many continuum spectra cut off in midbandpass. It may even be possible to see features in a display of the spectral image which aptrace will fail to find. As a quick diagnostic, note the scale of the Y-axis: if it spans more than 10 pixels or so, the trace has probably wandered off-course, as will also be visible from the fit line. If the Y-scale spans only 1–2 pixels, you are off to a good start. You are using icfit, the same subtask used for the background fitting, with the same commands: :vshow to display fit parameters in the text window, and ? to display the available key and colon commands. You may wish to adjust the sample region, naverage, order, or grow, among others, to change the fit, using f each time to recalculate. Aim for a peak function that shifts by no more than 1-2 pixels from left to right on the display. We offer the following (EUVE specific) suggestions:

• In regions where no peak is found, you must add some points yourself to guide aptrace. Use the key command a, which also allows you to assign a weight to the new point. The commands x and y can be used to adjust point locations. At least one or two artificial fit points may be needed to make the trace behave reasonably at the ends.

• If there are profiles or points with large deviations at the ends of the fit, the sampling region may include bright rim areas, and should be shortened.

• Increasing the number of profiles combined with naverage can smooth out noise in the profile locations. Setting a small step with a large naverage can help spread the influence of bright parts of the spectrum.

• Single points whose profiles vary greatly from the average can be eliminated by moving the cursor to that point and typing a “d.” Points where the peak-finding algorithm has found the bright detector rim or a stim pin (check the locations) are obvious candidates. Deleting individual profiles (small crosses) forces recomputation of the averages used as the fit points the next you type f.

• One can also vary the parameters width, radius, and threshold in order to prevent the trace from wandering off of the spectrum. Values of 10, 2, and 0 are good for a start.

5/97

To further explore the intricacies of icfit, type ? to display a full list of the available commands. Always use q to return from the help listing to interactive mode (it may require two keystrokes) before typing more commands. Please read the icfit help page in IRAF for a full description of how the remaining task parameters affect the fit.

4. Extract the spectrum. When satisfied with the trace, once again type q to leave aptrace. apall will ask you if you want to save the aperture and trace in a database directory; if you want to continue, type “y”. You will then be queried whether if you want to extract the spectrum and review the results (the display will not change until you answer). If you answer “yes,” apsum is run with the EXTRACTION parameters (figure 3-5, d). After another query, you can look at the spectrum that has been extracted, to decide whether to save it in the output image with the background in the second band. apall has now done its job; the task exits when you type “q” in the spectrum display.

apall and the apextract package have a number of other capabilities that have not been introduced here. Read the apextract task help pages and IRAF User’s Guides if you wish to try variations on any of the steps.

Example 3.2 : Extracting a continuum spectrum with apall

This example illustrates the extraction of a long-wavelength spectrum of HZ43, as set up in our sample parameter file. The GO has remade the two-dimensional spectral image with a user-defined time filter. A section of the [2048×2048] image is used for extraction, to eliminate stim pins and most of the detector rim. After editing the parameters as shown in figure 3-5, she types:

          ap> apall sw hz43[*,720:1299]

The graphics display shows summed profile of 100 columns centered about column 745. The GO makes an aperture with an m command, and uses :upper and :lower to widen it to ±8 rows. The resulting display is shown in figure 3-6.

To continue, the GO types b to enter background parameter setting mode, in icfit. The display is not wide enough to include the entire background region selected, but using vshow to display the fit parameters gives:

        Wed 18:03:08 29-Mar-95
        Set Background Subtraction for Aperture 1
        function = spline1
        grow = 3.
        naverage = 20
        order = 1
        low_reject = 3.
        high_reject = 3.
        niterate = 3
        sample = -114:-15,15:114

5/97

Figure 3-6: Display in apedit with a user-defined aperture.

        total points = 401
        sample points = 10
        nrejected = 0
        deleted = 0
        RMS =   1.089518
        square root of reduced chi square =   5.425122
                  coefficient      error
                    2.839199E1     1.352863E-1
                    2.695773E1     1.354585E-1

The number of sample points is 10 = (100 + 100) /20 as expected. Redoing the fit or using the graph keys hijkl shifts to the icfit display, which shows all of the sample area. The GO uses a few Z and M commands to zoom in on a section of the fit, decides that it looks adequate, and types q to return to apedit and the aperture display. Another q elicits the prompt lines:

           Trace apertures for hz43 sw?  (yes):
           Fit traced positions for hz43 sw interactively?  (yes):

The GO types a <cr> (default affirmative) for each query to enter aptrace, and the display blinks and returns with yet another query. The GO, now mildly irritated, again types a <cr> and the display returns, showing the initial spectrum

5/97

trace (figure 3-7).

Figure 3-7: Spectrum trace of a bright continuum displayed in aptrace. Sample region is indicated by horizontal bar, and small crosses are individual spectrum profile peaks. Large crosses are averaged points used in the fit.

The initial results are fairly acceptable. The trace shifts by only about one pixel from one end of he image to the other; less than a pixel over the length of the spectrum. Note the long horizontal hash marks, which show the sample area where the profiles are averaged to form one fit point. The GO tries several methods of improving the fit. Increasing naverage to 20 averages out some of the oscillations in the profile locations. Redefining the sample region to exclude points at the long end and add some at the short end also reduces the RMS a bit. After plotting the residuals using the j key command, the GO tries deleting all the profiles with residuals of |r |> .5 (figure 3-8), and finds that since only two are within the sample region, this makes very little difference. Since there is no obvious reason why these profiles might not be valid, the GO undeletes them (u), and reruns the fitting routine one more time to return to the previous fit. This seems quite good enough, so the GO enters a q to move on.

The GO types <cr> to the next queries, writing the aperture and trace to a new database directory file. The contents of database/aphz43 sw, are shown below:

        # Wed 18:49:47 29-Mar-95
        begin   aperture hz43_sw 1 745. 202.6001
                image   hz43_sw
                aperture        1
                beam    1

5/97 Figure 3-8: Plot of fit point residuals in aptrace. ×’s show where profiles were deleted from the fit.

center  745. 202.6001
low     -744. -8.
high    1303. 8.
background
xmin -114.
xmax 114.
function spline1
order 1
sample -114:-15,15:114 41.12367:46.12561
naverage 20
niterate 3
low_reject 3.
high_reject 3.
grow 3.
axis    2
curve   6
2.
2.
1.
2048.
0.2402294
0.3580123

The spectrum then is extracted, after several more queries, and the result is

5/97

displayed, before being written to the output image. The peaks at the ends are normal; as the detector is round, the lines of most background regions don’t extend as far as those of the spectrum aperture, causing the under-subtractions at both ends. Also, the particularly bright rim at the short wavelength end of the SW detector is fairly broad in X, and often produces some background over-subtractions in this area. The spectrum is written to the multispec format output image hz43 swsp, with the background in the second image band. Both the spectrum and background are plotted in figure 3-9.

Figure 3-9: SW spectrum for HZ43, extracted with apall. The bottom curve is the averaged background subtracted. The spectrum plot is offset by 60 counts.

3.1.3 MKPROFILE fits profiles at specified intervals

The tool euvtools.mkprofile is written to create spatial profiles of spectra for use in performing optimal extraction. A sample parameter file is shown in figure 3-10.

The user can specify either the sample section and size of profile sections or the number of profiles to make. The specified number of dispersion lines are summed, a background level is subtracted, and each profile is normalized over a specified FWHM. Background fitting can be done interactively. The profiles are output as lines of a two-dimensional image.

5/97

Spectral Corrections and Flux Calibration 3-19

          eu> lpar mkprofile

        input = "sw"            input image
       output = "prof_sw"       output image
          (x1 = 350)            starting column of profiles
          (x2 = 1500)           ending column of profiles
          (nx = INDEF)          number of columns per profile
   (nprofiles = 10)             number of profiles
    (aperture = "35:535")       profile aperture
    (function = "chebyshev")    Fitting function
       (order = 2)              Number of polynomial terms or spline pieces
      (sample = "*")            Sample points to fit
    (naverage = 3)              Number of sample points to combine
    (niterate = 4)              Number of rejection iterations
  (low_reject = 2.5)            Low sigma rejection factor
 (high_reject = 2.5)            High sigma rejection factor
        (grow = 0.)             Rejection growing radius
        (trim = yes)            Trim low S/N regions of profile?
   (threshold = 0.02)           Profile trim threshold
       (nfwhm = 4)              Number of fwhm's to use in normalization
 (interactive = yes)            Interactive display?
    (graphics = "stdgraph")     Device for display
      (cursor = "")             Cursor
        (mode = "al")

Figure 3-10: Parameter file for mkprofile.

3.2 Spectral Corrections and Flux Calibration

This section covers the use of tasks for processing one-dimensional extracted spectra. The EUVE spectra output by euvextract, as well as extractions done with apall are binned counts vs. wavelength. The tasks in this section can be used to divide counts by effective area to create flux spectra, and to subtract counts from so-called “higher” inside spectral orders (n < −1) from the appropriate bins. In addition to the IRAF/EUV tasks described here, there are spectral analysis tasks in noao.onedspec which may be used on EUVE spectra as well. See Appendix D for a listing.

All examples of reference data path names in this section assume that the reference data is located in a directory called reference, located immediately below the current working directory. This is in keeping with the standard directory setup illustrated in the Data Products Guide, and provided for GO’s working at the EGO Center. When using any task which requires access to any reference data file, make sure the path to be used file is complete and accurate for your system. Newer versions of the reference data set can be obtained as needed from the CEA anonymous ftp site, in the directory /pub/reference. See section D.2 in Appendix D for the ftp site address.

5/97

3.2.1 DIVEA divides by effective area

The task euvtools.divea provides the basic functionality for converting first-order EUVE spectra in counts to flux units. It performs a straightforward division by the first-order effective area and the exposure time, and rebins the spectrum at a user-specified resolution. A sample parameter file is shown in figure 3-11.

          eu> lpar divea
        input = "sw_spec"     Input spectrum
       output = "sw_flux"     Output spectrum
          (dx = 0.1)            Bin width in Angstroms
     (exptime = 1.)             Exposure time
     (refdata = "reference/detector") Reference file
    (detector = "sw")           Detector name [sw|mw|lw]
        (mode = "al")

Figure 3-11: Parameter file for fluxing a short wavelength spectrum with divea.

Notes on running divea

□ The value of dx determines the binning of the output spectrum. Setting dx to “INDEF” suppresses rebinning of the spectrum. Otherwise, the spectrum, and spectrum error band if one exists, will both be rebinned.

□ When the input is rebinned, the error band will be divided by exposure time and effective area, and corrected using standard error propagation. The correction does not include the cross-correlation terms introduced in the spectrum by rebinning, and the error band will no longer fully account for the standard deviation. Do any rebinning before extraction, and set dx to “INDEF” if preserving the validity of the error band is important.

□ The parameters exptime and detector are used only if the task fails to find this information in the image header. These header cards should be present in any spectrum extracted from an EUVE two-dimensional spectral image.

□ divea needs the reference data table detector.tab, so the parameter reference must contain the complete path, including the filename, for your directory setup.

GO’s whose data do not contain appreciable contributions from higher spectral orders can use divea from the command line to convert their extracted spectra to flux units without going through the order-subtraction process in suborder, for example by typing:

          eu> divea lw spec lw spdiv dx=.2

If the header card INSTRUME is not found for some reason, and the value of detector does not match that of the input spectrum (recorded or not), the task attempts to do the division for any part of the spectral bandpasses that overlap. The input

5/97

spectrum is divided by the effective area of the detector, or if the error involves the LW and SW detectors, the task fails with the error message: “ divea: Current parameters yield no pixels.” Neither of these outputs is especially useful, but they may demonstrate to the user that something is amiss.

3.2.2 SUBORDER removes higher order counts

Because of the fact that each channel spans a wavelength range greater than 2×λ_{min}, the Spectrometer has little or no free spectral range. Light from wavelengths shorter than λ_{min} in each channel is collected by the mirror, and may contribute to spectra in inside orders other than the principal order, n = −1. Contamination from higher order light is sometimes a significant problem in continuum spectra, where higher order counts form an underlying continuum, rather than discrete features.

The task euvtools.suborder uses the EUVE spectra and reference data for higherorder effective areas to calculate the number of counts due to all possible higher orders in each bin. The higher order contributions to shorter wavelengths are calculated first. They are then subtracted from the spectrum, and the subtracted part of the spectrum is used to find the higher order intensities at longer wavelengths with a “bootstrapping” technique.

suborder uses the exposure time in the card EXPTIME and first-order effective area^{3} to convert the subtracted count spectrum to flux units before creating the output images. The end results are corrected spectra for each of the spectrometer channels input, which can be written to equispec type spectral images in flux units (photons/cm^{2}/sec/Å). suborder does not provide any facility to merge overlapping parts of the spectra, or to join the spectra from the different channels^{4}.

5/97 Will running suborder improve my spectrum?

3The effective areas for the first two inside orders in each spectrometer are plotted in the EUVE Guest Observer Program Handbook. The data for all measured orders are available from the CEA ftp site.

4suborder replaces part of the functionality of the task euvcombine, which is now in the euv.obsolete package.

When used with version 1.9 or later of the EUVE reference data on smooth continuum spectra, suborder can produce relatively good order-subtracted spectra with reasonable agreement at the join points. However there are still some inconsistencies in the effective area functions in regions where the bandpasses overlap. In contrast, suborder does NOT work well near emission features. Line profiles in the higher-order spectra are not well characterized, and so suborder effectively constructs scaled versions of first order features. These often do not match the shapes predicted by ray tracing the higher orders, and may be asymmetrical. When higher orders calculated near such features are subtracted, spurious features can be generated at the wings. The same problem arises in attempts to subtract higher orders from continuum spectra with absorption features. It is usually only worthwhile to run suborder on spectra from bright sources with fairly smooth continua; they are the most affected by higher order light, and are corrected the most successfully.

A sample parameter set for suborder is shown in figure 3-12. Higher order counts will be subtraced from three EUVE spectra of the white dwarf GD153.

suborder can be run interactively, allowing the user to view each higher-order contribution as it is calculated. After viewing the display, the user can vary many of the task parameters before any subtractions are made. Each order is then calculated in turn, allowing the user to elect whether to perform each subtraction. Refer to the IRAF help page for complete information on the parameters and how higher-order contributions are calculated.

Notes on running suborder

□ When operating on spectra such as those made by euvextract the task looks for a BANDID card in the header. If the spectrum band is not identified in some header card, band 1 is assumed.

□ The spectra from all three channels should be included as input, unless the user only wants a subtracted spectrum for the SW channel. The task will not abort if only the medium and long spectra are input, but counts in both the short and medium wavelength channels should be available to predict any higher order counts in the longer channels.

□ The parameter putout is used to determine whether the results of the task should be saved in one-dimensional spectral images. If putout is “no”, the task will exit without writing, and the subtracted spectra are discarded. This feature saves cleaning up after trial runs.

□ The bin widths used to rebin the spectra are given in Å units, and can be smaller than the existing spectral bins.

□ All wavelength values must be given in Å. Users may wish to plot each spectrum in splot first to find the desired limits.

5/97

      eu> lpar suborder

   sw_in = "sw_spec"       sw input spectrum
   mw_in = "mw_spec"       mw input spectrum
   lw_in = "lw_spec"       lw input spectrum
  sw_out = "GD153_swsub"   sw output spectrum
  mw_out = "GD153_mwsub"   mw output spectrum
  lw_out = "GD153_lwsub"   lw output spectrum
 (putout = yes)            out put results
(sw_delta = 0.1)            sw delta
(mw_delta = 0.1)            mw delta
(lw_delta = 0.1)            lw delta
 (sw_low = 75.)            sw low range
(sw_high = 180.)           sw high range
 (mw_low = 145.)           mw low range
(mw_high = 365.)           mw high range
 (lw_low = 300.)           lw low range
(lw_high = 735.)           lw high range
  (sw_mw = 160.)           sw mw join point
  (mw_lw = 340.)           mw lw join point
(refdata = "reference/detector") reference file
(exptime = "exptime")      exposer time imheader
(graphics = "stdgraph")     graphics
   (mode = "ql")

Figure 3-12: Parameter file for starting the task suborder in interactive mode.

□ Set lw high or sw low inside the spectrometer bandpass limits, as shown in figure 3-12, rather than use the default values. This is necessary because including bins from too near the detector edges, which may be under- or over-subtracted, can produce pathological results. The output spectra will then have unsubtracted portions at each end which can be trimmed or masked off.

□ The join points sw mw and mw lw are used to specify which spectrum is used to calculate higher order fluxes in regions where the bandpasses overlap. The join points should also be well into the longer wavelength spectral range, and inside the upper limit of the shorter range, to ensure that no over-bright or over-subtracted edges are used for order calculations.

Starting suborder

When started, suborder first rebins the input spectra, and divides the counts in each bin by the bin width specified in sw delt, mw delt, or lw delt. The initial display shows all spectra plotted as counts/Å vs. Å. All calculations are carried out on the binned fluxes, and the results are converted to counts/Å for display. Figure 3-13 shows the initial display for a set of GD153 spectra.

5/97

Figure 3-13: Start-up display of GD153 spectra in suborder, showing input spectra and the 2nd order contribution from light at 75–150 Å to the short wavelength spectrum.

The task starts by assuming that the spectrum from sw low to [sw low × 2] is free of any higher-order contributions. Although the short wavelength (SW) spectrometer can produce orders -2, -3, etc. for wavelengths shortward of 75 Å, their contribution is assumed to be negligible because the throughput of the Lexan filter at these wavelengths is very low. Therefore, if the spectrum limits are as in figure 3-12, the 2nd order contribution to SW from SW begins at 2 × 75 = 150 Å.

IRAF shifts into interactive cursor mode when the task starts. The command line shifts to the graphics screen, and the task waits for the first command. Any adjustments to the spectrum limits or the join points must be made at this point, before the first subtraction. To change the spectrum limits, place the cursor at the desired lower limit within the spectrum you want to change, and type the command z. The task will record the X-position as the new lower limit and you will be prompted to enter a new upper limit by repeating the process. To change the join points, position the cursor in the same way and type the command j. The 2nd order contribution of SW to SW will be recalculated with each change of parameters.

Running in interactive mode

Once in interactive mode, the user views each calculated order overplotted on the appropriate spectrum, and elects to subtract the contribution, or to move on to the next calculation without subtracting, or to undo the last subtraction. Order contri-

5/97

butions are subtracted by giving the :subtract command after each new calculation is displayed. Subtraction of any single order can be bypassed in interactive mode by typing the c command (for “continue”). The next contribution is then calculated immediately from the current versions of the subtracted spectra. The u command undoes the last subtraction, returning the spectrum its former state.

The task proceeds from short to long wavelengths, calculating and applying each successive order for each channel. After the initial subtraction of 2nd order from SW, there should be no more higher orders in this channel, as 3 × sw low = 225 is outside thw SW bandpass. Contributions of SW light to second order in the medium wavelength (MW) are then calculated from the entire spectrum between sw low and sw mw, the first join point.

The process continues, calculating all higher orders in each Spectrometer channel from the shorter ones until some order misses the bandpass entirely. All contributions from wavelengths longer than sw mw and shorter than mw lw are calculated from the MW spectrum. Contributions from wavelengths longer than mw lw are calculated from the long wavelength (LW) spectrum. All subtractions are performed on the entire spectrum between the low and high limits, including regions between join points and limits, like that between sw mw and sw high.

When all possible orders have been computed and subtracted, the final subtracted spectra are written out in flux units, obtained by dividing the subtracted counts in the display by exptime and the first order effective area. The user can command the task to complete the sequence without further reviews (or choices) at any time by typing :auto. One can quit from the task at any time by typing q. Spectra are recorded if putout is “yes”.

Manipulating the display

The following commands can be given at any time, to change some aspect of the display. They involve no subtractions or order calculations. Complete definitions of the commands can be displayed in the text window by typing a ?.

• To view one spectrum at a time, position the cursor over the spectrum you want and type l.

• The a command restores all the spectra to the display.

• After any single subtraction, the s command will overplot the spectrum before the last subtraction on the subtracted result, allowing the user to view the effects of the subtraction. Type s again to remove the overplot from the display. The s command is actually a toggle which is always turned “off” (unsubtracted spectrum not displayed) when suborder is started.

• When the difference between subtracted and unsubtracted spectra is small, it may be necessary to enlarge part of the plot with the standard IRAF window commands to see the difference. Typing w(indow) a will restore the plot under the cursor to its default range and scale. The Y axis will be autoscaled when subtract or undo are issued, but other window commands remain in effect.

5/97

• The d and f commands are also toggles; they turn the display of the current subtracted spectrum and the next higher-order contribution off and on, respectively. They are both “on” at startup.

The user should always be aware of the status of the display and the task algorithm, to avoid being confused about the results of the subtractions. For example, a combination of d and s make it possible to create a display in which the version of the spectrum before the last subtraction is plotted along with the next higher-order contribution to be subtracted. Then the next :subtract command produces a spectrum showing the affects of the last two subtractions.

If you do become confused, using the undo (u) and various window commands to back up and compare the subtracted and unsubtracted states may be helpful.

Example 3.3 : Subtracting higher-order contributions with suborder

The spectra from an observation of hot DA white dwarf GD153 are to have higher order contributions subtracted. The GO sets up the parameter file as in figure 3-12, and types :go in epar to start the task. After viewing the initial display of the second order contribution to the short wavelength spectrum shown back in figure 3-13, the GO types :subtract to perform subtraction of second order contributions of SW light from the SW spectrum.

The SW spectrum has no third order contributions from wavelengths with λ > 75 Å, as 75×3 > sw high, and shorter wavelengths were assumed to be negligible. So the next contribution calculated is the second order contribution from SW light to the MW spectrum. The contribution is quite small, but the GO subtracts this order also, as the subtraction adds relatively little noise and the compounded effects of several orders may be significant. The same is repeated for the third and fourth orders of SW in MW, and then the second order MW light in the MW spectrum, which begins at 2 × sw mw = 320 Å.

After this is subtracted, the next contribution is from second order SW light to the LW spectrum. The GO now displays only the LW plot by using the l command. This eliminates waiting for the (now static) SW and MW spectra to be redrawn after each order. The second order contribution of SW in LW is small, and all at wavelengths lower than the join point mw lw, so the subtracted bins would not affect later order calculations. The GO decides to skip over the subtraction by typing c (for “continue”).

The third and fourth order contributions of SW light in LW are very small. After subtracting the fourth order, the GO tries to assess the difference in the spectrum. He types s to overplot the previous spectrum, then a series of window commands enlarge a small region of the spectrum. There is no detectible difference in the spectrum, so it does not seem that the fourth order (or higher orders) is significant. The GO types w a to return to the autoscaled axes, and skips over all of the rest of the SW orders (n = 5,6,7, and 8) in LW using c. The next significant contribution is the second order of MW in the LW spectrum, shown in figure 3-14.

5/97 Figure 3-14: Second order MW in LW from suborder. The vertical line at 340 Å is the join point between the MW and LW spectra. The spectrum is seen before subtraction of the -2nd order counts.

Near the mw lw join point, it is about 1 count in 6, and so its subtraction makes a visible difference in the spectrum height and shape. Figure 3-15 shows a close-up view of the subtracted result overplotted with the spectrum before subtraction. The plot was made using the following commands:

cursor position command  effect
(any,any)  s  overplots spectrum before subtraction
(550,350)  w j  sets left edge of window
(560,363)  w k  sets right edge of window
(any,450)  w b  sets bottom edge of window
(any,790)  w t  sets top edge of window
(any,any)  :.snap sends screen dump to printer

One can see that the shape of the spectrum is preserved after the subtraction– very little additional noise is added.

The third order of MW in LW is also significant, but does not contribute until 480 Å, or 3×sw mw. The fourth order results in some increased over-subtraction between 600 and 700 Å. Subtracting the higher orders from LW itself polishes off most of the signal longward of about 640 Å. The final spectra are divided by the first order effective areas before being written to the output images. They are shown in figure 3-16, after smoothing with a boxcar of ≈ Δλ.

5/97

Figure 3-15: Enlarged section of LW spectrum. Top line is the spectrum before subtraction of second order MW light; bottom is after. suborder is ready to subtract 3rd order.

Although there has been some increase in the noise levels, especially near the ends, the spectra are now cleaned of higher order contamination, and fluxed.

3.2.3 EUVHCLEAN strips obsolete header cards from spectra

The task euvtools.euvhclean is supplied to remove obsolete header cards which are commonly written into EUVE spectra when they are created. The task has only one input parameter: the name of the image to be cleaned. It has a predefined list of header keywords, and removes cards matching a certain template. euvhclean is normally used from the command line:

          eu> euvhcl GD153 swsp

Cards removed include the macros inherited from parent QPOE files. These macros occupy a large block of space in the header file and are no longer useful once the QPOE has been binned into an image. Other cards removed are also inherited from the parent QPOE file, and their functions may be obsolete or redundant with other header cards in the image. See the euvhclean help page for a full description of the keyword templates applied.

5/97 Figure 3-16: Final output from suborder. The subtracted spectra have units of photons/cm^{2}/sec/Å. The spectra have been boxcar smoothed by about 1 resolution element, and are offset in intensity.

3.2.4 SPECPRINT prints spectrum and error band

The task euvtools.specprint is a tool for creating ASCII listings of extracted spectra. A spectral image with multiple bands is input, the spectrum and error band numbers are given through parameters, and the task writes the wavelength of each bin, the spectrum counts, and the subtraction errors to STDOUT. Figure 3-17 shows a sample parameter file for printing one of the EGO Center extracted spectra.

          eu> lpar specprint

        input = "lw_spec"       Input image file
    (specband = 1)              Image band with spectrum
     (errband = 2)              Image band with spectrum errors
      (format = "%13.3f %g")    Output formats
        (mode = "al")

Figure 3-17: Parameter file for specprint

5/97

3-30 CHAPTER 3. EXTRACTING AND FLUXING EUVE SPECTRA

The output from specprint can be easily redirected to a file:

          eu> specprint lw spec > lwspec.txt

The default values of the parameters specband and errband are correct for the extracted spectrum and error bands of spectra produced with euvextract. The parameter format may also be used to specify the format of the output:

          eu> specprint mw spec specb=1 errb=2 format=%9.3f,%g,%g > mwspec.txt

specprint should not be run with either specband or errband set to “INDEF”.

5/97

SIMULATIONS AND MODEL COMPARISONS

Many GO’s will want to compare their spectra to models of the target spectrum. To make the two comparable, either the EUVE spectra must be flux calibrated, or else the instrument signature of the Spectrometer must be added to the model spectrum. Many EUVE spectra have relatively low signal to noise, even after the spectrum has been re-extracted, so that flux calibration is difficult. Comparisons may be done in these cases by converting the model spectrum to a set of three simulated EUVE channel spectra in counts/second or counts for a comparable exposure time.

4.1 Making Simulated EUVE Spectra

This section gives instructions and examples on creating simulated one-dimensional EUVE spectra from spectral models. Typically, the models consist of output files from a modeling program that simulates the flux from the target. However, the fluxes used in simulations should be estimates of flux at the telescope, including the effects of interstellar absorption along the line of sight.

4.1.1 ISM calculates transmission of the interstellar medium

To create simulated EUVE spectra from a theoretical model, one must determine the spectrum resulting from the model at the telescope. The task euvspec.ism is a tool for calculating the effect of absorption by the interstellar medium on spectra produced by modeling programs. The task currently supports a single ISM model, presented in the paper by Rumph, et. al^{1}. The code also accounts for metals as described in the R. Morrison and D. MacCammon paper (1983, ApJ, 270, 119). The code uses the best available photoionization cross sections from the atomic data literature and combines them in an arbitrary mixture of HI, HeI, and HeII.

A typical parameter file for ism is shown in figure 4-1. The input parameters are the column density of neutral hydrogen in the direction of the target, in cm^{−2}, the ratios of HeI/HI, and HeII/HI, and a two-column list with wavelengths in Å in the first column and fluxes in the second, for which to compute the local flux. The fluxes may be expressed in any units. Each flux value is multiplied by a dimensionless transmission for that wavelength. The value of the parameter model should not be

5/97

1Interstellar Medium Continuum, Autoionization, and Line Absorption in the Extreme Ultraviolet, T. Rumph, S.

Bowyer, and S. Vennes, AJ, 107(6), 2108–2114, 1994

4-2 CHAPTER 4. SIMULATIONS AND MODEL COMPARISONS

changed, as only the Rumph et. al. model is currently supported. Figure 4-2 shows a model of a hot DA white dwarf spectrum before and after processing by ism.

          eu> lpar ism

      hcolumn = 1.0000000000000E18 neutral hydrogen column density (cm**-2)
   (he1hratio = 0.1)            HeI/HI ratio
   (he2hratio = 0.)             HeII/HI ratio
       (model = "rumph")        ism model
       (input = "hotDA")        Name of input file (or stdin)
      (output = "hotDA.ism")    Name of output file (or stdout)
    (document = yes)            output comment lines that document parameters
        (mode = "al")

Figure 4-1: Sample parameters for calculating the transmission of flux from a model white dwarf spectrum.

Figure 4-2: Effect of the task ims on a hot DA white dwarf model. The calculation assumes no HeII, and HeI/HI of .1. The offset between the spectra is entirely the result of estimated absorption.

Spectra created with ism are suitable for input to specmod.

5/97

Making Simulated EUVE Spectra 4-3

4.1.2 SPECMOD makes simulated EUVE spectra

The task euvspec.specmod simulates the effect of the EUVE Spectrometer on model spectra, producing a spectrum of binned integrated counts from a model spectrum of flux at the telescope. Input model spectra must have columns of wavelengths and fluxes, and can be ASCII text files, IRAF images, or ST tables. Input may be either a list of emission lines, or a continuum whose values are sampled or integrated bins. The user defines the integration time, the binning of the output images, and can choose whether to convolve the spectra with a point spread function or the spectrometer collimator response, where appropriate.

The input model may be a file or image of: a) the centers and integrated fluxes of a number of narrow emission lines, or values of b) photons/cm^{2}/sec or c) photons/cm^{2}/sec/Å for any number of wavelengths. Fluxes input as ASCII files or ST tables need not be regularly spaced; specmod will perform a linear interpolation where necessary to create the specified output binning. The output from specmod is a one-dimensional spectrum for one or more Spectrometer channels, each with a linear WCS determined by the input values of the binning parameters in a detector pset. The output spectrum has units of counts integrated over the specified exposure time.

A sample parameter setup is shown in figure 4-3.

          eu> lpar specmod

       (model = "W7")           Input model
     (wavecol = "")             Input model wavelength column
     (fluxcol = "")             Input model flux column
      (output = "wd7_spn")      Basename of output files
    (outtable = no)             Produce ST table as output?
     (detpars = "detsw.par,detmw.par,detlw.par") List of detector psets
     (refdata = "reference/detector") Reference dataset
    (linelist = no)             Is input model a linelist?
    (linefwhm = 1.0000000000000E-5) FWHM of lines in linelist (angstroms)
     (sampled = yes)            Is input model sampled?
    (convolve = yes)            Apply convolution?
         (psf = yes)            Use point spread function for convolution?
      (nsigma = 4.)             Number of sigma for convolution gaussian
    (addnoise = no)             Add poisson noise?
        (seed = 2)              Seed for random number generator
        (mode = "al")

Figure 4-3: Specmod parameters for making simulated EUVE spectra from an ASCII line list.

Notes on running specmod

□ specmod reads the parameter linelist to determine whether the input is a line list, with wavelengths vs. integrated photons/cm^{2}/sec for emission lines of finite width, or a continuum in photons/cm^{2}/sec or photons/cm^{2}/sec/Å.

5/97

□ If linelist is “yes,” specmod uses the parameter linefwhm as the intrinsic line width. This parameter should not be changed from its default value (<< F W HMofP SF) unless the user knows that the lines are unusually broad. Very broad and/or blended lines should have fluxes specified at intervals of .05 Å or less.

□ IRAF images can be used only for the two varieties of continuum input, and must have a WCS relating the pixel index to wavelength. Sampled continua are assumed to be in photons/cm^{2}/sec/Å and unsampled, or histogram images are assumed to be photons/cm^{2}/sec.

□ The output may be stored in an ST table, as opposed to an image, by setting the boolean parameter outtable.

□ If a continuum is input, the user must specify whether it is a sampled piecewise linear function (PLF), with pixel values in photons/cm^{2}/sec/Å or a binned histogram, in photons/cm^{2}/sec, using the boolean parameter sampled.

□ Values for wavecol and fluxcol need not be supplied for ASCII files or ST tables that have wavelengths in the first column and fluxes in the second. For ASCII files with some other format, specify values of the column parameters using “c1,” “c2,” “c3,” etc. For ST tables, use the column names.

□ specmod produces an output spectrum in each channel for which it has a detector pset (short for ‘parameter set’) listed in the value of detpars. Each detector pset is a parameter file which is specific to one of the detectors, and must be created by the user before specmod can be run. The parameters that must be supplied by each pset are listed in the specmod help page. Information on creating detector psets is given in the help page, and below.

□ specmod uses the instrument reference data: wavelength solution, effective areas, and PSF. The default value of refdata, reference/detector.tab, should be correct for the standard EUVE directory setup shown in Chapter 0 of the EGO

Center Data Products Guide.

□ The parameter nsigma determines the width of the kernel (gaussian PSF or triangular collimator transmission) used in the convolution, in units of the standard deviation. It should be set to at least 4, so that the convolution is not cut off in the wings, but higher values can make specmod run slowly.

Creating detector psets

Each pset supplies specialized information for processing the simulated output of one detector, including parameters like the exposure time, and the desired output binning. A typical detector pset is shown in figure 4-4 below.

Sample detector psets are included with every copy of the euv package in the library

euvlib. The files are called swdetpars.par, mwdetpars.par, and lwdetpars.par, and are in the

location indicated by the environmental variable euvlib. They have the default values for each detector.

5/97

          eu> lpar detsw.par

    (detector = "sw")           Name of detector to process
     (exptime = 51964.9)        Exposure time (seconds)
     (binsize = 0.2)            Size of output bins (angstroms)
     (minwave = 70.)            Wavelength of first bin (angstroms)
     (maxwave = 190.)           Wavelength of last bin (angstroms)
       (nbins = INDEF)          Number of output bins
    (convstep = 0.1)            Wavelength step for convolution (angstroms)
    (backrate = 5.4000000000000E-4) Background rate (counts/sec/angstrom)
      (orders = "1")            List of spectrometer orders to include
   (outsuffix = "sw")           Suffix appended to output filename
        (mode = "al")

Figure 4-4: Example of a short wavelength detector pset for use with specmod. The filename —in this case detsw.par— must have the suffix .par.

Since each observation may require different parameter values, and specmod may be run several times for each, it is convenient to create separate psets for each EUVE observation. This can be done from the IRAF parameter file editor. Edit the short wavelength pset file to create your own SW pset by typing:

          eu> epar euvlib$swdetpars.par

Edit the file to insert the desired values. Make sure you enter the data in the required units. When finished, write the parameters to a unique filename by typing

           :w mysw.par

(insert an appropriate filename with the .par suffix). Then you can fetch and edit the MW file without leaving the editor:

           :e euvlib$mwdetpars.par

and the same for the LW Spectrometer.

Notes on creating psets

□ The value of exptime should match the value of the header card EXPTIME in the spectrum to which you wish to compare the simulation. Use imhead on your extracted EUVE spectra to determine the values.

□ Three of the four binning parameters, binsize, minwave, maxwave, and nbins, must have values to completely determine the output range and binsize.

□ The value of orders determines the number of spectral orders that are calculated, allowing the creation of simulated spectra comparable to the output

5/97

from suborder. The default files include all possible orders for each channel, producing spectra equivalent to the 1-d spectra supplied on the data tape. Of course which higher spectral orders can be calculated depends on the range of wavelength/flux data supplied.

□ The default value of convstep should generally not be changed from the values in the default psets. (.1 for SW, .2 for MW, .4 for LW).

Since specmod has no “command line” parameters, it can be run repeatedly on the same model without much typing^{2}. This facilitates making many simulations from the same model; one need change only the input or the detector/pset each time. Several sets of psets can be created and referenced with the “@filename” notation, along with a list of output image names. Most of the task parameters, and those from the relevant pset, are saved as header cards in the output image.

Example 4.1 : Creating spectra with specmod

The following command lines create simulated spectra with and without Poisson noise from a hot DA white dwarf model in a two-column ASCII file:

          eu> specmod model=hotDA.ism output=DAmod3 detpars=detsw.par,detmw.par,detlw.par
          linel- sampled+ addnoi-
          eu> specmod output=DAmod3noi addnoi+

The model spectrum is the same one produced by the example in section 4.1.1. The two simulated EUVE spectra for the LW detector are shown in figure 4-5. Header cards preserve a record of most of specmod’s parameters, as can be seen from the LW image header shown below:

hotdanoi.lw[481][real]:
    No bad pixels, no histogram, min=unknown, max=unknown
    Line storage mode, physdim [512], length of user area 1297 s.u.
    Created Tue 19:44:45 30-May-95, Last modified Tue 19:44:45 30-May-95
    Pixel file "HDR$hotdanoi.lw.pix" [ok]
    WCSDIM  =                    1
    CTYPE1  = 'LINEAR  '
    CRVAL1  =                 280.
    CRPIX1  =                   1.
    CDELT1  =                   1.
    CD1_1   =                   1.
    LTM1_1  =                   1.
    WAT0_001= 'system=world
    MODEL   = 'hotdamod.ism'
    WAVECOL = '        '
    FLUXCOL = '        '

2However, specmod will not overwrite files, as a rule, so unwanted output from previous runs must be deleted or moved to a new image name with imrename.

5/97

Statistical Comparisons Between Spectra 4-7

    OUTPUT  = 'hotdapoi.lw'
    OUTTABLE=                    F
    REFDATA = 'reference$detector'
    LINELIST=                    F
    LINEFWHM=  1.00000000000000E-5
    SAMPLED =                    T
    CONVOLVE=                    T
    PSF     =                    T
    NSIGMA  =                   4.
    ADDNOISE=                    T
    SEED    =                    2
    DETECTOR= 'lw      '
    EXPTIME =              50658.3
    BINSIZE =                   1.
    MINWAVE =                 280.
    MAXWAVE =                 760.
    NBINS   =                  481
    CONVSTEP=                  0.5
    BACKRATE=  1.70000000000000E-4
    ORDERS  = '1       '
    BANDID1 = 'spectrum - generated from model'

Other simulation tasks

Users may also wish to try the tasks in the package noao.artdata which can be used to construct artificial spectra or add noise to spectra. Keep in mind that the actual extracted spectra have noise properties which depend on the number of lines in the spectrum aperture, the size of the background sample region, and the method of background subtraction.

4.2 Statistical Comparisons Between Spectra

4.2.1 COMPSPEC makes statistical comparisons

The task euvtools.compspec^{3} compares two spectra in either counts or flux units, by performing pixel-by-pixel computation of the χ^{2} statistic and the associated probability that the two spectra sample the same parent distribution. The task can compare models to data, models to models, or compare observation spectra to each other, using a logical hierarchy which chooses an appropriate data variance for each type of calculation. Multiple models can also be compared to a single spectrum and

5/97

3Since the stsdas package also has a task called compspec. You may need to reload the euvtools package or type the package and task names, e.g. euvtools.compspec when summoning compspec, its parameter file, or help file.

Figure 4-5: Output spectra from specmod. Input model spectrum is shown in figure 4-2. The upper spectrum has had Poisson noise added, and has been offset by 8230 on the intensity scale.

vice versa, if one of the input arguments is a list or file of image names. A sample parameter file for comparing a single observation spectrum, euvspsm lw, to a list of model spectra is shown in figure 4-6.

          eu> lpar compspec

     spectra1 = "euspsm_lw"     one set of spectral 1d images
     spectra2 = "@models_lw"    comparison set of spectral 1d images
  (poiss1errs = yes)            are errors poisson for spectra1
  (poiss2errs = yes)            are errors poisson for spectra2
  (resolution = INDEF)          Resolution
          (x1 = 290.)           Minimum wavelength
          (x2 = 740.)           Maximum wavelength
 (constraints = 0)              No. of constraints
       (title = yes)            Print output title
        (mode = "al")

Figure 4-6: Sample compspec parameters for comparing an EUVE spectrum to model spectra.

5/97

Notes on running compspec

□ Images to be compared must be one-dimensional spectra or have 1-d spectra identified with a BANDID# card in the header.

□ A number of model spectra can be compared to a single observation by using a comma-separated list or the cl pointer notation @filename for the value of

     spectra1 or spectra2.

□ If two lists of spectra are to be compared, they must have the same number of entries.

□ The variance used in each χ^{2} computation is determined by the presence of a BANDID# = ...SIGMA... header card indicating an error band with σ values, the values of poiss1errs and poiss2errs, and the types of the images being compared: observation data or model, Please see the figure 4-7 below, and the compspec help page.

□ The values of x1 and x2 specify the upper and lower limits of the wavelength range to be compared, in Å.

□ For best results, the value of resolution should be “INDEF”, which forces a minimal rebinning to bring the input spectra to the same resolution. A value matching the observation spectrum may be specified if a model comparison spectrum has extremely small bins, but the χ^{2} test results are not as meaningful if the data are substantially rebinned.

How compspec determines the test data variance

As noted above, compspec will compute χ^{2} for different combinations of observation and model spectra. The χ^{2} computation assumes one of the inputs, the “test data,” is being tested against some other set of “comparison data”. It is assumed that the test data will usually be a model of the target spectrum, with an EUVE observation spectrum as the comparison data. When compspec receives a pair of spectra, it first determines what to use as the variance of each input, then which input of the pair is the model fit being tested. Only the variance of the fit being tested is used in the calculation of χ^{2}.

compspec first checks the header of each input image for a card of the form BANDID# = sigma. If the card is not found, Poisson errors are assumed. Figure 4-7 shows what is used for σ^{2} of each type of input spectrum. Whether the spectrum is considered a model or an observation depends on the value of the parameter poisserrsX. The symbol # stands for the band number of the error band, if present, and X is 1 or 2. Table 4-1 shows which spectrum’s variance is used in the computation of χ^{2}, based on the spectrum type, model or observation.

If model spectra are being input as, say, spectra2, the value of poiss2errs should be “no”. to force compspec to use each model variance in calculating χ^{2}. If either or both inputs are multiple spectra, the determination of which input is the “test data” is done each time compspec compares a new pair of spectra, although poiss1errs and poiss2errs have the same values for each pair.

5/97

4-10 CHAPTER 4. SIMULATIONS AND MODEL COMPARISONS Figure 4-7: Logic to determine variance (σ^{2}) of spectrum input to compspec. If no error band is identified in the header, Poisson errors are assumed. 5/97 Table 4-1: Which input variance is used in χ^{2}?

                    spectra1     spectra2  σ^{2} in χ^{2}

model spectrum obs. spectrum σ_{1}^{2}
obs. spectrum model spectrum σ_{2}^{2}
model spectrum model spectrum σ_{1}^{2}
obs. spectrum  obs. spectrum [σ_{1}^{2} + σ_{2}^{2}]

Example 4.2 : Compare specmod outputs with compspec

A nearly trivial use of compspec is to see how much difference the addition of Poisson noise makes in a spectrum produced by specmod. If we compare the simulated hot DA white dwarf spectra with and without noise to each other, assuming that the one without noise is a “model” while the other is not:

          eu> compspec hotda.lw hotdanoi.lw poiss1- poiss2+

The task assumes that spectra1 is fit to the data in spectra2. The output shows that compspec concurs with the fit:

spectrum1 spectrum2 statistic probability degfree
==================================================
hotda.lw hotdanoi.lw 460.2904968 0.7440797 481

It gives a probability of nearly 75% that the two are sampling the same parent distribution. This is hardly surprising, given the brightness of the spectrum; the Poisson noise added by specmod to the model was relatively small.

5/97

4-12 CHAPTER 4. SIMULATIONS AND MODEL COMPARISONS 5/97

DOING YOUR OWN DATA REDUCTION

This chapter is about redoing the basic data reduction step which corrects event positions for aspect and detector distortions, applies the wavelength mapping to Spectrometer data, and remaps events to sky positions in Deep Survey data. This is done by using the event pipeline task euvred.cep and the EUVE DS/S reference data to convert the time-ordered event list in table0.tab to one or more QPOE files.

There are two main reasons why GO’s will want to rerun cep. First, if there are problems with unpacking the FITS encoded QPOE files in your IRAF environment, running the pipeline on your own IRAF system will produce a set of locally compatible QPOE files. Second, as new knowledge about the operation of the instruments becomes available, some GO’s may profit by reprocessing their data with a new reference data set or revised version of the pipeline. The event pipeline is now capable of producing QPOE files for any or all of the DS/S detectors: short, medium, and long wavelength Spectrometers, and the Deep Survey.

New versions of the reference data are announced to all GO’s, and may be obtained or from the EGODATA directory in the CEA ftp site. Some versions of the reference data may require a newer version of the IRAF/EUV software than that supplied on the GO data tape. The software and installation instructions are available in the

ftp directory /pub/software/euv1.#.#.

If the reference data changes, do I also need new software?

New versions of the EUVE reference data do not automatically require a newer version of the IRAF/EUV software than that supplied on the GO data tape. The message announcing the new reference data version should also indicate which version(s) of the software can be used. The IRAF/EUV software and the compatible versions of the reference data will not in general have the same version number. If you are unsure what software version to use with a particular data set, it is usually best to use the latest available. The latest software version can always be obtained from the CEA/EUVE ftp site, in the directory /pub/software/euv1.#.#.

5.1 Running The EUVE Event Pipeline, CEP

The task cep (for comprehensive event pipeline) implements the steps of wavelength correction and aspect correction, and makes QPOE files with two-dimensional linear

5/97

5-2 CHAPTER 5. DOING YOUR OWN DATA REDUCTION

world coordinate systems from the event list in table0.tab. It processes by reading each event as a line from the table, processing the event, and writing to a separate QPOE file for each of the detectors. cep uses aspect reports from the telemetry table table1.tab and instrument data from a reference data set. Each data tape comes with a copy of the reference data used to produce the products on the tape.

5.1.1 Preparation

Most of the time required of the user in running cep goes into preparation. Once the task starts, although it may take quite a while, the user need only check its progress occasionally. This section contains instructions and checklists for getting ready to run the pipeline.

cep needs to know the names and locations of several tables and the reference directory it will use. Each of the pointers it uses to get this information has a default value. If used with the standard directory setup described in Chapter 0 of the EGO Center Data Products Guide, the default values are correct, with one minor exception (noted below). This is the easiest arrangement to work with. Confirm that each of the required items below is in place before running cep, and that all files have been converted from FITS format to their internal IRAF formats^{1}.

Preparing to run cep

□ The reference data should be in a subdirectory of the working directory called reference, and should include the file called reference. These data are placed in the proper directory when the tarfile of reference data is read from the data

tape with softools.rtar.

Make sure sufficient disk space is available. By default, cep will make a temporary file for each QPOE, in the working directory. Each file is the size of the finished QPOE, which can be 10 – 60 Mgb. All four DS/S files together roughly equal the size of table0.tab. So there must be at least twice this much space (for both at once) in the working directory. To use another location for the temporary files, search for lines in the reference file which have the form:

                TempDir ./

and change them to point to the desired location, e.g.:

                TempDir path.to/lotsoroom/

□ □ The file called pipeline should be moved from the reference directory, where it is normally placed by rtar, to the working directory where cep is to be run.

1See section 0.3 of the Data Products Guide for instructions on FITS conversion.

5/97

Running The EUVE Event Pipeline, CEP 5-3

□ The telemetry tables table0.tab, table1.tab, and table7.tab should be in a subdirectory of the working directory called observation. They are normally written in this location when read from the tarfile of observation/engineering files with rtar.

□ The file called observation should be read from tape into the observation directory.

□ The user should check the information in the table catalog.tab to make sure the target coordinates and units are correct for the current application.

□ cep always creates QPOE files called sw.qp, mw.qp, lw.qp, and ds.qp. If these files already exist, it will either write over them or fail at the very end of processing (depending on how IRAF clobber options are set). Be sure to remove or rename old QPOE files from the working directory before running cep.

Figure 5-1 shows a schematic of the directory structure for running cep, with the required tables and ASCII files. Use this figure as a map to check the file locations before running cep.

Figure 5-1: Schematic of directories and files for running the event pipeline cep. Existing versions of QPOE files should be moved from the working directory.

5/97

5.1.2 Processing scripts control cep

The parameter file for cep is shown in figure 5-2; it is deceptively simple. This complex task uses its two parameters as pointers to other ASCII files which contain descriptions and locations of the telemetry tables, the reference data and the subprograms cep uses at each stage of the reduction.

          eu> lpar cep

     database = "pipeline"      Processing text database name
     pipeline = "defpipe"       Pipeline record name
        (mode = "al")

Figure 5-2: Parameters for the task cep with default values.

The first parameter, database, points to the file called pipeline. The file contains a number of pipeline scripts which can be used to control processing by cep and produce different kinds of QPOE files. The scripts may be specific to a certain version of the reference data, so it is important always to use the version of the pipeline file that was bundled with the reference data you are using. If you have not already done so, move the file to the working directory where cep will be run. The second parameter, pipeline, indicates which of the scripts in the file is to be used. In our example, pipeline is set to defpipe, which is an alias for the default GO processing pipeline.

The default processing script

cep will start reading file pointed to by database (usually pipeline) and will expect its processing directions to start with a line containing the value of pipeline (usually “defpipe”). The listing below shows the pipeline script used to process observations at the EGO Center. Comments have been omitted, and users may wish to read the actual file for more detail. Most GO’s who run cep on their data will use this pipeline script. Notes on the other scripts in the pipeline file and their uses are given later in this section.

The three lines starting with begin defpipe under the banner, “Highest Level Indirection to Pipeline,” define defpipe as a shorter name for the script Spectrometer Wavelength Calibration Plots, which begins under the banner “Wavelength Calibration Image.” The first lines of the actual script tell cep where to find the observation data and reference data. The string RefPath/Reference must give the correct path to the reference file, which is a directory to of reference data files and processing commands. Likewise, the string ObsPath/Observation must be the path to the observation file, in the directory with the telemetry tables. These fields need not be changed if the user has created the standard directory and file setup shown in figure 5-1. To use some other location for the reference or observation directories and files, change

the values of RefPath, Reference, ObsPath, and Observation in pipeline.

5/97

One can see the general flow of the processing by looking over the rest of the script, under ProcessList, Events are read from the telemetry; W,S, and Z pulse-heights are converted to detector X,Y coordinates^{2}. Subpixel randomization is done to remove binning distortion effects (described in section 4.2.1 of the EGO Center Data Products Guide). After wavelength mapping and aspect correction, events are sorted using several Copy commands, and written to the QPOE files. cep translates each line of the ProcessList into lower-level processing commands using definitions in the reference file. These commands in turn point to the subroutines which are executed by cep. We will not describe pipeline scripts in any more detail here, though users can read section 4.2.1 of the EGO Center Data Products Guide, which describes the steps in the standard script.

######################################################################
#
#               Highest Level Indirection to Pipeline
#
######################################################################
begin   defpipe
        Type    Indirect
        Name    Spectrometer Wavelength Calibration Plots
######################################################################
#
#               Top Level Pipelines
#
######################################################################

#+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#
#                       Wavelength Calibration Image
#
#+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
begin   Spectrometer Wavelength Calibration Plots
        Type            Pipeline
        RefPath         reference
        Reference       reference
        ObsPath         observation
        Observation     observation
        ProcessList
                Read Spectrometer Photon Events
                Convert Spectrometer WSZ to XY
                Copy Spectrometer X to S
                Copy Spectrometer Y to Z

5/97

2Events are telemetered in WSZ format, starting on March 1, 1995. Prior observations were in XY format, and did not require the conversion.

                Randomize Spectrometer X SubPixel Position
                Randomize Spectrometer Y SubPixel Position
                Correct Spectrometer Wavelength
                Source Spectrometer Aspect
                Aspect Spectrometer Correction
                Correct Spectrometer Wavelength Aspect
                Correct Spectrometer Imaging Aspect
                Copy Spectrometer S to TYPE
                Copy Spectrometer Z to FLAT
                Copy Spectrometer X to W
                Copy Spectrometer Y to S
                Write Spectrometer Photon Events
                EOL

Target coordinates for aspect correction

The target position on the sky is needed to perform aspect correction. The position is specified in a table called catalog.tab. When the pipeline gets to the step Aspect Spectrometer Correction it looks up the table in the section of the pipeline file starting at the line begin SourcePosition, listed below. Under SourcePosition, the entry Catalog specifies the table name, which happens to be catalog. IRAF automatically supplies the .tab extension. cep is instructed to look in the first row (Column = ROW and Value = 1) and get the RA from the column “RA” and the Dec from the column “DEC”. The RA can be in either hours or degrees; cep looks at the units field of the table column to determine which is used.

######################################################################
#
#                       Source Position Record
#
######################################################################
begin   SourcePosition
        Type    Starcat
        Catalog catalog
        Column  ROW
        Value   1
        ColumnRA        RA
        ColumnDEC       DEC

A copy of the catalog table used in EGO Center processing is supplied on each data tape, in the fourth tarfile. Using tlcol on this copy of catalog.tab will show the columns, formats, and units that are required in the file, as well as the content seen with tread. An alternate name, target priority and HISTORY card are shown in

5/97

the header, and the values are printed after the column formats:

          tt> tlcol catalog

            # catalog.tab
            ID               CH*15       %-15s  ""
            RA               R          %15.7g  degrees
            DEC              R          %15.7g  degrees
            exposure         I            %11d  ""
            priority         CH*4         %-4s  ""
            name             CH*15       %-15s  ""
            HISTORY  t  Created Thu 17:21:36 08-Dec-94
                   WD1254+223        12.95064        22.03194              0.
                         0.      500000     1A             ALGOL

This listing can be used with tcreate to make a new catalog table, if one is needed.

Telemetry tables and the observation file

cep does not use all the telemetry tables in processing event data; only a few of the tables need to be transferred from the data tape to disk just to run the pipeline. The subset of tables required to process most observations is only table0.tab, the events, and table1.tab, which has the aspect quaternion components. GO’s whose data were telemetered in WSZ mode (observations made after March 1, 1995) will also need

table7.tab.

cep gets information about the format of the telemetry tables from the file observation, which is in the same directory with the tables. Users should not change this file; it is created by the program that builds the tables and is not meant to be modified.

Example 5.1 : Rerunning the pipeline with new reference data

We will describe a situation in which A new release of the EUVE reference data has been announced by the EGO Center, and this new data contains important changes which the GO can take advantage of by rerunning the pipeline. This is a straightforward repeat of the processing that creates the standard EUVE QPOE files.

The GO first obtains the new version of the reference data. In the target directory with the original QPOE files, she removes the old files from the reference directory (she still has a copy on the data tape). She starts ftp by typing:

           % ftp ftp.cea.berkeley.edu

5/97

After changing to binary mode, she uses the get command to copy the new tarfile

from /pub/reference/egodata1.# to her reference directory. She checks the README

file to confirm that she already has the correct version of the euv package installed. She then unpacks the tarfile in IRAF with rtar and converts the various FITS files to IRAF images or ST tables with the tables task fitsio.strfits.

The GO did not have her telemetry on disk, so she mounts the data tape and positions the tape drive at the beginning of the sixth tarfile, which contains the engineering telemetry tables. She transfers to the target directory, and reads the telemetry files table0.FITS and table1.FITS, and the file observation from the tape with rtar. rtar creates the directory observation and places the FITS files in it. The GO converts the telemetry tables to ST format with strfits.

The GO moves any existing QPOE files to the names filenamew old.qp, to prevent them from being overwritten. The old files are also backed up in ST tables. She moves the file pipeline from the reference directory to the target directory.

The GO checks the size of the file table0.tab in observation. cep will need roughly twice this much space in the target directory for the temporary files and finished QPOE files. The GO decides to use a system temporary storage space for the temporary files, so she must edit the reference file. She opens the file and goes to the section labeled “ Primitives for writing photon events,” and changes the lines TempDir ./ to /tmp1/user/target/ (with the terminating /).

Using tread, the GO examines the contents of the file catalog.tab, in the target directory, and makes sure that the values in the columns RA and DEC are the same as those recorded in the observation log.

At this point, she can run the default pipeline by simply typing:

          eu> cep pipeline defpipe

cep prints a message to the screen as it finishes reading each 10% of the telemetry events. Since cep processes each event as soon as it is read, one can estimate how long it will take to finish the processing by multiplying the time from startup to the first message by 10. It is often several hours for a typical observation. After all the events are read and processed, the message sorting... appears on the screen, as the QPOE files are sorted. When cep finishes, it writes the sorted QPOE files in the working directory with the usual names, and deletes the temporary files.

5.1.3 Other processing options with cep

There are now several other options for running the event pipeline, which produce different subsets of the QPOE files. These options can be applied by running cep with a different value for the parameter pipeline. Scripts for the first two options below are included in the pipeline file on all GO data tapes. The option for creating QPOE files with event pulse heights is included with version 1.5.1 of the software.

5/97 Single Spectrometer QPOE

It is possible to run cep with a script which makes a QPOE file for only one Spectrometer channel. This provides economy of both disk space and run time for GO’s who must rerun the pipeline, but know that they have interesting data in only one of the detectors. The scripts for creating single Spectrometer QPOE’s are called SW

Spectrometer Wavelength Image, MW Spectrometer Wavelength Image, and LW Spectrometer

Wavelength Image. Each script reads through all of table0.tab, but processes events from only one detector. Make a new QPOE file (after all required preparations are done) by typing:

          eu> cep pipeline 'SW Wavelength Calibration Image'

To make QPOE files for two of the three Spectrometer channels, it is necessary to run cep twice, so those who want two new QPOE files, and have sufficient disk space, will probably still want to run the standard pipeline, and discard the unwanted QPOE(s) after processing.

5.1.3.1 Remapped QPOE’s for imaging instruments

Scripts for making remapped images for all the EUVE imaging instruments, the three all-sky survey Scanners and the Deep Survey instrument, are also supplied in the standard pipeline file. At present, the Deep Survey is the only imaging instrument for which data are available in ST table format. Every GO event table (table0.tab) produced with version 1.5 or later of the IRAF/EUV software and 1.9 or later of the reference data includes the Deep Survey events. If it is necessary to rerun cep to create a new DS QPOE file, GO’s can use the script DS Remapped Image:

          eu> cep pipeline 'DS Remapped Image'

Pulse height QPOE files for WSZ observations

Starting March 1, 1995, the default mode of operation for EUVE is to telemeter event data in WSZ format, rather than converting the signals to detector X,Y locations on board the satellite. In WSZ mode each event is transmitted as the wedge, strip and zigzag pulses from the detector anode. This means that events are recorded by three pulse heights in table0.tab, not the two detector coordinates used for X,Y data. GO’s whose data was taken after 3/1/95, and processed with version 1.10 of the reference data should have this new version of table0.tab. Refer to the cover letter included with the tape, or use tlcol to look for the WSZ columns in table0. Telemetry tables for earlier observations do not contain the WSZ information for events. The format of the standard EUVE QPOE files has not changed; X,Y coordinates are now calculated in the course of processing by cep.

The change to recording events in WSZ mode is advantageous for observers, since pulse height information is preserved in the raw data, while no other information is

5/97

lost. Energetic particle events that have pulse heights outside the photon distribution can be removed later by filtering on event pulse height. Pulse height filtering is implemented by rerunning cep on the WSZ-format telemetry to create a QPOE file with pulse height as one of the event attributes. Once a pulse height QPOE file has been made, the event list can be sorted into an image using a QPOE/IO expression to filter on pulse heights and screen out unwanted events. This is expected to produce a substantial improvement in S/N for low-signal spectra in the short and medium wavelength Spectrometer channels. See the EUVE Electronic News, vol. 5, no. 2b, for test statistics^{3}.

All GO’s who receive telemetry in the WSZ format can make pulse height QPOE files with the software and reference data supplied on the data tape. The pipeline file on the data tape includes new processing scripts for producing pulse height QPOE’s with cep. The operation requires version 1.5.1 of the IRAF/EUV software, and version 1.10 of the EUVE reference data. The observation file is created by the telemetry translation task euvred.gettm, and the copy from the data tape with the WSZ-format data should be used to ensure the version is correct.

The pipeline file in the version 1.10 reference data includes the script, Spectrometer Wavelength Pulse Height Image, which create pulse height QPOE files for all three Spectrometers. The user runs cep with this as the value for pipeline:

          eu> cep pipeline 'Spectrometer Wavelength Pulse Height Image'
The pipeline file also has scripts called SW/MW/LW/DS Wavelength Pulse Height Image,

which create pulse height QPOE’s for each DS/S detector.

5.2 Rerunning CEP to Correct Aspect Problems

As discussed in section 1.3, the aspect record in telemetry may sometimes have errors, because of pointing problems. Also, if the target coordinates used to remap the data in cep are in error, or at all different from those used to point the telescope, the results will be the same as if a systematic error had occurred in the aspect reporting. In all these cases, cep must be rerun on the data with some kind of “corrected” input to force the correct remapping of events. Aspect errors that change over time are addressed by rerunning cep with a “corrected” aspect table. The table can be generated with the task aspcorr, as described in section 1.3.2 and example 1.3. We complete the application of aspect-corrected mapping here with the rerunning of cep to make the corrected QPOE file.

Related topics

◇ See the material in section 1.3 for more information on diagnosing aspect errors. and the corrections that can be applied.

3CEA Web pages, at the URL in section D.2.

5/97

Rerunning CEP to Correct Aspect Problems 5-11

Example 5.2 : Reprocessing an observation of the Moon

Recall example 1.3 in Chapter 1, which demonstrated the creation of a revised aspect table for an observation of the moon. The moon was moving through the field of view, and a table of apparent positions in the Spectrometer field was used to add compensating offsets to the aspect components in table1.tab with the task aspcorr. To produce QPOE files with the corrected event positions, the event pipeline must be rerun with the new table as the aspect reference.

Since cep expects the table to be named table1.tab, the original table must be

moved from observation/table1.tab to observation/table1old.tab and the new aspect table

copied from newtable1.tab to observation/table1.tab. The pipeline uses only the aspect columns, Aspect(T,W-Z), so this change will be transparent to cep.

The GO moves the original QPOE files to different names first, to keep them for comparisons with the new files. IRAF’s handy global substitution syntax saves a little typing:

          eu> rename sw.qp sw old.qp
          eu> ^{∧}sw^{∧}mw^{∧}g
          eu> ^{∧}mw^{∧}lw^{∧}g

He then makes the other required preparations and reruns cepto produce only the LW QPOE file. Since he wishes to do other processing IRAF, he puts the task in the background:

          eu> cep pipeline `LW Wavelength Calibration Image' >&cepmess &

The messages cep usually displays on STDOUT will be redirected to the file cepmess.

The image created from the new long wavelength QPOE is shown in figure 1-19, and may be compared to figure 1-16. The corrected picture shows each spectral feature centered in Y at the position of a boresight spectrum. The height of each feature is determined by the size of the moon in the field of view, and the width by the collimator response (≈ 20′ ⌢ FWHM).

5.2.1 Correcting constant aspect error

Some cases of error in the aspect report involve constant offsets between the actual pointing direction and the aspect reported. This may cause the spectra to be displaced from boresight in wavelength and/or position, and to be slightly blurred. A constant aspect error can occur for any (or a combination) of several reasons:

5/97

• Telescope flexure – The structure that supports the DS/S instrument appears to flex slightly in some orientations of the satellite. This changes the orientation of the DS/S instruments to the spacecraft attitude control system (ACS), producing an unrecorded offset between the telescope pointing direction and the ACS coordinate system. As a result, the aspect reports do not have the assumed relationship to the DS/S look direction, whether the telescope is pointed directly at the target, or not, and imaging is degraded.

• Guide star interference – A start tracker may lock onto another star near the intended guide star, or become stuck in an intermediate position, while reporting that it has settled on the guide star. Again a constant aspect offset can result.

• Incorrect coordinates – If the coordinates used to point the telescope and reduce the observation are in error, or if the target is an extended object with an EUV source of unknown location, this may produce an off-boresight spectrum.

As described in chapter 4 of the EGO Center Data Products Guide, cep compares the aspect report for each event to the target position in catalog.tab, figures the difference, and applies this factor to the wavelength and imaging angle (or the sky coordinates, for DS data) as a correction. This correction scheme uses the aspect information to remap the events as if the telescope were pointed with the target perfectly on-axis. The usual pointing errors reported in the aspect are within about ±20 − 30 seconds. All events that occur during each aspect reporting period (1 MF) are given the same correction.

Small, unreported errors in aspect occur in many observations because of pointing or coordinate offsets. However, the problems caused by constant aspect offsets are systematic and relatively easy to correct by rerunning the pipeline with an adjusted source position. This is an easier way to add the necessary compensating factor to the event adjustments calculated in cep; because the compensation is always the same, it is added only once, to the coordinates in catalog.tab, instead of having to be calculated for each aspect report (the job done by aspcorr).

cep always produces a Deep Survey image with the catalog.tab coordinates at the center. If the source is not really on-axis, but the aspect reports reflect only the normal jitter, the image will appear not only off-center, but at a different location, in the World Coordinate System applied by cep. This situation is depicted in figure 5-3. The telescope is actually pointed in the direction p⃗, but the aspect reports locations near the intended pointing t⃗. After processing with cep, the image appears up at the apparent location t⃗′ in the remapped coordinates.

To correct for an unreported offset, the coordinates in catalog.tab should cause cep to compute coordinate corrections that compensate for both the recorded aspect jitter and the offset. This is equivalent to running cep with the apparent coordinates of the target from the remapped DS image replacing the coordinates in catalog.tab.

5/97 Figure 5-3: Incorrect imaging due to coordinate/pointing offset. The offset Δp⃗ causes the target image to be mapped to the location t⃗′ in the Deep Survey detector. The remapped DS image always has north along the positive Y axis.

5.2.2 Finding Deep Survey image centroids

Some GO’s will find it helpful to produce apparent coordinates by making centroids from the Deep Survey QPOE file. If there is a systematic, constant pointing offset, make a single image for the entire observation. If multiple centroids are needed, use QPOE time filters to create several images for different periods of the observation. For example, with a high Lexan count rate, the pairs of start and end times from the filter macro string in ds gt.tab might be dumped and edited to make several files, each with valid times in a segment of the observation. Then each file, orbits1 3, orbits4 6, etc., can be used to make a separate image for those valid times:

          eu> qpmkim 'ds.qp[time=(@orbits1 3),rect=[1026:1048,1009:1031]]' @img1 3
          applycor- detector=ds
          eu> ^{∧}1 3^{∧}4 6^{∧}g

etc., where each file img# # has an image made from the start : end times of three orbits. Determine values for the rect filter, including a margin of background, from displays or plots of the QPOE file. If the observation count rate is limited, it may not be possible to generate frequent DS centroids, so experiment first.

The task proto.imcntr can then be used to find the centroid of each image along the rows and columns. A sample parameter file for imcntr is shown in figure 5-4.

5/97

          pr> lpar imcntr

        input = "img*.imh"      Image names
       x_init = 1037.           Approx x position of star
       y_init = 1020.           Approx y position of star
    (cboxsize = 23)             Size of extraction box
        (mode = "ql")

Figure 5-4: Sample parameter file for using imcntr to find the centroid of a DS image.

The value of cboxsize should be an odd number. imcntr requires an estimate of the centroid as a starting point, such as one might read from the cursor location in a 2-d image display. The following runs the task on each of the images made by qpmkim above, directing the output to an ASCII file called centroids:

          pr> imcntr img* 1037 1020 > rawcents

Check the output centroids against the displayed images, and if necessary, remove any extra characters before using the rawcents file as input for the next step.

cep uses coordinates in decimal degrees, and aspcorr needs the apparent coordinates in hours (RA) and decimal degrees (Dec), so the centroids output by imcntr need to be converted from logical coordinates (image section pixels) to celestial coordinates. The orientation of the DS image in celestial coordinates is given in the WCS system placed in the image header by cep. The Y axis of the remapped image always corresponds to north on the sky. Use the task xray.xspatial.wcscoords to convert centroids to RA and Dec using the WCS in the DS image header:

          xs> wcscoords ds 11 rawcents ofile=dscents

wcscoords uses the WCS in the reference image ds.imh, and an internal code (11) to perform a logical-to-WCS conversion on the pairs of X,Y coordinates in the input file (rawcents) and places the converted coordinates in the file dscents. Recall that the coordinates in an image made from a section of another image, in this case the partial DS images input to imcntr, are the logical coordinates, referenced to the section boundaries. See the wcscoords help page for a complete list of the conversion codes.

To correct for a systematic aspect error, where the pointing offset is constant, the remapped DS image coordinates from a centroid of the entire observation are used in place of the original target coordinates in catalog.tab, as in next example. If multiple DS centroids have been created, to map a drift in aspect, the coordinates can be used to create a simple position table for input to aspcorr, as in section 1.3.2.

5/97 Example 5.3 : Reprocessing an observation to correct for flexure

Suppose that a GO displays the Deep Survey detector remapped image for an observation of FK Aqr, and finds it is fuzzy and not well-centered in the field of view. He locates the approximate center of the image with the cursor, and uses wcscoords to find that the image position from the internal image WCS:

          xs> wcscoords ds 5 STDIN

            ⋮  (Terminate interactive list with <cntl>-D)

           1023 1021

           Convert image pixels to sky coordinates

           1023.  1021.  22:38:41.6 -20:37.19.5

These coordinates correspond to an RA and Dec of (339.6732,-20.6221) in degrees: about 35.8 seconds from the requested position of (339.6829,-20.6199). Next the GO checks the target coordinates in the observation log and the copy of catalog.tab from the data tape, to make sure that his requested coordinates were indeed used to point the telescope and run the pipeline.

He then inspects the table mkaspectds.tab, which contains the satellite aspect reports from table1.tab, converted to celestial coordinates. The aspect report suggests the satellite reported it was pointing just over half an arc minute away from the target, and jittering systematically around that spot. The GO spotchecks some rows at the beginning, middle and end of the aspect file, making sure that the aspect reports change by no more than ≈ 30−40 seconds about this position– the normal aspect jitter. So there is a constant pointing offset, but the pointing and the aspect report are stable. The problem can be fixed fairly easily, by finding the apparent position of the source in the field of view, and inserting this position in catalog.tab, causing cep to perform the correct remapping, event with to incorrect aspect.

The GO first determines the image centroid more precisely, using imcntr, and converts them to degrees with wcscoords:

          xs> imcntr ds 1024 1021 cbox=21
           [ds] x:  1023.117 y:  1021.105

5/97

          xs> wcscoords ds 5 STDIN

             ⋮  (Terminate interactive list with <cntrl>-D)

           1023.117 1021.105

           Convert image pixels to sky coordinates

           1023.117 1021.105 22:38:41.6 -20:37.23.6

He then converts the coordinates to decimal hours and degrees, and inserts them into catalog.tab with tedit. Now he is ready to reprocess the data using the default pipeline in cep and the new version of catalog.tab.

A combination of the methods in this section and section 1.3.2 may be necessary to correctly reduce observations with, for example, telescope flexure and roll angle changes, or conditions that produce bad affect reporting over a fraction of the exposure.

5/97

CONVERTING EUVE DATA PRODUCTS TO FITS FORMAT

Users who come to the EGO Center to reduce their data may have new data products to transport when they leave. To convert most IRAF data products to FITS format, the general rule is: IRAF image ⇒ basic FITS file and ST table ⇒ FITS file with binary table extension. Many GO’s like to transport the new FITS files directly to their home machines via ftp, and make a tape copy to take with them as backup. QPOE files may be tar’ed directly onto tape if you are going to read them back on a SUN Sparcstation with the same binary encoding conventions. Otherwise, it is possible to create FITS files from EUVE QPOE files, using a special task which first packs the QPOE into a number of ST tables. We will deal with the more general cases first.

6.1 Creating FITS Files with STWFITS

The EGO Center recommends using only fitsio.stwfits to convert any new IRAF images or ST tables, to FITS files before putting them on tape. A sample parameter file for stwfits is shown below in figure 6-1. Some of the parameters should be changed from their default values to write a table to a binary extension. Read the help page for more details on the parameter options.

Notes on running stwfits

□ To convert ordinary IRAF images (as opposed to GEIS or STF images) to FITS, supply the .imh extension with the header file name. Likewise, supply the .tab extension when converting ST tables.

□ format file is an option to customize what appears on STDOUT as the task runs, by pointing at file of user-defined ouput. It has no effect on the format of the output FITS file.

□ Letting bitpix equal zero (the default) causes the task to take the data type from the type of the input.

□ The extensions and binary tables parameters should be set to “yes” for translating ST tables to binary-encoded table extensions with full precision. binary tables should be set to “no” for converting images.

□ The parameter gftoxdim is used only for translating certain Space Telescope data formats, and should be turned off (not the default) for converting EUVE data files.

5/97

6-2 CHAPTER 6. CONVERTING EUVE DATA PRODUCTS TO FITS FORMAT

          ta> lpar stwfits

   iraf_files = mw_mygt.tab     IRAF image
   fits_files = mw_mygt.FITS    FITS filename
      newtape =                 Blank tape?
 (long_header = no)             Print FITS header cards?
(short_header = yes)            Print short header?
 (format_file = "default")      Format filename
    (log_file = "none")         Log filename
      (bitpix = 0)              IRAF data type
(blocking_fac = 1)              FITS tape blocking factor
  (extensions = yes)            Allow FITS extensions in the same file?
(binary_table = yes)            Binary FITS tables?
    (gftoxdim = no)             Multigroup to xtra dimension?
        (ieee = yes)            Convert to ieee standard?
       (scale = yes)            Scale data?
   (autoscale = yes)            Auto_scaling?
      (bscale = 1.)             FITS bscale
       (bzero = 0.)             FITS bzero
    (dadsfile = "null")         Internal use.
    (dadsclas = "null")         Internal use.
    (dadsdate = "null")         Internal use.
        (mode = "ql")

Figure 6-1: Sample parameter file for stwfits. The task is set to write a good times table to tape in a FITS file with a binary table extension.

□ EUVE ST tables use the IEEE floating point format, so ieee should be “yes” (the default).

Running stwfits by typing :go from eparam with the above parameters is equivalent to typing:

          fi> stwfits mw mygt.tab mw mygt.FITS no exten+ binary+ gftoxd-

6.2 QPWST converts QPOE files to sets of ST tables

Users who create new QPOE files at some remote site, like the EGO Center, may wish to transport them between different machines in a portable format. EUVE QPOE files can be converted to FITS files using the two tasks euvtools.qpwst and fitsio.stwfits in the tables packages. The same process in reverse was used to create the FITS-encoded QPOE files on your data tape, and it generally ensures that QPOE files you create will be usable when transported to another system ^{1}.

5/97 QPWST converts QPOE files to sets of ST tables 6-3

1GO’s with DEC computers, such as VAX, should take care to follow the special instructions for unpacking FITS’d QPOE files in VMS systems. See section 0.3.2 of the EGO Center Data Products Guide.

The conversion from QPOE files to FITS files is a two-stage process. First, each QPOE file is converted to a standard set of ST tables using the IRAF/EUV task qpwst. A sample parameter file is shown below in figure 6-2.

          eu> lpar qpwst

   qpfilename = "newsw.qp"      input QP filename
       tabdir = "/tmp1/drego1/" directory to put the ST tables in
   (tabprefix = "newsw_qp_")    prefix to use on ST table filenames
   (parmtable = "pars")         Name of table to contain QP parameters
     (verbose = yes)            Verbose processing information
        (mode = "al")

Figure 6-2: Sample parameter file for qpwst, to convert a QPOE file into ST tables in the directory /tmp1/drego1/.

Alternatively, convert the new short wavelength QPOE to a set of ST tables in the directory /tmp1/drego1/, from the command line by typing:

          eu> qpwst sw.qp /tmp1/drego1/ newsw qp pars

Notes on running qpwst

□ Make sure the output directory specified by tabdir has more than enough room for the QPOE file. We have used a temporary location in the example. IRAF requires the trailing slash for directory locations.

□ The parameter file is quite simple, and the task itself specifies all the table names, using the prefix specified by tabprefix. To distinguish new QPOE’s don’t use a prefix like “sw” that is used elsewhere on the data tape.

□ The parameter parmtable names a table where the task writes the QPOE file’s internal parameters. The user needs to supply the parmtable value later, when unpacking the QPOE file in IRAF.

The list of ST tables resulting from the parameters in figure 6-2 would be:

newsw_qp_pars.tab       newsw_qp_lambda.tab       newsw_qp_modtime.tab
newsw_qp_EQUINOX.tab    newsw_qp_dc_flag.tab      newsw_qp_naxes.tab
newsw_qp_INSTRUME.tab   newsw_qp_defattr1.tab     newsw_qp_qpwcs.tab
newsw_qp_OBSERVAT.tab   newsw_qp_dispaxis.tab     newsw_qp_sw_adct.tab
newsw_qp_TELESCOP.tab   newsw_qp_dx.tab           newsw_qp_mysw_gt.tab
newsw_qp_axlen.tab      newsw_qp_dy.tab           newsw_qp_theta.tab
newsw_qp_cretime.tab    newsw_qp_earth.tab        newsw_qp_time.tab
newsw_qp_datamax.tab    newsw_qp_events.tab       newsw_qp_x.tab
newsw_qp_datamin.tab    newsw_qp_limtime.tab      newsw_qp_y.tab

5/97

You may want to compare this list to the one produced by strfits from an EUVE QPOE FITS file, shown in section 0.3.2 of the EGO Center Data Products Guide.

The table newsw qp events.tab contains the event data, while other tables contain various parameters that are stored in areas of the QPOE called the header and symbol table. You will recognize many of these from the listings produced by imhead. In addition, each macro contained in the QPOE file is transformed into a separate table. The tables newsw qp (time,x,y).tab, newsw qp sw gt.tab, and several others are made from the standard EUVE QPOE file macros. For instance the list above implies that the person who created newsw.qp chose to insert the EGO Center macros earth and sw adct, in addition to a (user-defined) mysw gt.

The second step is to convert the set of ST tables to a single FITS file, with fitsio.stwfits. It may be helpful to move the ST tables into their own directory with an obvious name, like newsw qp tables first. Each table will be attached to the file as a separate binary table extension.

Notes on making FITS files from qpwsttables

□ All the ST tables should be selected by running stwfits with the value of

     iraf files equal to “tabprefix*.tab”.

□ The parameter fits files should contain the original name of the QPOE file, as in “newsw qp.FITS.” File extensions will be shortened to three characters.

□ Set the parameter extensions to “yes”, to pack all the tables as extensions to a single file.

□ Other parameters may be set as in the example in figure 6-1.

The resulting FITS table contains an empty image and one binary table extension for each of the ST tables. It will be slightly larger than the QPOE file.

5/97

Appendix A

USING THE QPOE I/O INTERFACE

This appendix fills in some details on what the QPOE interface can do; what it can operate on, processes that can be executed, and more complete description of the syntax for interface expressions. It is not exhaustive, and unfortunately, there doesn’t seem to be any really complete description of the interface that is meant for end users. If you encounter problems or have further questions while processing your EUVE data, please send them to the EGO Center, at egoinfo.cea.berkeley.edu. The EUVE science and programming staff will support Guest Observer’s efforts as completely as possible.

A.1 QPOE File Headers

QPOE file headers can contain quite a lot of information, packaged in several different formats. As noted in the EGO Center Data Products Guide and in section 2.2 of this User’s Guide, a partial representation of QPOE file headers can be obtained with the tasks imhead and qpmedit. To understand the operation of the the QPOE I/O interface, it is helpful to have a more complete picture of how non-event information is stored and maintained in QPOE files. This section describes QPOE file header parameters which, although they look the same in the output from imhead, are different from the header cards in FITS files or IRAF images This information is not necessary for doing most standard data reduction on EUVE QPOE files, but may prove useful to GO’s who process their QPOE’s with tasks in the xray package, or who wish to write their own QPOE-compatible IRAF scripts. Review the general information about QPOE files in the EGO Center Data Products Guide, Chapter 2, if some of the terms used are not familiar to you.

A.1.1 Types of QPOE header parameters

QPOE header parameters can be separated into several basic categories:

• Scalars – The simplest parameters contain a single number or a single ASCII character. They are read directly by imhead and the literal values are printed in the output as if they were IRAF image cards.

• Vectors – Unlike IRAF images, QPOE files may have header parameters with two or more elements: vector parameters. The individual vector elements can contain numbers, ASCII characters, or more complex structures. Every QPOE file has a character vector called EVENT, containing attribute names and

5/97

A-2 APPENDIX A. USING THE QPOE I/O INTERFACE

definitions, which defines the structure of the event record. The event list itself is a very long, complex vector called EVENTS. These two vectors appear in different forms in the output from imhead, and illustrate a basic limitation of imhead in dealing with vectors. imhead can print QPOE vector parameters that are character strings, and not too long, but when it encounters a non-character vector, it cannot print the literal value. Instead imhead prints the vector type, and the size in some appropriate units, using the usual IRAF data types: b=boolean, c=character, d=double, i=integer, l=long, r=real, x=complex^{1}. So if the vector appears as

       NAME = r[010]

it is an array of 10 real numbers. EUVE QPOE files do not include many noncharacter vector parameters as of this writing, so they are all printed entire except for some special cases.

• Special Classes of Parameters – In addition to scalars and the usual types of vectors, there are three other types of parameters which may be included in the QPOE file header. In EUVE QPOE files they are always vectors, though this is not a general requirement.

1. macros – Macro definitions are stored as vectors of ASCII characters. Although the value is technically an ASCII string, imhead cannot print the value, because the vector is not of type c. Instead the task produces the entry macro[#chars], with the number of characters, followed by the string `macro definition'. Use the task qpmedit (section 2.2.2) to examine the contents of macro definitions. The standard EUVE macros are more fully described in Chapter 4 of the Data Products Guide.

2. opaque vectors – These are parameters that the QPOE interface cannot interpret for use. They may be intended for use by some other task in IRAF. imhead only gets the type and the number of bytes in the value of the parameter, with a short label that describes the contents. All EUVE QPOE’s contain a qpwcs, which is a special case of opaque parameter. The qpwcs vector stores the spectral world coordinate system (WCS) that maps the attributes x,y to lambda,theta, in an internal binary format. The parameter is listed as opaque in the imhead output, but the QPOE interface also translates its elements into a set of header cards that specify the WCS in the usual IRAF format. The interface performs this translation so that when the interface passes the QPOE file to an image task, the task have a WCS to work with. The IRAF document The IRAF/NOAO Spectral World Coordinate Systems, listed in Appendix D, explains the IRAF WCS notation.

QPOE File Headers A-3

3. user-defined parameters – These are in fact usually defined by the program that creates the QPOE file. Again, imhead cannot print the value; instead it prints name[#elements], followed by a short descriptive label. The most important example of a user-defined parameter is the event list itself. The vector events is made by the “user” cep, and has elements of a type defined by the value of event. The number of elements printed by imhead is the number of events in the event list.

1“c” only appears in the case of an empty character vector, as c[000].

5/97

A.1.2 Reading the imhead display

When a QPOE file is processed by imhead, the output has a number of lines which appear identical to those produced by IRAF image header cards:

           CAPSNAME = some string or numerical value

Other lines, like those produced by the QPOE macros, show only some labels and numbers; their literal values cannot be printed. imhead truncates all keywords to eight characters and displays them in capitals, although QPOE header parameters are not subject to these limitations like the image cards.

Unfortunately, imhead is the only tool in IRAF for examining most of the QPOE header parameters. Table A-1 shows each of the parameters in a standard EUVE QPOE file, in the order they appear in the imhead long output. We have identified each parameter with both the label printed by imhead and the internal QPOE name; the latter are often spelled the same, but are not capitalized. On the other hand, parameters which are inserted by cep for compatibility with IRAF image tasks are all-capitals. Any time a header parameter is used as input to a QPOE task, it must be spelled as it is stored in the file.

The table shows the type of each parameter as well as some information about how the imhead “value” is produced. All the entries except BLOCKFAC are from a part of the QPOE called the symbol table, having been put there by cep when it created the QPOE. Many of the lines that translate most literally, like the EUVE global observation parameters^{2}, are put in the symbol table for the user’s information only.

The values of some imhead entries are generated “on the fly” by the QPOE interface when it opens a QPOE file for processing as an image. This is because the symbol table is also used to pass information about the QPOE to IRAF image tasks, with the QPOE interface operating on the table entries to package them into “card-like” values. The values may be printed (as by imhead) or used in some other way by another task. Some examples:

• The parameters naxes and axlen# are required by many IRAF image tasks.

• The opaque vector qpwcs is stored in binary and expanded into a series of WCS cards by the QPOE I/O interface.

• BLOCKFAC is the value of a QPOE I/O parameter that sets a default bin size.

5/97

2Described in Appendix B of the EGO Center Data Products Guide.

A-4 APPENDIX A. USING THE QPOE I/O INTERFACE

Table A-1: EUVE QPOE file header parameters, as generated by imhead.

imhead label QPOE name param type  origin  comments
Created  cretime  scalar  inserted by cep  not FITS card format
Last modified modtime  scalar  inserted by IRAF  not FITS card format
min  datamin  scalar  generated by QPOE I/O minimum data value
max  datamax  scalar  generated by QPOE I/O maximum data value
NAXES  naxes  scalar  generated by QPOE I/O 2 axes for EUVE QPOE’s
AXLEN1  AXLEN1  scalar  generated by QPOE I/O 2048 for EUVE QPOE’s
AXLEN2  AXLEN2  scalar  generated by QPOE I/O 2048 for EUVE QPOE’s
BLOCKFAC  blockfactor  scalar  generated by QPOE I/O QPOE I/O parameter
QPFILT01  PFILT01  char vector  generated by QPOE I/O filter used whenever the QPOE is

opened. Up to 32 cards.

WCSDIM  N/A  scalar  generated by QPOE I/O element of QPWCS
CTYPE1  N/A  char vector  ”  element of QPWCS
CTYPE2  N/A  char vector  ”  ”
CRVAL1  N/A  scalar  ”  ”
CRPIX1  N/A  scalar  ”  ”
CRPIX2  N/A  scalar  ”  ”
CDELT1  N/A  scalar  ”  ”
CDELT2  N/A  scalar  ”  ”
CD1 1  N/A  scalar  ”  ”
CD2 2  N/A  scalar  ”  ”
LTM1 1  N/A  scalar  ”  ”
LTM2 2  N/A  scalar  ”  ”
WAT0 001  N/A  char vector  ”  ”
WAT1 001  N/A  char vector  ”  ”
WAT2 001  N/A  char vector  ”  ”
QPWCS  qpwcs  opaque  inserted by cep  world coordinate system
ORIGIN  ORIGIN  char vector  inserted by cep  global observation info
OBSERVAT  OBSERVAT  char vector  ”  ”
TIMESYS  TIMESYS  char vector  ”  ”
EQUINOX  EQUINOX  scalar  ”  ”
JDREF  JDREF  scalar  ”  ”
TIMEUNIT  TIMEUNIT  char vector  ”  ”
RA UNIT  RA UNIT  char vector  ”  ”
DEC UNIT  DEC UNIT  char vector  ”  ”
HIST001  HIST001  char vector  ”  ”
REFVERS  REFVERS  char vector  ”  ”
EGOCSVER  EGOCSVER char vector  ”  ”
DATE-OBS  DATE-OBS  char vector  ”  ”
TIME-OBS  TIME-OBS  char vector  ”  ”
DATE-END  DATE-END  char vector  ”  ”
TIME-END  TIME-END  char vector  ”  ”
INSTRUME  INSTRUME  char vector  ”  ”

5/97

Table A-1, Cont’d: EUVE QPOE header parameters

imhead label QPOE name param type  origin  comments
TELESCOP  TELESCOP  char vector  inserted by cep global observation info
PI NAME  PI NAME  char vector  ”  ”
OBJECT  OBJECT  char vector  ”  ”
RA OBJ  RA OBJ  char vector  ”  ”
DEC OBJ  DEC OBJ  char vector  ”  ”
PROPID  PROPID  char vector  ”  ”
DEFATTR1  defattr1  char vector  ”  defined as: integral time:d
TIME  time  macro def’n  ”  EUVE attribute name
X  x  macro def’n  ”  ”
Y  y  macro def’n  ”  ”
LAMBDA  lambda  macro def’n  ”  ”
THETA  theta  macro def’n  ”  ”
DX  dx  macro def’n  ”  ”
DY  dy  macro def’n  ”  ”
XS-EVENT  XS-EVENT  char vector  ”  for use with xray
XS-SORT  XS-SORT  char vector  ”  for use with xray
EVENT  event  user defined  ”  char vector defining event format
EVENTS  events  user defined  ”  the event list
DISPAXIS  dispaxis  scalar  ”  1 for dispersion along columns
DC-FLAG  dc-flag  scalar  ”  0 for linear WCS
EVENTSOR  EVENTSOR scalar  ”  number of events processed
EVENTSIR  EVENTSIR  scalar  ”  number of events rejected
(DET) GT  (det) gt  macro def’n  ”  time filter
EARTH GT  earth gt  macro def’n  ”  time filter
(DET) ADCT  (det) adct  macro def’n  ”  time filter

• The parameter defattr is a special case; it’s string value defines exptime as the integral of time over any time filter present when the QPOE is opened. When the QPOE event list is binned to make an image, the interface calculates the integral and places the result in a card called exptime in the image header. The exptime card is not displayed in the output from imhead because imhead does not require the event list to be binned.

A.1.3 Editing QPOE header parameters

Only a few QPOE header parameters are accessible to users; the rest are created and used only by tasks that operate on QPOE’s. Users can edit the default parameters,

deffilt and defmask.

5/97

The default filter deffilt has the form:

           deffilt = att1r=(string1),attr2=(string2)...

which is applied to the event list every time the QPOE file is opened. Default header parameters can be added, deleted, or edited using the task ximages.qphedit. Using qphedit is very similar to using qpmedit, which is described in section 2.2.2. One reason to do this is to use the xray.xtiming tasks, as discussed in section A.1.4 and example A.1. The task qphedit will allow any string to be placed in the QPOE file, but a valid deffilt must contain complete filter syntax. The default filter should contain attribute filters only; the xray tasks cannot interpret default filters with any of the function keywords, such as block or rect.

Tasks in the xray packages will ignore command line filters and use the deffilt instead. euv package tasks merge the default filter with any other filters present, which may produce invalid results due to a QPOE interface bug (see box on page 2-5). Special syntax can be used to negate a deffilt (see example A.1).

One can also use default region masks of the form:

           DEFMASK = PLIO region specification

PLIO region masks are described briefly in the subsection on mask in section 2.1.3.

A.1.4 Processing EUVE QPOE files with the XRAY package

EUVE QPOE files now contain the header parameters XS-EVENTS and XS-SORT, which provide compatibility with tasks in the xray packages. These include xray.xtiming, xray.xspatial, and xray.xplot. Type “help xray” for a listing of packages and topics. Another requirement for compatibility with most xray tasks is the addition of the deffilt header parameter, whose value is a default attribute filter.

If xray tasks are used on a file with no deffilt, or no time filter in deffilt, the task will run without error, but no time filtering will take place. This is true even if a time

filter is included on the command line. The task will display the message:

           No GTI records in file - no TIME update done

The note (; no exp.) will appear in the header of the resulting QPOE or image file, and the value of exptime will not be calculated correctly by the xray task. This is because xray tasks are written to look for filtering information in the QPOE file header parameters deffilt and defmask. It is not a characteristic of the QPOE interface itself. Tasks in the euv package can parse command line filters and a deffilt, or a combination, but then filter merging may become an issue (see the box in section 2.1.2).

5/97

Additional Notes on QPOE Expression Syntax A-7

Once a deffilt card has been inserted, this filter is always applied to the QPOE events when the QPOE is opened by the interface program, unless a special QPOE I/O syntax is used. This means that those who use tasks in the xray packages should exercise extra care when processing these QPOE’s, as the deffilt will interact with any on-line filters on the same attributes when processing or making images. Because of the QPOE interface bug outlined early in section 2.1.2, if a filter specified

on the command line is applied to the same attribute as deffilt, the results may not be correct.

If, for example, both filters are applied to the time attribute, the resulting exposure time may be incorrect.

If deffilt can be used to hold all the exposure and data quality filters required, then processing the QPOE can be done without any other filters on the command line. But if filters other than deffilt are needed for some phases of processing, it is easiest to remove the deffilt with qphedit beforehand. See section A.2.2 to read about the QPOE I/O interface syntax for nulling the effect of deffilt on the command line.

One xray task which may be useful for GO’s doing timing analysis is xtiming.timsort. This task resorts the QPOE file in time order, which makes subsequent time-filtering operations run much more efficiently. Although timsort is set up to operate most easily on PROS QPOE files, a complete resorted copy of an EUVE QPOE can be made by specifying the value of the task parameter esize as follows:

           esize = "d:time,s:x,s:y,r:lambda,r:theta,s:dx,s:dy"

Note that each attribute to be included must be specified by both type and name, the latter of which is used as the column name in the new QPOE file.

A.2 Additional Notes on QPOE Expression Syntax

The description of the QPOE interface given in section 2.1 should enable you produce valid basic QPOE expressions. While it is not complete, this section adds a bit more detail, and some variations of the QPOE expression syntax that some GO’s may find useful.

A.2.1 Multiple intervals and filters

In QPOE expressions with attribute filters, the filter string can be quite long and complex, as we have seen. The exact selection rule produced depends on punctuation as well as on the interval specifications.

5/97

Commas and parentheses

Generally, the commas between intervals function as logical OR’s; any event in any of the comma-separated ranges passes this expression. So the filter expression:

           dx=(5:10,11,16:17,20:)

Will select all events with a value of dx between 5 and 10, OR equal to 11, 16, OR 17, OR greater than 20. In an expression like:

           sw.qp[dx=(5:10,11,16:17,20:),dy=(1:5),time=(sw gt)]

the commas between the three attribute filters are ANDs; an event passes this set of filters only if it matches all three filter expressions.

Commas and parentheses can also be used to produce logical ANDs within a filter string for a more stringent criterion. For example:

           'mw.qp[y=((900:1100),!(1018:1032)),x=((140:1780),!(1200:1400))]

will produce a background region equivalent to a 200-row strip in Y, excluding 15 rows where the spectrum falls, and a section in X with 200 columns, where a bright airglow line raises the normal background. The outer ( )’s delimit the filter string associated with the attribute, and the inner ones delimit the ranges (intervals) that are being ANDed together for each one. The resulting QPOE can be trimmed of its 0 “pixels” by using a rect mask as well.

In addition, when filters on different attributes are ANDed together, the interface applies the individual filters in the order in which they appear in the QPOE expression. This means users can save processing time by putting filters that eliminate the most events first. Then the interface doesn’t waste as much time putting events through one filter only to have them eliminated by later ones. For example, in processing EUVE QPOE’s, good times filters should generally come last, since they do not normally eliminate many events. On the other hand, the order of the attribute ranges that are ORed together within each filter is unimportant.

Examples of specialized QPOE expressions

1. Make an image of a QPOE in raw detector coordinates and block the resulting image by a factor of 2. The key expression comes first, and the blocking is then performed on the detector coordinates. The output image will be 1024 × 1024 pixels in size.

               im> qpmkim 'test.qp[key=(dx,dy),block=2]' test.imh applyc+ adjexp+

All other parameters of qpmkim must be set in advance.

5/97

2. Filter the QPOE according to the x coordinate. The interface bins all events with an x value between 100 and 2000. The only events that pass the filter are those from the intervals (100–1200) and (1400–2000). Others are 0.

               im> imcopy 'test.qp[rect=[100:2000,*],x=!(1200:1400)]' test.imh

3. Select a section of a Deep Survey QPOE file by filtering on RA and Dec:

               im> imcopy 'sw1.qp[ra=(19.853:19.953),dec=(-210.10:-210.20)]'
               sw blkspec.imh

A.2.2 Special punctuation for attribute filters

The QPOE I/O interface provides some variations on the ‘=’ operator for combining or selecting among multiple QPOE filters, which may appear in deffilt, macro definition files (described below), and command line expressions. When the QPOE interface finds multiple filters associated with a single QPOE file, the default action is to merge (AND) all the filters, including filters on the same attribute, to form the logical intersection. As noted in section 2.1.2, the interface cannot be relied on to merge two or more filters on the same attribute reliably at this time (version 2.10.4 and earlier). The conventions we describe here are subject to the caveat that at present, combining two or more filters on any one attribute should still be done with the IRAF/EUV task tfilter. The following syntax can be used successfully if they do not imply merging filters this way.

One variant on attribute =(string) is attribute:=(string). In the case of :=, any

filters already entered for the named attribute, in deffilt, in a macro, or earlier on the command line, will be REPLACED by the new expression.

Another special case is the syntax !attribute=(string). In this case, Any previously entered filter on any attribute will be cancelled and replaced by the part of the expression after the symbol !. This should not be confused with the use of ! to negate an interval within a filter string.

Example A.1 : Working around a deffilt

In the particular case of processing QPOE files with the xtiming package, users may need to know how to include or exclude the default filter in deffilt with other filters. Suppose, for example, that a default filter has been added to a QPOE file, in order to resort two orbits’ worth of events in time order with timsort. The parameter deffilt is added to the QPOE header with the task

ximages.qphedit:

          xi> qphedit sw deffilt time=(780800001.4596:780802014.6436,780805
          525.9396:780807708.0837) add+ update+

5/97

When imhead is run on the file sw.qp, the following line appears:

           DEFFILT = 'time=(780800001.4596:780802014.6436,780805525.9396:78080
          7708.0837)'

This might not be quite the time filter needed for some other operation, such as making a new spectral image for the entire observation. To replace this filter with one using the good times macro, sw mygt, in the QPOE, one could type:

          eu> qpmkim 'sw.qp[!time=(sw mygt),rect=[*,720:1299]]' swgt adjexp+
          dead+ prim+

This would result in a standard sized (2048 × 580) 2-dimensional image, using the GO’s specification of good time intervals, instead of the deffilt, to filter the event list and calculate the value of exptime.

If, on the other hand, the GO has created a deffilt with filters on more than one attribute, and wishes to replace only part of the expression, a different syntax is used. The user may have placed a wavelength filter in the deffilt as well:

           DEFFILT= 'time=(780800001.4596:780802014.6436,780805525.9396:780807
          708.0837),lambda=(90:120)'

In this case the GO replaces the time filter with her own good times, while keeping the spatial filter for trimming the image, using the following QPOE expression in the command to qpmkim:

          eu> qpmkim 'sw.qp[time:=(sw mygt)]' swgt adjexp+ dead+ prim+

This results in only the time filter of deffilt being replaced; the wavelength filter is still applied to produce an image of part of the spectrum.

Users should bear in mind that any spatial filters other than rect, including PLIO region masks, will not make the image smaller, but produce a 2048×2048 image with zero values outside the filter or mask region.

A.3 Macro Types and Hierarchies

Macros can be defined several ways and in several places, allowing for a hierarchy of precedence. We will discuss the major types of macros and how to use them, but this section is not meant to be an exhaustive treatment of the possibilities. There are three major types of macro references, which are summarized in table A-2, and defined in more detail below.

5/97 Macro Types and Hierarchies A-11

Table A-2: Form of QPOE file macro references

macro type  action  examples

text substitution macro expanded to part of expression from definition ’sw.qp[time=(1:300,900:1200)]’

                                            time⇒ d0

file inclusion file is opened to find macro definition ’mw.qp[@timefilter]’

timefilter⇒ time=(1:30,90:120,180:210)

command subst. output of the named task inserted to complete expression ’lw.qp[time=(’tfilter’)]’

tfilter⇒ 1:300,900:1200,1800:2100

1. text replacement: This is by far the most common type of macro. When a defined macro name is encountered in a QPOE expression, the interface searches the QPOE file and any macro definition files (see section A.3.1) for a macro definition with the same name, and the name is replaced by the definition of the macro. For example, EUVE QPOE files have macros that assign a mnemonic name to each attribute. The macro “x” is defined in EUVE QPOE files as “s8”, the short integer attribute beginning at byte 8 of each record. When the expression [x=50:100] is interpreted, the interface converts it to [s8=50:100] before it is applied to the event list.

2. file include macro: This looks the same as the IRAF mechanism for suppling a list of arguments from a file, arguments, to a task parameter with “@arguments”. Instead, the file should contain one or more complete keyword expressions. As always, the filename must include the complete path to the file from the current working directory. For example, a user who wanted to create a number of images from different time-slices of his QPOE file could use the command:

          im> imcopy 'mw.qp[@home$timeseg]' mw orb30-33

where the file timeseg is in the IRAF home directory, and contains a keyword expression of the form: time=(774683832.7481:774685492.6521,774689532.3321:774691192.2361, 774695231.9169:774696886.7002...

⋮

The QPOE interface converts all EOL characters to spaces. This example also makes use of text substitution macros: the string ‘time’ is a macro for ‘d0’. Up to 20 levels of nested macros are permitted.

5/97

3. command substitution: The macro reference is given as a cl command surrounded by quotes, which is replaced by the output of the command. For example the expression:

                [block=4,time='tfilter']

where tfilter is a task which renders a table of start and end times into a set of time intervals in QPOE filter format (see section 2.2.3). The parameter file of the task to be run may be set up beforehand, or the task may be referenced with arguments, as in:

                [block=4,time='tfilter(sw gt,my gt mode=a)']

This includes setting the task mode to a (for automatic), to suppress queries. Since there are not many commands that produce output in valid QPOE expression format, use of this type of macro is limited.

In all three macro types, the replacement text may contain embedded spaces, tabs, and newlines, each of which is converted to just one space on expansion. Recursion is also allowed, meaning that the replacement text may contain references to other macros, which will be expanded as encountered.

A.3.1 Global macros and the QPDEFS file

The QPOE interface recognizes two categories of macros: “global” macros, which reside in a separate file and may be referenced in any QPOE expression, and “local” macros, which are stored in the QPOE file and are specific to that file. Macros may be defined in more than one place; for example, a macro called time may be defined in both a QPOE file and in an external file such as the standard file QPDEFS. If a macro of the same name is defined in more than one place, then a locally defined macro takes precedence over a global macro of the same name.

Global macros can be defined in external ASCII files, called macro definition files (MDF’s). Such macros are global because they are available to any QPOE file expression. The interface looks at the cl environment parameter qmfiles to find files which contain global macros. The value should be a comma-separated list of files to be searched for macro definitions. IRAF virtual paths are allowed. There is no default setting for this variable; it must be explicitly set by the user. If multiple definitions of the same macro exist among the macro definition files available, the last occurrence, as determined by the order of filenames in qmfiles, is used. An existing local QPOE file macro with the same name takes precedence over a global macro in an MDF.

Definitions of macros in an MDF have a different form than macros installed directly into a QPOE file. Macros in MDF’s must be defined with the syntax:

           define macroname <replacement text>

5/97

QPOE Interface Parameters A-13

with one such definition per line.^{3} Remember that any MDF with macro definitions must appear in the list displayed by

          eu> show qmfiles

for the macros to be applied. Macro definition files may also contain lines which set the values of QPOE interface parameters, as discussed in section A.4 below.

An example of a macro definition file exists in all IRAF environments, in the file /qpoe/QPDEFS. It contains a set of QPOE interface parameter settings and some examples of macro definitions, but its contents are not required to run any of the IRAF/EUV software packages. It’s just a handy template for making one’s own macro definition files. You can copy the file into your IRAF startup directory and add definitions as you create global macros. If you use only your personal version of QPDEFS to store global macros, qmfiles can be set with the command:

          cl> set qmfiles = "home$QPDEFS"

which can be added to your login.cl file.

When the QPOE interface encounters a macro name in parsing an expression, the QPOE file being accessed is searched for the macro definition. If it is not found, then the interface searches the macro definition files named in qmfiles. This is why local macro definitions take precedence. Be aware, however, that if you change the contents of a macro definition file during an IRAF session, the changes may not take effect until you reset qmfiles or restart IRAF.

Note on MDF’s for IRAF/EUV users

The euv package contains a package-wide MDF, in euv$lib/qp.defs. Whenever someone starts the euv package, the variable qmfiles is checked – if it already has a value, it is not changed, but otherwise the value is set to point to the qp.defs file. Part of the function of this MDF is to expand the system ability to deal with long QPOE macros; an ability many GO’s will need. Therefore, users who create new MD files should always include the standard euv file euv$lib/qp.defs in the list in qmfiles (see example A.2).

A.4 QPOE Interface Parameters

The QPOE interface has a number of parameters that set assumed sizes for buffers, QPOE file buckets, and other I/O tools. These values are normally stored in a part of the QPOE file that is transparent to users. When an EUVE QPOE file

5/97

3This syntax is different from that used in file include macros–files included in [ ]’s must contain complete keyword expressions.

is converted to a FITS file using qpwst and stwfits, the list of interface parameter values is saved in a binary table extension. Compare the following list with the list of entries in qp AAA qpparam.tab in Appendix B, section B.5.1 of the EGO Center Data

Products Guide, version 1.5.A.

   bucketlen     - bucket length, number of event records
   cachesize     - of file descriptors in lfile cache
   indexlen      - of hash entries in symbol table index
   maxlfiles     - number of lfiles in datafile
   pagesize      - size, bytes, of datafile
   sbufsize      - symbol table size
   stablen       - symbol table string buffer size, chars
   progbuflen    - program buffer size (compiled instructions)
   databuflen    - data buffer size (initialized data space)
   nodeffilt     - the use of any default event filters
   nodefmask     - the use of any default region masks
   maxpushback   - characters pushback (for macro expansion)
   maxfrlutlen   - full resolution lookup table length
   maxrrlutlen   - reduced resolution lookup table length
   lutminranges  - ranges required before a lookup table is used
   lutscale      - factor to convert nranges to n LUT bins
   blockfactor   - blocking factor for output pixel arrays
   optbufsize    - I/O buffer size for IMIO access to QPOE file, chars
   debuglevel    - debug level, 0 for no runtime debug messages

Each file carries a list of interface parameter values appropriate for processing that file, which is not seen in the output of imhead. Default values of QPOE interface parameters can be set globally by inserting the syntax:

           set parameter value

in a user-defined macro definitions file, such as QPDEFS. Any such file must be included in the value of the environmental parameter qmfiles. Most of the time, most of these parameters are set at the system level, and need not be changed. However, the application in section A.4.1 (below) is one instance in which GO’s may need to alter the values of some interface parameters.

A.4.1 Accommodating long macros

One situation in which it may be necessary to reconfigure the QPOE interface is when one wishes to use a very long macro, such as one with a large number of short intervals. In terms of the usual default settings of interface parameters, a long macro means something over 8000 characters, or 64,000 characters with the euv package tasks installed.

A symptom of exceeding the limit is an error when trying to make an image from

5/97

the QPOE file with the commands qpmkim or imcopy. The error message:

           ERROR: Cannot open pixel file

is most common. It is also possible that there will be a different error message or no message at all. The command will finish unusually quickly when this problem arises, and the image produced will be incorrect or unusable. This problem will also occur with other commands, such as display. When the problem arises, the user can generally accommodate the large filter by adjusting some of the QPOE interface parameters. The QPOE interface is not designed for simple adjustment parameters by users, and GO’s who do not have this problem will definitely want to skip the rest of this section.

It is difficult to devise values for the interface parameters which will work for all possible filters, so the values suggested below should be considered as starting points for your empirical research. To help you in this endeavor, we first, give an (optional) detailed explanation of the interface parameters involved, and follow this with a step-by-step recipe for changing the parameters. Please refer to Chapter 2 for definitions of terms regarding QPOE files with which you are unfamiliar.

Optional: Why long macros don’t work

There are two stages at which a QPOE macro can be too large. The first is during the macro text replacement process. The maximum number of characters which can be pushed back into the input stream when replacing the name of a macro with its text is set by the parameter maxpushback. The default value for maxpushback is normally 8192 characters. When the euv package is loaded, this parameter is reset to 65536 characters. You can check the size of the macros in a QPOE file with the task imhead, as in:

          im> imhead foo.qp long+

Find the macro name at the left, near the bottom of the listing. The number in square brackets is the size in characters. If this exceeds the value of maxpushback, you will get an error.

The obvious solution is to look at your macro lengths and set maxpushback to a value larger than the largest macro you will be using. Then this stage of the filtering process cannot fail. This may solve the problem, or it may result in a different error message from the next stage of the interface program.

The second time the filter can fail is when the expression is compiled. When reading a QPOE file, keyword and filter expressions are translated into an interpreted program and stored in an internal buffer. This buffer must be large enough to hold the program; the more intervals in the filters, the more space it needs. The buffer size is controlled by the parameters progbuflen and databuflen, with default values of 8192 bytes each. A related parameter is maxrrlutlen, which controls nesting of

5/97

lookup-tables (don’t worry about what this means) and has a default value of 1024.

The method for setting all these parameters to values other than their default is to edit one of the macro definition files, or MDF’s (see section A.3.1), which are pointed to by the QPOE environment variable qmfiles. The default MDF supplied with the PROS xray package is called QPDEFS. Macro definition files may contain lines of the form:

           set foo bar

where foo is the name of a QPOE interface parameter and “bar” is the value to which it is to be set. For example, if you look at the file euv$euv.cl, which is executed when you start the euv package, you will see the line:

           set qmfiles = ("euv$lib/qp.defs")

which places the file euv$lib/qp.defs on the list of MDF’s. If you look at euv$lib/qp.defs, you will see that it contains only two lines (the first is a comment):

        \# set up the 64k buffer for qpoes macros
        set maxpushback 65536

This is how the parameter maxpushback is increased from 8192 (its default) to 65536 characters when the euv package is loaded.

Unfortunately, because there are no tasks that will directly probe the length of macro programs, their data buffers, or their nesting habits, there is no easy way to predict the buffer lengths required for progbuflen, databuflen, and maxrrlutlen when using any particular filter. You will need to use trial and error to arrive at good values. What you must do to choose your own values is reset qmfiles to point to an MDF of your own which contains lines to change the values of all the parameters of interest to you. The step-by-step recipe given next explains how to do this.

Example A.2 : Adapting the QPOE interface for large macros

Here are step-by-step instructions for changing the parameters from inside IRAF:

1. Copy the file euv$lib/qp.defs to your IRAF home directory:

               cl> copy euv$lib/qp.defs home$qp.defs

2. Edit the file qp.defs in your IRAF home directory. Change the value for maxpushback to a number larger than your longest macro (if you don’t know what this means, it is described further above). Add the three lines:

5/97 Trouble-shooting your QPOE interface expressions A-17

             set    progbuflen      50000
             set    databuflen      150000
             set    maxrrlutlen     8192

to the file. The suggested values ought to work with quite large filters.

3. Change qmfiles to point to your file

               cl> set qmfiles = "home$qp.defs"

Be sure not to reload the euv package after this or you will overwrite the value of qmfiles with the euv package default value.

4. That should do it. Go to your data directory and try your large filter. If it still doesn’t work, re-edit the file and increase progbuflen and databuflen to larger values. It shouldn’t be necessary to increase maxrrlutlen further. If you need more information to choose appropriate values for the parameters, read the section above on the gory details.

If you need to use large filters all the time, you can place the command in step 3 into your login.cl file, where it will be executed every time you start IRAF. If you choose that option, and you also load the euv package in your login.cl file, be sure to set qmfiles after the euv package is loaded, to prevent overwriting qmfiles.

A.5 Trouble-shooting your QPOE interface expressions

The QPOE interface can seem quite finicky and obstinate about accepting your expression syntax. Here is a short list of error messages, and some (incomplete) suggestions about may be wrong.

Unfortunately, it is also known that sometimes the QPOE interface produces some output, but not the right or expected output, and gives no error message whatsoever. You have probably already read about some of these possibilities elsewhere in this User’s Guide. The best insurance against unknowingly using bad results from QPOE files is to test every product as carefully as possible and as soon as it is created. Address your unsolved problems with EUVE QPOE files to egoinfo@cea.berkeley.edu; we want to know about them, and you can usually get help.

PS: Don’t forget to flush the IRAF process cache:

          eu> flprc

whenever an attempt to run any task fails.

   ? ERROR on line 70:  QPEX: bad range specifier in event attribute filter

There may be misspelled attributes or macro names. The [ ]’s around a rect specifier may be missing or replaced by ( )’s. This message can also appear

5/97

when an xray task gets some input it isn’t expecting. If using xtiming.timsort, it may not like the way you’ve specified the output event record in esize, or the number in ssize may be too small. It will also occur if there are non-attribute keywords in deffilt.

? QPIO: unexpected token `:'

Some ( )’s may be missing around some filter string with commas in it.

? ERROR: Newline while processing string

The single quote is probably missing from the end of a QPOE expression used on the command line.

? ERROR: Cannot open pixel file

This often means that the task was not expecting a QPOE, but an image file, because the “.qp” suffix was omitted from the input file name. However it may mean that a macro used in a QPOE expression is too long to be parsed by the interface– see section A.4.1.

? ERROR: segmentation violation

This may mean that your directories are nested too deeply, and you have created a path length to the QPOE (or other file) that exceeds the buffer size in IRAF.

5/97

Appendix B

PLOTTING ST TABLE COLUMNS WITH TBPLOT.IGI

The interactive graphics interpreter in the tables package, tbplot.igi, is a versatile plotting tool capable of handling data in many IRAF formats. GO’s are particularly encouraged to use igi in situations requiring a flexible graphic representation of data in ST table format. igi is capable of reading data from images or ST tables and allows the user to plot any two image coordinates or columns against each other. In this Appendix, we describe using igi to plot various columns of EUVE ST tables. This is a good alternative to using dqselect as a display tool, since igi reads the entire column instead of sampling the input. This means that, by adjusting plot limits for the data set and the display device, one can view at least a subset of the table data at full resolution. By contrast, display in dqselect is always limited to the sample of data points read in when the table is first loaded on startup.

One caveat: igi cannont handle double precision numbers. This means that to plot any table column with doubles, such as time columns in EUVE ST tables, the user must first add a column to the table with the values offset by the first value, and converted to real numbers. This is quite simple to do with tasks in tables.ttools:

1. Look at the table using tread, and record the first time stamp, T_{0}.

2. Use ttools.tcalc to add a new column to the table with the difference:

                timecolumn - T_{0}

3. Check the new column using tread. If, for example you want to plot an aspect table, with times every major frame, the new column should have the values 0, 1.024, 2.048, etc., with orbit daytime gaps occurring about every 3600 lines.

When you have done any necessary preparation on the table, load the tbplot package and start igi. This will put you in interactive mode with a status line at the bottom of the STDGRAPH screen.

          tt> tbplot
            axispar dvpar igi pltpar sgraph
          tp> igi

(switch to STDGRAPH)

           STScI igi Version 24Jan94 -- type HELP for info, END to stop igi>

5/97

B-2 APPENDIX B. PLOTTING ST TABLE COLUMNS WITH TBPLOT.IGI

The task uses the standard commands available in the mongo graphics command interpreter, with extensions to take advantage of IRAF facilities like “snap”. All the commands can be listed by typing the help command with no argument. Type help command to see the message for command. Help messages are directed to the text window; use two ‘q’ commands to return to the interactive mode.

The following lines give a basic script for loading the table mkaspectds.tab, which was produced for the Deep Survey pointing by the example parameters in figure 1-8. The table has had a real-number time column added, and the user first plots the roll angle column against this time. The left column shows command syntax, with comments on the right.

data mkaspectds  Causes igi to locate the table.
xcol reltime  igi reads the named columns into its plot buffers.
ycol roll  Column names are not case sensitive.
limits  igi finds the limiting values of each buffer.
erase  Erase the graphics frame before plotting.
box  Draw box with labeled tick marks using limits.
connect  Connects all data points.

The plot in figure B-1 shows a gap in the roll angle reports where the telescope was slewed away from the target, perhaps for an engineering or calibration test.

Figure B-1: ROLL vs. time - T_{0} in igi. There has been a slew away from the target, as seen by the gap in the roll. The resulting large ROLL scale does not show smaller drifts of 7 − 11′′ ⌢.

5/97

APPENDIX B. PLOTTING ST TABLE COLUMNS WITH TBPLOT.IGI B-3

Telemetry from the test period would be omitted from that delivered to the GO^{1}. The user can enter IRAF interactive cursor mode with the curses command and use cursor reads (C) to determine times just before the start and after the end of the “bad” interval.

The user can then search the times in table1 for the expected gap, remembering that the value of T_{0} must be added to times in the plot. It is a good idea for GO’s with interrupted pointings to make sure all off-target telemetry has been omitted from the telemetry, and that any parts of the telescope slews have been masked out with a time filter on the QPOE.

Plotting the RA vs. Dec from the mkaspectds.tab table can also give a good idea of how stable pointing was during an observation, and the size of any drifts. Without exiting from igi, the user now loads the RA column of the table into xcol, and the Dec column into ycol, and recomputes the limits with the new data vectors before plotting them as in figure B-2.

Figure B-2: RA vs. Dec for an interrupted observation. Because of the excursion, most of the pointing has been compressed into the upper left corner.

This is a “worst case” example, with the telemetry from the test included. The plot limits and area are magnified to include a location very far from the target coordinates, and the target pointings are compressed into the upper left corner of the plot. This is the kind of situation where the flexibility of igi is useful. The plot limits in igi can be reset to show the area around most of the pointings in greater

5/97

1Observations interrupted by tests are extended by the duration of the test to ensure the full awarded time.

detail. igi will use the full resolution of the graphics display to plot as many of the data values as possible in the new range, as shown in figure B-3.

Figure B-3: Plot of RA vs. Dec for a dithered observation. The plot area is expanded about the target coordinates to show the dithering pattern of three clusters of pointing locations.

The commands for producing this plot are:

curses  Enter interactive cursor mode
C C  Read coordinates of lower left and upper right

 limits .058 .134 5.75 6.45  Close in on this area

era  Abbreviations allowed
box  Draw new coordinate axes
conn  Draw new plot

This view shows that the observation seems to consist of several pointings, that range over about .067 hours in RA and 3/4 degree in Declination, with small clusters of corrections or dithers about each main point. Normal dithered GO observations should vary within ±1 − 2′ ⌢ of the target coordinates. A hard copy of the plot with some labels can then be produced by the commands:

5/97

xlabel RA (hours)  Label under X axis
ylabel Declination (degrees)  Label to the left of Y axis

limits 0 1 0 1  Simplifies scales for locating labels in the box
relocate .15 .8  Go to desired label location

label Beta CMa pointing 3/13/95  Label the plot

curses  Shift to IRAF cursor mode
:.snap  Send snapshot to hardcopy device
q  Return to igi command line mode
end  Go back to IRAF command line

A plot of RA vs. Dec in igi for a typical dithered observation is shown in Chapter 1, figure 1-10.

Tips for new igi users

▽ After the curses command, all the regular capital letter graphics commands in the IRAF interactive cursor mode, e.g. C,X,Y,Z,M, etc., can be used to display more detail and so take advantage of the data resolution. The X-Y aspect is not preserved by the X and Y commands.

▽ Arguments to the label command can use a number of alternate fonts, each denoted with a letter and with a ‘\’. An Angstrom symbol is produced by the capital ‘V’ in the greek font: \gV.

▽ Superscripts and subscripts are also produced by \ escapes. The \u (for “up”) in “cm\u2” produces cm^{2}, and the \d (for “down”) in “T\d0” produces T_{0}.

▽ The user can edit, replay, and write the history buffer to a file:

list Lists previous commands, as in mongo.

edit Puts you into the buffer in your default editor (text screen). playback replays the command history. write plotit save this buffer to file plotit.

▽ Internal macros can be created with the define command:

              igi> define limbo
              macro> xcol &1
              macro> ycol &2
              macro> limits; erase; box
              macro> end
              igi> limbo table1.tab 3 5

creates the macro limbo and uses it to draw a box with labeled axes.

5/97

B-6 APPENDIX B. PLOTTING ST TABLE COLUMNS WITH TBPLOT.IGI 5/97

Appendix C

OBSOLETE IRAF/EUV TASKS

The following tasks have been moved to the euv subpackage obsolete. Their use is not recommended in any current processing described in this document, but they may still be used with older EUVE data sets.

1. listspec is not compatible with the EUVE spectral data products in multispec format.

2. euvcombine has been replaced by suborder and divea.

3. mkeuvspec is no longer needed, due to improvements in the task specmod.

4. editwcs is not fully functional, and not needed for normal data processing.

In addition, the task rowcount is no longer available through the user interface, but is still included with the software. The following sections have not been substantially updated since the release of the Software User’s Guide, version 1.4.

C.1 LISTSPEC Lists Spectra in WCS Units

The task euvtools.listspec is analagous to images.listpixels. It takes a 1-d spectrum as input and creates a listing of wavelength vs. the pixel values, counts or flux, according to the input. listspec can determine the wavelengths from the WCS in the image header, and use a bin size supplied by the user in the parameter dx to calculate the appropriate fluxes by linear interpolation. The user also has the option of setting the parameter average to divide each value by CD1 1, the bin width in Å. This produces an average over the bin represented by each pixel, and converts a spectrum image in photons/cm^{2}/sec to an ASCII listing in photons/cm^{2}/sec/Å. listspec can be used to list the output from mkeuvspec for comparison to the input model, or applied to flux-calibrated 1-d EUVE spectra of continuum sources to convert them to the same units as model spectra, as in the following example:

          eu> listspec g191 all x1=70 average=yes > g191 list

C.2 EUVCOMBINE Joins Channel Spectra

GO’s whose spectra have very good signal-to-noise can use the task euvcombine to join their background-subtracted EUVE spectra before using spectral comparison

5/97

C-2 APPENDIX C. OBSOLETE IRAF/EUV TASKS

tools. This task uses the instrument effective areas for the three channels to adjust the levels of overlapping spectral regions and combine the three channels, with options for fluxing the spectrum in first order and specifying the wavelength range of the output spectrum.

Limitations of euvcombine

Although the functionality to remove counts from orders other than first order has been implemented in euvcombine, this algorithm does not work reliably because of problems with noise and behavior near sharp spectral edges and features.

Therefore, euvcombine should only be used when the effects from higher orders can be safely ignored, as in a continuum with very little short and medium wavelength flux, or easily separated, as with an emission source that has no lines that overlap with higher order features. In these cases, the EUVE spectra can be joined using euvcombine, but dividing by only the first order effective area. This produces a flux-calibrated first order spectrum that can be compared directly to model spectra. GO’s should look at the ST tables sw ea.tab, mw ea.tab, and lw ea.tab in the reference data to see the relative efficiencies for higher orders.

An example parameter file setup for using euvcombine is shown in figure C-1.

          eu> lpar euvcombine

      spectra = "g191sim.sw,g191sim.mw,g191sim.lw" Input spectra raw count 1d i
    detectors = "sw,mw,lw"      Input detectors for images
       output = "g191sim_com3"  Output spectra 1d image
     (refdata = "reference/detector.tab") Detector table
    (exposure = 40000.)         Exposure time in seconds
   (maxorders = 1)              Maximum number of orders
      (counts = no)             Output counts
    (autosize = no)             Determine output range from inputs
    (resample = 2)              Sampling factor to conserve memory
(minwavelengt = 70.)            Minimum output wavelength
(maxwavelengt = 760.)           Maximum output wavelength
        (mode = "al")

Figure C-1: Sample parameters for combining three spectra with euvcombine.

Notes on running euvcombine

□ Both spectra and detectors should be specified in order of increasing wavelength, and combinations of adjacent channels are allowed, as in (sw,mw,lw), (sw,mw), or (mw,lw).

□ When counts = “no”, the parameter exposure is used to compute photon fluxes at the telescope. In this case, the raw counts in each bin of the output spec-

5/97

MKEUVSPEC C-3

trum are divided by exposure and the first order effective area to compute the intensity in photons/cm^{2}/sec for each pixel.

□ maxorders was intended to determines the maximum spectral order to be included in the combined spectrum. The value of maxorders should always be set to 1, so that all counts are assumed to be in first order and none are subtracted (see the box above). If it is an integer greater than one, euvcombine attempts to remove counts from higher orders using effective areas for orders 2–maxorder, but it may misjudge the first order signal if it has a significant fraction of noise, and will then subtract too many counts. GO’s with bright sources who ardently desire to remove higher orders are strongly urged to use the task suborder.

□ Set counts to “yes” to make a raw count spectrum, “no” to make a flux calibrated spectrum.

□ Set the range of the output spectrum in Å using autosize, to automatically select limits from the input images, or explicitly, using minwave and maxwave.

□ The parameter resample, which specifies the binning of the output spectrum, is not currently implemented. The resolution of the output spectrum defaults to the lowest resolution present in the input spectra.

□ The value of table should be set to the location of the ST table detector.tab in the directory reference. The table contains pointers to instrument reference data. The value of table should include the “.tab” extension.

To combine three channel spectra from a white dwarf observation:

      eu> euvcombine grumpy.sw,grumpy.mw,grumpy.lw sw,mw,lw grumpy.com
      exposure=39873 maxorders=1 counts-

To combine raw counts from the medium and long wavelength spectra of a late star:

      eu> euvcombine latest.mw,latest.lw detectors=mw,lw counts+

C.3 MKEUVSPEC Creates Images from Model Spectra

If the source model (to be input to specmod, version 1.4) represents a continuum with units of photons/cm^{2}/sec/Å, or an emission source with broad or blended features, it must first be rebinned into a 1-d spectral image using the task euvtools.mkeuvspec to supply data in the correct units^{1}. mkeuvspec treats an input list of fluxes as a piecewise continuous spectrum. It integrates and rebins the flux into bins with a size specified by the user.

Continuum input is simulated by a sufficiently dense array of lines, where sufficiently dense means that the line spacing is narrow compared to the spectrometer PSF that

5/97

1No longer true. See section 4.1.2.

will be applied to the output by specmod ^{2}. To convert a model continuum spectrum called cont.model, edit the parameter file for mkeuvspec as shown in figure C-2.

          eu> lpar mkeuvspec

       inputs = "cont.model"    Input file list
      outputs = "cmod_in"       Output file list
        (xcol = 1)              Wavelength column
        (ycol = 2)              Y value column
          (x1 = INDEF)          First bin's wavelength
          (x2 = INDEF)          Last bin's wavelength
          (dx = 0.05)           Wavelengths per pixel
    (linelist = no)             Line list or plf
   (integrate = yes)            Is the output to be multiplied by wpc
        (mode = "al")

Figure C-2: Sample mkeuvspec parameters to make an IRAF image from an ASCII spectrum.

Notes on running mkeuvspec

□ Letting x1 and x2 equal “INDEF” causes mkeuvspec to determine the wavelength range from the first and last values in the file.

□ Setting linelist to “no” tells it to treat the input as a continuum and do a linear interpolation between data points.

□ Setting integrate to “yes” causes the function to be integrated over each bin, changing the units of photons/cm^{2}/sec/Å to photons/cm^{2}/sec.

□ To produce input to specmod, the value of the bin size in dx should be no more than one fourth of one resolution element in the simulated EUVE spectrum (see footnote). Values comparable to the resolution size may cause the bin size of the spectral image to “beat” against the PSF used in specmod, producing spurious features.

Creating simulated EUVE spectra of a continuum model in photons/cm^{2}/sec/Å from the command line requires two steps:

          eu> mkeuvspec contin 1e18 contin18 dx=.05 linelist=no integrate=yes
          eu> specmod cont18.imh contin18 sim time=50000 wavbin=.2

The use of specmod is discussed in section 4.1.2.

Making emission line lists into IRAF spectra for use as input to specmod is not recommended, as it will either produce inaccurate results (input bins larger) or slow down processing by specmod (input bins smaller). However, if an emission spectrum has lines or blends which already span several bins, the spectrum should be treated as a continuum by setting linelist to “no”, and integrate to “yes”. If emission spectra in photons/cm^{2}/sec are to be rebinned for some other purpose, set the parameter linelist to “yes” and integrate to “no”. Then each line will be recorded in only one bin in the output spectrum.

2The spectrometer resolution element is equivalent to the half width of the local PSF. Resolution elements in the short, medium and long wavelength channels are approximately .5 Å, 1.0 Å, and 2.0 Å, respectively.

5/97

5/97

C-6 APPENDIX C. OBSOLETE IRAF/EUV TASKS 5/97

Appendix D

OTHER RESOURCES

D.1 Other Useful IRAF Packages

D.1.1 Basic image operations with generic IRAF tasks

There are a great many facilities in IRAF that you can use to move, copy, and change the one- and two- dimensional spectral images. A number of tasks in the images package are often useful:

blkavg - Block average or sum a list of N-D images
boxcar - Boxcar smooth a list of 1 or 2-D images
 fit1d - Fit a function to image lines or columns
 hedit - Header editor
imarith - Simple image arithmetic
imcombine - Combine images pixel-by-pixel using various algorithms
imcopy - Copy an image
imdelete - Delete a list of images
imgets - Return the value of an image parameter as a string
imheader - Print an image header
imhistogram - Compute image histogram
imrename - Rename one or more images
imstatistics - Compute and print statistics for a list of images
 imsum - Compute the sum, average, or median of a set of images
listpixels - Convert an image section into a list of pixels
minmax - Compute the minimum and maximum pixel values in an image
sections - Expand an image template on the standard output

D.1.2 Operating on EUVE spectra with the onedspec package

Many of the tasks in the onedspec package can be used on EUVE spectral images, as the WCS is compatible with the onedspec spectral format. Tasks include profile fitting, spectrum arithmetic, line identification, and display routines. The following is an abbreviated listing of some of the useful tasks for 1-d specta.

continuum - Fit the continuum in spectra
fitprofs - Fit gaussian profiles
sarith - Spectrum arithmetic
scombine - Combine spectra having different wavelength ranges

5/97

D-2 APPENDIX D. OTHER RESOURCES

scopy - Select and copy apertures in different spectral formats
setjd - Compute and set Julian dates in images
sfit - Fit spectra and output fit, ratio, or difference
sinterp - Interpolate a table of x,y pairs to create a spectrum
slist - List spectrum header parameters
specplot - Stack and plot multiple spectra
splot - Preliminary spectral plot/analysis

splot and specplot are also introduced briefly in section 1.2.1 of the EGO Center Data

Products Guide.

D.1.3 Manipulating QPOES with PROS XRAY tasks

The xray package contains a number of subpackages and many tasks that are written for use with QPOE files and the IRAF/QPOE interface. Many of the xray package tasks assume that the QPOE file input has a standard EINSTEIN or ROSAT event definition, making them unusable with EUVE QPOE’s. However, euv/xray compatibility has recently been improved by the addition of the header keywords XS-SORT and XS-EVENT to EUVE QPOE files.

The tasks most often used with EUVE QPOE’s are in the packages xspatial and ximages. The complete list of tasks is displayed by typing ‘help xray’. Those with an asterisk may be more useful to EUVE GO’s.

rootxray.xray:
xapropos - List all tasks related to a given subject
packages:
xdataio - Tasks to convert data to/from FITS format
xplot - Plotting routines for XRAY data
xproto - Crude tasks, currently under development
xobsolete - Tasks that will soon be removed from the system
*      ximages - Additions to IRAF images package
imcalc - IRAF image calculator
plcreate - create a pixel list from a region
pllist - display a pixel list
qpsort - sort a qpoe file
qphedit - edit header parameters in a qpoe or image file
*     xspatial - X-ray spatial analysis package
wcscoords - use WCS to convert between IRAF coordinate systems
(logical,physical)
xspectral - X-ray spectral analysis package

5/97 Other Useful IRAF Packages D-3

*      xtiming - X-ray timing analysis package
          fft      - compute fast fourier transform on table or qpoe data
          timsort  - create a time-sorted qpoe file for timing analysis

The xray package also has several on-line help documents on various related subjects.

( Type help <topic> for info on any of the following: )
pros          -     overview of all IRAF/PROS tasks
*     bugs           -  description of known IRAF/PROS bugs
coords         -  description of IRAF coordinates used in PROS
exposure       -     description of PROS exposure correction support
explain_errors -  description of PROS error computations for low
                    count/bin data
explain_xdata  -  description of new RDF file formats
explain_screen -  description of PROS support for temporal screening
extensions     -  description of conventions for PROS file extensions
file_compare   -  comparison of new/RDF formats to OLD formats in PROS
*     filter         -  description of filtering options in PROS
missions       -  list of missions and instruments support by PROS
prf            -  information on the Point Response Function (or psf)
*     qpoe           -  description of the QPOE data file and user interface
*     regions        -  description of PROS region masks
release_xray   -  release notes for this build
revisions      -  revisions for the patch

D.1.4 The TABLES layered packages

Several of the tables tasks can be useful with the EUVE telemetry tables, or to construct limit and filter specification tables. Load the tables package and type ‘help fitsio’, ‘help ttools’, and ‘help tbplot’ to produce a listing of tasks in each subpackage. Tasks which are referenced in this document and the EGO Center Data Products Guide are marked with an asterisk (*).

    ta> help fitsio

tables.fitsio:
Menu for the package tables.fitsio
catfits - Produce a catalog of FITS files on tape or on disk
fitscopy - Copy FITS files: disk <--> tape and disk <--> disk
fits_exampl - Detailed examples of `fitsio' tasks
 geis - Provide background information about GEIS format files

5/97

   gftoxdim - Convert a GEIS file to one with xtra dimension plus a table
*   stwfits - Convert tables and/or GEIS files to FITS files
*   strfits - Convert FITS files to tables or to GEIS files
    r49fits - Same as strfits but handle WFPC2 data correctly.
   xdimtogf - Convert a GEIS file with xtra dimension plus a table to GEIS
(Type "help fitsio opt=sys" for more information about the fitsio package)
          ta> help ttools

tables.ttools:
Menu for the package tables.ttools
       keypar - Copy an image or table header keyword to an IRAF parameter.
    keyselect - Copy selected image header keywords to sdas table.
       keytab - Copy an image or table header keyword to a table element.
       parkey - Put an IRAF parameter into an image or table header keyword.
       partab - Transfer an IRAF parameter to a table element.
       tabkey - Copy a table element to an image or table header keyword.
       tabpar - Transfer a table element to an IRAF parameter.
       gtedit - Graphically edit a table.
        gtpar - Pset to specify graph parameters for 'gtedit' task.
        imtab - Copy an image to a table column.
        tabim - Copy a table column to an image.
*       tcalc - Perform arithmetic operations on table columns.
       tchcol - Change column name, print format, or units.
*      tcheck - Check STSDAS table element values.
      tchsize - Change allocated sizes of various sections of a table.
        tcopy - Copy tables.
*     tcreate - Create a STSDAS table from an ASCII descriptor table.
      tdelete - Delete tables.
      tdiffer - Form a table which is the difference of two tables.
*       tdump - Dump the contents of a table to an ASCII file.
*       tedit - Edit a table.
      texpand - Expand tables according to a set of rules.
   thistogram - Make a histogram of a column in a table.
*       tinfo - Display table size information.
   tintegrate - Numerically integrate one column with respect to another.
        tjoin - Perform a relational join of two tables.
*       tlcol - List column information for a table.
      tlinear - Use linear regression to fit one or two table columns.
       tmatch - Find closest match between rows in two tables
       tmerge - Either merge or append tables.
       tprint - Print tables--both headers and data.
     tproduct - Form the Cartesian product of two tables.

5/97 Electronic Resources for Guest Observers D-5

     tproject - Create new table from selected columns in a table.
       tquery - Create a new table from selected rows and columns in a table.
*       tread - Browse through a table.
       trebin - Resample a table to uniform spacing.
      trename - Rename tables.
      tselect - Create a new table from selected rows of a table.
        tsort - Sort a table.
*       tstat - Get mean, standard deviation, min, and max for a column.
   ttranspose - Transpose or flip a table.
        tupar - Edit table header parameters.
          ta> help tbplot

tables.tbplot:
Menu for the package tables.tbplot
         axispar - Specify scaling attributes (pset for 'sgraph')
         devpar - Specify device parameters (pset for 'sgraph')
*           igi - Interactive Graphics Interpreter.
         pltpar - Specify graph parameters (pset for 'sgraph').
         sgraph - Graph one or more image sections or lists.
(Type "help tbplot option=sysdoc" for detailed information about 'tbplot').

The interactive graphics interpret, igi, is the subject of Appendix B.

D.2 Electronic Resources for Guest Observers

The EUVE Guest Observer Center makes a number of resources available through the ftp server and World Wide Web (WWW) pages of the Center for EUV Astrophysics.

∘ The Universal Resource Locator (URL) for the EGO Center WWW page:

      http://www.cea.berkeley.edu/

telnet www.cea.berkeley.edu 200 For those without a WWW browser, this will start a session with the all-text browser lynx.

∘ EUVE Guest Observer Center HomePage:

      http://www.cea.berkeley.edu/ego/HomePage.html

∘ anonymous FTP:

      ftp ftp.cea.berkeley.edu
               Name:  anonymous
               Password:  type_your_e-mail_address

5/97

∘ anonymous gopher:

  gopher ftp.cea.berkeley.edu

∘ EUVE Electronic Newsletter:

Past issues of the EUVE electronic news are available from the CEA WWW site (URL above). Click on \underline{EUVE Public Archive of Mission Data and Information}, then \underline{Documentation}, then on \underline{Electronic Newsletters}. Subscribe to the newsletter by sending mail to majordomo@cea.berkeley.edu with the one-line message

body, subscribe euvenews.

∘ EUVE Guest Investigator (GI) Program: If you want to find out about obtaining public EUVE data, the EUVE GI Program was recently set up to help support users of the EUVE archive. For further information click on \underline{GUEST INVESTIGATOR PROGRAM} in the CEA WWW Archive page, or contact the EUVE public data archive at

archive@cea.berkeley.edu.

∘ Public Public Right Angle Program (RAP): RAP is an easy method for proposing long-exposure observations with EUVE imaging instruments. For information on the Public RAP and its streamlined proposal process, click on \underline{Public Right Angle Program Information} in the EGO Program Web page Quick Reference list, or contact the EGO Center

(egoinfo@cea.berkeley.edu). Mail all proposals to: euverap@cea.berkeley.edu.

D.3 Documents You Can Get From IRAF

The following documents are among those available in the directory iraf/docs/ in the IRAF anonymous ftpsite. The ftp address is:

iraf.noao.edu or 140.252.1

A Beginner’s Guide to Using IRAF,

Version 2.10, Jeannette Barnes, August, 1993, 76 pages. (this file is in iraf/pub/beguide.ps.Z)

A User’s Introduction to the IRAF Command Language

Version 2.3, Peter MB Shames and Doug Tody, August 1986, 97 pp. (iraf/docs/cluser.ps.Z)

The IRAF/NOAO Spectral World Coordinate Systems

Francisco Valdez, 12 pp. (iraf/docs/specwcs.ps.Z)

The IRAF Packages, IRAF Version 2.10,

6 pp. (iraf/docs/glos210a.ps.Z)

5/97 Writing IRAF-Compatible Software D-7

The NOAO Packages, IRAF Version 2.10,

13 pp. (iraf/docs/glos210b.ps.Z)

Rectifying and Registering Images

Lisa A. Wells, April 1994, 29 pp. (iraf/docs/reg.ps.Z)

The IRAF APEXTRACT Package (and revisions notes)

Frank Valdez, July 1990, 11 pp. (iraf/docs/apex.ps.Z)

A User’s Guide to Reducing Slit Spectra with IRAF

(descriptions of apall and splot. Phil Massey, Frank Valdes, Jeannette Barnes, 15 April 1992, 59 pp. (iraf/docs/spect.ps.Z)

VMS/IRAF Installation and Site Manager’s Guide

Doug Today, Suzanne Jacoby, and Nigel Sharp, July 1993, 30 pp. (vmsiraf.ps.Z file size: 262143 uncomp.)

D.4 Writing IRAF-Compatible Software

All these documents and many others can be gotten by anonymous ftp to iraf.noao.edu, in the directory iraf/docs. The name in parentheses is the filename under which the document is stored.

A User’s Guide to Fortran Programming in IRAF, The IMFORT Interface

Doug Tody, September 1986, 29 pages. (imfort.ps.Z)

An Introductory User’s Guide to IRAF Scripts,

Ed. Anderson, revised by Rob Seaman, September 1989,, 49 pp. (script.ps.Z)

An Introductory User’s Guide to IRAF SPP Programming,

Rob Seaman, October 1992, 76 pages. (sppguide.ps.Z)

5/97

D.5 General Information on FITS

The following documents can be obtained from:

FITS Support Office NASA/OSSA Office of Standards and Technology Code 933 Goddard Space Flight Center Greenbelt, MD 20771 USA

Telephone: (301) 513-1634 Internet: fits@nssdca.gsfc.nasa.gov DECnet: NCF::FITS

Anonymous ftp site: nssdca.gsfc.nasa.gov

A User’s Guide for the Flexible Image Transport System (FITS)

Version 3.0, October 1992.

5/97