CalFUSE, the FUSE calibration pipeline, was originally designed before the launch of the satellite. As unexpected instrumental effects were discovered during the telescope operations, changes were made to the pipeline to correct for these, leading to a cumbersome design.
In 2002, a new pipeline design was conceived. The goal was not only to take into account all the newly discovered instrumental effects, but to make it easier to add further instrumental corrections. The new design is more flexible, thus maintenance is easier. Moreover it makes the pipeline work considerably faster.
Up to version 2.4, CalFUSE converts time-tag data to a 2-dimensional image in an early step, making it hard to add a module to correct for time-dependent effects such as mirror motions. CalFUSE 3 keeps the data in the format of a time tagged events list until the extraction. Histogram files are also converted to a list of events, but in that case all events are tagged with the same time value.
This events list is stored in the Intermediate Data File (IDF). This FITS file is an unified input/output format for most of the pipeline modules. The pipeline subroutines read or write the keywords, rows and columns in this file. The unified input/output format for most of the pipeline routines means that the order of operation of the pipeline modules can be changed (to a certain extent) or new modules can easily be added.
The Intermediate Data File (IDF) is at the center of the new design. It is a FITS file with 4 Header Data Units (HDU).
The first HDU consists of the header originally copied from the raw file which gets modified as the IDF goes through the different pipeline steps. It contains information about the proposal, the exposure, the observation and the calibration as well as engineering and housekeeping data.
The second HDU is a time-tagged list of events with their raw XY positions, weights, pulse height, as well as other parameters set to dummy values at the creation of the file (like their XY corrected positions and assigned wavelengths) and computed when the IDF goes through the appropriate pipeline module.
The third HDU is the list of good time intervals.
The fourth and last HDU lists, for each second of the exposure, housekeeping parameters like limb angle, orbital velocity, latitude and longitude, as well as the LiF and SiC Y centroids. If present, a housekeeping file, generated from the engineering telemetry, is used to fill some columns of this time-line. Otherwise these columns are filled with the best guess from the engineering keywords of the main header. This is the first time the pipeline makes use of the housekeeping files. It allows for a time-dependent correction of such effects as dead-time and count-rate Y distortion as well as screening for detector high voltage variations.
Since in the IDF file events are flagged as good or bad, but never discarded, users can change their selection criteria without rerunning the pipeline.
This document deals with the analysis tools that work on IDF files. The tools described are from CalFUSE 3.0.8. If you are using a different version of the pipeline, the examples in this document might not work. Note also that a lot of these tools are not available in pipeline 3.0 before 3.0.8. The goal of this document is to help you: