Data Retrieval Help Page

To retrieve IUE data, mark the rows in the search results page of the desired exposures (or click on the "more retrieval options" page and enter camera and image numbers in the list under "image IDs"). Once the desired files are marked, several options are available for formatting and compressing the data. Generally, MAST files are most efficiently (and quickly) downloaded as a bundled tar or zip file. This means however, that the user is left with the task of extracting the files. The descriptions below list the programs available on different operating systems for reading the various file formats.

Clicking the "Download NEWSIPS MX Files as a .tar file" from the IUE search results page will download a tar file of NEWSIPS FITS files. Users desiring a zip file, or a gzipped tar file, should click the "More retrieval options" button after selecting the desired files from the search results list.

Other options available from this page include downloading NEWSIPS, IUESIPS-GO, or IUESIPS-RDAF format files, selecting various NEWSIPS or IUESIPS file types, or requesting files by entering the IUE camera names and image numbers. Note the current retrieval options allow only one data type and one file format to be requested at a time and the two selected options must agree (e.g. don't select NEWSIPS files and an IUESIPS file type). If both IUESIPS and NEWSIPS files are desired, two separate file downloads are required.

Data Type
Requests can be submitted for either IUESIPS or NEWSIPS files. NEWSIPS files are in FITS format while IUESIPS data is archived in IUE GO format and needs special software to be read. In general, NEWSIPS data is recommended for most users.

Conversion Options
IUESIPS data can be requested in either GO or RDAF formats. The default format is GO. IUE RDAF software uses RDAF format files, however the IDL program GOTORDAF can be used to convert GO format files to RDAF format (see also RDAFTOGO).

File Download Options The current IUE file download options include:

Data Format:
Data Compression Options:
Users know when the request is completed when the browsers "Download window" disappears. Requests for a few files are normally completed in a couple of minutes.

It should be pointed out that the "time Left" gauge in the pop-up window will not accurately show the remaining download time. The file transfer speeds will eventually be displayed, but the download is basically not complete until the pop-up window disppears.

In addition to the requested files, users will also receive an ASCII text file called filetypes.txt which describes the various IUE data types, and, an ASCII text file called status.txt describing the status of the users data request. More information on status.txt is provided below.




UNIX

For unix users, the downloaded tar files can be extracted using the command:
$ tar xvf filename

Gzipped tar files can be read the same way, after first running the command:
$ gunzip filename

Requested ZIP files can be opened using unzip:
$ unzip filename

If FITS files were requested, the uncompressed files can be read with a standard FITS reader (although it must support binary table extensions). If IUESIPS GO format files were downloaded, they can either be read using custom software, or converted to rdaf format using the gotordaf routine from the IUEDAC IDL library. RDAF format files can also be requested directly from MAST.

VMS

Since gzipped, tar, and zip file formats are not as common on VMS systems, users may first need to download the software necessary to extract the downloaded files. GUNZIP, TAR2VMS, and UNZIP are all public domain software available from DECUS. To see if these programs are already available on your system, type:
$ show symbol tar2vms
$ show symbol gunzip
$ show symbol unzip
Depending on the format you select for downloading the files, if the program you want is undefined, (and assuming it doesn't exist under another name) they can be downloaded as follows:

Download the TAR2VMS executable by clicking on the appropriate OpenVMS system below:

  • (VAX tar2vms executable) (15 blocks)
  • (ALPHA tar2vms executable) (20 blocks)

    Once you have downloaded the executable, type the following commands (note the first command defining tar2vms could be added to your login.com file):

    VAX:

    $ tar2vms :== $disk:[account]tar2vms.exe
    $ tar2vms xvf iue.tar
    or ALPHA:
    $ tar2vms :== $disk:[account]tar2vms_alpha.exe
    $ tar2vms xvf iue.tar

    where "disk" and "account" are the names of the disk and directory where the executable resides.(Note the "$" in front of the disk name!) The file "iue.tar" listed above is an example of the file name downloaded from MAST. The actual name will depend on the file format requested.

    The gunzip program (as well as unzip for ZIPed files) can be dowloaded directly from Digital. Be sure to select the appropriate executable for your system (i.e., *-vax for vaxes, *-axp for alphas). To install gunzip or unzip, follow the directions above for tar2vms. That is, define a symbol for the command name (with the "$" added to the file path) and then just type (for example):

    $ unzip iue.zip   or
    $ gunzip iue.tar.gz
    (Note after downloading and unpacking RDAF format IUESIPS files, VMS users will need to convert their files with a program such as CON_RDAF (i.e., using the SPARC to VMS mode option).

    The above commands will create a subdirectory called IUE containing all the requested files. NEWSIPS files will have names in the form swp12345.mxlo, while IUESIPS files will have names such as swp12345llg.dat & .lab, or swp12345.melol.

    IUESIPS files can be requested from the "More Retrieval Options" page in either GO format, or RDAF format. After extracting, GO format files will have names such as swp12345.melol, while RDAF format files will have names such as swp12345llg.dat and swp12345llg.lab. The main differences between the 2 formats is that the RDAF format stores the VICAR label in a separate file from the data and, the data stored in the RDAF .dat files are assumed to be converted to a format compatible with the local operating system. Because of the latter difference, VMS users planning to use the IUERDAF IDL software should run the program con_rdaf on all rdaf-format files. (i.e., using the SPARC to VAX mode option). (Since the MAST archive is stored on a unix computer, no conversion is needed when the files are downloaded to unix or Macintosh computers.) Alternatively, users may request GO format files and convert them to RDAF format using the IUERDAF program GOTORDAF. For example, to convert the file described in the example above from GO to RDAF format, simply type (from IDL):

    IDL> gotordaf,'swp07971.melol'
    This will produce a standard .dat and .lab file in the format appropriate for the host computer. (The RDAF IDL routines can be downloaded from the IUEDAC web page.)

    Windows

    If you are using a PC running Win3.x, Win95, or WinNT, you can request the data in either tar or zip format and use the shareware program WinZip to both uncompress and extract the files. Extracting the tar or zip file from WinZip will create an IUE subdirectory containing the images requested. If you don't have WinZip, the programs pkzip/pkunzip should also work, but they have not been tested.

    The tar.gz format can also be read on a pc, but in our experience with WinZip, the system renames the downloaded file to be iue_tar.gz (instead of the original iue.tar.gz), and therefore the file may need to be renamed to iue.tar to be recognized as a tar file after unzipping.

    IUESIPS files can be requested from the "More Retrieval Options" page in either GO format, or RDAF format. After extracting, GO format files will have names such as swp12345.melol, while RDAF format files will have names such as swp12345llg.dat and swp12345llg.lab. The main differences between the 2 formats is that the RDAF format stores the VICAR label in a separate file from the data and, the data stored in the RDAF .dat files are assumed to be converted to a format compatible with the local operating system. Because of the latter difference, Window users planning to use the IUERDAF IDL software should run the program con_rdaf on all rdaf-format files. (i.e., using the SPARC to MIPSEL mode option). (Since the MAST archive is stored on a unix computer, no conversion is needed when the files are downloaded to unix or Macintosh computers.) Alternatively, users may request GO format files and convert them to RDAF format using the IUERDAF program GOTORDAF. For example, to convert the file described in the VMS example above from GO to RDAF format, simply type (from IDL):

    IDL> gotordaf,'swp07971.melol'
    This will produce a standard .dat and .lab file in the format appropriate for the host computer. (The RDAF IDL routines can be downloaded from the IUEDAC web page.)

    Macintosh

    Mac users can now use a free file-decompressor program called MindExpander from MindVision, to extract .zip and .gz files. Alternatively, the standard (but not free-ware) Stuffit Deluxe utility can also be used. Other utilities include: UnZip, ZipIt, and UnTar. Since most utilities only support zip or gz files, Mac users may want to request zip format rather than tar from the "more retrieval options" page. Tests run with Stuffit Deluxe however, showed that tar files can be read without any trouble.

    IUE Data Types

    The available IUE data types are displayed on the "More Retrieval Options" page and are described below. Normally users request the merged extracted spectral files (i.e., NEWSIPS MXLO or MXHI files, or IUESIPS MELO or MEHI files). Additional documentation is available on both the NEWSIPS and the (GO-format) IUESIPS data products.

    NEWSIPS Files

    • MX - The merged extracted spectral files containing the absolutely-calibrated fluxes, wavelengths, data quality flags, and sigmas (or noise estimate for MXHI files) in a FITS binary table extension. The high dispersion files are named MXHI and the low dispersion files are MXLO. Note that although low dispersion double aperture exposures are listed as two separate entries on the search results page, both spectra are contained in one MXLO file.
    • EXTRACT - This group contains the MXLO, MXHI files described above, plus the resampled image files (SILO & SIHI) from which the fluxes were extracted. The SI files contain the photometrically-corrected and rotated image as a primary array, wavelength information in a binary table extension, and an image extension containing an array of data qualiy flags. These files are useful for identifying camera artifacts such as cosmic ray hits, etc.
    • All-FA - All the archived NEWSIPS data products for a given observation can be requested by selecting this group. In addition to the files above, the RILO, RIHI raw image files (a 768x768 primary array image) and the LILO, LIHI linearized image files (a 768x768 primary array file with associated data quality flags as a 768x768 array image extension) will be downloaded.
    • SI - Select this group if only the SILO, SIHI resampled image files are to be downloaded.
    • RI - This group allows only the RILO, RIHI raw image files to be downloaded.
    • LI - This group allows only the LILO, LIHI linearized image files to be downloaded.

    IUESIPS Files

    • ME - This group includes the merged extracted spectral files containing the absolutely-calibrated fluxes, wavelengths, and data quality flags in a project-defined data format based on VICAR. The high dispersion files are named MEHI and the low dispersion files are MELO.
    • EXTRACT - This group contains the MELO, MEHI files described above, plus the low dispersion resampled line-by-line image (ELBL) files from which the fluxes were extracted (note no IUESIPS high dispersion resampled images were generated). The ELBL files contain the photometrically-corrected image data, wavelength information, and data qualiy flags. These files are particularly useful for identifying cosmic ray hits, reseau, microphonic noise, etc.
    • ALL - All the IUESIPS archived data products for a given observation can be requested by selecting this group. In addition to the files above, the RAW image file (a 768x768 array) will be downloaded.
    • ELBL - This group allows only the low dispersion resampled image (ELBL) files described above to be downloaded.
    • RAW - This group allows only the RAW image files to be downloaded.

    Status.txt File

    The status.txt file is downloaded along with the users requested files to provide information on the users request. This file may be useful for explaining why an expected file was not received. The following information is currently included in these files:
    • Observations requested - The number of camera name/image sequence number entries specified by the user.
    • Requested data types - A list of the IUE data types (e.g., melo, sihi, etc.) to be retrieved from the MAST archive as specified by the user in the data type menus on the "more retrieval options" page, or as specified on the search results page if downloading MX files was requested,
    • Compression type - The method used to compress/bundle the requested files. The possible values are:
      • tar - files will be bundled as a .tar file.
      • zip - Files bundled and compressed as a zip file with no further compression applied to individual files.
      • tar.gz - Files bundled as a gzipped tar file
    • Data format - Describes whether NEWSIPS, IUESIPS-GO, or IUESIPS-RDAF format files were requested,
    • File list - The list of downloaded files showing the (uncompressed) file name and the uncompressed file size in kilobytes. A message will appear next to any entries either "not found in the database" or, more rarely, "found in the database but not located on the jukebox". (Users should report any missing files to archive@stsci.edu.)
    • Total number of found files - The total number of files found in the database that were requested by the user.
    • Total file size - The total amount of disk space (in kilobytes) required to uncompress the requested files. Note that RDAF format IUESIPS file sizes are approximate.