next up previous contents
Next: How FITS Evolves Up: History Previous: Image Extension

2.8 Binary Tables

One drawback of the ASCII table format was the space required by tables with a large number of entries, such as gain and calibration tables, because every number appeared in coded form. In addition, conversion of the entries in these tables to ASCII would be very time-consuming. Initially, the ASCII form had been needed to represent floating point numbers. The precise definition and general acceptance of the IEEE floating point format made it possible to include binary floating point numbers in table fields. Meanwhile, developers of software to write FITS files for results of the Very Long Baseline Array project found that it would be useful to be able to put vectors in table fields. A design for binary tables with vector fields was developed by W. Cotton (NRAO). Because the vector field might be thought of as a column rising out of the page, it was imagined to be a third dimension of the table, and the extension was given the type name A3DTABLE. This name was chosen rather than, for example, 3DTABLE, to signify that the extension might still undergo further development; the final version would have a new name. This 3-D table extension was released as part of the NRAO Astronomical Image Processing System (AIPS) early in 1987.

At the beginning of 1990, NASA Headquarters specified that all NASA or NASA-supported projects must make their data available in FITS format. Many of the projects then in the process of developing their data plans were high energy astrophysics experiments. Their data would usually appear as event lists. These event tables could be very large. Map arrays would contain mostly zeroes and be inefficient. Binary tables would be the best format. Many of the FITS files designed by the high energy astrophysics community were based on the AIPS 3D table concept. This growing demand gave impetus to the development of a formal proposal for binary tables. Cotton distributed an initial proposal for a standard binary table extension, to be called BINTABLE, in April 1991. It incorporated all the features of the original A3DTABLE extension and included a number of additional keywords and field formats that had been suggested by readers of the original A3DTABLE description.

In late 1989, at a meeting at Green Bank to develop a standard format for single dish data, Wells had suggested the concept of including multidimensional matrices, not just vectors, in the fields of binary tables. The participants in the meeting had decided to adopt such a format and proceeded to discuss possible designs. Subsequently, a number of prospective binary table users expressed an interest in a mechanism for including arrays whose dimension might vary from row to row in the table. This issue was raised by D. Tody at the April, 1991 European FITS Committee meeting. Following discussions outside the formal sessions by a number of the participants, Cotton and Tody presented a design with a pointer data type, which was modified slightly in the discussion at the meeting. A formal text with appendixes describing possible formats for multidimensional fields and variable length arrays, and incorporating the keywords and field formats to make these structures possible, was released by Cotton and Tody in October, 1991.

As early as July, 1991, W. Pence (GSFC/HEASARC) had raised questions about how an array of strings was to be distinguished from a single long string. The ensuing electronic discussion led to the addition of a third appendix, describing a substring array convention. The revised BINTABLE proposal with this appendix added was released by Cotton, Tody, and Pence in May, 1993. In the spring of 1994, after a few points had been clarified, the IAUFWG endorsed the main proposal as part of standard FITS. The three appendixes - multidimensional arrays, variable length arrays, and arrays of strings - were not included as part of the endorsed standard; they are considered recommended but not required conventions. However, they have a special place in that the BINTABLE extension contains a number of provisions especially designed to make them possible.

Initial testing began in 1992, with an exchange of IMAGE and BINTABLE files among ESO, IUE, and the High Energy Science Archive Research Center (HEASARC) at Goddard Space Flight Center. In line with the evolution of FITS from a tape standard to a bit stream standard, the exchange was carried out by making files available through anonymous ftp rather than through exchange of tapes. Early in 1994, following the revision of BINTABLE, additional tests were carried out in preparation for the votes by the FITS committees. Files from the Space Telescope Science Institute (STScI) containing IMAGE and BINTABLE extensions were read at ESO, and another set of files from ESO was read at GSFC/HEASARC. The successful tests allowed the two extensions to proceed to a vote. On June 15, 1994, Chair P. Grosbøl announced that the IAUFWG had endorsed the blocking rules and the IMAGE and BINTABLE extensions. As both extensions are now standard FITS, they are now described in Section 3, rather than in Section 5 as was done in earlier versions of the Guide. The discussion of the BINTABLE appendixes remains in Section 5, as they were not part of the proposal adopted by the IAUFWG.


next up previous contents
Next: How FITS Evolves Up: History Previous: Image Extension