Use the Kepler FFI Search form to
search and retrieve
FFI, KEP, or REF files from the Kepler archive.
Searches are limited to queries based on file name, class,
start date, end date, release date and/or generation date.
Search values can be specified in several different ways, depending
on the data type of the field. In all cases, a single value can
be entered (although this is not recommented for floating point values).
In addition,
various operators can be included depending on the data type of the
field as described below.
The data types for each column can be displayed using the "Field Descriptions"
link at the top of all MAST search forms.
- Numerical fields - Real (i.e., real,
numeric, float, or double) and Integer (i.e., long) fields can be
specified as a single value, as a single value with numerical operators
such as "< n", ">= n", and "!= n", "\null",
or as an inclusive range (e.g., "1990 .. 2000").
Bcause of small differences in stored floating point values, specifying
a single floating point value may not return the expected results.
Therefore, numerical or range operators are recommended with floating point
fields queries. Integers may be requested either way.
The number of digits displayed to the right of the
decimal point for floating point numbers is determined solely by the
data type. If the data type is "real" only 1 digit is displayed, and
for "numeric" 2 digits are displayed. "Float" data types have 3, and "double"
can have up to 8 or more.
- String fields - String fields, also
known as "char" or "varchar" fields,
can use the = (equal), != (not equal, e.g., != SMALL), \null ,
or the "*" , "%" wildcard operators (e.g. "Jup*).
By default, string searches use equals ("=") which runs faster
without wild cards but implies matches must be exact. For example, searching on
Target description = "Planet" will not
return an entry for "Planet:Jupiter". (There are some exceptions though such as
searching on Kepler Investigation ID will automatically include wild cards.)
Wildcards are allowed
and encouraged when an exact match is not desired, so in the previous
example, searching on Planet* would return all entries beginning with the word Planet.
Quotes are not necessary for string values.
Since moving to the Microsoft SQL Server database system in 2009, string
searches are no longer case-sensitive. Values will still be displayed in the same
case they were originally entered, but entries will be found regardless of the
case of the searched string (i.e., searching for HST id "go-5916" or "GO-5916"
will return the same entries).
Also, as with any data type, commas can be used to search for multiple entries.
For example, to search for all O3 and B3 stars from the Skiff catalog,
just enter O3*,B3*. Likewise, entering \null,<5 will return values that are
either null or less than 5.
- Bibstring fields - Bibstrings are a special
string field for storing ADS bibcodes. A bibstring value has 19 characters,
and its assumed that no operators other than wild cards are included in
queries. This data type was mainly added because ".." was normally interpreted
as a range search request. For a bibstring field, specifying 2101ApJ..*
means find all entries whose bibcode begins with 2010ApJ..
- Coordinate fields - Generally you can specify a variety
of formats for Right Ascension and Declination using either decimal
degrees or sexagesimal values (RA in decimal hours is
also available, but only as an output format option). The allowed search
formats are described in detail below.
- Date fields - Dates can also be specified in a variety
of formats and can also use operators and inclusive range searches.
Here are some allowed/recommended examples:
- > Jul 15 1994
- < 01-jan-2010
- Dec 1 1995 .. Dec 6 1995
- 01-jan-2009 .. 15-feb-2010
- 2009-05-11 17:51:31 (but date match must be exact)
- 20090115 .. 20100101
- dec 2009 .. jan 2010
- 2009 jan .. 2010 jan
- 2009 .. 2010
Formats found not to work or not recommended include:
- 15 Jul 2005 (valid format but only returns entries with a value of 15 Jul 2005 00:00:00)
- 15JUL2009 (same problem as above)
- 15-jul-2009 (same problem as above)
- 2009-01 (same problem as above)
- 2009-10 .. 2010-01 (doesn't seem to work)
- 2009-10-01 .. jan-01-2010 (don't mix formats in range searches)
- jan-15-2009 .. feb-01-2010 (putting month before day doesn't work with dashes!)
- Dec 15 2009 .. Dec 01 2009 (earlier date should be listed first)
Often queries on a single date will fail because the database can store
datetime fields to the millisecond and the matches must be exact. It is
preferable to use a range or the <, > operators.
Note that when the time is not specified, the query will default to 00:00:00.
Therefore without specifying times, a range search would include the starting date
but exclude the ending date. Leaving off the day or month would work similarly.
To see the data type of a particular field, click on the form element
label or any of the help page links.
Note quotes are not needed for any values.
Note, searches on "null" values in fields of any data type
are now possible by entering \null.
By default, the various search criteria will be submitted
using logical AND's. Logical OR's
are not supported on most mission search forms except when using
commas within a single form element such as entering "hc230,srhlw"
for IUE program ID to return entries with ID hc230 OR srhlw..
Information on individual search form elements is listed below.
Note that specific examples given below do not necessarily apply
to all missions. The examples are merely intended to show valid
formats for data entry.
- Data set name
- Data set or root name
examples: KPLR2009069022529
- Archive Class
- DADS archive classification. Current options include:
full frame images (ffi), long and short cadence collateral
files (CLL, CLS), artifact removal pixel files (ARP),
background pixel files (BKG), and Ancillary Engineering data (ENG).
- Release_date
- Date data becomes public
examples: currently all
set to Jan 1, 2020
- Actual Start Time
- Actual Observation Start time
currently all nulls.
- Actual End time
- End of observation time
examples: Jan 10 2008
although currently all values are null.
- Archive Date
- Date file was ingested into MAST archive
examples: Jan 10 2008
although currently all values are null.
- Quarter
- A sequential number indicating the quarter in which exposure was made.
0 to greater than 11
- Skygroup ID
- An integer value from 1 to 84 describing the module/channel where the target falls.
1 to 84
- Module
- Each pair of CCDs comprise a module and corresponds to four output channels (two per CCD). Each output is read out by a unique analog signal chain (e.g. amplifier).
1 to 21
- Output
- OUTPUT is an integer ranging from 1 to 4 describing one of the four output channels. An object can have a non-zero output number if it is off, but near, a CCD. Each module has four output channels (two per CCD). Each output is read out by a unique analog signal chain (e.g. amplifier).
1 to 4
- Season
- Time of year in which file was obtained.
0 to 3. See the Kepler Seasons table .
- Channel
- CHANNEL is an integer ranging from 1 to 84 that uniquely specifies the MODULE/OUTPUT pair.There are 21 modules, each with 4 outputs for a total of 84 channels.
1 to 84
- User Specified Field n
- You may use these form elements to search on any column(s)
in the mission table. First, select the field
you wish to search from the pulldown menu under the "User-specified field n" heading.
Then, type in the qualification in the corresponding "Field Descriptions" box.
Clicking on the "Field Descriptions" link, will display information on the allowed fields
including the allowed range of values.
As an example, a Kepler user might select "E(B-V)" from the pulldown menu and
enter "< 0.5" in the "Field Descriptions" box.
NOTE only fields which are not already
included on the search form should be selected. Specifying search criteria
for a field that is listed in BOTH the form and in the User Specified field, may
cause either the query to fail or return unexpected results.
- Output Columns
- This form element allows you to choose the columns to be displayed
and their order, for the search results.
A set of columns that are commonly requested has been chosen as a default.
You remove output columns by highlighting the column to be removed and
then clicking on the remove button to the right of the output columns list.
You may determine the order of columns by highlighting a column and
then clicking the up or down buttons to the right of the list of
chosen output columns.
You may also add more columns to the list, either singly or all
at once. To add individual columns, select the desired
column from the pull down menu beneath the list of chosen output columns,
then click the add button. The column will be added to the bottom
of the output column list. To add all the available columns, click the
"add all" button.
Note the output column form element has its own reset button to restore
the list of output columns to the values initially displayed when the page
was drawn. The reset button at the top of the form is
used to reset the other form elements.
Clicking the "clear form" button will restore the original defaults to
all isections of the form.
One warning, the amount of memory required for a search is proportional
to the number of columns requested. For large requests, users may want to
reduce the number of output columns to the minimum required.
- Suppress Null Result Message
- By default, when using "file upload mode", the
message "No Records Found Matching Query" (or for non-html output,
"no rows found") is displayed
for each entry with no search results. Clicking this button
will prevent these messages from appearing which may be useful
for reducing output from large search results.
- Verb
- Verb is an integer parameter used by the VO community for specifying the
amount of output returned for a given search request. It is only
available when retrieving data as a web service but works with every
MAST service. Setting verb=3 in a search request is equivalent to
specifying "add all" from a search form;
it will return all the available columns in the output not just the standard
default fields. Currently setting verb to any other value has no effect.
- Sort output by:
- Choose how you want the output rows sorted. You can select
up to three fields to sort on. The rows will be sorted in the order of
the first sort field; if two rows have the same sort field, they will be
sorted in order of the second sort field, and so on. Default sort fields
may be listed, but any field from the pulldown list can be used.
Specifying multiple sort fields may increase executon time.
If you prefer no sorting, you can specify "null" for all 3 fields.
This may speed up the query, but results will be displayed in
the order in which they were originally stored in the database table.
For each field, you can select that the rows be sorted in reverse
order on that field by selecting the reverse checkbox. For example,
you can sort the rows with the most recent observations first by selecting
Observation Date for the first sort field and selecting
the reverse checkbox next to it.
One word of caution: the selected sort field can change the search results
when the query finds more rows than are displayed ( i.e, when the number of
found rows exceeds the value of "maximum records"). For example, for a search
on a particular coordinate that finds 5,000 entries, if the search
is sorted on exposure time
and 1,001 rows are to be displayed (the default), then the 1,001
shortest exposures
from the 5,000 found entries will be displayed
which may not include the entries closest to the
desired position. (It is a good idea to always sort on "ang_sep" for target or
coordinate searches).
- Display Coordinates
- Specifies the format for displaying the primary
equatorial (i.e., RA and Dec) coordinates. The options include:
- Sexagesimal - The default format with Right Ascension specified
as hh mm ss.sss and Declination as +/-dd mm ss.ss
( e.g., RA = 12 46 11.091, Dec = -00 30 12.08).
Note an extra digit was added as of June, 2012.
- Degrees - Decimal degrees for both RA and Dec with 7
significant figures to the right of the decimal point
(e.g., RA=191.5461912, Dec=-0.5033333).
Note 2 extra digits were added as of June, 2012.
or
- Hours - Decimal hours for RA and decimal degrees for Dec
in same format as for decimal degrees (e.g., RA=12.7697512, Dec=-0.5033333).
Note decimal hours = decimal degrees/15.0.
(Note 2 extra digits were added as of June, 2012.)
Any other coordinate fields contained in the searched mission catalog will be
displayed in their original format.
- Maximum Records
- This value specifies the maximum number of rows returned in a
single query. For the standard mission search forms, the current
default is 1,001, but values from 1 to 25,001 are allowed.
For the file upload forms in which multiple targets, data ID's, or
coordinates can be specified, the default is set to 20 rows per file entry,
with allowed values ranging from 1 to 5,000.
Note when displayed as HTML, the latest search scripts will display
100 records per page. Links to the additional pages are shown
on the results page.
This paging feature however does require javascript to be turned on.
Users should be cautioned about retrieving a large number of records
(i.e., > 10,000 - 15,000) in HTML format. This can cause memory
problems for the browser (particulrly Safari) and prevent javascript
commands from functioning. It may also cause the browser to freeze
and require restarting.
Using the output format options which download results in a file can reduce the
problem.
Another option for large requests is to use "Casjobs".
Casjobs requires requesting a user name and password, and submitting
queries in SQL, but it allows users to submit large search results and
save them online. For Kepler, the link ito Casjobs is
http://mastweb.stsci.edu/kplrcasjobs/.
For other missions, check the Search_Retrieve page in the left gutter.
When queries are submitted as a web service, the default number of rows
returned is 2,000, but any value is allowed when max_rows is specified
as a query parameter. (See the
MAST Web services page for more information.)
- Records per page
- This parameter controls the number of records displayed per web page.
By default, 100 rows or records are displayed per HTML page. Therefore if
400 records are returned, links to 4 pages will be displayed at the
top and bottom of the results table.
The paging feature however uses
javascript, so if javascript is turned off, paging won't work
and only the rows shown on the first page can be displayed. This is
one reason why increasing the default value may be helpful.
The current limit is 1000 rows per page which is recommended for large requests.
Note this value is ignored when output formats other than HTML are selected.
- Make Rows Distinct
- Selecting this checkbox will restrict the display to only rows in which
every output column value for a given row is unique. This option is primarily useful
when only a small number of columns are displayed (i.e., using the "remove"
button to remove default output columns) and when the selected columns
have duplicate values. Including columns which already have unique values (e.g.
Kepler ID or Data ID)
will make the "Distinct" option ineffective.
As an example, a IUE or FUSE user might want to create a list of unique target
names for a specific object class/category. He or she would specify the desired object
class, select only "object Class" and "target name" for the output columns,
click the "Make Rows Distinct" button, then click "Search".
Note that for some missions, columns such as RA, Dec and Magnitude were frequently defined
by the observer and often have different values for the same target.
If columns such as these are chosen as output columns, there will often
be more than one row listed per object name.
- Output Format
- You may choose any of the
following formats for displaying/storing search results.
If you are using a browser (e.g., Firefox, etc) to submit a query
from a MAST web form, the formats labelled "File: ..." offer a
way to download results directly to your local computer. Choose any of
the other format options if you want to display the results in the browser,
or if you are submitting the request from a program
(see MAST web services).
- HTML_Table (default) - results returned as a
standard HTML table including various links for retrieving data,
displaying previews, literature references, plotting spectra, etc.
- File: Excel_Spreadsheet - results are stored as an Excel
spreadsheet file.
(Note: assumes users computer/browser provides support for Excel-format files).
The default file name when downloaded is "mission"_search.xls where "mission"
is the mission name (e.g., fuse_search.xls).
- VOTable - an XML format adopted by the Virtual Observatory (VO)
project and displayed in the user's browser. Note coordinates in VOTable
format are always in decimal degrees rather than sexagesimal format.
For searches returning results from more than one mission and/or target,
multiple "resource" tags are created. Searches with a radius of 0
will return a VOTable file listing the output fields for that particular
mission/catalog. (For more information on the XML file format,
see VOTable documentation.)
- Comma-separated values - a simple ASCII array containing
column headings followed by rows of comma-separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
- Space-separated values - a simple ASCII array containing
column headings followed by rows of space-separated values.
In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
Note users may want to select
coordinates in decimal rather than sexagesimal format to maintain the
correspondence between column headings and entries although some column
headings may contain blanks as well.
(Semi-colons, pipes, or commas are probably safer delimiters.)
- IRAF Space-separated values with INDEFs - Like the space-separated format
above except empty fields are replaced with the string "INDEF". This format is useful
for IRAF-compatibility.
- Semi-colon separated values - a simple ASCII array containing
column headings followed by rows of semi-colon separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
- Tab-separated values - a simple ASCII array containing
tabs for delimiters. May be useful for ingesting into Excel spreadsheets.
- Pipe-separated values - a simple ASCII array containing
column headings followed by rows of pipe or vertical bar separated values.
(Note: not offered in file upload mode.)
- Json format - Javascript Object Notation (json) is a simple machine and
human-readable, name/value ASCII format supported by many programming languages.
(Note: not offered in file upload mode.)
- File: comma-separated values -
a simple ASCII text file containing
column headings followed by rows of comma-separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
- File: Space-separated values -
a simple ASCII text file containing
column headings followed by rows of space-separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
- File: IRAF Space-separated values with INDEFs -
Like the space-separated format
above except empty fields are replaced with the string "INDEF". This file format is useful
for IRAF-compatibility.
- File: Semicolon-separated values -
a simple ASCII text file containing
column headings followed by rows of semicolon-separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
- File: Tab-separated values - a simple ASCII file containing
tabs for delimiters. May be useful for ingesting into Excel spreadsheets.
- File: Pipe-separated values -
a simple ASCII text file containing
column headings followed by rows of pipe-separated values.
(Note: not offered in file upload mode.)
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
- File: Json format - Javascript Object Notation (json) is a
simple machine and
human-readable, name/value ASCII format supported by many programming languages.
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
- File: WGET Commands - This option is only available for
certain missions. If selected, a shell script file is
output which the user
can execute from his own computer to download all the selected light
curves or extracted spectral files with one command.
The shell script
file uses the "WGET" program which is available for most operating systems.
Note for Kepler, two WGET options are available:
"File: WGET LC commands" will create a script for downloading available light curves,
"File: WGET TPF commands" will create a script for downloading target pixel files.
For Swift, the WGET option will create a script for downloading
the Sky coordinate images.
Note: If no entries are found for an entry a
"no entries found" message is written in the selected format and
the program continues.
In all cases, error conditions will cause the database search to abort.