5.1.1.2 Keyword inspection and manipulation ¶

The meta-data in each header data unit, or HDU (also known as extension, see Fits) is stored as “keyword”s. Each keyword consists of a name, value, unit, and comments. The Fits program (see Fits) options related to viewing and manipulating keywords in a FITS HDU are described below.

First, let’s review the --keyvalue option which should be called separately from the rest of the options described in this section. Also, unlike the rest of the options in this section, with --keyvalue, you can give more than one input file.

-l STR[,STR[,...]

--keyvalue=STR[,STR[,...]

Only print the value of the requested keyword(s): the STRs. --keyvalue can be called multiple times, and each call can contain multiple comma-separated keywords. If more than one file is given, this option uses the same HDU/extension for all of them (value to --hdu). For example, you can get the number of dimensions of the three FITS files in the running directory, as well as the length along each dimension, with this command:

$ astfits *.fits --keyvalue=NAXIS,NAXIS1 --keyvalue=NAXIS2
image-a.fits 2      774    672
image-b.fits 2      774    672
image-c.fits 2      387    336

If only one input is given, and the --quiet option is activated, the file name is not printed on the first column, only the values of the requested keywords.

$ astfits image-a.fits --keyvalue=NAXIS,NAXIS1 \
          --keyvalue=NAXIS2 --quiet
2      774    672

Argument list too long: if the list of input files are too long, the shell is going to complain with the Argument list too long error! To avoid this problem, you can put the list of files in a plain-text file and give that plain-text file to the Fits program through the --arguments option discussed below.

The output is internally stored (and finally printed) as a table (with one column per keyword). Therefore just like the Table program, you can use --colinfoinstdout to print the metadata like the example below (also see Invoking Table). The keyword metadata (comments and units) are extracted from the comments and units of the keyword in the input files (first file that has a comment or unit). Hence if the keyword does not have units or comments in any of the input files, they will be empty. For more on Gnuastro’s plain-text metadata format, see Gnuastro text table format.

$ astfits *.fits --keyvalue=NAXIS,NAXIS1,NAXIS2 \
          --colinfoinstdout
# Column 1: FILENAME [name,str10,] Name of input file.
# Column 2: NAXIS    [    ,u8   ,] number of data axes
# Column 3: NAXIS1   [    ,u16  ,] length of data axis 1
# Column 4: NAXIS2   [    ,u16  ,] length of data axis 2
image-a.fits 2      774    672
image-b.fits 2      774    672
image-c.fits 2      387    336

Another advantage of a table output is that you can directly write the table to a file. For example, if you add --output=fileinfo.fits, the information above will be printed into a FITS table. You can also pipe it into Table to select files based on certain properties, to sort them based on another property, or any other operation that can be done with Table (including Column arithmetic). For example, with the command below, you can select all the files that have a size larger than 500 pixels in both dimensions.

$ astfits *.fits --keyvalue=NAXIS,NAXIS1,NAXIS2 \
          --colinfoinstdout \
          | asttable --range=NAXIS1,500,inf \
                     --range=NAXIS2,500,inf -cFILENAME
image-a.fits
image-b.fits

Note that --colinfoinstdout is necessary to use column names when piping to other programs (like asttable above). Also, with the -cFILENAME option, we are asking Table to only print the final file names (we do not need the sizes any more).

The commands with multiple files above used *.fits, which is only useful when all your FITS files are in the same directory. However, in many cases, your FITS files will be scattered in multiple sub-directories of a certain top-level directory, or you may only want those with more particular file name patterns. A more powerful way to list the input files to --keyvalue is to use the find program in Unix-like operating systems. For example, with the command below you can search all the FITS files in all the sub-directories of /TOP/DIR.

astfits $(find /TOP/DIR/ -name "*.fits") --keyvalue=NAXIS2

--arguments=STR

A plain-text file containing the list of input files that will be used in --keyvalue. Each word (group of characters separated by SPACE or new-line) is assumed to be the name of the separate input file. This option is only relevant when no input files are given as arguments on the command-line: if any arguments are given, this option is ignored.

This is necessary when the list of input files are very long; causing the shell to abort with an Argument list too long error. In such cases, you can put the list into a plain-text file and use this option like below:

$ ls $(path)/*.fits > list.txt
$ astfits --arguments=list.txt --keyvalue=NAXIS1

-O

--colinfoinstdout

Print column information (or metadata) above the column values when writing keyword values to standard output with --keyvalue. You can read this option as column-information-in-standard-output.

Below we will discuss the options that can be used to manipulate keywords. To see the full list of keywords in a FITS HDU, you can use the --printallkeys option. If any of the keyword modification options below are requested (for example, --update), the headers of the input file/HDU will be changed first, then printed. Keyword modification is done within the input file. Therefore, if you want to keep the original FITS file or HDU intact, it is easiest to create a copy of the file/HDU first and then run Fits on that (for copying a HDU to another file, see HDU information and manipulation. In the FITS standard, keywords are always uppercase. So case does not matter in the input or output keyword names you specify.

Most of the options can accept multiple instances in one command. For example, you can add multiple keywords to delete by calling --delete multiple times, since repeated keywords are allowed, you can even delete the same keyword multiple times. The action of such options will start from the top most keyword.

The precedence of operations are described below. Note that while the order within each class of actions is preserved, the order of individual actions is not. So irrespective of what order you called --delete and --update. First, all the delete operations are going to take effect then the update operations.

1. --delete
2. --rename
3. --update
4. --write
5. --asis
6. --history
7. --comment
8. --date
9. --printallkeys
10. --verify
11. --copykeys

All possible syntax errors will be reported before the keywords are actually written. FITS errors during any of these actions will be reported, but Fits will not stop until all the operations are complete. If --quitonerror is called, then Fits will immediately stop upon the first error.

If you want to inspect only a certain set of header keywords, it is easiest to pipe the output of the Fits program to GNU Grep. Grep is a very powerful and advanced tool to search strings which is precisely made for such situations. for example, if you only want to check the size of an image FITS HDU, you can run:

$ astfits input.fits | grep NAXIS

The keyword related options to the Fits program are fully described below.

-d STR

--delete=STR

Delete one instance of the STR keyword from the FITS header. Multiple instances of --delete can be given (possibly even for the same keyword, when its repeated in the meta-data). All keywords given will be removed from the headers in the same given order. If the keyword does not exist, Fits will give a warning and return with a non-zero value, but will not stop. To stop as soon as an error occurs, run with --quitonerror.

-r STR,STR

--rename=STR,STR

Rename a keyword to a new value (for example, --rename=OLDNAME,NEWNAME. STR contains both the existing and new names, which should be separated by either a comma (,) or a space character. Note that if you use a space character, you have to put the value to this option within double quotation marks (") so the space character is not interpreted as an option separator. Multiple instances of --rename can be given in one command. The keywords will be renamed in the specified order. If the keyword does not exist, Fits will give a warning and return with a non-zero value, but will not stop. To stop as soon as an error occurs, run with --quitonerror.

-u STR

--update=STR

Update a keyword, its value, its comments and its units in the format described below. If there are multiple instances of the keyword in the header, they will be changed from top to bottom (with multiple --update options).

The format of the values to this option can best be specified with an example:

--update=KEYWORD,value,"comments for this keyword",unit

If there is a writing error, Fits will give a warning and return with a non-zero value, but will not stop. To stop as soon as an error occurs, run with --quitonerror.

The value can be any numerical or string value¹²⁹. Other than the KEYWORD, all the other values are optional. To leave a given token empty, follow the preceding comma (,) immediately with the next. If any space character is present around the commas, it will be considered part of the respective token. So if more than one token has space characters within it, the safest method to specify a value to this option is to put double quotation marks around each individual token that needs it. Note that without double quotation marks, space characters will be seen as option separators and can lead to undefined behavior.

-w STR

--write=STR

Write a keyword to the header. For the possible value input formats, comments and units for the keyword, see the --update option above. The special names (first string) below will cause a special behavior:

/

Write a “title” to the list of keywords. A title consists of one blank line and another which is blank for several spaces and starts with a slash (/). The second string given to this option is the “title” or string printed after the slash. For example, with the command below you can add a “title” of ‘My keywords’ after the existing keywords and add the subsequent K1 and K2 keywords under it (note that keyword names are not case sensitive).

$ astfits test.fits -h1 --write=/,"My keywords" \
          --write=k1,1.23,"My first keyword"    \
          --write=k2,4.56,"My second keyword"
$ astfits test.fits -h1
[[[ ... truncated ... ]]]

                      / My keywords
K1      =                 1.23 / My first keyword
K2      =                 4.56 / My second keyword
END

Adding a “title” before each contextually separate group of header keywords greatly helps in readability and visual inspection of the keywords. So generally, when you want to add new FITS keywords, it is good practice to also add a title before them.

The reason you need to use / as the keyword name for setting a title is that / is the first non-white character.

The title(s) is(are) written into the FITS with the same order that --write is called. Therefore in one run of the Fits program, you can specify many different titles (with their own keywords under them). For example, the command below that builds on the previous example and adds another group of keywords named A1 and A2.

$ astfits test.fits -h1 --write=/,"My keywords"   \
          --write=k1,1.23,"My first keyword"      \
          --write=k2,4.56,"My second keyword"     \
          --write=/,"My second group of keywords" \
          --write=a1,7.89,"First keyword"         \
          --write=a2,0.12,"Second keyword"

checksum ¶

When nothing is given afterwards, the header integrity keywords DATASUM and CHECKSUM will be calculated and written/updated. The calculation and writing is done fully by CFITSIO, therefore they comply with the FITS standard 4.0¹³⁰ that defines these keywords (its Appendix J).

If a value is given (e.g., --write=checksum,MyOwnCheckSum), then CFITSIO will not be called to calculate these two keywords and the value (as well as possible comment and unit) will be written just like any other keyword. This is generally not recommended since CHECKSUM is a reserved FITS standard keyword. If you want to calculate the checksum with another hashing standard manually and write it into the header, it is recommended to use another keyword name.

In the FITS standard, CHECKSUM depends on the HDU’s data and header keywords, it will therefore not be valid if you make any further changes to the header after writing the CHECKSUM keyword. This includes any further keyword modification options in the same call to the Fits program. However, DATASUM only depends on the data section of the HDU/extension, so it is not changed when you add, remove or update the header keywords. Therefore, it is recommended to write these keywords as the last keywords that are written/modified in the extension. You can use the --verify option (described below) to verify the values of these two keywords.

datasum

Similar to checksum, but only write the DATASUM keyword (that does not depend on the header keywords, only the data).

-a STR

--asis=STR

Write the given STR exactly as it is, into the given FITS file header with no modifications. If the contents of STR does not conform to the FITS standard for keywords, then it may (most probably: it will!) corrupt your file and you may not be able to open it any more. So please be very careful with this option (its your responsibility to make sure that the string conforms with the FITS standard for keywords).

If you want to define the keyword from scratch, it is best to use the --write option (see below) and let CFITSIO worry about complying with the FITS standard. Also, you want to copy keywords from one FITS file to another, you can use --copykeys that is described below. Through these high-level instances, you don’t have to worry about low-level issues.

One common usage of --asis occurs when you are given the contents of a FITS header (many keywords) as a plain-text file (so the format of each keyword line conforms with the FITS standard, just the file is plain-text, and you have one keyword per line when you open it in a plain-text editor). In that case, Gnuastro’s Fits program won’t be able to parse it (it doesn’t conform to the FITS standard, which doesn’t have a new-line character!). With the command below, you can insert those headers in headers.txt into img.fits (its HDU number 1, the default; you can change the HDU to modify with --hdu).

$ cat headers.txt \
      | while read line; do \
              astfits img.fits --asis="$line"; \
        done

Don’t forget a title: Since the newly added headers in the example above weren’t originally in the file, they are probably some form of high-level metadata. The raw example above will just append the new keywords after the last one. Making it hard for human readability (its not clear what this new group of keywords signify, where they start, and where this group of keywords end). To help the human readability of the header, add a title for this group of keywords before writing them. To do that, run the following command before the cat ... command above (replace Imported keys with any title that best describes this group of new keywords based on their context): $ astfits img.fits --write=/,"Imported keys"

-H STR

--history STR

Add a HISTORY keyword to the header with the given value. A new HISTORY keyword will be created for every instance of this option. If the string given to this option is longer than 70 characters, it will be separated into multiple keyword cards. If there is an error, Fits will give a warning and return with a non-zero value, but will not stop. To stop as soon as an error occurs, run with --quitonerror.

-c STR

--comment STR

Add a COMMENT keyword to the header with the given value. Similar to the explanation for --history above.

-t

--date

Put the current date and time in the header. If the DATE keyword already exists in the header, it will be updated. If there is a writing error, Fits will give a warning and return with a non-zero value, but will not stop. To stop as soon as an error occurs, run with --quitonerror.

-p

--printallkeys

Print the full metadata (keywords, values, units and comments) in the specified FITS extension (HDU). If this option is called along with any of the other keyword editing commands, as described above, all other editing commands take precedence to this. Therefore, it will print the final keywords after all the editing has been done.

--printkeynames

Print only the keyword names of the specified FITS extension (HDU), one line per name. This option must be called alone.

-v

--verify

Verify the DATASUM and CHECKSUM data integrity keywords of the FITS standard. See the description under the checksum (under --write, above) for more on these keywords.

This option will print Verified for both keywords if they can be verified. Otherwise, if they do not exist in the given HDU/extension, it will print NOT-PRESENT, and if they cannot be verified it will print INCORRECT. In the latter case (when the keyword values exist but cannot be verified), the Fits program will also return with a failure.

By default this function will also print a short description of the DATASUM AND CHECKSUM keywords. You can suppress this extra information with --quiet option.

--copykeys=INT:INT/STR,STR[,STR]

Copy the desired set of the input’s keyword records, to the to the output (specified with the --output and --outhdu for the filename and HDU/extension respectively). The keywords to copy can be given either as a range (in the format of INT:INT, inclusive) or a list of keyword names as comma-separated strings (STR,STR), the list can have any number of keyword names. More details and examples of the two forms are given below:

Range

The given string to this option must be two integers separated by a colon (:). The first integer must be positive (counting of the keyword records starts from 1). The second integer may be negative (zero is not acceptable) or an integer larger than the first.

A negative second integer means counting from the end. So -1 is the last copy-able keyword (not including the END keyword).

To see the header keywords of the input with a number before them, you can pipe the output of the Fits program (when it prints all the keywords in an extension) into the cat program like below:

$ astfits input.fits -h1 | cat -n

List of names

The given string to this option must be a comma separated list of keyword names. For example, see the command below:

$ astfits input.fits -h1 --copykeys=KEY1,KEY2 \
          --output=output.fits --outhdu=1

Please consider the notes below when copying keywords with names:

• If the number of characters in the name is more than 8, CFITSIO will place a HIERARCH before it. In this case simply give the name and do not give the HIERARCH (which is a constant and not considered part of the keyword name).
• If your keyword name is composed only of digits, do not give it as the first name given to --copykeys. Otherwise, it will be confused with the range format above. You can safely give an only-digit keyword name as the second, or third requested keywords.
• If the keyword is repeated more than once in the header, currently only the first instance will be copied. In other words, even if you call --copykeys multiple times with the same keyword name, its first instance will be copied. If you need to copy multiple instances of the same keyword, please get in touch with us at bug-gnuastro@gnu.org.

--outhdu

The HDU/extension to write the output keywords of --copykeys.

-Q

--quitonerror

Quit if any of the operations above are not successful. By default if an error occurs, Fits will warn the user of the faulty keyword and continue with the rest of actions.

-s STR ¶

--datetosec STR

Interpret the value of the given keyword in the FITS date format (most generally: YYYY-MM-DDThh:mm:ss.ddd...) and return the corresponding Unix epoch time (number of seconds that have passed since 00:00:00 Thursday, January 1st, 1970). The Thh:mm:ss.ddd... section (specifying the time of day), and also the .ddd... (specifying the fraction of a second) are optional. The value to this option must be the FITS keyword name that contains the requested date, for example, --datetosec=DATE-OBS.

This option can also interpret the older FITS date format (DD/MM/YYThh:mm:ss.ddd...) where only two characters are given to the year. In this case (following the GNU C Library), this option will make the following assumption: values 68 to 99 correspond to the years 1969 to 1999, and values 0 to 68 as the years 2000 to 2068.

This is a very useful option for operations on the FITS date values, for example, sorting FITS files by their dates, or finding the time difference between two FITS files. The advantage of working with the Unix epoch time is that you do not have to worry about calendar details (for example, the number of days in different months, or leap years).

--wcscoordsys=STR ¶

Convert the coordinate system of the image’s world coordinate system (WCS) to the given coordinate system (STR) and write it into the file given to --output (or an automatically named file if no --output has been given).

For example, with the command below, img-eq.fits will have an identical dataset (pixel values) as image.fits. However, the WCS coordinate system of img-eq.fits will be the equatorial coordinate system in the Julian calendar epoch 2000 (which is the most common epoch used today). Fits will automatically extract the current coordinate system of image.fits and as long as it is one of the recognized coordinate systems listed below, it will do the conversion.

$ astfits image.fits --wcscoordsys=eq-j2000 \
          --output=img-eq.fits

The currently recognized coordinate systems are listed below (the most common one today is eq-j2000):

eq-j2000

2000.0 (Julian-year) equatorial coordinates. This is also known as FK5 (short for “Fundamental Katalog No 5” which was the source of the star coordinates used to define it).

This coordinate system is based on the motion of the Sun and has epochs when the mean equator was used (for example eq-b1950 below). Furthermore, the definition of year is different: either the Besselian year in 1950.0, or the Julian year in 2000. For more on their difference and links for further reading about epochs in astronomy, please see the description in Wikipedia.

Because of these difficulties, the equatorial J2000.0 coordinate system has been deprecated by the IAU in favor of International Celestial Refernece System (ICRS) but is still used extensively. ICRS is defined based on extra-galactic quasars, so it does not depend on the dynamics of the solar system any more. But to enable historical continuity, ICRS has been defined to be equivalent to the equatorial J2000.0 within its accepted error bars of the latter (tens of milli-arcseconds). This justifies the reason that moving to ICRS has been relatively slow.

eq-b1950

1950.0 (Besselian-year) equatorial coordinates.

ec-j2000

2000.0 (Julian-year) ecliptic coordinates.

ec-b1950

1950.0 (Besselian-year) ecliptic coordinates.

galactic

Galactic coordinates.

supergalactic

Supergalactic coordinates.

--wcsdistortion=STR ¶

If the argument has a WCS distortion, the output (file given with the --output option) will have the distortion given to this option (for example, SIP, TPV). The output will be a new file (with a copy of the image, and the new WCS), so if it already exists, the file will be delete (unless you use the --dontdelete option, see Input/Output options).

With this option, the Fits program will read the minimal set of keywords from the input HDU and the HDU data. It will then write them into the file given to the --output option but with a newly created set of WCS-related keywords corresponding to the desired distortion standard.

If no --output file is specified, an automatically generated output name will be used which is composed of the input’s name but with the -DDD.fits suffix, see Automatic output. Where DDD is the value given to this option (desired output distortion).

Note that all possible conversions between all standards are not yet supported. If the requested conversion is not supported, an informative error message will be printed. If this happens, please let us know and we will try our best to add the respective conversions.

For example, with the command below, you can be sure that if in.fits has a distortion in its WCS, the distortion of out.fits will be in the SIP standard.

$ astfits in.fits --wcsdistortion=SIP --output=out.fits