---

6.2.4.14 Bitwise operators ¶

Astronomical images are usually stored as an array multi-byte pixels with different sizes for different precision levels (see Numeric data types). For example, images from CCDs are usually in the unsigned 16-bit integer type (each pixel takes 16 bits, or 2 bytes, of memory) and fully reduced deep images have a 32-bit floating point type (each pixel takes 32 bits or 4 bytes).

On the other hand, during the data reduction, we need to preserve a lot of meta-data about some pixels. For example, if a cosmic ray had hit the pixel during the exposure, or if the pixel was saturated, or is known to have a problem, or if the optical vignetting is too strong on it. A crude solution is to make a new image when checking for each one of these things and make a binary image where we flag (set to 1) pixels that satisfy any of these conditions above, and set the rest to zero. However, processing pipelines sometimes need more than 20 flags to store important per-pixel meta-data, and recall that the smallest numeric data type is one byte (or 8 bits, that can store up to 256 different values), while we only need two values for each flag! This is a major waste of storage space!

A much more optimal solution is to use the bits within each pixel to store different flags! In other words, if you have an 8-bit pixel, use each bit as a flag to mark if a certain condition has happened on a certain pixel or not. For example, let’s set the following standard based on the four cases mentioned above: the first bit will show that a cosmic ray has hit that pixel. So if a pixel is only affected by cosmic rays, it will have this sequence of bits (note that the bit-counting starts from the right): 00000001. The second bit shows that the pixel was saturated (00000010), the third bit shows that it has known problems (00000100) and the fourth bit shows that it was affected by vignetting (00001000).

Since each bit is independent, we can thus mark multiple metadata about that pixel in the actual image, within a single “flag” or “mask” pixel of a flag or mask image that has the same number of pixels. For example, a flag-pixel with the following bits 00001001 shows that it has been affected by cosmic rays and it has been affected by vignetting at the same time. The common data type to store these flagging pixels are unsigned integer types (see Numeric data types). Therefore when you open an unsigned 8-bit flag image in a viewer like DS9, you will see a single integer in each pixel that actually has 8 layers of metadata in it! For example, the integer you will see for the bit sequences given above will respectively be: \(2^0=1\) (for a pixel that only has cosmic ray), \(2^1=2\) (for a pixel that was only saturated), \(2^2=4\) (for a pixel that only has known problems), \(2^3=8\) (for a pixel that is only affected by vignetting) and \(2^0 + 2^3 = 9\) (for a pixel that has a cosmic ray and was affected by vignetting).

You can later use this bit information to mark objects in your final analysis or to mask certain pixels. For example, you may want to set all pixels affected by vignetting to NaN, but can interpolate over cosmic rays. You therefore need ways to separate the pixels with a desired flag(s) from the rest. It is possible to treat a flag pixel as a single integer (and try to define certain ranges in value to select certain flags). But a much more easier and robust way is to actually look at each pixel as a sequence of bits (not as a single integer!) and use the bitwise operators below for this job. For more on the theory behind bitwise operators, see Wikipedia.

bitand

Bitwise AND operator: only bits with values of 1 in both popped operands will get the value of 1, the rest will be set to 0. For example, (assuming numbers can be written as bit strings on the command-line): 00101000 00100010 bitand will give 00100000. Note that the bitwise operators only work on integer type datasets.

bitor

Bitwise inclusive OR operator: The bits where at least one of the two popped operands has a 1 value get a value of 1, the others 0. For example, (assuming numbers can be written as bit strings on the command-line): 00101000 00100010 bitand will give 00101010. Note that the bitwise operators only work on integer type datasets.

bitxor

Bitwise exclusive OR operator: A bit will be 1 if it differs between the two popped operands. For example, (assuming numbers can be written as bit strings on the command-line): 00101000 00100010 bitand will give 00001010. Note that the bitwise operators only work on integer type datasets.

lshift

Bitwise left shift operator: shift all the bits of the first operand to the left by a number of times given by the second operand. For example, (assuming numbers can be written as bit strings on the command-line): 00101000 2 lshift will give 10100000. This is equivalent to multiplication by 4. Note that the bitwise operators only work on integer type datasets.

rshift

Bitwise right shift operator: shift all the bits of the first operand to the right by a number of times given by the second operand. For example, (assuming numbers can be written as bit strings on the command-line): 00101000 2 rshift will give 00001010. Note that the bitwise operators only work on integer type datasets.

bitnot

Bitwise not (more formally known as one’s complement) operator: flip all the bits of the popped operand (note that this is the only unary, or single operand, bitwise operator). In other words, any bit with a value of 0 is changed to 1 and vice-versa. For example, (assuming numbers can be written as bit strings on the command-line): 00101000 bitnot will give 11010111. Note that the bitwise operators only work on integer type datasets/numbers.