12.1.2 Linking ¶

To enhance modularity, similar functions are defined in one source file (with a .c suffix, see Headers for more). After running make, each human-readable, .c file is translated (or compiled) into a computer-readable “object” file (ending with .o). Note that object files are also created when building programs, they are not particular to libraries. Try opening Gnuastro’s lib/ and bin/progname/ directories after running make to see these object files²⁵¹. Afterwards, the object files are linked together to create an executable program or a library.

The object files contain the full definition of the functions in the respective .c file along with a list of any other function (or generally “symbol”) that is referenced there. To get a list of those functions you can use the nm program which is part of GNU Binutils. For example, from the top Gnuastro directory, run:

$ nm bin/arithmetic/arithmetic.o

This will print a list of all the functions (more generally, ‘symbols’) that were called within bin/arithmetic/arithmetic.c along with some further information (for example, a T in the second column shows that this function is actually defined here, U says that it is undefined here). Try opening the .c file to check some of these functions for yourself. Run info nm for more information.

To recap, the compiler created the separate object files mentioned above for each .c file. The linker will then combine all the symbols of the various object files (and libraries) into one program or library. In the case of Arithmetic (a program) the contents of the object files in bin/arithmetic/ are copied (and re-ordered) into one final executable file which we can run from the operating system.

There are two ways to link all the necessary symbols: static and dynamic/shared. When the symbols (computer-readable function definitions in most cases) are copied into the output, it is called static linking. When the symbols are kept in their original file and only a reference to them is kept in the executable, it is called dynamic, or shared linking.

Let’s have a closer look at the executable to understand this better: we will assume you have built Gnuastro without any customization and installed Gnuastro into the default /usr/local/ directory (see Installation directory). If you tried the nm command on one of Arithmetic’s object files above, then with the command below you can confirm that all the functions that were defined in the object file above (had a T in the second column) are also defined in the astarithmetic executable:

$ nm /usr/local/bin/astarithmetic

These symbols/function have been statically linked (copied) in the final executable. But you will notice that there are still many undefined symbols in the executable (those with a U in the second column). One class of such functions are Gnuastro’s own library functions that start with ‘gal_’:

$ nm /usr/local/bin/astarithmetic | grep gal_

These undefined symbols (functions) are present in another file and will be linked to the Arithmetic program every time you run it. Therefore they are known as dynamically linked libraries ²⁵². As we saw above, static linking is done when the executable is being built. However, when a program is dynamically linked to a library, at build-time, the library’s symbols are only checked with the available libraries: they are not actually copied into the program’s executable. Every time you run the program, the (dynamic) linker will be activated and will try to link the program to the installed library before the program starts.

If you want all the libraries to be statically linked to the executables, you have to tell Libtool (which Gnuastro uses for the linking) to disable shared libraries at configure time²⁵³:

$ configure --disable-shared

Try configuring Gnuastro with the command above, then build and install it (as described in Quick start). Afterwards, check the gal_ symbols in the installed Arithmetic executable like before. You will see that they are actually copied this time (have a T in the second column). If the second column does not convince you, look at the executable file size with the following command:

$ ls -lh /usr/local/bin/astarithmetic

It should be around 4.2 Megabytes with this static linking. If you configure and build Gnuastro again with shared libraries enabled (which is the default), you will notice that it is roughly 100 Kilobytes!

This huge difference would have been very significant in the old days, but with the roughly Terabyte storage drives commonly in use today, it is negligible. Fortunately, output file size is not the only benefit of dynamic linking: since it links to the libraries at run-time (rather than build-time), you do not have to rebuild a higher-level program or library when an update comes for one of the lower-level libraries it depends on. You just install the new low-level library and it will automatically be used/linked next time in the programs that use it. To be fair, this also creates a few complications²⁵⁴:

• Reproducibility: Even though your high-level tool has the same version as before, with the updated library, you might not get the same results.
• Broken links: if some functions have been changed or removed in the updated library, then the linker will abort with an error at run-time. Therefore you need to rebuild your higher-level program or library.

To see a list of all the shared libraries that are needed for a program or a shared library to run, you can use GNU C library’s ldd²⁵⁵ program, for example:

$ ldd /usr/local/bin/astarithmetic

Library file names (in their installation directory) start with a lib and their ending (suffix) shows if they are static (.a) or dynamic (.so), as described below. The name of the library is in the middle of these two, for example, libgsl.a or libgnuastro.a (GSL and Gnuastro’s static libraries), and libgsl.so.23.0.0 or libgnuastro.so.4.0.0 (GSL and Gnuastro’s shared library, the numbers may be different).

• A static library is known as an archive file and has the .a suffix. A static library is not an executable file.
• A shared library ends with the .so.X.Y.Z suffix and is executable. The three numbers in the suffix, describe the version of the shared library. Shared library versions are defined to allow multiple versions of a shared library simultaneously on a system and to help detect possible updates in the library and programs that depend on it by the linker.

  It is very important to mention that this version number is different from the software version number (see Version numbering), so do not confuse the two. See the “Library interface versions” chapter of GNU Libtool for more.

  For each shared library, we also have two symbolic links ending with .so.X and .so. They are automatically set by the installer, but you can change them (point them to another version of the library) when you have multiple versions of a library on your system.

Libraries that are built with GNU Libtool (including Gnuastro and its dependencies), build both static and dynamic libraries by default and install them in prefix/lib/ directory (for more on prefix, see Installation directory). In this way, programs depending on the libraries can link with them however they prefer. See the contents of /usr/local/lib with the command below to see both the static and shared libraries available there, along with their executable nature and the symbolic links:

$ ls -l /usr/local/lib/

To link with a library, the linker needs to know where to find the library. At compilation time, these locations can be passed to the linker with two separate options (see Summary and example on libraries for an example) as described below. You can see these options and their usage in practice while building Gnuastro (after running make):

-L DIR

Will tell the linker to look into DIR for the libraries. For example, -L/usr/local/lib, or -L/home/yourname/.local/lib. You can make multiple calls to this option, so the linker looks into several directories at compilation time. Note that the space between L and the directory is optional and commonly ignored (written as -LDIR).

-lLIBRARY

Specify the unique library identifier/name (not containing directory or shared/dynamic nature) to be linked with the executable. As discussed above, library file names have fixed parts which must not be given to this option. So -lgsl will guide the linker to either look for libgsl.a or libgsl.so (depending on the type of linking it is suppose to do). You can link many libraries by repeated calls to this option.

Very important: The place of this option on the compiler’s command matters. This is often a source of confusion for beginners, so let’s assume you have asked the linker to link with library A using this option. As soon as the linker confronts this option, it looks into the list of the undefined symbols it has found until that point and does a search in library A for any of those symbols. If any pending undefined symbol is found in library A, it is used. After the search in undefined symbols is complete, the contents of library A are completely discarded from the linker’s memory. Therefore, if a later object file or library uses an unlinked symbol in library A, the linker will abort after it has finished its search in all the input libraries or object files.

As an example, Gnuastro’s gal_fits_img_read function depends on the fits_read_pix function of CFITSIO (specified with -lcfitsio, which in turn depends on the cURL library, called with -lcurl). So the proper way to link something that uses this function is -lgnuastro -lcfitsio -lcurl. If instead, you give: -lcfitsio -lgnuastro the linker will complain and abort. To avoid such linking complexities when using Gnuastro’s library, we recommend using BuildProgram.

If you have compiled and linked your program with a dynamic library, then the dynamic linker also needs to know the location of the libraries after building the program: every time the program is run afterwards. Therefore, it may happen that you do not get any errors when compiling/linking a program, but are unable to run your program because of a failure to find a library. This happens because the dynamic linker has not found the dynamic library at run time.

To find the dynamic libraries at run-time, the linker looks into the paths, or directories, in the LD_LIBRARY_PATH environment variable. For a discussion on environment variables, especially search paths like LD_LIBRARY_PATH, and how you can add new directories to them, see Installation directory.