6.19.1 Foreign Libraries

Just as Guile can load up Scheme libraries at run-time, Guile can also load some system libraries written in C or other low-level languages. We refer to these as dynamically-loadable modules as foreign libraries, to distinguish them from native libraries written in Scheme or other languages implemented by Guile.

Foreign libraries usually come in two forms. Some foreign libraries are part of the operating system, such as the compression library libz. These shared libraries are built in such a way that many programs can use their functionality without duplicating their code. When a program written in C is built, it can declare that it uses a specific set of shared libraries. When the program is run, the operating system takes care of locating and loading the shared libraries.

The operating system components that can dynamically load and link shared libraries when a program is run are also available programmatically during a program’s execution. This is the interface that’s most useful for Guile, and this is what we mean in Guile when we refer to dynamic linking. Dynamic linking at run-time is sometimes called dlopening, to distinguish it from the dynamic linking that happens at program start-up.

The other kind of foreign library is sometimes known as a module, plug-in, bundle, or an extension. These foreign libraries aren’t meant to be linked to by C programs, but rather only to be dynamically loaded at run-time – they extend some main program with functionality, but don’t stand on their own. Sometimes a Guile library will implement some of its functionality in a loadable module.

In either case, the interface on the Guile side is the same. You load the interface using load-foreign-library. The resulting foreign library object implements a simple lookup interface whereby the user can get addresses of data or code exported by the library. There is no facility to inspect foreign libraries; you have to know what’s in there already before you look.

Routines for loading foreign libraries and accessing their contents are implemented in the (system foreign-library) module.

(use-modules (system foreign-library))

Scheme Procedure: load-foreign-library [library] [#:extensions=system-library-extensions] [#:search-ltdl-library-path?=#t] [#:search-path=search-path] [#:search-system-paths?=#t] [#:lazy?=#t] [#:global=#f] [#:host-type-rename?=#t] [#:allow-dll-version-suffix?=#t] ¶

This procedure finds the shared library denoted by library (a string) and dynamically links it into the running Guile application. On success, the procedure returns a Scheme object suitable for representing the linked object file. Otherwise an error is thrown.

In the common usage, the library parameter is a filename with no path and with no filename extension, such as .so, .dylib or .dll. The procedure will search for the library in a set of standard locations using the common filename extensions for the OS. The optional parameters can customize this behavior.

When library has directory elements or a filename extension, a more targeted search is performed.

For each system, Guile has a default set of extensions that it will try. On GNU systems, the default extension set is just .so; on Windows, just .dll; and on Darwin (Mac OS), it is .bundle, .so, and .dylib. Pass #:extensions extensions to override the default extensions list. If library contains one of the extensions, no extensions are tried, so it is possible to specify the extension if you know exactly what file to load.

Unless library denotes an absolute file name or otherwise contains a directory separator (/, and also \ on Windows), Guile will search for the library in the directories listed in search-paths. The default search path has three components, which can all be overridden by colon-delimited (semicolon on Windows) environment variables:

GUILE_EXTENSIONS_PATH

This is the environment variable for users to add directories containing Guile extensions to the search path. The default value has no entries. This environment variable was added in Guile 3.0.6.

LTDL_LIBRARY_PATH

When search-ltdl-library-path? is true, this environment variable can also be used to add directories to the search path. For each directory given in this environment variable, two directories are added to the search path: the given directory (for example, D) and a .libs subdirectory (D/.libs).

For more information on the rationale, see the note below.

GUILE_SYSTEM_EXTENSIONS_PATH

The last path in Guile’s search path belongs to Guile itself, and defaults to the libdir and the extensiondir, in that order. For example, if you install to /opt/guile, these would probably be /opt/guile/lib and /opt/guile/lib/guile/3.0/extensions, respectively. See Parallel Installations, for more details on extensionsdir.

For DLL-using systems, it searches bindir rather than libdir, so /opt/guile/bin in this example.

Finally, if no library is found in the search path, and if library is not absolute and does not include directory separators, and if search-system-paths? is true, the operating system may have its own logic for where to locate library. For example, on GNU, there will be a default set of paths (often /usr/lib and /lib, though it depends on the system), and the LD_LIBRARY_PATH environment variable can add additional paths. On DLL-using systems, the PATH is searched. Other operating systems have other conventions.

Falling back to the operating system for search is usually not a great thing; it is a recipe for making programs that work on one machine but not on others. Still, when wrapping system libraries, it can be the only way to get things working at all.

If lazy? is true (the default), Guile will request the operating system to resolve symbols used by the loaded library as they are first used. If global? is true, symbols defined by the loaded library will be available when other modules need to resolve symbols; the default is #f, which keeps symbols local.

If host-type-rename? is true (the default) library names may be modified based on the current %host-type. On Cygwin hosts, the search behavior is modified such that a filename that starts with “lib” will be searched for under the name “cyg”, as is customary for Cygwin. Similarly, for MSYS hosts, “lib” becomes “msys-”.

If dll-version-suffix? is true (the default), the search behavior is modified such that when searching for a DLL, it will also search for DLLs with version suffixes. For example, a search for libtiff.dll will also allow libtiff-1.dll. When the unversioned DLL is not found and multiple versioned DLLs exists, it will return the versioned DLL with the highest version. Note that when searching, directories take precedence. It does not return the highest versioned DLL among all search directories collectively; it returns the highest versioned in the first directory to have the DLL.

If library argument is omitted, it defaults to #f. If library is false, the resulting foreign library gives access to all symbols available for dynamic linking in the currently running executable.

The environment variables mentioned above are parsed when the foreign-library module is first loaded and bound to parameters. Null path components, for example the three components of GUILE_SYSTEM_EXTENSIONS_PATH="::", are ignored.

Scheme Parameter: guile-extensions-path ¶

Scheme Parameter: ltdl-library-path ¶

Scheme Parameter: guile-system-extensions-path ¶

Parameters whose initial values are taken from GUILE_EXTENSIONS_PATH, LTDL_LIBRARY_PATH, and GUILE_SYSTEM_EXTENSIONS_PATH, respectively. See Parameters. The current values of these parameters are used when building the search path when load-foreign-library is called, unless the caller explicitly passes a #:search-path argument.

Scheme Procedure: foreign-library? obj ¶

Return #t if obj is a foreign library, or #f otherwise.

Before Guile 3.0.6, Guile loaded foreign libraries using libltdl, the dynamic library loader provided by libtool. This loader used LTDL_LIBRARY_PATH, and for backwards compatibility we still support that path.

However, libltdl would not only open .so (or .dll and so on) files, but also the .la files created by libtool. In installed libraries – libraries that are in the target directories of make install – .la files are never needed, to the extent that most GNU/Linux distributions remove them entirely. It is sufficient to just load the .so (or .dll and so on) files, which are always located in the same directory as the .la files.

But for uninstalled dynamic libraries, like those in a build tree, the situation is a bit of a mess. If you have a project that uses libtool to build libraries – which is the case for Guile, and for most projects using autotools – and you build foo.so in directory D, libtool will put foo.la in D, but foo.so gets put into D/.libs.

Users were mostly oblivious to this situation, as libltdl had special logic to be able to read the .la file to know where to find the .so, even from an uninstalled build tree, preventing the existence of .libs from leaking out to the user.

We don’t use libltdl now, essentially for flexibility and error-reporting reasons. But, to keep this old use-case working, if search-ltdl-library-path? is true, we add each entry of LTDL_LIBRARY_PATH to the default extensions load path, additionally adding the .libs subdirectories for each entry, in case there are .so files there instead of alongside the .la files.