3.3.2 Multiple Files

Sometimes you need to process files one at a time. But usually this is not necessary, and, it is faster to run a command on as many files as possible at a time, rather than once per file. Doing this saves on the time it takes to start up the command each time.

The ‘-execdir’ and ‘-exec’ actions have variants that build command lines containing as many matched files as possible.

Action: -execdir command {} +

This works as for ‘-execdir command ;’, except that the result is always true, and the ‘{}’ at the end of the command is expanded to a list of names of matching files. This expansion is done in such a way as to avoid exceeding the maximum command line length available on the system. Only one ‘{}’ is allowed within the command, and it must appear at the end, immediately before the ‘+’. A ‘+’ appearing in any position other than immediately after ‘{}’ is not considered to be special (that is, it does not terminate the command).

Action: -exec command {} +

This insecure variant of the ‘-execdir’ action is specified by POSIX. The main difference is that the command is executed in the directory from which find was invoked, meaning that ‘{}’ is expanded to a relative path starting with the name of one of the starting directories, rather than just the basename of the matched file. The result is always true.

Before find exits, any partially-built command lines are executed. This happens even if the exit was caused by the ‘-quit’ action. However, some types of error (for example not being able to invoke stat() on the current directory) can cause an immediate fatal exit. In this situation, any partially-built command lines will not be invoked (this prevents possible infinite loops).

At first sight, it looks like the list of filenames to be processed can only be at the end of the command line, and that this might be a problem for some commands (cp and rsync for example).

However, there is a slightly obscure but powerful workaround for this problem which takes advantage of the behaviour of sh -c:

find startpoint -tests … -exec sh -c 'scp "$@" remote:/dest' sh {} +

In the example above, the filenames we want to work on need to occur on the scp command line before the name of the destination. We use the shell to invoke the command scp "$@" remote:/dest and the shell expands "$@" to the list of filenames we want to process.

Another, but less secure, way to run a command on more than one file at once, is to use the xargs command, which is invoked like this:

xargs [option] [command [initial-arguments]]

xargs normally reads arguments from the standard input. These arguments are delimited by blanks (which can be protected with double or single quotes or a backslash) or newlines. It executes the command (the default is echo) one or more times with any initial-arguments followed by arguments read from standard input. Blank lines on the standard input are ignored. If the ‘-L’ option is in use, trailing blanks indicate that xargs should consider the following line to be part of this one.

Instead of blank-delimited names, it is safer to use ‘find -print0’ or ‘find -fprint0’ and process the output by giving the ‘-0’ or ‘--null’ option to GNU xargs, GNU tar, GNU cpio, or perl. The locate command also has a ‘-0’ or ‘--null’ option which does the same thing.

You can use shell command substitution (backquotes) to process a list of arguments, like this:

grep -l sprintf `find $HOME -name '*.c' -print`

However, that method produces an error if the length of the ‘.c’ file names exceeds the operating system’s command line length limit. xargs avoids that problem by running the command as many times as necessary without exceeding the limit:

find $HOME -name '*.c' -print | xargs grep -l sprintf

However, if the command needs to have its standard input be a terminal (less, for example), you have to use the shell command substitution method or use either the ‘--arg-file’ option or the ‘--open-tty’ option of xargs.

The xargs command will usually process all of its input, building command lines and executing them. The processing stops earlier and immediately if the tool reads a line containing the end-of-file marker string specified with the ‘--eof’ option, or if one of the launched commands exits with a status of 255. The latter will cause xargs to issue an error message and exit with status 124.