Chapter 17. Here Documents

A here document uses a special form of I/O redirection to feed a command script to an interactive program, such as ftp, telnet, or ex. Typically, the script consists of a command list to the program, delineated by a limit string. The special symbol << precedes the limit string. This has the effect of redirecting the output of a file into the program, similar to interactive-program < command-file, where command-file contains

command #1
command #2
...

The "here document" alternative looks like this:

#!/bin/bash
interactive-program <<LimitString
command #1
command #2
...
LimitString

Choose a limit string sufficiently unusual that it will not occur anywhere in the command list and confuse matters.

Note that here documents may sometimes be used to good effect with non-interactive utilities and commands.

Example 17-1. dummyfile: Creates a 2-line dummy file

#!/bin/bash

# Non-interactive use of 'vi' to edit a file.
# (Will not work with 'vim', for some reason.)
# Emulates 'sed'.

E_BADARGS=65

if [ -z "$1" ]
then
  echo "Usage: `basename $0` filename"
  exit $E_BADARGS
fi

TARGETFILE=$1

# Insert 2 lines in file, then save.
#--------Begin here document-----------#
vi $TARGETFILE <<x23LimitStringx23
i
This is line 1 of the example file.
This is line 2 of the example file.
^[
ZZ
x23LimitStringx23
#----------End here document-----------#

# Note that ^[ above is a literal escape
# typed by Control-V Escape.

exit 0

The above script could just as effectively have been implemented with ex, rather than vi. Here documents containing a list of ex commands are common enough to form their own category, known as ex scripts.

Example 17-2. broadcast: Sends message to everyone logged in

#!/bin/bash

wall <<zzz23EndOfMessagezzz23
E-mail your noontime orders for pizza to the system administrator.
    (Add an extra dollar for anchovy or mushroom topping.)
# Additional message text goes here.
# Note: Comment lines printed by 'wall'.
zzz23EndOfMessagezzz23

# Could have been done more efficiently by
#         wall <message-file
# However, saving a message template in a script saves work.

exit 0

Example 17-3. Multi-line message using cat

#!/bin/bash

# 'echo' is fine for printing single line messages,
#  but somewhat problematic for for message blocks.
#  A 'cat' here document overcomes this limitation.

cat <<End-of-message
-------------------------------------
This is line 1 of the message.
This is line 2 of the message.
This is line 3 of the message.
This is line 4 of the message.
This is the last line of the message.
-------------------------------------
End-of-message

exit 0


#--------------------------------------------
# Code below disabled, due to "exit 0" above.

# S.C. points out that the following also works.
echo "-------------------------------------
This is line 1 of the message.
This is line 2 of the message.
This is line 3 of the message.
This is line 4 of the message.
This is the last line of the message.
-------------------------------------"
# However, text may not include double quotes unless they are escaped.

The - option to mark a here document limit string (<<-LimitString) suppresses tabs (but not spaces) in the output. This may be useful in making a script more readable.

Example 17-4. Multi-line message, with tabs suppressed

#!/bin/bash
# Same as previous example, but...

#  The - option to a here document <<-
#  suppresses tabs in the body of the document, but *not* spaces.

cat <<-ENDOFMESSAGE
        This is line 1 of the message.
        This is line 2 of the message.
        This is line 3 of the message.
        This is line 4 of the message.
        This is the last line of the message.
ENDOFMESSAGE
# The output of the script will be flush left.
# Leading tab in each line will not show.

# Above 5 lines of "message" prefaced by a tab, not spaces.
# Spaces not affected by   <<-  .


exit 0

A here document supports parameter and command substitution. It is therefore possible to pass different parameters to the body of the here document, changing its output accordingly.

Example 17-5. Here document with parameter substitution

#!/bin/bash
# Another 'cat' here document, using parameter substitution.

# Try it with no command line parameters,   ./scriptname
# Try it with one command line parameter,   ./scriptname Mortimer
# Try it with one two-word quoted command line parameter,
#                           ./scriptname "Mortimer Jones"

CMDLINEPARAM=1     # Expect at least command line parameter.

if [ $# -ge $CMDLINEPARAM ]
then
  NAME=$1          # If more than one command line param,
                   # then just take the first.
else
  NAME="John Doe"  # Default, if no command line parameter.
fi  

RESPONDENT="the author of this fine script"  
  

cat <<Endofmessage

Hello, there, $NAME.
Greetings to you, $NAME, from $RESPONDENT.

# This comment shows up in the output (why?).

Endofmessage

# Note that the blank lines show up in the output.
# So does the "comment".

exit 0

Quoting or escaping the "limit string" at the head of a here document disables parameter substitution within its body. This has very limited usefulness.

Example 17-6. Parameter substitution turned off

#!/bin/bash
#  A 'cat' here document, but with parameter substitution disabled.

NAME="John Doe"
RESPONDENT="the author of this fine script"  

cat <<'Endofmessage'

Hello, there, $NAME.
Greetings to you, $NAME, from $RESPONDENT.

Endofmessage

#  No parameter substitution when the "limit string" is quoted or escaped.
#  Either of the following at the head of the here document would have the same effect.
#  cat <<"Endofmessage"
#  cat <<\Endofmessage

exit 0

This is a useful script containing a here document with parameter substitution.

Example 17-7. upload: Uploads a file pair to "Sunsite" incoming directory

#!/bin/bash
# upload.sh

# Upload file pair (Filename.lsm, Filename.tar.gz)
# to incoming directory at Sunsite (metalab.unc.edu).

E_ARGERROR=65

if [ -z "$1" ]
then
  echo "Usage: `basename $0` filename"
  exit $E_ARGERROR
fi  


Filename=`basename $1`           # Strips pathname out of file name.

Server="metalab.unc.edu"
Directory="/incoming/Linux"
# These need not be hard-coded into script,
# but may instead be changed to command line argument.

Password="your.e-mail.address"   # Change above to suit.

ftp -n $Server <<End-Of-Session
# -n option disables auto-logon

user anonymous "$Password"
binary
bell                # Ring 'bell' after each file transfer
cd $Directory
put "$Filename.lsm"
put "$Filename.tar.gz"
bye
End-Of-Session

exit 0

It is possible to use : as a dummy command accepting output from a here document. This, in effect, creates an "anonymous" here document.

Example 17-8. "Anonymous" Here Document

#!/bin/bash

: <<TESTVARIABLES
${HOSTNAME?}${USER?}${MAIL?}  # Print error message if one of the variables not set.
TESTVARIABLES

exit 0

Note

Here documents create temporary files, but these files are deleted after opening and are not accessible to any other process.

bash$ bash -c 'lsof -a -p $$ -d0' << EOF
> EOF
lsof    1213 bozo    0r   REG    3,5    0 30386 /tmp/t1213-0-sh (deleted)
              

Caution

Some utilities will not work in a here document.

For those tasks too complex for a "here document", consider using the expect scripting language, which is specifically tailored for feeding input into interactive programs.