I/O Redirection
In this lesson, we will explore a powerful feature used
by many command line programs called input/output redirection.
As we have seen, many commands such as ls print their output on the
display. This does not have to be the case, however. By using some
special notation we can redirect the output of many commands
to files, devices, and even to the input of other commands.
Most command line programs that display their results do
so by sending their results to a facility called standard output.
By default, standard output directs its contents to the display. To
redirect standard output to a file, the ">" character is used like
this:
[me@linuxbox me]$ ls > file_list.txt
In this example, the ls command is executed and the
results are written in a file named file_list.txt. Since the output of
ls was redirected to the file, no results appear on the display.
Each time the command above is repeated, file_list.txt
is overwritten (from the beginning) with the output of the command ls.
If you want the new results to be appended to the file
instead, use ">>" like this:
[me@linuxbox me]$ ls >> file_list.txt
When the results are appended, the new results are added
to the end of the file, thus making the file longer each time the
command is repeated. If the file does not exist when you attempt to
append the redirected output, the file will be created.
Many commands can accept input from a facility called standard
input. By default, standard input gets its contents from the
keyboard, but like standard output, it can be redirected. To redirect
standard input from a file instead of the keyboard, the "<"
character is used like this:
[me@linuxbox me]$ sort < file_list.txt
In the above example we used the sort
command to process the contents of file_list.txt. The results are
output on the display since the standard output is not redirected in
this example. We could redirect standard output to another file like
this:
[me@linuxbox me]$ sort < file_list.txt >
sorted_file_list.txt
As you can see, a command can have both its input and
output redirected. Be aware that the order of the redirection does not
matter. The only requirement is that the redirection operators (the
"<" and ">") must appear after the other options and arguments in
the command.
By far, the most useful and powerful thing you can do
with I/O redirection is to connect multiple commands together with what
are called pipes. With pipes, the standard output of one
command is fed into the standard input of another. Here is my absolute
favorite:
[me@linuxbox me]$ ls -l | less
In this example, the output of the ls command is fed
into less. By using this "| less" trick, you can make any command have
scrolling output. I use this technique all the time.
By connecting commands together, you can acomplish
amazing feats. Here are some examples you'll want to try:
Examples of commands used together with pipes
| Command |
What it does |
|
ls -lt | head
|
Displays the 10 newest files in the current
directory.
|
|
du | sort -nr
|
Displays a list of directories and how much space
they consume, sorted from the largest to the smallest.
|
|
find .
-type f -print | wc -l
|
Displays the total number of files in the current
working directory and all of its subdirectories.
|
One class of programs you can use with pipes is called filters.
Filters take standard input and perform an operation upon it and send
the results to standard output. In this way, they can be used to
process information in powerful ways. Here are some of the common
programs that can act as filters:
Common filter commands
| Program |
What it does |
|
sort
|
Sorts standard input then outputs the sorted
result on standard output.
|
|
uniq
|
Given a sorted stream of data from standard input,
it removes duplicate lines of data (i.e., it makes sure that every line
is unique).
|
|
grep
|
Examines each line of data it receives from
standard input and outputs every line that contains a specified pattern
of characters.
|
|
fmt
|
Reads text from standard input, then outputs
formatted text on standard output.
|
|
pr
|
Takes text input from standard input and splits
the data into pages with page breaks, headers and footers in
preparation for printing.
|
|
head
|
Outputs the first few lines of its input. Useful
for getting the header of a file.
|
|
tail
|
Outputs the last few lines of its input. Useful
for things like getting the most recent entries from a log file.
|
|
tr
|
Translates characters. Can be used to perform
tasks such as upper/lowercase conversions or changing line termination
characters from one type to another (for example, converting DOS text
files into Unix style text files).
|
|
sed
|
Stream editor. Can perform more sophisticated text
translations than tr.
|
|
awk
|
An entire programming language designed for
constructing filters. Extremely powerful.
|
Performing tasks with pipes
- Printing from the command line. Linux
provides a program called lpr that
accepts standard input and sends it to the printer. It is often used
with pipes and filters. Here are a couple of examples:
cat poorly_formatted_report.txt | fmt | pr | lpr cat unsorted_list_with_dupes.txt | sort | uniq | pr | lpr
In the first example, we use cat to read the file
and output it to standard output, which is piped into the standard
input of fmt. fmt formats the text into neat paragraphs and outputs it
to standard output, which is piped into the standard input of pr. pr
splits the text neatly into pages and outputs it to standard output,
which is piped into the standard input of lpr. lpr takes its standard
input and sends it to the printer.
The second example starts with an unsorted list of
data with duplicate entries. First, cat sends the list into sort which
sorts it and feeds it into uniq which removes any duplicates. Next pr
and lpr are used to paginate and print the list.
- Viewing the contents of tar files Often
you will see software distributed as a gzipped tar file. This
is a traditional Unix style tape archive file (created with tar) that
has been compressed with gzip. You
can recognize these files by their traditional file extensions,
".tar.gz" or ".tgz". You can use the following command to view the
directory of such a file on a Linux system:
tar tzvf name_of_file.tar.gz | less
Linux® is a registered trademark of Linus Torvalds.
|