See some examples of how to manipulate and redirect with file descriptors.
- [Narrator] Playing with file descriptor and redirection. In the previous video, we've learned how to math with the shell. Let's move on to the topic. In this video, we'll show examples on how to manipulate and redirect with file descriptors. File descriptors are integers that are associated with file input and output. They keep track of open files. The best known file descriptors are stdin, stdout, and stderr. We can even redirect the contents of one file descriptor to another.
While writing scripts, we use standard input, stdin, standard output, stdout, and standard error, stderr frequently. Redirection of an output to a file by filtering its contents is one of the essential things we need to perform. While a command outputs in text, it can be either an error or an output, non-error message. We cannot distinguish whether it is an output text or an error text just by looking at it. However, we can handle them with file descriptors. We can extract text that is attached to a specific descriptor.
File descriptors are integers associated with an open file or data stream. File descriptors zero, one, and two are reserved as zero, stdin, standard input, one stdout, standard output two, stderr, standard error. Let's see how to do it. Redirecting or saving output text to a file can be done as-- This would store the echo text in temp.text. By truncating the file the contents would be emptied before writing.
To append text to a file, consider this as an example. You can view the contents of the file as follows. Let's see what a standard error is and how you can redirect. Stderr messages are printed when commands output an error message. Consider this as an example. Here, plus is an invalid argument and hence an error is returned. Successful and unsuccessful commands. When a command returns after an error, it returns a non-zero exit status.
The command returns zero when it terminates after a successful completion. The return status can be read from special variable dollar/question mark. Run echo, dollar, question mark immediately after the command execution statement to print the exit status. The following command prints the stderr text to the screen rather than to a file. And because there is no stdout output, out.text will be empty. In the following command, we redirect stderr to out.text.
You can redirect stderr exclusively to a file and stdout to another file as-- It is also possible to redirect stderr and stdout to a single file by converting stderr to stdout using this preferred method. Or the alternative approach. Sometimes the output may contain unnecessary information such as debug messages. If you don't want the output terminal burdened with the stderr details, then you should redirect the stdout ouput to /dev, /null which removes it completely.
For example, consider that we have three files: b1, b2, and b3. However, b1 does not have the read/write execute permission for the user. When you need to print the contents of a file starting with a, we use the cat command. Set up the test files as-- While displaying contents of the files using wildcards, b*, it will show an error message for file b1 as it does not have the proper read permission. Here, cat:b1:permissiondenied belongs to the stderr data.
We can direct the stderr data into a file whereas stdout remains printed on the terminal. Consider the code. Take a look at the code. When redirection is performed for stderr or stdout, the redirected text flows into a file. After text has already been redirected and has gone into the file, no text remains to flow to the next command through pipe. And it appears to the next set of command sequences through stdin.
However, there is a way to redirect data to a file as well. Let's provide a copy of redirected data as stdin for the next set of commands. This can be done using the t command. For example, to print stdout in the terminal as well as redirect stdout into a file, the syntax for t is-- In this code, the stdin data is received by the t command. It writes the copy of stdout to the out.text file and sends another copy as stdin for the next command.
The cat/end command puts a line number for each line received from stdin and writes it to stdout. Examine the contents of out.text as-- Note that cat b1 permission denied does not appear because it belongs to stderr. The t command can read from stdin only. By default the t command overwrites the file. But, it can be used with appended options by providing the /a option.
For example, $ cat b*, pipe, tee-b out.text, pipe cat/end. Commands appear with these arguments in the format command, file one, file two, or simply command/file. We can use stdin as a command argument. It can be done by using the dash as the file name argument for the command as-- For example, alternatively we can use /dev, /stdin as the output file name to use stdin.
Similarly, use /dev, /stderr for standard error and /dev, /stdout for standard output. These are special device files that correspond to stdin, stderr, and stdout. Let's see how it works. For output redirection, these operators are different. Both of them redirect text to a file but the first one empties the file and then writes to it whereas the latter one adds the output to the end of the existing file. When we use a redirection operator, the output won't print in the terminal but it is directed to a file.
When redirection operators are used, by default they operate on standard output. To explicitly take a specific file descriptor, you must prefix the descriptor number to the operator. This operator is equivalent to one. This operator and similarly, it applies for this operator. When working with errors, the stderr output is dumped into the /dev, /null file. /dev, /null is a special device file where any data received by the file is discarded. The null device is often known as a black hole as all the data that goes into it is lost forever.
A command that reads stdin for input can receive data in multiple ways. Also, it's possible to specify file descriptors of our own using cat and pipes for example. Redirection from a file to a command. By using redirection, we can read data from a file as stdin as-- Redirecting from a text block enclosed within a script. Sometimes, we need to redirect a block of text as standard input.
Consider a particular case where the source text is places within the shell script. A practical usage example is writing a log file header data. It can be performed as follows. Lines that appear between cat eof log.text and the next eof line will appear as the stdin data. Print the contents of log text as-- Custom file descriptors. A file descriptor is an abstract indicator for accessing a file.
Each file access is associated with a special number called a file descriptor. Zero, one, and two are reserved descriptor numbers for stdin, stdout, and stderr. We can create our own custom file descriptors using the exec command. If you are already familiar with file programming with any other programming language, you might have noticed modes for opening files. Usually, the following three modes are used: read mode, write with truncate mode, write with append mode.
This is not pro to use to read from the file to stdin. This is the operator used to write to a file with truncation. Data is written to a target file after truncating the contents. This is not pro to use to write to a file by appending. Data is appended to the existing file contents and the contents of a target file will not be lost. File descriptors can be created with one over three modes. Create a file descriptor for reading a file as-- We could use it this way. Now, you can use file descriptor three with commands.
For example, we will use cat<&3 as-- If a second read is required, we cannot reuse the file descriptor three. It's required that we reassign the file descriptor three for read using exec before making a second read. Create a file descriptor for writing truncate mode as-- For example-- Create a file descriptor for writing append mode as-- For example-- Awesome.
In this video, we have successfully learned how to play with file descriptors and redirection. Hope you enjoyed watching this video. In the next video, we'll see about arrays and associative arrays.
Note: This course was created by Packt Publishing. We are pleased to host this training in our library.
- Printing in the terminal
- Performing math in the Linux shell
- Getting and setting dates
- Working with functions and arguments
- Reading output
- Making comparisons
- Concatenating text
- Finding, editing, generating, and deleting files
- Running parallel processes
- Using regular expressions
- Downloading webpages
- Parsing data from a website
- Finding broken links
- Backing up and archiving
- Transferring files and data through the network
- Monitoring your Linux system
- Gathering data for system administration