New Feature: Playlist Center! Pick a topic and let our playlists guide the way.

Start learning with our library of video tutorials taught by experts. Get started

Unix for Mac OS X Users
Illustration by

tr: Deleting and squeezing characters


From:

Unix for Mac OS X Users

with Kevin Skoglund

Video: tr: Deleting and squeezing characters

In the previous movie we saw how to use tr by looking at its standard translate usage. The tr has several options that we can specify that will allow us to work with it in other ways. Instead of translating, it can also allow you to delete or filter out characters into dedupe repeating characters, a process that it refers to as squeezing. The three options we are going to be looking out are the -d, -s and -c options. -d will delete characters in the set that we specify. So it'd just simply remove them. Instead of translating them that will just get removed. The -s option will squeeze repeats that are in the listed set.
Expand all | Collapse all
  1. 3m 57s
    1. Introduction
      1m 14s
    2. Using the exercise files
      2m 43s
  2. 32m 2s
    1. What is Unix?
      7m 27s
    2. The terminal application
      4m 23s
    3. Logging in and using the command prompt
      5m 19s
    4. Command structure
      5m 22s
    5. Kernel and shells
      5m 25s
    6. Unix manual pages
      4m 6s
  3. 15m 58s
    1. The working directory
      2m 49s
    2. Listing files and directories
      3m 59s
    3. Moving around the filesystem
      4m 58s
    4. Filesystem organization
      4m 12s
  4. 1h 4m
    1. Naming files
      5m 41s
    2. Creating files
      2m 19s
    3. Unix text editors
      6m 39s
    4. Reading files
      5m 35s
    5. Reading portions of files
      3m 27s
    6. Creating directories
      2m 40s
    7. Moving and renaming files and directories
      8m 32s
    8. Copying files and directories
      3m 7s
    9. Deleting files and directories
      3m 38s
    10. Finder aliases in Unix
      4m 10s
    11. Hard links
      5m 30s
    12. Symbolic links
      6m 36s
    13. Searching for files and directories
      6m 32s
  5. 34m 58s
    1. Who am I?
      4m 3s
    2. Unix groups
      1m 52s
    3. File and directory ownership
      6m 41s
    4. File and directory permissions
      4m 27s
    5. Setting permissions using alpha notation
      6m 49s
    6. Setting permissions using octal notation
      3m 49s
    7. The root user
      1m 57s
    8. sudo and sudoers
      5m 20s
  6. 52m 34s
    1. Command basics
      4m 4s
    2. The PATH variable
      4m 13s
    3. System information commands
      3m 40s
    4. Disk information commands
      6m 8s
    5. Viewing processes
      5m 0s
    6. Monitoring processes
      3m 36s
    7. Stopping processes
      3m 19s
    8. Text file helpers
      6m 50s
    9. Utility programs
      7m 28s
    10. Using the command history
      8m 16s
  7. 20m 39s
    1. Standard input and standard output
      1m 24s
    2. Directing output to a file
      4m 13s
    3. Appending to a file
      2m 44s
    4. Directing input from a file
      5m 28s
    5. Piping output to input
      4m 40s
    6. Suppressing output
      2m 10s
  8. 41m 28s
    1. Profile, login, and resource files
      9m 11s
    2. Setting command aliases
      6m 59s
    3. Setting and exporting environment variables
      4m 54s
    4. Setting the PATH variable
      6m 10s
    5. Configuring history with variables
      6m 17s
    6. Customizing the command prompt
      6m 5s
    7. Logout file
      1m 52s
  9. 1h 25m
    1. grep: Searching for matching expressions
      5m 21s
    2. grep: Multiple files, other input
      4m 28s
    3. grep: Coloring matched text
      2m 57s
    4. Introduction to regular expressions
      3m 22s
    5. Regular expressions: Basic syntax
      3m 19s
    6. Using regular expressions with grep
      5m 20s
    7. tr: Translating characters
      8m 17s
    8. tr: Deleting and squeezing characters
      5m 30s
    9. sed: Stream editor
      7m 45s
    10. sed: Regular expressions and back-references
      7m 8s
    11. cut: Cutting select text portions
      7m 42s
    12. diff: Comparing files
      4m 35s
    13. diff: Alternative formats
      4m 30s
    14. xargs: Passing argument lists to commands
      7m 25s
    15. xargs: Usage examples
      7m 59s
  10. 42m 25s
    1. Finder integration
      4m 45s
    2. Clipboard integration
      5m 5s
    3. Screen capture
      3m 42s
    4. Shut down, reboot, and sleep
      3m 34s
    5. Text to speech
      2m 36s
    6. Spotlight integration: Searching metadata
      3m 41s
    7. Spotlight integration: Metadata attributes
      4m 24s
    8. Using AppleScript
      5m 23s
    9. System configurations: Viewing and setting
      5m 51s
    10. System configurations: Examples
      3m 24s
  11. 1m 26s
    1. Conclusion
      1m 26s

Watch this entire course now—plus get access to every course in the library. Each course includes high-quality videos taught by expert instructors.

Become a member
please wait ...
Unix for Mac OS X Users
6h 35m Beginner Apr 29, 2011

Viewers: in countries Watching now:

Unix for Mac OS X Users unlocks the powerful capabilities of Unix that underlie Mac OS X, teaching how to use command-line syntax to perform common tasks such as file management, data entry, and text manipulation. The course teaches Unix from the ground up, starting with the basics of the command line and graduating to powerful, advanced tools like grep, sed, and xargs. The course shows how to enter commands in Terminal to create, move, copy, and delete files and folders; change file ownership and permissions; view and stop command and application processes; find and edit data within files; and use command-line shortcuts to speed up workflow. Exercise files accompany the course.

Topics include:
  • Moving around the file system
  • Creating and reading files
  • Copying, moving, renaming, and deleting files and directories
  • Creating hard links and symbolic links
  • Understanding user identity, file ownership, and sudo
  • Setting file permissions with alpha and octal notation
  • Changing the PATH variable
  • Using the command history
  • Directing input and output
  • Configuring the Unix working environment
  • Searching and replacing using grep and regular expressions
  • Manipulating text with tr, sed, and cut
  • Integrating with the Finder, Spotlight, and AppleScript
Subjects:
Developer Web
Software:
Mac OS X Unix
Author:
Kevin Skoglund

tr: Deleting and squeezing characters

In the previous movie we saw how to use tr by looking at its standard translate usage. The tr has several options that we can specify that will allow us to work with it in other ways. Instead of translating, it can also allow you to delete or filter out characters into dedupe repeating characters, a process that it refers to as squeezing. The three options we are going to be looking out are the -d, -s and -c options. -d will delete characters in the set that we specify. So it'd just simply remove them. Instead of translating them that will just get removed. The -s option will squeeze repeats that are in the listed set.

So it won't necessarily squeeze everything, but the things that we've told it to, it will compress down. If we had for example five Xs in a row, it would turn it into one X instead, and then the -c option says Use the complementary set. It's the opposite. So you would use this option either with either the -d or the -s option. With the -d option I mean delete characters that are not in the listed set, right. It's the opposite of that, the reverse. It will become clear once we actually look at some examples. So let's say that I have a simple string. I have "abc1233deee567f".

The threes and the Es are repeated. I am going to take that string. I am going to pipe it into tr, I am going to use the -d option, and I am going to say the set that I want to specify, the thing I'm looking for, is this class called digit. So what I am doing is deleting all of the digits and you can see the output there to the far right. It would then just give me the letters. If use the -c that's the complementary set. So now I'm saying delete everything that is not a digit. So what I'm left with is just the digits. Now that's not just stripping out the letters. That's also stripping out tabs, line returns, anything else that might be in that file at all, anything that's not a digit.

So a lot of times it's easier to specify not the thing you're looking for, the thing that you don't want, right. We are filtering out the opposite version. It can be very handy. Notice that in both of those cases there is only one argument tr now. Before we had to. We were saying translate from a to b; now we are just saying we want to delete from set a. We don't need to translate to the other set. So there is no reason to have a second argument. Squeeze works the same way. We just have one argument. This time I have tr -s digit and that squeezes all the digits. So you notice in the result there to the far right that now the threes have just become a single 3. It's just 1, 2, 3.

If we use the complementary set, well then it's the opposite. Squeeze everything that's not a digit, so that squeezes the es. So now there's only one e. There are still two 3s in there. It's 1233. It's the es that got compressed. Now again that's the complementary set, so everything that's not a digit. That would be line returns, that would be tabs, anything else that's in that file that's not a digit would get squeezed. We can use these two options together. So for example, if I have tr -ds, the first argument you are going to specify is going to be the thing you want to delete.

The second argument is going to be the thing that you want to squeeze. So in the first example I have there, it's going to delete all the digits and then it's going to squeeze all of the letters, and then we come up with abcdef. Now if we use the -c option with that as well, it's important to note that the -c option only applies to the delete. So what the example I have there will do is it will delete everything that's not a digit and then squeeze the digits. It's not squeezing the opposite of the digits, right. -c just applies to the first argument to the -d. So that's how it works, but you may be thinking well why in the world would I need something like this? I mean if you have something like a file that has an essay in it, why would you want to transform your letters so that the word sweet became swet? That's not that useful.

Well, admittedly you won't use it that often, but it is a good tool to have it in your toolbox for a couple of special cases. Let's say for example that you want to remove certain characters from the file. We can remove all no-nprintable characters from the file for example, right. So remove everything that's not a printable character out of the file and then presumably I'd be able to print the file, right? You could also remove other things. You can remove all of the tabs from a file. You can remove all the line returns from a file. You can remove all the punctuation, right. Any of those things are possibilities, but essentially we are saying go through the file and remove the things that I don't want in there.

One common use case that I want to give you just so that you have it and can refer to it is that you can remove the surplus carriage returns and end of file characters that Windows files often have. For the end of line in a Windows file it has both a carriage return and the line feed, but on Mac and Unix, that's just a line feed character. So what we want to do is get rid of extra carriage return character and then Windows must also have an end of file character to indicate the end of the file, which Unix sometimes doesn't like as well. So we can strip those out and essentially take something that was a Windows file and make it a happy Unix file by stripping out the hex codes that I have there. That /015/032? There is no way you would know this.

You would have to look those up to know what they are, but I wanted to give them to you because this is a handy tool to have in your toolbox. And an example where you might you squeeze? Well, you might want to remove all the double spaces from your file, right. So every time you hit a period, if you accidentally hit the spacebar twice, you really wish you'd only done it once, well, we can squeeze all those spaces out and not just double spaces, but if you accidentally hit triple spaces or however many more there was, this would squeeze them all down, so that they would only be one space. There are all sorts of variations on these. There are ways that you can combine these. You could for example remove all characters that are not printable and squeeze the spaces in the file, do those together.

The main thing I want you to see is that tr doesn't just translate. It also has these other special features which almost could be programs on their own, but they're really just options to tr. So tr -d and tr -s. So make sure that you have tr -d and tr -s in your toolbox.

Find answers to the most frequently asked questions about Unix for Mac OS X Users.


Expand all | Collapse all
please wait ...
Q: The exercise files for the following movies appear to be broken:
07_02_files
07_03_files
07_04_files
07_05_files
08_03_files

Is there something wrong with them?
These exercises include one or more "dot files", whose file names start with a period. These files are normally hidden from view by the Finder.  So that they would show up in the Finder, the period has been removed from the file names. Additionally, "_example" has been added at the end of the file name to make it clear that the file will not work as-is. To make the dot files usable, either:

1) Open the file in a text editor to view its contents. Note that it may not be possible to double-click the file to open it because there is no file extension (such as .txt).
2) Resave the file under a new name (usually by choosing File > Save As), adding a "." to the beginning of the file name and removing "_example" from the end.

OR

1) Copy and rename the file from the Unix command line using the techniques discussed in this course. Rename the file by adding a "." to the start and removing "_example" from the end. Include the "-i" option to prevent overwriting an existing file unexpectedly.
Example:  cp -i ~/Desktop/Exercise\ Files/Chapter_07/07_02_files/bashrc_example ~/.bashrc
 
Share a link to this course

What are exercise files?

Exercise files are the same files the author uses in the course. Save time by downloading the author's files instead of setting up your own files, and learn by following along with the instructor.

Can I take this course without the exercise files?

Yes! If you decide you would like the exercise files later, you can upgrade to a premium account any time.

Become a member Download sample files See plans and pricing

Please wait... please wait ...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.
Upgrade now


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ.

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

join now Upgrade now

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed Unix for Mac OS X Users.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferencesfrom the dropdown menu.

Continue to classic layout Stay on new layout
Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Are you sure you want to delete this note?

No

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.