Easy-to-follow video tutorials help you learn software, creative, and business skills.Become a member

sed: Stream editor

From: Unix for Mac OS X Users

Video: sed: Stream editor

In this movie, we will take a look at a Unix program called sed. sed is short for Stream Editor. What that means is that sed modifies a stream of input according to a list of commands before passing it on to the output. It's very similar to the way that tr works, but with sed, we can modify the stream of input in many more ways. sed is a complex tool that offers several modes of working and mini features. In this movie, we are just going to focus on the basics and get to see the most common usage. It's most common usage is for doing substitution and so what we'll be doing is sed, space, and then providing an expression for substituting.

sed: Stream editor

In this movie, we will take a look at a Unix program called sed. sed is short for Stream Editor. What that means is that sed modifies a stream of input according to a list of commands before passing it on to the output. It's very similar to the way that tr works, but with sed, we can modify the stream of input in many more ways. sed is a complex tool that offers several modes of working and mini features. In this movie, we are just going to focus on the basics and get to see the most common usage. It's most common usage is for doing substitution and so what we'll be doing is sed, space, and then providing an expression for substituting.

That will always begin with the letter s inside the quotes to indicate that it's going to be doing substitution. Then we have got some delimiters, the forward slash in my example, and then we've got patterns of searching and replacing, just like we had with tr. So we are going to search for whatever is in a and replace it with whatever is in b. Notice that the big difference between this and tr though is that tr actually had two different arguments. It had tr, then the first argument, and then the second argument. Here, they're all in one argument and they are inside those delimiters.

sed is really based on this one expression. So everything that we wanted to do, the sort of command that we are sending to sed, is going to be inside those quotes. Let's take a look at some examples. So, for example, let's take some input, echo upstream, and we are going to pipe that into sed. So then what we needed here is an expression inside quotes for substitution. I am going to go ahead and put my delimiters. Now, inside each of these, we are going to need to fill in what we are searching for. I am going to search for up and we want to replace it with down.

So, upstream becomes downstream, which is appropriate since we are doing stream modification. Upstream, it was upstream. Once it gets downstream, it's downstream. In between is sed doing the stream editing. Now, notice how this is different from what we were doing with tr. tr was translating each of those. What we are doing here is we are searching for one thing and we are replacing it with something else. It's much more like the find and replace that you might do in a word processing program. Now there is one important thing I need to show you about sed, which is that by default it doesn't search globally.

I will show you what I mean. So let's say we have upstream and upward and then let's just hit Return. Notice now it changed the first occurrence of up, not both of them, just the first one got changed. That's different from what we were doing with grep. When we were greping for something, it was finding all occurrences of it and that's because in grep, the g stands for global. It's globally searching. So, just like grep, we need to provide a g modifier here to say that we want to do this globally. So after this pattern of s and then a replace string, then we put a g to say that we would like to do that globally. Downstream and downward now does both of them.

Now, the delimiters that we are using here, these forward slashes, actually are modifiable. It doesn't actually matter which one we have. Let's do, for example, colons. What sed does is it says, all right, what's the first thing that comes after the s? All right, I see the s, I know we are going to do a substitution, what delimiters should I use? And whatever we have is that next character is what it will use as the delimiter. So, we can use pipes like that. It still works. I will just paste in an example to show you why you might want to do this.

The forward slashes are really the most traditional usage of it. But let's say, for example, that I had a phrase like MacOS/Unix: awesome and I wanted to change that. Then the search string that I'm looking for has a forward slash and a colon in it in it, so I don't wan to use either of those first two choices. Therefore I'm using the upright pipes. Now, I could still use the forward slashes. I will just have to escape each of those values. Every time it occurred, I would have to put a backslash before the forward slash to escape it. By switching to a different delimiter, I can just make it a little cleaner and easier to read.

So far, the stream that I've been using has been texted that I'm piping into sed, but sed also works with files. So, let's say we have sed s and we will use pear to mango inside our fruit file. Now, I am already inside my Unix files here. Inside my user directory. We have a file of just fruit there. Now, notice that I did not put the pipe here. I certainly can. That still works too. This is different from tr. In tr, you had to put this there. with sed, you don't have to. It will take a second argument, which is the file that you want it to do, and then we can take that and we can pipe that and we can have something like mango_fruit.txt.

Now, we've redirected the output into this other file. Now, notice here that we did not use the g modifier. We did that, we didn't use the g modifier, but at the same time it replaced pear here and it replaced pear down here, two times. Why is that? It was because each line gets treated as a stream. Let me show you an example where you can really see this in action. s/a/x and we will do that with the fruit file. Now, notice here when we had, for example, banana, it only changed it the first time.

It didn't change it the second time but it did do it on the next line. It found the first occurrence and then the first occurrence and so on. But then it got on papaya. It only used the first one. We didn't do it globally. If we switch that and you do the g in front of our global, now notice the papaya all got changed. So, it's line by line. So when we say global, we are saying per stream and each one of these lines is being treated as a new stream that's coming in. Now, you can search for just about anything inside a file and replace it. It's useful for things like let's say we had a file that was written in Britain.

So it has the word colour in it, spelled with o-u-r. We want to change that to color, the American spelling without the u. Then we put that global and whatever file we want to work with. Now, obviously, I don't have the word color in there. That's the kind of thing you could do globally to a file and sed would make it easy. You can also provide multiple sed commands. Let's say for example that I have a phrase like echo. During day time, we have sunlight and let's take that and we'll do sed.

We want to first do s for day and night. That will simply replace the day with nights and that says during nighttime, we have sunlight. If we also wanted to replace sun with moon, we can provide a second command and the way we do that is we put -e. It's important that it's lowercase and that says that we are going to have multiple commands. We are essentially going to build them up. So, -e in the first one, -e, and then next sed command, s/sun/moon. There we go and now it does both of them.

So, all you have to do is prefix it with that -e option and it says "Oh, what comes next is an argument to -e. Therefore, I am going to take that as one of my sed arguments and I am going to wait until I get another -e." And I will take that and just sort of build up the set of filters that we want to apply. So, it's nice. You can actually put together a five or six of these and change a whole bunch of things all at one time. And you can remember that it's the e because e is for edit. So we are providing the edits. So each one of these is an edit that we want for our stream editor to use.

Now, we won't go over it, but if you have a lot of sed commands that you want to run on a single input stream, what you can do is you can put all these edit commands into a file and you can use the f option to run all of them at once and that will run every sed command that's in the file. So you can have a set of regular things that you want to change, sort of filters you want to apply. Save them to a file and they're available to you. Anytime you want them, you can just call up that sed file and say hey, run all of these commands on this file for me. Now, these will give you some examples of just the simplest substitution that you can do just by using literal characters.

Where sed really becomes powerful was when we start working with regular expressions and that's what we will look at in the next movie.

Show transcript

This video is part of

Image for Unix for Mac OS X Users
Unix for Mac OS X Users

82 video lessons · 27725 viewers

Kevin Skoglund
Author

 
Expand all | Collapse all
  1. 3m 57s
    1. Introduction
      1m 14s
    2. Using the exercise files
      2m 43s
  2. 32m 2s
    1. What is Unix?
      7m 27s
    2. The terminal application
      4m 23s
    3. Logging in and using the command prompt
      5m 19s
    4. Command structure
      5m 22s
    5. Kernel and shells
      5m 25s
    6. Unix manual pages
      4m 6s
  3. 15m 58s
    1. The working directory
      2m 49s
    2. Listing files and directories
      3m 59s
    3. Moving around the filesystem
      4m 58s
    4. Filesystem organization
      4m 12s
  4. 1h 4m
    1. Naming files
      5m 41s
    2. Creating files
      2m 19s
    3. Unix text editors
      6m 39s
    4. Reading files
      5m 35s
    5. Reading portions of files
      3m 27s
    6. Creating directories
      2m 40s
    7. Moving and renaming files and directories
      8m 32s
    8. Copying files and directories
      3m 7s
    9. Deleting files and directories
      3m 38s
    10. Finder aliases in Unix
      4m 10s
    11. Hard links
      5m 30s
    12. Symbolic links
      6m 36s
    13. Searching for files and directories
      6m 32s
  5. 34m 58s
    1. Who am I?
      4m 3s
    2. Unix groups
      1m 52s
    3. File and directory ownership
      6m 41s
    4. File and directory permissions
      4m 27s
    5. Setting permissions using alpha notation
      6m 49s
    6. Setting permissions using octal notation
      3m 49s
    7. The root user
      1m 57s
    8. sudo and sudoers
      5m 20s
  6. 52m 34s
    1. Command basics
      4m 4s
    2. The PATH variable
      4m 13s
    3. System information commands
      3m 40s
    4. Disk information commands
      6m 8s
    5. Viewing processes
      5m 0s
    6. Monitoring processes
      3m 36s
    7. Stopping processes
      3m 19s
    8. Text file helpers
      6m 50s
    9. Utility programs
      7m 28s
    10. Using the command history
      8m 16s
  7. 20m 39s
    1. Standard input and standard output
      1m 24s
    2. Directing output to a file
      4m 13s
    3. Appending to a file
      2m 44s
    4. Directing input from a file
      5m 28s
    5. Piping output to input
      4m 40s
    6. Suppressing output
      2m 10s
  8. 41m 28s
    1. Profile, login, and resource files
      9m 11s
    2. Setting command aliases
      6m 59s
    3. Setting and exporting environment variables
      4m 54s
    4. Setting the PATH variable
      6m 10s
    5. Configuring history with variables
      6m 17s
    6. Customizing the command prompt
      6m 5s
    7. Logout file
      1m 52s
  9. 1h 25m
    1. grep: Searching for matching expressions
      5m 21s
    2. grep: Multiple files, other input
      4m 28s
    3. grep: Coloring matched text
      2m 57s
    4. Introduction to regular expressions
      3m 22s
    5. Regular expressions: Basic syntax
      3m 19s
    6. Using regular expressions with grep
      5m 20s
    7. tr: Translating characters
      8m 17s
    8. tr: Deleting and squeezing characters
      5m 30s
    9. sed: Stream editor
      7m 45s
    10. sed: Regular expressions and back-references
      7m 8s
    11. cut: Cutting select text portions
      7m 42s
    12. diff: Comparing files
      4m 35s
    13. diff: Alternative formats
      4m 30s
    14. xargs: Passing argument lists to commands
      7m 25s
    15. xargs: Usage examples
      7m 59s
  10. 42m 25s
    1. Finder integration
      4m 45s
    2. Clipboard integration
      5m 5s
    3. Screen capture
      3m 42s
    4. Shut down, reboot, and sleep
      3m 34s
    5. Text to speech
      2m 36s
    6. Spotlight integration: Searching metadata
      3m 41s
    7. Spotlight integration: Metadata attributes
      4m 24s
    8. Using AppleScript
      5m 23s
    9. System configurations: Viewing and setting
      5m 51s
    10. System configurations: Examples
      3m 24s
  11. 1m 26s
    1. Conclusion
      1m 26s

Start learning today

Get unlimited access to all courses for just $25/month.

Become a member
Sometimes @lynda teaches me how to use a program and sometimes Lynda.com changes my life forever. @JosefShutter
@lynda lynda.com is an absolute life saver when it comes to learning todays software. Definitely recommend it! #higherlearning @Michael_Caraway
@lynda The best thing online! Your database of courses is great! To the mark and very helpful. Thanks! @ru22more
Got to create something yesterday I never thought I could do. #thanks @lynda @Ngventurella
I really do love @lynda as a learning platform. Never stop learning and developing, it’s probably our greatest gift as a species! @soundslikedavid
@lynda just subscribed to lynda.com all I can say its brilliant join now trust me @ButchSamurai
@lynda is an awesome resource. The membership is priceless if you take advantage of it. @diabetic_techie
One of the best decision I made this year. Buy a 1yr subscription to @lynda @cybercaptive
guys lynda.com (@lynda) is the best. So far I’ve learned Java, principles of OO programming, and now learning about MS project @lucasmitchell
Signed back up to @lynda dot com. I’ve missed it!! Proper geeking out right now! #timetolearn #geek @JayGodbold
Share a link to this course

What are exercise files?

Exercise files are the same files the author uses in the course. Save time by downloading the author's files instead of setting up your own files, and learn by following along with the instructor.

Can I take this course without the exercise files?

Yes! If you decide you would like the exercise files later, you can upgrade to a premium account any time.

Become a member Download sample files See plans and pricing

Please wait... please wait ...
Upgrade to get access to exercise files.

Exercise files video

How to use exercise files.

Learn by watching, listening, and doing, Exercise files are the same files the author uses in the course, so you can download them and follow along Premium memberships include access to all exercise files in the library.


Exercise files

Exercise files video

How to use exercise files.

For additional information on downloading and using exercise files, watch our instructional video or read the instructions in the FAQ .

This course includes free exercise files, so you can practice while you watch the course. To access all the exercise files in our library, become a Premium Member.

Join now Already a member? Log in

Are you sure you want to mark all the videos in this course as unwatched?

This will not affect your course history, your reports, or your certificates of completion for this course.


Mark all as unwatched Cancel

Congratulations

You have completed Unix for Mac OS X Users.

Return to your organization's learning portal to continue training, or close this page.


OK
Become a member to add this course to a playlist

Join today and get unlimited access to the entire library of video courses—and create as many playlists as you like.

Get started

Already a member ?

Become a member to like this course.

Join today and get unlimited access to the entire library of video courses.

Get started

Already a member?

Exercise files

Learn by watching, listening, and doing! Exercise files are the same files the author uses in the course, so you can download them and follow along. Exercise files are available with all Premium memberships. Learn more

Get started

Already a Premium member?

Exercise files video

How to use exercise files.

Ask a question

Thanks for contacting us.
You’ll hear from our Customer Service team within 24 hours.

Please enter the text shown below:

The classic layout automatically defaults to the latest Flash Player.

To choose a different player, hold the cursor over your name at the top right of any lynda.com page and choose Site preferences from the dropdown menu.

Continue to classic layout Stay on new layout
Exercise files

Access exercise files from a button right under the course name.

Mark videos as unwatched

Remove icons showing you already watched videos if you want to start over.

Control your viewing experience

Make the video wide, narrow, full-screen, or pop the player out of the page into its own window.

Interactive transcripts

Click on text in the transcript to jump to that spot in the video. As the video plays, the relevant spot in the transcript will be highlighted.

Thanks for signing up.

We’ll send you a confirmation email shortly.


Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

Keep up with news, tips, and latest courses with emails from lynda.com.

Sign up and receive emails about lynda.com and our online training library:

Here’s our privacy policy with more details about how we handle your information.

   
submit Lightbox submit clicked
Terms and conditions of use

We've updated our terms and conditions (now called terms of service).Go
Review and accept our updated terms of service.