Join Kevin Skoglund for an in-depth discussion in this video Lazy expressions, part of Learning Regular Expressions (2011).
After our discussion in the previous movie about how regular expression engines choose greediness by default, you may be wondering, could we make the regular expression engine make a set of different choices? And we can do that but to do it, we need to introduce a metacharacter--now not a new metacharacter but one we've already seen before, the question mark. Previously we learned the question mark was the modifier that said the preceding item occurred either zero or one time. Now in this context, this question marks makes the preceding quantifier into a lazy quantifier.
So what that means is that we stick it right after all those qualifiers that we've seen before. So we have a star and a question mark, plus and a question mark, our curly braces with a question, or question mark question mark. The asterisk question mark is actually called the lazy star, and the plus question mark is nicknamed the lazy plus, so you can call them those for short. And all of these had the exact same effect that they did before, the same meaning. The only difference is that instead of adopting a greedy strategy, now they're going to adopt a lazy strategy for making their choices.
Remember, the greedy strategy matches as much as possible before giving control over to the next expression part. The lazy strategy does the opposite; it says matches little as possible before giving control to the next expression part. It still differs to the overall match just like the greedy one does, and it's not necessarily any faster or slower to choose a lazy strategy or a greedy strategy, but it will probably match different things. Now as far as is faster or slower, it's a little bit like saying, if you've lost your car keys and your sunglasses inside your house, is it better to start looking in the kitchen or to start looking in the living room? You don't know which one's going to yield the best result, and you don't know which one's going to find the sunglasses first or the keys first; it's just about different strategies of starting the search.
So you will likely get different results depending on where you start, but it's not necessarily faster to start in one place or the other. So let me just show you this lazy quantifier in action, just so you can sort of see how it works in context. These look just like expressions we had before; we've just stuck the question mark after each one. Now the last one, I want you to pay particular attention, to apples??. It's totally a valid regular expression to do that. In the first instance that question mark is saying that the S can occur zero or one time.
By default it's greedy, so it's going to prefer one time; it's always going to prefer, if we have the string apples, it's going to want to take a-p-p-l-e-s and be greedy about it, not just to stop at the e. The question mark after it though tells it to be lazy, which says if you see the word apples, then I want you to just take a-p-p-l-e. So essentially then we've made the s meaningless, because when is there ever a case when you'd be able to find that a-p-p-l-e and fail and still need to get that s afterwards? There is never an instance.
That's really meaningless to have it, and some tutorials leave it out altogether. So I just want to show you it so that you see it, but it actually can be completely omitted and ignored. Now as far as support goes, most regular expression engines support this lazy quantifier, but UNIX tools are always greedy. BREs and EREs, all of those tools do not support it. It was really something that came along with Perl, Perl-compatible regular expressions. So most programming languages and regular expression engines that are out there will allow you to adopt a lazy strategy instead of a greedy one.
So let's look at how the regular expression engine parses it, just like we did before. We have the same string, Page 266, and the same regular expression, but I have put the question mark after the star. That makes it a lazy star. So it's wildcard lazy star, any digit one or more times. So what happens is the regular expression engine starts out at the beginning, and actually, before it even goes to the first letter, it says, oh you know what this wildcard is zero or more times. It could be zero. Boy, that would be really lazy. I could get away with doing no work at all. Let me first just try doing nothing and see if the next part of the expression can take over and do all the work for me.
So the next part of the expression then goes to the P and says nope, that's not a digit. So then the wildcard says okay, okay, I'll try and match it. Oh, it does match for me. All right, so I'll take the P. All right, I am done now. I am going to take a break. Next part of the regular expression, you do your job. So it goes to the a and it says is that a digit? No, that's not. So therefore, so it says, okay I'll get up and do a little bit more work. I'll take the P and the A--those both match me, the wildcard. Now, it asks once again, hey digit, can you take over? The digits, or whatever the next part of the expression happens to be, takes over and tries again.
And it keeps doing that until it keeps going along the string, and finally then when it gets here, then the digits start matching. The digit says, oops! I found it a 2. That works for me. So now the wildcard is equal to page space, and the digits say ah! This works for me, this matches me, the six matches me, and the next six matches me; therefore I now have a match. So in this case it still matched exactly the same thing, but it did it by using a different strategy. One note of caution I want to give you is that if you were to take this same expression but you changed the plus sign instead to be a lazy star as well, what do you think it would match in this case? Both of them are being lazy.
Take a second and think about it. What happens is the first one says, oh I can get away with matching nothing, so I match zero. Then it goes to the next one and says, hey, why don't you take over? The 0-9 says oh, I can get away with matching nothing too, so then it says, go along to the next regular expression, and both of them have succeeded in fulfilling their requirements. So it says I made a match, but you know what your matched result was? Nothing. It matched absolutely nothing. So you have to be careful. If everything in the expression is optional then it's going to match nothing at all.
And just to round this out, let's take one last look at those same samples that we were looking at in the last movie where we had an Excel file or we had a comma-delimited file of names and companies. And let's look what would be matched if you used lazy expressions instead of using greedy expressions. So notice that I have now got a lazy plus after the word character in the first example and a lazy plus after each of those wildcards in the second example. So what does it match now? Remember before, the first one match the entire thing except for the .xls.
Well now, it matches just the first part, because that word character tries to give up as quickly as it can and see if it can get a match. Then in the second example, instead of having the entire string be matched, now it says, all right, I am going to try the first one, but I am going to give up as quickly as I can and see if the next one can do it's the thing. So again, two different strategies. One is not necessarily better than the other, but they do often yield different results, so it's something that you definitely need to be mindful of as you are constructing your regular expressions.
- Creating flexible patterns using character sets
- Achieving efficiency when using repetition
- Understanding different types of search strategies
- Writing logical and efficient alternations
- Capturing groups and reusing them with backreferences
- Developing complex patterns with lookaround assertions
- Working with Unicode and multibyte characters
- Matching email addresses, URLs, dates, HTML tags, and credit card numbers
- Using search and replace to format a document