Because lemmatization is really important for the improvement of your search results, explore this technique a little further. Learn about how lemmatization is the process of determining the lemma of a word based on its intended meaning. Unlike stemming, lemmatization depends on correctly identifying the intended part of speech and meaning of a word in a sentence, as well as within the larger context surrounding that sentence, such as neighboring sentences or even an entire document.
- [Instructor] Lemmatization plays a very important role in our search logic, so I'd like to give you a few more details on this process, and I'd like to start with a little definition because lemmatization is the process of grouping together the inflected forms of a word so that they can be analyzed as a single item. And in this case, this is easy. We have the walk, and inflected forms could be walked, walks, or walking, and what they all have in common is their stem, walk, but lemmatization is closely related to stemming, but the difference is that a stemmer operates on a single word without the knowledge of the context.
So let's have a look at some more examples to see that lemmatization is really more than stemming. So, what about the word better has good as its lemma, and this link that we see here is missed by stemming because it requires a dictionary lookup. Another case would be in our last meeting, and we are meeting again tomorrow. Here we have two words that could be a noun or a verb, so unlike stemming, lemmatization attempts to select the correct lemma depending on the context.
So, in our context of natural language processing, lemmatization is the process of determining the lemma of a word based on its intended meaning. And unlike stemming, lemmatization depends on correctly identifying the intended part of speech and meaning of a word, and this could be in a sentence as well as within the larger context surrounding that sentence, such as neighboring sentences or even in an entire document.
- What are machine learning, Core ML, Vision, and NLP?
- Adding a machine learning model to a project
- Getting predictions from machine learning models
- Converting existing machine learning models for Core ML
- Classifying images and detecting objects with Vision and Core ML
- Analyzing natural language text with NSLinguisticTagger