- In this lecture, we're going to cover natural language processing. I'm back again with Eric Nyberg. Eric, you're a professor in the Language Technologies Institute in the Department of Computer Science at Carnegie Mellon. Tell us what language technologies are. - Well, David, language technologies is a term that's used to refer to software that processes any human language, any one of its many forms. - So natural language processing enables computers to work with text and language the way that we do, like extracting meaning from text or creating text that's readable and stylistically natural and grammatically correct.
NLP does not enable computers to understand text the way that we do, but it does enable them to manipulate text in sophisticated ways, such as automatically identifying all of the people and places mentioned in a document, or identifying the main topic of a document. NLP is not to be confused with speech recognition, the subject of our next lecture. An example of natural language processing you may be familiar with is machine translation. If you ask Google to translate the sentence, I love to learn, into Italian, it will automatically produce, correctly, mi piace imparare.
Unfortunately, it doesn't translate I learn to love correctly, at least not yet. Natural language processing is hard because language is often ambiguous. Consider the sentence, he saw her duck. Did he see her waterfowl? Or did he see her bow down quickly? Out of context, you can't know for sure. And humans use context to understand text, information that computers may not have. Eric, can you talk us through at a high level how natural language processing really works? - Well, David, natural language processing of text begins with a document, like a web page, and it has to begin by segmenting that document into sections, paragraphs, and finally into sentences.
And then each individual sentence can be processed by breaking it up into words. And then finally, the meanings of the words are combined using the rules of grammar for that language in order to understand the deeper meaning of each sentence. - So it breaks the job of processing text into subtasks connected into a pipeline? - Correct. - Excellent. So there are many applications of natural language processing, including summarizing documents, extracting information, like pulling financial information out of news reports, translation, question answering in scientific, medical, or customer service applications, or writing stories, such as corporate earnings or sports stories, and even analyzing customer feedback.
Looking forward, as the field progresses, what applications are we likely to see more of in the marketplace? - So David, I think by now there's a very large demand for natural language processing systems that can help humans cope with a very large volume of textual information in certain domains. For example, in the legal domain, you have many legal precedents that you have to research in order to build a case. Or in the medical domain there are many, many articles published every year that you might want to research if you're a doctor doing a complex diagnosis, so I think we're going to see more examples of NLP being applied to these domains where there's a very large amount of text that humans need to process.
- Excellent, so there's a lot more to come. And just to summarize, natural language processing enables computers to work with language the way that we do. It works by breaking down the task of processing language into subtasks connected in a pipeline. And machine learning is widely used in modern NLP systems.
- Artificial intelligence explained
- Cognitive technologies explained
- Supervised, unsupervised, and reinforcement learning
- Machine learning models and algorithms
- Language, speech, and visual processing
- Business applications of cognitive tech
- The impact of cognitive technologies at work
- Future of cognitive technologies