From the course: Learning the Elastic Stack (2020)

Working with Logstash - Elastic Stack Tutorial

From the course: Learning the Elastic Stack (2020)

Start my 1-month free trial

Working with Logstash

“

- [Instructor] Logstash is much more than a simple log ingestion tool. It's a fully customizable data processing engine. As we saw in the example when we were setting up Logstash, there are three blocks of code in a config file. Input is how you get data in, like from files or when data is sent over a network port. Output sends the data somewhere. Generally, this is an Elasticsearch cluster, but it can also be a file on disk or even another network service. It's worth mentioning here that Logstash can be really powerful without the rest of the stack. For example, it could be used to ingest data that's in different formats from multiple sources and output it to one unified standard. The filter block is where data is transformed and manipulated. It's where things like geolocation happens, but it's also where unstructured log lines can be tagged and parsed into more usable structured data. That's why we can see a JSON representation of a web server log. The Logstash filter was set up to make that conversion. It's also what lets us do complex queries that just aren't possible on flat text files. Let's look at an example of a config file. This is a very handy one when you're learning Logstash. Input comes from standard in, whatever you type into the terminal, and then you have a filter block, and output goes to standard out, so it's printed right back in your terminal window. Running Logstash with this config file gives you an interactive shell to see how your filter block will respond to various inputs. So if we had this in the filter block, anything we typed would be returned in a structured format with a new greeting field. That's it for basic config. We'll go deeper into some of the config options in the next lesson.

Contents