It's good to know where a platform came from in order to understand how best to leverage it for your needs. This video goes into the history and origins of Elasticsearch.
- [Instructor] I like to start my courses by showing where the platform we're digging into came from, because I believe it helps us understand a little bit more about why things are the way they are. To think about Elasticsearch, we have to go back to 1999 when the platform Lucene came out. Doug Cutting originally wrote this back then and it was available on SourceForge at the time. Lucene was added to the Apache Software Foundation in 2001 and became its own top-level project in February of 2005.
Now, Lucene included many top-level projects, such as Mahout and Nutch, which you now may know as HDFS and Apache Mahout. The nature of Lucene was that it helped all the search engines back then index the data that they were adjusting from the internet and provide reasonable ways of retrieving that information based on fuzzy matching. That means that if you searched for something, like funny cat videos, it would return documents or websites that were indexed, which contained information related to your search.
A real innovation here was how the search engine was able to extract text from almost any type of content and let you later retrieve it without knowing those exact terms. A few years later, Shay Banon created Compass, which was built on top of Lucene and provided, essentially, the same services, but in a more scalable manner. The idea here was to provide a distributed search solution that used common web transfer protocols and document formats. This is where Elasticsearch was born.
Elasticsearch is a distributed, RESTful search and analytics engine that helps with all kinds of use cases in today's technology landscape. For its data format, Elasticsearch uses JSON and, for its interface, HTTP. Both incredibly common on the web. Elasticsearch is developed in Java and open source with the Apache license. There are clients available in Java, .NET, Python, and many other languages. Across the landscape, Elasticsearch is, by far, the most popular enterprise search engine.
In short, if you were looking to do search today and don't want to reinvent the wheel, this is probably your best bet. Also, because of its incredible ability to scan documents and find information, it is increasingly becoming useful for data scientists and analysts as well.
In this course, join Ben Sullins as he dives into the inner workings of Elasticsearch combined with Kibana. Ben provides an overview of the architecture, and then goes over the different deployment methods, and how to best structure your data. From there, he demonstrates how to query data, and how to work with Kibana to present your insights.
- Reviewing key Elasticsearch concepts
- Running Elasticsearch in the cloud and locally
- Bulk loading data
- Setting up mappings of data types
- Querying data
- Running basic aggregations
- Creating visualizations and dashboards in Kibana