Mastering Elastic Stack

Even structured data is useless if it can’t help you to take strategic decisions and improve existing system. If you love to play with data, or your job requires you to process custom log formats, design a scalable analysis system, and manage logs to do real-time data analysis, this book is your one-stop solution.

By combining the massively popular Elasticsearch, Logstash, Beats, and Kibana, elastic.co has advanced the end-to-end stack that delivers actionable insights in real time from almost any type of structured or unstructured data source. If your job requires you to process custom log formats, design a scalable analysis system, explore a variety of data, and manage logs, this book is your one-stop solution. You will learn how to create real-time dashboards and how to manage the life cycle of logs in detail through real-life scenarios.

This book brushes up your basic knowledge on implementing the Elastic Stack and then dives deeper into complex and advanced implementations of the Elastic Stack. We’ll help you to solve data analytics challenges using the Elastic Stack and provide practical steps on centralized logging and real-time analytics with the Elastic Stack in production. You will get to grip with advanced techniques for log analysis and visualization. Newly announced features such as Beats and X-Pack are also covered in detail with examples. Toward the end, you will see how to use the Elastic stack for real-world case studies and we’ll show you some best practices and troubleshooting techniques for the Elastic Stack.

Chapters Overview

• Chapter 1, Elastic Stack Overview, covers the shift from ELK Stack to Elastic Stack followed by setup of various components of Elastic Stack.
• Chapter 2, Stepping into Elasticsearch, takes us to how Elasticsearch started as a project, how Elasticsearch works and covering various Elasticsearch API’s and Aggregations.
• Chapter 3, Exploring Logstash and Its Plugins, covers introduction of Logstash along with understanding it’s architecture. It also covers the various plugins with suitable examples. At the end, a Logstash configuration file is shown for parsing logs.
• Chapter 4, Kibana Interface, teaches about the various interfaces present in Kibana in depth along with an example to demonstrate how to combine all the interfaces to create a dashboard.
• Chapter 5, Using Beats, takes us to introducing the beats, understanding how beat differs from Logstash followed by exploring various beats, their functionalities and setup steps. At the end, we explored how to use Beats in Elastic Stack.
• Chapter 6, Elastic Stack in Action, covers a real-world use-case of an Intranet Portal server and showcases and how to use Elastic Stack components to solve the problem.
• Chapter 7, Customizing Elastic Stack, teaches us how to extend each component of Elastic Stack and how to create a plugin for our use-cases.
• Chapter 8, Elasticsearch APIs, takes us to various Elasticsearch API’s along with understanding Elasticsearch modules, Ingest nodes, Discovery plugins and how to use Java client to access various Elasticsearch operations.
• Chapter 9, X-Pack: Security and Monitoring, covers introduction of X-Pack along with installation of X-Pack. It also covers the usage and functionalities provided by Shield, Marvel and Profiler.
• Chapter 10, X-Pack: Alerting, Graph, and Reporting, teaches us about the usage and functionalities of Watcher, Graph and Reporting features.
• Chapter 11, Best Practices, takes us to understand why do we need to follow best practices along with listing of various best practices which should be followed which has been categorized into multiple sub-sections.
• Chapter 12, Case Study-Meetup, covers complete coverage of understanding the problem statement followed by extending Logstash and creating a plugin to fetch required information. It then takes us to understand how to utilize Elastic Stack components to cover end-to-end understanding of Meetup data and showcasing the powerful capabilities of Elastic Stack for data analytics.

Excerpt from the preface of my book “Mastering the Elastic Stack”.