DevOps Classroom Series – 25/Nov/2020

Elastic Search

• This is realtime distributed search & analytics engine that is horizontally scalable & capable of solving wide variety of use-cases
• Elastic Search is built on top of Apache Lucene Refer Here
• Key features/benifits of elastic stack

  • Schemaless, doument-oriented:

    • We can store any JSON documents. To understand Json and YAML Refer Here

  • Searching:

    • Full text searching can be done with elastic search

  • Analytics:

  • Rich client libarary & REST API

  • Easy to operate and easy to scale

  • Near realtime

  • Fault-tolerant

Architectural View of Components of Elastic Stack

• Logstash:
  • Assume two application store logs in different formats
  • Logstash can perform number of transformations on the logs before sending it to elastic search
  • Logstash can transform our logs into meaningful formats b4 sending to Elastic Search
• Beats:
  • This is a lightweight data shipper which is installed on the source and sends the logs to either logstash/elastic search
  • Beats is complimentary to logstash
• Kibana:
  • This is a visualization tool where we configure dashboards, graphs etc to view the application logs
• X-Pack:
  • This adds essential features to make Elastic Stack Production ready.
  • They add
    • Security: Authentication & authorization capabilities into Elastic Search & Kibana
    • Monitor: You can montior elastic stack components, so there will be no down time.
    • Reporting: Printable Reports can be generated
    • Alerting: Alerting (email, Slack, PagerDuty) to user in the cases of failure can be configured

Use case of Elastic Stack

1. Log analytics
2. Product Search
3. Web Searches & Website Search
4. Metrics

Installing Elastic Stack

• Lab Setup
• To install elastic search the follow the documentation Refer Here
• If you would like to install elastic search on redhat distributions Refer Here
• Instructions

sudo apt update
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list
sudo apt-get update && sudo apt-get install elasticsearch
sudo systemctl status elasticsearch.service
sudo systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service
sudo systemctl status elasticsearch.service

• Run the following command curl -X GET "localhost:9200/?pretty" in the terminal
• But we are unable to access elastic stack from other system, we need to resolve this, so we need to understand elastic search configuration