DevOps Classroom Notes 30 Oct 2019 – Elastic Stack Usecase

Scenario

  • To understand Elastic Stack, We assume the organization as AdventureWorks in this series.
  • Lets see different application/db servers used by Adventure Works Preview

Application-1 Architecture

Preview

  • To make the above application observable, we need to monitor
    • Logs
    • Traces
    • Metrics

Enabling Log Monitoring

  • It is very difficult to login into each server for the logs
  • We need a central location for looking into logs of each applciation/server
  • To acheive this, we have the following options
    • Store all the logs into Database like mysql/oracle
    • Use Syslog / Event log services

Lets Examine SQL Approach

Preview

  • In this Approach we send the logs of application to the Central SQL Server
  • To send the logs of mysql and mongodb we might use ETL (Extract, Transform and Load) to see the logs in SQL Server
  • It looks like the problem is solved.
  • Problems with this approach:
    • Logs in the SQL server needs to be queried. But logs are text. Text searching in SQL is not that great.
    • Logs of all the different servers generally will be in different formats
Possible Solution
  • We need a system which can process different log formats and do the necessary adjustments
  • We also need a system which can search text quickly
  • Elastic Stack comes to the rescue

Elastic Stack

  • It is suite of Products
  • In our Application’s Scenario we use Elastic Stack as our monitoring/logging platform.
  • Elastic Stack = ELBK

Elastic Search

  • Effecient Text Searching Distributed System.

LogStash

  • Log Parsing and Conversions

Kibana

  • Building Dashboards

Beats

  • Agents which run on nodes which export logs to LogStash or ElasticSearch and/or report metrics.

Preview

One thought on “DevOps Classroom Notes 30 Oct 2019 – Elastic Stack Usecase

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About continuous learner

devops & cloud enthusiastic learner