DevOps Classroom Series – 13/Apr/2021

Scenario 1: qt-restaurant

• QT has an application which is for restaurant order management

• This application has the following architecture and is developed in Java language

• Each Service and database in this application is running on a different virtual machine

• We need approximately 10 servers to run this app.

• When this application is running

  • can there be server failures?
  • Can there be application failures?

• Can we stop all failures?

• Since we cannot stop all the failures, we can atleast react to failures quickly and resolve.

• For known failures, we need to pro-act and for unknown failures we need to react quickly

• In some cases we can even automate to the error resolution.

• To do all of this we need some kind of system which can monitor

  • server’s health
  • application’s health
  • analyze server logs
  • analyze application logs
  • analyze different metrics from servers and applications

• Building this kind of system can also help in understanding usage patterns of the users which can be analyzed and simplified.

• Generally we have two kinds of monitoring

  • Server Monitoring
  • Application Monitoring

• Whenever any application runs, it generates logs. To understand application behavior we need to analyze logs.

• Analyzing logs is a challenge

  • logs are plain text and are unstructured
  • logs donot have standard format.

• In your organization you might be running multiple applications and having multiple log analysis and monitoring solutions might be challenging, so we need a centralized log monitoring and analysis

• To solve these kind of scenarios there are many centralized log analytics applications, we are going to learn about

  • Elastic Stack

• Logs which are generated by applications or servers are directed into elastic search using logstash or beats.

• Logstash/Beats will forward the logs to elastic search which stores the logs and also has features to analyze the logs

• Kibana which is a UI for elastic search system will help us by providing user interface to

  • create dashboards
  • analyzing the logs