DevOps Classroomnotes 14/Dec/2022

Understanding Log Messages

Apache Server

• Lets create a free tier linux vm (ubuntu)
• Install apache server

sudo apt updata
sudo apt install apache2 -y

• Now navigate to /var/log/apache2
• Access apache and view the logs generated in /var/log/apache2/access.log

183.82.7.20 - - [14/Dec/2022:02:47:37 +0000] "GET / HTTP/1.1" 200 3477 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36 Edg/108.0.1462.46"

• Every log has some format
• Lets use apache access log format. Refer Here

LogFormat "%h %l %u %t \"%r\" %>s %b" common

• mysql sample error log:

060516 22:38:54 [ERROR] Fatal error: Can't open privilege tables: Table 'mysql.host' doesn't exist

• Exercise:
  • postgresql: Find a sample log and its format.
  • tomcat: find a sample log and its format.
• There is a need for a tool which can break the log message into different fields, so that we can make log data queryable. A System admin has come up with an open source tool written in jruby called as logstash was released. Then elastic team picked this tool and rewritten it in java added to ELK stack which now has become elastic stack
• Logstash has a configuration file where we perform
  • input:
    • This section describes where to collect the logs
    • logstash supports lot of input plugins Refer Here
  • filter:
    • This section describes the parsing of the logs
    • Refer Here for plugins
  • output
    • This section describes where to stored the parsed logs
    • Refer Here for output plugins