DevOps Classroomnotes 16/Feb/2022

More information about Logs

• Log Data: This is the intrinsic meaning that the log message has. These messages are classified into the following general categories
  • Information
  • Debug
  • Warning
  • Error
  • Alert
• Collecting Logs
  • Syslog: UDP based client-server protocol
  • Windows Event Log: Microsoft’s propietary logging format
  • Databses: Structured way to store and retrieve logs
  • Shared Folders / Network Storage
• Log Message: Basic contents of the Log message
  • Timestamp
  • Source
  • Data
• Challenges with Logs:
  • No Standard Format
    “`

  </ul>
  <h1>apache</h1>
  83.149.9.216 – – [17/May/2015:10:05:03 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/&quot; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
  <h1>windows</h1>
  <ul>
  <li><Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"></li&gt;
  <li><System>
  <Provider Name="Microsoft-Windows-Winlogon" Guid="{DBE9B383-7CF3-4331-91CC-A3CB16A3B538}" EventSourceName="Wlclntfy" />
  <EventID Qualifiers="32768">6003</EventID>
  <Version>0</Version>
  <Level>4</Level>
  <Task>0</Task>
  <Opcode>0</Opcode>
  <Keywords>0x80000000000000</Keywords>
  <TimeCreated SystemTime="2022-02-16T00:53:47.9542325Z" />
  <EventRecordID>2021</EventRecordID>
  <Correlation />
  <Execution ProcessID="0" ThreadID="0" />
  <Channel>Application</Channel>
  <Computer>DESKTOP-MUBT2L2</Computer>
  <Security />
  </System></li>
  <li><EventData>
  <Data>SessionEnv</Data>
  <Binary>D9060000</Binary>
  </EventData>
  </Event></li>
  </ul>
  <h1>SQL</h1>
  070823 21:00:32 1 Connect root@localhost on test1
  070823 21:00:48 1 Query show tables
  070823 21:00:56 1 Query select * from category
  070917 16:29:01 21 Query select * from location
  070917 16:29:12 21 Query select * from location where id = 1 LIMIT 1
  “`
  * Generally logs have text information which makes querying difficult
  * Collecting the logs from different applications with each application having a different format and different storage will be difficult.

• Options for Log Analysis and Visualization
• Splunk: Versions
  • Splunk Enterprise
  • Splunk light
  • Splunk Cloud
• Elastic Stack:
  • Open Source
  • Most of the Components are free for usage
  • Some of the Components have licensing
  • Almost all the clouds are providing Elastic Stack as a Service.
  • This has Server Monitoring and Application Performance Monitoring features (APM)
• Options for Server and Application Monitoring
• Nagios
• Prometheus
• Elastic Stack
• Options for APM
• App Dynamics
• New-Relic
• Elastic Stack

Elastic Stack Components

• Overview
  
• Basic usage of the components