DevOps Classroom Series – 01/Dec/2020

Analyzing the log data with logstash

  • Lets install apache server and read the logs of apache server from logstash pipeline

  • The first basic configuration looks as mentioned over Here Preview Preview

Parse and enrich logs using logstash

  • We need to use filter plugins to transform the data.
  • Lets use a csv file and parse the data using csv filter
  • For this lets use the dataset from grouplens Refer Here
  • Lets look at filter plugins for csv Refer Here for all plugins & Refer Here for csv plugin. Refer Here for the changeset Preview Preview
  • Movie id is read as string, lets change that to integer, for that we will be using mutate filter Refer Here Preview
  • Refer Here for the change set Preview

Using GROK Filters

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About learningthoughtsadmin