DevOps Classroom Series – 01/Dec/2020

Analyzing the log data with logstash

  • Lets install apache server and read the logs of apache server from logstash pipeline

  • The first basic configuration looks as mentioned over Here Preview Preview

Parse and enrich logs using logstash

  • We need to use filter plugins to transform the data.
  • Lets use a csv file and parse the data using csv filter
  • For this lets use the dataset from grouplens Refer Here
  • Lets look at filter plugins for csv Refer Here for all plugins & Refer Here for csv plugin. Refer Here for the changeset Preview Preview
  • Movie id is read as string, lets change that to integer, for that we will be using mutate filter Refer Here Preview
  • Refer Here for the change set Preview

Using GROK Filters

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Please turn AdBlock off
Floating Social Media Icons by Acurax Wordpress Designers
Visit Us On FacebookVisit Us On LinkedinVisit Us On Youtube