DevOps Classroom Series – 24/Jul/2020

Codec plugins

  • Codec plugins are used to encode or decode incoming or outgoing events from logstash Refer Here

Exploring Input Plugins

  • File:

    • file plugin is used to stream events from file(s) line by line.
    • Refer file input plugin from here
    • Lets create a configuration file-hello.conf
    input 
    {
        file {
            path => "/home/ubuntu/logfiles/*"
        }
    }
    output
    {
        stdout {
            codec => rubydebug
        }
    }
    

    Preview

    • The sample configuration to read the logs from /var/log folder will be
    input {
        file {
            path => "/var/log/*.log"
        }
    }
    output {
        stdout {
            codec => rubydebug
        }
    }
    
  • Beats: Beats plugin allows Logstash to recievd events from elastic beats framework, Process them & send them to destinations

    • Lets write a configuration file and start logstash executable
    input {
        beats {
            port => 1234
        }
    }
    output {
        stdout {
            codec => rubydebug
        }
    }
    

    Preview

    • In reality our output from beats is always elastic search
  • JDBC: This plugin is used to import the data from the database to Logstash. JDCB configuration generally have the following important conf . Refer Here to understand connectivity terms used

    • jdbc_driver_libary: Path to Jar file which is jdbc driver
    • jdbc_driver_class: This is name of the driver
    • jdbc_connection_string: jdbc connection string
    • jdbc_user: username of db
    • jdbc_password: password of db
    • statement: this is sql statement or query to pull logs
    • parameters: query paramteres
    • schedule => when to run this query (* * * * *) (Cronjob)
    • Sample configuration
    input {
        jdbc {
            jdbc_driver_library => "/usr/share/mysql-connector-java"
            jdbc_driver_class => "com.mysql.jdbc.Driver"
            jdbc_connection_string => "jdbc:mysql://localhost:3306/sonoo"
            jdbc_user => "root"
            jdbc_password => "root@123"
            schedule => "30 * * * *"
            statement => "Select * from applogs"
    
        }
    }
    output{
        stdout {
            codec => rubydebug
        }
    }
    
  • IMAP: This plugin is used to read emails from IMAP server.

  • Play with Heroku Grok debugger Refer Here

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About learningthoughtsadmin