Logstash Experiments

• Logstash can be configured with conf file
• Basic structure is

input {
    <input-plugin> {
        property1  => <value1>
        ...
        ...
        propertyn  => <valuen>
    }
}
filter {
    <filter-plugin> {
        property1  => <value1>
        ...
        ...
        propertyn  => <valuen>
    }

}
output {
    <output-plugin> {
        property1  => <value1>
        ...
        ...
        propertyn  => <valuen>
    }
}

• Input Plugins:

  • For the complete list refer here
  • For experiment1 i would be using a simple plugin for input stdin
  • Navigate to docs and look into options and ensure you atleast fill the required options
  • Since there are no required fields my test.conf will look like

  input {
      stdin {}
  }
  

• Output Plugins:

  • For the Complete list refer here
  • For the first experiment lets use a stdout
  • Have a look at options like you did in input plugins
  • adding output to test.conf will look like

  input {
      stdin {}
  }
  output {
      stdout {}
  }
  

• Experiment1: Take the input from stdin and output to stdout

  • Create a test.conf file and execute the following command

  sudo /usr/share/logstash/bin/logstash -f test.conf
  

  • If every thing is configured correctly, it takes some time and then you can enter input and output will be displayed by logstash on terminal.

• Experiment2: Take the input from some log file and output to stdout

  • Conf file:

  input {
      file {
          path => "/var/log/dummy.log"
      }
  }
  output {
      stdout {
  
      }
  }
  

  • Now execute the logstash with new conf file.

Configured Elastic Search

• Refer Here for the configuration of elastic search.
• Refer Here for important settings in /etc/elasticsearch/elasticsearch.yml
• Refer Here for previous docs.

Install Kibana for Visualizations

• Install kibana using docs from here
• Kibana can be used to search logs and create visualizations and dashboards