## logstash工作原理  ## example 将apache的日志发送到elasticsearch ~~~ vim apache.conf ~~~ ~~~ input { file { path => "/etc/httpd/logs/*log" start_position => beginning ignore_older => 0 } } filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} } geoip { source => "clientip" } } output { elasticsearch { hosts => [ "localhost:9200" ] } } ~~~ #### 测试文件语法 ~~~ /opt/logstash/bin/logstash -f apache.conf --configtest ~~~ #### 执行文件 ~~~ /opt/logstash/bin/logstash -f apache.conf ~~~ #### 查看结果 ~~~ curl -XGET 'localhost:9200/logstash-2017.05.22/_search?pretty&q=response=200' ~~~ ## input插件 ### stdin插件 ~~~ input { stdin { } } output { stdout { codec => rubydebug } } ~~~ ### udp插件 collectd收集服务器信息，使用udp协议发送给logstash ~~~ input { udp { port => 1991 codec => collectd {} type => "collectd" } } output { stdout { codec => rubydebug } } ~~~ ### file插件 将指定的文件作为logstash的输入 ~~~ input { file { path => "/path/to/file/*.log" start_position => beginning ignore_older => 0 } } ~~~ * 默认的file插件像tail -f一样会从日志的最后读取新的日志， `start_position`设置为`beginning`让其从文件头读取。 * 忽略文件的时间 ``` Setting ignore_older to 0 disables file age checking so that the tutorial file is processed even though it’s older than a day ``` ### redis插件 ~~~ input { redis { port => "6379" host => "192.168.101.201" data_type => "list" type => "apachelog" key => "logstash-apachelog" } } ~~~ ## filter插件 对input的内容进行过滤，切割词语 ### grok插件 ~~~ filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} } } ~~~ ### geoip插件 从input中获得IP ~~~ filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} overwrite => [ "message" ] } geoip { source => "clientip" } } ~~~ ## output插件 ### elasticsearch插件 将logstash加工过的内容发送到elasticsearch ~~~ output { elasticsearch { hosts => [ "localhost:9200" ] } } ~~~ ### redis插件 ~~~ output { redis { port => "6379" host => ["192.168.101.201"] data_type => "list" key => "logstash-%{type}" } } ~~~