ELK企业应用-elk快速搭建-logstash

ELK企业应用-elk快速搭建-logstash

1、安装JDK
elasticsearch，logstash的运行需要依赖java环境。
下载并解压jdk二进制包。

tar xf jdk-8u144-linux-x64.tar.gz -C /usr/local
mv /usr/local/jdk1.8.0_144 /usr/local/java
cd ~

配置java环境变量。
在~/.bashrc文件末尾添加如下内容：

export JAVA_HOME=/usr/local/java
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/bin/tools.jar:$JRE_HOME/lib
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

使配置生效。

source ~/.bashrc

2、安装Logstash

建议Linux类的服务器下载rmp包安装。
2.1.下载logstash安装包

touch /etc/default/logstash
ln -s /usr/local/java/bin/java /usr/bin/java
rpm -ivh logstash-6.2.4.rpm
cd ~

2.2.配置systemd启动

rpm安装时，创建启动脚本的配置文件是/etc/logstash/startup.options

/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

注：当脚本启动失败后，自创启动脚本即可

[root@l ~]# cat /etc/systemd/system/logstash.service 
[Unit]
Description=logstash
?
[Service]
Type=simple
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
ExecStop=/bin/kill -s QUIT $MAINPID
ExecReload=/bin/kill -s HUP $MAINPID
WorkingDirectory=/usr/share/logstash/bin
?
[Install]
WantedBy=multi-user.target
?
[root@l ~]# systemctl daemon-reload #####更新
[root@l ~]# 
[root@l ~]# systemctl list-unit-files |grep logstash
logstash.service                              disabled
[root@l ~]# 
[root@l ~]# systemctl restart logstash.service  ####重启

2.3.遇到的错误

[root@l opt]# /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd
Using provided startup.options file: /etc/logstash/startup.options
Manually creating startup for specified platform: systemd
/usr/share/logstash/vendor/jruby/bin/jruby:行401: /usr/bin/java: 没有那个文件或目录
Unable to install system startup script for Logstash.

解决方法

ln -s /usr/local/java/bin/java /usr/bin/java
/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

3、配置

cd /etc/logstash/conf.d/
chown -R logstash /etc/logstash/conf.d
mkdir /opt/logstash
touch /opt/logstash/messages
chown -R logstash /opt/logstash
chown -R logstash /opt/logstash/messages
chown -R logstash /var/log/messages

Shipper配置文件（logstash_shipper.conf）

vim logstash_shipper.conf
###########################################3
input{file{type => "messages"path => "/var/log/messages"start_position => "beginning"sincedb_path => "/dev/null"}
}output{if [type] == "messages"{redis{host => "10.0.0.132"data_type => "list"key => "messages"port => 6379db => 2password => "123456"}}
}

Indexer配置文件（logstash_indexer.conf）注：该配置文件得重新搭个node节点，否则两个output会重复输出日志，加上redis缓存就会无限输出。

vim logstash_indexer.conf
######################################
input{redis{host => "10.0.0.132"data_type => "list"key => "messages"password => "123456"db => 2}
}output{if [type] == "messages" {elasticsearch{hosts => ["10.0.0.130"]index => "messages-%{+YYYY-MM-dd}"}}
}

4、测试

cd /usr/share/logstash/bin/
./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
[root@l bin]# ./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
Configuration OK

5、启动

systemctl start logstash.service
systemctl enable logstash.service