Meaning of the colors in this guide:
If not working on CentOS7, these links will help:
https://www.elastic.co/guide/en/logstash/7.6/dir-layout.html
https://www.elastic.co/guide/en/logstash/7.6/running-logstash.html
Java version 8 or 11 is required for the installation:
yum install java-1.8.0-openjdk-devel -y
java -version
openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)
First:
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
and create the repo
vi /etc/yum.repos.d/logstash.repo
Then copy the following into it:
[logstash-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
The command to install is:
sudo yum install logstash -y
Start Logstack:
sudo systemctl enable logstash
sudo systemctl start logstash
journalctl –unit logstash
localhost.localdomain systemd[1]: Started logstash.
Note: From now on the IP 192.168.20.100 will be entered, this must always be adjusted to the respective own IP of the VM on which Elasticsearch is installed.
Execute the following command:
/usr/share/logstash/bin/logstash -e 'input { stdin { } } output {
elasticsearch { hosts => "192.168.20.100:9200" } }'
Now wait for the following output:
[Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
After that you can just start typing. The output can be seen in the browser at http://192.168.20.100:9200/_search?pretty.
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 9,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "logstash-2020.05.11-000001",
"_type" : "_doc",
"_id" : "-hDWAnIB5PIPaJRIY6Ll",
"_score" : 1.0,
"_source" : {
"message" : "Hallo",
"@timestamp" : "2020-05-11T08:25:03.941Z",
"host" : "localhost.localdomain",
"@version" : "1"
}
},
After creating an index logs in Elasticsearch, you can also move the output to the index by:
/usr/share/logstash/bin/logstash -e 'input { stdin { } } output { elasticsearch { hosts => "192.168.20.100:9200" index => "logs" } }'
Now wait for the following output:
[Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
After that you can just start typing. The output can then be seen in the browser at http://192.168.20.100:9200/logs/_search?pretty
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "logs",
"_type" : "_doc",
"_id" : "4usEA3IBWUPpcn4FpXBV",
"_score" : 1.0,
"_source" : {
"host" : "localhost.localdomain",
"@timestamp" : "2020-05-11T09:15:35.106Z",
"@version" : "1",
"message" : "Hi"
}
},
After creating an index logs in Elasticsearch, you can also move the output to the index by:
/usr/share/logstash/bin/logstash -e 'input { file { path => "/etc/resolv.conf"
start_position => beginning } } output { elasticsearch { hosts =>
"192.168.20.100:9200" index => "logs" } }'
Now wait for the following output:
[Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
Now you can change something in the folder /etc/resolv.conf and see the change
in the browser at http://192.168.20.100:9200/logs/_search?pretty.
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 16,
"relation" : "eq"
},
{
"_index" : "logs",
"_type" : "_doc",
"_id" : "VcKTA3IByzccBleDawQ9",
"_score" : 1.0,
"_source" : {
"@version" : "1",
"@timestamp" : "2020-05-11T11:51:32.291Z",
"path" : "/etc/resolv.conf",
"message" : "# Generated by NetworkManager Gentian Rrafshi",
"host" : "localhost.localdomain"
}
After creating an index logs in Elasticsearch, create a configfile elastic.conf in the subfolder /etc/logstash/conf.d/elastic.conf,
which looks like this:
eht:
input {
file {
path => "/etc/resolv.conf"
start_position => beginning
}
}
output {
elasticsearch {
hosts => ["192.168.20.100:9200"]
index => "logs"
}
}
Test if the config fits:
sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash -t
Now execute the following command:
/usr/share/logstash/bin/logstash -f elastic.conf --path.settings=/etc/logstash/
Now wait for the following output:
[logstasg.agent] Successfully started Logstash API endpoint {:port=>9600}
Now you can change something in the folder /etc/resolv.conf and save it.
The change can then be seen in the browser at http://192.168.20.100:9200/logs/_search?pretty
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "logs",
"_type" : "_doc",
"_id" : "bsITBHIByzccBleDHATP",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2020-05-11T14:11:00.834Z",
"path" : "/etc/resolv.conf",
"host" : "localhost.localdomain",
"message" : "# Generated by Networkmanager Gentian Rrafshi",
"@version" : "1"
}
After each run systemctl restart logstash before!
You can display a list of all available indexes with this command: http://192.168.20.100:9200/_cat/indices
Please feel free to contact us for any question that is not answered yet.
We are looking forward to get in contact with you!
KubeOps GmbH
Hinter Stöck 17
72406 Bisingen
Germany
+49 7433 93724 90