Hi Furkan,
As I can see your configuration has two output
blocks enabled:
output.elasticsearch:
...
output.logstash.hosts:
...
You must choose just one of them depending on your needs, but if it’s a fresh installation just remove output.logstash.hosts:
line and leave the Elasticsearch lines. Then, restart the Filebeat service.
In addition, I can see pipeline: geo
, unless you followed our geolocation docs it won’t work.
Let us know your results.
Kind regards,
Jesús
Hi Furkan,
Let’s dig into your Filebeat issue.
Is the template inside the Filebeat directory?
ls -lh /etc/filebeat/wazuh-template.json
# Expected output
-rw-r--r--. 1 root root 42K May 22 11:28 /etc/filebeat/wazuh-template.json
Can Filebeat reach Elasticsearch?
filebeat test output
# Expected output
elasticsearch: http://x.x.x.x:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: x.x.x.x
dial up... OK
TLS... WARN secure connection disabled
talk to server... OK
version: 7.1.0
Is Filebeat reading your alerts.json
properly?
lsof /var/ossec/logs/alerts/alerts.json
# Expected output
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
ossec-ana 2631 ossec 9w REG 8,1 443788 67478332 /var/ossec/logs/alerts/alerts.json
filebeat 10346 root 5r REG 8,1 443788 67478332 /var/ossec/logs/alerts/alerts.json
If the above steps seem to be fine, try to restart your Wazuh manager or some other action that is expected to generate an alert (you can simply SSH in/out your instance).
If some of the above steps differ from my examples, let us know too and stop just there, then we'll guide you on how to fix the deployment.
Let us your results, thanks.
Best regards,
Jesús
Hi Furkan,
In fact, there is something wrong regarding your Elasticsearch.
Filebeat seems to be fine now, so let’s dig into the Elasticsearch issue, here are some useful commands that would help us to determine
what’s failing for you.
Service status:
systemctl status elasticsearch
Elasticsearch logs:
cat /var/log/elasticsearch/<cluster-name|elasticsearch>.log | grep -i -E "warn|error|critic"
If there are no errors and the service is working as expected, try the next two curl commands inside the Elasticsearch instance too:
curl localhost:9200
curl 172.31.45.42:9200
And as a final note, your Elasticsearch configuration would really help with this, so please paste it here too:
cat /etc/elasticsearch/elasticsearch.yml
Best regards,
Jesús
Hi Furkan,
As I can see, we have some points to be fixed:
172.31.45.42:9200
172.31.12.48:9200
localhost
Elasticsearch
Edit your Elasticsearch configuration file, it should look like the next one:
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 172.31.12.48
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["172.31.12.48"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["172.31.12.48"]
Once done, please restart Elasticsearch service:
systemctl restart elasticsearch
Elasticsearch uses to take a bit of time to be ready again after a restart, my suggestion is to wait for a response using the next command:
curl 172.31.12.48:9200/_cluster/health?pretty
Filebeat
Point Filebeat to the right Elasticsearch address: 172.31.12.48:9200
. Edit your /etc/filebeat/filebeat.yml
:
output.elasticsearch:
hosts: ['http://172.31.12.48:9200']
Restart Filebeat:
systemctl restart filebeat
Check the connection again:
filebeat test output
Let us know if it works for you.
Regards,
Jesús