Commit Graph

15 Commits

Author SHA1 Message Date
Andrew Bonney
2b65764a2e Enable cluster ID monitoring setting for beats using logstash
Change-Id: I59e7c92ebd91ceca161577e73d9f24826abedda1
2022-11-02 09:47:04 +00:00
Andrew Bonney
5437ddbd86 Add support for enabling ELK stack security
Change-Id: I661662c0784010ca2fcc3b3d31df1a1d79dbed1e
2022-11-02 09:17:45 +00:00
Erik Berg
f97ebbf990 Update beat templates from 7.11 reference
Depends-On: Ida8668f5548a15a8f597839bd9002585aeea5d1a
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-ops/+/843423

Change-Id: Ib889cc7ac7ad2540031016075cb5baab091bd6e3
2022-05-26 12:19:17 +00:00
Zuul
d12caed056 Merge "Alter default shard count in index templates" 2020-05-12 17:16:02 +00:00
Andrew Bonney
4f7995fe1a logstash: add cisco ios/nexus syslog parsing to pipeline
Adds parsing for Cisco NXOS/IOS syslog format, along with the
RFC5424 variant some devices can use. Messages which match these
patterns are fingerprinted based upon the message and host to
de-duplicate them when storing in elasticsearch.

Change-Id: I42fd441913d2095997c3493c37515362a5d732fc
2020-05-06 16:35:17 +01:00
Duncan Martin Walker
0cd1f76757 Alter default shard count in index templates
Currently, we use only a single primary shard for each beat index,
as configured by the index templates. This is not usually an optimal
setup, as all index searching is handled on a single node. A more
efficient configuration for a small number of different indices is
to balance, with around one primary shard per node. This
commit exposes the config option to set the number of primary shards,
which was previously hardcoded to 1. This now defaults to one shard
per data node in the cluster.

Change-Id: Icacf76c2270c98f8676d57952c00773646ca7b51
2020-05-04 17:18:29 +01:00
Duncan Martin Walker
4851e29e9c Subdivision of elastic-logstash group
It is currently assumed that the elasticsearch data nodes and the
logstash nodes are co-located within the same container during
deployment of the ELK stack. This commit is intended to remove that
assumption by requiring specification of separate elastic and
logstash groups. This is particularly important where instances of
elasticsearch and logstash are separated for resource reasons, a
common setup for an elastic cluster.

The default setup for the elastic nodes has been simplified, with
all non-coordinating nodes being both data and ingest nodes unless
otherwise specified. This was previously defined algorithmically,
with the elastic node setup tightly coupled to the placement of the
logstash instances.

Change-Id: Id06eb78b52705aefea9cfe1247f53bac58badd52
2020-03-18 16:44:58 +00:00
Duncan Martin Walker
81ab7008e1 Exposed beat logging levels
The elastic beats are currently deployed with a default logging level of
info, which can produce large amounts of data, particularly in the
case that beats are monitoring one another's logs, and then logging this fact.
This commit exposes the configuration, allowing it to be set by the
variable log_level. The default level is unchanged.

Change-Id: Idbf715fc99450003920a45323123fcba919f2a83
2020-02-18 10:09:48 +00:00
Sam Choraria
81eb58f1e8 Allow beat processors to be defined through configuration data
Processors for each of the beats are currently defined per template and
can be configured directly via template or once deployed. This change
allows processors for all or a subset of beats to be configured through
role data or via an overlay inventory.

Change-Id: I71fc96611082555b43fd0f505219c42c890464ef
2020-02-10 18:34:48 +00:00
Zuul
f73c2d05dc Merge "Exposed config for logstash elasticsearch endpoints" 2020-02-10 17:07:45 +00:00
Duncan Martin Walker
c30ad4c447 Increase logstash output connection timeout for beats
When using Metricbeat in an environment where only the system module
is enabled, the configuration reports system metrics to logstash every
60s. However, the current connection timout is shorter than this,
leading to the connection resetting and re-opening repeatedly due to
i/o timeouts. This commit increases the timeout to 90s, such that the
connection stays open and errors aren't reported to the metricbeat
logs.

Change-Id: Ib30f983df71c0b0381bfa23d9d2c7f1220184a4b
2020-02-06 10:49:55 +00:00
Duncan Martin Walker
b21034c4b8 Exposed config for logstash elasticsearch endpoints
The Elasticsearch endpoints in use by logstash are currently hardcoded
to localhost in both the logstash monitoring config and the default
logstash pipeline template. This commit allows this to be configured,
such that (multiple) remote endpoints can be specified for the case that
logstash is not co-located with elasticsearch. The default behaviour is
unchanged.

Change-Id: Id871f201168ff882a5b37677747c3484f908298f
2020-02-05 16:41:24 +00:00
Duncan Martin Walker
a192fb3129 Elastic beat index template configuration
Templates for the beat config files have been updated to allow more
configuration options to be passed through to the associated
Elasticsearch index templates. In particular, one can optionally
specify values of index_template_max_docvalue_search to be set at
the creation of the beat index template. This can prevent shard failure
errors when viewing output in Kibana relating to "Trying to retrieve
too many docvalue_fields". Any similar config options can in future
be passed into the template via the elastic_beat_settings object.

Change-Id: Ic9136c8e063bbd231ed280bb446661b251879407
2020-01-23 17:23:48 +00:00
Georgina Shippey
68664a9dc1 Config updates for elk 7.x
Updated ELK config files to elk 7.x reference samples, bringing over
existing customisation from elk_metrics_6x.

Removed deprecated use of --pipeline in elastic_beat_setup/tasks/main.yml,
--pipeline is no longer a valid cli argument.

Updated logstash-pipelines and removed the dynamic insertion of the date into
index names. This function is now done with the new ILM feature in elasticsearch
rather than logstash.

Installation of each beat creates an ILM policy for that beat and this patch
does not change the default policy. It is possible that the default policy
will exhaust the available storage and future work needs to be done to address
this.

The non-beat elements of the logstash pipeline (syslog, collectd and others)
are not yet updated to be compatible with ILM.

Change-Id: I735b64c2b7b93e23562f35266134a176a00af1b7
2019-08-05 07:47:35 +00:00
Georgina Shippey
5e96844123 Duplicate of elk_metrics_6x to elk_metrics_7x
Change-Id: I92a894e31f725a20c684165f93dd4c34b9c8b450
2019-07-10 17:52:49 +01:00