This updates the fluentd buffer output configurations to account
for the restraints of the jobs deploying fluentd. This also
renames the fluentd configuration key from td_agent to fluentd to
reflect the fact we're no longer deploying td-agent
This also updates the Elasticsearch default replicas and overrides
the replica counts in each Elasticsearch deployment to account for
resource constraints
Change-Id: I55dee410eced99c3e1645f7452e4306ad646e601
This updates the osh-infra charts to use a secret for their
configuration files instead of a configmap, allowing for the
storage of sensitive information
Change-Id: Ia32587162288df0b297c45fd43b55cef381cb064
This updates Fluentd to use the stable v1.2 debian fluentd
image instead of the kolla image. This images comes bundled
with the elasticsearch plugin, and provides more
flexibility in configuring the buffer behavior of the output
plugins
Change-Id: Id446ef1e050f5d9c005c94dae661cf9ae88fffea
This PS simply moves functions within the chart to their correct location.
Change-Id: Ia5ac02a25a76ff759160cc352404b71b4208b216
Signed-off-by: Pete Birley <pete@port.direct>
This updates the fluent-logging chart to support the creation of
an arbitrary number of templates for elasticsearch. This allows
for the definition of multiple index mappings driven via the
chart's values. This provides flexibility in determining specific
structures for indexes that may differ between log types.
This also moves to define these mappings via json instead of XML.
As gotpl can convert yaml directly to json, and elasticsearch can
ingest json directly for index creation, we no longer need an XML
helper function to generate the required configuration. This helps
reduce the number of helper functions we need to maintain
Change-Id: I3c85fb9a1e700eb1592d96f83e632172d0eb2681
Fluent-logging stores logs on a elasticsearch by default. Elasticsearch
stores all fields as tokens by default, but some fields shoud be stored
for purposes such as retrieval, without splitting. Mapping in elasticsearch
is used to define a property of fields and Template can defines the mapping
for an index. fluent-logging use it to define the index structure. Specific
index type can be defined on the value file.
Change-Id: Id597111f478fcddf709b36d2db9ac5a5d6d8206f
Defines configuration files for fluentbit and fluentd via the
values.yaml file for fluent-logging. This provides flexibility in
defining parsers and routes for log gathering and routing
This functionality is added via helm-toolkit helper functions for
both fluentd and fluentbit to make the values configuration
cleaner
Change-Id: I8a43f36e487e651561bec8abf7752c8fac68aefc
This introduces an initial helm chart for fluent logging.
It provides a functional fluent-bit and fluentd deployment to
use in conjunction with elasticsearch and kibana to consume
and aggregate logs from all resource types in a cluster.
It can deliver logs to kafka for external tools to consume.
This PS moves fluent-logging chart from osh-addons, osh to
osh-infra repo.
previous ps(addon): https://review.openstack.org/#/c/507023/
previous ps(osh): https://review.openstack.org/#/c/514622/
Specification: https://review.openstack.org/#/c/505491/
Partially implements: blueprint osh-logging-framework
Change-Id: I72e580aa3a197550060fc07af8396a7c8368d40b