openstack/kolla-ansible
Code Issues Proposed changes
17d8332604
kolla-ansible/ansible/roles/monasca/templates/monasca-log-transformer/monasca-log-transformer.json.j2

26 lines
866 B
Plaintext
Raw Normal View History

Support deploying Monasca Log Transformer The Monasca Log Transformer takes raw, unstandardised logs from one Kafka topic, standardises them with whatever rules the operator wants to use, and then writes them to a standardised logs topic in Kafka. It is currently implemented as a Logstash config file. Since Kolla does a fairly good job of standardising logs, this service does very little processing. However, when other sources of logs are used, it may be useful to add rules to the Transformer, particularly if it's not possible to standardise the logs at source. Ingesting the logs from this service with Fluentd will be covered under a different commit. Change-Id: I31cbb7e9a40a848391f517a56a67e3fd5bc12529 Partially-Implements: blueprint monasca-roles
2018-06-04 15:34:38 +01:00
{
Logstash 6 support Co-Authored-By: Doug Szumski <doug@stackhpc.com> Closes-Bug: #1884090 Depends-On: https://review.opendev.org/#/c/736768 Change-Id: If2d0dd1739e484b14e3c15a185a236918737b0ab
2020-07-07 13:43:18 +00:00
"command": "/usr/share/logstash/bin/logstash --path.settings /etc/logstash/ --log.format json --path.logs /var/log/kolla/logstash/monasca-log-transformer -f /etc/logstash/conf.d/log-transformer.conf",
Support deploying Monasca Log Transformer The Monasca Log Transformer takes raw, unstandardised logs from one Kafka topic, standardises them with whatever rules the operator wants to use, and then writes them to a standardised logs topic in Kafka. It is currently implemented as a Logstash config file. Since Kolla does a fairly good job of standardising logs, this service does very little processing. However, when other sources of logs are used, it may be useful to add rules to the Transformer, particularly if it's not possible to standardise the logs at source. Ingesting the logs from this service with Fluentd will be covered under a different commit. Change-Id: I31cbb7e9a40a848391f517a56a67e3fd5bc12529 Partially-Implements: blueprint monasca-roles
2018-06-04 15:34:38 +01:00
"config_files": [
{
"source": "{{ container_config_directory }}/log-transformer.conf",
"dest": "/etc/logstash/conf.d/log-transformer.conf",
"owner": "logstash",
"perm": "0600"
Add support for using custom Logstash patterns A user may want to define and use Logstash patterns. This commit adds support to copy them into the Monasca Log Transformer container. In the future support could be added for other Logstash containers. Change-Id: Id8cde14af6dc7f49714f6b1cb878882d0048d293
2019-04-23 15:46:00 +01:00
},
{
"source": "{{ container_config_directory }}/logstash_patterns/*",
"dest": "/etc/logstash/conf.d/patterns/",
"owner": "logstash",
"perm": "0600",
"optional": true
Support deploying Monasca Log Transformer The Monasca Log Transformer takes raw, unstandardised logs from one Kafka topic, standardises them with whatever rules the operator wants to use, and then writes them to a standardised logs topic in Kafka. It is currently implemented as a Logstash config file. Since Kolla does a fairly good job of standardising logs, this service does very little processing. However, when other sources of logs are used, it may be useful to add rules to the Transformer, particularly if it's not possible to standardise the logs at source. Ingesting the logs from this service with Fluentd will be covered under a different commit. Change-Id: I31cbb7e9a40a848391f517a56a67e3fd5bc12529 Partially-Implements: blueprint monasca-roles
2018-06-04 15:34:38 +01:00
}
],
"permissions": [
{
"path": "/var/log/kolla/logstash",
"owner": "logstash:kolla",
"recurse": true
}
]
}
Copy Permalink