openstack/ci-log-processing
Code Issues Proposed changes

Update Opensearch user, role, ISM policy

Browse Source 
After merging change related to create 'performance-' index [1],
it was necessary to add new permissions into the logstash role.
Also added ISM policy to remove performance data after 14 days.

[1] https://review.opendev.org/c/openstack/ci-log-processing/+/840466

Change-Id: I9f3b922652829ba577186cff2af5e0debe5df088
This commit is contained in:
Daniel Pawlik
2022-06-28 09:26:51 +02:00
parent 91a71a5002
commit 9f226fa5e2
1 changed files with 67 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

67
opensearch-config/README.md
View File

@@ -48,9 +48,22 @@ Details:
tenant permissions:
tenant: global_tenant
* Logstash role (modify)
Details:
name: logstash
cluster permissions: cluster_monitor, cluster_composite_ops, indices:admin/template/get, indices:admin/template/put, cluster:admin/ingest/pipeline/put, cluster:admin:ingest/pipeline/get
index permissions:
index: logstash-*, performance-*, *beat*
index permissions: crud, create_index
tenant permissions:
tenant: global_tenant
NOTE:
The `cluster:monitor/main` role is required to use Python Opensearch client.
NOTE:
The index `*beat*` is optional.
### Create role mapping
After creating the role, inside the role you will be able to attach the user that should use it.
@@ -59,6 +72,10 @@ After creating the role, inside the role you will be able to attach the user tha
In the OpenSearch Dashboard select `Index Management`, `State management policies`, and then `Create Policy`. Make a policy with the following policy statement:
* For logstash-logs-*
Delete data for logstash-logs index after 14 days
```json
{
"policy": {
@@ -86,6 +103,13 @@ In the OpenSearch Dashboard select `Index Management`, `State management policie
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"logstash-logs-*"
]
}
]
}
}
@@ -93,6 +117,49 @@ In the OpenSearch Dashboard select `Index Management`, `State management policie
This will delete all indices that are at least 14 days old (e.g. the `logstash-logs-2021.12.15` index will be deleted on 2021-12-22).
* For performance-*
Policy ID: Delete data for performance index after 14 days
```json
{
"policy": {
"description": "Delete performance data after 14 days",
"default_state": "hot",
"states": [
{
"name": "hot",
"actions": [],
"transitions": [
{
"state_name": "delete",
"conditions": {
"min_index_age": "14d"
}
}
]
},
{
"name": "delete",
"actions": [
{
"delete": {}
}
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"performance-*"
]
}
]
}
}
```
## Advenced settings in Opensearch Dashboards
There is only few changes applied comparing to default settings.