openstack/ci-log-processing
Code Issues Proposed changes

Update Opensearch user, role, ISM policy

Browse Source 
After merging change related to create 'performance-' index [1],
it was necessary to add new permissions into the logstash role.
Also added ISM policy to remove performance data after 14 days.

[1] https://review.opendev.org/c/openstack/ci-log-processing/+/840466

Change-Id: I9f3b922652829ba577186cff2af5e0debe5df088
This commit is contained in:
Daniel Pawlik
2022-06-28 09:26:51 +02:00
parent 91a71a5002
commit 9f226fa5e2
1 changed files with 67 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

67
opensearch-config/README.md
View File

@@ -48,9 +48,22 @@ Details:
tenant permissions: tenant permissions:
tenant: global_tenant tenant: global_tenant
* Logstash role (modify)
Details:
name: logstash
cluster permissions: cluster_monitor, cluster_composite_ops, indices:admin/template/get, indices:admin/template/put, cluster:admin/ingest/pipeline/put, cluster:admin:ingest/pipeline/get
index permissions:
index: logstash-*, performance-*, *beat*
index permissions: crud, create_index
tenant permissions:
tenant: global_tenant
NOTE: NOTE:
The `cluster:monitor/main` role is required to use Python Opensearch client. The `cluster:monitor/main` role is required to use Python Opensearch client.
NOTE:
The index `*beat*` is optional.
### Create role mapping ### Create role mapping
After creating the role, inside the role you will be able to attach the user that should use it. After creating the role, inside the role you will be able to attach the user that should use it.
@@ -59,6 +72,10 @@ After creating the role, inside the role you will be able to attach the user tha
In the OpenSearch Dashboard select `Index Management`, `State management policies`, and then `Create Policy`. Make a policy with the following policy statement: In the OpenSearch Dashboard select `Index Management`, `State management policies`, and then `Create Policy`. Make a policy with the following policy statement:
* For logstash-logs-*
Delete data for logstash-logs index after 14 days
```json ```json
{ {
"policy": { "policy": {
@@ -86,6 +103,13 @@ In the OpenSearch Dashboard select `Index Management`, `State management policie
], ],
"transitions": [] "transitions": []
} }
],
"ism_template": [
{
"index_patterns": [
"logstash-logs-*"
]
}
] ]
} }
} }
@@ -93,6 +117,49 @@ In the OpenSearch Dashboard select `Index Management`, `State management policie
This will delete all indices that are at least 14 days old (e.g. the `logstash-logs-2021.12.15` index will be deleted on 2021-12-22). This will delete all indices that are at least 14 days old (e.g. the `logstash-logs-2021.12.15` index will be deleted on 2021-12-22).
* For performance-*
Policy ID: Delete data for performance index after 14 days
```json
{
"policy": {
"description": "Delete performance data after 14 days",
"default_state": "hot",
"states": [
{
"name": "hot",
"actions": [],
"transitions": [
{
"state_name": "delete",
"conditions": {
"min_index_age": "14d"
}
}
]
},
{
"name": "delete",
"actions": [
{
"delete": {}
}
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"performance-*"
]
}
]
}
}
```
## Advenced settings in Opensearch Dashboards ## Advenced settings in Opensearch Dashboards
There is only few changes applied comparing to default settings. There is only few changes applied comparing to default settings.