Reduce memory consumption in Cinder services

This patch reduces memory usage for Cinder Volume and Backup services by
tuning glibc.

The specific tuning consist on disabling the per thread arenas and
disabling dynamic thresholds.

The Cinder Backup service suffers from high water mark memory usage and
uses excessive memory.  As an example just after 10 restore operations
the service uses almost 1GB of RAM and does not ever free it afterwards.
With this patch the memory consumption of the service is reduced down to
almost 130MB.  If we add a revert from Cinder (Change-Id
I43a20c8687f12bc52b014611cc6977c4c3ca212c) it goes down to 100MB during
my tests.

It's even worse on real deployments, where we have seen a reduction in
peak memory usage, going down from 2.9GB to 1.2GB, and fixing the high
water mark issue that previously left the service constantly using 2GB
going down to 140MB.

This glibc tuning is not applied to all Python services because I
haven't done proper testings on them and at first glance they don't seem
to have such great improvements.

This is the equivalent of the devstack proposed patch from Change-Id
Ic9030d01468b3189350f83b04a8d1d346c489d3c

Related-bug: #1908805
Change-Id: I65b32f4ce3fddeb694fb33ca65076d45d23a3bb6
This commit is contained in:
Gorka Eguileor 2022-06-14 20:24:36 +02:00
parent 1ad453cca1
commit 6d8b07b7f9
3 changed files with 13 additions and 0 deletions

View File

@ -1,3 +1,7 @@
tcib_envs:
MALLOC_ARENA_MAX: 1
MALLOC_MMAP_THRESHOLD_: 131072
MALLOC_TRIM_THRESHOLD_: 262144
tcib_actions:
- run: dnf -y install {{ tcib_packages.common | join(' ') }} && dnf clean all && rm -rf /var/cache/dnf
- run: mkdir -p /etc/libqb

View File

@ -1,3 +1,7 @@
tcib_envs:
MALLOC_ARENA_MAX: 1
MALLOC_MMAP_THRESHOLD_: 131072
MALLOC_TRIM_THRESHOLD_: 262144
tcib_actions:
- run: dnf -y install {{ tcib_packages.common | join(' ') }} && dnf clean all && rm -rf /var/cache/dnf
- run: cp /usr/share/openstack-tripleo-common-containers/container-images/kolla/cinder-volume/extend_start.sh /usr/local/bin/kolla_extend_start

View File

@ -0,0 +1,5 @@
---
fixes:
- |
Fix high water mark memory usage on Cinder Volume and Backup services and
reduce peak memory usage.