Defining fs.aio-max-nr for hosts with libvirt service
When we install libvirt on a host, the system parameter ``fs.aio-max-nr`` is to 1048576. Since we containerized libvirtd, we lost this system parameter. We now make sure it's defined by adding it from the nova-libvirt-common template. Conflict: - roles/ComputeVdpa.yaml is only on Wallaby - adapted for rocky template version Related: https://bugzilla.redhat.com/show_bug.cgi?id=2036195 Closes-Bug: 1959449 Change-Id: I4d82e2a467c4250b23cc7de97233f934b3dbbe46 (cherry picked from commit97c1306b27
) (cherry picked from commite8acb586dd
)
This commit is contained in:
parent
371bb6528f
commit
e01fe8b342
|
@ -102,6 +102,19 @@ parameters:
|
||||||
type: json
|
type: json
|
||||||
tags:
|
tags:
|
||||||
- role_specific
|
- role_specific
|
||||||
|
FsAioMaxNumber:
|
||||||
|
default: 0
|
||||||
|
description: >
|
||||||
|
The kernel allocates aio memory on demand, and this number limits the
|
||||||
|
number of parallel aio requests; the only drawback of a larger limit is
|
||||||
|
that a malicious guest could issue parallel requests to cause the kernel
|
||||||
|
to set aside memory. Set this number at least as large as
|
||||||
|
128 * (number of virtual disks on the host)
|
||||||
|
Libvirt uses a default of 1M requests to allow 8k disks, with at most
|
||||||
|
64M of kernel memory if all disks hit an aio request at the same time.
|
||||||
|
type: number
|
||||||
|
tags:
|
||||||
|
- role_specific
|
||||||
|
|
||||||
# DEPRECATED: the following options are deprecated and are currently maintained
|
# DEPRECATED: the following options are deprecated and are currently maintained
|
||||||
# for backwards compatibility. They will be removed in future release.
|
# for backwards compatibility. They will be removed in future release.
|
||||||
|
@ -109,7 +122,6 @@ parameters:
|
||||||
default: 1024
|
default: 1024
|
||||||
description: Configures sysctl fs.inotify.max_user_instances key
|
description: Configures sysctl fs.inotify.max_user_instances key
|
||||||
type: number
|
type: number
|
||||||
|
|
||||||
parameter_groups:
|
parameter_groups:
|
||||||
- label: deprecated
|
- label: deprecated
|
||||||
description: Do not use deprecated params, they will be removed.
|
description: Do not use deprecated params, they will be removed.
|
||||||
|
@ -120,6 +132,10 @@ conditions:
|
||||||
ipv6_disabled: {equals: [{get_param: KernelDisableIPv6}, 1]}
|
ipv6_disabled: {equals: [{get_param: KernelDisableIPv6}, 1]}
|
||||||
# TODO: remove when misseplt/deprecated parameter InotifyIntancesMax is removed.
|
# TODO: remove when misseplt/deprecated parameter InotifyIntancesMax is removed.
|
||||||
is_inotify_intances_max_default: {equals: [{get_param: InotifyIntancesMax}, 1024]}
|
is_inotify_intances_max_default: {equals: [{get_param: InotifyIntancesMax}, 1024]}
|
||||||
|
fs_aio_max_number_set:
|
||||||
|
and:
|
||||||
|
- not: {equals: [{get_param: [RoleParameters, FsAioMaxNumber]}, 0]}
|
||||||
|
- not: {equals: [{get_param: [RoleParameters, FsAioMaxNumber]}, '']}
|
||||||
|
|
||||||
resources:
|
resources:
|
||||||
# Merging role-specific parameters (RoleParameters) with the default parameters.
|
# Merging role-specific parameters (RoleParameters) with the default parameters.
|
||||||
|
@ -192,4 +208,9 @@ outputs:
|
||||||
- is_inotify_intances_max_default
|
- is_inotify_intances_max_default
|
||||||
- {get_param: InotifyInstancesMax}
|
- {get_param: InotifyInstancesMax}
|
||||||
- {get_param: InotifyIntancesMax}
|
- {get_param: InotifyIntancesMax}
|
||||||
|
- if:
|
||||||
|
- fs_aio_max_number_set
|
||||||
|
- fs.aio-max-nr:
|
||||||
|
value: {get_param: [RoleParameters, FsAioMaxNumber]}
|
||||||
|
- {}
|
||||||
- {get_attr: [RoleParametersValue, value, extra_sysctl_settings]}
|
- {get_attr: [RoleParametersValue, value, extra_sysctl_settings]}
|
||||||
|
|
|
@ -0,0 +1,7 @@
|
||||||
|
---
|
||||||
|
fixes:
|
||||||
|
- |
|
||||||
|
When we install libvirt on a host, the system parameter ``fs.aio-max-nr`` is
|
||||||
|
to 1048576. Since we containerized libvirtd, we lost this system parameter.
|
||||||
|
We now make sure it's defined by adding it from the nova-libvirt-common
|
||||||
|
template.
|
|
@ -18,6 +18,7 @@
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
HostnameFormatDefault: '%stackname%-novacompute-%index%'
|
HostnameFormatDefault: '%stackname%-novacompute-%index%'
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
# Deprecated & backward-compatible values (FIXME: Make parameters consistent)
|
# Deprecated & backward-compatible values (FIXME: Make parameters consistent)
|
||||||
# Set uses_deprecated_params to True if any deprecated params are used.
|
# Set uses_deprecated_params to True if any deprecated params are used.
|
||||||
|
|
|
@ -16,6 +16,7 @@
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
HostnameFormatDefault: '%stackname%-novacomputealt-%index%'
|
HostnameFormatDefault: '%stackname%-novacomputealt-%index%'
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -17,6 +17,7 @@
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
HostnameFormatDefault: '%stackname%-novacompute-dvr-%index%'
|
HostnameFormatDefault: '%stackname%-novacompute-dvr-%index%'
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -16,6 +16,7 @@
|
||||||
StorageMgmt:
|
StorageMgmt:
|
||||||
subnet: storage_mgmt_subnet
|
subnet: storage_mgmt_subnet
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "throughput-performance"
|
TunedProfileName: "throughput-performance"
|
||||||
# CephOSD present so serial has to be 1
|
# CephOSD present so serial has to be 1
|
||||||
update_serial: 1
|
update_serial: 1
|
||||||
|
|
|
@ -19,6 +19,7 @@
|
||||||
# CephOSD present so serial has to be 1
|
# CephOSD present so serial has to be 1
|
||||||
update_serial: 1
|
update_serial: 1
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "cpu-partitioning"
|
TunedProfileName: "cpu-partitioning"
|
||||||
VhostuserSocketGroup: "hugetlbfs"
|
VhostuserSocketGroup: "hugetlbfs"
|
||||||
NovaLibvirtRxQueueSize: 1024
|
NovaLibvirtRxQueueSize: 1024
|
||||||
|
|
|
@ -16,6 +16,7 @@
|
||||||
StorageMgmt:
|
StorageMgmt:
|
||||||
subnet: storage_mgmt_subnet
|
subnet: storage_mgmt_subnet
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "cpu-partitioning"
|
TunedProfileName: "cpu-partitioning"
|
||||||
# CephOSD present so serial has to be 1
|
# CephOSD present so serial has to be 1
|
||||||
update_serial: 1
|
update_serial: 1
|
||||||
|
|
|
@ -16,6 +16,7 @@
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
HostnameFormatDefault: '%stackname%-novacomputeiha-%index%'
|
HostnameFormatDefault: '%stackname%-novacomputeiha-%index%'
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -19,6 +19,7 @@
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
HostnameFormatDefault: '%stackname%-novacompute-local-%index%'
|
HostnameFormatDefault: '%stackname%-novacompute-local-%index%'
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
NovaEnableRbdBackend: false
|
NovaEnableRbdBackend: false
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -17,6 +17,7 @@
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
deprecated_nic_config_name: 'compute-dpdk.yaml'
|
deprecated_nic_config_name: 'compute-dpdk.yaml'
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
VhostuserSocketGroup: "hugetlbfs"
|
VhostuserSocketGroup: "hugetlbfs"
|
||||||
TunedProfileName: "cpu-partitioning"
|
TunedProfileName: "cpu-partitioning"
|
||||||
NovaLibvirtRxQueueSize: 1024
|
NovaLibvirtRxQueueSize: 1024
|
||||||
|
|
|
@ -17,6 +17,7 @@
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
ImageDefault: overcloud-realtime-compute
|
ImageDefault: overcloud-realtime-compute
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
VhostuserSocketGroup: "hugetlbfs"
|
VhostuserSocketGroup: "hugetlbfs"
|
||||||
TunedProfileName: "realtime-virtual-host"
|
TunedProfileName: "realtime-virtual-host"
|
||||||
NovaLibvirtRxQueueSize: 1024
|
NovaLibvirtRxQueueSize: 1024
|
||||||
|
|
|
@ -13,6 +13,7 @@
|
||||||
- Tenant
|
- Tenant
|
||||||
- Storage
|
- Storage
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
VhostuserSocketGroup: "hugetlbfs"
|
VhostuserSocketGroup: "hugetlbfs"
|
||||||
TunedProfileName: "cpu-partitioning"
|
TunedProfileName: "cpu-partitioning"
|
||||||
NovaLibvirtRxQueueSize: 1024
|
NovaLibvirtRxQueueSize: 1024
|
||||||
|
|
|
@ -14,6 +14,7 @@
|
||||||
- Storage
|
- Storage
|
||||||
ImageDefault: overcloud-realtime-compute
|
ImageDefault: overcloud-realtime-compute
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
VhostuserSocketGroup: "hugetlbfs"
|
VhostuserSocketGroup: "hugetlbfs"
|
||||||
TunedProfileName: "realtime-virtual-host"
|
TunedProfileName: "realtime-virtual-host"
|
||||||
NovaLibvirtRxQueueSize: 1024
|
NovaLibvirtRxQueueSize: 1024
|
||||||
|
|
|
@ -17,6 +17,7 @@
|
||||||
HostnameFormatDefault: '%stackname%-novacomputeppc64le-%index%'
|
HostnameFormatDefault: '%stackname%-novacomputeppc64le-%index%'
|
||||||
ImageDefault: ppc64le-overcloud-full
|
ImageDefault: ppc64le-overcloud-full
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -19,6 +19,7 @@
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
HostnameFormatDefault: '%stackname%-novacompute-rbd-%index%'
|
HostnameFormatDefault: '%stackname%-novacompute-rbd-%index%'
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
NovaEnableRbdBackend: true
|
NovaEnableRbdBackend: true
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -21,6 +21,7 @@
|
||||||
HostnameFormatDefault: '%stackname%-computerealtime-%index%'
|
HostnameFormatDefault: '%stackname%-computerealtime-%index%'
|
||||||
ImageDefault: overcloud-realtime-compute
|
ImageDefault: overcloud-realtime-compute
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "realtime-virtual-host"
|
TunedProfileName: "realtime-virtual-host"
|
||||||
KernelArgs: "" # these must be set in an environment file or
|
KernelArgs: "" # these must be set in an environment file or
|
||||||
IsolCpusList: "" # similar according to the hardware of
|
IsolCpusList: "" # similar according to the hardware of
|
||||||
|
|
|
@ -15,6 +15,7 @@
|
||||||
Storage:
|
Storage:
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "cpu-partitioning"
|
TunedProfileName: "cpu-partitioning"
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -15,6 +15,7 @@
|
||||||
Storage:
|
Storage:
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "cpu-partitioning"
|
TunedProfileName: "cpu-partitioning"
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -16,6 +16,7 @@
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
ImageDefault: overcloud-realtime-compute
|
ImageDefault: overcloud-realtime-compute
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "realtime-virtual-host"
|
TunedProfileName: "realtime-virtual-host"
|
||||||
NovaLibvirtMemStatsPeriodSeconds: 0
|
NovaLibvirtMemStatsPeriodSeconds: 0
|
||||||
NovaComputeDisableIrqBalance: True
|
NovaComputeDisableIrqBalance: True
|
||||||
|
|
|
@ -15,6 +15,7 @@
|
||||||
Storage:
|
Storage:
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -16,6 +16,7 @@
|
||||||
StorageMgmt:
|
StorageMgmt:
|
||||||
subnet: storage_mgmt_subnet
|
subnet: storage_mgmt_subnet
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "throughput-performance"
|
TunedProfileName: "throughput-performance"
|
||||||
# CephOSD present so serial has to be 1
|
# CephOSD present so serial has to be 1
|
||||||
update_serial: 1
|
update_serial: 1
|
||||||
|
|
|
@ -18,6 +18,7 @@
|
||||||
StorageDashboard:
|
StorageDashboard:
|
||||||
subnet: storage_dashboard_subnet
|
subnet: storage_dashboard_subnet
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "throughput-performance"
|
TunedProfileName: "throughput-performance"
|
||||||
# CephOSD present so serial has to be 1
|
# CephOSD present so serial has to be 1
|
||||||
update_serial: 1
|
update_serial: 1
|
||||||
|
|
|
@ -16,6 +16,7 @@
|
||||||
StorageMgmt:
|
StorageMgmt:
|
||||||
subnet: storage_mgmt_subnet
|
subnet: storage_mgmt_subnet
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "throughput-performance"
|
TunedProfileName: "throughput-performance"
|
||||||
# CephOSD present so serial has to be 1
|
# CephOSD present so serial has to be 1
|
||||||
update_serial: 1
|
update_serial: 1
|
||||||
|
|
|
@ -15,6 +15,7 @@
|
||||||
Storage:
|
Storage:
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
update_serial: 25
|
update_serial: 25
|
||||||
ServicesDefault:
|
ServicesDefault:
|
||||||
|
|
|
@ -209,6 +209,7 @@
|
||||||
subnet: storage_subnet
|
subnet: storage_subnet
|
||||||
HostnameFormatDefault: '%stackname%-novacompute-%index%'
|
HostnameFormatDefault: '%stackname%-novacompute-%index%'
|
||||||
RoleParametersDefault:
|
RoleParametersDefault:
|
||||||
|
FsAioMaxNumber: 1048576
|
||||||
TunedProfileName: "virtual-host"
|
TunedProfileName: "virtual-host"
|
||||||
# Deprecated & backward-compatible values (FIXME: Make parameters consistent)
|
# Deprecated & backward-compatible values (FIXME: Make parameters consistent)
|
||||||
# Set uses_deprecated_params to True if any deprecated params are used.
|
# Set uses_deprecated_params to True if any deprecated params are used.
|
||||||
|
|
Loading…
Reference in New Issue