From 695343a40de93821eda5304b49b12f8065fb6ad8 Mon Sep 17 00:00:00 2001 From: Svetlana Karslioglu Date: Tue, 14 Jun 2016 10:23:59 -0700 Subject: [PATCH] Added the cgroups treshold paramter. Broke the cgroups.rst file in multiple files by sections. Added the recommended treshold parameters. Change-Id: I311117ed7fde5f3a25ee82aa83d4dcea08ff107e --- .../fuel-user-guide/manage-environment.rst | 4 + .../manage-environment/cgroups.rst | 175 +++--------------- .../cgroups/cgroups-configure.rst | 63 +++++++ .../cgroups/cgroups-example.rst | 44 +++++ .../cgroups/cgroups-modify-multiple-nodes.rst | 42 +++++ .../cgroups/cgroups-modify-single-node.rst | 47 +++++ 6 files changed, 224 insertions(+), 151 deletions(-) create mode 100644 userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-configure.rst create mode 100644 userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-example.rst create mode 100644 userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-modify-multiple-nodes.rst create mode 100644 userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-modify-single-node.rst diff --git a/userdocs/fuel-user-guide/manage-environment.rst b/userdocs/fuel-user-guide/manage-environment.rst index fe05c2a62..9effebabb 100644 --- a/userdocs/fuel-user-guide/manage-environment.rst +++ b/userdocs/fuel-user-guide/manage-environment.rst @@ -12,4 +12,8 @@ This section includes the following sections: :maxdepth: 1 manage-environment/cgroups.rst + manage-environment/cgroups/cgroups-configure.rst + manage-environment/cgroups/cgroups-modify-multiple-nodes.rst + manage-environment/cgroups/cgroups-modify-single-node.rst + manage-environment/cgroups/cgroups-example.rst manage-environment/role-operations.rst diff --git a/userdocs/fuel-user-guide/manage-environment/cgroups.rst b/userdocs/fuel-user-guide/manage-environment/cgroups.rst index dd298dd5b..b75041479 100644 --- a/userdocs/fuel-user-guide/manage-environment/cgroups.rst +++ b/userdocs/fuel-user-guide/manage-environment/cgroups.rst @@ -11,8 +11,10 @@ to the underlying middleware components, such as RabbitMQ, MySQL, and others. Rational assignment may significantly improve the performance of your system. Fuel implementation of control groups enables you to create one group for one -process and then create a hierarchy of groups. By default, ``cgroups`` -are enabled, but no limits are configured. +process and then create a hierarchy of groups. By default, Fuel enables +``cgroups``, however, Fuel does not configure any limits. You can configure +``cgroups`` using the ``settings.yaml`` file before you deploy an OpenStack +environment or after deployment. .. note:: Fuel supports control groups only for Ubuntu 14.04. If you have integrated @@ -22,7 +24,7 @@ are enabled, but no limits are configured. Fuel supports all standard Linux ``cgroups`` resource controllers, or subsystems. However, to optimize the performance of your system you will mostly use -the following resource controllers and limits. +the resource controllers and limits described in the following table. .. list-table:: **Resource controllers and limits** :widths: 10 10 @@ -56,7 +58,8 @@ the following resource controllers and limits. ``memory.swappiness`` Controls swap priority. Determines whether kernel can claim memory - from the control group. + from the control group. Set this value to 0 for high memory consuming + processes, such as MySQL, MongoDB, RabbitMQ. * - ``cpu`` - Controls the use of CPU. @@ -71,157 +74,27 @@ the following resource controllers and limits. for all control groups is 100_000. Which means 100% usage. To restrict the use of CPU, modify this setting. For example, to use 30% of CPU, change the - value of `cpu.cfs_quota_us`` to 30000. + value of `cpu.cfs_quota_us`` to 30000. Controlling the use of the + CPU resources through the ``cpu.cfs_quota_us`` and + ``cpu.cfs_period_us`` parameters provides more accurate resource + allocation compared to using ``cpu.shares``. + + You can set the following CPU limits: + + * ``beam.smp`` - for RabbitMQ up to 40%. + * ``pangine`` - for Pacemaker up to 10%. + * ``nova-api`` - for Nova 20 - 45%. + * ``nova-conductor`` - for Nova up to 20%. + * ``mysqld`` - for MySQL up to 10%. + * ``neutron-server`` - for Neutron - 10%. ``cpu.shares`` Defines a share of CPU resources available to each control group. The default value is 1024. For example, if you assign *1024* to one process and *512* to the other process, kernel proportionally allocates more CPU for the first process and less for the second. + Using the ``cpu.shares`` parameter to control CPU usage may not + result in accurate CPU resource allocation. - -.. _cgroups-configure: - -Configure control groups -++++++++++++++++++++++++ - -You can configure ``cgroups`` for multiple nodes before you deploy an -OpenStack environment by editing the ``settings_1.yaml`` file on the -Fuel Master node. - -**To configure control groups:** - -#. Log in to the Fuel Master node CLI. -#. Download the Fuel configuration: - - .. code-block:: console - - fuel settings --env-id --download - -#. Open the ``settings_1.yaml`` file for editing. -#. Add the required services to the ``cgroups`` section. - - **Example:** - - .. code-block:: console - - editable: - cgroups: - metadata: - always_editable: true - group: general - label: Cgroups conguration for services - restrictions: - - action: hide - condition: 'true' - weight: 90 - mysqld: - label: mysqld - type: text - value: '{"memory":{"memory.swappiness":0, - "memory.soft_limit_in_bytes":"%5, 10, 20"}}' - beam.smp: - label: beam.smp - type: text - value: '{"memory":{"memory.swappiness":0}}' - cinder-api: - label: cinder-api - type: text - value: '{"blkio":{"blkio.weight":500}}' - keystone-api: - label: keystone-api - type: text - value: '{"cpu":{"cpu.shares":70}}' - neutron-server: - label: neutron-server - type: text - value: '{"memory":{"memory.soft_limit_in_bytes":"%total, min, max"}}' - -#. Save and exit. -#. Upload the new configuration file to Fuel: - - .. code-block:: console - - fuel settings --env-id --upload - -.. _cgroups-modify-multiple-nodes: - -Modify control groups for multiple nodes -++++++++++++++++++++++++++++++++++++++++ - -You can modify ``cgroups`` for a particular process on multiple nodes by -creating a separate file with the ``cgroups`` configuration, uploading -the new configuration file to fuel, and restarting the ``cgroups`` task. -You can modify control groups before or after you deploy an OpenStack -environment. - -**To modify control groups for multiple nodes:** - -#. Log in to the Fuel Master node CLI. -#. #. Download the Fuel configuration: - - .. code-block:: console - - fuel settings --env-id --download - -#. Open the ``settings.yaml`` file for editing. -#. Copy the ``cgroups`` section. -#. Create a blank ``.yaml`` file. -#. Paster the copied ``cgroups`` configuration into the file. -#. Edit as required. -#. Upload the new configuration file to Fuel: - - .. code-block:: console - - fuel settings --dir --env-id --upload - -#. Restart the ``cgroups`` task: - - .. code-block:: console - - fuel node --node-id --tasks cgroups - - -.. _cgroups-modify-single-node: - -Modify control groups for a single node -+++++++++++++++++++++++++++++++++++++++ - -If you want to change the control group settings on a single node, you must -edit the control groups configuration file ``/etc/cgconfig.conf``, as well -as create, if needed, and configure the ``/etc/cgrules.conf`` file. -You can modify control groups before or after you deploy an OpenStack -environment. - -**To modify control groups for a single node:** - -#. Log in to the CLI of corresponding node. -#. Open the ``/etc/cgconfig.conf`` file for editing. -#. Apply the required changes. -#. Save and exit. -#. Add the corresponding parameters to the ``/etc/cgrules.conf`` file. - - **Example:** - - .. code-block:: console - - * :keystone-api cpu keystone-api - * :mysqld cpu mysqld - -#. Restart ``cgconfigparser``: - - .. code-block:: console - - service cgconfigparser restart - -#. For each running process, type: - - .. code-block:: console - - cgclassify 'pidof -x ' - -#. Restart ``cgrulesengd``: - - .. code-block:: console - - service cgrulesengd restart + * - ``cpuset`` + - Controls the use of memory and processor processes. diff --git a/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-configure.rst b/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-configure.rst new file mode 100644 index 000000000..8b5cd10fe --- /dev/null +++ b/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-configure.rst @@ -0,0 +1,63 @@ +.. _cgroups-configure: + +Configure control groups +++++++++++++++++++++++++ + +You can configure ``cgroups`` for multiple nodes before you deploy an +OpenStack environment by editing the ``settings_1.yaml`` file on the +Fuel Master node. + +**To configure control groups:** + +#. Log in to the Fuel Master node CLI. +#. Download the Fuel configuration: + + .. code-block:: console + + fuel settings --env-id --download + +#. Open the ``settings_1.yaml`` file for editing. +#. Add the required services to the ``cgroups`` section. + + **Example:** + + .. code-block:: console + + editable: + cgroups: + metadata: + always_editable: true + group: general + label: Cgroups conguration for services + restrictions: + - action: hide + condition: 'true' + weight: 90 + mysqld: + label: mysqld + type: text + value: '{"memory":{"memory.swappiness":0, + "memory.soft_limit_in_bytes":"%5, 10, 20"}}' + beam.smp: + label: beam.smp + type: text + value: '{"memory":{"memory.swappiness":0}}' + cinder-api: + label: cinder-api + type: text + value: '{"blkio":{"blkio.weight":500}}' + keystone-api: + label: keystone-api + type: text + value: '{"cpu":{"cpu.shares":70}}' + neutron-server: + label: neutron-server + type: text + value: '{"memory":{"memory.soft_limit_in_bytes":"%total, min, max"}}' + +#. Save and exit. +#. Upload the new configuration file to Fuel: + + .. code-block:: console + + fuel settings --env-id --upload diff --git a/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-example.rst b/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-example.rst new file mode 100644 index 000000000..366d2e002 --- /dev/null +++ b/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-example.rst @@ -0,0 +1,44 @@ +.. _cgroups-example: + +Example of the control groups configuration +------------------------------------------- + +The following text is an example of the section +in the ``settings.yaml`` file that describes the ``cgroups`` +configuration: + +.. code-block:: console + + editable: + cgroups: + mysqld: + label: mysqld + type: text + value: '{"memory":{"memory.swappiness":0, "memory.limit_in_bytes":"%95, + 2048, 4096"}, , "cpu":{"cpu.shares":100}}' + beam.smp: + label: rabbitmq + type: text + value: '{"memory":{"memory.swappiness":0}, "cpu":{"cpu.shares":400, + "cpu.cfs_quota_us":25000}}' + cinder-volume: + label: cinder-volume + type: text + value: '{"blkio":{"blkio.weight":1000}}' + nova-api: + label: nova-api + type: text + value: '{"cpu":{"cpu.shares":700}}' + neutron-server: + label: neutron-server + type: text + value: '{"cpuset":{"cpuset.sched_load_balance":0,"cpuset.cpus":1, + "cpuset.mems":0}}' + metadata: + always_editable: true + group: general + label: Cgroups conguration for services + restrictions: + - action: hide + condition: 'true' + weight: 90 diff --git a/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-modify-multiple-nodes.rst b/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-modify-multiple-nodes.rst new file mode 100644 index 000000000..acefdbafd --- /dev/null +++ b/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-modify-multiple-nodes.rst @@ -0,0 +1,42 @@ +.. _cgroups-modify-multiple-nodes: + +Modify control groups for multiple nodes +++++++++++++++++++++++++++++++++++++++++ + +You can modify ``cgroups`` for a particular process on multiple nodes by +creating a separate file with the ``cgroups`` configuration, uploading +the new configuration file to fuel, and restarting the ``cgroups`` task. +You can modify control groups before or after you deploy an OpenStack +environment. + +**To modify control groups for multiple nodes:** + +#. Log in to the Fuel Master node CLI. +#. Download the Fuel configuration: + + .. code-block:: console + + fuel settings --env-id --download + +#. Open the ``settings.yaml`` file for editing. +#. Copy the ``cgroups`` section. +#. Create a ``.yaml`` file with the same name as the ``.yaml`` + file that you have downloaded in step 2. +#. Paste the copied ``cgroups`` configuration into the file. +#. Edit as required.. + +#. Upload the new configuration file to Fuel: + + .. code-block:: console + + fuel settings --dir --env-id --upload + +#. Restart the ``cgroups`` task: + + .. code-block:: console + + fuel node --node-id --tasks cgroups + +.. seealso:: + + - :ref:`cgroups-example` diff --git a/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-modify-single-node.rst b/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-modify-single-node.rst new file mode 100644 index 000000000..d195e1ea6 --- /dev/null +++ b/userdocs/fuel-user-guide/manage-environment/cgroups/cgroups-modify-single-node.rst @@ -0,0 +1,47 @@ +.. _cgroups-modify-single-node: + +Modify control groups for a single node ++++++++++++++++++++++++++++++++++++++++ + +If you want to change the control group settings on a single node, you must +edit the control groups configuration file ``/etc/cgconfig.conf``, as well +as create, if needed, and configure the ``/etc/cgrules.conf`` file. +You can modify control groups before or after you deploy an OpenStack +environment. + +**To modify control groups for a single node:** + +#. Log in to the CLI of corresponding node. +#. Open the ``/etc/cgconfig.conf`` file for editing. +#. Apply the required changes. +#. Save and exit. +#. Add the corresponding parameters to the ``/etc/cgrules.conf`` file. + + **Example:** + + .. code-block:: console + + * :keystone-api cpu keystone-api + * :mysqld cpu mysqld + +#. Restart ``cgconfigparser``: + + .. code-block:: console + + service cgconfigparser restart + +#. For each running process, type: + + .. code-block:: console + + cgclassify 'pidof -x ' + +#. Restart ``cgrulesengd``: + + .. code-block:: console + + service cgrulesengd restart + +.. seealso:: + + - :ref:`cgroups-example`