Default performance schema to off
The 5.6 version of mysql defauls to performance_schema=on at server startup. This allocates all the memory that would be required to handle max-connections plus several other memory settings. We occasionally saw connection exhaustion in HA clouds with max-connections near 2000. The common practice became to set max-connections unrealistically high near 10k or 20k. In the move to 5.6 on Xenial this became a problem as the memory requirements for values that high are unrealistic. This change adds performance schema as a configuration option and defaults to off giving 5.5 like behavior. It also considerably updates documentation and highlights the need to carefully consider memory related configuration settings for production environments. Make the amulet test python2 and python3 capable. Change-Id: Id01095ac2f73fa041b25b2602496a4c04a13e3ef Partial-Bug: #1654086 Closes-Bug: #1600001
This commit is contained in:
parent
23a76721f9
commit
e47c4ad135
40
README.md
40
README.md
@ -49,6 +49,46 @@ deployment:
|
|||||||
|
|
||||||
A minimium cluster size of three units is recommended.
|
A minimium cluster size of three units is recommended.
|
||||||
|
|
||||||
|
Memory Configuration
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
Percona Cluster is extremely memory sensitive. Setting memory values too low
|
||||||
|
will give poor performance. Setting them too high will create problems that are
|
||||||
|
very difficult to diagnose. Please take time to evaluate these settings for
|
||||||
|
each deployment environment rather than copying and pasting bundle
|
||||||
|
configurations.
|
||||||
|
|
||||||
|
The Percona Cluster charm needs to be able to be deployed in small low memory
|
||||||
|
development environments as well as high performance production environments.
|
||||||
|
The charm configuration opinionated defaults favor the developer environment in
|
||||||
|
order to ease initial testing. Production environments need to consider
|
||||||
|
carefully the memory requirements for the hardware or cloud in use. Consult a
|
||||||
|
MySQL memory calculator [2] to understand the implications of the values.
|
||||||
|
|
||||||
|
Between the 5.5 and 5.6 releases a significant default was changed.
|
||||||
|
The performance schema [1] defaulted to on for 5.6 and later. This allocates
|
||||||
|
all the memory that would be required to handle max-connections plus several
|
||||||
|
other memory settings. With 5.5 memory was allocated during runtime as needed.
|
||||||
|
|
||||||
|
The charm now makes performance schema configurable and defaults to off (False).
|
||||||
|
With the performance schema turned off memory is allocated when needed during
|
||||||
|
run time. It is important to understand this can lead to run time memory
|
||||||
|
exhaustion if the configuration values are set too high. Consult a MySQL memory
|
||||||
|
calculator [2] to understand the implications of the values.
|
||||||
|
|
||||||
|
Particularly consider the max-connections setting, this value is a balance
|
||||||
|
between connection exhaustion and memory exhaustion. Occasionally connection
|
||||||
|
exhaustion occurs in large production HA clouds with max-connections less than
|
||||||
|
2000. The common practice became to set max-connections unrealistically high
|
||||||
|
near 10k or 20k. In the move to 5.6 on Xenial this became a problem as Percona
|
||||||
|
would fail to start up or behave erratically as memory exhaustion occurred on
|
||||||
|
the host due to performance schema being turned on. Even with the default now
|
||||||
|
turned off this value should be carefully considered against the production
|
||||||
|
requirements and resources available.
|
||||||
|
|
||||||
|
[1] http://dev.mysql.com/doc/relnotes/mysql/5.6/en/news-5-6-6.html#mysqld-5-6-6-performance-schema
|
||||||
|
[2] http://www.mysqlcalculator.com/
|
||||||
|
|
||||||
|
|
||||||
HA/Clustering
|
HA/Clustering
|
||||||
-------------
|
-------------
|
||||||
|
21
config.yaml
21
config.yaml
@ -47,6 +47,13 @@ options:
|
|||||||
Maximum connections to allow. A value of -1 means use the server's
|
Maximum connections to allow. A value of -1 means use the server's
|
||||||
compiled-in default. This is not typically that useful so the
|
compiled-in default. This is not typically that useful so the
|
||||||
charm will configure PXC with a default max-connections value of 600.
|
charm will configure PXC with a default max-connections value of 600.
|
||||||
|
Note: Connections take up memory resources. Either at startup time with
|
||||||
|
performance-schema=True or during run time with performance-schema=False.
|
||||||
|
This value is a balance between connection exhaustion and memory
|
||||||
|
exhaustion.
|
||||||
|
Consult a MySQL memory calculator like http://www.mysqlcalculator.com/ to
|
||||||
|
understand memory resources consumed by connections.
|
||||||
|
See also performance-schema.
|
||||||
wait-timeout:
|
wait-timeout:
|
||||||
type: int
|
type: int
|
||||||
default: -1
|
default: -1
|
||||||
@ -207,3 +214,17 @@ options:
|
|||||||
Sets the expire_logs_days mysql configuration option, which will make
|
Sets the expire_logs_days mysql configuration option, which will make
|
||||||
mysql server automatically remove logs older than configured number of
|
mysql server automatically remove logs older than configured number of
|
||||||
days.
|
days.
|
||||||
|
performance-schema:
|
||||||
|
default: False
|
||||||
|
type: boolean
|
||||||
|
description: |
|
||||||
|
The performance schema attempts to automatically size the values of
|
||||||
|
several of its parameters at server startup if they are not set
|
||||||
|
explicitly. When set to on (True) memory is allocated at startup time.
|
||||||
|
The implications of this is any memory related charm config options such
|
||||||
|
as max-connections and innodb-buffer-pool-size must be explicitly set for
|
||||||
|
the environment percona is running in or percona may fail to start.
|
||||||
|
Default to off (False) at startup time giving 5.5 like behavior. The
|
||||||
|
implication of this is one can set configuration values that could lead
|
||||||
|
to memory exhaustion during run time as memory is not allocated at
|
||||||
|
startup time.
|
||||||
|
@ -150,6 +150,7 @@ def render_config(clustered=False, hosts=None):
|
|||||||
'enable_binlogs': config('enable-binlogs'),
|
'enable_binlogs': config('enable-binlogs'),
|
||||||
'binlogs_max_size': config('binlogs-max-size'),
|
'binlogs_max_size': config('binlogs-max-size'),
|
||||||
'binlogs_expire_days': config('binlogs-expire-days'),
|
'binlogs_expire_days': config('binlogs-expire-days'),
|
||||||
|
'performance_schema': config('performance-schema'),
|
||||||
}
|
}
|
||||||
|
|
||||||
if config('prefer-ipv6'):
|
if config('prefer-ipv6'):
|
||||||
|
@ -114,6 +114,15 @@ wsrep_sst_auth="sstuser:{{ sst_password }}"
|
|||||||
wsrep_provider_options = {{ wsrep_provider_options }}
|
wsrep_provider_options = {{ wsrep_provider_options }}
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
|
||||||
|
#
|
||||||
|
# * Performance Schema
|
||||||
|
#
|
||||||
|
{% if performance_schema -%}
|
||||||
|
performance_schema=On
|
||||||
|
{% else -%}
|
||||||
|
performance_schema=Off
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
#
|
#
|
||||||
# * IPv6 SST configuration
|
# * IPv6 SST configuration
|
||||||
#
|
#
|
||||||
|
@ -1,6 +1,7 @@
|
|||||||
import amulet
|
import amulet
|
||||||
import re
|
import re
|
||||||
import os
|
import os
|
||||||
|
import socket
|
||||||
import time
|
import time
|
||||||
import telnetlib
|
import telnetlib
|
||||||
import yaml
|
import yaml
|
||||||
@ -41,7 +42,8 @@ class BasicDeployment(OpenStackAmuletDeployment):
|
|||||||
compatible with the local charm (e.g. stable or next).
|
compatible with the local charm (e.g. stable or next).
|
||||||
"""
|
"""
|
||||||
this_service = {'name': 'percona-cluster',
|
this_service = {'name': 'percona-cluster',
|
||||||
'units': self.units}
|
'units': self.units,
|
||||||
|
'constraints': {'mem': '3072M'}}
|
||||||
other_services = []
|
other_services = []
|
||||||
if self.units > 1:
|
if self.units > 1:
|
||||||
other_services.append({'name': 'hacluster'})
|
other_services.append({'name': 'hacluster'})
|
||||||
@ -185,10 +187,10 @@ class BasicDeployment(OpenStackAmuletDeployment):
|
|||||||
try:
|
try:
|
||||||
telnetlib.Telnet(addr, port)
|
telnetlib.Telnet(addr, port)
|
||||||
return True
|
return True
|
||||||
except TimeoutError: # noqa this exception only available in py3
|
except socket.error as e:
|
||||||
|
if e.errno == 113:
|
||||||
print("ERROR: could not connect to %s:%s" % (addr, port))
|
print("ERROR: could not connect to %s:%s" % (addr, port))
|
||||||
return False
|
if e.errno == 111:
|
||||||
except ConnectionRefusedError: # noqa - also only in py3
|
|
||||||
print("ERROR: connection refused connecting to %s:%s" % (addr,
|
print("ERROR: connection refused connecting to %s:%s" % (addr,
|
||||||
port))
|
port))
|
||||||
return False
|
return False
|
||||||
|
Loading…
Reference in New Issue
Block a user