openstack/neutron
Code Issues Proposed changes
neutron/doc/source/devref/db_transient_failure_injection.rst
Kevin Benton 1e1e7a842f Add Loki service plugin for optional DB havoc 
This adds a service plugin to introduce random delays
and deadlocks to DB operations to make it easier for
us to see that retry decorators are correctly applied
and race conditions are handled.

Change-Id: I8e283c1b53165faee548d26b3560a2c883dfb977
2017-02-03 05:12:33 -08:00

1.6 KiB
Raw Blame History

Transient DB Failure Injection

Neutron has a service plugin to inject random delays and Deadlock exceptions into normal Neutron operations. The service plugin is called 'Loki' and is located under neutron.services.loki.loki_plugin.

To enable the plugin, just add 'loki' to the list of service_plugins in your neutron-server neutron.conf file.

The plugin will inject a Deadlock exception on database flushes with a 1/50 probability and a delay of 1 second with a 1/200 probability when SQLAlchemy objects are loaded into the persistent state from the DB. The goal is to ensure the code is tolerant of these transient delays/failures that will be experienced in busy production (and Galera) systems.