411852892d
There's a potential deadlock scenario in PowerMax's masking.py
"do_move_volume_between_storage_groups" method.
The method uses 2 locks, one for the source Storage Group and another
for the destination Storage Group, and it could happen that if 2
requests going in opposite directions are received simultaneously their
first lock acquisition interleaves resulting in a deadlock situation.
@coordination.synchronized(
"emc-sg-{source_storagegroup_name}-{serial_number}")
@coordination.synchronized(
"emc-sg-{target_storagegroup_name}-{serial_number}")
def do_move_volume_between_storage_groups(
serial_number, source_storage_group_name,
target_storage_group_name):
The scenario would be like this:
- User requests an instance migration from A to B
- User requests an instance migration from B to A
- Driver acquires the first lock for A-to-B for example something like
cinder-emc-sg-SGA-###
- Driver acquires the first lock for B-to-A for example something like
cinder-emc-sgSGB-###
The deadlock happens because A-to-B waits forever for the lock held by
the B-to-A operation, which in turn cannot proceed because it’s waiting
for lock held by A-to-B.
This patch fixes it using the new coordination.synchronized
functionality that ensures that a series of locks are always acquired in
the same order, preventing deadlocks.
Closes-Bug: #1980870
Change-Id: I7eda4645575cfaedcf45d73ab3a215976d3fac3a
OpenStack Cinder
OpenStack Cinder is a storage service for an open cloud computing service.
You can learn more about Cinder at:
Getting Started
If you'd like to run from the master branch, you can clone the git repo:
git clone https://opendev.org/openstack/cinder
If you'd like to contribute, please see the information in CONTRIBUTING.rst
You can raise bugs on Launchpad