Browse Source

rbd: Warn if ceph udev rules are not configured

The LUKS encryptor feature expects devices to have a symbolic link that
it can overwrite in order to enable transparent encryption/decryption
for instances [1]. This is generally the case for RBD volumes, as Ceph
uses udev rules [2] to create a '/dev/rbd/{pool}/{device}' ->
'/dev/rbdN' symlink. However, in an environment where udev daemon is not
present or configured correctly, this symlink will never be configured.
This causes things to crash and burn in a rather non-obvious manner when
locally attaching an encrypted RBD volume:

  oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
  Command: cryptsetup luksOpen --key-file=- /dev/rbd/volumes/volume-foo crypt-volume-foo
  Exit code: 4
  Stdout: ''
  Stderr: "Device /dev/rbd/volumes/foo doesn't exist or access denied.\n"

('foo' being a stand-in for a very long 'device-$UUID' name)

The long term fix here is to probably stop relying on the side effects
of these udev rules, i.e. the symlinks, but that is a far more involved
fix that would not be backportable. Instead, for now we simply leave a
breadcrumb for the user, informing them as to what's gone wrong and
encouraging them to look at the bug report for more information.


Change-Id: I2775f55039695c7ec029106c0dafe4d46255b336
Signed-off-by: Stephen Finucane <>
Related-Bug: #1884114
(cherry picked from commit ee34d925ff)
Stephen Finucane 2 months ago
committed by Brian Rosmaita
1 changed files with 23 additions and 9 deletions
  1. +23

+ 23
- 9
os_brick/initiator/connectors/ View File

@@ -186,20 +186,34 @@ class RBDConnector(base.BaseLinuxConnector):
# via the rbd kernel module.
pool, volume = connection_properties['name'].split('/')
rbd_dev_path = RBDConnector.get_rbd_device_name(pool, volume)
if (not os.path.islink(rbd_dev_path) or
not os.path.exists(os.path.realpath(rbd_dev_path))):

if (
not os.path.islink(rbd_dev_path) or
not os.path.exists(os.path.realpath(rbd_dev_path))
cmd = ['rbd', 'map', volume, '--pool', pool]
cmd += self._get_rbd_args(connection_properties)
self._execute(*cmd, root_helper=self._root_helper,
LOG.debug('volume %(vol)s is already mapped to local'
' device %(dev)s',
{'vol': volume,
'dev': os.path.realpath(rbd_dev_path)})

return {'path': rbd_dev_path,
'type': 'block'}
'Volume %(vol)s is already mapped to local device %(dev)s',
{'vol': volume, 'dev': os.path.realpath(rbd_dev_path)}

if (
not os.path.islink(rbd_dev_path) or
not os.path.exists(os.path.realpath(rbd_dev_path))
'Volume %(vol)s has not been mapped to local device '
'%(dev)s; is the udev daemon running and are the '
'ceph-renamer udev rules configured? See bug #1884114 for '
'more information.',
{'vol': volume, 'dev': rbd_dev_path},

return {'path': rbd_dev_path, 'type': 'block'}

rbd_handle = self._get_rbd_handle(connection_properties)
return {'path': rbd_handle}