1c7b51b248
Change-Id: If95ad33afb609efd64bf7e1c9bfe289c35d49c2c Signed-off-by: Ngairangbam Mili <ngairangbam.mili@windriver.com>
18 KiB
18 KiB
Rook Ceph OSD Management
This section describes the steps to add, remove, and replace in Rook Container-based Ceph.
Add OSDs
Obtain the of the target disk(s).
[sysadmin@controller-1 ~(keystone_admin)]$ system host-disk-list compute-0 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+--------------------+--------------------------------------------+ | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_id | device_path | | | de | num | type | gib | gib | | | | +--------------------------------------+-----------+---------+---------+-------+------------+--------------+--------------------+--------------------------------------------+ | bc48b2ba-068d-4fc4-8c5a-f75cbd26eee4 | /dev/sda | 2048 | HDD | 292. | 0.0 | Undetermined | VB734fe32a- | /dev/disk/by-path/pci-0000:00:0d.0-ata-1.0 | | | | | | 968 | | | 9839f8bf | | | | | | | | | | | | | 1aa1e19b-474a-46ef-9877-12afe2759046 | /dev/sdb | 2064 | HDD | 9.765 | 0.0 | Undetermined | VB29b38d6e- | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0 | | | | | | | | | 28b7e2ae | | | | | | | | | | | | | ab35233a-9952-4437-87ed-80cc4d6817d7 | /dev/sdc | 2080 | HDD | 5.859 | 5.855 | Undetermined | VB816fb0b6-2ace762 | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0 | | | | | | | | | a | | | | | | | | | | | | | ed4c51a7-372f-4460-915d-43049cf23dd5 | /dev/sdd | 2096 | HDD | 5.859 | 5.855 | Undetermined | VBc059f8a9-953642e | /dev/disk/by-path/pci-0000:00:0d.0-ata-4.0 | | | | | | | | | e | | | | | | | | | | | | | 68f4ec86-3d3c-41e0-8356-ef75275f9c88 | /dev/sde | 2112 | HDD | 5.859 | 5.855 | Undetermined | VB627293e5-cdeb561 | /dev/disk/by-path/pci-0000:00:0d.0-ata-5.0 | | | | | | | | | 4 | | | | | | | | | | | | | 65e2238c-a355-460e-ab08-8e5d6ccd97aa | /dev/sdf | 2128 | HDD | 5.859 | 5.855 | Undetermined | VB93e0fe15-45d4840 | /dev/disk/by-path/pci-0000:00:0d.0-ata-6.0 | | | | | | | | | d | | | | | | | | | | | | +--------------------------------------+-----------+---------+---------+-------+------------+--------------+--------------------+--------------------------------------------+Add the target disk(s) as .
[sysadmin@controller-1 ~(keystone_admin)]$ system host-stor-add compute-0 ed4c51a7-372f-4460-915d-43049cf23dd5 [sysadmin@controller-1 ~(keystone_admin)]$ system host-stor-add compute-0 ab35233a-9952-4437-87ed-80cc4d6817d7At this point, the will be in the configuring-with-app state.
[sysadmin@controller-1 ~(keystone_admin)]$ system host-stor-list compute-0 +--------------------------------------+----------+-------+----------------------+--------------------------------------+-----------------------------+------------+----------+-----------+ | uuid | function | osdid | state | idisk_uuid | journal_path | journal_no | journal_ | tier_name | | | | | | | | de | size_gib | | +--------------------------------------+----------+-------+----------------------+--------------------------------------+-----------------------------+------------+----------+-----------+ | 4c612b3b-93f1-42b1-9f65-7ebd0cb3d397 | osd | None | configuring-with-app | ed4c51a7-372f-4460-915d-43049cf23dd5 | /dev/disk/by-path/pci-0000: | /dev/sdd2 | 1 | storage | | | | | | | 00:0d.0-ata-4.0-part2 | | | | | | | | | | | | | | | 61e6566b-52b4-4ba8-b8db-e31b092523a6 | osd | None | configuring-with-app | ab35233a-9952-4437-87ed-80cc4d6817d7 | /dev/disk/by-path/pci-0000: | /dev/sdc2 | 1 | storage | | | | | | | 00:0d.0-ata-3.0-part2 | | | | | | | | | | | | | | +--------------------------------------+----------+-------+----------------------+--------------------------------------+-----------------------------+------------+----------+-----------+Apply rook-ceph application and wait for the application to become applied.
[sysadmin@controller-1 ~(keystone_admin)]$ system application-apply rook-cephThe rook-ceph application may take several minutes to apply the changes and provision the before reaching the applied state.
Use the
system application-show rook-cephcommand to monitor progress.List the and verify that they are now in the configured state.
[sysadmin@controller-1 ~(keystone_admin)]$ system host-stor-list compute-0 +--------------------------------------+----------+-------+------------+--------------------------------------+-----------------------------+------------+----------------+-----------+ | uuid | function | osdid | state | idisk_uuid | journal_path | journal_no | journal_size_g | tier_name | | | | | | | | de | ib | | +--------------------------------------+----------+-------+------------+--------------------------------------+-----------------------------+------------+----------------+-----------+ | 4c612b3b-93f1-42b1-9f65-7ebd0cb3d397 | osd | 0 | configured | ed4c51a7-372f-4460-915d-43049cf23dd5 | /dev/disk/by-path/pci-0000: | /dev/sdd2 | 1 | storage | | | | | | | 00:0d.0-ata-4.0-part2 | | | | | | | | | | | | | | | 61e6566b-52b4-4ba8-b8db-e31b092523a6 | osd | 1 | configured | ab35233a-9952-4437-87ed-80cc4d6817d7 | /dev/disk/by-path/pci-0000: | /dev/sdc2 | 1 | storage | | | | | | | 00:0d.0-ata-3.0-part2 | | | | | | | | | | | | | | +--------------------------------------+----------+-------+------------+--------------------------------------+-----------------------------+------------+----------------+-----------+
Remove OSDs
Obtain the of the to be removed.
[sysadmin@controller-1 ~(keystone_admin)]$ system host-stor-list compute-0 +--------------------------------------+----------+-------+------------+--------------------------------------+-----------------------------+------------+----------------+-----------+ | uuid | function | osdid | state | idisk_uuid | journal_path | journal_no | journal_size_g | tier_name | | | | | | | | de | ib | | +--------------------------------------+----------+-------+------------+--------------------------------------+-----------------------------+------------+----------------+-----------+ | 4c612b3b-93f1-42b1-9f65-7ebd0cb3d397 | osd | 0 | configured | ed4c51a7-372f-4460-915d-43049cf23dd5 | /dev/disk/by-path/pci-0000: | /dev/sdd2 | 1 | storage | | | | | | | 00:0d.0-ata-4.0-part2 | | | | | | | | | | | | | | | 61e6566b-52b4-4ba8-b8db-e31b092523a6 | osd | 1 | configured | ab35233a-9952-4437-87ed-80cc4d6817d7 | /dev/disk/by-path/pci-0000: | /dev/sdc2 | 1 | storage | | | | | | | 00:0d.0-ata-3.0-part2 | | | | | | | | | | | | | | +--------------------------------------+----------+-------+------------+--------------------------------------+-----------------------------+------------+----------------+-----------+Mark the for removal.
[sysadmin@controller-1 ~(keystone_admin)]$ system host-stor-delete 4c612b3b-93f1-42b1-9f65-7ebd0cb3d397At this point, the will be in the deleting-with-app state.
[sysadmin@controller-1 ~(keystone_admin)]$ system host-stor-list compute-0 +--------------------------------------+----------+-------+-------------------+--------------------------------------+-----------------------------+------------+----------+-----------+ | uuid | function | osdid | state | idisk_uuid | journal_path | journal_no | journal_ | tier_name | | | | | | | | de | size_gib | | +--------------------------------------+----------+-------+-------------------+--------------------------------------+-----------------------------+------------+----------+-----------+ | 4c612b3b-93f1-42b1-9f65-7ebd0cb3d397 | osd | 0 | deleting-with-app | ed4c51a7-372f-4460-915d-43049cf23dd5 | /dev/disk/by-path/pci-0000: | /dev/sdd2 | 1 | storage | | | | | | | 00:0d.0-ata-4.0-part2 | | | | | | | | | | | | | | | 61e6566b-52b4-4ba8-b8db-e31b092523a6 | osd | 1 | configured | ab35233a-9952-4437-87ed-80cc4d6817d7 | /dev/disk/by-path/pci-0000: | /dev/sdc2 | 1 | storage | | | | | | | 00:0d.0-ata-3.0-part2 | | | | | | | | | | | | | | +--------------------------------------+----------+-------+-------------------+--------------------------------------+-----------------------------+------------+----------+-----------+Apply rook-ceph application and wait for the application to become applied.
[sysadmin@controller-1 ~(keystone_admin)]$ system application-apply rook-cephThe rook-ceph application may take several minutes to apply the changes and provision the before reaching the applied state.
Use the
system application-show rook-cephcommand to monitor progress.Wait until the are removed.
The will be removed asynchronously after rook-ceph reaches the applied state. Then, once the entry disappears from system host-stor-list, the are successfully removed.
[sysadmin@controller-1 ~(keystone_admin)]$ system host-stor-list 3 +--------------------------------------+----------+-------+------------+--------------------------------------+-----------------------------+------------+----------------+-----------+ | uuid | function | osdid | state | idisk_uuid | journal_path | journal_no | journal_size_g | tier_name | | | | | | | | de | ib | | +--------------------------------------+----------+-------+------------+--------------------------------------+-----------------------------+------------+----------------+-----------+ | 61e6566b-52b4-4ba8-b8db-e31b092523a6 | osd | 1 | configured | ab35233a-9952-4437-87ed-80cc4d6817d7 | /dev/disk/by-path/pci-0000: | /dev/sdc2 | 1 | storage | | | | | | | 00:0d.0-ata-3.0-part2 | | | | | | | | | | | | | | +--------------------------------------+----------+-------+------------+--------------------------------------+-----------------------------+------------+----------------+-----------+
Replace OSDs
Repeat the following steps for each target host, one at a time so that it does not impact Ceph availability.
Follow the steps in the
Remove OSDs <removeosd>section to remove the target from the host.Note
If after several minutes the entry is reverted to the configured state, then the removal is considered unsafe and is aborted to avoid data loss.
You can proceed using
system host-stor-delete --force <uuid>. This will skip data integrity checks and will forcefully remove the . However, this may cause loss of data. Thus, use this only when you are sure that there is enough replicas of data. Otherwise, before using the --force option, always attempt the standard deletion first.Lock the host.
[sysadmin@controller-1 ~(keystone_admin)]$ system host-lock <host>After the host is locked, power off the host.
[sysadmin@controller-1 ~(keystone_admin)]$ system host-power-off <host>Replace the disk(s) while the host is powered off.
Power on the host and wait for the host to become online.
Unlock the host.
[sysadmin@controller-1 ~(keystone_admin)]$ system host-unlock <host>Wait for the host to become unlocked/available.
Follow the steps in
Add OSDs <addosd>to add the new replaced disk(s) as .