Browse Source

Merge "Cinder volumes backup using os-brick from Freezer"

changes/91/446591/1
Jenkins 2 years ago
parent
commit
710167e17c
1 changed files with 284 additions and 0 deletions
  1. 284
    0
      specs/ocata/approved/cinder_os_brick.rst

+ 284
- 0
specs/ocata/approved/cinder_os_brick.rst View File

@@ -0,0 +1,284 @@
1
+..
2
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+ License.
4
+
5
+ http://creativecommons.org/licenses/by/3.0/legalcode
6
+
7
+=============================================
8
+Freezer Cinder Volumes backup using OS Bricks
9
+=============================================
10
+
11
+* https://blueprints.launchpad.net/freezer/+spec/cinder-osbrick
12
+
13
+Provide efficient way to backup Cinder Volumes leveraging os_bricks
14
+
15
+Problem description
16
+===================
17
+
18
+Currently Freezer provides basic features to execute Cinder volumes backup.
19
+The current approach present significant challenges,
20
+due mainly to the difficulty of downloading Cinder Volumes without passing
21
+through Glance. This can be an issue for time and scalability reasons,
22
+(i.e. volumes of few hundreds GB size, potential error probability increase,
23
+as more services are part of the process, unailability of cinder-backup)
24
+
25
+Use Cases
26
+---------
27
+
28
+* Users that want to backup cinder volumes.
29
+
30
+* Store backed up volumes in a different storage media than Swift.
31
+  This is important for disaster recovery purpose, as it should be
32
+  possible to restore the volume even if the swift or other services are
33
+  down in the original OpenStack deployment.
34
+
35
+* OpenStack distributions deployed without cinder-backup module.
36
+
37
+* Provide a more efficient way of executing incrementals backup.
38
+
39
+* Avoid uploading volumes image to Glance to be processed.
40
+
41
+* Volumes can be backed up while attached or detached (hot and cold).
42
+
43
+- Hot backup will be provide a crach consistent backup and the data  present
44
+  in the volumes can be accesses at all times during backup
45
+- If Cold backup is executed, the Volume is detached first,
46
+  then the backup is executed.
47
+
48
+
49
+Proposed change
50
+===============
51
+
52
+Implement in the freezer-agent a new engine called cinder-osbrick.
53
+
54
+The new engine cinder-osbrick execute backup and restore related
55
+operations direclty on the Volumes, without passing through Glance API.
56
+
57
+The freezer-agent needs to back up a single volume, all the volumes
58
+owned by the tenant or all volumes from all tenants (admin).
59
+Volumes backup and restore can happen in parallel (i.e. 10 Volumes
60
+simultaneously can be backup or restore)
61
+
62
+
63
+Technical details
64
+-----------------
65
+
66
+Openstack provide the os_brick library to attach volumes:
67
+
68
+* https://github.com/openstack/os-brick
69
+
70
+It mainly provides the following features:
71
+
72
+* Volumes discovery
73
+* Volumes attach
74
+* Volumes removal
75
+
76
+Related docs:
77
+
78
+* http://docs.openstack.org/developer/os-brick/api/index.html
79
+* http://docs.openstack.org/developer/os-brick/tutorial.html
80
+
81
+The python client module that could be used is brick-cinderclient-ex:
82
+
83
+* https://github.com/openstack/python-brick-cinderclient-ext
84
+
85
+It is preferrable to implement the Volumes related operations from cinder
86
+in python, rather wrapping around any possible related os-brick command.
87
+
88
+* The freezer-scheduler and the freezer-agent needs to support,
89
+  in the json and ini config file respectively, engine specific settings.
90
+
91
+
92
+Backup workflow with osbrick:
93
+-----------------------------
94
+
95
+* freezer-agent workflow:
96
+
97
+(Common steps)
98
+
99
+single-vol-backup:
100
+ - Backup any available metadata of the Volume
101
+ - A Snapshot is execute on the volume (--force if volume is attached)
102
+ - Snapshot is converted to Volume, in oder to be mounted using osbrick
103
+ - The new Volume is attached using os_brick. The Volume can be attached
104
+   using iSCSI, Local or FC according the information provided by
105
+   os_brick about the volume.
106
+ - The new Volume is mounted on the node where the freezer-agent is executing
107
+ - The freezer-agent will execute a backup of the volume content, starting
108
+   from the volume mount (i.e. volume root /)
109
+ - Every single file in the volume is backed up.
110
+ - If the execution is part of an incremental backup, each file/block is
111
+   compared against the previous execution.
112
+ - Data can be storage on each supported freezer storage backend
113
+ - When finished, the new Volume is detached
114
+ - Once detached, the new volume is removed
115
+
116
+Backup of a Single Volume:
117
+ 1) The freezer-agent take the volume id as input param (either from ini
118
+    file or json file provided to the scheduler):
119
+ 2) single-vol-backup from Common steps
120
+
121
+Backup of all Volumes owned by a tenant:
122
+ 1) freezer-agent discover all the volumes owned by the tenant from Cinder API
123
+ 2) Iterate over each Volume
124
+ 3) single-vol-backup from Common steps
125
+
126
+Backup of all Volumes (admin):
127
+ 1) freezer-agent get the list of all Volumes available from Cinder API
128
+ 2) Iterate over each volume
129
+ 3) single-vol-backup from Common steps
130
+
131
+Backup of all volumes part of a Consistency Group
132
+ 1) get list of all volumes from the Consistency Group. It can be provided
133
+    as a single element id or a list of elements comma separated:
134
+ 2) freezer-agent get the list of all Volumes available from Cinder API
135
+ 3) Iterate over each volume
136
+ 4) single-vol-backup from Common steps
137
+
138
+
139
+Restore workflow with osbrick
140
+-----------------------------
141
+
142
+* freezer-agent workflow:
143
+
144
+(Common steps)
145
+
146
+single-vol-restore:
147
+  - Get the original Volume metadata
148
+  - Check if the volume id exists
149
+  - If the same volume id exists
150
+
151
+    + snapshot the volume
152
+    + convert from snap to volume
153
+    + attach the volume
154
+    + mount the volume
155
+    + restore the backup data in the volume filesystem
156
+    + if meta-override option is provided, the volume metadata from backup
157
+      is applied to the current Volume meta
158
+  - If the volume id does not exist
159
+    + Create a new Volume with the same metadata from backup
160
+    + attach the volume with os-brick
161
+    + mount the volume
162
+    + restore the backup data in the volume filesystem
163
+  - unmount
164
+  - deattach the volum
165
+  - if remove_old_vol is provided, any existing volume not matching with the
166
+    new ones will be removed (Dangerous Option)
167
+
168
+Restote of a single volume:
169
+ 1) The freezer-agent take the volume id as input param (either from ini
170
+    file or json file provided to the scheduler):
171
+ 2) single-vol-restore from Common steps
172
+
173
+Restore of all volumes owned by a tenant:
174
+ 1) freezer-agent discover all the volumes owned by the tenant from Cinder API
175
+ 2) Iterate over each volume
176
+ 3) single-vol-restore from Common steps
177
+
178
+Restore of all volumes from all tenants (admin):
179
+ 1) freezer-agent get the list of all Volumes available from Cinder API
180
+ 2) Iterate over each volume
181
+ 3) single-vol-restore from Common steps
182
+
183
+Restore of all volumes part of a Consistency Group
184
+ 1) get list of all volumes from the Consistency Group. It can be provided as
185
+    a single element id or a list of elements comma separated:
186
+ 2) Iterate over each volume
187
+ 3) single-vol-restore from Common steps
188
+
189
+Data model impact
190
+-----------------
191
+* new engine in the db
192
+* DB model for single, all tenant, tenant owned volumes and consistency groups
193
+
194
+New Options to be added:
195
+------------------------
196
+* engine-os-brick
197
+* recreate-vol-on-error
198
+* meta-override
199
+* consistency groups [id]
200
+* all_tenants
201
+* all_tenant_volumes
202
+* single_volume_id
203
+* remove_old_vol
204
+
205
+Alternatives
206
+------------
207
+
208
+
209
+Impacts
210
+-------
211
+* freezer-agent
212
+* freezer-api
213
+* freezer-web-ui
214
+
215
+REST API impact
216
+-----------------------
217
+
218
+* API needs to support this new engine
219
+
220
+Security impact
221
+---------------------
222
+
223
+None
224
+
225
+Notifications impact
226
+---------------------------
227
+
228
+TBD.
229
+
230
+Other end user impact
231
+------------------------------
232
+
233
+None. TBD.
234
+
235
+Performance Impact
236
+------------------
237
+
238
+None.
239
+
240
+Other deployer impact
241
+------------------------------
242
+
243
+
244
+Developer impact
245
+------------------------
246
+
247
+
248
+Implementation
249
+==============
250
+
251
+Assignee(s)
252
+-----------------
253
+
254
+Primary assignee:
255
+
256
+Other contributors:
257
+  daemontool
258
+
259
+Work Items
260
+----------
261
+
262
+
263
+Dependencies
264
+============
265
+
266
+
267
+Testing
268
+=======
269
+
270
+TBD.
271
+
272
+Documentation Impact
273
+====================
274
+
275
+
276
+* Freezer API installation doc
277
+* Freezer agent docu
278
+* Freezer web ui doc
279
+
280
+References
281
+==========
282
+
283
+.. https://etherpad.openstack.org/p/freezer_cinder-os-brick
284
+

Loading…
Cancel
Save