Browse Source

Add SR-IOV docs

Change-Id: I2a8c11617f6cc2078aac6548ca972e65965f05c2
Elena Ezhova 2 years ago
parent
commit
7ac6307fe5
2 changed files with 278 additions and 0 deletions
  1. 277
    0
      doc/source/enable_sriov.rst
  2. 1
    0
      doc/source/index.rst

+ 277
- 0
doc/source/enable_sriov.rst View File

@@ -0,0 +1,277 @@
1
+.. _enable_sriov:
2
+
3
+============
4
+SR-IOV guide
5
+============
6
+
7
+This guide provides an instruction for enabling SR-IOV functionality in Fuel CCP.
8
+
9
+Introduction
10
+============
11
+
12
+The SR-IOV specification defines a standardized mechanism to virtualize PCIe devices. This mechanism can virtualize
13
+a single PCIe Ethernet controller to appear as multiple PCIe devices. Each device can be directly assigned to
14
+an instance, bypassing the hypervisor and virtual switch layer. As a result, users are able to achieve low latency and
15
+near-line wire speed.
16
+
17
+The following terms are used throughout this document:
18
+
19
+====  ======================================================================================
20
+Term  Definition
21
+====  ======================================================================================
22
+PF    Physical Function. The physical Ethernet controller that supports SR-IOV.
23
+VF    Virtual Function. The virtual PCIe device created from a physical Ethernet controller.
24
+====  ======================================================================================
25
+
26
+Prerequirements
27
+---------------
28
+
29
+1. Ensure that a host has a SR-IOV capable device. One way of identifying whether a device supports SR-IOV is to check
30
+for an SR-IOV capability in the device configuration. The device configuration also contains the number of VFs
31
+the device can support.  The example below shows a simple test to determine if the device located at the bus, device,
32
+and function number 1:00.0 can support SR-IOV.
33
+
34
+::
35
+
36
+    # lspci -vvv -s 02:00.0 | grep -A 9 SR-IOV
37
+        Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
38
+                IOVCap: Migration-, Interrupt Message Number: 000
39
+                IOVCtl: Enable+ Migration- Interrupt- MSE+ ARIHierarchy+
40
+                IOVSta: Migration-
41
+                Initial VFs: 32, Total VFs: 32, Number of VFs: 7, Function Dependency Link: 00
42
+                VF offset: 16, stride: 1, Device ID: 154c
43
+                Supported Page Size: 00000553, System Page Size: 00000001
44
+                Region 0: Memory at 0000000090400000 (64-bit, prefetchable)
45
+                Region 3: Memory at 0000000092c20000 (64-bit, prefetchable)
46
+                VF Migration: offset: 00000000, BIR: 0
47
+
48
+2. Enable IOMMU in Linux by adding `intel_iommu=on` to the kernel parameters, for example, using GRUB.
49
+
50
+3. Bring up the PF.
51
+
52
+::
53
+
54
+    # ip l set dev ens2f1 up
55
+
56
+4. Allocate the VFs, for example via the PCI SYS interface:
57
+
58
+::
59
+
60
+    # echo '7' > /sys/class/net/ens2f1/device/sriov_numvfs
61
+
62
+5. Verify that the VFs have been created.
63
+
64
+::
65
+
66
+    # ip l show ens2f1
67
+    5: ens2f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
68
+    link/ether 0c:c4:7a:bd:42:ac brd ff:ff:ff:ff:ff:ff
69
+    vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
70
+    vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
71
+    vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
72
+    vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
73
+    vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
74
+    vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
75
+    vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
76
+
77
+
78
+Deploy CCP with SR-IOV
79
+======================
80
+
81
+Neutron
82
+-------
83
+
84
+In OpenStack SR-IOV doesn't work with VxLAN tunneling, that is why it is required to enable either VLAN of
85
+flat tenant network type in the `configs.neutron` section of the CCP configuration file:
86
+
87
+::
88
+
89
+    ml2:
90
+      tenant_network_types:
91
+        - "vlan"
92
+
93
+All Neutron SR-IOV specific parameters are located in the `configs.neutron.sriov` section. Example configuration:
94
+
95
+::
96
+
97
+    sriov:
98
+      enabled: true
99
+      devices:
100
+        - name: "ens2f1"
101
+          physnets:
102
+            - "physnet1"
103
+          exclude_vfs:
104
+            - 0000:02:00.2
105
+            - 0000:02:00.3
106
+
107
+* `enabled` - Boolean. Enables and disables the SR-IOV in Neutron, `false` by default.
108
+
109
+* `devices` - List. A node-specific list of SR-IOV devices. Each element of the list has 2 mandatory fields: `name` and `physnets`.
110
+    * `name` is a name of the SR-IOV interface.
111
+    * `physnets` is a list of of names of physical networks a given device maps to.
112
+    * If `exclude_vfs` is omitted all the VFs associated with a given device may be configured by the agent. To exclude specific VFs, add them to the `exclude_devices` parameter as shown above.
113
+
114
+A new role should be added to compute nodes: `neutron-sriov-nic-agent`.
115
+
116
+Nova
117
+----
118
+
119
+All Nova SR-IOV specific parameters are located in the `configs.nova.sriov` section. Example configuration:
120
+
121
+::
122
+
123
+    sriov:
124
+      enabled: true
125
+      pci_alias:
126
+        - name: "82599ES"
127
+          product_id: "10fb"
128
+          vendor_id: "8086"
129
+        - name: "X710"
130
+          product_id: "1572"
131
+          vendor_id: "8086"
132
+      pci_passthrough_whitelist:
133
+         - devname: "ens2f1"
134
+           physical_network: "physnet1"
135
+
136
+* `enabled` - Boolean. Enables and disables the SR-IOV in Nova, `false` by default.
137
+
138
+* `pci_alias` - List, optional. An alias for a PCI passthrough device requirement. This allows users to specify the alias in the
139
+extra_spec for a flavor, without needing to repeat all the PCI property requirements.
140
+
141
+* `pci_passthrough_whitelist` - List. White list of PCI devices available to VMs.
142
+    * `devname` is a name of the SR-IOV interface.
143
+    * `physical_network` - name of a physical network to map a device to.
144
+
145
+Additionally it is required to add `PciPassthroughFilter` to the list of enable filters in Nova scheduler:
146
+
147
+::
148
+
149
+   scheduler:
150
+     enabled_filters:
151
+       - RetryFilter
152
+       - AvailabilityZoneFilter
153
+       - RamFilter
154
+       - DiskFilter
155
+       - ComputeFilter
156
+       - ComputeCapabilitiesFilter
157
+       - ImagePropertiesFilter
158
+       - ServerGroupAntiAffinityFilter
159
+       - ServerGroupAffinityFilter
160
+       - SameHostFilter
161
+       - DifferentHostFilter
162
+       - PciPassthroughFilter
163
+
164
+Sample CCP configuration
165
+------------------------
166
+::
167
+
168
+    services:
169
+      database:
170
+        service_def: galera
171
+      rpc:
172
+        service_def: rabbitmq
173
+      notifications:
174
+        service_def: rabbitmq
175
+    nodes:
176
+      node1:
177
+        roles:
178
+          - db
179
+          - messaging
180
+          - controller
181
+          - openvswitch
182
+      node[2-3]:
183
+        roles:
184
+          - db
185
+          - messaging
186
+          - compute
187
+          - openvswitch
188
+    roles:
189
+      db:
190
+        - database
191
+      messaging:
192
+        - rpc
193
+        - notifications
194
+      controller:
195
+        - etcd
196
+        - glance-api
197
+        - glance-registry
198
+        - heat-api-cfn
199
+        - heat-api
200
+        - heat-engine
201
+        - horizon
202
+        - keystone
203
+        - memcached
204
+        - neutron-dhcp-agent
205
+        - neutron-l3-agent
206
+        - neutron-metadata-agent
207
+        - neutron-server
208
+        - nova-api
209
+        - nova-conductor
210
+        - nova-consoleauth
211
+        - nova-novncproxy
212
+        - nova-scheduler
213
+      compute:
214
+        - neutron-sriov-nic-agent
215
+        - nova-compute
216
+        - nova-libvirt
217
+      openvswitch:
218
+        - neutron-openvswitch-agent
219
+        - openvswitch-db
220
+        - openvswitch-vswitchd
221
+    configs:
222
+      private_interface: ens1f0
223
+      neutron:
224
+        physnets:
225
+          - name: "physnet1"
226
+            bridge_name: "br-ex"
227
+            interface: "ens1f1"
228
+            flat: false
229
+            vlan_range: "50:1030"
230
+        ml2:
231
+          tenant_network_types:
232
+            - "vlan"
233
+        sriov:
234
+          enabled: true
235
+          devices:
236
+            - name: "ens2f1"
237
+              physnets:
238
+                - "physnet1"
239
+              exclude_vfs:
240
+                - 0000:02:00.2
241
+                - 0000:02:00.3
242
+      nova:
243
+        sriov:
244
+          enabled: true
245
+          pci_alias:
246
+            - name: "82599ES"
247
+              product_id: "10fb"
248
+              vendor_id: "8086"
249
+            - name: "X710"
250
+              product_id: "1572"
251
+              vendor_id: "8086"
252
+          pci_passthrough_whitelist:
253
+             - devname: "ens2f1"
254
+               physical_network: "physnet1"
255
+        scheduler:
256
+          enabled_filters:
257
+            - RetryFilter
258
+            - AvailabilityZoneFilter
259
+            - RamFilter
260
+            - DiskFilter
261
+            - ComputeFilter
262
+            - ComputeCapabilitiesFilter
263
+            - ImagePropertiesFilter
264
+            - ServerGroupAntiAffinityFilter
265
+            - ServerGroupAffinityFilter
266
+            - SameHostFilter
267
+            - DifferentHostFilter
268
+            - PciPassthroughFilter
269
+
270
+
271
+Known limitations
272
+=================
273
+
274
+* When using Quality of Service (QoS), `max_burst_kbps` (burst over `max_kbps`) is not supported. In addition, `max_kbps` is rounded to Mbps.
275
+* Security groups are not supported when using SR-IOV, thus, the firewall driver is disabled.
276
+* SR-IOV is not integrated into the OpenStack Dashboard (horizon). Users must use the CLI or API to configure SR-IOV interfaces.
277
+* Live migration is not supported for instances with SR-IOV ports.

+ 1
- 0
doc/source/index.rst View File

@@ -29,6 +29,7 @@ Advanced topics
29 29
    galera
30 30
    ceph
31 31
    ceph_cluster
32
+   enable_sriov
32 33
    using_calico_instead_of_ovs
33 34
    using_odl_instead_of_ovs
34 35
    ironic

Loading…
Cancel
Save