merge in more networking docs - still a work in progress
This commit is contained in:
root
commit
dc154f98bd
|
|
@ -75,6 +75,8 @@ Networking
|
|||
:maxdepth: 1
|
||||
|
||||
multi.node.install
|
||||
network.vlan.rst
|
||||
network.flat.rst
|
||||
|
||||
|
||||
Advanced Topics
|
||||
|
|
|
|||
|
|
@ -16,74 +16,42 @@
|
|||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
OpenStack Network Overview
|
||||
==========================
|
||||
Networking Overview
|
||||
===================
|
||||
In Nova, users organize their cloud resources in projects. A Nova project consists of a number of VM instances created by a user. For each VM instance, Nova assigns to it a private IP address. (Currently, Nova only supports Linux bridge networking that allows the virtual interfaces to connect to the outside network through the physical interface. Other virtual network technologies, such as Open vSwitch, could be supported in the future.) The Network Controller provides virtual networks to enable compute servers to interact with each other and with the public network.
|
||||
|
||||
Introduction
|
||||
------------
|
||||
..
|
||||
Introduction
|
||||
------------
|
||||
|
||||
Nova consists of seven main components, with the Cloud Controller component representing the global state and interacting with all other components. API Server acts as the Web services front end for the cloud controller. Compute Controller provides compute server resources, and the Object Store component provides storage services. Auth Manager provides authentication and authorization services. Volume Controller provides fast and permanent block-level storage for the comput servers. Network Controller provides virtual networks to enable compute servers to interact with each other and with the public network. Scheduler selects the most suitable compute controller to host an instance.
|
||||
Nova consists of seven main components, with the Cloud Controller component representing the global state and interacting with all other components. API Server acts as the Web services front end for the cloud controller. Compute Controller provides compute server resources, and the Object Store component provides storage services. Auth Manager provides authentication and authorization services. Volume Controller provides fast and permanent block-level storage for the comput servers. Network Controller provides virtual networks to enable compute servers to interact with each other and with the public network. Scheduler selects the most suitable compute controller to host an instance.
|
||||
|
||||
.. todo:: Insert Figure 1 image from "An OpenStack Network Overview" contributed by Citrix
|
||||
.. todo:: Insert Figure 1 image from "An OpenStack Network Overview" contributed by Citrix
|
||||
|
||||
Nova is built on a shared-nothing, messaging-based architecture. All of the major components, that is Compute Controller, Volume Controller, Network Controller, and Object Store can be run on multiple servers. Cloud Controller communicates with Object Store via HTTP (Hyper Text Transfer Protocol), but it communicates with Scheduler, Network Controller, and Volume Controller via AMQP (Advanced Message Queue Protocol). To avoid blocking each component while waiting for a response, Nova uses asynchronous calls, with a call-back that gets triggered when a response is received.
|
||||
Nova is built on a shared-nothing, messaging-based architecture. All of the major components, that is Compute Controller, Volume Controller, Network Controller, and Object Store can be run on multiple servers. Cloud Controller communicates with Object Store via HTTP (Hyper Text Transfer Protocol), but it communicates with Scheduler, Network Controller, and Volume Controller via AMQP (Advanced Message Queue Protocol). To avoid blocking each component while waiting for a response, Nova uses asynchronous calls, with a call-back that gets triggered when a response is received.
|
||||
|
||||
To achieve the shared-nothing property with multiple copies of the same component, Nova keeps all the cloud system state in a distributed data store. Updates to system state are written into this store, using atomic transactions when required. Requests for system state are read out of this store. In limited cases, the read results are cached within controllers for short periods of time (for example, the current list of system users.)
|
||||
To achieve the shared-nothing property with multiple copies of the same component, Nova keeps all the cloud system state in a distributed data store. Updates to system state are written into this store, using atomic transactions when required. Requests for system state are read out of this store. In limited cases, the read results are cached within controllers for short periods of time (for example, the current list of system users.)
|
||||
|
||||
.. note:: The database schema is available on the `OpenStack Wiki <http://wiki.openstack.org/NovaDatabaseSchema>_`.
|
||||
.. note:: The database schema is available on the `OpenStack Wiki <http://wiki.openstack.org/NovaDatabaseSchema>_`.
|
||||
|
||||
Nova Network Strategies
|
||||
-----------------------
|
||||
|
||||
In Nova, users organize their cloud resources in projects. A Nova project consists of a number of VM instances created by a user. For each VM instance, Nova assigns to it a private IP address. (Currently, Nova only supports Linux bridge networking that allows the virtual interfaces to connect to the outside network through the physical interface. Other virtual network technologies, such as Open vSwitch, could be supported in the future.)
|
||||
|
||||
Currently, Nova supports three kinds of networks, implemented in three "Network Manager" types respectively: Flat Network Manager, Flat DHCP Network Manager, and VLAN Network Manager. The three kinds of networks can c-exist in a cloud system. However, the scheduler for selecting the type of network for a given project is not yet implemented. Here is a brief description of each of the different network strategies, with a focus on the VLAN Manager in a separate section.
|
||||
|
||||
Flat Network
|
||||
++++++++++++
|
||||
Read more about Nova network strategies here:
|
||||
|
||||
IP addresses for VM instances are grabbed from a subnet specified by the network administrator, and injected into the image on launch. All instances of the system are attached to the same Linux networking bridge, configured manually by the network administrator both on the network controller hosting the network and on the computer controllers hosting the instances.
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
Flat Network with DHCP
|
||||
++++++++++++++++++++++
|
||||
|
||||
IP addresses for VM instances are grabbed from a subnet specified by the network administrator. Similar to the flat network, a single Linux networking bridge is created and configured manually by the network administrator and used for all instances. A DHCP server is started to pass out IP addresses to VM instances from the specified subnet.
|
||||
|
||||
VLAN Network
|
||||
++++++++++++
|
||||
|
||||
Each project gets its own VLAN, Linux networking bridge, and subnet. The subnets are specified by the network administrator, and are assigned dynamically to a project when required. A DHCP Server is started for each VLAN to pass out IP addresses to VM instances from the subnet assigned to the project. All instances belonging to one project are bridged into the same VLAN for that project. The Linux networking bridges and VLANs are created by Nova when required, described in more detail in Nova VLAN Network Management Implementation.
|
||||
|
||||
Nova VLAN Networks
|
||||
------------------
|
||||
|
||||
Because the flat network and flat DhCP network are simple to understand and yet do not scale well enough for real-world cloud systems, this section focuses on the VLAN network implementation by the VLAN Network Manager.
|
||||
|
||||
In the VLAN network mode, all the VM instances of a project are connected together in a VLAN with the specified private subnet. Each running VM instance is assigned an IP address within the given private subnet.
|
||||
|
||||
.. todo:: Insert Figure 2 from "An OpenStack Network Overview" contributed by Citrix
|
||||
|
||||
While network traffic between VM instances belonging to the same VLAN is always open, Nova can enforce isolation of network traffic between different projects by enforcing one VLAN per project.
|
||||
|
||||
In addition, the network administrator can specify a pool of public IP addresses that users may allocate and then assign to VMs, either at boot or dynamically at run-time. This capability is similar to Amazon's 'elastic IPs'. A public IP address may be associated with a running instances, allowing the VM instance to be accessed from the public network. The public IP addresses are accessible from the network host and NATed to the private IP address of the project.
|
||||
|
||||
.. todo:: Describe how a public IP address could be associated with a project (a VLAN)
|
||||
|
||||
Nova VLAN Network Management Implementation
|
||||
-------------------------------------------
|
||||
|
||||
This section describes the current (November 2010) implementation of the network structure of Nova.
|
||||
|
||||
The network assignment to a project, and IP address assignment to a VM instance, are triggered when a user starts to run a VM instance. When running a VM instance, a user needs to specify a project for the instances, and the security groups (described in Security Groups) when the instance wants to join. If this is the first instance to be created for the project, then Nova (the cloud controller) needs to find a network controller to be the network host for the project; it then sets up a private network by finding an unused VLAN id, an unused subnet, and then the controller assigns them to the project, it also assigns a name to the project's Linux bridge, and allocating a private IP within the project's subnet for the new instance.
|
||||
|
||||
If the instance the user wants to start is not the project's first, a subnet and a VLAN must have already been assigned to the project; therefore the system needs only to find an available IP address within the subnet and assign it to the new starting instance. If there is no private IP available within the subnet, an exception will be raised to the cloud controller, and the VM creation cannot proceed.
|
||||
network.flat.rst
|
||||
network.vlan.rst
|
||||
|
||||
|
||||
.. todo:: insert the name of the Linux bridge, is it always named bridge?
|
||||
Network Management Commands
|
||||
---------------------------
|
||||
|
||||
|
||||
Managing Networks
|
||||
=================
|
||||
Admins and Network Administrators can use the 'nova-manage' command to manage project resources:
|
||||
|
||||
VPN Management
|
||||
~~~~~~~~~~~~~~
|
||||
|
|
|
|||
|
|
@ -0,0 +1,60 @@
|
|||
..
|
||||
Copyright 2010 United States Government as represented by the
|
||||
Administrator of the National Aeronautics and Space Administration.
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
|
||||
Flat Network Mode (Original and Flat)
|
||||
=====================================
|
||||
|
||||
Flat network mode removes most of the complexity of VLAN mode by simply
|
||||
bridging all instance interfaces onto a single network.
|
||||
|
||||
There are two variations of flat mode that differ mostly in how IP addresses
|
||||
are given to instances.
|
||||
|
||||
|
||||
Original Flat Mode
|
||||
------------------
|
||||
IP addresses for VM instances are grabbed from a subnet specified by the network administrator, and injected into the image on launch. All instances of the system are attached to the same Linux networking bridge, configured manually by the network administrator both on the network controller hosting the network and on the computer controllers hosting the instances. To recap:
|
||||
|
||||
* Each compute host creates a single bridge for all instances to use to attach to the external network.
|
||||
* The networking configuration is injected into the instance before it is booted or it is obtained by a guest agent installed in the instance.
|
||||
|
||||
Note that the configuration injection currently only works on linux-style systems that keep networking
|
||||
configuration in /etc/network/interfaces.
|
||||
|
||||
|
||||
Flat DHCP Mode
|
||||
--------------
|
||||
IP addresses for VM instances are grabbed from a subnet specified by the network administrator. Similar to the flat network, a single Linux networking bridge is created and configured manually by the network administrator and used for all instances. A DHCP server is started to pass out IP addresses to VM instances from the specified subnet. To recap:
|
||||
|
||||
* Like flat mode, all instances are attached to a single bridge on the compute node.
|
||||
* In addition a DHCP server is running to configure instances.
|
||||
|
||||
Implementation
|
||||
--------------
|
||||
|
||||
The network nodes do not act as a default gateway in flat mode. Instances
|
||||
are given public IP addresses.
|
||||
|
||||
Compute nodes have iptables/ebtables entries created per project and
|
||||
instance to protect against IP/MAC address spoofing and ARP poisoning.
|
||||
|
||||
|
||||
Examples
|
||||
--------
|
||||
|
||||
.. todo:: add flat network mode configuration examples
|
||||
|
|
@ -0,0 +1,173 @@
|
|||
..
|
||||
Copyright 2010 United States Government as represented by the
|
||||
Administrator of the National Aeronautics and Space Administration.
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
|
||||
VLAN Network Mode
|
||||
=================
|
||||
VLAN Network Mode is the default mode for Nova. It provides a private network
|
||||
segment for each project's instances that can be accessed via a dedicated
|
||||
VPN connection from the Internet.
|
||||
|
||||
In this mode, each project gets its own VLAN, Linux networking bridge, and subnet. The subnets are specified by the network administrator, and are assigned dynamically to a project when required. A DHCP Server is started for each VLAN to pass out IP addresses to VM instances from the subnet assigned to the project. All instances belonging to one project are bridged into the same VLAN for that project. The Linux networking bridges and VLANs are created by Nova when required, described in more detail in Nova VLAN Network Management Implementation.
|
||||
|
||||
..
|
||||
(this text revised above)
|
||||
Because the flat network and flat DhCP network are simple to understand and yet do not scale well enough for real-world cloud systems, this section focuses on the VLAN network implementation by the VLAN Network Manager.
|
||||
|
||||
|
||||
In the VLAN network mode, all the VM instances of a project are connected together in a VLAN with the specified private subnet. Each running VM instance is assigned an IP address within the given private subnet.
|
||||
|
||||
.. todo:: Insert Figure 2 from "An OpenStack Network Overview" contributed by Citrix
|
||||
|
||||
While network traffic between VM instances belonging to the same VLAN is always open, Nova can enforce isolation of network traffic between different projects by enforcing one VLAN per project.
|
||||
|
||||
In addition, the network administrator can specify a pool of public IP addresses that users may allocate and then assign to VMs, either at boot or dynamically at run-time. This capability is similar to Amazon's 'elastic IPs'. A public IP address may be associated with a running instances, allowing the VM instance to be accessed from the public network. The public IP addresses are accessible from the network host and NATed to the private IP address of the project.
|
||||
|
||||
.. todo:: Describe how a public IP address could be associated with a project (a VLAN)
|
||||
|
||||
|
||||
Goals
|
||||
-----
|
||||
|
||||
* each project is in a protected network segment
|
||||
|
||||
* RFC-1918 IP space
|
||||
* public IP via NAT
|
||||
* no default inbound Internet access without public NAT
|
||||
* limited (project-admin controllable) outbound Internet access
|
||||
* limited (project-admin controllable) access to other project segments
|
||||
* all connectivity to instance and cloud API is via VPN into the project segment
|
||||
|
||||
* common DMZ segment for support services (only visible from project segment)
|
||||
|
||||
* metadata
|
||||
* dashboard
|
||||
|
||||
|
||||
Limitations
|
||||
-----------
|
||||
|
||||
* Projects / cluster limited to available VLANs in switching infrastructure
|
||||
* Requires VPN for access to project segment
|
||||
|
||||
|
||||
Implementation
|
||||
--------------
|
||||
Currently Nova segregates project VLANs using 802.1q VLAN tagging in the
|
||||
switching layer. Compute hosts create VLAN-specific interfaces and bridges
|
||||
as required.
|
||||
|
||||
The network nodes act as default gateway for project networks and contain
|
||||
all of the routing and firewall rules implementing security groups. The
|
||||
network node also handles DHCP to provide instance IPs for each project.
|
||||
|
||||
VPN access is provided by running a small instance called CloudPipe
|
||||
on the IP immediately following the gateway IP for each project. The
|
||||
network node maps a dedicated public IP/port to the CloudPipe instance.
|
||||
|
||||
Compute nodes have per-VLAN interfaces and bridges created as required.
|
||||
These do NOT have IP addresses in the host to protect host access.
|
||||
Compute nodes have iptables/ebtables entries created per project and
|
||||
instance to protect against IP/MAC address spoofing and ARP poisoning.
|
||||
|
||||
The network assignment to a project, and IP address assignment to a VM instance, are triggered when a user starts to run a VM instance. When running a VM instance, a user needs to specify a project for the instances, and the security groups (described in Security Groups) when the instance wants to join. If this is the first instance to be created for the project, then Nova (the cloud controller) needs to find a network controller to be the network host for the project; it then sets up a private network by finding an unused VLAN id, an unused subnet, and then the controller assigns them to the project, it also assigns a name to the project's Linux bridge, and allocating a private IP within the project's subnet for the new instance.
|
||||
|
||||
If the instance the user wants to start is not the project's first, a subnet and a VLAN must have already been assigned to the project; therefore the system needs only to find an available IP address within the subnet and assign it to the new starting instance. If there is no private IP available within the subnet, an exception will be raised to the cloud controller, and the VM creation cannot proceed.
|
||||
|
||||
.. todo:: insert the name of the Linux bridge, is it always named bridge?
|
||||
|
||||
External Infrastructure
|
||||
-----------------------
|
||||
|
||||
Nova assumes the following is available:
|
||||
|
||||
* DNS
|
||||
* NTP
|
||||
* Internet connectivity
|
||||
|
||||
|
||||
Example
|
||||
-------
|
||||
|
||||
This example network configuration demonstrates most of the capabilities
|
||||
of VLAN Mode. It splits administrative access to the nodes onto a dedicated
|
||||
management network and uses dedicated network nodes to handle all
|
||||
routing and gateway functions.
|
||||
|
||||
It uses a 10GB network for instance traffic and a 1GB network for management.
|
||||
|
||||
|
||||
Hardware
|
||||
~~~~~~~~
|
||||
|
||||
* All nodes have a minimum of two NICs for management and production.
|
||||
|
||||
* management is 1GB
|
||||
* production is 10GB
|
||||
* add additional NICs for bonding or HA/performance
|
||||
|
||||
* network nodes should have an additional NIC dedicated to public Internet traffic
|
||||
* switch needs to support enough simultaneous VLANs for number of projects
|
||||
* production network configured as 802.1q trunk on switch
|
||||
|
||||
|
||||
Operation
|
||||
~~~~~~~~~
|
||||
|
||||
The network node controls the project network configuration:
|
||||
|
||||
* assigns each project a VLAN and private IP range
|
||||
* starts dnsmasq on project VLAN to serve private IP range
|
||||
* configures iptables on network node for default project access
|
||||
* launches CloudPipe instance and configures iptables access
|
||||
|
||||
When starting an instance the network node:
|
||||
|
||||
* sets up a VLAN interface and bridge on each host as required when an
|
||||
instance is started on that host
|
||||
* assigns private IP to instance
|
||||
* generates MAC address for instance
|
||||
* update dnsmasq with IP/MAC for instance
|
||||
|
||||
When starting an instance the compute node:
|
||||
|
||||
* sets up a VLAN interface and bridge on each host as required when an
|
||||
instance is started on that host
|
||||
|
||||
|
||||
Setup
|
||||
~~~~~
|
||||
|
||||
* Assign VLANs in the switch:
|
||||
|
||||
* public Internet segment
|
||||
* production network
|
||||
* management network
|
||||
* cluster DMZ
|
||||
|
||||
* Assign a contiguous range of VLANs to Nova for project use.
|
||||
* Configure management NIC ports as management VLAN access ports.
|
||||
* Configure management VLAN with Internet access as required
|
||||
* Configure production NIC ports as 802.1q trunk ports.
|
||||
* Configure Nova (need to add specifics here)
|
||||
|
||||
* public IPs
|
||||
* instance IPs
|
||||
* project network size
|
||||
* DMZ network
|
||||
|
||||
.. todo:: need specific Nova configuration added
|
||||
Loading…
Reference in New Issue