openstack/openstack-manuals
Code Issues Proposed changes

Merge "Convert compute sections to rst"

Browse Source
This commit is contained in:
Jenkins 2015-07-21 22:44:44 +00:00 committed by Gerrit Code Review
commit e5da3f1042
12 changed files with 2680 additions and 1064 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

382
doc/admin-guide-cloud-rst/source/compute-configure-migrations.rst
View File

@ -1,382 +0,0 @@
.. _section_configuring-compute-migrations:
====================
Configure migrations
====================
.. :ref:`_configuring-migrations-kvm-libvirt`
.. :ref:`_configuring-migrations-xenserver`
.. note::
Only cloud administrators can perform live migrations. If your cloud
is configured to use cells, you can perform live migration within
but not between cells.
Migration enables an administrator to move a virtual-machine instance
from one compute host to another. This feature is useful when a compute
host requires maintenance. Migration can also be useful to redistribute
the load when many VM instances are running on a specific physical
machine.
The migration types are:
- **Non-live migration** (sometimes referred to simply as 'migration').
The instance is shut down for a period of time to be moved to another
hypervisor. In this case, the instance recognizes that it was
rebooted.
- **Live migration** (or 'true live migration'). Almost no instance
downtime. Useful when the instances must be kept running during the
migration. The different types of live migration are:
- **Shared storage-based live migration**. Both hypervisors have
access to shared storage.
- **Block live migration**. No shared storage is required.
Incompatible with read-only devices such as CD-ROMs and
`Configuration Drive (config\_drive) <http://docs.openstack.org/user-guide/cli_config_drive.html>`_.
- **Volume-backed live migration**. Instances are backed by volumes
rather than ephemeral disk, no shared storage is required, and
migration is supported (currently only available for libvirt-based
hypervisors).
The following sections describe how to configure your hosts and compute
nodes for migrations by using the KVM and XenServer hypervisors.
.. _configuring-migrations-kvm-libvirt:
KVM-Libvirt
~~~~~~~~~~~
.. :ref:`_configuring-migrations-kvm-shared-storage`
.. :ref:`_configuring-migrations-kvm-block-migration`
.. _configuring-migrations-kvm-shared-storage:
Shared storage
--------------
.. :ref:`_section_example-compute-install`
.. :ref:`_true-live-migration-kvm-libvirt`
**Prerequisites**
- **Hypervisor:** KVM with libvirt
- **Shared storage:** :file:`NOVA-INST-DIR/instances/` (for example,
:file:`/var/lib/nova/instances`) has to be mounted by shared storage.
This guide uses NFS but other options, including the
`OpenStack Gluster Connector <http://gluster.org/community/documentation//index.php/OSConnect>`_
are available.
- **Instances:** Instance can be migrated with iSCSI-based volumes.
.. note::
- Because the Compute service does not use the libvirt live
migration functionality by default, guests are suspended before
migration and might experience several minutes of downtime. For
details, see :ref:Enabling True Live Migration.
- This guide assumes the default value for ``instances_path`` in
your :file:`nova.conf` file (:file:`NOVA-INST-DIR/instances`). If you
have changed the ``state_path`` or ``instances_path`` variables,
modify the commands accordingly.
- You must specify ``vncserver_listen=0.0.0.0`` or live migration
will not work correctly.
- You must specify the ``instances_path`` in each node that runs
nova-compute. The mount point for ``instances_path`` must be the
same value for each node, or live migration will not work
correctly.
.. _section_example-compute-install:
Example Compute installation environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- Prepare at least three servers. In this example, we refer to the
servers as ``HostA``, ``HostB``, and ``HostC``:
- ``HostA`` is the Cloud Controller, and should run these services:
nova-api, nova-scheduler, ``nova-network``, cinder-volume, and
``nova-objectstore``.
- ``HostB`` and ``HostC`` are the compute nodes that run
nova-compute.
Ensure that ``NOVA-INST-DIR`` (set with ``state_path`` in the
:file:`nova.conf` file) is the same on all hosts.
- In this example, ``HostA`` is the NFSv4 server that exports
``NOVA-INST-DIR/instances`` directory. ``HostB`` and ``HostC`` are
NFSv4 clients that mount ``HostA``.
**Configuring your system**
#. Configure your DNS or ``/etc/hosts`` and ensure it is consistent across
all hosts. Make sure that the three hosts can perform name resolution
with each other. As a test, use the :command:`ping` command to ping each host
from one another:
.. code:: console
$ ping HostA
$ ping HostB
$ ping HostC
#. Ensure that the UID and GID of your Compute and libvirt users are
identical between each of your servers. This ensures that the
permissions on the NFS mount works correctly.
#. Export ``NOVA-INST-DIR/instances`` from ``HostA``, and ensure it is
readable and writable by the Compute user on ``HostB`` and ``HostC``.
For more information, see: `SettingUpNFSHowTo <https://help.ubuntu.com/community/SettingUpNFSHowTo>`_
or `CentOS/Red Hat: Setup NFS v4.0 File Server <http://www.cyberciti.biz/faq/centos-fedora-rhel-nfs-v4-configuration/>`_
#. Configure the NFS server at ``HostA`` by adding the following line to
the :file:`/etc/exports` file:
.. code:: ini
NOVA-INST-DIR/instances HostA/255.255.0.0(rw,sync,fsid=0,no_root_squash)
Change the subnet mask (``255.255.0.0``) to the appropriate value to
include the IP addresses of ``HostB`` and ``HostC``. Then restart the
NFS server:
.. code:: console
# /etc/init.d/nfs-kernel-server restart
# /etc/init.d/idmapd restart
#. On both compute nodes, enable the 'execute/search' bit on your shared
directory to allow qemu to be able to use the images within the
directories. On all hosts, run the following command:
.. code:: console
$ chmod o+x NOVA-INST-DIR/instances
#. Configure NFS on ``HostB`` and ``HostC`` by adding the following line to
the :file:`/etc/fstab` file
.. code:: console
HostA:/ /NOVA-INST-DIR/instances nfs4 defaults 0 0
Ensure that you can mount the exported directory
.. code:: console
$ mount -a -v
Check that ``HostA`` can see the :file:`NOVA-INST-DIR/instances/`
directory
.. code:: console
$ ls -ld NOVA-INST-DIR/instances/
drwxr-xr-x 2 nova nova 4096 2012-05-19 14:34 nova-install-dir/instances/
Perform the same check on ``HostB`` and ``HostC``, paying special
attention to the permissions (Compute should be able to write)
.. code-block:: console
:linenos:
$ ls -ld NOVA-INST-DIR/instances/
drwxr-xr-x 2 nova nova 4096 2012-05-07 14:34 nova-install-dir/instances/
$ df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 921514972 4180880 870523828 1% /
none 16498340 1228 16497112 1% /dev
none 16502856 0 16502856 0% /dev/shm
none 16502856 368 16502488 1% /var/run
none 16502856 0 16502856 0% /var/lock
none 16502856 0 16502856 0% /lib/init/rw
HostA: 921515008 101921792 772783104 12% /var/lib/nova/instances ( <--- this line is important.)
#. Update the libvirt configurations so that the calls can be made
securely. These methods enable remote access over TCP and are not
documented here.
- SSH tunnel to libvirtd's UNIX socket
- libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
- libvirtd TCP socket, with TLS for encryption and x509 client certs
for authentication
- libvirtd TCP socket, with TLS for encryption and Kerberos for
authentication
Restart libvirt. After you run the command, ensure that libvirt is
successfully restarted
.. code:: console
# stop libvirt-bin && start libvirt-bin
$ ps -ef | grep libvirt
root 1145 1 0 Nov27 ? 00:00:03 /usr/sbin/libvirtd -d -l\
#. Configure your firewall to allow libvirt to communicate between nodes.
By default, libvirt listens on TCP port 16509, and an ephemeral TCP
range from 49152 to 49261 is used for the KVM communications. Based on
the secure remote access TCP configuration you chose, be careful which
ports you open, and always understand who has access. For information
about ports that are used with libvirt,
see `the libvirt documentation <http://libvirt.org/remote.html#Remote_libvirtd_configuration>`_.
#. You can now configure options for live migration. In most cases, you
will not need to configure any options. The following chart is for
advanced users only.
.. TODO :include :: /../../common/tables/nova-livemigration.xml/
.. _true-live-migration-kvm-libvirt:
Enabling true live migration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Prior to the Kilo release, the Compute service did not use the libvirt
live migration function by default. To enable this function, add the
following line to the ``[libvirt]`` section of the :file:`nova.conf` file:
.. code:: ini
live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_TUNNELLED
On versions older than Kilo, the Compute service does not use libvirt's
live migration by default because there is a risk that the migration
process will never end. This can happen if the guest operating system
uses blocks on the disk faster than they can be migrated.
.. _configuring-migrations-kvm-block-migration:
Block Migration
^^^^^^^^^^^^^^^
Configuring KVM for block migration is exactly the same as the above
configuration in :ref:`configuring-migrations-kvm-shared-storage`
the section called shared storage, except that ``NOVA-INST-DIR/instances``
is local to each host rather than shared. No NFS client or server
configuration is required.
.. note::
- To use block migration, you must use the :option:`--block-migrate`
parameter with the live migration command.
- Block migration is incompatible with read-only devices such as
CD-ROMs and `Configuration Drive (config_drive) <http://docs.openstack.org/user-guide/cli_config_drive.html>`_.
- Since the ephemeral drives are copied over the network in block
migration, migrations of instances with heavy I/O loads may never
complete if the drives are writing faster than the data can be
copied over the network.
.. _configuring-migrations-xenserver:
XenServer
~~~~~~~~~
.. :ref:Shared Storage
.. :ref:Block migration
.. _configuring-migrations-xenserver-shared-storage:
Shared storage
--------------
Prerequisites
^^^^^^^^^^^^^
- **Compatible XenServer hypervisors**. For more information, see the
`Requirements for Creating Resource Pools <http://docs.vmd.citrix.com/XenServer/6.0.0/1.0/en_gb/reference.html#pooling_homogeneity_requirements>`_ section of the XenServer
Administrator's Guide.
- **Shared storage**. An NFS export, visible to all XenServer hosts.
.. note::
For the supported NFS versions, see the
`NFS VHD <http://docs.vmd.citrix.com/XenServer/6.0.0/1.0/en_gb/reference.html#id1002701>`_
section of the XenServer Administrator's Guide.
To use shared storage live migration with XenServer hypervisors, the
hosts must be joined to a XenServer pool. To create that pool, a host
aggregate must be created with specific metadata. This metadata is used
by the XAPI plug-ins to establish the pool.
**Using shared storage live migrations with XenServer Hypervisors**
#. Add an NFS VHD storage to your master XenServer, and set it as the
default storage repository. For more information, see NFS VHD in the
XenServer Administrator's Guide.
#. Configure all compute nodes to use the default storage repository
(``sr``) for pool operations. Add this line to your :file:`nova.conf`
configuration files on all compute nodes:
.. code:: ini
sr_matching_filter=default-sr:true
#. Create a host aggregate. This command creates the aggregate, and then
displays a table that contains the ID of the new aggregate
.. code:: console
$ nova aggregate-create POOL_NAME AVAILABILITY_ZONE
Add metadata to the aggregate, to mark it as a hypervisor pool
.. code:: console
$ nova aggregate-set-metadata AGGREGATE_ID hypervisor_pool=true
$ nova aggregate-set-metadata AGGREGATE_ID operational_state=created
Make the first compute node part of that aggregate
.. code:: console
$ nova aggregate-add-host AGGREGATE_ID MASTER_COMPUTE_NAME
The host is now part of a XenServer pool.
#. Add hosts to the pool
.. code:: console
$ nova aggregate-add-host AGGREGATE_ID COMPUTE_HOST_NAME
.. note::
The added compute node and the host will shut down to join the host
to the XenServer pool. The operation will fail if any server other
than the compute node is running or suspended on the host.
.. _configuring-migrations-xenserver-block-migration:
Block migration
^^^^^^^^^^^^^^^
- **Compatible XenServer hypervisors**.
The hypervisors must support the Storage XenMotion feature.
See your XenServer manual to make sure your edition
has this feature.
.. note::
- To use block migration, you must use the :option:`--block-migrate`
parameter with the live migration command.
- Block migration works only with EXT local storage storage
repositories, and the server must not have any volumes attached.

120
doc/admin-guide-cloud-rst/source/compute-configure-service-groups.rst
View File

@ -1,120 +0,0 @@
.. highlight:: ini
:linenothreshold: 5
.. _configuring-compute-service-groups:
================================
Configure Compute service groups
================================
The Compute service must know the status of each compute node to
effectively manage and use them. This can include events like a user
launching a new VM, the scheduler sending a request to a live node, or a
query to the ServiceGroup API to determine if a node is live.
When a compute worker running the nova-compute daemon starts, it calls
the join API to join the compute group. Any service (such as the
scheduler) can query the group's membership and the status of its nodes.
Internally, the ServiceGroup client driver automatically updates the
compute worker status.
.. _database-servicegroup-driver:
Database ServiceGroup driver
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
By default, Compute uses the database driver to track if a node is live.
In a compute worker, this driver periodically sends a ``db update``
command to the database, saying “I'm OK” with a timestamp. Compute uses
a pre-defined timeout (``service_down_time``) to determine if a node is
dead.
The driver has limitations, which can be problematic depending on your
environment. If a lot of compute worker nodes need to be checked, the
database can be put under heavy load, which can cause the timeout to
trigger, and a live node could incorrectly be considered dead. By
default, the timeout is 60 seconds. Reducing the timeout value can help
in this situation, but you must also make the database update more
frequently, which again increases the database workload.
The database contains data that is both transient (such as whether the
node is alive) and persistent (such as entries for VM owners). With the
ServiceGroup abstraction, Compute can treat each type separately.
.. _zookeeper-servicegroup-driver:
ZooKeeper ServiceGroup driver
-----------------------------
The ZooKeeper ServiceGroup driver works by using ZooKeeper ephemeral
nodes. ZooKeeper, unlike databases, is a distributed system, with its
load divided among several servers. On a compute worker node, the driver
can establish a ZooKeeper session, then create an ephemeral znode in the
group directory. Ephemeral znodes have the same lifespan as the session.
If the worker node or the nova-compute daemon crashes, or a network
partition is in place between the worker and the ZooKeeper server
quorums, the ephemeral znodes are removed automatically. The driver
can be given group membership by running the :command:`ls` command in the
group directory.
The ZooKeeper driver requires the ZooKeeper servers and client
libraries. Setting up ZooKeeper servers is outside the scope of this
guide (for more information, see `Apache Zookeeper <http://zookeeper.apache.org/>`_). These client-side
Python libraries must be installed on every compute node:
**python-zookeeper**
The official Zookeeper Python binding
**evzookeeper**
This library makes the binding work with the eventlet threading model.
This example assumes the ZooKeeper server addresses and ports are
``192.168.2.1:2181``, ``192.168.2.2:2181``, and ``192.168.2.3:2181``.
These values in the :file:`/etc/nova/nova.conf` file are required on every
node for the ZooKeeper driver:
.. code:: ini
# Driver for the ServiceGroup service
servicegroup_driver="zk"
[zookeeper]
address="192.168.2.1:2181,192.168.2.2:2181,192.168.2.3:2181"
To customize the Compute Service groups, use these configuration option
settings:
.. TODO ../../common/tables/nova-zookeeper.xml
.. _memcache-servicegroup-driver:
Memcache ServiceGroup driver
----------------------------
The memcache ServiceGroup driver uses memcached, a distributed memory
object caching system that is used to increase site performance. For
more details, see `memcached.org <http://memcached.org/>`_.
To use the memcache driver, you must install memcached. You might
already have it installed, as the same driver is also used for the
OpenStack Object Storage and OpenStack dashboard. If you need to install
memcached, see the instructions in the `OpenStack Installation Guide <http://docs.openstack.org/>`_.
These values in the :file:`/etc/nova/nova.conf` file are required on every
node for the memcache driver:
.. code:: ini
# Driver for the ServiceGroup service
servicegroup_driver="mc"
# Memcached servers. Use either a list of memcached servers to use for
caching (list value),
# or "<None>" for in-process caching (default).
memcached_servers=<None>
# Timeout; maximum time since last check-in for up service
(integer value).
# Helps to define whether a node is dead
service_down_time=60

165
doc/admin-guide-cloud-rst/source/compute-images-instances.rst
View File

@ -4,9 +4,6 @@ Images and instances
.. TODO (bmoss)
compute-image-mgt.rst
compute-instance-building-blocks.rst
compute-instance-mgt-tools.rst
instance-scheduling-constraints.rst
@ -39,14 +36,14 @@ flavors that you can edit or add to.
services <http://docs.openstack.org/kilo/config-reference/content/ch_configuring-openstack-image-service.html>`__
section of the OpenStack Configuration Reference.
- For more information about flavors, see ? or
- For more information about flavors, see :ref:`compute-flavors` or
`Flavors <http://docs.openstack.org/openstack-ops/content/flavors.html>`__
in the OpenStack Operations Guide.
You can add and remove additional resources from running instances, such
as persistent volume storage, or public IP addresses. The example used
in this chapter is of a typical virtual system within an OpenStack
cloud. It uses the cinder-volume service, which provides persistent
cloud. It uses the ``cinder-volume`` service, which provides persistent
block storage, instead of the ephemeral storage provided by the selected
instance flavor.
@ -54,14 +51,14 @@ This diagram shows the system state prior to launching an instance. The
image store, fronted by the Image service (glance) has a number of
predefined images. Inside the cloud, a compute node contains the
available vCPU, memory, and local disk resources. Additionally, the
cinder-volume service provides a number of predefined volumes.
``cinder-volume`` service provides a number of predefined volumes.
|Base image state with no running instances|
To launch an instance select an image, flavor, and any optional
attributes. The selected flavor provides a root volume, labeled ``vda``
in this diagram, and additional ephemeral storage, labeled ``vdb``. In
this example, the cinder-volume store is mapped to the third virtual
this example, the ``cinder-volume`` store is mapped to the third virtual
disk on this instance, ``vdc``.
|Instance creation from image and runtime state|
@ -74,8 +71,8 @@ images, as less data needs to be copied across the network.
A new empty ephemeral disk is also created, labeled ``vdb`` in this
diagram. This disk is destroyed when you delete the instance.
The compute node connects to the attached cinder-volume using ISCSI. The
cinder-volume is mapped to the third disk, labeled ``vdc`` in this
The compute node connects to the attached ``cinder-volume`` using ISCSI. The
``cinder-volume`` is mapped to the third disk, labeled ``vdc`` in this
diagram. After the compute node provisions the vCPU and memory
resources, the instance boots up from root volume ``vda``. The instance
runs, and changes data on the disks (highlighted in red on the diagram).
@ -98,6 +95,65 @@ process.
|End state of image and volume after instance exits|
Image management
~~~~~~~~~~~~~~~~
The OpenStack Image service discovers, registers, and retrieves virtual
machine images. The service also includes a RESTful API that allows you
to query VM image metadata and retrieve the actual image with HTTP
requests. For more information about the API, see the `OpenStack API
Complete Reference <http://developer.openstack.org/api-ref.html>`__ and
the `Python
API <http://docs.openstack.org/developer/python-glanceclient/>`__.
The OpenStack Image service can be controlled using a command-line tool.
For more information about using the OpenStack Image command-line tool,
see the `Manage
Images <http://docs.openstack.org/user-guide/common/cli_manage_images.html>`__
section in the OpenStack End User Guide.
Virtual images that have been made available through the Image service
can be stored in a variety of ways. In order to use these services, you
must have a working installation of the Image service, with a working
endpoint, and users that have been created in OpenStack Identity.
Additionally, you must meet the environment variables required by the
Compute and Image service clients.
The Image service supports these back-end stores:
File system
The OpenStack Image service stores virtual machine images in the
file system back end by default. This simple back end writes image
files to the local file system.
Object Storage
The OpenStack highly available service for storing objects.
Block Storage
The OpenStack highly available service for storing blocks.
VMware
ESX/ESXi or vCenter Server target system.
S3
The Amazon S3 service.
HTTP
OpenStack Image service can read virtual machine images that are
available on the Internet using HTTP. This store is read only.
RADOS block device (RBD)
Stores images inside of a Ceph storage cluster using Ceph's RBD
interface.
Sheepdog
A distributed storage system for QEMU/KVM.
GridFS
Stores images using MongoDB.
Image properties and property protection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
An image property is a key and value pair that the cloud administrator
@ -280,6 +336,97 @@ Information on the configuration options for caching on compute nodes
can be found in the `Configuration
Reference <http://docs.openstack.org/kilo/config-reference/content/>`__.
Instance building blocks
~~~~~~~~~~~~~~~~~~~~~~~~
In OpenStack, the base operating system is usually copied from an image
stored in the OpenStack Image service. This results in an ephemeral
instance that starts from a known template state and loses all
accumulated states on shutdown.
You can also put an operating system on a persistent volume in Compute
or the Block Storage volume system. This gives a more traditional,
persistent system that accumulates states that are preserved across
restarts. To get a list of available images on your system, run:
.. code:: console
$ nova image-list
+---------------------------+------------------+--------+----------------+
| ID | Name | Status | Server |
+---------------------------+------------------+--------+----------------+
| aee1d242-730f-431f-88c1- | | | |
| 87630c0f07ba | Ubuntu 14.04 | | |
| | cloudimg amd64 | ACTIVE | |
| 0b27baa1-0ca6-49a7-b3f4- | | | |
| 48388e440245 | Ubuntu 14.10 | | |
| | cloudimg amd64 | ACTIVE | |
| df8d56fc-9cea-4dfd-a8d3- | | | |
| 28764de3cb08 | jenkins | ACTIVE | |
+---------------------------+------------------+--------+----------------+
The displayed image attributes are:
``ID``
Automatically generated UUID of the image.
``Name``
Free form, human-readable name for the image.
``Status``
The status of the image. Images marked ``ACTIVE`` are available for
use.
``Server``
For images that are created as snapshots of running instances, this
is the UUID of the instance the snapshot derives from. For uploaded
images, this field is blank.
Virtual hardware templates are called ``flavors``. The default
installation provides five predefined flavors.
For a list of flavors that are available on your system, run:
.. code:: console
$ nova flavor-list
+----+----------+----------+-----+----------+-----+------+------------+----------+
| ID | Name | Memory_MB| Disk| Ephemeral| Swap| VCPUs| RXTX_Factor| Is_Public|
+----+----------+----------+-----+----------+-----+------+------------+----------+
| 1 | m1.tiny | 512 | 1 | 0 | | 1 | 1.0 | True |
| 2 | m1.small | 2048 | 20 | 0 | | 1 | 1.0 | True |
| 3 | m1.medium| 4096 | 40 | 0 | | 2 | 1.0 | True |
| 4 | m1.large | 8192 | 80 | 0 | | 4 | 1.0 | True |
| 5 | m1.xlarge| 16384 | 160 | 0 | | 8 | 1.0 | True |
+----+----------+----------+-----+----------+-----+------+------------+----------+
By default, administrative users can configure the flavors. You can
change this behavior by redefining the access controls for
``compute_extension:flavormanage`` in :file:`/etc/nova/policy.json` on the
``compute-api`` server.
Instance management tools
~~~~~~~~~~~~~~~~~~~~~~~~~
OpenStack provides command-line, web interface, and API-based instance
management tools. Third-party management tools are also available, using
either the native API or the provided EC2-compatible API.
The OpenStack python-novaclient package provides a basic command-line
utility, which uses the :command:`nova` command. This is available as a native
package for most Linux distributions, or you can install the latest
version using the pip python package installer:
::
# pip install python-novaclient
For more information about python-novaclient and other command-line
tools, see the `OpenStack End User
Guide <http://docs.openstack.org/user-guide/index.html>`__.
Control where instances run
~~~~~~~~~~~~~~~~~~~~~~~~~~~
The `OpenStack Configuration

1011
doc/admin-guide-cloud-rst/source/compute-networking-nova.rst Normal file
View File

File diff suppressed because it is too large Load Diff

333
doc/admin-guide-cloud-rst/source/compute-recover-nodes.rst
View File

@ -1,333 +0,0 @@
.. _section_nova-compute-node-down:
==================================
Recover from a failed compute node
==================================
If Compute is deployed with a shared file system, and a node fails,
there are several methods to quickly recover from the failure. This
section discusses manual recovery.
.. TODO ../../common/section_cli_nova_evacuate.xml
.. _nova-compute-node-down-manual-recovery:
Manual Recovery
~~~~~~~~~~~~~~~
To recover a KVM or libvirt compute node, see
the section called :ref:`nova-compute-node-down-manual-recovery`. For
all other hypervisors, use this procedure:
**Recovering from a failed compute node manually**
#. Identify the VMs on the affected hosts. To do this, you can use a
combination of ``nova list`` and ``nova show`` or
``euca-describe-instances``. For example, this command displays
information about instance i-000015b9 that is running on node np-rcc54:
.. code:: console
$ euca-describe-instances
i-000015b9 at3-ui02 running nectarkey (376, np-rcc54) 0 m1.xxlarge 2012-06-19T00:48:11.000Z 115.146.93.60
#. Query the Compute database to check the status of the host. This example
converts an EC2 API instance ID into an OpenStack ID. If you use the
:command:`nova` commands, you can substitute the ID directly (the output in
this example has been truncated):
.. code:: ini
mysql> SELECT * FROM instances WHERE id = CONV('15b9', 16, 10) \G;
*************************** 1. row ***************************
created_at: 2012-06-19 00:48:11
updated_at: 2012-07-03 00:35:11
deleted_at: NULL
...
id: 5561
...
power_state: 5
vm_state: shutoff
...
hostname: at3-ui02
host: np-rcc54
...
uuid: 3f57699a-e773-4650-a443-b4b37eed5a06
...
task_state: NULL
...
.. note::
The credentials for your database can be found in :file:`/etc/nova.conf`.
#. Decide which compute host the affected VM should be moved to, and run
this database command to move the VM to the new host:
.. code:: console
mysql> UPDATE instances SET host = 'np-rcc46' WHERE uuid = '3f57699a-e773-4650-a443-b4b37eed5a06';
#. If you are using a hypervisor that relies on libvirt (such as KVM),
update the :file:`libvirt.xml` file (found in
:file:`/var/lib/nova/instances/[instance ID]`) with these changes:
- Change the ``DHCPSERVER`` value to the host IP address of the new
compute host.
- Update the VNC IP to `0.0.0.0`
#. Reboot the VM:
.. code:: console
$ nova reboot 3f57699a-e773-4650-a443-b4b37eed5a06
The database update and :command:`nova reboot` command should be all that is
required to recover a VM from a failed host. However, if you continue to
have problems try recreating the network filter configuration using
``virsh``, restarting the Compute services, or updating the ``vm_state``
and ``power_state`` in the Compute database.
.. _section_nova-uid-mismatch:
**Recover from a UID/GID mismatch**
In some cases, files on your compute node can end up using the wrong UID
or GID. This can happen when running OpenStack Compute, using a shared
file system, or with an automated configuration tool. This can cause a
number of problems, such as inability to perform live migrations, or
start virtual machines.
This procedure runs on nova-compute hosts, based on the KVM hypervisor:
#. Set the nova UID in :file:`/etc/passwd` to the same number on all hosts (for
example, 112).
.. note::
Make sure you choose UIDs or GIDs that are not in use for other
users or groups.
#. Set the ``libvirt-qemu`` UID in :file:`/etc/passwd` to the same number on
all hosts (for example, 119).
#. Set the ``nova`` group in :file:`/etc/group` file to the same number on all
hosts (for example, 120).
#. Set the ``libvirtd`` group in :file:`/etc/group` file to the same number on
all hosts (for example, 119).
#. Stop the services on the compute node.
#. Change all the files owned by user or group nova. For example:
.. code:: console
# find / -uid 108 -exec chown nova {} \;
# note the 108 here is the old nova UID before the change
# find / -gid 120 -exec chgrp nova {} \;
#. Repeat all steps for the :file:`libvirt-qemu` files, if required.
#. Restart the services.
#. Run the :command:`find` command to verify that all files use the correct
identifiers.
.. _section_nova-disaster-recovery-process:
Recover cloud after disaster
----------------------------
This section covers procedures for managing your cloud after a disaster,
and backing up persistent storage volumes. Backups are mandatory, even
outside of disaster scenarios.
For a definition of a disaster recovery plan (DRP), see
`http://en.wikipedia.org/wiki/Disaster\_Recovery\_Plan <http://en.wikipedia.org/wiki/Disaster_Recovery_Plan>`_.
A disaster could happen to several components of your architecture (for
example, a disk crash, network loss, or a power failure). In this
example, the following components are configured:
- A cloud controller (nova-api, nova-objectstore, nova-network)
- A compute node (nova-compute)
- A storage area network (SAN) used by OpenStack Block Storage
(cinder-volumes)
The worst disaster for a cloud is power loss, which applies to all three
components. Before a power loss:
- Create an active iSCSI session from the SAN to the cloud controller
(used for the ``cinder-volumes`` LVM's VG).
- Create an active iSCSI session from the cloud controller to the compute
node (managed by cinder-volume).
- Create an iSCSI session for every volume (so 14 EBS volumes requires 14
iSCSI sessions).
- Create iptables or ebtables rules from the cloud controller to the
compute node. This allows access from the cloud controller to the
running instance.
- Save the current state of the database, the current state of the running
instances, and the attached volumes (mount point, volume ID, volume
status, etc), at least from the cloud controller to the compute node.
After power is recovered and all hardware components have restarted:
- The iSCSI session from the SAN to the cloud no longer exists.
- The iSCSI session from the cloud controller to the compute node no
longer exists.
- The iptables and ebtables from the cloud controller to the compute
node are recreated. This is because nova-network reapplies
configurations on boot.
- Instances are no longer running.
Note that instances will not be lost, because neither ``destroy`` nor
``terminate`` was invoked. The files for the instances will remain on
the compute node.
- The database has not been updated.
**Begin recovery**
.. warning::
Do not add any extra steps to this procedure, or perform the steps
out of order.
#. Check the current relationship between the volume and its instance, so
that you can recreate the attachment.
This information can be found using the :command:`nova volume-list` command.
Note that the ``nova`` client also includes the ability to get volume
information from OpenStack Block Storage.
#. Update the database to clean the stalled state. Do this for every
volume, using these queries:
.. code:: console
mysql> use cinder;
mysql> update volumes set mountpoint=NULL;
mysql> update volumes set status="available" where status <>"error_deleting";
mysql> update volumes set attach_status="detached";
mysql> update volumes set instance_id=0;
Use ``nova volume-list`` commands to list all volumes.
#. Restart the instances using the :command:`nova reboot INSTANCE` command.
.. important::
Some instances will completely reboot and become reachable, while
some might stop at the plymouth stage. This is expected behavior, DO
NOT reboot a second time.
Instance state at this stage depends on whether you added an
`/etc/fstab` entry for that volume. Images built with the
cloud-init package remain in a ``pending`` state, while others skip
the missing volume and start. This step is performed in order to ask
Compute to reboot every instance, so that the stored state is
preserved. It does not matter if not all instances come up
successfully. For more information about cloud-init, see
`help.ubuntu.com/community/CloudInit/ <https://help.ubuntu.com/community/CloudInit/>`__.
#. Reattach the volumes to their respective instances, if required, using
the :command:`nova volume-attach` command. This example uses a file of
listed volumes to reattach them:
.. code:: bash
#!/bin/bash
while read line; do
volume=`echo $line | $CUT -f 1 -d " "`
instance=`echo $line | $CUT -f 2 -d " "`
mount_point=`echo $line | $CUT -f 3 -d " "`
echo "ATTACHING VOLUME FOR INSTANCE - $instance"
nova volume-attach $instance $volume $mount_point
sleep 2
done < $volumes_tmp_file
Instances that were stopped at the plymouth stage will now automatically
continue booting and start normally. Instances that previously started
successfully will now be able to see the volume.
#. Log in to the instances with SSH and reboot them.
If some services depend on the volume, or if a volume has an entry in
fstab, you should now be able to restart the instance. Restart directly
from the instance itself, not through ``nova``:
.. code:: console
# shutdown -r now
When you are planning for and performing a disaster recovery, follow
these tips:
- Use the ``errors=remount`` parameter in the :file:`fstab` file to prevent
data corruption.
This parameter will cause the system to disable the ability to write
to the disk if it detects an I/O error. This configuration option
should be added into the cinder-volume server (the one which performs
the iSCSI connection to the SAN), and into the instances' :file:`fstab`
files.
- Do not add the entry for the SAN's disks to the cinder-volume's
:file:`fstab` file.
Some systems hang on that step, which means you could lose access to
your cloud-controller. To re-run the session manually, run this
command before performing the mount:
.. code:: console
# iscsiadm -m discovery -t st -p $SAN_IP $ iscsiadm -m node --target-name $IQN -p $SAN_IP -l
- On your instances, if you have the whole ``/home/`` directory on the
disk, leave a user's directory with the user's bash files and the
:file:`authorized_keys` file (instead of emptying the ``/home`` directory
and mapping the disk on it).
This allows you to connect to the instance even without the volume
attached, if you allow only connections through public keys.
If you want to script the disaster recovery plan (DRP), a bash script is
available from `https://github.com/Razique <https://github.com/Razique/BashStuff/blob/master/SYSTEMS/OpenStack/SCR_5006_V00_NUAC-OPENSTACK-DRP-OpenStack.sh>`_
which performs the following steps:
#. An array is created for instances and their attached volumes.
#. The MySQL database is updated.
#. All instances are restarted with euca2ools.
#. The volumes are reattached.
#. An SSH connection is performed into every instance using Compute
credentials.
The script includes a ``test mode``, which allows you to perform that
whole sequence for only one instance.
To reproduce the power loss, connect to the compute node which runs that
instance and close the iSCSI session. Do not detach the volume using the
:command:`nova volume-detach` command, manually close the iSCSI session.
This example closes an iSCSI session with the number 15:
.. code:: console
# iscsiadm -m session -u -r 15
Do not forget the ``-r`` flag. Otherwise, you will close all sessions.

102
doc/admin-guide-cloud-rst/source/compute-rootwrap.rst
View File

@ -1,102 +0,0 @@
.. _root-wrap-reference:
====================
Secure with rootwrap
====================
Rootwrap allows unprivileged users to safely run Compute actions as the
root user. Compute previously used :command:`sudo` for this purpose, but this
was difficult to maintain, and did not allow advanced filters. The
:command:`rootwrap` command replaces :command:`sudo` for Compute.
To use rootwrap, prefix the Compute command with :command:`nova-rootwrap`. For
example:
.. code:: console
$ sudo nova-rootwrap /etc/nova/rootwrap.conf command
A generic ``sudoers`` entry lets the Compute user run :command:`nova-rootwrap`
as root. The :command:`nova-rootwrap` code looks for filter definition
directories in its configuration file, and loads command filters from
them. It then checks if the command requested by Compute matches one of
those filters and, if so, executes the command (as root). If no filter
matches, it denies the request.
.. note::
Be aware of issues with using NFS and root-owned files. The NFS
share must be configured with the ``no_root_squash`` option enabled,
in order for rootwrap to work correctly.
Rootwrap is fully controlled by the root user. The root user
owns the sudoers entry which allows Compute to run a specific
rootwrap executable as root, and only with a specific
configuration file (which should also be owned by root).
The :command:`nova-rootwrap` command imports the Python
modules it needs from a cleaned, system-default PYTHONPATH.
The root-owned configuration file points to root-owned
filter definition directories, which contain root-owned
filters definition files. This chain ensures that the Compute
user itself is not in control of the configuration or modules
used by the :command:`nova-rootwrap` executable.
Rootwrap is configured using the :file:`rootwrap.conf` file. Because
it's in the trusted security path, it must be owned and writable
by only the root user. The file's location is specified in both
the sudoers entry and in the :file:`nova.conf` configuration file
with the ``rootwrap_config=entry`` parameter.
The :file:`rootwrap.conf` file uses an INI file format with these
sections and parameters:
.. list-table:: rootwrap.conf configuration options
:widths: 64 31
* - Configuration option=Default value
- (Type) Description
* - [DEFAULT]
filters\_path=/etc/nova/rootwrap.d,/usr/share/nova/rootwrap
- (ListOpt) Comma-separated list of directories
containing filter definition files.
Defines where rootwrap filters are stored.
Directories defined on this line should all
exist, and be owned and writable only by the
root user.
If the root wrapper is not performing correctly, you can add a
workaround option into the :file:`nova.conf` configuration file. This
workaround re-configures the root wrapper configuration to fall back to
running commands as sudo, and is a Kilo release feature.
Including this workaround in your configuration file safeguards your
environment from issues that can impair root wrapper performance. Tool
changes that have impacted
`Python Build Reasonableness (PBR) <https://git.openstack.org/cgit/openstack-dev/pbr/>`__
for example, are a known issue that affects root wrapper performance.
To set up this workaround, configure the ``disable_rootwrap`` option in
the ``[workaround]`` section of the :file:`nova.conf` configuration file.
The filters definition files contain lists of filters that rootwrap will
use to allow or deny a specific command. They are generally suffixed by
``.filters`` . Since they are in the trusted security path, they need to
be owned and writable only by the root user. Their location is specified
in the :file:`rootwrap.conf` file.
Filter definition files use an INI file format with a ``[Filters]``
section and several lines, each with a unique parameter name, which
should be different for each filter you define:
.. list-table:: filters configuration options
:widths: 72 39
* - Configuration option=Default value
- (Type) Description
* - [Filters]
filter\_name=kpartx: CommandFilter, /sbin/kpartx, root
- (ListOpt) Comma-separated list containing the filter class to
use, followed by the Filter arguments (which vary depending
on the Filter class selected).

36
doc/admin-guide-cloud-rst/source/compute-security.rst
View File

@ -1,36 +0,0 @@
.. _section-compute-security:
==================
Security hardening
==================
OpenStack Compute can be integrated with various third-party
technologies to increase security. For more information, see the
`OpenStack Security Guide <http://docs.openstack.org/sec/>`_.
.. :ref:section_trusted-compute-pools.rst
Encrypt Compute metadata traffic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
**Enabling SSL encryption**
OpenStack supports encrypting Compute metadata traffic with HTTPS.
Enable SSL encryption in the :file:`metadata_agent.ini` file.
#. Enable the HTTPS protocol::
nova_metadata_protocol = https
#. Determine whether insecure SSL connections are accepted for Compute
metadata server requests. The default value is ``False``::
nova_metadata_insecure = False
#. Specify the path to the client certificate::
nova_client_cert = PATH_TO_CERT
#. Specify the path to the private key::
nova_client_priv_key = PATH_TO_KEY

1544
doc/admin-guide-cloud-rst/source/compute-system-admin.rst
View File

File diff suppressed because it is too large Load Diff

14
doc/admin-guide-cloud-rst/source/compute.rst
View File

@ -17,21 +17,11 @@ web-based API.
compute_arch.rst
compute-images-instances.rst
common/support-compute.rst
compute-networking-nova.rst
compute-system-admin.rst
compute_config-firewalls.rst
compute-rootwrap.rst
compute-configure-migrations.rst
compute-configure-service-groups.rst
compute-security.rst
compute-recover-nodes.rst
common/support-compute.rst
.. TODO (bmoss)
compute/section_compute-networking-nova.xml
compute/section_compute-system-admin.xml
../common/section_support-compute.xml
../common/section_cli_nova_usage_statistics.xml
../common/section_cli_nova_volumes.xml
../common/section_cli_nova_customize_flavors.xml
../common/section_compute-configure-console.xml

33
doc/admin-guide-cloud-rst/source/compute_config-firewalls.rst
View File

@ -1,33 +0,0 @@
.. _default_ports:
==========================================
Compute service node firewall requirements
==========================================
Console connections for virtual machines, whether direct or through a
proxy, are received on ports ``5900`` to ``5999``. The firewall on each
Compute service node must allow network traffic on these ports.
This procedure modifies the iptables firewall to allow incoming
connections to the Compute services.
**Configuring the service-node firewall**
#. Log in to the server that hosts the Compute service, as root.
#. Edit the :file:`/etc/sysconfig/iptables` file, to add an INPUT rule that
allows TCP traffic on ports from ``5900`` to ``5999``. Make sure the new
rule appears before any INPUT rules that REJECT traffic:
.. code:: ini
-A INPUT -p tcp -m multiport --dports 5900:5999 -j ACCEPT
#. Save the changes to :file:`/etc/sysconfig/iptables` file, and restart the
iptables service to pick up the changes:
.. code:: console
$ service iptables restart
#. Repeat this process for each Compute service node.

2
doc/admin-guide-cloud-rst/source/networking.rst
View File

@ -1,3 +1,5 @@
.. _networking:
==========
Networking
==========

2
doc/admin-guide-cloud-rst/source/networking_adv-features.rst
View File

@ -165,6 +165,8 @@ administrative user can view and set the provider extended attributes
through Networking API calls. See the section called
:ref:`Authentication and authorization` for details on policy configuration.
.. _L3-routing-and-NAT:
L3 routing and NAT
~~~~~~~~~~~~~~~~~~