788b49f554
This patch: - renames all the RST files in the ops-guide folder to use a hyphen instead of underscore; - adds redirects to the renamed files to .htacces; - removes /([a-z-]+) from Admin Guide redirects in .htacces. Change-Id: I4c35a4c89ae9900a2e9bfe1a7a3bcb94ab72454b Implements: blueprint consistency-file-rename
306 lines
12 KiB
ReStructuredText
306 lines
12 KiB
ReStructuredText
=============
|
|
Compute Nodes
|
|
=============
|
|
|
|
In this chapter, we discuss some of the choices you need to consider
|
|
when building out your compute nodes. Compute nodes form the resource
|
|
core of the OpenStack Compute cloud, providing the processing, memory,
|
|
network and storage resources to run instances.
|
|
|
|
Choosing a CPU
|
|
~~~~~~~~~~~~~~
|
|
|
|
The type of CPU in your compute node is a very important choice. First,
|
|
ensure that the CPU supports virtualization by way of *VT-x* for Intel
|
|
chips and *AMD-v* for AMD chips.
|
|
|
|
.. tip::
|
|
|
|
Consult the vendor documentation to check for virtualization
|
|
support. For Intel, read `“Does my processor support Intel® Virtualization
|
|
Technology?” <http://www.intel.com/support/processors/sb/cs-030729.htm>`_.
|
|
For AMD, read `AMD Virtualization
|
|
<http://www.amd.com/en-us/innovations/software-technologies/server-solution/virtualization>`_.
|
|
Note that your CPU may support virtualization but it may be
|
|
disabled. Consult your BIOS documentation for how to enable CPU
|
|
features.
|
|
|
|
The number of cores that the CPU has also affects the decision. It's
|
|
common for current CPUs to have up to 12 cores. Additionally, if an
|
|
Intel CPU supports hyperthreading, those 12 cores are doubled to 24
|
|
cores. If you purchase a server that supports multiple CPUs, the number
|
|
of cores is further multiplied.
|
|
|
|
.. note::
|
|
|
|
**Multithread Considerations**
|
|
|
|
Hyper-Threading is Intel's proprietary simultaneous multithreading
|
|
implementation used to improve parallelization on their CPUs. You might
|
|
consider enabling Hyper-Threading to improve the performance of
|
|
multithreaded applications.
|
|
|
|
Whether you should enable Hyper-Threading on your CPUs depends upon your
|
|
use case. For example, disabling Hyper-Threading can be beneficial in
|
|
intense computing environments. We recommend that you do performance
|
|
testing with your local workload with both Hyper-Threading on and off to
|
|
determine what is more appropriate in your case.
|
|
|
|
Choosing a Hypervisor
|
|
~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
A hypervisor provides software to manage virtual machine access to the
|
|
underlying hardware. The hypervisor creates, manages, and monitors
|
|
virtual machines. OpenStack Compute supports many hypervisors to various
|
|
degrees, including:
|
|
|
|
* `KVM <http://www.linux-kvm.org/page/Main_Page>`_
|
|
* `LXC <https://linuxcontainers.org/>`_
|
|
* `QEMU <http://wiki.qemu.org/Main_Page>`_
|
|
* `VMware ESX/ESXi <https://www.vmware.com/support/vsphere-hypervisor>`_
|
|
* `Xen <http://www.xenproject.org/>`_
|
|
* `Hyper-V <http://technet.microsoft.com/en-us/library/hh831531.aspx>`_
|
|
* `Docker <https://www.docker.com/>`_
|
|
|
|
Probably the most important factor in your choice of hypervisor is your
|
|
current usage or experience. Aside from that, there are practical
|
|
concerns to do with feature parity, documentation, and the level of
|
|
community experience.
|
|
|
|
For example, KVM is the most widely adopted hypervisor in the OpenStack
|
|
community. Besides KVM, more deployments run Xen, LXC, VMware, and
|
|
Hyper-V than the others listed. However, each of these are lacking some
|
|
feature support or the documentation on how to use them with OpenStack
|
|
is out of date.
|
|
|
|
The best information available to support your choice is found on the
|
|
`Hypervisor Support Matrix
|
|
<http://docs.openstack.org/developer/nova/support-matrix.html>`_
|
|
and in the `configuration reference
|
|
<http://docs.openstack.org/mitaka/config-reference/compute/hypervisors.html>`_.
|
|
|
|
.. note::
|
|
|
|
It is also possible to run multiple hypervisors in a single
|
|
deployment using host aggregates or cells. However, an individual
|
|
compute node can run only a single hypervisor at a time.
|
|
|
|
Instance Storage Solutions
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
As part of the procurement for a compute cluster, you must specify some
|
|
storage for the disk on which the instantiated instance runs. There are
|
|
three main approaches to providing this temporary-style storage, and it
|
|
is important to understand the implications of the choice.
|
|
|
|
They are:
|
|
|
|
* Off compute node storage—shared file system
|
|
* On compute node storage—shared file system
|
|
* On compute node storage—nonshared file system
|
|
|
|
In general, the questions you should ask when selecting storage are as
|
|
follows:
|
|
|
|
* What is the platter count you can achieve?
|
|
* Do more spindles result in better I/O despite network access?
|
|
* Which one results in the best cost-performance scenario you are aiming for?
|
|
* How do you manage the storage operationally?
|
|
|
|
Many operators use separate compute and storage hosts. Compute services
|
|
and storage services have different requirements, and compute hosts
|
|
typically require more CPU and RAM than storage hosts. Therefore, for a
|
|
fixed budget, it makes sense to have different configurations for your
|
|
compute nodes and your storage nodes. Compute nodes will be invested in
|
|
CPU and RAM, and storage nodes will be invested in block storage.
|
|
|
|
However, if you are more restricted in the number of physical hosts you
|
|
have available for creating your cloud and you want to be able to
|
|
dedicate as many of your hosts as possible to running instances, it
|
|
makes sense to run compute and storage on the same machines.
|
|
|
|
We'll discuss the three main approaches to instance storage in the next
|
|
few sections.
|
|
|
|
Off Compute Node Storage—Shared File System
|
|
-------------------------------------------
|
|
|
|
In this option, the disks storing the running instances are hosted in
|
|
servers outside of the compute nodes.
|
|
|
|
If you use separate compute and storage hosts, you can treat your
|
|
compute hosts as "stateless." As long as you don't have any instances
|
|
currently running on a compute host, you can take it offline or wipe it
|
|
completely without having any effect on the rest of your cloud. This
|
|
simplifies maintenance for the compute hosts.
|
|
|
|
There are several advantages to this approach:
|
|
|
|
* If a compute node fails, instances are usually easily recoverable.
|
|
* Running a dedicated storage system can be operationally simpler.
|
|
* You can scale to any number of spindles.
|
|
* It may be possible to share the external storage for other purposes.
|
|
|
|
The main downsides to this approach are:
|
|
|
|
* Depending on design, heavy I/O usage from some instances can affect
|
|
unrelated instances.
|
|
* Use of the network can decrease performance.
|
|
|
|
On Compute Node Storage—Shared File System
|
|
------------------------------------------
|
|
|
|
In this option, each compute node is specified with a significant amount
|
|
of disk space, but a distributed file system ties the disks from each
|
|
compute node into a single mount.
|
|
|
|
The main advantage of this option is that it scales to external storage
|
|
when you require additional storage.
|
|
|
|
However, this option has several downsides:
|
|
|
|
* Running a distributed file system can make you lose your data
|
|
locality compared with nonshared storage.
|
|
* Recovery of instances is complicated by depending on multiple hosts.
|
|
* The chassis size of the compute node can limit the number of spindles
|
|
able to be used in a compute node.
|
|
* Use of the network can decrease performance.
|
|
|
|
On Compute Node Storage—Nonshared File System
|
|
---------------------------------------------
|
|
|
|
In this option, each compute node is specified with enough disks to
|
|
store the instances it hosts.
|
|
|
|
There are two main reasons why this is a good idea:
|
|
|
|
* Heavy I/O usage on one compute node does not affect instances on
|
|
other compute nodes.
|
|
* Direct I/O access can increase performance.
|
|
|
|
This has several downsides:
|
|
|
|
* If a compute node fails, the instances running on that node are lost.
|
|
* The chassis size of the compute node can limit the number of spindles
|
|
able to be used in a compute node.
|
|
* Migrations of instances from one node to another are more complicated
|
|
and rely on features that may not continue to be developed.
|
|
* If additional storage is required, this option does not scale.
|
|
|
|
Running a shared file system on a storage system apart from the computes
|
|
nodes is ideal for clouds where reliability and scalability are the most
|
|
important factors. Running a shared file system on the compute nodes
|
|
themselves may be best in a scenario where you have to deploy to
|
|
preexisting servers for which you have little to no control over their
|
|
specifications. Running a nonshared file system on the compute nodes
|
|
themselves is a good option for clouds with high I/O requirements and
|
|
low concern for reliability.
|
|
|
|
Issues with Live Migration
|
|
--------------------------
|
|
|
|
We consider live migration an integral part of the operations of the
|
|
cloud. This feature provides the ability to seamlessly move instances
|
|
from one physical host to another, a necessity for performing upgrades
|
|
that require reboots of the compute hosts, but only works well with
|
|
shared storage.
|
|
|
|
Live migration can also be done with nonshared storage, using a feature
|
|
known as *KVM live block migration*. While an earlier implementation of
|
|
block-based migration in KVM and QEMU was considered unreliable, there
|
|
is a newer, more reliable implementation of block-based live migration
|
|
as of QEMU 1.4 and libvirt 1.0.2 that is also compatible with OpenStack.
|
|
However, none of the authors of this guide have first-hand experience
|
|
using live block migration.
|
|
|
|
Choice of File System
|
|
---------------------
|
|
|
|
If you want to support shared-storage live migration, you need to
|
|
configure a distributed file system.
|
|
|
|
Possible options include:
|
|
|
|
* NFS (default for Linux)
|
|
* GlusterFS
|
|
* MooseFS
|
|
* Lustre
|
|
|
|
We've seen deployments with all, and recommend that you choose the one
|
|
you are most familiar with operating. If you are not familiar with any
|
|
of these, choose NFS, as it is the easiest to set up and there is
|
|
extensive community knowledge about it.
|
|
|
|
Overcommitting
|
|
~~~~~~~~~~~~~~
|
|
|
|
OpenStack allows you to overcommit CPU and RAM on compute nodes. This
|
|
allows you to increase the number of instances you can have running on
|
|
your cloud, at the cost of reducing the performance of the instances.
|
|
OpenStack Compute uses the following ratios by default:
|
|
|
|
* CPU allocation ratio: 16:1
|
|
* RAM allocation ratio: 1.5:1
|
|
|
|
The default CPU allocation ratio of 16:1 means that the scheduler
|
|
allocates up to 16 virtual cores per physical core. For example, if a
|
|
physical node has 12 cores, the scheduler sees 192 available virtual
|
|
cores. With typical flavor definitions of 4 virtual cores per instance,
|
|
this ratio would provide 48 instances on a physical node.
|
|
|
|
The formula for the number of virtual instances on a compute node is
|
|
``(OR*PC)/VC``, where:
|
|
|
|
OR
|
|
CPU overcommit ratio (virtual cores per physical core)
|
|
|
|
PC
|
|
Number of physical cores
|
|
|
|
VC
|
|
Number of virtual cores per instance
|
|
|
|
Similarly, the default RAM allocation ratio of 1.5:1 means that the
|
|
scheduler allocates instances to a physical node as long as the total
|
|
amount of RAM associated with the instances is less than 1.5 times the
|
|
amount of RAM available on the physical node.
|
|
|
|
For example, if a physical node has 48 GB of RAM, the scheduler
|
|
allocates instances to that node until the sum of the RAM associated
|
|
with the instances reaches 72 GB (such as nine instances, in the case
|
|
where each instance has 8 GB of RAM).
|
|
|
|
.. note::
|
|
Regardless of the overcommit ratio, an instance can not be placed
|
|
on any physical node with fewer raw (pre-overcommit) resources than
|
|
the instance flavor requires.
|
|
|
|
You must select the appropriate CPU and RAM allocation ratio for your
|
|
particular use case.
|
|
|
|
Logging
|
|
~~~~~~~
|
|
|
|
Logging is detailed more fully in :doc:`ops-logging-monitoring`. However,
|
|
it is an important design consideration to take into account before
|
|
commencing operations of your cloud.
|
|
|
|
OpenStack produces a great deal of useful logging information, however;
|
|
but for the information to be useful for operations purposes, you should
|
|
consider having a central logging server to send logs to, and a log
|
|
parsing/analysis system (such as logstash).
|
|
|
|
Networking
|
|
~~~~~~~~~~
|
|
|
|
Networking in OpenStack is a complex, multifaceted challenge. See
|
|
:doc:`arch-network-design`.
|
|
|
|
Conclusion
|
|
~~~~~~~~~~
|
|
|
|
Compute nodes are the workhorse of your cloud and the place where your
|
|
users' applications will run. They are likely to be affected by your
|
|
decisions on what to deploy and how you deploy it. Their requirements
|
|
should be reflected in the choices you make.
|