1091 Commits

Author SHA1 Message Date
Mark McLoughlin
7cb17d63f1 Move global service networking opts to new module
The my_ip, host and use_ipv6 options are used all over the codebase
and they're pretty well related to each other. Create a new netconf
module for them to live in.

There are now no options registered globally in nova.config!

blueprint: scope-config-opts
Change-Id: Ifde37839ae6f38e6bf99dff1e80b8e25fd68ed25
2013-01-09 22:09:17 +00:00
Mark McLoughlin
d8c80c6b14 Move service_down_time to nova.service
blueprint: scope-config-opts
Change-Id: I5fddb3768348c43a38b72dbf738b0c7fa2967691
2013-01-09 22:09:17 +00:00
Mark McLoughlin
41ef13a34c Move enabled_apis option into nova.service
The enabled_apis option is only used in nova-api and nova-all but
there's no more obvious place to put it than nova.service. At least
this location is consistent, since we quite have a number of other
options related to the APIs in nova.service.

blueprint: scope-config-opts
Change-Id: I86e200fa5ad91ac0b7ab6837cc61e5927b2ebeb7
2013-01-09 08:15:43 +00:00
Mark McLoughlin
39ce4032bb Move manager options into nova.service
Move the compute_manager, console_manager, cert_manager, network_manager
and scheduler_manager options into nova.service.

Apart from the tests, these options are mostly just used in nova.service
so it makes sense for them to live there. Also, metadata_manager already
lives there.

blueprint: scope-config-opts
Change-Id: I180859817b57c081b83d6a6f075a6ff76bb0ef6d
2013-01-09 08:15:42 +00:00
Mark McLoughlin
eac6e73223 Move compute_topic into nova.compute.rpcapi
Even though the compute_topic opt is used outside nova.compute.rpcapi,
it makes sense for the RPC module to "own" the topic option.

blueprint: scope-config-opts
Change-Id: I453951825481b9a5ba1865f6bf1223f7b1e25f98
2013-01-09 08:13:14 +00:00
Jenkins
32cf601fc1 Merge "Move scheduler_topic into nova.scheduler.rpcapi" 2013-01-09 07:58:16 +00:00
Jenkins
579c5860ab Merge "Move console_topic into nova.console.rpcapi" 2013-01-09 07:03:07 +00:00
Jenkins
d7b7f10855 Merge "Move network_topic into nova.network.rpcapi" 2013-01-09 07:02:52 +00:00
Jenkins
aa46eb0a70 Merge "Move cert_topic into nova.cert.rpcapi" 2013-01-09 07:02:38 +00:00
Joe Gordon
1ab2fc6477 Remove availability_zones from service table
This is the final step in enabling availability_zones using aggregate
metadata. Previously all services had an availability_zone, but the
availability_zone is only used for nova-compute.  Services such as
nova-scheduler, nova-network, nova-conductor have always spanned all
availability_zones.

After this change only compute nodes (nova-compute), will have an
availability_zone. In order to preserve current APIs, when running:
* nova host-list (os-hosts)
* euca-describe-availability-zones verbose
* nova-manage service list
Internal services will appear in there own internal availability_zone
(CONF.internal_service_availability_zone)
Internal zone is hidden in euca-describe-availability_zones
(non-verbose)

CONF.node_availability_zone has been renamed to
CONF.default_availability_zone and is only used by the nova-api and
nova-scheduler. CONF.node_availability_zone still works but is
deprecated

DocImpact

Completes blueprint aggregate-based-availability-zones

Change-Id: Ib772df5f9ac2865f20df479f8ddce575a9ce3aff
2013-01-08 14:01:30 -08:00
Mikyung Kang
eeff3ddcb6 CLI for bare-metal database sync.
Part 3 of 6: blueprint general-bare-metal-provisioning-framework.

Change-Id: Ia19ce00edb84aa924c2ab2c9c2217f6b49073d69
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
2013-01-08 22:56:14 +09:00
Mark McLoughlin
b4af878634 Move scheduler_topic into nova.scheduler.rpcapi
The scheduler_topic opt is only used in nova.scheduler.rpcapi and it
makes sense for the RPC module to "own" the topic option.

blueprint: scope-config-opts
Change-Id: Iafb8705eb716b650b63e060fe1dd36666bcee812
2013-01-08 10:23:40 +00:00
Mark McLoughlin
7a5e72d80d Move console_topic into nova.console.rpcapi
Even though the console_topic opt is used outside nova.console.rpcapi,
it makes sense for the RPC module to "own" the topic option.

blueprint: scope-config-opts
Change-Id: I0f00927ca1c0bba7b21315240165e83e7075ebb7
2013-01-08 10:23:40 +00:00
Mark McLoughlin
580df54260 Move network_topic into nova.network.rpcapi
The network_topic opt is only used in nova.network.rpcapi and it makes
sense for the RPC module to "own" the topic option.

blueprint: scope-config-opts
Change-Id: Icd779f34df47ac6dd048b3e41995e872d8859872
2013-01-08 10:23:40 +00:00
Mark McLoughlin
9714f31f37 Move cert_topic into nova.cert.rpcapi
The cert_topic opt is only used in nova.cert.rpcapi and it makes sense
for the RPC module to "own" the topic option.

blueprint: scope-config-opts
Change-Id: If3584b8e1984e0425d9f33133f9d291739b04d49
2013-01-08 10:23:40 +00:00
Jenkins
0fc1e10d40 Merge "PXE bare-metal provisioning helper server" 2013-01-08 07:02:17 +00:00
Jenkins
2f7aa071dc Merge "Cells: Add cells commands to nova-manage" 2013-01-08 05:52:11 +00:00
Mikyung Kang
abe1db6f88 PXE bare-metal provisioning helper server
a part of blueprint general-bare-metal-provisioning-framework.

Implement nova-baremetal-deploy-helper.

This service listens for HTTP requests from baremetal deploy ramdisk,
formats the remote disk and writes an image to it, as part of baremetal
PXE provisioning.

blueprint improve-baremetal-pxe-deploy shows how we plan to improve this
process.

Change-Id: I0a1b020cc5f81d49559acd4dcc781397a58e2c01
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
2013-01-08 08:55:02 +13:00
Chris Behrens
f9a868e86c Cells: Add the main code.
This introduces *EXPERIMENTAL* compute cells functionality as a way to
scale nova in a more distributed fashion without having to use complicated
technologies like DB and message queue clustering.

Cells are configured as a tree and the top level cell should contain
nova-api without any nova-computes while child cells contain everything
except nova-api.  One can think of a cell as a normal nova deployment in
that each cell has its own DB server and message queue broker.

The top level cell keeps a subset of data about ALL instances in all
cells in its DB.  Child cells send messages to the top level cell when
instances change state.  Data in 1 child cell is not shared with another
child cell.

A new service, nova-cells, is introduced that handles communication
between cells and picking of a cell for new instances.  This service is
required for every cell.  Communication between cells is pluggable with
the only option currently implemented being communnication via RPC.

Cells scheduling is separate from host scheduling.  nova-cells first picks
a cell (currently randomly -- future patches add filtering/weighing
functionality and decisions can be based on broadcasts of
capacity/capabilities).  Once a cell has been selected and the new build
request has reached its nova-cells service, it'll be sent over to the host
scheduler in that cell and the build proceeds as it does without cells.

New config options are introduced for enabling and configuring the cells
code.  Cells is disabled by default.  All of the config options below go
under a '[cells]' section in nova.conf.  These are the options that one
may want to tweak:

enable -- Turn on cells code (default is False)
name -- Name of the current cell.
capabilities -- List of arbitrary key=value pairs defining capabilities
                of the current cell.  These are sent to parent cells,
                but aren't used in scheduling until later filter/weight
                support is added.
call_timeout -- How long to wait for replies from a calls between cells

When using cells, the compute API class must be changed in the API cell,
so that requests can be proxied via nova-cells down to the correct cell
properly.  Thus, config requirements for API cell:

--
[DEFAULT]
compute_api_class=nova.compute.cells_api.ComputeCellsAPI.
[cells]
enable=True
name=api-cell
--

Config requirements for child cell:

--
[cells]
enable=True
name=child-cell1
--

Another requirement is populating the 'cells' DB table in each cell.
Each cell needs to know about its parent and children and how to
communicate with them (message broker location, credentials, etc).

Implements blueprint nova-compute-cells

DocImpact

Change-Id: I1b52788ea9d7753365d175abf39bdbc22ba822fe
2013-01-04 20:45:05 +00:00
Daniel P. Berrange
977f4e0f5a Clarify & fix docs for nova-novncproxy
Clarify docs for nova-novncproxy to indicate that it only serves
noVNC web content. Also remove duplicated link from manpage

Change-Id: Ia1f4aff21591132a5c41810a371d248ea0001e1f
2013-01-03 11:01:33 +00:00
Chris Behrens
56eb0df00c Cells: Add cells commands to nova-manage
Add commands to nova-manage for configuring cells in the DB.

Implements blueprint nova-compute-cells

DocImpact

Change-Id: Ic1a950181d5ce191780647838c5d06e5410676f7
2013-01-02 21:37:55 +00:00
Jenkins
4f4ef9f556 Merge "Fix stack trace on incorrect nova-manage args." 2012-12-29 00:39:53 +00:00
Daniel P. Berrange
481d6ff142 Remove obsolete VCS version info completely
The VCS version string in nova/version.py has been hardcoded
to 'LOCALBRANCH:LOCALVERSION' since change

  0bbb0e8cb1daa351af91cf97b2bcababfc0c7f9f

The only places which use this info are some logging calls,
and since 'nova-manager version' command. The latter would
more usefully output the distro specific package string.

Delete the VCS related methods in version.py and replace their
usage with a new method version_string_with_package() which
appends the distro-specific package tag

Change-Id: I972b79c329198cb5376d537c6b672830094cd759
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2012-12-19 18:54:17 +00:00
Michael Still
e865f12001 Fix stack trace on incorrect nova-manage args.
This error reporting needs more work, but this at least stops us
from presenting a stack trace to the user and is as broken as
before.

Resolves bug 1091403.

Change-Id: I941983b60cc3762a916d57b7ea67264e093cfa44
2012-12-18 18:36:37 +11:00
Jenkins
84363a2e91 Merge "Add syslogging to nova-rootwrap" 2012-12-17 17:09:15 +00:00
Thierry Carrez
a5b12b675c Add syslogging to nova-rootwrap
Add syslogging capabilities to nova-rootwrap, if you set parameter
use_syslog to True. You can specify a facility (syslog_log_facility)
and level (syslog_log_level) to use. Finalizes bp nova-rootwrap-options.

In doing so, it moves rootwrap config parsing to a nova.rootwrap.wrapper
object (and adds unit testing for it). It also improves log messages
content (including the name of the matching filter and the escalation
path used). Incidentally fixes bug 1084766.

Change-Id: Idb8cd9c9febd6263dafab4bc2bff817f00c53dc0
2012-12-13 10:09:46 +01:00
Jenkins
9202a8457a Merge "Port to argparse based cfg" 2012-12-12 23:30:43 +00:00
Mark McLoughlin
5e09b2cab7 Port to argparse based cfg
Import latest cfg from oslo-incubator with these changes:

 Add deprecated --logdir common opt
 Add deprecated --logfile common opt.
 Allow nova and others to override some logging defaults
 Fixing the trim for ListOp when reading from config file
 Fix set_default() with boolean CLI options
 Improve cfg's argparse sub-parsers support
 Hide the GroupAttr conf and group attributes
 Fix regression with cfg CLI arguments
 Fix broken --help with CommonConfigOpts
 Fix ListOpt to trim whitespace
 updating sphinx documentation
 Don't reference argparse._StoreAction
 Fix minor coding style issue
 Remove ConfigCliParser class
 Add support for positional arguments
 Use stock argparse behaviour for optional args
 Use stock argparse --usage behaviour
 Use stock argparse --version behaviour
 Remove add_option() method
 Completely remove cfg's disable_interspersed_args()
 argparse support for cfg

The main cfg API change is that CONF() no longer returns the un-parsed
CLI arguments. To handle these args, you need to use the support for
positional arguments or sub-parsers.

Switching nova-manage to use sub-parser based CLI arguments means the
following changes in behaviour:

 - no more lazy matching of commands - e.g. 'nova-manage proj q' will
   no longer work. If we find out about common abbreviations used in
   peoples' scripts, we can easily add those.

 - the help output displayed if you run nova-manage without any args
   (or just a category) has changed

 - 'nova-manage version list' is no longer equivalent to
   'nova-manage version'

Change-Id: I19ef3a1c00e97af64d199e27cb1cdc5c63b46a82
2012-12-11 06:47:35 +00:00
Chuck Short
c454606ada i18n raise Exception messages
Provide i18n to some exceptions that were not
provided before.

Change-Id: Ia320ba5f12e82686c8ee7973a77f152eb4b02bd4
Signed-off-by: Chuck Short <chuck.short@canonical.com>
2012-12-10 14:43:18 -06:00
Monty Taylor
70965c3ec5 Use fixtures library for nova test fixtures.
Moving the test-case setup and teardown code into managed fixtures
so that we can better interact with them when we start running things
in parallel.

Part of blueprint grizzly-testtools

Change-Id: I406be0a88b14c0dba2d22b4957e26a53442bbae3
2012-12-07 23:14:58 -08:00
Andrew Glen-Young
963fcdb673 Fix regression allowing quotas to be applied to projects.
Change-Id: I9c7dfa7664badcafb800c02ee608699e56b344ab
Fixes: bug #1084261
2012-12-06 10:22:35 -05:00
Joe Gordon
59f6d3b63b Improve nova-manage usability
Prevent nova-manage from stack tracing when the wrong number of
arguments are passed in.

Fixes bug 1083315

Change-Id: I7b9a237a82b4255d88c01c6a56ad0ccd4a6ee984
2012-12-06 13:28:15 +02:00
Thierry Carrez
12e264d58f Configurable exec_dirs to find rootwrap commands
Adds support for a configurable set of trusted directories to search
executables in (exec_dirs), which defaults to system PATH. If your
filter specifies an exec_path that doesn't start with '/', then it
will be searched in exec_dirs. Avoids having to write multiple
filters to care for distro differences. Fixes bug 1079723.

Also returns a specific error rather than try to run absent executables.

Change-Id: Idab03bb0be6832a75ffeed4e78d25d0543f5caf9
2012-12-03 15:12:06 +01:00
Dan Prince
47ddb8ca76 Validate rxtx_factor as a float.
Updates the rxtx_factor validations in the instance_types modules so
that rxtx_factor is validated as a float. Given that rxtx_factor is stored
as a float in the database this makes sense... and also adheres
to some of the extension documentation as well (although some of
the extension docs were incorrect as well).

Previously rxtx_factor was being cast into an int which caused it to
be stored and displayed incorrectly via the API.

This patchset adds a test which fails with the old code.

Additionally some of the API docs are corrected so that rxtx_factor is
listed as a float in all examples.

Fixes LP Bug #1081287.

Change-Id: Iae98522a1f2ed63cbd2497b0b0af5ac2d9bb935c
2012-11-27 11:12:41 -05:00
Mark McLoughlin
baa1006747 Make nova-dhcpbridge use CONFIG_FILE over FLAGFILE
nova-dhcpbridge's FLAGFILE environment variable is one of the few
remaining references we have to "flags".

Switching to use a new environment variable is easy, but we need
to take care that when you upgrade the code the compute and network
services might not be restarted immediately.

If this happens, the services may launch a new dnsmasq (so rootwrap
needs to continue to support FLAGFILE) or an existing dnsmasq might run
the lease script (so the script needs to support FLAGFILE).

Change-Id: Ieb55daa2507abb8327c7f5a2ef2f682740f31a5d
2012-11-26 07:53:38 +00:00
Mark McLoughlin
d3c68d6e63 nova-dhcpbridge should require the FLAGFILE is set
We always launch dnsmasq with FLAGFILE set, so something is very broken
if FLAGFILE isn't set when nova-dhcpbridge is launched.

Also, the fallback to CONF.dhcpbridge_flagfile is a little strange - at
this point, we haven't parsed any config files so the only possible
value of this config option is its default value.

There's no real need to gracefully handle this condition either - a
KeyError is a straightforward enough error.

Change-Id: I734d7b739f17c8a9ab48d8ba13baacff3f00e1c4
2012-11-23 12:15:22 +00:00
Mark McLoughlin
e237c86446 Use CONF.import_opt() for nova.config opts
The only reason for importing nova.config now is where one of the
options defined in that file is needed. Rather than importing
nova.config using an import statement, use CONF.import_opt() so
that it is clear which option we actually require.

In future, we will move many options out of nova.config so many
of these import_opt() calls will either go away or cause a module
other than nova.config to be imported.

Change-Id: I0646efddecdf2530903afd50c1f4364cb1d5dce1
2012-11-20 00:04:49 +00:00
Mark McLoughlin
94d87bce21 Remove nova.config.CONF
Modules import nova.config for two reasons right now - firstly, to
reference nova.config.CONF and, secondly, if they use one of the
options defined in nova.config.

Often modules import nova.openstack.common.cfg and nova.config
which is a bit pointless since they could just use cfg.CONF if
they just want to nova.config in order to reference CONF.

Let's just use cfg.CONF everywhere and we can explicitly state
where we actually require options defined in nova.config.

Change-Id: Ie4184a74e3e78c99658becb18dce1c2087e450bb
2012-11-20 00:00:21 +00:00
Dan Smith
4ea757b880 Add the beginnings of the nova-conductor service
This adds a new service called "nova-conductor" which will provide
services for nova-compute, such as completing database updates and
handling long-running tasks.

Right now, this just adds a fraction of the database-related work
so that we can proceed with incremental changes to nova-compute.

For compatibility/flexibility, both LocalAPI and (RPC)API classes
are provided. Users concerned with performance or compatibility
who wish to maintain compute-node-local database accesses can
set the [conductor] use_local=True, which will mimic the legacy
behavior.

DocImpact

Change-Id: Ib81d879f40b7daceeffad5af0631c7db68168661
2012-11-19 11:41:34 -08:00
Chris Behrens
e374fb0389 Remove nova.flags imports from bin/*
nova.flags is no longer needed except to load nova config options shared
across multiple daemons.

This removes nova.flags from the bin programs and makes sure that
nova.flags is imported in nova.config for now.

Change-Id: If066ac0070387bee4b41e6a78ad972f7a0955c75
2012-11-15 23:03:04 +00:00
Jenkins
39958a83a9 Merge "Fix import ordering in /bin scripts." 2012-11-09 18:05:55 +00:00
Dan Prince
a3318d88f2 Fix import ordering in /bin scripts.
Fix a couple hacking violations in Nova's bin scripts.

Change-Id: I5cab72a5a94f761f429ad45f94a95cd4bc7e30e8
2012-11-09 12:17:13 -05:00
Joe Gordon
29f7bdc045 Remove nova-volume DB
*  Remove volume calls from nova.db.api
*  Remove volume calls from nova.db.sqlalchemy.api
*  Remove tables from nova/db/sqlalchemy/models.py
*  Remove Xen Storage Manager Volume Driver code
*  No migration to preserve data in unused tables
*  Remove quota support for volumes

DocImpact

part of bp delete-nova-volume

Change-Id: I2c82c96d67f3746e5de28f917dd8ceb0c8037e27
2012-11-08 21:31:31 +00:00
Jenkins
aa88eaba6d Merge "Switch from FLAGS to CONF in bin" 2012-11-06 21:39:50 +00:00
Jenkins
52cf3cae69 Merge "Remove flags.DECLARE" 2012-11-06 19:12:46 +00:00
Jenkins
f45c67e356 Merge "Fixes volume attach issue on Hyper-V" 2012-11-06 16:02:00 +00:00
Alessandro Pilotti
bcf8af965e Fixes volume attach issue on Hyper-V
Fixes Bug: #1074717

During volume attach on Windows a process is spawned. The process
currently fails with the following exception, due to eventlet monkey
patching of the os module.

NotImplementedError: set_nonblocking() on a file object with no
setblocking() method (Windows pipes don't support non-blocking I/O)

The fix consists in avoiding monkey patching the os module in
nova-compute on Hyper-V

Change-Id: Ib513bc218da83b751439bad114baac24b08e3d02
2012-11-05 17:12:24 +02:00
Mark McLoughlin
5749b8deb0 Switch from FLAGS to CONF in bin
Use the global CONF variable instead of FLAGS. This is purely a cleanup
since FLAGS is already just another reference to CONF.

We leave the nova.flags imports until a later cleanup commit since
removing them may cause unpredictable problems due to config options not
being registered.

Change-Id: Id0c59b2dc8002ec89ccbc5e5d7986fb68d3a693d
2012-11-04 21:39:49 +00:00
Mark McLoughlin
8ce58defbe Remove flags.DECLARE
The cfg.ConfigOpts class has an equivalent method, so lets use that.

Change-Id: I5860230336d00d7531a0ffd255c766ff77661625
2012-11-04 21:38:38 +00:00
Mark McLoughlin
efede80046 Move parse_args to nova.config
The flags module will eventually be removed and this is a first step
towards that.

Change-Id: I729b08900e53e2ae6db10633dcff3be59720fa6f
2012-11-04 21:37:39 +00:00