When there's 2 options with the same name, the script gets confused and
doesn't know in which group it goes. This fixes it, and fixes also the
template generation which is broken for various reasons:
- gettext.install was missing
- nova-rootwrap when imported calls exit() because it doesn't find a valid
configuration file, so let's exclude it entirely anyway
- eventlet/greendns needs to be ignored for this
Change-Id: Iaa4e9a806e79032ce1675b46a6b7a7628c3eff89
Signed-off-by: Julien Danjou <julien@danjou.info>
This prepares for having the conductor query the scheduler for a list of
hosts to build an instance on. In order to accomplish this the API
sends a build_instances message to conductor, or spawns a greenthread in
local mode, rather than sending a message to the scheduler. This is
being done so that conductor can handle the orchestration of long
running tasks like spawning instances. By making this move, the API is
free to return to the caller while conductor queries the scheduler for a
host to provision to.
In the case of cells the build_instances message first goes to the
cells scheduler in order to pick a cell to send it to, and then it is
sent to the conductor in that cell.
Part of bp query-scheduler
Change-Id: I4539888e78ebdbb8cef6647273b959a012280110
Mechanical translation of the deprecated
except x,y: construct with except x as y:
The latter works with any Python >= 2.6.
Add Hacking check.
Change-Id: I845829d97d379c1cd9b3a77e7e5786586f263b64
It's ugly, and potentially racy, to mess around with other modules'
global variables. Instead, pass CONF.tempdir into tempfile.mkdtemp
as the 'dir' keyword argument.
Because we already pass **kwargs to mkdtemp, inspect **kwargs and
only set 'dir' if it's not already there.
Change-Id: I8a2b34cd051919db29facabc1664cf357073b1d7
This will be necessary as we start to introduce objects with
real dicts in system_metadata. These are the common spots that
need to be ready ahead of time. The rest are at the actual use
sites and can/will be replaced when individual uses are
converted to objects.
Related to bp/unified-internal-objects
Change-Id: I8314e0d52ec2ae800765f60ce58ce9b309d7d513
Oslo's `bool_from_string` learned the `strict` keyword which allows
callers to detect invalid boolean values, so we can use that instead
of having a new Nova-specific function.
Change-Id: I61bfa4029897c7304bd54d6cdae9f9a9bc4c1f78
Oslo provides an equivalent implmentation of `bool_from_str` so we
should switch the code to use that instead.
Change-Id: I382f23af2468e276ae4342dff18cf06e1c24b755
The lockfile module has a new convenience API which sets the lockfile prefix.
Using this API, the prefix is not required everytime synchronized is used.
Change-Id: Iac1cfcc83b59108164de924d20127c1cf4dd7dcd
This change moves nova over to using the newly imported trycmd() and
ssh_execute() implementations from oslo's processutils.
Change-Id: Ied2bb1eeb0cbdc6903a4028a60c89ca9e23656b4
There are some small warts (mostly around other execute() like
methods which reuse the exception). I will fix these warts in later
reviews.
Change-Id: Ice9cdbdc5f3e5a9f8365f5d99acf1863a9fe3e7a
Import the oslo looping call implementation (which is a copy of
nova's), delete nova's local copy, convert all users to the new
location.
It should be noted that the oslo implementation of
FixedIntervalLoopingCall measures time from the start of the
periodic task, not the end, so periodic tasks will run with a
constant frequency instead of the frequency changing depending on
how long the periodic task takes to run.
Change-Id: Ia62ce1988f5373c09146efa6b3b1d1dc094d50c4
When we parse incoming XML safely, if there was an error raised it would
be an expat.Expat() error, which would bubble up to the api and turn
into a HTTP 500 (Internal Error)
It turns out that all the places we use the safe_xml parsing are in
Deserializers, close to the API, so in this patch we just change the
error it raises straight to nova.exception.MalformedRequest().
This causes the api to fail with the proper 400 (Malformed Request) when
it encounters corrupt XML. This is caught at
nova.api.openstack.wsgi._process_stack and __call__.
We also take the opportunity to move the new safe parser from nova.utils
to nova.api.openstack.xmlutil as the openstack api is the only thing
that uses it.
Fixes: bug #1133111
Change-Id: Ifa2ed7ee128241cfe8dbcdc5bd75194d96b6cdb5
Note that some tests were verifying things that can no longer fail, namely
rebuilding instances who use flavors that have since been disabled. These
tests are removed here.
Also, this changes the order of a piece of logic in the resize/migrate
path where before we would have ended up checking the instance type that
we fetched from sys_meta to see if it was disabled (which makes no sense
now). Reversing the check (to see if we're doing a resize or a migrate
before checking the new flavor) eliminates the problem of the stashed
type not having the 'disabled' attribute.
This is one change in a series aimed at removing the use of instance-linked
instance_type objects, in favor of the decoupled type data in
system_metadata. See bug 1140119 for more details.
Change-Id: I214a693e3bb16c0a365eb9b3afe73601beba4a22
compute.manager methods often takes password parameters in methods
that are wrapped with exception handlers and notifiers. What can
happen is these passwords will get bundled up and sent outside of
the system.
This patch will strip out any parameter with *_pass* in the name.
The side effect of this is that all notification will have the
error parameters in the 'args' part of the notification payload.
Previously only the positional args were in the 'args' part and
keyword args were placed in the payload directly.
This may affect consumers of the error notifications.
Change-Id: I2e7822eb5416d315ceb690f739e4dba9d52a7954
Details:
- Use the nova/utils.py validations along the code
- refactor ipv4 validator to looks like ipv6 validator
- interprets ip validations exceptions as Invalid IP values
Change-Id: Ia506c00510a066e167d3dcd9dd371a371129a9dd
For nova, forbid_dtd is going to be true always, however
if someone picks up this code and tries forbid_dtd = False
then the existing code is not good enough. we need to protect
against external entities/dtd and not allow notations as well.
Added a few more handlers and test cases to cover that use
case.
Change-Id: If50f690e015f2bf837b403edf552e35d7af8c907
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.
Add the 2013.1b4 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.
Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.
Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.
Make tools/hacking.py include oslo in IMPORT_EXCEPTIONS like it already
does for paste. It turns out imp.find_module() doesn't correct handle
namespace packages.
Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).
Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
Adds a new utils.safe_minidom_parse_string function and
updates external API facing Nova modules to use it.
This ensures we have safe defaults on our incoming API XML parsing.
Internally safe_minidom_parse_string uses a ProtectedExpatParser
class to disable DTDs and entities from being parsed when using
minidom.
Fixes LP Bug #1100282.
Change-Id: Ib90d6379320ff1d007f8a661f7ddaa286ba6918e
it turns out that the 149 migration was overly agressive in its
IPAddress conversion, as we actually have a few columns that are
really CIDR values. This means that 39 chars isn't enough space to
store even a normalized IPv6 cidr in the worst case (you need 4
more to support /127).
We must also normalize IPv6 address cidrs otherwise 43 chars isn't
long enough. This was more of a problem in theory, than in practice,
as real IPv6 addresses rarely are non compressible.
This adds a migration to bump up the CIDR columns to 43 characters.
There is an infinitessimal chance that someone using mysql and long
IPv6 values might loose data in 149 because of truncation. This
doesn't address that, which would require modifying 149.
The native pg CIDR column type is not used because it would require
additional scrubbing of the data as CIDR is invalid if any host bits
are set (and it will fail on type conversion).
Fixes bug #1127696
Change-Id: I54539ac9c257d726bc4db5943169b5284cc847d3
This is normalize IPv6 to be inserted always in shortened and no mixed form
into db, this way postgresql, which uses INET type, and other databases like
mysql will have equivalent contents.
Fix bug 1116236
Related to bp migration-testing-with-data
Change-Id: Iae5aa8a28e3ccc0d3a1a96459232b827c3a19d5c
Fixes bug 1102280
The length of flavor name is defined 255 charaters in database.
But flavor name can be more than 255 characters when a flavor
is created. This patch adds the length check of flavor name.
Change-Id: If9db879e5f6340594b215b057a29d03c6fef1503
Most of the allocation for floating ips can be done on the calling
side, including finding the proper host to send the message to.
This saves us from making an rpc call for allocate/deallocate and
makes sure that we only need 1 call for associate/disassociate by
finding the proper host to send the message to immediately.
Getting exceptions to work properly required pulling in the helper
that was used by the conductor to regenerate exceptions that are
wrapped for rpc. Since this is now a shared class, it was moved
to utils.
Also a few config options were moved to avoid circular imports.
Part of blueprint optimize-nova-network
Change-Id: I6ec65b1f3e8d00cab778b10eec620760886567e0
This function was used with deprecated auth and is no longer used
anywhere in the code, so just remove it.
Change-Id: I1e5472c312bfc49c57bf031593cf82c32bc77dfd
The default value of monkey_patch_modules has not kept up with
the move of notifier to openstack.common
Without this change setting 'monkey_patch=True' in nova.conf
will cause runtime errors unless the monkey_patch_modules
is also specified pointing to the correct notifier
Change-Id: Ib22faa7404b09791799338de2edfe0ddcd662d5f
This patch reimplements the is_valid_ipv4() function is nova.utils.
Instead of open-coding the validity check, just make use of the netaddr
module, which is already used elsewhere in nova.utils.
Also add a unit test for this code.
Part of indigoprint bored-on-an-airplane.
Change-Id: Ia89cbbd94a0ac4631d794d658b53c244237e1ca2
This function used a few backslashes for continuation. According to
HACKING, we prefer to not use this. This code gets rid of them by
changing the way the checks are done.
Look at those backslashes ... sitting there at the end of the line.
It's like they're looking down on the code that precedes them, as if
they are somehow superior to the other characters. Banish them from
this utility function!
Also add a unit test for this code.
Part of greenprint bored-on-an-airplane.
Change-Id: If88b6e63ab078916ce1b9cf2f5e99623c402996c
This makes the system_metadata updated in-place like regular
metadata during instance_update(). It also modifies that function
to avoid actually removing the item from the list, which will cause it
to be deleted from the database when the session ends. The soft_delete
is sufficient for our purposes.
Related to blueprint no-db-compute
Change-Id: I0da28ec485dc7850d246dab0a2bb95ae10c05c3e
Record when an action is initiated on an instance, and the underlying
events related to completing that action.
Actions will typically occur at the API level and should match what a
user intended to do with an instance. Events will typically track what
happens behind the scenes and may include things that would be unclear
for a user if exposed, but should be beneficial to an admin/deployer.
Adds a new wrapper to the compute manager. The wrapper will record when
an event begins and finishes from the point of view of the compute
manager. It will also record errors if they occur.
Blueprint instance-actions
Change-Id: I801f3e796d091e146413f84c2ccfab95ad2e1af4
* Removes extra call to get block-device-mappings when detaching during
recreate.
* Untangle network handling from block-device handling in rebuild
* DRYs up evacuate hosts tests
* Temporary mutation supports dicts as well as objs
* Make instance recreate exceptions more specific to aid testing
Change-Id: I242f9f6a7d329f27b06b7e81b9a418e01682c82e
Fixes bug 981104
Although the temporary directory used can be controlled via
environment variables, this patch provides a way to define it
explicitly via a config option.
The default value is None, which behaves per the doc below:
http://docs.python.org/2/library/tempfile.html#tempfile.tempdir
Flags: DocImpact
Change-Id: I47b6d8bac734f453c80d541b46086a30e847c859
The method info_from_instance() in nova.notifications was doing a db
lookup for the system_metadata for an instance. This patch updates that
code to get that data from the instance that's passed in instead. The
rest of the patch are related changes to make that happen.
metadata_to_dict() was needed here. It lived in nova.compute.utils.
nova.compute.utils already imported nova.notifications, so using it from
there would have created a circular import. Move the method to
nova.utils instead and update the tree to use it from its new location.
I also noticed that the xen driver had a copy of metdata_to_dict().
This patch removes it and uses the common implementation in nova.utils.
'system_metadata' was added to _extra_keys of the Instance db model so
that it would show up in a serialized instance. Tests failed without it
as the result of getting instances via the conductor API did not include
system_metadata. Now it's there.
Part of bp no-db-compute.
Change-Id: I451355fb26ae29f13b71438f7896c448b59f97b0
Use @markmc's suggestion to enhance LazyPluggable with an
optional config group. Also fix the baremetal database
backend option to use the "baremetal" config group.
Fixes LP #1093043
Change-Id: I28cf51a2962f516fcef4ced19e30c985220e86dc
the get() function takes an arbitrary id coming from the rest url
for the server. In our current code it checks if this is a proper uuid
and sends it down the uuid path, everything else gets dispatched to
the version that selects by integer id. This means that arbitrary
garbage fuzzed server ids will get sent down the int path, all the way
to the database.
In postgresql, where the db is strongly typed, this causes a type error.
This error was found by tempest nightly runs where we send some 35 and 37
character strings in.
This patch creates and equivalent is_int_like function. If the server_id
is neither uuid_like nor int_like, we throw the InstanceNotFound exception
early. This also saves us a trip to the database in these cases.
Make the is_int_like a little more robust, and don't succeed on floats
Once more with feeling, to let us actually use is_int_like on ints, not
just strings.
Fixes bug #1100253
Change-Id: If4ae8005fd33a23ac50a6408ecd5933a2ff6425c
