311 Commits

Author SHA1 Message Date
Alex Kavanagh
50a7253b80 Fix guardmap as a function object issue in service_guard()
The issue was the original commit had a bug (using the wrong variable),
and also that there were no tests to verify the functionality.  This
patchset fixes the bug and adds tests to verify the contract.

Change-Id: Ia8b5580132c5a592461a57fb514994ab64a9ed42
Closes-Bug: #1805128
2018-11-26 12:44:01 +00:00
Liam Young
c88155d17a Purge old packages on upgrade-charm
On charm upgrade the charm may switch to py3 packages. If so, ensure
the old py2 packages are purged. If the purge occurs then restart
services.

Change-Id: I984a227b3fe12a0086c926ae69c27d6e4d9741d3
Closes-Bug: 1803451
2018-11-15 12:52:44 +00:00
Liam Young
314aff9e53 Fix type error when encoding certificate
The keystone_ca_cert_b64 function retrieved the CA cert from a file
before encoding it but it was passing a str to the b64encode which
requires a bytes-like object.

Change-Id: Iafaf5916b04746eb045fcd3dfe9676a80c88b464
2018-10-30 17:13:50 +00:00
Zuul
40ef19e78e Merge "Switch the charm to support py3" 2018-10-23 16:35:53 +00:00
Alex Kavanagh
9c12812735 Switch the charm to support py3
Some major changes:
* the charm has been rebased (from a Python perspective) to be rooted in
  the charm directory.  This is a single root.
* Imports have been changed so that the don't add lots of imports to the
  namespace of the module doing the import.
* The code that used to run at module import time has been made lazy
  such that it only has to run if the relevant functions are called.
  This includes restart_on_change parameters, the harden function and
  the parameters to the guard_map.  Appropriate changes will be
  submitted to charm-helpers.
* Several tests had to be re-written as (incorrect) mocking meant that
  text fixtures didn't actually match what the code was doing.  Thus,
  the tests were meaningless.
* This has had a net positive impact on the unit tests wrt to importing
  modules and mocking.

Change-Id: Id07d9d1caaa9b29453a63c2e49ba831071e9457f
2018-10-18 15:43:03 +01:00
James Page
1d8946151a py3: Don't purge or install python-six
Drop python-six from the list of packages to install/purge on
upgrade to py3 workload execution.

This was a legacy requirement in obsolete packaging versions
of Nova.

Purging six has the side effect of removing crmsh from the unit
in clustered deployments.

Change-Id: I9dfe7b031562c8f3e85445486ba16b3c7b2465ca
2018-10-15 14:45:01 +01:00
Liam Young
0f9ada1713 Fix assess status for super-conductor
In a cells deployment its possible that the top-level
nova-cloud-controller will have no compute nodes associated with it.
This change fixes the work load status in that scenario.

There is also a drive by fix to return empty contexts for the cell
db and amqp contexts if the relation do not yet exists.

Change-Id: Ia8eeccb6794dd016185eb0cfb05339b76cef9348
2018-10-09 10:32:51 +00:00
Liam Young
6695d79c95 Add support for cells v2
This change adds relations necessary for registering a compute cell
with the superconductor. For a cell to be registered this charm
must have relations with the compute cells conductor, database and
message queue. Only when all these relations are complete can the
registration happen. Below are major changes included in this PR.

* Add nova-cell-api relation for communicating with the
  nova-cell-conductor
* Add shared-db-cell relation for communicating with the
  a compute cells database.
* Add amqp-cell relation for communicating with the
  a compute cells message queue.
* Add methods for registering cells with the
  superconductors database.
* Charm helper sync

Change-Id: Ic6ddc29426319b98b147c29031f60485fccc513f
2018-10-05 11:50:21 +00:00
Corey Bryant
d5c5cccb0a py3: Switch to using Python 3 for rocky or later
Switch package install to Python 3 for OpenStack Rocky or later.

When upgrading, remove any python-* packages that were explicitly
installated and then autoremove --purge any dependencies that are
no longer required.

Change-Id: I87ee4a0a854f88b931093f6245a3a1057bf56c57
2018-10-04 11:19:06 +01:00
Liam Young
e20db83c7d Add nova-metadata service
Add a service for handling nova metadata api services. This was
previously handled by the neutron-gateway and still is for
deployemnts up to and including Pike, For the neutron metadata
service and the nova service to communicate they need a shared
secret. To achieve this, the change includes:

* A charmhelper sync to get support for multiple wsgi vhosts
* Rendering new wsgi vhost and corresponding haproxy config.
* Setting a shared-secret down the relation with the neutron
  gateway.
* Remove fragile keystone authtoken checks as they are failing
  after a ch sync and any issues will be caught by the instance
  launch functional test.

Change-Id: I5ad15ba782cb87b6fdb3c0941a6482d201670bff
2018-10-03 07:24:05 +00:00
James Page
935d30571c Use v2.1 API for >= Queens
Switch to using the v2.1 API endpoint for OpenStack Queens or later.

Closes-Bug: 1794271

Change-Id: Ia21b3a6782ab944b7c41564e889e6b12d264b731
2018-09-27 15:08:17 +01:00
James Page
36ccf4ee97 Use lowercase hostnames for SSH know hosts
OpenSSH will lowercase any hostname; ensure that hostnames
for known_host entries are also lower case avoiding any
authenticity of host type issues during live migration and
resize operators.

Change-Id: Ie5ab0774b49fc0d753ff1c26eb041f1ceb35e8fb
Closes-Bug: 1628216
2018-07-13 15:55:54 +01:00
Liam Young
21b17020d8 Remove support for nova cells v1
This change removes support for nova cells v1.

Change-Id: Id86697b6a2e893e023769ea214671e958e7166ca
2018-06-15 05:57:52 +00:00
Frode Nordahl
edc18d4937
Make charm agnostic of underlying init system
Replace charm custom init control functions with `service_pause`
and `service_resume` functions from charm-helpers.

Change-Id: I235af30a19294316f65fba0e13fe10ae50164a42
Closes-Bug: #1765215
2018-04-20 19:10:59 +02:00
Billy Olsen
d43e264d02 Sync charm-helpers and use VolumeAPIContext
Sync in the charm-helpers to use the new VolumeAPIContext object
in order to determine the volume catalog info to use in the
configuration file.

This is simply an alternative implementation for commit 16ae06bf
which separates the concerns for volume endpoint context and
internal endpoint context.

Change-Id: I187bc8ef5644a59dc5a0b5f7ad66774812a78b78
Related-Bug: #1733566
2018-02-08 19:01:44 -07:00
Liam Young
a0778a5304 Only run map_instances for Ocata
map_instances only needs to be run during the transition from
Newton to Ocata to map existing instances into a cell. All
new Ocata instances will be automatically mapped to a call.
This change limits the slow running map_instances command to
just run for upgrades to Ocata

Change-Id: Ic2e2df530504284d28cfcab26a71d211342203fa
Closes-Bug: #1743357
2018-01-23 13:27:00 +00:00
Liam Young
c276dfdb96 Batch up map_instances call
nova-manage map_instances maps all instances into a cell in batches
of 50 if max-count is not set. Setting max-count causes the script
to run a single batch of size max-count. The return code of the
script shows if there are still more to do. This change runs
map_instances repeatedly with a batch size of 50000 while rc is 1
and then exists cleanly when a rc 0 is recieved.

Change-Id: Id1184778a5ae94bb3b57348b10d12077b093d6dd
Partial-Bug: #1742115
2018-01-17 14:43:10 +00:00
Liam Young
edd9b1face Add action for running archive-deleted-rows
Add an action for moving stale data to shadow tables using
nova-manage *1. This will speed up other operations such as
map_instances which no longer need to work against stale
data.

*1 https://docs.openstack.org/nova/pike/cli/nova-manage.html

Change-Id: I03f3d641b50cfc6f02262edb0f714ba6e9566775
Partial-Bug: #1742115
2018-01-12 14:11:16 +00:00
James Page
0f14eac672 Remove deploy from source support
Drop support for deployment from Git repositories, as deprecated
in the 17.02 charm release.  This feature is unmaintained and has
no known users.

Change-Id: I2fe15b648d485e5b03965a00dee6324669ebe9fa
2018-01-12 10:42:50 +00:00
James Page
5c79af1bd1 Drop postgresql support
Remove postgresql DB support; This feature is untested as part
of the charms, is not in use and was deprecated as part of
the 1708 charms release.

Change-Id: I866559f519ace3476e0cc85661d99e5f5491227d
2017-12-21 12:01:58 +00:00
James Page
0196625084 Drop zeromq support
Support for the ZeroMQ messaging driver has bit-rotted over
the last few years across the OpenStack charms; drop support
for ZMQ inline with deprecation notices issued in 17.02 charm
release.

Change-Id: I39e464b289c9df842ca73e64c7576ba0531963ae
2017-12-15 17:27:01 +00:00
Edward Hope-Morley
16ae06bf42 Use volumev3 for internal endpoint in >= Pike
volumev2 api is deprecated for >= Pike so switch
internal endpoint to v3 for Pike onwards.

Change-Id: Icb9861efbd650b079f507ca585818e95921dc42a
Partial-Bug: 1733566
2017-12-08 10:06:49 +00:00
Corey Bryant
3b873932be Add nova-manage failure verbosity and clean up Cells V2 code
Update all nova-manage commands to use subprocess.check_output()
and log subprocess.CalledProcessError.output on failure. This
will help capture nova-manage error details on first failure.

Specify cell1 uuid on discover_hosts call. This doesn't change
behavior, it is just more explicit and useful if we move to
multiple cells in the future.

Introduce an is_cellv2_init_ready() function that uses contexts to
check if cells_v2 is ready to initialize. This cleans up the
corresponding TODOs.

Move checks for cell v2 init readiness to update_cell_db_if_ready(),
also cleaning up corresponding TODOs.

Change-Id: I313edce84d3d249031e020a4fbb4baf216c01ddb
Related-Bug: 1720846
2017-10-16 12:19:55 +00:00
David Ames
0161dde796 Resolve OS upgrade bugs
Ensure that the os_release cache is cleared during the openstack
upgrade process, ensuring that package list and configuration
options are correctly set for the new OpenStack version.

Ensure that map_instances gets run on upgrade. This updates the
nova_api.instance_mappings table with pre-existing instances.

Change-Id: Idfcdc48c25b24d0cc9ded3eda2bc4d13d3b04f6d
Closes-Bug: 1715624
2017-09-07 14:34:55 -07:00
David Ames
0681179aa3 Reset os_release after an upgrade
determine_endpoints was failing to recognize the new OS release after an
upgrade. Specifically newton --> ocata failed to register the
placement api.

Reset os_release after an upgrade.

Change-Id: I04ae30aea435c8c39f58d1a814d070a057bb5ce0
Closes-Bug: #1715505
2017-09-06 15:56:27 -07:00
Felipe Reyes
eacd234ed5 Run nova-manage db online_data_migrations on upgrades
In mitaka an extra operation called online_data_migrations was added to
assist rolling upgrades.

This patch runs this before and after, this is to make sure the old
upgrades where this migration was not being performed are run before
installing a newer release.

References:
- https://git.openstack.org/cgit/openstack/nova/commit/?id=7d5069ce20ec0d792a3974b2c53d01ae005cde98
- https://docs.openstack.org/nova/latest/user/upgrade.html

Change-Id: Ic4bc2c5ae86c99c3af2a1d0ee08badb09835d932
Closes-Bug: 1711209
2017-09-01 17:45:47 -03:00
Jenkins
f185d001a3 Merge "Drop nova-cert >= newton" 2017-06-26 14:08:24 +00:00
James Page
e8f5341034 Drop nova-cert >= newton
nova-cert was deprecated @ newton, and only supports ec2 services
which we dropped in the charms @ liberty so remove installation
and configuration of this service for >= newton.

Change-Id: I6d0ed6a1e058f38cb5d556a34b6f4299bf6d2055
2017-06-23 13:59:24 +01:00
Jorge Niedbalski
51b7a600c8 Evaluate 'None' as None in console_attributes.
The config.yaml file suggests None as one
of the possible values of the directive console-access-protocol,
this is wrongly evaluated as a string in the console_attributes
function, which causes the charm to install the pacemaker
resource even when is not required (console-access-protocol = None).

Change-Id: I367b7fd813e044bc6eedca687e837e8adab2dd40
Closes-Bug: #1697693
Depends-On: I98ebbf9f2c7582fae5f466dfc1d224bf66e38a24
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@canonical.com>
2017-06-16 19:20:40 +00:00
Jorge Niedbalski
0d28006e1c Disable the nova-consoleauth service on HA.
When using the nova-cloud-controller charm in HA with the config
option single-nova-consoleauth set to true, its expected for the
nova-consoleauth service to be run in just a single unit at the time.

The service management (start/stop) is performed by pacemaker in
accordance with the cluster health using the OCF resource agent[0].

Its required for the service to be disabled by default on upstart (trusty)
or systemd (>=xenial).

This change disables the service by using the service_pause
charmhelpers call which considers both cases (upstart/systemd) when
the ha relation is present and the single-nova-consoleauth option is
used.

Also, this change fixes LP: #1660244 (Services not running that should be:
nova-consoleauth) by removing it from the resource_map when
ha + single-nova-consoleauth is used.

[0] https://github.com/openstack/openstack-resource-agents/blob/master/ocf/nova-consoleauth

Closes-Bug: #1693629
Closes-Bug: #1660244

Change-Id: Iaffe0456cceb42ee124cb8881d3379d78cac0f3a
Signed-off-by: Jorge Niedbalski <jorge.niedbalski@canonical.com>
2017-05-30 20:13:46 -04:00
Alex Kavanagh
f9b97ac66c Fix alphanumeric comparisons for openstack and ubuntu releases
- sync charmhelpers with fix-alpha helpers
- fix up code where the alpha comparisons are done
- fix tests which assumed mocks would just work on os_release()

Change-Id: I48b4853ed343bd3188c16f2bc432b4ca0badc473
Related-Bug: #1659575
2017-04-27 10:52:43 +01:00
Corey Bryant
7dd8357b06 Enable SSL termination for nova-placement-api
determine_ports() was not adding the nova-placement-api port to the
ports list because it is run under the apache2 service. This fix
enables determine_ports() to get 'nova-placement-api' from the list
of services rather than 'apache2', allowing it to determine the port
that nova-placement-api uses.

Also enable tests/gate-basic-xenial-ocata in this commit.

Change-Id: I8113e911272ee671026bace9f6e0e8c4ce072e6b
2017-03-01 14:27:35 +00:00
Corey Bryant
1e2647a9b5 Enable minimal cells v2 support
A minimal cells v2 setup is required in Ocata, which includes the
following databases:
 1. An API database (named nova_api)
 2. A special cell0 database (named cell0)
 3. A "cell1" database (named nova - we're using the nova db as cell1)

The order of nova-manage commands is also updated in this commit.
The 'nova-manage api_db sync' must occur first because 'nova-manage
cell_v2' commands use the API database. 'nova-manage db sync' must
then be run after the 'nova-manage cell_v2' commands.

Finally, 'nova-manage cell_v2 discover_hosts' must be run whenever a
new compute node is introduced to add the host to cell1.

This commit includes a sync of charm-helpers to pick up AMQPContext
updates for transport_url.

Change-Id: Ia6f36ca8a360dc8490e9f41b62d499fa4d73d0b9
2017-02-16 18:23:21 +00:00
Corey Bryant
2564cf5a5f Enable nova-placement-api
This change installs the nova-placement-api, runs it under
mod_wsgi with apache2, updates nova.conf accordingly, and
registers its endpoints.

Change-Id: I4af4afa591cf68964e6146ca0ca0c51d1ddb8a64
2017-02-09 21:30:13 +00:00
Frode Nordahl
2eef644a5c Revert "Update policy.json to give service role access"
This reverts commit ec2579a8448281cdf1154018c0a56c9d4b174e36.

We are not quite ready for fine grained RBAC for service
accounts.

Change-Id: I210685d7b9036abb191073d512f8a65ebff30613
Closes-Bug: 1655028
2017-01-11 15:33:11 +01:00
Felipe Reyes
020e22eeae Allow to use any repository in openstack-origin to upgrade
do_openstack_upgrade() assumes that a cloud archive repository
will be used to upgrade the unit, this is not true for environments
where a mirror is used or a PPA.

Change-Id: I96a20e926f42dc5df4ddb3cb7ac5428b882711e8
Closes-Bug: #1646884
2016-12-14 16:18:26 -03:00
Liam Young
dfd8208354 Enable Memcache service for Token Caching
With the release of 4.2.0 of keystonemiddleware using the
in-process token cache is no longer recommended. It is recommended
that a memcache backend to store tokens is used instead,

This installs and configures memcache and configures neutron-server
to use memcache for token caching.

http://docs.openstack.org/releasenotes/keystonemiddleware/mitaka.html#id2

Change-Id: I747d107d28474d545e4a3612a927d46cae34e6b6
2016-12-13 09:31:19 +00:00
Frode Nordahl
ec2579a844 Update policy.json to give service role access
Role check is scoped to the configured service project and access
is granted to os_compute_api:servers:detail:get_all_tenants.

This change is required to allow Ceilometer to operate using a
non-Admin user.

Change-Id: I17729e2089cc658588eaea93f8de5051369e5dff
Closes-Bug: 1636098
2016-11-29 08:41:49 +01:00
James Page
366398e0b3 Guard return value from get_hostname
It's possible that the get_hostname call use when resolving the
full hostname of a IP address may return None in the event that
a hostname cannot be resolved via DNS.

Ensure that None values are not added to the list of hostnames
to validated for SSH key configuration.

Change-Id: I910224f9c22b7e0b8e53faddc64e3715389cc691
Closes-Bug: 1641614
2016-11-14 14:25:52 +00:00
Jenkins
7fe23b9dc3 Merge "Test resolvability of short names" 2016-11-10 15:48:54 +00:00
David Ames
d5d2e5064e Test resolvability of short names
There is no guarantee short hostnames will be DNS resolvable. Test
first before adding to the compute hosts list.

Change-Id: Ieddf098f97ebdfdf9074016998308e2887532d25
2016-11-07 15:16:37 -08:00
Jenkins
e8cb556926 Merge "Add support for serial console proxy access" 2016-09-27 13:04:26 +00:00
James Page
58bf5b0628 Add support for application version
Juju 2.0 provides support for display of the version of
an application deployed by a charm in juju status.

Insert the os_application_version_set function into the
existing assess_status function - this gets called after
all hook executions, and periodically after that, so any
changes in package versions due to normal system updates
will also be reflected in the status output.

This review also includes a resync of charm-helpers to
pickup hookenv and contrib.openstack support for this
feature.

Change-Id: I8c1d6b9b6d42bcc357d764ec1bd04a36e9982f57
2016-09-20 12:35:19 +01:00
Corey Bryant
88f2b0a648 Set install hook status when deploying from source
Change-Id: I8e576112b128fb307cb0b14dce61063fc4ac7559
2016-09-19 12:09:37 +00:00
James Page
469ef8dced Add support for serial console proxy access
Support access to instances via optionally enabled serial console
feature provided in Nova.

Seria console access is enabled using a new config flag; this flag
plus the required base_url for the nova-serialproxy are also passed
over the cloud-compute relation for use in nova-compute units.

This is only supported in OpenStack Juno or later, and replaces
the standard output to the nova console-log.

Change-Id: I3bfcca88bd6147be337e6d770db7348170b914e6
2016-09-15 12:37:33 +01:00
Felipe Reyes
68b0afed6d Add disable-aws-compat config to disable AWS compatibility layer
For OpenStack >= Liberty EC2 and objectstore services aren't installed and
deprecated, but there is no way to disable it, this patch adds a way to
permanently disable and enable them

Change-Id: Ie78ba5af1d953e8ebed9195202522ff73efcde9f
Closes-Bug: #1533255
2016-09-12 05:46:43 +00:00
Corey Bryant
a946731d24 Use os_release() to determine OpenStack release
Use os_release() instead of get_os_codename_install_source() to
determine the OpenStack release in resolve_services(). This is
needed when deploying from source because it checks both
openstack-origin and openstack-origin-git when determining the
release.

Change-Id: I39a61fc4394967bd70af493f6a69fe9d428274b1
2016-08-10 15:31:18 +00:00
Corey Bryant
01148cc7a9 Add systemd init support for deploy from source
systemd is used instead of upstart by default since Ubuntu 15.10
(Wily).  This adds systemd init file support for nova services
that are deployed from source.

Change-Id: Ie71aed8c1f28204d5fbd735d51cea6d77977a637
2016-07-07 11:05:45 +01:00
James Page
c103450dbb Re-license charm as Apache-2.0
All contributors to this charm have agreed to the switch
from GPL v3 to Apache 2.0; switch to Apache-2.0 license
as agreed so we can move forward with official project status.

Change-Id: I20448e0370138e103a00c7fcf0f49949c82d33a9
2016-07-03 16:38:27 +00:00
Corey Bryant
18e19ef50e Add defaults for openstack-origin-git config option
openstack-origin-git currently only supports YAML that specifies
the git repositories to deploy from.

This adds support for default openstack-origin-git values. The
default values supported are: icehouse, kilo, liberty, mitaka,
and master.  For example: openstack-origin-git=master

Change-Id: I6b737272deed067c2f71dbd36b201aea884265ee
2016-06-20 08:37:43 -04:00