Cleaning up comments and logging to make sure they properly adhere
to Openstack standards.
* Consistently use """ instead of ''' for comments.
* Always lazy-load logging parameters.
* Fixed bad log line in cert_manager.
Change-Id: I547f5dfa61609a899aef9b1470be8d8a6d8e4b81
Currently, Magnum is running periodic tasks to collect k8s cluster
metrics to message bus. Unfortunately, it's collecting pods info
only from "default" namespace which makes this function useless.
What's more, even Magnum can get all pods from all namespaces, it
doesn't make much sense to keep this function in Magnum. Because
operators only care about the health of cluster nodes. If they
want to know the status of pods, they can use heapster or other
tools to get that.
Task: 22619
Story: 1775116
Change-Id: I3ca0f2e96fe63870406cc5323f08fa018ac6e8be
Added configuration parameter, send_cluster_metrics, to magnum.conf
with default value of True. If set to True, periodic tasks will pull
COE data and send to ceilometer. This parameter can be set to False to
disable periodic collection of data to avoid unnecessary load from the
cluster.
Closes-Bug: #1668330
Related-Bug: #1746510
Change-Id: I9945293e7b2b52731f6e220d0925c1f6ad097caa
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who wants to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds first wsgi trace point, with info about HTTP request
* Add initialization of osprofiler at start of service
Currently that includes oslo.messaging notifer instance creation
to send Ceilometer backend notifications.
* Traces HTTP/RPC/DB API calls
Demo: https://hieulq.github.io/cluster-create-false-new-html.html
Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com>
Implements: blueprint osprofiler-support-in-magnum
Change-Id: I7d68995aab81d365433950aada078ef1fcd5469b
Move cluster status change notifications into the
periodic task so that drivers do not have to have
any knowledge of Magnum notification strategy.
Change-Id: I5c71dd780f7bd6d4b683e491f5b4ce22cecb396c
Partial-Blueprint: bp-driver-consolodation
This is an alternative implementation to:
https://review.openstack.org/#/c/397961
This version implements an earlier proposal from the
spec that adds a driver method for synchronizing
cluster state. This method is optional so that drivers
that do not wish to leverage the existing periodic
synchronization task can do so in whatever manner
they wish and Magnum will not force them to do anything
unnecessarily.
1. add an update_cluster_status method to the driver
interface
2. implment update_cluster_status for Heat drivers
using the existing tested logic
3. Remove cluster status updates from the cluster conductor
in favor of the periodic sync_cluster_status task - this
should avoid timeouts and race conditions possible in the
previous implementation
4. Update the periodic sync_cluster_status method to use
the driver to update cluster status rather than calling
Heat directly
Change-Id: Iae0ec7af2542343cc51e85f0efd21086d693e540
Partial-Blueprint: bp-driver-consolodation
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel. This patch updates Bay to Cluster in DB and
Object as well as all the usages. No functionality should be
changed by this patch, just naming and db updates.
Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
This patch set modifies lines which are importing objects
instead of modules. As per openstack import guide lines, user should
import modules in a file not objects.
http://docs.openstack.org/developer/hacking/#imports
Closes-Bug: #1620161
Change-Id: I7ec9022a6b1cec36c678a2cec2a1856e70a51c68
This is the first of several patches to add new Cluster commands
that will replace the Bay terminalogy in Magnum. This patch adds
the new Cluster and ClusterTemplate commands in addition to the
Bay and Baymodel commands. Additional patches will be created
for client, docs, and additional functional tests.
Change-Id: Ie686281a6f98a1a9931158d2a79eee6ac21ed9a1
Implements: blueprint rename-bay-to-cluster
There is a rollback mechanism in heat after the stack
update failed. There should be a rollback mechanism in
magnum after bay update failed.
This patch add new microversion 1.3 to add rollback
support for Magnum bay, user can enable rollback on bay
update failure by specifying microversion 1.3 in header(
{'OpenStack-API-Version': 'container-infra 1.3'}) and
passing 'rollback=True'(http://XXX/v1/bays/XXX/?rollback=True)
when issuing bay update reqeust.
Change-Id: Idd02769f98078702404a11dc9f7a3339ce4e22eb
Partially-Implements: blueprint bay-rollback-on-update-failure
Added i18n translation to all log messages. Also added
hacking check for this to ensure in future all log
messages are translated.
Change-Id: I77fbb18ffe47b12232125cc1fa7aae97ea9b1214
Closes-Bug: #1603759
The periodic task unneccessarily lists Heat stacks in the
global tenant (across all tenants) which the Magnum service
user may lack permission for. Also, the most restrictive way
to let it use global stack-list is chose a Keystone role and
open that operation to any user in any project holding that
role.
This commit substitutes a direct lookup of all bays' stack_id
attributes for this global stack list. This direct lookup will
yield the same net result. In order to get the neccessary
permissions it will use each bay's stored Keystone trust to
act on behalf of the bay's creating user.
Co-Authored-By: Jiri Suchomel <jiri.suchomel@suse.com>
Closes-Bug: #1589955
Change-Id: I67b176c137c463e37e037970cc4e468d51db30c9
There are files containing string format arguments inside logging
messages. Using logging function parameters should be preferred.
Change-Id: Id558f66de13146f6ae76a7a69f49721b6c3d6257
Closes-Bug: #1321274
Updated import order as per the OpenStack import standards [1].
Modules under magnum/common/pythonk8sclient are not covered in
this patch because this code is generated via a third party
tool "swagger-codegen" which does not follows openstack
standards yet.
[1] http://docs.openstack.org/developer/hacking/#import-order-template
Change-Id: Ia87f3e797ee627bf3c04d2bae9639fd305b41b0d
Liberty tech-debt-bug
Magnum service was initially utilizing periodic task within magnum
conductor. This commit lays out independent periodic framework for
magnum service, by decoupling from periodic task.
Change-Id: Ie0e725b75069309a9fa783f77c1a454f9dae549e
Partial-Bug: 1500998
Not removing existing try-catch block for sync_bay_status and other
periodic tasks. Rather leaving a note with the pointers and explanation
for developers to add new periodic task.
Change-Id: I2560d3961c19d73ab78a8e9812d69742c6bd2117
Closes-Bug:1496094
The cyclomatic complexity of sync_bay_status was greater than 10 and
very hard to comprehend. This patch reduces the complexity and makes the
task more open to refactoring and maintence.
Partial-Bug: #1501331
Change-Id: If415aa7710b586b0429ed1bc5077976791b424d1
* Implement a bay monitoring framework that provides APIs to pull data from
bay, and compute metrics based on the data.
* Add support for computing memory utilization of a swarm bay.
* Add a periodic task that sends metrics to ceilometer.
Change-Id: I32730e116c59a4e5f9d1ccb466d464c2fc1f6193
Partially-Implements: blueprint autoscale-bay
Partially-Implements: blueprint magnum-notifications
To support 'magnum service-list' after 'nova service-list', we need to
introduce periodic status update functionality for internal services.
Change-Id: Ia0c09222405c87cb61e5de4a43ba345ae3405b50
Partially-Implements: blueprint magnum-service-list
Closes-bug: #1492501
In periodic task, when bay status is found to be different with
corresponding stack, or the corresponding stack cannot be found in
heat, status reason should be synced as well as status. Otherwise,
status reason will be meaningless in some situation.
Change-Id: I27d6a7ecb0c74bd4d4a2bb2bfcd81d7632cd2147
Closes-Bug: #1488709
In before, bay and x509keypair supported list resources from all
tenants. It is desirable to generalize this capability while reducing
duplicated codes. Therefore, moving 'all_tenants' options to context.
Change-Id: Icfe31a6f2ac2e21fa7f377e244764f10892d25c7
Partially-Implements: blueprint autoscale-bay
Statuses are currently defined as a StringField when they should
be defined as an Enum object. This fix changes these objects from
using StringField by extending the Enum object in
oslo.versionedobjects
Change-Id: I12760e1d72f4af9931ee97bbad298f92f785118f
Partial-Bug: #1489136
The filters in method 'sync_bay_status' in periodic.py should be a dict,
instead of a list.
Change-Id: Id12e8768abb17b70bbc92d951b0d95e765d94e38
Closes-Bug: #1487812
In order to run several magnum-conductor, we should make periodic
tasks can run simultaneously. We should not destroy a bay twice.
Change-Id: Ie3c66442839197196f532734e67d810749ec5052
Partial-Implements: blueprint horizontal-scale
This patch makes some general cleanup changes to help strings to tidy
up various areas of the code. No functional impacts result from the
changes within this patch.
Closes-Bug: #1477837
Change-Id: I25a81059e974ac0083bb439f1cd37fb82157b3bb
This patch adds periodic task `sync_bay_status` which will be used for syncing
bay's status from heat stack, we will pull bays which status in
[bay_status.CREATE_IN_PROGRESS,
bay_status.UPDATE_IN_PROGRESS,
bay_status.DELETE_IN_PROGRESS]
which are all in a temporary status, and try to sync up the status with heat's
stack.
status changes will be like this:
bay_status stack_status sync up bay_status to
---------- ------------ ---------------------
CREATE_IN_PROGRESS CREATE_COMPLETE CREATE_COMPLETE
UPDATE_IN_PROGRESS UPDATE_COMPLETE UPDATE_COMPLETE
DELETE_IN_PROGRESS DELETE_COMPLETE DELETE_COMPLETE
CREATE_IN_PROGRESS CREATE_FAILED CREATE_FAILED
UPDATE_IN_PROGRESS UPDATE_FAILED UPDATE_FAILED
DELETE_IN_PROGRESS DELETE_FAILED DELETE_FAILED
CREATE_IN_PROGRESS Not Found CREATE_FAILED
UPDATE_IN_PROGRESS Not Found UPDATE_FAILED
DELETE_IN_PROGRESS Not Found destroy
Partial-Implements: blueprint add-periodic-task
Co-Authored-By: ShaoHe Feng <shaohe.feng@intel.com>
Change-Id: Ie9cc4d3f03c7938a8d988010604da79c9b8a22fd
This patch adds periodic timer to conductor service.
Any new periodic tasks need to be added to magnum/service/periodic.py
We add 2 default configure options to support periodic task:
'periodic_enable', default=True
'periodic_interval_max', default=60
These 2 options are for rpc_service.
PS:
We don't need to supervisor timers because timer itself won't have any
exception, and it is not a thread, which means can not be killed.
Partial-Implements: blueprint add-periodic-task
Co-Authored-By: ShaoHe Feng <shaohe.feng@intel.com>
Change-Id: I132d9cc89b26baf249b62b713ccd3776f0078d7d
Erik Olof Gunnar Andersson