Commit Graph

35 Commits (8f4643d85ce9d5d8ad7442a0534e4e5f84fb8c48)

Author SHA1 Message Date
Erik Olof Gunnar Andersson f2fd732ce2 Trivial code cleanups
Cleaning up comments and logging to make sure they properly adhere
to Openstack standards.

* Consistently use """ instead of ''' for comments.
* Always lazy-load logging parameters.
* Fixed bad log line in cert_manager.

Change-Id: I547f5dfa61609a899aef9b1470be8d8a6d8e4b81
5 years ago
Feilong Wang a26c2225b6 Deprecate send_cluster_metrics
Currently, Magnum is running periodic tasks to collect k8s cluster
metrics to message bus. Unfortunately, it's collecting pods info
only from "default" namespace which makes this function useless.
What's more, even Magnum can get all pods from all namespaces, it
doesn't make much sense to keep this function in Magnum. Because
operators only care about the health of cluster nodes. If they
want to know the status of pods, they can use heapster or other
tools to get that.

Task: 22619
Story: 1775116

Change-Id: I3ca0f2e96fe63870406cc5323f08fa018ac6e8be
5 years ago
Kirsten G cf84683940 Add send_cluster_metrics configuration parameter
Added configuration parameter, send_cluster_metrics, to magnum.conf
with default value of True. If set to True, periodic tasks will pull
COE data and send to ceilometer. This parameter can be set to False to
disable periodic collection of data to avoid unnecessary load from the

Closes-Bug: #1668330
Related-Bug: #1746510

Change-Id: I9945293e7b2b52731f6e220d0925c1f6ad097caa
5 years ago
coldmoment ba8ad5e37f Add a hacking rule for string interpolation at logging
String interpolation should be delayed to be handled
by the logging code, rather than being done at the point
of the logging call.
See the oslo i18n guideline

Change-Id: I8a4f5f896865aebbff88ee894f0081e58cfce9ef
6 years ago
yuanpeng 71d25456d2 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.


Change-Id: If1f4bd2f6be967368f52fb367c5a428d3eb58a9d
6 years ago
Tovin Seven 32d088b2c1 Integrate OSProfiler in Magnum
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
  1) It checks that person who wants to trace is trusted and knows
     secret HMAC key.
  2) It starts tracing in case of proper trace headers
     and adds first wsgi trace point, with info about HTTP request

* Add initialization of osprofiler at start of service
  Currently that includes oslo.messaging notifer instance creation
  to send Ceilometer backend notifications.

* Traces HTTP/RPC/DB API calls


Co-Authored-By: Hieu LE <>
Implements: blueprint osprofiler-support-in-magnum
Change-Id: I7d68995aab81d365433950aada078ef1fcd5469b
6 years ago
Randall Burt 84a9464957 Move cluster status notifications out of driver
Move cluster status change notifications into the
periodic task so that drivers do not have to have
any knowledge of Magnum notification strategy.

Change-Id: I5c71dd780f7bd6d4b683e491f5b4ce22cecb396c
Partial-Blueprint: bp-driver-consolodation
7 years ago
Randall Burt 759c1b3b2b Move cluster status updates into driver
This is an alternative implementation to:

This version implements an earlier proposal from the
spec that adds a driver method for synchronizing
cluster state. This method is optional so that drivers
that do not wish to leverage the existing periodic
synchronization task can do so in whatever manner
they wish and Magnum will not force them to do anything

1. add an update_cluster_status method to the driver
2. implment update_cluster_status for Heat drivers
   using the existing tested logic
3. Remove cluster status updates from the cluster conductor
   in favor of the periodic sync_cluster_status task - this
   should avoid timeouts and race conditions possible in the
   previous implementation
4. Update the periodic sync_cluster_status method to use
   the driver to update cluster status rather than calling
   Heat directly

Change-Id: Iae0ec7af2542343cc51e85f0efd21086d693e540
Partial-Blueprint: bp-driver-consolodation
7 years ago
Hieu LE 1a97476593 Centralize config option: rpc periodic section
Centralize config option of rpc periodic section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: Ic499369d16e7be79449a0e8ed5416b0ab92abc07
Implements: blueprint centralize-config-magnum
7 years ago
Jaycen Grant 729c2d0ab4 Rename Bay DB, Object, and internal usage to Cluster
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel.  This patch updates Bay to Cluster in DB and
Object as well as all the usages.  No functionality should be
changed by this patch, just naming and db updates.

Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
7 years ago
Cao Xuan Hoang 33bd24252f Clean imports in code
This patch set modifies lines which are importing objects
instead of modules. As per openstack import guide lines, user should
import modules in a file not objects.

Closes-Bug: #1620161

Change-Id: I7ec9022a6b1cec36c678a2cec2a1856e70a51c68
7 years ago
Jaycen Grant eaddb942fd Rename Bay to Cluster in api
This is the first of several patches to add new Cluster commands
that will replace the Bay terminalogy in Magnum. This patch adds
the new Cluster and ClusterTemplate commands in addition to the
Bay and Baymodel commands.  Additional patches will be created
for client, docs, and additional functional tests.

Change-Id: Ie686281a6f98a1a9931158d2a79eee6ac21ed9a1
Implements: blueprint rename-bay-to-cluster
7 years ago
Jenkins 5372cd92b4 Merge "Rollback bay on update failure" 7 years ago
Wenzhi Yu 63b5c21c8d Rollback bay on update failure
There is a rollback mechanism in heat after the stack
update failed. There should be a rollback mechanism in
magnum after bay update failed.

This patch add new microversion 1.3 to add rollback
support for Magnum bay, user can enable rollback on bay
update failure by specifying microversion 1.3 in header(
{'OpenStack-API-Version': 'container-infra 1.3'}) and
passing 'rollback=True'(http://XXX/v1/bays/XXX/?rollback=True)
when issuing bay update reqeust.

Change-Id: Idd02769f98078702404a11dc9f7a3339ce4e22eb
Partially-Implements: blueprint bay-rollback-on-update-failure
7 years ago
yatin karel fb0057c105 Add i18n translation for Log messages
Added i18n translation to all log messages. Also added
hacking check for this to ensure in future all log
messages are translated.

Change-Id: I77fbb18ffe47b12232125cc1fa7aae97ea9b1214
Closes-Bug: #1603759
7 years ago
Johannes Grassler f895b2bd09 Fix global stack list in periodic task
The periodic task unneccessarily lists Heat stacks in the
global tenant (across all tenants) which the Magnum service
user may lack permission for. Also, the most restrictive way
to let it use global stack-list is chose a Keystone role and
open that operation to any user in any project holding that

This commit substitutes a direct lookup of all bays' stack_id
attributes for this global stack list. This direct lookup will
yield the same net result. In order to get the neccessary
permissions it will use each bay's stored Keystone trust to
act on behalf of the bay's creating user.

Co-Authored-By: Jiri Suchomel <>
Closes-Bug: #1589955
Change-Id: I67b176c137c463e37e037970cc4e468d51db30c9
7 years ago
PanFengyun f9c662e709 Fix string declaration in
The "%(reason)" of has losed 's'.
"%(reason)"  -->  "%(reason)s"

Change-Id: I5bd1b723e5bf8615e11b1d182cc8a978c6b01b8b
7 years ago b9aa042293 Replace string format arguments with function parameters
There are files containing string format arguments inside logging
messages. Using logging function parameters should be preferred.

Change-Id: Id558f66de13146f6ae76a7a69f49721b6c3d6257
Closes-Bug: #1321274
7 years ago
Dinesh Bhor e7be462a24 Fix invalid import order
Updated import order as per the OpenStack import standards [1].
Modules under magnum/common/pythonk8sclient are not covered in
this patch because this code is generated via a third party
tool "swagger-codegen" which does not follows openstack
standards yet.


Change-Id: Ia87f3e797ee627bf3c04d2bae9639fd305b41b0d
7 years ago
Wanlong Gao 6ac4ce2eb1 Python 3 deprecated the logger.warn method in favor of warning
Python 3 deprecated the logger.warn method, see:, so we
prefer to use warning to avoid DeprecationWarning.

Closes-Bug: 1508442
Change-Id: I245d24b30e783f1c6dcec1b425d78d271edff49b
8 years ago
Surojit Pathak d024bc60c4 Decoupling magnum service from periodic task
Liberty tech-debt-bug

Magnum service was initially utilizing periodic task within magnum
conductor. This commit lays out independent periodic framework for
magnum service, by decoupling from periodic task.

Change-Id: Ie0e725b75069309a9fa783f77c1a454f9dae549e
Partial-Bug: 1500998
8 years ago
Surojit Pathak ab5a173e6d Adding dev-notes for try-catch block in periodic task
Not removing existing try-catch block for sync_bay_status and other
periodic tasks. Rather leaving a note with the pointers and explanation
for developers to add new periodic task.

Change-Id: I2560d3961c19d73ab78a8e9812d69742c6bd2117
8 years ago
Tom Cammann d2e05d4e9b Refactor periodic task sync_bay_status
The cyclomatic complexity of sync_bay_status was greater than 10 and
very hard to comprehend. This patch reduces the complexity and makes the
task more open to refactoring and maintence.

Partial-Bug: #1501331
Change-Id: If415aa7710b586b0429ed1bc5077976791b424d1
8 years ago
Hongbin Lu b88085c93c Implement bay monitoring and notifications
* Implement a bay monitoring framework that provides APIs to pull data from
  bay, and compute metrics based on the data.
* Add support for computing memory utilization of a swarm bay.
* Add a periodic task that sends metrics to ceilometer.

Change-Id: I32730e116c59a4e5f9d1ccb466d464c2fc1f6193
Partially-Implements: blueprint autoscale-bay
Partially-Implements: blueprint magnum-notifications
8 years ago
Surojit Pathak 3674ce278d Backend support for magnum service
To support 'magnum service-list' after 'nova service-list', we need to
introduce periodic status update functionality for internal services.

Change-Id: Ia0c09222405c87cb61e5de4a43ba345ae3405b50
Partially-Implements: blueprint magnum-service-list
Closes-bug: #1492501
8 years ago
Jenkins 6c43071d3f Merge "Move 'all_tenants' options to context" 8 years ago
Jenkins b10538847a Merge "Sync bay status reason in periodic task" 8 years ago
mathspanda 235877bebc Sync bay status reason in periodic task
In periodic task, when bay status is found to be different with
corresponding stack, or the corresponding stack cannot be found in
heat, status reason should be synced as well as status. Otherwise,
status reason will be meaningless in some situation.

Change-Id: I27d6a7ecb0c74bd4d4a2bb2bfcd81d7632cd2147
Closes-Bug: #1488709
8 years ago
Hongbin Lu fa5bcee4ab Move 'all_tenants' options to context
In before, bay and x509keypair supported list resources from all
tenants. It is desirable to generalize this capability while reducing
duplicated codes. Therefore, moving 'all_tenants' options to context.

Change-Id: Icfe31a6f2ac2e21fa7f377e244764f10892d25c7
Partially-Implements: blueprint autoscale-bay
8 years ago
Cale Rath e4add16a69 Use oslo.versionedobjects enums instead of status strings
Statuses are currently defined as a StringField when they should
be defined as an Enum object.  This fix changes these objects from
using StringField by extending the Enum object in

Change-Id: I12760e1d72f4af9931ee97bbad298f92f785118f
Partial-Bug: #1489136
8 years ago
mathspanda a1ffe63454 Fix wrong usage of filters in periodic task
The filters in method 'sync_bay_status' in should be a dict,
instead of a list.

Change-Id: Id12e8768abb17b70bbc92d951b0d95e765d94e38
Closes-Bug: #1487812
8 years ago
Hua Wang e495f81b39 Fix race condition when syncing bay status across conductors
In order to run several magnum-conductor, we should make periodic
tasks can run simultaneously. We should not destroy a bay twice.

Change-Id: Ie3c66442839197196f532734e67d810749ec5052
Partial-Implements: blueprint horizontal-scale
8 years ago
Joe Cropper 16688d42a7 Clean up miscellaneous help strings
This patch makes some general cleanup changes to help strings to tidy
up various areas of the code. No functional impacts result from the
changes within this patch.

Closes-Bug: #1477837
Change-Id: I25a81059e974ac0083bb439f1cd37fb82157b3bb
8 years ago
Eli Qiao fd57cb4372 Add periodic task to sync up bay status
This patch adds periodic task `sync_bay_status` which will be used for syncing
bay's status from heat stack, we will pull bays which status in
which are all in a temporary status, and try to sync up the status with heat's

status changes will be like this:

bay_status              stack_status           sync up bay_status to
----------              ------------           ---------------------


CREATE_IN_PROGRESS      Not Found              CREATE_FAILED
UPDATE_IN_PROGRESS      Not Found              UPDATE_FAILED
DELETE_IN_PROGRESS      Not Found              destroy

Partial-Implements: blueprint add-periodic-task
Co-Authored-By: ShaoHe Feng <>
Change-Id: Ie9cc4d3f03c7938a8d988010604da79c9b8a22fd
8 years ago
Eli Qiao fbe5b8065f Add periodic task framework
This patch adds periodic timer to conductor service.
Any new periodic tasks need to be added to magnum/service/
We add 2 default configure options to support periodic task:
    'periodic_enable', default=True
    'periodic_interval_max', default=60
These 2 options are for rpc_service.
We don't need to supervisor timers because timer itself won't have any
exception, and it is not a thread, which means can not be killed.

Partial-Implements: blueprint add-periodic-task
Co-Authored-By: ShaoHe Feng <>
Change-Id: I132d9cc89b26baf249b62b713ccd3776f0078d7d
8 years ago