Commit Graph

83 Commits (c98e9525c7db34734afb29d1b9fb409a08d16ef7)

Author SHA1 Message Date
Clenimar Filemon ec950be894 federation api: api endpoints
this commit introduces a new '/federations'
endpoint to Magnum API, as well as its controllers,
entities and conductor handlers.

this corresponds to the first phase of the
federation-api spec. please refer to [1] for more


Change-Id: I662ac2d6ddec07b50712109541486fd26c5d21de
Partially-Implements: blueprint federation-api
5 years ago
Spyros Trigazis d11f87d0ca Start RPC service before waiting
Stoping magnum-cond without having invoke start(),
results in "WARNING oslo_messaging.server Possible
hang: stop is waiting for start to complete".

A magnum instance with 16 workers was taking 1m to stop
with this change it takes 1 to 10 seconds. This change
doesn't break the fix in [1].

[1] If9e13effc14fd35e646d02bb4f568e79786aa958

Related-Bug: #1702349
Related issue in sahara:
Related-Bug: #1546119

Change-Id: Ied7ab43398d4e499514fa0bd5dba64971d1956bf
5 years ago
yatin 604bbe5f33 Fix Usage of cliff commandmanager
Set namespace to "magnum" when setting up cliff
commandmanager. Earlier None used to work but after
[1] and it's u-c release as cliff 2.10.0([2]) it requires
to be set as some string.


Change-Id: I036386d758a9351f053338f41a595fc030df5a33
Closes-Bug: #1741911
5 years ago
Mohammed Naser 8ce15c4510 Avoid running periodic processes inside each worker process
The periodic jobs are currently getting registered per each worker
which means that in cases with large number of workers, the APIs
for services such as Heat and Keystone will be hit very hard.

This patch resolves this issue by registering the jobs only to the
main process, ensuring that they run once per instance (or group
of workers).

Closes-Bug: #1702349

Change-Id: If9e13effc14fd35e646d02bb4f568e79786aa958
6 years ago
coldmoment ba8ad5e37f Add a hacking rule for string interpolation at logging
String interpolation should be delayed to be handled
by the logging code, rather than being done at the point
of the logging call.
See the oslo i18n guideline

Change-Id: I8a4f5f896865aebbff88ee894f0081e58cfce9ef
6 years ago
wangbo c50d869670 Support magnum-conductor multiple process workers
Multiple process workers support for magnum-conductor.
Adds new option 'workers' to group [conductor] of magnum.conf.

Change-Id: If4d47769c97f756dbf5f45ac4413df7971731f21
Implements: blueprint magnum-multiple-process-workers
6 years ago
yuanpeng 71d25456d2 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.


Change-Id: If1f4bd2f6be967368f52fb367c5a428d3eb58a9d
6 years ago
Jenkins 085058bfb2 Merge "Support magnum-api multiple process workers" 6 years ago
Tovin Seven 32d088b2c1 Integrate OSProfiler in Magnum
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
  1) It checks that person who wants to trace is trusted and knows
     secret HMAC key.
  2) It starts tracing in case of proper trace headers
     and adds first wsgi trace point, with info about HTTP request

* Add initialization of osprofiler at start of service
  Currently that includes oslo.messaging notifer instance creation
  to send Ceilometer backend notifications.

* Traces HTTP/RPC/DB API calls


Co-Authored-By: Hieu LE <>
Implements: blueprint osprofiler-support-in-magnum
Change-Id: I7d68995aab81d365433950aada078ef1fcd5469b
6 years ago
wangbo b462a3b8cd Support magnum-api multiple process workers
Multiple process workers support for magnum-api.
Adds new option 'workers' to group [api] of magnum.conf.

Change-Id: I0e8327ada6926602d577d1f36d384dd49426c7ee
Implements: blueprint magnum-multiple-process-workers
6 years ago
yatin af59ea68b8 Fix magnum-template-manage
After patch:
magnum-template-manage didn't worked because load_entry_point
method moved to Driver class.
With this patch, magnum-template-manage can be used to list
all available drivers in magnum.

The patch also renames magnum-template-manage cli to
Drivers can now be listed using:-
magnum-driver-manage list-drivers
magnum-driver-manage list-drivers -d -p


Change-Id: I17ba94b0e2000486b5fcbf792991ad98183bd26c
Partially-Implements: blueprint bay-drivers
Closes-Bug: #1632630
7 years ago
Hieu LE 4f8b26f822 Centralize config option: conductor section
Centralize config option of Conductor section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: I7deea1cba16495af2081bbb12444da7e62bdf235
Implements: blueprint centralize-config-magnum
7 years ago
Hieu LE 494cb2198c Centralize config option: cluster section
Centralize config option of Cluster section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: I2f7e796c08b4b2a97e4644cb6aa9dc7a4a4e692d
Implements: blueprint centralize-config-magnum
7 years ago
Hieu LE cfe5b4ed5c Centralize config option: api section
Centralize config option of API section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: I0589014ab8eb70f0f6551bb23808d3b9edc580c1
Implements: blueprint centralize-config-magnum
7 years ago
Jenkins 4548b310e5 Merge "Rename Bay DB, Object, and internal usage to Cluster" 7 years ago
Jaycen Grant 729c2d0ab4 Rename Bay DB, Object, and internal usage to Cluster
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel.  This patch updates Bay to Cluster in DB and
Object as well as all the usages.  No functionality should be
changed by this patch, just naming and db updates.

Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
7 years ago
Jenkins 9db3f369d5 Merge "Improve unit test coverage for cmd/" 7 years ago
Jenkins 0beb06cb67 Merge "Make magnum manage_template read config file and increase coverage" 7 years ago
Jenkins 30194b55d6 Merge "Use werkzeug to run Magnum API with SSL" 7 years ago
Hieu LE 85d4c68365 Improve unit test coverage for cmd/
Add new unit tests for cmd/
Increase the coverage for cmd/ from 0 to 100%.

Move the subcommand 'command' of db_manage to out side
of method main for easier testing.

Change-Id: I686fbc25fd58aea91b5a862fc61c832f4e0d8684
Partial-Bug: #1511667
7 years ago
Hieu LE 99cfbcbc8b Make magnum manage_template read config file and increase coverage
Currently magnum-manage-template command doesn't read config file
properly. This patch make manage-template script read config file
and display template list the right way.

Add new unit tests for cmd/
Increase the coverage for cmd/ from 0 to 100%.

Change-Id: I3a4df33834201148f824ebe013c5a22aee3ec1f8
Partial-Bug: #1511667
Closes-Bug: #1612596
7 years ago
Stephen Watson 6ead3e4780 Updates CONF usage from bay to cluster.
Renames bay-related CONF options to their respective cluster names.
Adds release notes for CONF changes.

Change-Id: I7bbe0927e54c1f40a47bfdea448a88b467fef106
Implements: blueprint rename-bay-to-cluster
7 years ago
Hieu LE 70c803bfc1 Use werkzeug to run Magnum API with SSL
wsgiref.simple_server is mono threaded process that can not
support SSL context. This patch aim to replace wsgiref.simple_server
with werkzeug for running development API server supporting SSL.

Change-Id: Ib4360d77030e4cce8abf5ea543d87b7982e0e285
Closes-Bug: #1614596
7 years ago
Spyros Trigazis e6a29fb252 Remove ReplicationController object
Following the removal of service [1], pod [2] and container [3], remove
COE specific object ReplicationController.

This change also removes k8s_conductor.

[1] I4f06bb779caa0ad369a2b96b4714e1bf2db8acc6
[2] I8c2499ccb97aae39d80868ce02fbef292d762c10
[3] I288fa7a9717519b1ae8195820975676d99b4d6d2

Change-Id: Ica100c8d2dfdd7dc709feb1f5cdc5a3f3d6c7318
Partially-Implements: blueprint delete-container-endpoint
Partially-Implements: blueprint bay-drivers
7 years ago
Tom Cammann 40aa6550f1 Remove container object
Following on from removing the k8s specific APIs in
I1f6f04a35dfbb39f217487fea104ded035b75569 the objects associated with
these APIs need removal.

Remove the container object, drop the db table and remove references to
the container object. The docker_conductor has also been removed as this
was used for managing containers using Magnum objects.

Change-Id: I288fa7a9717519b1ae8195820975676d99b4d6d2
Partially-Implements: blueprint delete-container-endpoint
Co-Authored-By: Spyros Trigazis <>
7 years ago
Vijendar Komalla a5ae8b37aa Bay driver: k8s Fedora Atomic
Change-Id: I1fae89e902ef73fe18ef78733a18e3690b67f312
Paritially-Implements: blueprint bay-drivers
7 years ago
Murali Allada 4080d4a84c Bay driver implementation
Moved all the swarm templates and template_definition code
to the magnum/drivers folder.

Moved base template_definition classes to drivers/common

Change-Id: Ieff57f0f47835c35d9f17c3d7d1b7e6a40907462
Partially-Implements: blueprint bay-drivers
Co-Authored-by: Spyros Trigazis <>
7 years ago
Spyros Trigazis c3491ad526 Fix string format in cmd/conductor
When testing debian packages for magnum, this raised an exception
on the conductor.

Change-Id: Ic0e9861a7ed0b4e41642d5203268b30e318c9a9e
7 years ago
Madhuri Kumari 303d14dde0 X509keypair cleanup
This patch does following:

* Removes X509keypair controller as there is already Certificate
  controller for same purpose.
* Removes X509keypair conductor.
* Removes name, ca_cert and bay_uuid from x509keypair model as
  Bay model already holds certificate references.
* Add intermediates and private_key_passphrase to x509keypair
* Remove related tests and changes.

Change-Id: I9271221cd1d07c672c4a380a4ae3593237fca66a
Partially-Implements: blueprint barbican-alternative-storeX
7 years ago
Eli Qiao ba0f715f40 Fix config parser error magnum-template-manage list-templates
`magnum-template-manage list-templates` reports 'str' object is not

Closes-Bug: #1558391
Change-Id: If36b0eafcce2e18cdda6480e7bd18bb19b8d40a4
7 years ago
Eric Brown 6253f70a43 Remove bandit.yaml in favor of defaults
* Remove bandit.yaml so bandit just uses internal defaults. Results
  in much less maintenance.
* Raise the severity level of the scan to just medium and high
* Added nosec to line that makes use of file:// scheme. That support
  should probably be droppped at a later time.
* Removed a misleading log message noting what address it was
  serving on. Bandit also flagged this as a potential security issue
  serving on

Change-Id: I3e08e462255f7b4ba8405d69f5843ed3c001d055
7 years ago
Aaron-DH db378a0ee5 Load wsgi app(api) with paste.deploy
This change replaces the hard coded WSGI app creation with a pipeline
of WSGI apps declared in a configuration file.
Paste Deploy was used to create the pipeline since it is used by many other
OpenStack projects and it is an active project
with new contributors and supports Python 3. Dependency on Paste is
localized so switching to another library would not be hard if OpenStack moves
to another package in the future.

Change-Id: I9a45f974c2c8c67a01748583639e6a6248003b85
7 years ago b9aa042293 Replace string format arguments with function parameters
There are files containing string format arguments inside logging
messages. Using logging function parameters should be preferred.

Change-Id: Id558f66de13146f6ae76a7a69f49721b6c3d6257
Closes-Bug: #1321274
7 years ago
Dinesh Bhor e7be462a24 Fix invalid import order
Updated import order as per the OpenStack import standards [1].
Modules under magnum/common/pythonk8sclient are not covered in
this patch because this code is generated via a third party
tool "swagger-codegen" which does not follows openstack
standards yet.


Change-Id: Ia87f3e797ee627bf3c04d2bae9639fd305b41b0d
7 years ago
Xi Yang 7e799929bb Replace logging with oslo_log
Use oslo_log for all logging options

Co-Authored-By: Corey O'Brien <>
Change-Id: Iaed3a18c868a532f0ce2455e2f4d3c6b06adc88e
7 years ago
houming-wang e5bdfb9b49 Trival: Remove unused logging import
In some Magnum source code files, oslo log is imported but not used.
Remove it.
from oslo_log import logging
LOG = logging.getLogger(__name__)

Closes-Bug: #1529253
Change-Id: I4d3911b94aed4c74b8163993b38e756139198049
8 years ago
houming-wang 6cf61f1421 Use cliff instead of cliutils
The (magnum/openstack/common/ has been graduate
[1]. We need to remove the dependency from Magnum and replace it with
cliff [2]. This patch refactor the Magnum template manager by cliff.


Change-Id: I25e2e2f653bdf6d1f484519894e00aa54f6477cc
Closes-Bug: #1528867
8 years ago
houming-wang 98875345c2 Performance: leverage dict comprehension in PEP-0274
PEP-0274 introduced dict comprehensions to replace dict constructor
with a sequence of length-2 sequences, these are benefits copied
from [1]:
  The dictionary constructor approach has two distinct disadvantages
  from the proposed syntax though.  First, it isn't as legible as a
  dict comprehension.  Second, it forces the programmer to create an
  in-core list object first, which could be expensive.
Magnum does not support python 2.6, we can leverage this.
There is deep dive about PEP-0274[2] and basic tests about
Note: This commit doesn't handle dict constructor with kwagrs.
This commit also adds a hacking rule.


Change-Id: I61992fa428d6760449afe3754b02506336e8b421
8 years ago
Surojit Pathak 3674ce278d Backend support for magnum service
To support 'magnum service-list' after 'nova service-list', we need to
introduce periodic status update functionality for internal services.

Change-Id: Ia0c09222405c87cb61e5de4a43ba345ae3405b50
Partially-Implements: blueprint magnum-service-list
Closes-bug: #1492501
8 years ago
Jenkins 82d961831f Merge "Indirection API implementation" 8 years ago
Madhuri cefc5a9ac3 Add Certificate controller for TLS support.
The Certificate controller has 2 operations:


Generate X509 certificate using bay's CA cert.
Below is an example of Certificate POST API request using magnum command:

Example Request:

    curl -X POST -H 'Content-Type: application/json' \
        -d '{"bay_uuid": "<bay_uuid>", "csr": "<csr>"}' \

This creates a X509 certificate signed by the given bay's CA and returns
it. No database information is stored in Magnum against it. For each POST
request, a new certificate is generated.

Example Response:

    {"bay_uuid": "<bay_uuid>", "csr": "<csr>",
     "pem": "<pem encoded certifiacte>"}

2. GET
Fetches the CA cert associated with a bay. Below is an example of CA GET
API request using magnum command:

Example Request:

    curl -X GET http://localhost:9511/v1/certificates/<bay_uuid>

This fetches stored CA cert for the given Bay, which can be used to validate
any client and node certificates signed by the Bay's CA. The value for each
is fetched from Barbican or Magnum db based on the different configuration
for storage of certificates.

Example Response:

    {"bay_uuid": "<bay_uuid>", "pem": "<pem encoded certifiacte>"}

Co-Authored-By: Andrew Melton <>

Change-Id: I4b72cc1e1bddc7a7c7eeb0ab22d3769a666ccb2b
Partially-Implements: bp secure-kubernetes
8 years ago
Grzegorz Grasza 2ed6b128d4 Indirection API implementation
When an object is received in an incompatible version,
IncompatibleObjectVersion is raised. Implementation of the indirection API
allows the object to be backported to a supported version by the conductor.

Related to blueprint versioned-objects-indirection-api

Change-Id: I99fe686b4b4e497be6b5d35a1d2e41833865799a
8 years ago
OTSUKA, Yuanying 45a2c4d1ff Unify using of migration tools
Currently magnum has two type of migration tools, one is oslo.db
migration_cli, other is calling alembic migration tool directly.
This fixes to get more consistency.

Closes-Bug: #1487248
Change-Id: I705dd2cc65c1f879ce1e2ebaaf2015dc6dc24c64
8 years ago
Jenkins 3df5e26a68 Merge "Add X509KeyPair controller and conductor." 8 years ago
Lin Yang fb168ffb47 Remove hardcoded config file in error message
Previously, it throw error message when database connection was
not defined,

"ValueError: Database connection not set in /etc/magnum/magnum.conf"

even if it was explicitly run with "--config-file /other/location".
So remove that hardcoded config file location.

Change-Id: I3af00f1f6572c3cd9e56141e1774acf373bbce8b
Closes-Bug: #1487258
8 years ago
Madhuri 4eb9425185 Add X509KeyPair controller and conductor.
This patch adds X509KeyPair controller and conductor to handle all
x509keypair related operation and also add test for it.

Change-Id: I5773fcd5bdf8a30fd195714e3e0fdc9d1b0c962d
Partially-Implements: bp secure-kubernetes
8 years ago
Jenkins c6b51d9130 Merge "Add guru meditation report for magnum" 8 years ago
Wanghua 6e22b25b11 Add guru meditation report for magnum
Oslo_reports enables OpenStack projects to dump Guru Meditation
Reports with useful debugging information to files or stderr.
So we introduce it into magnum.

Change-Id: Ibf78d87559f9646cc314cd064d7da4f94edc1ff7
Implements: blueprint guru-meditation-report
8 years ago
Kai Qiang Wu(Kennan) 8ec20a29f1 Fix the wrong platform usage
As 'vm' is not proper for platform, we now used server_type
to replace platfrom. As server_type can be vm or baremetal(bm) etc.

Closes-Bug: #1473257
Change-Id: I9e769e73ba3ea48069f0e69c4a93240bb1fe6c63
8 years ago
Wanghua af35cb173d Code refactor for prepare_service
Put the common codes used by api and conductor into a function.

Closes-Bug: 1472961
Change-Id: I3f7d431d9a91d9979d940d794db5bc383a077826
8 years ago