Commit Graph

216 Commits (06659759f1213df23f1d456503b1dd2ea8e90963)

Author SHA1 Message Date
Andreas Jaeger ae228bb5cc Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found.

Update local hacking checks for new flake8.

Remove hacking and friends from lower-constraints, those are not needed
for co-installing.

Change-Id: I926efaef501f190e78da9cab40c1e94203277258
3 years ago
Theodoros Tsioutsias cbe05aa97d ng-6: Add new fields to nodegroup objects
Since each nodegroup will be one independent stack, we have to add
more fields to the table and object in order to track each stack
contained in the cluster. This adds the stack_id, version, status,
status_reason and version fields to the nodegroup object.

Change-Id: I6d36b2d3bc6476efbef6a9f702ffc73cfa0fab8c
4 years ago
Zuul c099128a61 Merge "Hardcode the names of the default NGs" 4 years ago
Feilong Wang 32989b4f7b Allow setting network, subnet and FIP when creating cluster
When using a public cluster template, user still need the capability
to reuse their existing network/subnet, and they also need to be
able to turn of/off the floatingip to overwrite the setting in the
public template. This patch supports that by adding those three
items as parameters when creating cluster.

Story: 2006208
Task: 35797

Change-Id: I11579ff6b83d133c71c2cbf49ee4b20996dfb918
4 years ago
Theodoros Tsioutsias a1d385cc4a Hardcode the names of the default NGs
As decided, the names of the default NGs should follow the convention
"default-<role>". This change fixes the migration responsible for
populating the default nodegroups.

Change-Id: I7a3ea4c14e1353a02ed91cfe47f89e2b00d802a1
Story: 2005953
Task: 34318
4 years ago
Theodoros Tsioutsias 3f80cbab06 ng-4: Adapt cluster object
This commit removes the fields node_addresses, master_addresses,
node_count and master_count from the cluster object since this info
will be stored in the nodegroups. At the same time, provides the way
to adapt existing clusters to the new schema.

story: 2005266

Change-Id: Iaf2cef3cc50b956c9b6d7bae13dbb716ae54eaf7
4 years ago
Theodoros Tsioutsias 0607c7a9d6 ng-1: Add nodegroup representation
This adds the object and db schema changes needed for supporting
nodegroups.

story: 2005266

Change-Id: Ibf10277a52aa94c4b217cf3b364844b04baab1e0
4 years ago
Ricardo Rocha 6aee864954 Add hidden flag to cluster template
Add a new hidden flag to cluster templates. This allows an operator to
keep a cluster public (accessible to all users) while not showing them
in cluster template listing.

Story: 2004941
Task: 29342

Change-Id: Ia2717ca960041753f6e772bf2d41c7f5a196dae6
4 years ago
Feilong Wang 766a64a41a Allow cluster template being renamed
To get a better cluster template versioning and relieve the pain
of maintaining public cluster template, the patch is proposing
that the name of cluster template can be changed.

A folllowing patch/spec will be proposed to add a new field
'deprecated' to allow ops to hide old/deprecated templates.

Task: 26889
Story: 2003960

Change-Id: Id1db81d35bc3dccff0fac481be7801de200d52de
4 years ago
Erik Olof Gunnar Andersson daf34d9df8 Use oslo_serialization instead of the json module directly
* Always use oslo jsonutils.
* Consistently import jsonutils as-is.
* Use dump_as_bytes instead of dumps.
https://wiki.openstack.org/wiki/Python3#Serialization:_base64.2C_JSON.2C_etc.

Change-Id: I2b65faa7df43a1d58205e8ff106ff62f73d78198
4 years ago
Feilong Wang d80febb384 Add health_status and health_status_reason to cluster
APIImpact
Related blueprint cluster-healing

Change-Id: I78d4d14fb064996de77bdd6a381d2df53a9488b8
5 years ago
oorgeron 0e6e27eb7a Enables MySQL Cluster Support for Magnum
This enables the usage of the mysql_enable_ndb setting
in oslo_db from bug 1564110. This allows operators to
select from MySQL (InnoDB) or MySQL Cluster (NDB)
as the storage engine backend. There are only two
tables that need to have their columns auto adjusted
to fit with in the 14k row width limit of NDB, cluster
and cluster_templates. The modifications here will
have zero affect on users of MySQL (InnoDB). Additionally,
this fix will auto adjust those tables in models.py
module to support this enhancement.

Closes-Bug: 1691514
Change-Id: Ia0584059f998baf3db4a44bccbb6f6ff8a058ae1
Depends-On: I7f9c830073bf9a30abce0aa4bb55b5c9cf661afe
5 years ago
Clenimar Filemon 34754f36f0 federation api: federation table and db layer
this commit introduces a new `Federation` table to
Magnum database, as well as the necessary DB layer
APIs to access and manage it.

this belongs to the first phase of the implementation
of the federation api. check [1] for more details.

[1] https://review.openstack.org/#/c/489609/

Change-Id: Ie8a68cd3198c8fc7930069fd2e55f1cad55b6c9b
Partially-Implements: blueprint federation-api
5 years ago
Ricardo Rocha 84006f63d7 Allow flavor_id on cluster create
Add flavor_id as an option during cluster create. If not given,
the default is taken from the cluster template.

Add flavor_id in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1699245
Change-Id: Ib60c05cce1cf2639ca4740abdd264403033433f9
6 years ago
Ricardo Rocha 28fff8006a Make docker_storage_driver a str instead of enum
Allow any value to be passed on the docker_storage_driver field by turning it
into a StringField (was EnumField), and remove the constraints limiting the
values to 'devicemapper' and 'overlay'.

Change the docker storage setup to have a generic setup for all drivers with
the exception of 'devicemapper', which keeps its own specific storage config
function. For all others, do the same we already did for overlay (with two
cases for usage of a cinder volume or not) and simply set the storage driver
in the docker configuration to the value provided in the cluster template.

Change-Id: I9aa8f232ce64ece4d439c0a476f463820a499617
Closes-Bug: #1722522
6 years ago
Ricardo Rocha 446b159349 Allow master_flavor_id on cluster create
Add master_flavor_id as an option during cluster create. If not given,
the default is taken from the cluster template.

Add master_flavor_id in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1699247
Change-Id: Id1d973167b381538121583a0a9691304b39e98de
6 years ago
Jenkins 04b4df0d74 Merge "Allow labels on cluster create" 6 years ago
Ricardo Rocha 3d04ed4cbb Allow labels on cluster create
Add labels as an option during cluster create. If not given,
the default is taken from the cluster template.

Add labels in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1697651
Implements: blueprint flatten-attributes
Change-Id: I8990c78433dcbbca5bc4aa121678b02636346802
6 years ago
chestack c22f9cd882 Deal with db_exc.DBDuplicate of conductor startup
we have enable multiple workers of magnum-conductor that
result in multiple processes save same DB entry concurrently.

This patch logs warning message instead of raising exception

Change-Id: I548d50bed5d80e96042f88039e880075e1bffc53
Close-Bug: #1711324
6 years ago
Ricardo Rocha 79039bb419 Allow docker_volume_size on cluster create
Add docker_volume_size as an option during cluster create. If not given,
the default is taken from the cluster template.

Add docker_volume_size in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1697648
Implements: blueprint flatten-attributes
Change-Id: Ic6d77e6fdf5b068fa5319b238f4fd98b4d499be4
6 years ago
yuanpeng 71d25456d2 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

Change-Id: If1f4bd2f6be967368f52fb367c5a428d3eb58a9d
Closes-Bug:#1674551
6 years ago
Johannes Grassler e93d82e8b3 Fix CVE-2016-7404
This commit addresses multiple potential vulnerabilities in
Magnum. It makes the following changes:

* Permissions for /etc/sysconfig/heat-params inside Magnum
  created instances are tightened to 0600 (used to be 0755).
* Certificate retrieval is modified to work without the need
  for a Keystone trust.
* The cluster's Keystone trust id is only passed into
  instances for clusters where that is actually needed. This
  prevents the trustee user from consuming the trust in cases
  where it is not needed.
* The configuration setting trust/cluster_user_trust (False by
  default) is introduced. It needs to be explicitely enabled
  by the cloud operator to allow clusters that need the
  trust_id to be passed into instances to work. Without this
  setting, attempts to create such clusters will fail.

Please note, that none of these changes apply to existing
clusters. They will have to be deleted and rebuilt to benefit
from these changes.

Change-Id: I643d408cde0d6e30812cf6429fb7118184793400
6 years ago
Jenkins cb7d36ea0f Merge "Resource Quota - Limit clusters per project" 6 years ago
Vijendar Komalla 206d17f8ca Resource Quota - Limit clusters per project
Currently there is no limit on the number of clusters that can
be created in a project. This change limits number of clusters
in a project by checking cluster quota on cluster-create.

Change-Id: Ifa17d12692751fc6929e62be8bb59d481a2fd205
Partially-Implements: blueprint resource-quota
6 years ago
Jenkins 4e1ada7914 Merge "Integrate OSProfiler in Magnum" 6 years ago
Tovin Seven 32d088b2c1 Integrate OSProfiler in Magnum
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
  1) It checks that person who wants to trace is trusted and knows
     secret HMAC key.
  2) It starts tracing in case of proper trace headers
     and adds first wsgi trace point, with info about HTTP request

* Add initialization of osprofiler at start of service
  Currently that includes oslo.messaging notifer instance creation
  to send Ceilometer backend notifications.

* Traces HTTP/RPC/DB API calls

Demo: https://hieulq.github.io/cluster-create-false-new-html.html

Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com>
Implements: blueprint osprofiler-support-in-magnum
Change-Id: I7d68995aab81d365433950aada078ef1fcd5469b
6 years ago
Vijendar Komalla ccc04d67fc Resource Quota - DB layer changes
Change-Id: Ib1d4d8634eabdba76c5f7f858efc16702ebc0d2a
Partially-Implements: blueprint resource-quota
6 years ago
Jenkins 942fa495b6 Merge "Magnum stats API" 6 years ago
Vijendar Komalla 51e833137b Magnum stats API
This change introduces a new /stats REST endpoint that
provide the following basic information;
1) Total number of clusters and nodes for the given tenant.
2) Total number of clusters and nodes across all the tenants.
Follow-up patches include more stats.

Change-Id: Iac0bf9343549de31654545d5b1fd7601e56142a7
Partially Implements blueprint magnum-stats-api
6 years ago
Hongbin Lu 900c2af483 Remove unused context variable in db api
Change-Id: I78d7ddccddcc3630ddaf8fc6ddfc62dcff352bcd
6 years ago
yatin 5902daf8ae Remove provision_state parameters(specific to ironic)
The provision_state parameter was copied from ironic
with following patch:
https://review.openstack.org/#/c/138280

It should be removed as it's not required in magnum.

Change-Id: Id9e08ee731ae79531c8e29fd47e4fa650b1c5dcc
Partial-Bug: #1627663
7 years ago
Jaycen Grant 06f056f606 Allow keypair to be added during cluster create
Keypair can now be added during Cluster create command. This allows
Clusters to be created from the same ClusterTemplate but have
different keypair values when created. If not specified on create
then the value from the ClusterTemplate will be used. Adds keypair_id
to Cluster object and uses of keypair_id will use the value from Cluster
instead of ClusterTemplate.

Added release note for new features in magnum and magnum CLI.

Change-Id: I177a5aa06f881156944a9f74c9ccc3cd2abac492
Implements: blueprint keypair-override-on-create
7 years ago
Hieu LE d86b5735cd Centralize config option: database section
Centralize config option of database section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: Id12bbf3ad8d3342450cd64cf23761a60d49ee46a
Implements: blueprint centralize-config-magnum
7 years ago
Hieu LE 8f9eeb801a Centralize config option: cluster_heat section
Centralize config option of Cluster Heat section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: I9118eeb17061a0aa26269ea9deaba28e79f28b76
Implements: blueprint centralize-config-magnum
7 years ago
Jaycen Grant 729c2d0ab4 Rename Bay DB, Object, and internal usage to Cluster
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel.  This patch updates Bay to Cluster in DB and
Object as well as all the usages.  No functionality should be
changed by this patch, just naming and db updates.

Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
7 years ago
Jenkins 02ef5471b2 Merge "Correctly raising MagnumServiceNotFound exception" 7 years ago
Jaycen Grant 0b7c6401dd Rename BayModel DB, Object, and internal usage to ClusterTemplate
This patch is the first of 3 patches to change the internal
usage of the terms Bay and BayModel. This patch updates
BayModel to ClusterTemplate. No functionality should be
changed by this patch, just naming and db updates.

Change-Id: I0803e81be6482962be2878a8ea2c7480f89111ac
Implements: blueprint rename-bay-to-cluster
7 years ago
Rajiv Kumar b60c447d91 Correctly raising MagnumServiceNotFound exception
Passed arguments in key value form, while raising
MagnumServiceNotFound exception, as expected by the exception
message.

Change-Id: Iae767db0bbd264bb668b2845284fc5d876f7025d
Closes-Bug: #1619225
7 years ago
Stephen Watson 6ead3e4780 Updates CONF usage from bay to cluster.
Renames bay-related CONF options to their respective cluster names.
Adds release notes for CONF changes.

Change-Id: I7bbe0927e54c1f40a47bfdea448a88b467fef106
Implements: blueprint rename-bay-to-cluster
7 years ago
Vijendar Komalla 50bc376c4d Include version info in bay/cluster show operation
Currently bay-show operation does not return bay/cluster
version information. This change contain changes to return
bay/cluster version and container version info.

Change-Id: Ie12b6583e6d85faa3607f87295c04d72698034a5
Closes-Bug: #1613413
7 years ago
Jaycen Grant eaddb942fd Rename Bay to Cluster in api
This is the first of several patches to add new Cluster commands
that will replace the Bay terminalogy in Magnum. This patch adds
the new Cluster and ClusterTemplate commands in addition to the
Bay and Baymodel commands.  Additional patches will be created
for client, docs, and additional functional tests.

Change-Id: Ie686281a6f98a1a9931158d2a79eee6ac21ed9a1
Implements: blueprint rename-bay-to-cluster
7 years ago
OTSUKA, Yuanying 26370a2509 Add floating_ip_enabled field to baymodel
This patch adds floating_ip_enabled field to baymodel to specify
whether floating ip is used or not.

Change-Id: I99677221250480b43a4b95ebf460c43bc77090ad
Partially-Implements: blueprint bay-with-no-floating-ips
7 years ago
Jenkins fc9c1a8fc0 Merge "Fix for enum type docker_storage_driver" 7 years ago
Spyros Trigazis e6a29fb252 Remove ReplicationController object
Following the removal of service [1], pod [2] and container [3], remove
COE specific object ReplicationController.

This change also removes k8s_conductor.

[1] I4f06bb779caa0ad369a2b96b4714e1bf2db8acc6
[2] I8c2499ccb97aae39d80868ce02fbef292d762c10
[3] I288fa7a9717519b1ae8195820975676d99b4d6d2

Change-Id: Ica100c8d2dfdd7dc709feb1f5cdc5a3f3d6c7318
Partially-Implements: blueprint delete-container-endpoint
Partially-Implements: blueprint bay-drivers
7 years ago
Michal Jura e76ca364ab Fix for enum type docker_storage_driver
Create enum type "docker_storage_driver" for migration

This is fixing
   oslo_db.exception.DBError: (psycopg2.ProgrammingError) type
   "docker_storage_driver" does not exist

Closes-Bug: #1609776

Change-Id: I92d427e90bd73b4114d8688d3761cabac450fc9d
7 years ago
Tom Cammann 40aa6550f1 Remove container object
Following on from removing the k8s specific APIs in
I1f6f04a35dfbb39f217487fea104ded035b75569 the objects associated with
these APIs need removal.

Remove the container object, drop the db table and remove references to
the container object. The docker_conductor has also been removed as this
was used for managing containers using Magnum objects.

Change-Id: I288fa7a9717519b1ae8195820975676d99b4d6d2
Partially-Implements: blueprint delete-container-endpoint
Co-Authored-By: Spyros Trigazis <strigazi@gmail.com>
7 years ago
PanFengyun f3f0f9fef6 Add i18n support for some ERROR message
There are some ERROR message miss i18n support, this patch adds i18n
support for then and changes some marks in message.

Change-Id: I33ce248b5bfc4e09680a4c6ea1f1c0e1ef02ed5d
Closes-Bug: #1603759
7 years ago
OTSUKA, Yuanying 62c75e5883 Add fixed_subnet field to baymodel
Ironic template needs to specify fixed subnet name/ID.
And also user sometimes want to build his cluster on
his network which he built.
This patch adds fixed_subnet field to baymodel to specify
fixed subnet which bay will build on.

Change-Id: Ia21ad6e3e1472a0d078d08369ef62601034fcc17
Partial-Implements: blueprint magnum-baremetal-full-support
7 years ago
Drago Rosson 6d10dbf35d Add master_lb_enabled field to baymodel
master_lb_enabled is a boolean field that determines if a bay's master
nodes should be load balanced. Defaults to False.

Change-Id: Icf8317432dec70c99c29a3af9edb0499bfd2439d
Partially-Implements: blueprint decouple-lbaas
7 years ago
Tom Cammann 3e02840628 Remove service object
Following on from removing the k8s specific APIs in
I1f6f04a35dfbb39f217487fea104ded035b75569 the objects associated with
these APIs need removal.

Remove the service object, drop the db table and remove references to
the service object.

Change-Id: I4f06bb779caa0ad369a2b96b4714e1bf2db8acc6
Partially-Implements: blueprint delete-container-endpoint
7 years ago