Commit Graph

28 Commits (8f4643d85ce9d5d8ad7442a0534e4e5f84fb8c48)

Author SHA1 Message Date
Feilong Wang cff4823168 Rename scripts
Scripts are the core of Magnum for COE deployment. To be more
clear and consistent, two changes proposed in this patch:

1. Rename network related script to xxx-flannel-xxx given they
are all for flannel and now we have calico driver.

2. Adding .sh for some scripts to be consistent with others.

Change-Id: I97f3e53b4b43648a4896193fb4ce469dbf42c611
5 years ago
Lingxian Kong 2cc57c5386 Use Octavia for LoadBalancer type service
In the OpenStack deployment with Octavia service enabled, the octavia
service should be used not only for master nodes high availability, but
also for k8s LoadBalancer type service implementation as well.

Change-Id: Ib61f59507510253794a4780a91e49aa6682c8039
Closes-Bug: #1770133
5 years ago
Spyros Trigazis 30785acd3c Update kubernetes dashboard to v1.8.3
Add the RBAC enabled kubernetes-dashboard with
version v1.8.3.

Related-Bug: #1680900
Change-Id: I68a17d22dda9661c81f40bcc9db06f7456790958
5 years ago
Spyros Trigazis 2329cb7fb4 k8s: Fix kubelet, add RBAC and pass e2e tests
Due to a few several small connected patches for the
fedora atomic driver, this patch includes 4 smaller patches.

Patch 1:
k8s: Do not start kubelet and kube-proxy on master

Patch [1], misses the removal of kubelet and kube-proxy from and therefore they are started if they
exist in the image or the script will fail.
Closes-Bug: #1726482

Patch 2:
k8s: Set require-kubeconfig when needed

From kubernetes 1.8 [1] --require-kubeconfig is deprecated and
in kubernetes 1.9 it is removed.

Add --require-kubeconfig only for k8s <= 1.8.


Closes-Bug: #1718926

Patch 3:
k8s_fedora: Add RBAC configuration

* Make certificates and kubeconfigs compatible
  with NodeAuthorizer [1].
* Add CoreDNS roles and rolebindings.
* Create the system:kube-apiserver-to-kubelet ClusterRole.
* Bind the system:kube-apiserver-to-kubelet ClusterRole to
  the kubernetes user.
* remove creation of kube-system namespaces, it is created
  by default
* update client cert generation in the conductor with
  kubernetes' requirements
* Add --insecure-bind-address= to work on
  multi-master too. The controller manager on each
  node needs to contact the apiserver (on the same node)


Closes-Bug: #1742420
Depends-On: If43c3d0a0d83c42ff1fceffe4bcc333b31dbdaab

Patch 4:
k8s_fedora: Update coredns config to pass e2e

To pass the e2e conformance tests, coredns needs to
be configured with POD-MODE verified. Otherwise, pods
won't be resolvable [1].

Closes-Bug: #1738633

Change-Id: Ibd5245ca0f5a11e1d67a2514cebb2ffe8aa5e7de
5 years ago
Feilong Wang be0609ce88 Support soft-anti-affinity policy for nodes
Currently, there is no guarantee to make sure all nodes of one cluster are
created on different compute hosts. So it would be nice if we can create
a server group and set it with anti-affinity policy to get a better HA
for cluster. This patch is proposing to create a server group for master
and minion nodes with soft-anti-affinity policy by default.

Closes-Bug: #1737802

Change-Id: Icc7a73ef55296a58bf00719ca4d1cdcc304fab86
5 years ago
Spyros Trigazis 65dfb2009f Add openstack_ca_file configuration option
In the drivers section of magnum.conf add openstack_ca_file.
This file is expected to be a CA Certificate OR CA bundle
which will be passed on every node and it will be installed
on the host's CA bundle.

Update devstack plugin to use the ssl bundle if tls-proxy is

Install the CA for drivers:

Add doc in troubleshooting-guide.

Add release notes.

Closes-Bug: #1580704
Partially-Implements: blueprint heat-agent
Change-Id: Id48fbea187da667a5e7334694c3ec17c8e2504db
5 years ago
Spyros Trigazis 273405cb5a Leverage heat-container-agent for monitoring
Use the heat-container-agent from a system container.
It means that the docker daemon can be started later.

Pass as a software deployment with the heat-agent the following
* prometheus-monitoring
** pin prometheus to v1.8.2 since its config is not 2.0.0

Add heat-container-agent container image.

Implements: blueprint heat-agent
Related-Bug: #1680900
Change-Id: I084b7fe51eddb7b36c74f9fe76cda37e8b48f646
6 years ago
Ricardo Rocha 28fff8006a Make docker_storage_driver a str instead of enum
Allow any value to be passed on the docker_storage_driver field by turning it
into a StringField (was EnumField), and remove the constraints limiting the
values to 'devicemapper' and 'overlay'.

Change the docker storage setup to have a generic setup for all drivers with
the exception of 'devicemapper', which keeps its own specific storage config
function. For all others, do the same we already did for overlay (with two
cases for usage of a cinder volume or not) and simply set the storage driver
in the docker configuration to the value provided in the cluster template.

Change-Id: I9aa8f232ce64ece4d439c0a476f463820a499617
Closes-Bug: #1722522
6 years ago
Kirsten G b07b6f34d5 Add verify_ca configuration parameter
Added configuration parameter, verify_ca, to magnum.conf with default
value of True. This parameter is passed to the heat templates to
indicate whether the cluster nodes validate the Certificate Authority
when making requests to the OpenStack APIs (Keystone, Magnum, Heat).
This configuration parameter can be set to False to disable CA

Co-Authored-By: Vijendar Komalla <>

Change-Id: Iab02cb1338b811dac0c147378dbd0e63c83f0413
Partial-Bug: #1663757
6 years ago
Spyros Trigazis 69bb03fcbe k8s_fedora: Add container_infra_prefix label
Add a label to prefix all container image use by magnum:
* kubernetes components
* coredns
* node-exporter
* kubernetes-dashboard

Using this label all containers will be pulled from the specified
registry and group in the registry.

* grafana
* prometheus

Closes-Bug: #1712810
Change-Id: Iefe02f5ebc97787ee80431e0f16f73ae8444bdc0
6 years ago
Jenkins 62afc249d1 Merge "Clean-up server names in drivers" 6 years ago
Jenkins 79460caffa Merge "Remove TENANT_NAME from /etc/sysconfig/heat-params" 6 years ago
chestack 048400bd4b Remove TENANT_NAME from /etc/sysconfig/heat-params
1. It will fail to create cluster if there is chinese in tenant name
2. TENANT_NAME is unnecessary after changing to trustee

this patch is for k8s_fedora_atomic and k8s_fedora_ironic

Change-Id: Ie072f183110ae95861fb3694a913a3a4526549fb
Close-Bug: #1711308
6 years ago
Mathieu Velten 46255dd4b1 Add a kube_tag label to control the k8s containers to pull
Separate the tag from which to pull from the kubernetes version.
With the current state the tag and the version happen to be the
the same. But, it is not decided yet in the fedoraproject how the
images are going to be tag. Finally, operators might want to try
their own container images with custom tags.

Depends-On: Icddb8ed1598f2ba1f782622f86fb6083953c3b3f
Implements: blueprint run-kube-as-container

Change-Id: I4c4bc055d7df5e65aede93464bff51e6d5971504
6 years ago
Mohammed Naser 5153d912b2
Clean-up server names in drivers
The instance type of servers at the moment can become quite long
due to the Heat autogenerated names.  This patch cleans up the names
so that they are shorter yet contain all the info needed to be able
to know where they belong to.

Change-Id: I5bcbe73f08844242d049b8408221da40d22cd3dc
6 years ago
Mark Goddard 769f0eea41 Extract kubernetes baremetal ports
Previously the master's private IP address was not pushed through to the
minion configuration when the load balancer is disabled as the heat
templates were not wired up in this case.  This change resolves that
issue and makes it possible for security groups to be applied to the
master and minion ports.

Change-Id: If85a5434f014c5a09b54dda710d13739e9bff928
Related-Bug: #1544195
6 years ago
ArchiFleKs 6ea4a7872d [k8s-fedora-atomic] fix multimaster cluster
Same fix as CoreOS for Fedora which enable multimaster with
TLS and ETCD Load balancer.

Closes-Bug: #1679724
Change-Id: I45b62a20f0a89ebd1494ad61021384fc7a416e8e
6 years ago
Jenkins 65615282d9 Merge "Remove kube-examples software configs" 6 years ago
yatinkarel 44d102a65e Add kube dashboard and remove kube ui
kube-ui [2] is deprecated and not actively maintained since long time.
Instead kubernetes dashboard [1] has lot of features and is actively
With this patch kube-ui is removed and kubernetes dashboard is added
and enabled in k8s cluster by default.

The kubernetes dashboard is enabled by default. To disable it, set the
label 'kube_dashboard_enabled' to False


Change-Id: I8864c097a3da6a602e0f25d3ff8ade788aa134a9
Implements: blueprint add-kube-dashboard
6 years ago
Spyros Trigazis 58d1fc5523 Remove kube-examples software configs
In a default nova deployment user-data for software deployments
goes into nova's db. That field is 64KB, so we are contraint by
space and we need to pass only what is needed.

Change-Id: I84bb59dbedc6e97b3bb985db5621daf34b6e17ee
Closes-Bug: #1680900
Closes-Bug: #1650538
6 years ago
Cristovao Cordeiro 248e45f75c [k8s] Monitoring with Prometheus and Grafana
Profit from the default cAdvisor deployed by k8s to deploy the
remaining monitoring stack on top, made of node-exporter,
Prometheus and Grafana.

Node-exporter is ran as a normal pod through a manifest, while
Prometheus and Grafana are deployments with 1 replica.

Prometheus has compliance with Kubernetes, so the discovery of
the nodes and other k8s components is configured directly in
Prometheus configuration.

Change-Id: If2cab996b9458580a55b5212ab298c909622e7f3
Partially-Implements: blueprint container-monitoring
6 years ago
Bertrand NOEL 1f3b0500b7 K8S: Allows to specify admission control plugins to enable
If nothing is specified a set of recommended default plugins is used,
which includes the ServiceAccount one.

Change-Id: I1383aae09ba68f8e83b07e3eaae40ab071f7be94
Closes-Bug: #1646489
6 years ago
Bertrand NOEL 2a4ad9aa41 Make Kubernetes pods' health checks configurable
Make Kubernetes' kube-controller-manager and kube-scheduler
health checks configurable as a parameter to the cluster-template
Set their value higher for all deployments. And set their value
to a high number for tests, for the CI.

Change-Id: I65e2da12487c513419125f0525a4e21bac22210e
Closes-Bug: 1648826
6 years ago
ricolin f7bf92ae24 [k8s_ironic] Move wc out of master resource group
This patch let kubemasters share same wait condition and wait condition
handler resource instead of create same function of resource for each
kubemaster node.
Partial-Bug: #1646720

Change-Id: Ib529e695409ba4519bdf0ad159a4599ef0805d54
6 years ago
Spyros Trigazis 4fa049e037 [k8s_fedora_atomic] Remove podmaster
Podmaster is deprecated since k8s 1.2 and its docker
image is v1, incompatible with docker >=1.12.

* Remove podmaster pod
* Update manifests of kube-controller-manager and kube-scheduler
* Rename SoftwareConfig to reflect the new functionality

Closes-Bug: #1646109
Change-Id: Ibf4ce06cbf5b79a4241c58c67b13a7c68145d3ae
7 years ago
Mathieu Velten 004e0cd742 Remove underscores from Nova server names
It creates a mismatch between the generated Nova name
and its hostname which can lead to weird problems.

Closes-Bug: 1645730
Change-Id: I1c7dd459caefacaf41dd77e59c1a6e1df3ef0d42
7 years ago
Spyros Trigazis f82749457c Make cinder volume optional
In the swarm_atomic and k8s_atomic drivers container images are
stored in a dedicated cinder volume per cluster node. It is
proven that this architecture can be a scalability bottleneck.

Make the use of cinder volumes for container images and opt-in
option.  If docker-volume-size is not specified no cinder
volumes will be created. Before, if docker-volume-size wasn't
specified the default value was 25.

To use cinder volumes for container storage the user will
interact with magnum as before, (meaning the valid values are
integers starting from 1).

Closes-Bug: #1638006
Change-Id: I3394c62a43bbf950b7cf0b86a71b1d9b0481d68f
7 years ago
Spyros Trigazis 2c635692ae Split k8s atomic vm and ironic drivers
The 2 k8s atomic drivers we currently support are added to the
same driver. This breaks ironic support with the stevedore
work I'm currently doing.

With stevedore, we can choose only one driver based on the
server_type, os and coe. We won't be able to pick a driver and
then choose an implementation bases on server_type.

Partially-Implements: blueprint magnum-baremetal-full-support
Co-Authored-By: Spyros Trigazis <>
Change-Id: Ic1b8103551f48f85baa2ed9ff32d5b70b1fab84e
7 years ago