Without this, heat container agents using kubectl version
1.18.x (e.g. ussuri-dev) fail because they do not have the correct
KUBECONFIG in the environment.
Task: 39938
Story: 2007591
Change-Id: Ifc212478ae09c658adeb6ba4c8e8afc8943e3977
We kept introspecting the name of the instance with the assumption
that the network always existed under .novalocal
This is not always the case, with certain variables changed inside
Neutron it is possible to control this, therefore, leading in failing
deploys.
With this change, we pass the instance name directly to the cluster
and therefore we always have the accurate name.
Task: 36160
Story: 2006371
Change-Id: I2ba32844b822ffc14da043e6ef7d071bb62a22ee
Instead of using "print xxx", it'd better to use "print(xxx)"
to make it supports both py2.x and py3.x for make-cert.sh and
make-cert-client.sh
Task: 36024
Story: 2006283
Change-Id: Iaf6ab7172367bc96d19f2d1236cc038460a7d461
Rolling ugprade is an important feature for a managed k8s service,
at this stage, two user cases will be covered:
1. Upgrade base operating system
2. Upgrade k8s version
Known limitation: When doing operating system upgrade, there is no
chance to call kubectl drain to evict pods on that node.
Task: 30185
Story: 2002210
Change-Id: Ibbed59bc135969174a20e5243ff8464908801a23
USER_TOKEN variable is empty because the grep expression
is not ignoring case and certs are not created.
Change-Id: I175cb2d4c64d5f7024b13ce11c1184029f63c317
Task: 26189
Story: 2003671
Currently the option of selecting no floating IP will not apply to
a multimaster configuration and loadbalancers will be expected to use
floating IPs. This patch allows the floating IP resources to be
disabled among the load balancers.
Task: 22121
Story: 2002557
Change-Id: I8f96fba8aa41319ac209baedd9d3a927aad0eb91
Multi master deployments for k8s driver use different service account
keys for each api/controller manager server which leads to 401 errors
for service accounts. This patch will create a signed cert and private
key for k8s service account keys explicitly, dedicatedly for the k8s
cluster to avoid the inconsistent keys issue.
Task: 21653
Story: 1766546
Change-Id: I61547405f866d3c5a84da63de66724b55af1066a
By current design, pods under kube-system will run on minion nodes. And
given now we're not running kubelet on master node, so calico-node is
not running on k8s master node. As a result, kubectl proxy is not
working to access dashboard. And it's confirmed with calico team that
the calico-node container must be running on master node if user want
to use kubectl proxy, see [1]. So, the solution is enabling kubelet
on master but disallow the other pods scheduled on master with
taint/tolerations.
Besides, this patch includes another fix about running calico on
Fedora Atomic. Because Fedora Atomic is using NetworkManager, it
manipulates the routing table for interfaces in the default network
namespace where Calico veth pairs are anchored for connections to
containers. This can interfere with the Calico agent’s ability to
route correctly. Please see more information about this at [2].
[1] https://docs.projectcalico.org/v3.0/getting-started/kubernetes/
installation/integration#about-the-calico-components
[2] https://docs.projectcalico.org/master/usage/troubleshooting/
#configure-networkmanager
Closes-Bug: #1751978
Change-Id: Iacd964806a28b3ca6ba3e037c60060f0957d44aa
In Fedora Atomic 27 etcd and flanneld are removed from the base image.
Install them as a system containers.
* update docker-storage configuration
* add etcd and flannel tags as labels
Change-Id: I2103c7c3d50f4b68ddc11abff72bc9e3f22839f3
Closes-Bug: #1735381
Due to a few several small connected patches for the
fedora atomic driver, this patch includes 4 smaller patches.
Patch 1:
k8s: Do not start kubelet and kube-proxy on master
Patch [1], misses the removal of kubelet and kube-proxy from
enable-services-master.sh and therefore they are started if they
exist in the image or the script will fail.
https://review.openstack.org/#/c/533593/
Closes-Bug: #1726482
Patch 2:
k8s: Set require-kubeconfig when needed
From kubernetes 1.8 [1] --require-kubeconfig is deprecated and
in kubernetes 1.9 it is removed.
Add --require-kubeconfig only for k8s <= 1.8.
[1] https://github.com/kubernetes/kubernetes/issues/36745
Closes-Bug: #1718926https://review.openstack.org/#/c/534309/
Patch 3:
k8s_fedora: Add RBAC configuration
* Make certificates and kubeconfigs compatible
with NodeAuthorizer [1].
* Add CoreDNS roles and rolebindings.
* Create the system:kube-apiserver-to-kubelet ClusterRole.
* Bind the system:kube-apiserver-to-kubelet ClusterRole to
the kubernetes user.
* remove creation of kube-system namespaces, it is created
by default
* update client cert generation in the conductor with
kubernetes' requirements
* Add --insecure-bind-address=127.0.0.1 to work on
multi-master too. The controller manager on each
node needs to contact the apiserver (on the same node)
on 127.0.0.1:8080
[1] https://kubernetes.io/docs/admin/authorization/node/
Closes-Bug: #1742420
Depends-On: If43c3d0a0d83c42ff1fceffe4bcc333b31dbdaab
https://review.openstack.org/#/c/527103/
Patch 4:
k8s_fedora: Update coredns config to pass e2e
To pass the e2e conformance tests, coredns needs to
be configured with POD-MODE verified. Otherwise, pods
won't be resolvable [1].
[1] https://github.com/coredns/coredns/tree/master/plugin/kuberneteshttps://review.openstack.org/#/c/528566/
Closes-Bug: #1738633
Change-Id: Ibd5245ca0f5a11e1d67a2514cebb2ffe8aa5e7de
Added configuration parameter, verify_ca, to magnum.conf with default
value of True. This parameter is passed to the heat templates to
indicate whether the cluster nodes validate the Certificate Authority
when making requests to the OpenStack APIs (Keystone, Magnum, Heat).
This configuration parameter can be set to False to disable CA
validation.
Co-Authored-By: Vijendar Komalla <vijendar.komalla@rackspace.com>
Change-Id: Iab02cb1338b811dac0c147378dbd0e63c83f0413
Partial-Bug: #1663757
Kubernetes uses cetificates, kubeconfig and the kubernetes openstack
cloud provider configuration from /srv/kubernetes and /etc/sysconfig.
The upstream kubernetes system containers used with atomic hosts
mounts /etc/kubernetes, we can unify the location of all kubernetes
configuration and also be able to use the upstream containers
unmodified.
Implements: blueprint run-kube-as-container
Change-Id: I9b2da390745836d9a66b7c8fc995a35cb74993e9
In kubernetes with atomic we have a set of certificates that we use in
three places:
1. etcd
2. kubernetes apiserver
3. kubernetes service accounts
In order to make service accounts work we need to set the common name
properly in the certificates.
Partial-Bug: #1705694
Change-Id: I04ed3bba938f0d5f340e2141be94058c38c2ed2b
Same fix as CoreOS for Fedora which enable multimaster with
TLS and ETCD Load balancer.
Closes-Bug: #1679724
Change-Id: I45b62a20f0a89ebd1494ad61021384fc7a416e8e
Allow to specify a custom AUTH_URL for the templates in case instances
cannot reach internalURL which is the case in mose deployment.
A new variable in trust section: trustee_keystone_interface which
default to public is introduced.
Change-Id: I2a908c0752387e4ff4ad2b0fdf0c1025a73ce806
Closes-Bug: #1643197
By default, API service with service account is accessible from inside
the cluster at the address 10.254.0.1. This IP should be added to SANS
when generating the certs.
Fixes-bug: #1660811
Change-Id: I214b4296bea55bb0c4015165c56fbd8ca3cebd39
This commit addresses multiple potential vulnerabilities in
Magnum. It makes the following changes:
* Permissions for /etc/sysconfig/heat-params inside Magnum
created instances are tightened to 0600 (used to be 0755).
* Certificate retrieval is modified to work without the need
for a Keystone trust.
* The cluster's Keystone trust id is only passed into
instances for clusters where that is actually needed. This
prevents the trustee user from consuming the trust in cases
where it is not needed.
* The configuration setting trust/cluster_user_trust (False by
default) is introduced. It needs to be explicitely enabled
by the cloud operator to allow clusters that need the
trust_id to be passed into instances to work. Without this
setting, attempts to create such clusters will fail.
Please note, that none of these changes apply to existing
clusters. They will have to be deleted and rebuilt to benefit
from these changes.
Change-Id: I643d408cde0d6e30812cf6429fb7118184793400
Otherwise, the magnum certificates API will return a 406 Not
Acceptable error.
Change-Id: I0d59bf71b62bdd4204cd32d26ef3f2fc30f8f180
Closes-Bug: #1659423
With this patch following are done:-
- Configure Etcd with TLS support
Configure Following to commuicate with TLS enabled Etcd:-
- Flannel
Etcd also listens at http://127.0.0.1:2379, so on master nodes
etcdctl and kube apiserver can communicate without using
certificates.
if TLS_DISABLED="True" then TLS is not enabled for etcd.
Change-Id: I2147b67c4e346a4415e1f76c19ac68e94cb0a0fa
Partially-Implements: blueprint secure-etcd-cluster-coe
Similarly to pep8 checks, this allows enforcing a consistent
style of the shell scripts accross modfications. For now
only the indentation is enforced to reduce code churn.
Closes-Bug: 1648099
Change-Id: Ie66cbe1aea4bd01a8bba8833ef6cbd2cff6a7c6a
A lot of deployments use self signed certs. Curl breaks in those
cases trying to validate certs against known set of CAs
Change-Id: Ib36f9a99a91ce2c4d2141421ab7295303ead716f
The 2 k8s atomic drivers we currently support are added to the
same driver. This breaks ironic support with the stevedore
work I'm currently doing.
With stevedore, we can choose only one driver based on the
server_type, os and coe. We won't be able to pick a driver and
then choose an implementation bases on server_type.
Partially-Implements: blueprint magnum-baremetal-full-support
Co-Authored-By: Spyros Trigazis <strigazi@gmail.com>
Change-Id: Ic1b8103551f48f85baa2ed9ff32d5b70b1fab84e
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel. This patch updates Bay to Cluster in DB and
Object as well as all the usages. No functionality should be
changed by this patch, just naming and db updates.
Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
Followup to [1], to fix long lines and if expressions.
[1] I0c320ef392fb424755730d4572e744f9c3852c87
Change-Id: I0a29def2b6511f1d8f6190c2e07974ac8d235b46
Related-Bug #1599680
This changes were merged into upstream once[1], but lost now.
So I added this changes again.
And this fixes issues which occurs if "KUBE_API_PUBLIC_ADDRESS"
or "KUBE_API_PRIVATE_ADDRESS" are blank.
[1]: Idb30971e205e8fd94a478ef7d0bc6a30f5cab534
Change-Id: I0c320ef392fb424755730d4572e744f9c3852c87
Closes-Bug: #1599680
Some keystone auth requests respond with whitespace characters that
our cloud-init scripts do not handle properly. This patch removes the
problematic whitespace before the auth token is used.
Co-Authored By: Dolph Mathews <dolph.mathews@gmail.com>
Change-Id: I921a82a1ab61d8adc07be28370351092198742a1
Closes-bug: 1603240
`make-cert.sh` scripts uses metadata service to get node public/private
addresses. But it's better to remove dependency of metadata service
if it is possible.
This patch make `make-cert.sh` use KUBE_NODE_IP/KUBE_NODE_PUBLIC_IP
for setting SANs.
Change-Id: Idb30971e205e8fd94a478ef7d0bc6a30f5cab534
Closes-Bug: #1599680
Sometimes heat-params refer to undeclared variables when we update
`write-heat-params.yaml` and make-cert.sh will be broken.
This problem is difficult to find root cause, so we should load
`/etc/sysconfig/heat-params` before `set -o nounset` in make-cert.sh.
Change-Id: I276b22836833c10b9b6222c3cb4dc242da88dbd7
* Rename tenant to project in functional test In particular,
rename "admin_tenant_name" to "admin_project_name" and
rename "tenant_name" to "project_name". These configs have been
renamed in tempest [1], so Magnum needs to adapt the change.
* Temporarily remove "subjectAltName" from CSR config, because
it caused failure of certificate signing, possibly, due to a
bug or imcompatible change in pyOpenSSL 16.0.0.
[1] https://review.openstack.org/#/c/301167/
Change-Id: I41438cbfaefd9d04d7e73a4f46f5ece1494b349c
Closes-Bug: #1568212
Closes-Bug: #1567691
Change the devstack image to the default Atomic 23 image.
Update templates not to rely on interface naming because the names
changed between Atomic 21 and Atomic 23 images.
Add a functional test to validate minion node registration.
Update docs to refer to a generic Atomic image.
Co-Authored-By: Hongbin Lu <hongbin.lu@huawei.com>
Implements: blueprint atomic23
Change-Id: Id67244e3f452b3c7c3c46fdabcca3a5c141a76c0
Currently, we use the users auth token, which expires after a while.
We need to use a trust instead.
Change-Id: Ie18e6a1371871720fbfd4af0bd43e166075b0c6d
Closes-Bug: #1503863
Partially-Implements: blueprint use-trust-for-tls-cert-generation
At this time, server.key and client.key which are used by
kubernetes related daemon are readable. These keys should
be private.
Change-Id: I52232001f6983845f8d7de75486fe5672a57949a
Closes-Bug: #1515844
Rename heat-kubernetes to kubernetes, heat-mesos to mesos,
docker-swarm to swarm in templates. We use heat templates and
no other methods, so I think it is unnecessary to add heat before
coe. kubernetes, mesos, swarm are better than
heat-kubernetes, heat-mesos, docker-swarm.
Change-Id: I257b35c1c4ef55d3172095736f550f2c55c8d81f
Closes-Bug: #1514682
This patch is meant to cleanup indentations in the templates, as
well some other house keeping fixes.
Closes-Bug: #1504685
Change-Id: I43f6cca8f449dea2db4de6a5308bfa5e4551838e
This patch modifies template to generate certificates and configure TLS
settings for kube-apiserver/kubelet/kube-proxy.
Co-Authored-By: Andrew Melton <andrew.melton@rackspace.com>
Partially-Implements: bp secure-kubernetes
Change-Id: I76b0f91f0c44f9880980e35c6b8856ea48ed3ce1
Bharat Kunwar
Spyros Trigazis
Mohammed Naser
Feilong Wang
Manuel Rodriguez
Jim Bach
Spyros Trigazis
Kirsten G
Mathieu Velten
ArchiFleKs
Johannes Grassler
Jason Dunsmore
yatin
Dirk Mueller
Abhishek Chanda
Jaycen Grant
OTSUKA, Yuanying
Adrian Otto
Vijendar Komalla
PanFengyun
Hua Wang
Hongbin Lu
Corey O'Brien
Zachary Sais