4585 Commits

Author SHA1 Message Date
Theodoros Tsioutsias
14b46ea22b FakeLoopingCall raises IOError
All unittests using FakeLoopingCall raise an IOError if an initial
delay is not specified, because the default initial_dealy is -1.
Changing the default initial delay to 0.

story: 2005112
task: 29748
Change-Id: I6cbae0996c2347e25d8be617e4b3fd93f4d9cc95
2019-02-28 10:01:17 +00:00
Zuul
d76ab4da80 Merge "[k8s-fedora-atomic] Security group definition for worker nodes" 2019-02-27 23:59:12 +00:00
Lingxian Kong
31c82625d6 [k8s-fedora-atomic] Security group definition for worker nodes
Defines more strict security group rules for kubernetes worker nodes. The
ports that are open by default: default port range(30000-32767) for
external service ports; kubelet healthcheck port; Calico BGP network ports;
flannel overlay network ports. The cluster admin should manually config the
security group on the nodes where Traefik is allowed.

Story: #2005082
Task: #29661
Change-Id: Idbc67cb95133d3a4029105e6d4dc92519c816288
2019-02-27 22:15:46 +00:00
Zuul
07e48a1ed5 Merge "Add server group for cluster worker nodes" 2019-02-27 12:32:47 +00:00
Zuul
731499c460 Merge "Return instance ID of worker node" 2019-02-27 11:57:34 +00:00
Zuul
138472dcf1 Merge "Add reno for flannel reboot fix" 2019-02-27 10:00:52 +00:00
Feilong Wang
20d03919fb Return instance ID of worker node
Return the nova instance UUID of worker nodes in kubeminion
templates. We will be able to remove resources from the
ResourceGroups based on nova instance uuid.

Backstory:
In heat a ResourceGroup creates a stack of depth 2. ResourceGroups
support removal policies to declare which resources must be removed.
This can be done by passing the index of the resource or the stack_id
of the nested stack. If a stack update call receives a list of
indices (eg [0, 5, 3]) or nested stack uuid (eg [uuidA, uuidB]), it
will remove the corresponding nested stacks.

In magnum's heat templates, a nested stack logically represents a
nova compute instance which is a cluster node. Using composition in
heat, we can change the way a resources group references the nested
stacks. This proposes to use the nova instance uuid as
'OS::stack_id'.

With this change, an external consumer of the stack (the cluster
autoscaler or an actual user) can remove resources from the
ResourceGroup using the nova instance uuid or resource index. Without
this change, a user or system  (which typically knows the name,
server uuid or ip) would have to find in which nested stack a
kubernetes node belongs too.  Resulting multiple call to heat.

The end result of this patch can be verified like this:
nested_stack_id=$(openstack stack resource show <STACK_ID_OR_NAME> kube_minions -c physical_resource_id -f value)
openstack stack show "${nested_stack_id}"

Task: 29664
Story: 2005054

Change-Id: I6d776f62d640c72b3228460392b92df94fe56fe6
2019-02-27 10:46:41 +01:00
Feilong Wang
4f84c849f6 Add server group for cluster worker nodes
Now Magnums onlys has one server group for all master and worker nodes
per cluster, which is not very flexible for small cloud scale. For a
3+ master clusters, it's easily meeting the capacity when using hard
anti-affinity policy. This patch is proposing one server group for each
master and worker nodes group to have better flexibility.

story: 2004195

Change-Id: If11ba863a2aa538efe1e3e850084bdd33afd27d2
2019-02-27 09:09:20 +00:00
Nguyen Hai Truong
055384343f Add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: I5a92105f7cfbcabf521150d65f89b14cea62db0f
2019-02-23 18:01:18 +11:00
Spyros Trigazis
e6b3325120 Add reno for flannel reboot fix
Change [0] fixed the issue of reseting iptables on node reboot
when flannel was configured which made pods lose connectivity.

[0] I7f6200a4966fda1cc701749bf1f37ddc492390c5

Change-Id: I07771f2c4711b0b86a53610517abdc3dad270574
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2019-02-22 11:07:59 +01:00
Zuul
9323da7819 Merge "[k8s] Add trustee as a secret in kube-system" 2019-02-20 11:08:01 +00:00
Ricardo Rocha
ca442a7202 [k8s] Add trustee as a secret in kube-system
Add a new secret in kube-system holding the trustee information. This is
useful for any service running within kubernetes needing to contact
OpenStack services.

Change-Id: I1939fb6a33c9eb6a45697d070f58c9510be774b3
2019-02-20 09:45:14 +01:00
Zuul
91d84ff01d Merge "[k8s] Update cluster health status by native API" 2019-02-19 17:56:23 +00:00
Zuul
d239051751 Merge "[k8s] helm install metrics service" 2019-02-18 17:46:42 +00:00
Zuul
f45d62ea96 Merge "Add hidden flag to cluster template" 2019-02-18 01:28:15 +00:00
Feilong Wang
c38edc6929 [k8s] Update cluster health status by native API
Calling Kubernetes native API to update the cluster health status
so that it can used for cluster auto healing.

Task: 24593
Story: 2002742

Change-Id: Ia76eeeb2f1734dff38d9660c804d7d2d0f65b9fb
2019-02-18 11:03:10 +13:00
Diogo Guerra
230ad3f2db [k8s] helm install metrics service
* Add Folder specific for helm managed resources
* Add first use case of helm install script
* Install metrics-server with helm (parallel to heapster to allow back compatibility)
* Added extra ARGS to kube-apiserver to enable communication with metrics-server

Known Issues:
  * Tiller pod sometimes is presented as not active due to (possibly) Heartbeat/Healthz

story: 2004816
task: 28980
depends_on: I99d3a78085ba10030200f12bbfe58a72964e2326
Change-Id: I1b2432bc09ccde02e43124ed010120b99d853d65
Signed-off-by: Diogo Guerra <dy090.guerra@gmail.com>
2019-02-13 17:34:29 +01:00
Zuul
61173ec6fb Merge "[k8s_fedora] Add heat-agent to worker nodes" 2019-02-13 11:48:03 +00:00
Spyros Trigazis
b2a6a7715a [k8s_fedora] Add heat-agent to worker nodes
Start/Install heat agent in worker nodes.

task: 29140
story: 2002210
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>

Change-Id: If39d0dff3432ba132b8b56eb21b5aae80ba52450
2019-02-13 09:36:33 +00:00
Ricardo Rocha
6aee864954 Add hidden flag to cluster template
Add a new hidden flag to cluster templates. This allows an operator to
keep a cluster public (accessible to all users) while not showing them
in cluster template listing.

Story: 2004941
Task: 29342

Change-Id: Ia2717ca960041753f6e772bf2d41c7f5a196dae6
2019-02-12 11:38:15 +01:00
Spyros Trigazis
0b5f4260d9 k8s_fedora: Deploy tiller
Add enable_tiller  label to install tiller in k8s_fedora_atomic
clusters. Defaults to false.

Add tiller_tag label to select the version of tiller. If the
tag is not set the tag that matches the helm client version in
the heat-agent will be picked.  The tiller image can be stored
in a private registry and the cluster can pull it using the
container_infra_prefix label.

Install tiller securely using helper container.

TODO:

*add instructions on how RBAC is designed
https://docs.helm.sh/using_helm/#example-deploy-tiller-in-a-namespace-restricted-to-deploying-resources-in-another-namespace
* add docs on how to install addon in the cluster using this tiller
* how users can get the creds to talk to tiller

NOTE:
The main goal of this tiller is internal usage!
Users can still deploy other tillers in other namespaces.

story: 2003902
task: 26780

Change-Id: I99d3a78085ba10030200f12bbfe58a72964e2326
Signed-off-by: dioguerra <dy090.guerra@gmail.com>
2019-02-11 11:18:08 +01:00
Zuul
53e4b51e71 Merge "Fixing container-build job" 2019-02-11 08:57:54 +00:00
Erik Olof Gunnar Andersson
82344703ca Fixing container-build job
When the base zuul job moved from openstack-infra to
opendev the role openstack-zuul-jobs which we depend
on for the swap was removed.

This patch manually imports that repo.

Change-Id: I90acaf010014664879f5c0d2f1a1ef660baf498f
2019-02-10 17:31:17 -08:00
Lingxian Kong
87a743ad45 Fix typo in octavia-ingress-controller doc
Change-Id: I53eaab312fa1b3b3f5689aa0147a028ecaeb52f1
2019-02-11 11:51:55 +13:00
Zuul
1cca9cc809 Merge "Delete loadbalancers and floatingips for service and ingress" 2019-02-07 11:04:18 +00:00
Zuul
9cffb903d6 Merge "Allow overwriting labels on swarm mode creation" 2019-02-06 16:05:42 +00:00
Ricardo Rocha
b555f90c10 Allow overwriting labels on swarm mode creation
Allow passing label values on cluster creation for swarm mode. This is
available in all kubernetes drivers as well as swarm, but somehow missed
on swarm mode.

Story: 2004942
Task: 29343

Change-Id: Ie3ac66f45e27cc92993116c3df0b33873dc67e24
2019-02-06 14:28:59 +01:00
Zuul
4f0319d88b Merge "Allow cluster template being renamed" 2019-02-05 21:46:42 +00:00
Lingxian Kong
d2f36a2ed1 Delete loadbalancers and floatingips for service and ingress
This patch makes the regex more broad to cover both services and ingresses that
create load balancers in octavia. In cloud-provider-openstack repo, the load
balancer description is defined in:

- f056677572/pkg/cloudprovider/providers/openstack/openstack_loadbalancer.go (L459)
- f056677572/pkg/ingress/controller/openstack/octavia.go (L316)

The floating ip description is defined in:

- f056677572/pkg/cloudprovider/providers/openstack/openstack_loadbalancer.go (L1063)
- f056677572/pkg/ingress/controller/openstack/neutron.go (L72)

Change-Id: I7ce38e38de0d66fac340e287ce520de4c4bf679d
Story: 2004839
2019-02-05 10:12:35 +13:00
Zuul
f0b6d3e1fe Merge "Support octavia-ingress-controller" 2019-02-03 22:20:02 +00:00
Zuul
a4164eddd8 Merge "[k8s_fedora_atomic] Delete floating ip for load balancer" 2019-02-01 15:19:26 +00:00
Zuul
dad2bdfbbe Merge "[k8s-fedora-atomic] Update k8s default version" 2019-02-01 11:34:53 +00:00
Zuul
2d07719867 Merge "heat-agent: Add openssh-clients" 2019-02-01 11:34:51 +00:00
Lingxian Kong
a941822c8e Support octavia-ingress-controller
- Add "octavia" as one of the "ingress_controller" options.
- Add label "octavia_ingress_controller_tag".
- Use external network ID in the heat templates.

Story: 2004838
Change-Id: I7d889a054cd5feb2eeef523b20607a6c7630d777
2019-01-31 14:49:32 +13:00
Spyros Trigazis
e7efb49aac heat-agent: Add openssh-clients
Add openssh-client to the heat-agent to act on the
host.

story: 2002210
task: 29142

Change-Id: I6e52291e4fc750418c70a22cc386034fa729d765
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2019-01-30 23:08:27 +00:00
Feilong Wang
f69a54d979 [k8s-fedora-atomic] Update k8s default version
And also update the default value of keystone_auth_enabled from False
to True in favor of prompting the integration of OpenStack and K8s.

Change-Id: I0fcea762d467e1afeecb175f65b9b13ad9ee1f71
2019-01-31 11:15:45 +13:00
Feilong Wang
0a5fd471c3 Support multi k8s image versions
Change-Id: Icae30a7d6c5f779e75587f398d06dc149378e827
2019-01-30 12:59:24 +13:00
Feilong Wang
766a64a41a Allow cluster template being renamed
To get a better cluster template versioning and relieve the pain
of maintaining public cluster template, the patch is proposing
that the name of cluster template can be changed.

A folllowing patch/spec will be proposed to add a new field
'deprecated' to allow ops to hide old/deprecated templates.

Task: 26889
Story: 2003960

Change-Id: Id1db81d35bc3dccff0fac481be7801de200d52de
2019-01-28 18:57:16 +00:00
Spyros Trigazis
60f9799374 ci: Rebuild kubernetes v1.11.6 containers
Change-Id: I531bf1fef6a87d8fbbf8f16c22c3db6a80797b69
2019-01-28 11:56:57 +01:00
Zuul
77435e5797 Merge "Update kube cmd documentation links" 2019-01-28 09:32:44 +00:00
Zuul
f241da6a80 Merge "Enable CoreDNS prometheus metrics plugin" 2019-01-28 09:32:41 +00:00
Ricardo Rocha
6a9c490bd8 Update kube cmd documentation links
Change-Id: I2f6aca27d0f640f01e8e031f759479c776b2523a
2019-01-28 10:10:15 +01:00
akhiljain23
2ccf639a39 Add framework for magnum-status upgrade check
This commit adds the functionality of magnum-status CLI for performing
upgrade checks as part of the Stein cycle upgrade-checkers goal.
It only includes a sample check which must be replaced by real checks in
future.

Change-Id: Ia8a74fd8bd5a804e71bb04eb0615fa114a517bc4
Story: 2003657
Task: 26138
2019-01-27 04:26:30 +00:00
Lingxian Kong
f63761a804 [k8s_fedora_atomic] Delete floating ip for load balancer
When user creates LoadBalancer type service in k8s cluster, a floating
ip may be created and associated with the load balancer VIP. Magnum
now could delete the load balancers automatically in the cluster
pre-delete method, should also remove the floating ip as needed.

This patch depends on the github PR for cloud-provider-openstack:
https://github.com/kubernetes/cloud-provider-openstack/pull/433

Story: 2004836
Change-Id: Ia553aff4e66033346c6bfe120a72992bec79e136
2019-01-25 12:15:08 +13:00
Zuul
1b8a3ba163 Merge "Use oslo_serialization instead of the json module directly" 2019-01-23 22:53:24 +00:00
Zuul
7e0b4b32f0 Merge "support http/https proxy for discovery url" 2019-01-23 10:42:03 +00:00
Zuul
ed4223786e Merge "Fix prometheus monitoring" 2019-01-23 00:00:28 +00:00
Erik Olof Gunnar Andersson
daf34d9df8 Use oslo_serialization instead of the json module directly
* Always use oslo jsonutils.
* Consistently import jsonutils as-is.
* Use dump_as_bytes instead of dumps.
https://wiki.openstack.org/wiki/Python3#Serialization:_base64.2C_JSON.2C_etc.

Change-Id: I2b65faa7df43a1d58205e8ff106ff62f73d78198
2019-01-21 16:28:03 -08:00
Zuul
c0c5502577 Merge "Use python3 for functional tests" 2019-01-19 15:08:41 +00:00
Zuul
70b89f07e4 Merge "Use MultiType and types.text instead of str" 2019-01-18 12:56:38 +00:00