This patch replaces the "kuryr-kubernetes" handling used by tacker's
FT to build the k8s environment with "devstack-plugin-container".
Also, with the update of devstack-plugin-container, k8s, cri-o and
helm will be upgraded.
k8s: 1.26.8 -> 1.30.5
crio: 1.26 -> 1.30.5
helm: 3.11.3 -> 3.15.4
The following is a summary of the fixes in this patch.
* Remove plugins and settings related to kuryr-kubernetes
* Rename parameters with "kuryr"
* Modify devstack-plugin-container to be used in FT k8s environment
build
* Add parameters required by devstack-plugin-container
Also, the following is a list of problems that occurred when setting
up the k8s environment with devstack-plugin-container and how to fix
them.
Cannot get bearer_token value:
- modified file: roles/setup-default-vim/tasks/main.yaml
- The task "Get admin token from described secret" of the Ansible
role "setup-default-vim" failed to obtain the value of
bearer_token, which is set as a parameter when creating vim,
causing an error. Retrying to obtain token fixed the problem.
Unknown error in "Create clusterrolebinding on k8s server" task:
- modified file: roles/setup-k8s-nodes/tasks/main.yaml
- In task "Create clusterrolebinding on k8s server" in Ansible role
"setup-k8s-oidc", `failed to download openapi: unknown;` error
occurred. The cause was that the pod status of kube-apiserver was
"Pending" after executing the previous "Wait for k8s apiserver to
restart" task. The error was fixed by waiting for the Pod status
to reach the "Running" state.
"cni0" is not assigned the intended IP address:
- added file: roles/restart-kubelet-service/tasks/main.yaml
- When using devstack-plugin-container to create a k8s environment
and deploy a Pod, the Pod deployment fails with the error `network:
failed to set bridge addr: "cni0" already has an IP address
different from 10.x.x.x`. Removing the associated interface and
restarting the service cleared the error.
Depends-On: https://review.opendev.org/c/openstack/devstack-plugin-container/+/926709
Change-Id: I596a2339f6a3c78fee99b92d7bfb65a6b0244901
Previously there are a lot of warning messages of setuptools
in the log files of tox jobs. This is mainly caused by test
samples because these are in python codes directories.
This patch moves test samples from under {tacker_root}/tacker/
to under {tacker_root}/samples/.
This patch reduces log messages drastically. For example,
the log file of py38 tox job decreased about 34,000 lines.
Change-Id: I8187ef892c6fe8be323fa5cc20969d298843f1ea
This patch removes documentation for the following legacy features.
- Extensions
- Virtualized Network Function Descriptors (VNFDs)
- Virtualized Network Functions (VNFs)
- Events
- VNF Forwarding Graph Descriptors (VNFFGDs)
- VNF Forwarding Graphs (VNFFGs)
- Network Forwarding Paths (NFPs)
- Service Function Chains (SFCs)
- Classifiers
- Network Service Descriptors (NSD)
- Network Services (NSs)
And also remove the following tools that are currently not in use.
- tools/vnfc/build_image.sh
- tools/check_i18n_test_case.txt
- tools/check_i18n.py
- tools/i18n_cfg.py
- tools/clean.sh
- tools/prepare_function_test.sh
- tools/test-setup.sh
- tools/install_venv_common.py
- tools/install_venv.py
- tools/with_venv.sh
For the following tools used in FT, move them under
`tacker/tests/functional/tools`.
- tools/test-setup-default-vim.sh
- tools/test-setup-k8s-vim.sh
- tools/test-setup-mgmt.sh
- tools/test-setup-fake-prometheus-server.sh
Implements: blueprint deprecate-legacy-apis
Change-Id: Iea89c32f69ccbe47badcfddcf77430abda98362b
In v2 API, when using vim with `ETSINFV.HELM.V_3` type, you must
set the `ssl_ca_cert` information. Currently, when registering vim
with kubernetes type and use_helm parameter, it will succeed even
if you don't set `ssl_ca_cert` information. This causes v2 APIs to
fail when you use the vimConnectionInfo stored in the database.
This patch added a check in register vim to fix this issue. If the
helm is used to register the vim of kubernetes type, the
`ssl_ca_cert` parameter will be checked. If it's not set, the
registration will fail.
At the same time, FT test items have been added to verify that
v2 API operations perform properly when using the `ETSINFV.HELM.V_3`
type vim that exists in the database.
Implements: blueprint helmchart-k8s-vim
Change-Id: I629e347413b242ab9e1a5db16c52ca222adc3873
This patch modifies the task replacement rules in set-default-vim,
enabling zuul to create kubernetes vim using oidc. At the same time,
the FT code of v1-related oidc is modified so that it can use
oidc-related vim during testing.
Closes-Bug: #2007054
Change-Id: Idc27bd2609582ed949041d2f8c730eec686f5102
This patch adds openid token auth support when calling k8s APIs.
Openid token auth of k8s relies on an external openid provider,
and Keycloak acts as the openid provider in this implementation.
Implements: blueprint support-openid-k8s-vim
Change-Id: Ie5e080a20cba3ba0ed514ede7955eb16729d797c
When initializing k8s client in InfraDriverV2, the SSL CA
certificate is set incorrectly.
To fix the issue, the following modifies are made in this patch:
* A temp file for ssl_ca_cert is created before initializing
k8s client and the temp file path is set to k8s_config.ssl_ca_cert,
* The temp file is deleted until the lifetime of k8s client ends.
Note: This references the implementation in InfraDriverV1.
If set the ssl_ca_cert in instantiate request, the validation of
request is failed because of the length of ssl_ca_cert exceeds 1024.
For this issue, add a new type `keyvalue_pairs_no_length_limit`
which has no max length limitation to verify the request.
And the interfaceInfo, accessInfo, extra are all set to the new type
for unity.
In Zuul test environment, when registering default vim, ssl_ca_cert
is not set. So the case with ssl_ca_cert is not tested.
In this patch ssl_ca_cert is set into the default vim.
Closes-Bug: #1979413
Change-Id: I61dbd70690b737a72fc619e5a08b4bab51160a27
This patch supports MgmtDriver in the operation of modifying VNF.
It provides a sample script MgmtDriver, when modifying CNF,
If the ConfigMap and Secret are updated, the Pod and Deployment will
also be updated (image only).
Implements: blueprint container-update
Change-Id: I1e7a1b03fef13f4c7a83488f6d2fdd7efc2e454b
kuryr-kubernetes patch [1] that changes to use kubeadm for
installing Kubernetes on devstack.
A patch [2] was previously created to address the change, but the fix
used kuryr-Kubernetes stable/wallaby as a temporary fix.
This patch fixes to use latest kuryr-kubernetes.
* Change the ".zuul.yaml" setting to use the latest kuryr-kubernetes.
* Add the process of creating a ServiceAccount and change the vim
authentication method to bearer_token. This is because
kuryr-kubernetes does not create an admin ServiceAccount by default.
[1] https://review.opendev.org/c/openstack/kuryr-kubernetes/+/779250
[2] https://review.opendev.org/c/openstack/tacker/+/791252
Change-Id: Ib64183b5e978774811f51f8af0f4590a20ced856
This patch add a kubernetes cluster for the kubernetes related
functional tests of the VNF LCM in the zuul environment.
There is no impact to the existing jobs because this patch only
add a new job, however we may need to watch the load on the Zuul
environment due to its parallel jobs.
A new node-set consists of four nodes;
* Controller: Keystone, Nova, Neutron, Glance, Cinder, Octavia,
MySQL, MQ, ETCD
* Controller-tacker: Tacker, Tacker-conductor
* Controller-k8s: kuryr-k8s, kuryr-CNI, k8s-api, kubelet
* Compute: Nova-compute
All kubernetes resources are created on the controller-k8s node.
This patch includes the following changes:
* Added a execution command for the functional test of related
k8s for the VNF LCM in tox.ini.
* Registered a vim of the `kubernetes` type by ansible. Also
added related materials.
* Moved the functional test files for k8s to other new directory.
* Fixed a minor invalid definition in the definition file used
for functional testing.
Change-Id: I1621b904450e94d6793b4c524de6785520f2e805
The number of required plugins has increased to the extent that
controller on Zuul FT infrastructure almost runs out of memory [1].
This potentially induces various problems such as FT failure noises
or POST_FAILURE at the ansible task 'export-devstack-journal'.
To request expanded images such as ubuntu-focal-expanded (16GB) or
ubuntu-focal-32GB would be an option, but it turns out that would
lead us to another problematic situation. [2]
This patch, instead, addresses the issue by subdividing the memory
load. As a first step, introduce a new subnode 'tacker-controller',
on which both tacker-server and tacker-conductor are located.
Note:
* when we re-locate some other components to this new subnode, it might
better to rename it.
* `devstack_local_conf: {}` in .zuul.yaml is to cancel out the global
job.vars devstack_local_conf.post-config.$NEUTRON_DHCP_CONF, which
is not present on 'tacker-controller' in the first place.
* TACKER_MODE is set to 'standalone'. 'all-in-one' supposes core
services like nova, neutron, keystone, etc. api servers are located
on the same host as tacker-server.
* in devstack/lib/tacker:create_tacker_accounts, SERVICE_HOST should
have been TACKER_HOST. this minor fix is included.
* in roles/setup-default-vim/tasks/main.yaml, the same where conditions
were scattered but all tasks in it just needed to run on 'controller'
only. so let us wrap them all in a block.
* renamed devstack/plugin.sh:tacker_register_default_vim for clarity.
* policy file modification for Heat is now done by an ansible task.
it frees us from the co-location requirements for Tacker and Heat.
* drop devstack/lib/tacker:is_tacker_enabled as it's no longer needed.
[1]: we investigated how severe the memory load on 'controller' was
on Zuul FT infrastructure:
* The highest memory-consuming processes in desc order:
808.70MB (9.87%) 828112 /usr/sbin/mysqld
179.81MB (2.19%) 184124 ... /usr/local/bin/tacker-server ...
152.57MB (1.86%) 156232 ... /usr/local/bin/tacker-conductor .
146.67MB (1.79%) 150188 ... /usr/local/bin/neutron-server ...
132.96MB (1.62%) 136148 ... /usr/local/bin/neutron-server ...
129.08MB (1.58%) 132180 ... /usr/local/bin/heat-engine ...
127.48MB (1.56%) 130544 ... /usr/local/bin/heat-engine ...
122.16MB (1.49%) 125092 nova-apiuWSGI worker 1
121.00MB (1.48%) 123900 neutron-openvswitch-agent ...
119.50MB (1.46%) 122368 cinder-apiuWSGI worker 1
---(snip)---
* `free -m` output
total used free shared buff/cache available
Mem: 7955 7427 196 16 331 219
Swap: 1022 1019 3
[2]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/
%23openstack-infra.2020-11-25.log.html
Change-Id: I030ffd5fd11b7ca9abca56e85e449ed4c4d709bd
There are some placeholder files for running functional test in zuul.
The contents of the files, such as `auth_url` of the following files,
are updated while running devstack `stack.sh`.
* tacker/tests/etc/samples/local-vim.yaml
* tools/test-setup-default-vim.sh
These Git management files including the above, should not be
modified. Therefore, I will be fix as follow.
The need files to be change for Zuul are deploy by Ansible,
so the deploy process is run with Ansible instead of the
`devstack/lib/tacker` script.
Allow `tools/test-setup-default-vim.sh` script to be executed
by itself, so the git management files aren't updated by running
devstack `stack.sh`.
Also, as other improvements, replace the file path included in
Ansible role to definition value.
Change-Id: Iad88c7adfe56e926ee0324d94787577ba066989b
Closes-Bug: #1879303
In patch [1], the mode of tools/test-setup-default-vim.sh shell
script file was changed from 755 to 644 because of which it's fails to
run the script which registers a default vim 'VIM0' needed to run
for some of the functional tests.
This patch reverts back the mode of test-setup-default-vim.sh
from 644 to 755 and logs meaningful error message in case
the script file doesn't exists or it doesn't have execute permission.
[1] : https://review.opendev.org/#/c/674761
Change-Id: I4651bdd89da12720416e630173f80cc788911307
Closes-Bug: #1874007
