This patch fixes the following issues:
1. Change the QueuePool settings to use the values recommended.
The following error occurred in sol-vnflcm and sol-kubernetes job:
```
tacker.wsgi sqlalchemy.exc.TimeoutError:
QueuePool limit of size 10 overflow 20 reached,
connection timed out, timeout 10.00
```
Change to remove the process of overriding the default value of
oslo.db config. In Neutron, the process of overriding the default
value of oslo.db has already been removed [1].
However, errors still occur when using the recommended value, so
temporarily set `max_pool_size` to unlimited in FT.
2. Fix issue of `Show subscription (V1 API)`.
A previous patch [2] included `sqlalchemy.row.LegacyRow` for
backwards compatibility with oslo.db, which caused the error
"Show subscription (V1 API)" to fail by upgrading SQLAlchemy.
Fix it to work correctly by removing `row.LegacyRow`.
3. Fix to upload image successfully in FT.
There was a process to get the image from the glance and upload the
copy image, but it was not working properly.
Changed the argument to glance so that it uploads correctly.
4. Increase wait time Terminate after Heal in sol-vnflcm job.
A 409 error occurred at the time of Terminate because the process
was to wait a fixed number of seconds after Heal before Terminate.
The current 20 seconds may not be enough, so increase it to 60
seconds. This will be fixed to check if OpOcc is COMPLETED status
in future work.
5. Split test jobs for sol-vnflcm.
Split test jobs for sol-vnflcm because it takes longer to run and,
it is executed using the full limit time.
[1] https://review.opendev.org/c/openstack/neutron/+/456419
[2] https://review.opendev.org/c/openstack/tacker/+/910720
Closes-Bug: #2066047
Change-Id: If1cd835c8875fa844f9eac5bff5ba85103ad91aa
This commit squash the multiple changes:
Changes 1:
Keep new RBAC disable by default
oslo.policy has enabled the new RBAC config options
enforce_scope and enforce_new_defaults by default[1][2].
Tacker implemented the new RBAC in previous cycle with new
RBAC disable by default. To give more time to operator, let's
continue the same setting in this release also.
Also, there are many test modification is needed for the new
RBAC (using the new RBAC default role in tests)
Unit tests log a lot of policy warnings, suppressing those[3]
-https://fa3204066787dd37fd86-ea893277118f144d3b928cfbb4823c04.ssl.cf1.rackcdn.com/926089/1/check/openstack-tox-py311/9df6631/testr_results.html
As oslo.policy enable them by default, we override the setting
for the Tacker.
NOTE: there is no change in behaviour, tacker continue with the
old RBAC as default.
ref: https://review.opendev.org/c/openstack/requirements/+/925464
[1] https://review.opendev.org/c/openstack/oslo.policy/+/924283
[2] https://review.opendev.org/c/openstack/releases/+/925032
Change 2:
Co-Authored-By: Ayumu Ueha <ueha.ayumu@fujitsu.com>
This fixes the issues when "Set GLOBAL_VENV=true".
1: When running python in the subprocess on venv, the library import
may fail to execute properly.
This patch changes the command passed to subprocess from "python"
to "sys.executable" [4].
2: Remove compute node from nodeset for kubernetes jobs because it is
not used in kubernetes jobs and to save the resources.
Also left "GLOBAL_VENV=false" on the jobs related kubernetes to avoid
the issue of setup on kuryr-kubernetes.
[4] https://docs.python.org/3/library/sys.html#sys.executable
[3]
-----------
File "/home/zuul/src/opendev.org/openstack/tacker/tacker/policy.py", line 209, in authorize
result = _ENFORCER.authorize(action, target, credentials,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1189, in authorize
return self.enforce(
^^^^^^^^^^^^^
File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1093, in enforce
raise exc(*args, **kwargs)
tacker.common.exceptions.PolicyNotAuthorized: Policy doesn't allow os_nfv_orchestration_api:vnf_instances:cancel to be performed.
-----------
Change-Id: I4f736660e3b1079883a7434847222f6a6589377f
It's just a quick fix and intended to be not a drastic or complete
update but step-by-step.
Since the recent update of versions of sqlalchemy[1], several util
features such as autocommit or subtransaction have been dropped, or
some functions and attributes in alembic migration scripts called
from `tacker-db-manage` haven't work anymore.
All the things should be updated are described in the migration
guide[2] and will completed later.
This update is for following fixes, and including a fix of CI's
update [3] which is required to this update.
* Remove 'subtransaction=True' in `context.session.begin()`, or
replace with 'nested=True' for subtransaction part itself.
* Remove autocommit in `context_manager` and get_session().
* Add `commit()` to all sessions explicitly instead of dropping
autocommit. `flush()` is also replaced with `commit()`.
* In alembic migration scripts, change a way of creating session
in a manner of SQLAlchemy 2.0.
* Update `TackerBase` to add `__allow_unmapped__`.
* Replace argument of `joinedload()` from string to model object
because it's not supported in SQLAlchemy 2.0 anymore.
* Reemove argument table's `insert()`.
* Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0.
* Move to non-voting for unknown failures emerged recently.
* tacker-functional-devstack-multinode-sol-vnflcm
* tacker-functional-devstack-multinode-sol-kubernetes
* tacker-functional-devstack-multinode-sol-v2-vnflcm
* Skip functions for some failure of timeout cannot find out the
cause and fix.
* test_subscription_functionality in the job
tacker-functional-devstack-multinode-sol-multi-tenant
* test_inst_heal_term in the job
tacker-functional-devstack-multinode-sol-separated-nfvo
* Divide test scenarios under 'tacker/tests/functional/sol' and
'tacker/tests/functional/sol_v2' because the total time has become
over the limitation.
* Revise logging for better analysis.
[1] https://review.opendev.org/c/openstack/requirements/+/879743
[2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html
[3] https://review.opendev.org/c/openstack/tacker/+/921528
Closes-Bug: #2066047
Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
This patch obsoletes Legacy APIs excluding VIM feature.
And this patch mainly contains the following changes:
* Drop the implementation/db of NS and VNFFG functions.
* Remove the implementation that was used only by Legacy VNFM features.
* Remove the following components used only by Legacy features.
* ceilometer
* fenix
* blazar
* Drop the functional test jobs for Legacy features.
* Delete the unnecessary files that were used in Legacy tests.
Implements: blueprint deprecate-legacy-apis
Change-Id: I76ba79f42cf7c3f04c48a87de4ae893f2d53e467
This patch adds a workaround for the image volume cache issue[1]
in Cinder to the local.conf sample.
The above workaround has already been added to the Zuul environment.
[1] https://bugs.launchpad.net/cinder/+bug/1953704
Change-Id: I46f2ad99c2d524c60402af48a038434a547f684a
This patch provides terraform infra-driver with several unit and
functional tests and is build an environment for terraform. The
supported version of terraform is v1.4.0 or later.
To build the terraform environment that need to install as follow
components:
- awscli
- docker
- localstack or moto server
- terraform
Implements: blueprint terraform-infra-driver
Change-Id: I14414c42229dcdb8e0083d7c51d6be6b5f2fc841
This patch adds a rollback option to create and update stacks.
If "CONF.v2_vnfm.enable_rollback_stack" is True,
when a resource creation fails in the create and update stack,
rollback stack is executed and the create resource is deleted.
This feature is implemented to successful AZ reselection at
the VDU using volume.
Closes-Bug: #2034886
Change-Id: Icdc70f299c65a137672935338dd6d795a3dbea73
This patch provides to the encryption and decryption methods for
authentication credentials in Tacker by using an external key
manager service such as OpenStack Key Manager (barbican).
Impletemts: blueprint support-external-keymanager
Change-Id: I130fd57dd8a2bad8e3e93dca5cdb476fdbe68608
This patch updates the k8s version to 1.26.8 and helm version to 3.11.3.
Implements: blueprint update-k8s-helm-prometheus
Change-Id: Iab80295e4e0c22d8fd5e3350d15fd451893a60ba
This patch fixes broken gate jobs by the following two changes:
1: Set GLOBAL_VENV=false
Set GLOBAL_VENV=false in job definition to fix gate jobs broken by
devstack change [0].
This fix is a temporary workaround in reference to the patch of
neutron-tempest-plugin [1] and will be reverted after the issue
is resolved.
2: Define constraints file for docs, api-ref and renos
Right now we are not using any constraints for docs, api-ref and
releasenotes builds. This has resulted in docs job failures once
Sphinx 7.2.0 has been released.
The patch will ensure that constraints are used an we should not
face similar issue again.
TOX_CONSTRAINTS_FILE is updated by Release bot once new branch is
created, so it should always track relevant constraints.
[0] https://review.opendev.org/c/openstack/devstack/+/558930
[1] https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/891550
Related-bug: #2031415
Change-Id: I5dbf931c15215f39dec03f594c21ee225ebe145f
This patch extends the functional test items related to individual
vnfc mgmt (StandardUserData) to make the test coverage of this
function more comprehensive. The following seven patterns are added.
* Test basic LCMs Max pattern
* Test basic LCMs Min pattern
* Test retry/rollback scale out
* Test rollback instantiate
* Test rollback change external VNF connectivity and update VNF
* Test rollback change current VNF Package
* Test various LCM operations before/after
Implements: blueprint individual-vnfc-management
Change-Id: I0da080418a1f1de803b950a9d0c8676e833a1f3e
For fine-grained access control based on user and VNF information
for API resources, this patch does the following things:
1.Add three comparison attributes of area, vendor, and tenant
for the enhanced Tacker policy.
2.Convert special roles to API attributes in context.
3.Modify the API process to support Tacker policy authorize.
4.Add the Tacker policy filter to the list API processes.
Implements: blueprint enhance-api-policy
Change-Id: I5b4c39387860133a3bcf4544f18a6353c80773f6
This patch is a part of Legacy APIs deprecation.
A Mistral workflow is created when NS is created, however, Tacker
doesn't have actual action scripts. Therefore, even though a Mistral
workflow is created, it never runs successfully due to the absence of
specified scripts.
This patch removes the dependency on Mistral for the legacy NS
create/delete APIs and also remove Mistral from Tacker entirely, as
there's no longer any place to use Mistral.
Implements: blueprint deprecate-legacy-apis
Change-Id: Iee5d109e4fdb9546730164aea94985b1c535bbc6
Provide the option to verify the SSL certificate when accessing an
external server from Tacker. Several parameters have been added to
config to allow verification of SSL certificates when accessing
external NFVO servers, heat servers, and notification endpoints from
Tacker.
Implements: blueprint enhance-http-client
Change-Id: I55b2b53cfe0dc794040d0e46ac13a20524b1d9f0
Added two new interfaces to Prometheus Plugin. Tacker as a VNFM can
support AutoHeal and AutoScale operations for VNFs and CNFs through
External Monitoring Tools, without NFVO.
Implements: blueprint support-auto-lcm
Change-Id: Ib0b5fd9264d80b9666ce69190e0ee41bbde23fac
Provide the option to use mutual TLS client authentication when
accessing external servers from Tacker. Oauth2MtlsAuthHandle has been
added to support Mutual-TLS client authentication for access from Tacker
to external NFVO servers and notification endpoints using user-provided
Mutual-TLS client certificates.
Implements: blueprint support-oauth2-mtls
Change-Id: Ib1b33bccac85ba8c68aeebd460876bb38a4917fa
This patch enables availability zone reselected by stack retry
when "instantiate" "scale" "heal" operations in v2 API fails
due to availability zone is unavailable.
Note that precondition of using these functions is to use
StandardUserData as UserData class.
Implements: blueprint enhance-placement
Change-Id: Icc9eb7a8fffbf35d2e005a9703dcefe66f097584
This patch updates the k8s version to 1.25.6 and helm version to 3.10.3.
Implements: blueprint update-k8s-helm-prometheus
Change-Id: I1ce138b5579d66e000c4f1978db4e3916f706e7c
As per 2023.1 testing runtime[1], we need to test on Ubuntu
Jammy and at least run a single job on Ubuntu Focal(for smooth
upgrade).
This patch adds a new sol-v2 tacker job to run on Focal which
can be removed in the future cycle when testing runtime drops the
requirement of Focal testing.
[1] https://governance.openstack.org/tc/reference/runtimes/2023.1.html
Change-Id: I96a611dabc947a876c978a55208df48b59ec6421
This patch changes client_socket_timeout config value of neutron
to 60(sec) from default 900(sec).
CI fails sometimes due to Heat timeout. The cause of timeout is
that create server API to nova takes too long time (900 sec).
It seems the communication between nova and neutron takes too
long time (900 sec) from log files.
This patch mitigates its value (900) to 60 for avoiding Heat
timeout. Note that the root cause of communication problem
between nova and neutron is not revealed.
Change-Id: I2b278bff9d138f781072d5d6f7aefc353309aa1e
This patch makes to stable Zuul CI by switching the Legacy FT
to non-voting because the Legacy API, excluding VIM functionality,
will be deprecated in Antelope release and will not be updated
in the future.
Implements: blueprint deprecate-legacy-apis
Change-Id: Ib28d1f7e01e977b17877efa5df9d5258b10c8e21
This is improvement of functional test of
prometheus plugin. By this fix, the created
prometheus rule files are checked by promtool.
If the syntax of them are invalid, the test fails.
This check is disabled by default and enabled when
CONF.prometheus_plugin.test_rule_with_promtool
is true.
Implements: blueprint support-auto-lcm
Change-Id: Ica331befe225156c607d5c8267462b7281669c91
As per the 2023.1 cycle testing runtime and community goal [1],
we need to migrate upstream CI/CD to Ubuntu Jammy(22.04).
Additionally, this patch adds new Jammy nodesets for
tacker gate jobs.
Note: Temporary disable "Ceilometer" as its failing tacker
gate jobs.
[1] https://governance.openstack.org/tc/goals/selected/migrate-ci-jobs-to-ubuntu-jammy.html
Change-Id: Icb79f357dafe852888ccbf22da6702b5f2ad428b
This patch updates the k8s version to 1.25.2 and helm version to 3.10.1.
The following changes have been made in kubernetes 1.24, so this patch
supports them.
- Dockershim Removed from kubelet
-> Changed from Dockershim to CRIO
- Service account tokens are no longer auto-generated
for every ServiceAccount
-> Changed secrets to be created manually
Also fixes the following API versions that have already been removed
in kubernetes 1.25.
- autoscaling/v2beta1
- extensions/v1beta1
Implements: blueprint update-k8s-helm-prometheus
Change-Id: Ic9c2f66251c5d11a652184be5908d91e0ee3fb7d
This patch reverts the changes made by the [1] patch with some jobs set
to non-voting for the Z release RC.
[1] https://review.opendev.org/c/openstack/tacker/+/857551
Signed-off-by: Ayumu Ueha <ueha.ayumu@fujitsu.com>
Change-Id: If7893e5e2d4df868f844b9c769e9adc86a619e4f
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for antelope. Also,
updating the template name to generic one.
See also the PTI in governance [1].
[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html
Change-Id: I6c6fd22e6e4b85cd9005d7ea5d4187a885e0b3d9
Support AutoHealing using FaultNotification interface between
Tacker and VIM.
Provide sample mgmt driver script to register/unregister
FaultNotification interface.
Implements: blueprint support-autoheal-queue
Change-Id: I9ec727c24ba053e928ccebb99dd49ebcaa64bcbb
Support container based VNF AutoHeal and AutoScale operation with
External Monitoring Tools.
Add the Fault Management interfaces and CLI to support AutoHeal.
Add the Performance Management interfaces and CLI to support
AutoScale. The Fault Management and Performance Management
interfaces are based on ETSI NFV-SOL 002 v3.3.1 and ETSI NFV-SOL
003 v3.3.1, which are Version "2.0.0" API of Tacker. Add the
Prometheus Plugin that has a interface between tacker and Prometheus
as a sample of External Monitoring Tool.
Implements: blueprint support-auto-lcm
Change-Id: Ib74305f0b1da4eb8d996ebae400e75902aaa1321
This patch fixes an issue that triggered the sol-kubernetes related job
when the file under ``api-ref`` folder is updated.
Signed-off-by: Ayumu Ueha <ueha.ayumu@fujitsu.com>
Change-Id: I1b360eff07bd2677dfc2003a7105187cd1d8a721
This patch changes the followings to stabilize sol-kubernetes job.
* Set ``service.type=NodePort`` in ``helmparameter`` of input parameter
to reduce the time it takes to complete the deletion of resources.
* Modify ``vnf_instance_name`` and parameters of
``using_helm_install_param`` to unique values to avoid duplicate
entry.
* Change definitions for each test class run to avoid duplicate
resource names.
And this patch make the following tests non-voting as a temporary fix.
* tacker-functional-devstack-kubernetes-oidc-auth
* tacker-functional-devstack-multinode-legacy
* tacker-functional-devstack-multinode-libs-master
Signed-off-by: Ayumu Ueha <ueha.ayumu@fujitsu.com>
Change-Id: I33f4bbab43b6338ace4b4a9f96c803d6d8db150c
This patch adds openid token auth support when calling k8s APIs.
Openid token auth of k8s relies on an external openid provider,
and Keycloak acts as the openid provider in this implementation.
Implements: blueprint support-openid-k8s-vim
Change-Id: Ie5e080a20cba3ba0ed514ede7955eb16729d797c
When initializing k8s client in InfraDriverV2, the SSL CA
certificate is set incorrectly.
To fix the issue, the following modifies are made in this patch:
* A temp file for ssl_ca_cert is created before initializing
k8s client and the temp file path is set to k8s_config.ssl_ca_cert,
* The temp file is deleted until the lifetime of k8s client ends.
Note: This references the implementation in InfraDriverV1.
If set the ssl_ca_cert in instantiate request, the validation of
request is failed because of the length of ssl_ca_cert exceeds 1024.
For this issue, add a new type `keyvalue_pairs_no_length_limit`
which has no max length limitation to verify the request.
And the interfaceInfo, accessInfo, extra are all set to the new type
for unity.
In Zuul test environment, when registering default vim, ssl_ca_cert
is not set. So the case with ssl_ca_cert is not tested.
In this patch ssl_ca_cert is set into the default vim.
Closes-Bug: #1979413
Change-Id: I61dbd70690b737a72fc619e5a08b4bab51160a27
Since the version of jsonschema has been updated from 3.2.0 to 4.7.2,
when tox executes UT, it reports an error when checking IPv4, and now
the invalid IP address is fixed.
Due to FT failure caused by ceilometer, the installation of Ceilometer
was commented out in `.zuul.yaml`.
Note that this is only a temporary change and will need to be changed
back when the ceilometer issue is resolved.
Closes-Bug: #1983067
Change-Id: Ice9be4d71ee1689b57add263be5fadd77a7f5ccd
Only test cases using local NFVO are included in the FT of the
current v2 API.
This patch provides a new FT job for the v2 API to test separated
NFVO scenarios.
1. The separated NFVO in this patch is to be simulated using code.
2. This patch will merge the fake_grant_server.py of v1 as a common
fake_grant_server for users to test manually.
Change-Id: Ic67295d425b19fc6de6f5c49dc02478068c17224
This patch fixes the issue that k8s failed to initialize object due
to version upgrade. Skip initial validation by setting the
`client_side_validation` parameter and remove the must_param
variable that was previously set in order to pass the validation.
In kubernetes v23.3.0, 'available_replicas' must be set into status
of `StatefulSet`, so this patch also add a kuryr-kubernetes versioned
parameter temporarily to .zuul.yaml to make the response returned
by kubernetes correct.
Closes-Bug: #1968103
Change-Id: I9495ce0f0893e5f9a1d6c52b98c3db3928bd95a3
