The flake8 version was very old and has been updated
The py27 and py36 targets are removed
The pylint file has been cleaned up for python3
Zuul nodesets for this component are now debian
Minor fixes to cleanup simple flake8 and pylint warnings.
The use of 'file' and 'stat' as variables, which mask the
builtin or top-level imports have been cleaned up.
Main change was to replace yaml.load with yaml.safe_load
This is because yaml.safe_load exists in older and newer
versions of pyyaml, whereas yaml.load signature changes
between versions, and not all Loader types exist in both.
Test Plan:
PASS: tox
Story: 2010642
Task: 48156
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Ibdac877f40dc32140121724f19751aff6052e9db
Updating the rsa ssh host key based on:
https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/
Note: In the future, StarlingX should have a zuul job and
secret setup for all repos so we do not need to do this
for every repo.
Needed to rename the secret, because zuul fails if like-named
secrets have diffent values in different branches of the same
repo.
Partial-Bug: #2015246
Change-Id: Ic4c29da7646fea621002d5c152b8e4dce5a35c68
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
The dependencies were failing to install mostly because
this repo still has python2 for its tox and zuul.
tox.ini has been updated, and zuul now points to
a debian nodeset.
Some of the components are newer, and therefore additional
error checks are being suppressed for flake8 and pylint.
Closes-Bug: #2016171
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I0632ab4d4292a623a541109d4f53953113d59877
The python2.7 jobs will no longer be executed as part
of the zuul check and gate.
The nodeset with the python2.7 interpreter has been
updated around Feb 5th 2023.
tsconfig flake8 was running python2.7 and has been updated
Story: 2010531
Task: 47312
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I7bd3ec4fb3019356a344196f1a20a8ecdea0e018
sysinv will now use python3.9 for pylint, flake8, bandit
basepython is now set to python3.9
The zuul jobs for older versions are now updated or removed
The nodeset for python 3.9 jobs is set to debian-bullseye
pylint.rc changes include:
- Upversion to latest pylint (2.16)
- remove suppressions that no longer exist.
- add suppressions for new checks added in recent pylint
- update pylint.rc settings that no longer exist.
- remove explicit enabling of pylint checkers from python2
that are on by default in python3
Note: some of the newly suppressed pylint error codes will
be un-suppressed in later commits.
A subtle issue has also been fixed where a variable was
shadowing an imported method, and would have led to a
an exception when logging an error. The variable has been
renamed.
Example CLI service-parameter command for that code path:
system service-parameter-modify kubernetes \
kube_apiserver_volumes \
default-audit-policy-file='{"hostPath":"x", "name":"y"}'
The CLI of the previous code would show the following
in its exception case without the fix
'int' object is not callable
Test Plan:
PASS: run tox in a debian build environment.
PASS: build/install/unlock AIO-SX
PASS: service-parameter CLI command still works
Story: 2010531
Task: 47279
Change-Id: I989ec6e75463be0a4dcd2c7727cd69584c2a20a2
Signed-off-by: Al Bailey <al.bailey@windriver.com>
This change will allow this repo to pass zuul now
that this has merged:
https://review.opendev.org/c/zuul/zuul-jobs/+/866943
Tox 4 deprecated whitelist_externals.
Replace whitelist_externals with allowlist_externals
skipsdist and usedevelop are incompatible in tox 4.
Setting tox < 4 in .zuul.yaml rather than trying to
figure out why the sysinv component is not properly
installing in tox.
Partial-Bug: #1997255
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: If8e339d20e57e0830ea4ce27c098bacfef2f2ef8
sysinv tox could not be run in a developer environment
due to a relative path that only existed in a zuul env.
the py39 zuul jobs do not need root repo cloned since
the upper constraints are accessed by URL.
The py files are only being updated in their header
so that the tox jobs will be triggered to be executed
due to zuul filtering.
Story: 2006796
Task: 44861
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I184dc910717239fd343a70c343361a4c6b8fb9e3
Removing redundant py36 Zuul jobs since we now have py39 Zuul jobs in
place with the debian nodeset
Renaming tsconfig gates name
Story: 2006796
Task: 43481
Signed-off-by: Bernardo Decco <bernardo.deccodesiqueira@windriver.com>
Change-Id: I3ab977d6691ecbaf295469878e7830d07ee744b7
Add support for Debian bullseye which runs python3.9.
This was tested by running tox locally on a Debian bullseye
VM.
Story: 2006796
Task: 42931,42925,43099
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I2b18437db2249ecd7b358c6d1e49023afa3da21a
Add python 3.9 gate for sysinv. Make the gate non-voting for now
since the unit tests do not pass for now. A seperate commit is coming
to fix the unit tests.
Update the bindep.txt to install missing packages in the ubuntu-focal
image.
Also update the requirements to fix dependencies for python3.6 and
install dependencies that are python2.7 only.
Story: 2006796
Task: 42975
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I2512e276c8517132f952cdec0affa8cea8565035
Added support for python 3.9 in the zuul gate. Tested
by running tox locally.
Story: 2006796
Task: 42951
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I571ff28fc631215b66b4a88649e30e3d3b7368ac
Add gates for py36, py27, flake8, and pylint.
Story: 2006796
Task: 42774
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: Icca03a8b8c46c39a88f5aa76acea891ea87a34e7
The py2.7 jobs need to specify xenial
The py3.6 jobs need to specify bionic
The focal zuul nodes only have python 3.8 installed in them
The copyright date was updated for some files in order to trigger
the zuul jobs, as a no-delta type of change.
Partial-Bug: 1928978
Signed-off-by: albailey <Al.Bailey@windriver.com>
Change-Id: Ifc7904d4908a5dbe2ffbd9214e5e4c425932afad
pylint running in python3.8 will complain about some
code such as 'import contextlib'
Rather than destabilize the code, specify python3.6 for
the pylint job.
When python2 is fully dropped, the changes can be addressed.
The zuul nodeset for bionic sets up a venv for python3.6
Closes-Bug: #1928841
Signed-off-by: albailey <Al.Bailey@windriver.com>
Change-Id: Ib2e7f467ce20e5b31aef7242405ef034583d4e1a
The following changes have been introduced as a fix for this issue:
1. Changed the sorting on the migration script file names to be based
on the first number on the file name.
2. Added file name format validation: "nnn-*.*", where "nnn" string
shall contain only digits.
3. Fixed the name of two migration scripts that were not following the
correct format (not using "-" separator).
4. Added set of unit tests to test and validate the execution of
migration scripts code.
Manual upgrade testing to STX 5.0 has been executed.
Closes-Bug: 1887985
Signed-off-by: Adriano Oliveira <adriano.oliveira@windriver.com>
Change-Id: I04fdb8a3b3e177c609c4037825810a531954d99c
This commit updates to clean up the obsolete code in
controllerconfig as a result of switch to Ansible.
The cleanup includes:
- the whole config_controller with its relevant files
(ie..initial configuration, backup/restore, image clone...)
- config_subcloud
- config_region
Tests conducted:
- fresh installation of AIO-SX, AIO-DX, standard
and storage labs
- platform sanity performed on AIO-SX, AIO-DX,
standard and storage labs
- stx-openstack app apply/remove performed on AIO-SX
and AIO-DX
- stx-monitor app apply/remove on standard and
storage labs
- backup&restore performed on standard lab
- distributed cloud was deployed
Change-Id: I0d1eb91188dfb24dca68a19a497b9e4edb0b0ef0
Partial-Bug: 1834218
Depends-On: https://review.opendev.org/#/c/703517/
Signed-off-by: Angie Wang <angie.wang@windriver.com>
Deprecate cgtsclient.openstack.common files since
those files were mostly replaced by oslo.
Fix a wrapping_formatters import which was using a relative path.
A recursion error with deepcopy has been fixed which allows
the py36 unit tests to be enabled.
Newer flake8, hacking and bugbear are now enabled since python3
is able to be tested now.
Change-Id: I4978ab33b09e13b933b7f0fc9da22ed97e671938
Story: 2007082
Task: 38314
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
Running flake8 in python3 exposes bugs that will
impact the conversion to python3.
This also adjusts the unit tests to run py36
rather than py35.
Some of the notable issues are:
- cmp operator has been deprecated
- file operator has been deprecated
(replaced by call to open)
The behaviour in python2 has not changed.
For python3 unit tests should be added for:
- sysinv/agent/lldp/plugin.py (Key)
sort in python3 expects a different method (__lt__)
- sysinv/conductor/manager.py (evaluate_app_reapply)
The contents from file open when passed to the md5 routine
should fail unless opened with utf-8 encoding
ie:
TypeError: Unicode-objects must be encoded before hashing
Change-Id: Ifcec7d5539cba2350649f56057db0f5e78ef279d
Story: 2007082
Task: 38130
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
Since OBS is not working correctly for CentOS and we are stopping openSUSE
for now, no point it having this template run those OBS jobs.
Change-Id: I75310fbeff26bd4d6f022750e3921da3ec5c82cf
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Adding a zuul job so that any new bandit failures
will be caught.
Switching bandit to run in python3.
B111 plugin was removed from bandit by:
PyCQA/bandit@d93eed5
Cleanup the list of bandit checks that need to be skipped
in order for this tox job to pass. Those checks can be
removed when the code is updated by future commits.
Story: 2004515
Task: 37744
Change-Id: I1809b1124a5a693d288fb7ea893f30c24487af36
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
This will build the openSUSE rpms with a non-voting Zuul
job during the check stage
Change-Id: I3ba34011f1204dd6ba562feffcc76f24b91ac30d
Story: 2006673
Task: 37195
Signed-off-by: Saul Wold <sgw@linux.intel.com>
The repo re-org has removed 'stx-' from the name of the repos
that are checked out from the manifest, so there is no
need for special prefix handling in tox to accomodate
differences with checking out using "repo" or "git clone".
With the addition of tsconfig to this config repo, it will
generate an intermediate .stestr folder if tox is run
in that subdir. This has been added to the top level
.gitignore file.
The required-projects for the zuul jobs have been
cleaned up and references to STX-PREFIX are removed.
Change-Id: I8454305867445ad8b0feb9f46d44c7a570e9142a
Story: 2006166
Task: 36490
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
This change updates the dependency paths referenced
by tox.ini so that zuul jobs in sysinv can pass.
The STX prefix is no longer needed by tox and zuul
now that the manifest and repo names are the same,
which was fixed as part of the package re-org.
integ is no longer a required project for zuul.
The packages from integ are in utilities instead.
Change-Id: I0ff834e51667e21cbd75262a2c4178fb837f93ad
Story: 2006166
Task: 36488
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
Use templates instead of individual jobs so that these
can be changed in one place.
Depends-On: https://review.opendev.org/677606
Change-Id: I60b748616fc610e3521a26d37e8a09c4f27e2712
It looks like devstack zuul is failing when it sets up glance.
This review recently merged which does not seem to work with our
devstack configuration.
https://review.opendev.org/#/c/665606/
Removing this devstack job from zuul. It can be re-enabled if
the compatability issues between our environment and glance
are resolved.
Change-Id: I0da7fbbef6a6ab65b5146435c6e01fff347c53d1
Closes-Bug: 1840292
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
Zuul checks out the dependant projects by their repo names.
Repo checks out the project directory structure based on the
labels in the manifest.
Currently these directories have different names and so tox
passes when run by zuul, but fails when run in a developer env.
This submission uses an env variable: "STX_PREFIX" to make
both envs able to run tox.
Story: 2004515
Task: 30664
Change-Id: I06cefab7422f53ccc0b8af30ca06945311cec70e
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
Remove configutilities and move what is being used in other components
to controllerconfig.
Tested with a clean install on AIO-DX and running config_controller.
With the StarlingX move to supporting pure upstream OpenStack, the
majority of the SDK Modules are related to functionality no longer
supported. The remaining SDK Modules will be moved to StarlingX
documentation.
Story: 2005275
Task: 30262
Change-Id: Ie496548dfc6efee677a501c98c227c586df0a7d6
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
* Rename stx-devstack-config to flock-devstack-config
and inherit from flock-devstack-base
* Make devstack jobs voting
Story: 2005285
Change-Id: Ic8970f6c747a8b1ef3796db3810f4e81c67f0d32
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
The upstream OpenStack default node type changed recently
from Ubuntu Xenial to Bionic. THere is additional work to
cleanly run these test on Bionic (including bumping to py36
or py37) so just go back to Xenial for now.
Added pycrypto to trigger the jobs to run since we'll need it
for the bionic bits anyway.
FWIW https://review.openstack.org/#/c/643466/ is the attempt at
switching to py37 on bionic.
Change-Id: Ib09a07fa01388b5ff984720b6290d47a01a3e6fb
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
Allow cgtsclient to be installed without any of the other
sysinv services or prerequisites. Also allow some selection
within sysinv services.
Change-Id: Ie8f10cb11111e9545103df001976295fc4aba3a6
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
Use the DevStack-provided functions to do the Python installations
for configutilities and controllerconfig.
Prepare the plugin setting for declaring DevStack prereqs that
is available in master's DevStack playbook.
Also do not enable all services by default. sysinv-api is disabled
in the devstack job as it does not properly start under Bionic. We
will address this separately.
Change-Id: Ib57863526d285049b5964828e1b60bf215d25a23
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
The pylint checks that fail are being suppressed.
Those checks will be fixed and un-suppressed in later commits.
Story: 2004515
Task: 29298
Change-Id: Iacc18da64ed5e2d82ff2f3e64368c9b4a94bb1d5
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
- Replace mox with mox3 to support python3
- Change 'lambda (x, y): y - x' to 'lambda xy: xy[1] - xy[0]'
- Change iter to list for str operator
- Change urlparse to compatible python 2/3
- Adapt gettextutils
- Fix 'TypeError: unorderable types: NoneType() < int()' for python3
- Change dict key to list
- Change some requirement and test requirement
remove MySQL-python
remove Cheetah
remove version limitation <3.0.0 for python-ldap
- Fix encode/decode issue
- Avoid exception for obj_load_attr with optional fields
- Remove test_hash_file test case since it was never used.
- Use mock not CreateMockAnything to mock contentmanager
- Fix UnboundLocalError issue for python3
- Replace self.stubs.Set with stub_out
- Change e.message to str(e) in exception block
- Add self.mox.UnsetStubs() before self.mox.VerifyAll()
- Fix set() order dismatch for python 2/3
- Implement nested for context manager in python 3
- Change migration version of sqlalchemy from 0 in python 3
- Avoid expose exception for db not connected
- change hasattr(obj,attr) to getattr(obj,attr,None) since
hasattr shadows all exception in python2.7
- Avoid sqlalchemy.orm.exc.DetachedInstanceError in get_networks
Story: 2003433
Task: 28354
Depends-on: I4c601a72232402e45fe70e0d29de031ff294a4d7
Change-Id: I8382eba1bc3c91ca63d93e759021149914b12865
Signed-off-by: Sun Austin <austin.sun@intel.com>
This update adds the tox and zuul configuration to
run puppet-lint against the puppet manifests. The
initial update ignores all existing errors, which
will be cleaned up later.
Change-Id: I293abc2eac6bc6216cbbf6d939c1ba3474fb9384
Story: 2004515
Task: 28665
Signed-off-by: Don Penney <don.penney@windriver.com>
Previously when pylint was invoked, all warnings were suppressed.
Now, the only warnings suppressed are the ones that are failing.
By making this change, fixed issues will not be reintroduced
by later commits.
Additional tasks will be created in the future to address the
currently suppressed warnings and errors.
This also adds a zuul job for pylint in sysinv.
Story: 2004515
Task: 28268
Change-Id: Ie162c08a9e70a70bfc089ebee711ef9d6a374f2b
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
The zuul config file was not triggering correctly.
cgts-client changes should run jobs for cgts-client
sysinv changes should run jobs for sysinv
pep8 / flake8 jobs do not need dependent projects
This change also fixes the failing pep8 and pylint errors
The change in sysinv/tox.ini was to verify the zuul jobs for
sysinv are triggered by a change under that folder.
Story: 2003862
Task: 27870
Change-Id: Ia5887ef74fc0628e0dcf55f1f27cf0dc9e5ccbc7
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
need add sysinv-agent to ENABLED_SERVICES list
in devstack/localrc to enable this service.
Story: 2004370
Task: 27976
Change-Id: Id5a18a761bf288abe0ee145d116d09c9443848d3
Signed-off-by: Austin <austin.sun@intel.com>
* Add pointers in the main doc to api-ref and releasenotes pages
* Add publish-stx-api-ref and publish-stx-releasenotes jobs
* Add search at bottom of api-ref and relnotes pages to trigger the jobs
Change-Id: I6286efb94cab686c978b9d3924bcd621250107f6
Signed-off-by: Dean Troyer <dtroyer@gmail.com>