Merging sysinv-fpga-agent service with sysinv-agent
in order to reduce overall OS overhead.
Replaced calls "wait_for_n3000_reset()" and "wait_for_host_uuid()" in
previous fpga-agent-manager by checks that ensure fpga devices are
reset and host_uuid is available in agent-manager. Also, the content of
"fpga_pci_update()" and "report_fpga_inventory()" methods is directly
inserted in the body of "agent_audit()" method.
Test Plan:
On AIO-DX env (CentOS):
<sysinv-fpga-agent tests>
PASS: Check FPGA pod and its resources.
PASS: Check FPGA pod and its resources after lock/unlock.
PASS: Check FPGA pod and its resources after the system reboot.
PASS: Verify image upload with non-functional image with
retimer-included
PASS: Verify retimer_a_version and retimer_b_version after applying
BMC image with re-timer and bmc
PASS: Verify firmware update for BMC and retimer image with
retimer-include=False
PASS: Verify apply BMC image without re-timer first and then BMC
image with re-timer, only latest image is kept in
device-image-state-list
PASS: Test accelerator configuration is persistent after lock/unlock.
PASS: Test to verify that the accelerator configuration is persistent
after a graceful reboot.
<sysinv-agent tests>
PASS: Verify alarms raised by PTP feature
PASS: Verify the configuration and run of single ptp-instance
PASS: Verify the configuration and run of single phc2sys
PASS: Verify PTP CLI commands
On AIO-SX env (Debian):
PASS: Check FPGA pod and its resources.
PASS: Check FPGA pod and its resources after lock/unlock.
PASS: Check FPGA pod and its resources after system reboot.
PASS: Check if FPGA device can be detected, configured.
PASS: Test accelerator configuration is persistent after lock/unlock.
PASS: Test to verify that the accelerator configuration is persistent
after graceful reboot.
Story: 2010087
Task: 45628
Signed-off-by: Davi Frossard <dbarrosf@windriver.com>
Change-Id: I83edd261898498344001ca90bb53a5f65e66728c
This commit does two different changes: it changes the policy engine to
oslo_policy and restrict access to sysinv API to users of projects
'admin' or 'services'.
The policy engine deprecated is the one present in the file
"sysinv/sysinv/sysinv/sysinv/openstack/common/policy.py" (780 lines).
This file is no longer used by this repository and was not deleted
because it is used by other repositories, like starlingx/update. The
library oslo_policy is used in its place. In fact, the deprecated engine
seems to be an ancient version of oslo_policy. The library oslo_policy
changed the default format of configuration files from JSON to YAML, so
the configuration files named "policy.json" were changed to
"policy.yaml". The file that initializes and wraps oslo_policy
("sysinv/sysinv/sysinv/sysinv/common/policy.py") contains the minimal
implementation to use this library.
The access to sysinv API, before this commit, was restricted to users
with role "admin" or "administrator" from any project. This commit
restricts the access to users with role "admin" of projects "admin" or
"services". This change should not cause problems, because role
"administrator" doesn't exist and because all users from Starlingx are
from projects "admin" or "services". This change is needed to avoid
access from admin users of other projects.
To test custom policy rules set in the file "/etc/sysinv/policy.yaml",
it will be used the Service Parameter API actions create/apply/modify/
delete/get (commands "system service-parameter-[add/apply/modify/delete/
list]". To test default policy for sysinv API commands, it will be used
the command to change the system description (PATCH "/v1/isystems",
command "system modify --description='test'"). On test plan, these
commands will be reffered as "test commands". Any change in the file
"/etc/sysinv/policy.yaml" is detected by policy engine and rules are
updated.
Test Plan:
PASS: Successfully deploy an AIO-SX using an Debian image with this
commit present. Successfully create, through openstack CLI, the users:
'testreader' with role 'reader' in project 'admin',
'adminsvc' with role 'admin' in project 'services' and
'otheradmin' with role 'admin' in project 'notadminproject'.
Create openrc files for all new users. Note: the other user that will be
used is the already existing 'admin' with role 'admin' in project
'admin'.
PASS: In the deployed AIO-SX, check the behavior of test commands
through different users: for "admin" and "adminsvc" users, all commands
are successful; for user "testreader", only "service-parameter-list"
command is successful and for user "otheradmin" no command is
successful.
PASS: In the deployed AIO-SX, add the following lines in file
"/etc/sysinv/policy.yaml":
config_api:service_parameter:add: role:reader
config_api:service_parameter:apply: role:reader
config_api:service_parameter:delete: role:reader
config_api:service_parameter:get: role:reader
config_api:service_parameter:modify: role:reader
and check the behavior of test commands through different users:
for "admin" and "adminsvc" users, all commands are successful; for users
"testreader" and "otheradmin", all commands are successful except the
change in the system description ("system modify --description='test'").
PASS: In the deployed AIO-SX, to assert that public API works without
authentication, execute the commands:
"curl -v http://<MGMT_IP>:6385/v1/" and
"curl -v http://<MGMT_IP>:6385/v1/isystems/mgmtvlan" and
verify that they are accepted and that the HTTP response is 200,
and execute the commands:
"curl -v http://<MGMT_IP>:6385/v1/isystems/" and
"curl -v http://<MGMT_IP>:6385/v1/service_parameter" and
verify that they are rejected and that the HTTP response is 401.
PASS: Repeat all tests above changing the deploy to AIO-DX using an
CentOS image.
PASS: Successfully execute Debian AIO-SX daily regression and sanity
tests using an image containing this change.
Story: 2010149
Task: 45984
Signed-off-by: Joao Victor Portal <Joao.VictorPortal@windriver.com>
Change-Id: Id7aa387e154afb1441a8484b076cdc97f2fc46cb
The Host Hardware Profiles for creating re-usable configuration
templates for hosts are no longer maintained or supported and should be
removed from code. Including : CPU, Interface, Storage and Memory
profiles.
profile categories:
* cpu
* memory
* storage
* interfaces
topics:
* remove objects
* update models
* update documentation
* remove import/export profile apis
* remove and update unit tests
* new version script for migration upgrade
Test Plan / Failure Path:
PASS: Verify profile feature is removed on system upgrade
without existing previous profiles.
PASS: Verify profile feature is removed on system upgrade
with existing previous profiles.
PASS: Verify profile feature is removed on fresh install.
Regression:
PASS: Verify that the Horizon GUI remains navigable.
PASS: Verify non-affected system commands remains listed.
Story: 2009163
Task: 43159
Signed-off-by: Pablo Bovina <pablo.bovina@windriver.com>
Depends-On: https://review.opendev.org/c/starlingx/config/+/806800
Change-Id: Id828365920ce179e347acf0de5d3ed6af09efcbd
This commit does several things at once:
- Drop argparse, its no longer required.
- Add missing oslo.log, its been pulled in by other requirements
but explicitly include it.
- Use rfc3986 instead of Django, the requirement seem to be
a bit heavy to me, to check for a valid URL. Added unittest.
- Move run time requirements out of test-requirements.txt
that are actually run time requirements for sysinv.
- Also remove python2-mox from the centos7 spec file since
its no longer required.
- Updated debian/control and opensuse rpm as well.
tests:
- Did basic build test.
- Ran smoke test with resulting RPMS and iso.
Story: 2006796
Task: 42072
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I591ed52937092233c3ac7c2c2bf847b2b0c3f690
Deprecate sysinv.openstack.common.rootwrap in favor or oslo_rootwrap.
This was done so we maintain less code and we worry about Python3
a little less.
Story: 2006796
Task: 41868
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I979ff1c8045030cfcaf8b88678121c4d0d684743
This creates a new sysinv FPGA agent. On startup it will perform
an initial scan for supported FPGA devices and report the current
hardware status to sysinv-conductor via RPC.
It also provides basic support for flashing the specified device
images to the FPGA device using Intel-supplied tools running
in a Docker container.
Initially only the Intel N3000 FPGA is supported.
Story: 2006740
Task: 39927
Change-Id: Id8a6510a2d8cd072737a98c5d909f94dbf10a763
Depends-On: I63cfa7698285a1a43f1e9e4b98e9a536fc3dc682
The sysinv code has imports for python components but does
not have all of those specified as rpm dependencies.
Adding:
python-jsonpatch (used by several sysinv/api/controllers/v1/*)
python-keystoneauth1 (used by sysinv/common/fm)
python-keystonemiddleware (used by sysinv/api/acl)
python-oslo-serialization (used by sysinv/conductor/manager)
python-oslo-service (used by sysinv/common/wsgi_service)
python-paste (used by api config file)
python-psutil (used by sysinv/api/controllers/v1/host)
python-requests (used by sysinv/common/ceph)
python-retrying (used by sysinv/agent/manager)
python-stevedore (used by sysinv/puppet)
python-webob (used by sysinv/api/hooks)
This is a protective commit, since this only works because other
packages are pulling in the dependencies, and those packages can be
updated or removed in the future, thus breaking sysinv.
Change-Id: I9457e3817ae8d219ef0c582b0b0177510a8ec4b0
Story: 2004515
Task: 37985
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
Moving incorrectly located stand-alone scripts from the
sysinv/cmd directory to the scripts directory. These scripts
appeared to be using eventlet due to the __init__.py file in the
sysinv/cmd directory, but were not, as they are moved to the
/bin directory at installation time.
This is being done in preparation for upversioning eventlet.
Change-Id: I511d61e14b3eb1511cee03aa0e36ffd1277e79ab
Story: 2006781
Task: 37583
Signed-off-by: Bart Wensley <barton.wensley@windriver.com>
This commits creates a new utility command /usr/bin/sysinv-utils
that can be used by ansible playbooks to generate the host overrides
file that will be used for upgrades and backup/restore.
This commits also enhanced the following CLI commands to include a
--column and --format options for better CLI outputs data manipulation.
host-show
host-list
service-parameter-list
application-show
These two new options are optional. The default behavior is not changed.
Story: 2006590
Task: 36892
Change-Id: I36b8ea411a4eb74d6e1a2194982025905e9299e5
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
The gettextutils code dates back to when openstack projects
maintained common code and was later replaced by oslo_utils.
The new convention is to create a new _i18n.py file at the base
level and import it.
See: https://docs.openstack.org/oslo.i18n/latest
Most of the changes in this review are:
Old Code:
from sysinv.openstack.common.gettextutils import _
New Code:
from sysinv._i18n import _
This explicitly adds oslo_i18n to requirements.txt
Change-Id: I606ef9fef1e0431340669e9a2f153a038e72b88a
Story: 2006796
Task: 37340
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
These runtime dependencies are added since they are missing and
caused the execution of the scripts to fail.
Story: 2006723
Task: 37138
Change-Id: Ib34fae832b13394d3b2f041b27a7d02992ddc168
Signed-off-by: Saul Wold <sgw@linux.intel.com>
These files will be extracted by the _service file that is managed
by OBS.
These do not affect the CentOS build
Story: 2006723
Task: 37133
Change-Id: Iec8329ca2c7d6442cd41436c291eae79326052fb
Signed-off-by: Saul Wold <sgw@linux.intel.com>
This commit updates to support pulling application images from
aws ecr at application-apply time.
In commit https://review.opendev.org/686057, all system images
are pushed to local registry at Ansible bootstrap time. This
commit also updates armada container to use armada image from
local registry.
Allow to add "elastic-registry" in service parameter to be able
to deploy stx-monitor from private registries.
Tests conducted:
- deploy platform-app-integ from public/aws ecr registry
- deploy stx-monitor from public/aws ecr registry
- tiller image upgrade/downgrade test
Change-Id: I7a597a2bfdacf969462f166e8f416e570f243076
Story: 2006274
Task: 36874
Depends-On: https://review.opendev.org/686057
Signed-off-by: Angie Wang <angie.wang@windriver.com>
These will be used with the Open Build Service (OBS) an _service
will be used to automagically generate the tarball.
OBS is a generic system to build and distribute binary packages from
sources [0], StarlingX OBS Project:
- Cloud:starlingx:2.0 [1]
[0] openbuildservice.org
[1] https://build.opensuse.org/project/show/Cloud:StarlingX:2.0
Story: 2006508
Task: 36650
Task: 36651
Task: 36652
Task: 36653
Change-Id: Ie59e8403a6f3173a678f5fd1cd5cf03df4a7e961
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Davi Frossard