This commit implements the access control for all FM APIs. An incomplete
list of FM APIs can be found at
"https://docs.starlingx.io/api-ref/fault/api-ref-fm-v1-fault.html". Unit
tests will be created in other task.
All access control rules can be overwritten through file
"/etc/fm/policy.yaml". Any change in file "/etc/fm/policy.yaml" is
automatically detected by policy engine and the rules are updated.
Differently from other APIs, which have as default rule to enforce that
all users using the API are present in either project "admin" or
"services", all read-only actions (GET requests) of FM API are allowed
for any user, so it only requires "reader" role (that is the lowest
role). Other actions require the user to have "admin" role and to be
present in either project "admin" or "services".
As all system users of StarlingX have "admin" role and are present in
either project "admin" or "services", the default rules for FM API
allows any system users to execute any action, so there should be no
regression with the change introduced here.
To test the access control of FM API, the following commands will be
used:
fm alarm-list
fm alarm-show <uuid>
fm alarm-summary
fm alarm-delete <uuid>
fm event-list
fm event-show <uuid>
fm event-suppress --alarm_id <alarm_id>
fm event-suppress-list
fm event-unsuppress --alarm_id <alarm_id>
fm event-unsuppress-all
On test plan, these commands will be reffered as "test commands".
Note: there is one FM API that is not tested by the commands above,
that is the creation of alarms ("fm_api:alarm:create"). This API will
be tested indirectly by observing the system successfully creating
alarms in the deployed environment.
Test Plan:
PASS: Successfully deploy an AIO-SX using an Debian image with this
commit present. Successfully create, through openstack CLI, the users:
'testreader' with role 'reader' in project 'admin',
'adminsvc' with role 'admin' in project 'services' and
'otheradmin' with role 'admin' in project 'notadminproject'.
Create openrc files for all new users. Note: the other user that will be
used is the already existing 'admin' with role 'admin' in project
'admin'.
PASS: In the deployed AIO-SX, check the behavior of test commands
through different users: for "admin" and "adminsvc" users, all commands
are successful; for users "testreader" and "otheradmin", only the
commands "alarm-delete", "event-suppress", "event-unsuppress" and
"event-unsuppress-all" fail. Observe also that the system is able to
create alarms during its operation.
PASS: In the deployed AIO-SX, add the following lines in file
"/etc/fm/policy.yaml":
fm_api:alarm:create: role:admin
fm_api:alarm:delete: role:admin
fm_api:alarm:get: role:admin
fm_api:alarm:modify: role:admin
fm_api:event_log:get: role:admin
fm_api:event_suppression:get: role:admin
fm_api:event_suppression:modify: role:admin
and check that all test commands are successful through user
"otheradmin" and that all test commands fail through user "testreader".
Observe also that the system is able to create alarms during its
operation.
PASS: In the deployed AIO-SX, to assert that public API works without
authentication, execute the commands:
"curl -v http://<MGMT_IP>:18002/" and
"curl -v http://<MGMT_IP>:18002/v1/" and
verify that they are accepted and that the HTTP response is 200,
and execute the commands:
"curl -v http://<MGMT_IP>:18002/v1/alarms" and
"curl -v http://<MGMT_IP>:18002/v1/event_log" and
verify that they are rejected and that the HTTP response is 401.
PASS: In the deployed AIO-SX, check through Horizon interface that Fault
Management works correctly (showing alarms and events, allowing events
to be suppressed).
PASS: Repeat all tests above changing the deploy to AIO-DX using an
CentOS image.
Story: 2010149
Task: 46123
Signed-off-by: Joao Victor Portal <Joao.VictorPortal@windriver.com>
Change-Id: I3db6d0464d8d53c4dfbc761663be1712141b8b93
Here is added a hook to the WSGI compliant Pecan server that receives
the Fault Management API requests. This hook logs the needed request
data to "/var/log/fm-api.log". All requests are logged except the
ones of "GET" type.
The code is a port from the same hook that exists in
"starlingx/config" repository, also called "AuditLogging".
Note: there are no "POST" or "PUT" requests available in Fault
Management API v1.
Test Plan:
PASS: Successfully deploy an AIO-SX and verify that the logs of
"fm-api" service are present in file "/var/log/fm-api.log".
PASS: In the deployed AIO-SX, execute command "fm alarm-list" and
check that no GET requests was logged in "fm-api.log".
PASS: In the deployed AIO-SX, execute command "fm alarm-delete 1111"
and check that a "DELETE /v1/alarms/1111" request was logged in
"fm-api.log" with status "404".
PASS: In the deployed AIO-SX, execute command
"fm event-suppress --alarm_id <alarm_id>" and check that a
"PATCH /v1/event_suppression/<uuid>" request was logged in
"fm-api.log" with status "200".
PASS: Successfully build all packages of this repository as Debian
packages: fm-api, fm-common, fm-doc, fm-mgr, fm-rest-api and
python-fmclient.
Story: 2009824
Task: 44468
Depends-On: https://review.opendev.org/c/starlingx/config-files/+/828200
Signed-off-by: Joao Victor Portal <Joao.VictorPortal@windriver.com>
Change-Id: I25bf662ed9e792b30ae1f90329bd35b918f0a5f7
Add debian packaging infrastructure for fm-api
so we can create a debian packaging for it.
Also update the directory layout due to limitations
of the STX tool. Update the centos build information
to reflect the new layout.
Story: 2009101
Task: 43091
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I5c9784647581f31384d7475f596663adc0d26498
Add support for Debian bullseye which runs python3.9.
This was tested by running tox locally on a Debian bullseye
VM.
Story: 2006796
Task: 42941
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I5568b85a365104fdbca2af7adcf9f289001baea0
Handle http client exceptions for non-admin
users. This is because some alarms/events
metadata are not available for them.
For example if the call fails the user will see the a
value like:
"host=controller-1.port=enp0s8"
instead of:
"host=controller-1.port=0cbb441a-874d-418e-bd57-7aa1b8ab0044"
Before this if an error in translation happened, the user
experienced an alert badge saying "Unable to retrieve
Events", then the user seen an empty table.
Story: 2009077
Task: 43093
Signed-off-by: Pablo Bovina <pablo.bovina@windriver.com>
Change-Id: I68eeb103a6fe97beb1fb7969a1cd9503210cdcb0
Fixed a typo for accessing a configuration resource.
Fixed a python3 script called by fmManager on initialization. It was
modifying a dictionary while parsing it using an iterator, which doesn't
guarantee all initial elements are visited. The fm database
event_suppression table was left in a bad state, thus alarm list and
alarm summary were empty.
Tests:
- build and deploy from iso
- tested with the fm_api_test.py provided in the repo
- tested some of the commands by raising a config out of date and
targeting the raised alarm
- fm alarm-list, alarm-delete, alarm-show, alarm-summary, event-list,
event-show, event-suppress-list, event-suppress, event-unsuppress,
event-unsuppress-all produce relevant output
Story: 2008454
Task: 42632
Depends-On: I2d0f4c2c85ea8057258d56632a102b2eac7db388
Signed-off-by: Dan Voiculeasa <dan.voiculeasa@windriver.com>
Change-Id: Ib3d5276c4a669a3f3f123470c31edf7a07751eaa
(cherry picked from commit c18a7865c6)
- Replace 'range' with six.moves.range.
- Replace 'zip' with six.moves.zip.
- Replace 'map' with six.moves.map.
- Replace dict.keys() with list(dict.keys()) to get a list on Python
3. On Python 3, dict.keys() now returns a view.
- Fix 'TypeError: unorderable types: NoneType() < int()' for python3
- Python 3 changed the iso8601 module iso8601.iso8601.Utc()
function to iso8601.UTC. This is compatible with Python2.7.
Testing:
1. Built new fm-rest-api rpm package.
2. Built new ISO with newer fm-rest-api and installed fine.
3. Checked for an exception in the logs while running "fm event-list".
4. Ran "fm alarm-list" without a problem.
Story: 2006729
Task: 42256
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I2d0f4c2c85ea8057258d56632a102b2eac7db388
(cherry picked from commit b91aaaab90)
The fm-api service is not properly started since the status logic is
wrong.
Updated the status logic to drop the regex approach and use the PIDFILE
instead.
There is a confirm_stop logic step which still needs to use the regex
approach because the main process spawns children. Updated the regex
for confirm_stop logic step. Now looking for /usr/bin/pythton3 and
/usr/libexec/platform-python.
Story: 2008454
Task: 42631
Depends-On: I970c2600475e32f2c5fb815738a2fe79f99a5b17
Signed-off-by: Dan Voiculeasa <dan.voiculeasa@windriver.com>
Change-Id: Ia77988ce282ea17de84b89e833ec686df342b3c4
(cherry picked from commit 4d5c6c7f21)
Centos 8 uses python3 by default so check for running python3
process as well.
Story: 2008454
Task: 42370
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I970c2600475e32f2c5fb815738a2fe79f99a5b17
(cherry picked from commit 4958c7be50)
System UUID not required to be part of alarms summary.
This is also needed to allow non-admin users to read
alarms summary.
Story: 2009077
Task: 42995
Signed-off-by: Pablo Bovina <pablo.bovina@windriver.com>
Change-Id: I55c0cb297be7e86d31bce116e8ba20a2162d2f04
The tox jobs for fault are having problems with a
timeout, which appears to be related to how long it takes
to setup the python requirements.
This update tries to limit the scope for yamllint, and
eliminate some of the sdist related actions.
pylint has been relocated to the test-requirements file.
The usedevelop = False is required for the base tox.ini
because there are no setup files at that directory level.
Several pylint errors had to be suppressed, and will be
restored once py3 changes are merged from centos8 branch.
Story: 2008943
Task: 42967
Signed-off-by: albailey <Al.Bailey@windriver.com>
Change-Id: Icc8747226511f0be614df20800813fe4eb9e6b96
flake8 2.5.5 fails on ubuntu-focal zuul machines running python3.8
with the following error:
AttributeError: 'FlakesChecker' object has no attribute 'CONSTANT'
Fixed:
E117 over-indented
E741 ambiguous variable name
F841 local variable is assigned to but never used
Per-Line-Suppressed:
E402 module level import not at top of file
F632 use ==/!= to compare constant literals
Global Suppress:
W504 line break after binary operator
W605 invalid escape sequence
The suppressed errors can be fixed by later submissions.
Change-Id: I2df3ece427f0c84ce16c1a82f4d9f0c9a5a6982b
Partial-Bug: 1895054
Signed-off-by: albailey <Al.Bailey@windriver.com>
This update changed the TIS_PATCH_VER to use the PKG_GITREVCOUNT
variable to auto-version the packages.
Adds an upper constraint to fm-rest-api to pass zuul.
Story: 2006740
Task: 39842
Change-Id: I4a180f1395ff9764ecc9d617a514e265db670b9c
Signed-off-by: Teresa Ho <teresa.ho@windriver.com>
stestr was upgraded to 3.0.0 on Mar. 26 and it does not support py27.
Update test requirements accordingly so py27 tests will use earlier version.
Change-Id: Ia481a4609d81ad4d225aeb6ec3ab0cc3a9e18ae5
Closes-Bug: 1869777
Signed-off-by: Gerry Kopec <gerry.kopec@windriver.com>
add test class DbSyncTestCase
add test case for migration database sync
Story: 2007082
Task: 38152
Depends-on: https://review.opendev.org/#/c/702823/
Signed-off-by: SidneyAn <ran1.an@intel.com>
Signed-off-by: chenyan <yan.chen@intel.com>
Change-Id: I8121dedae7e4319a454dc128a591f74ef066c149
As of March 1, when jobs run on ubuntu-xenial, they fail
to run py27 and py35 jobs with an error similar to:
/usr/bin/python3 -m virtualenv --no-download
--python /usr/bin/python2.7 py27
ImportError: cannot import name 'ContextManager'
It does not appear to impact the ubuntu-bionic nodes so
updating zuul to use those nodes.
bionic nodes do not support python3.5, so the py35 jobs
is now converted to py36.
Partial-Bug: 1865554
Change-Id: Iad13d03ced21294d1a02c82f4e1e37cac3f27ee7
Signed-off-by: albailey <Al.Bailey@windriver.com>
As new unit tests are being added, the coverage target
helps to view their impact.
Story: 2007082
Task: 38836
Change-Id: I2d57aac7c633ab358a2d31bfc8d8a8635931527f
Signed-off-by: albailey <Al.Bailey@windriver.com>
Add add the first unit test for fm-rest-api/fm/fm/common/timeutils.
Add tox task for py27/35 as zuul checking and gating jobs.
Story: 2007082
Task: 38077
Change-Id: I2f259dcf2178f42546966063c0d724bc17e2a804
Signed-off-by: chenyan <yan.chen@intel.com>
The dependency resolver for RPM relies on the shebang to have the
interperter that will be used in order to get things right, so
change shebang to /usr/bin/python.
In other cases, the shebang is not needed at all as these are not
really executable python files, they are part of the python package.
RPMLINT finds these and complains that a non-executable has a shebang.
Change-Id: Ie90a846d6addffca1225e59fd8b6f3753d2991a4
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Flake8 currently ignores the following errors:
H233: Python 3.x incompatible use of print operator
H236: Python 3.x incompatible __metaclass__
H238: Old style class declaration, use new style
Enable them for python3 compability
Change-Id: I8959f3a78ce683eb93a16234908662189dd7c8a5
Story: 2004515
Task: 29304
Signed-off-by: Eric Barrett <eric.barrett@windriver.com>
This commit implements the methods PUT and POST in order
to insert and update alarms.
Story: 2004008
Task: 30270
Change-Id: I773e651165b3654684b95463167fc565aef1ffa4
Co-authored-by: Sun Austin <austin.sun@intel.com>
Signed-off-by: Mario Alfredo Carrillo Arevalo <mario.alfredo.c.arevalo@intel.com>
Signed-off-by: Sun Austin <austin.sun@intel.com>
Test has been done on Simplex. the fm alarm-list can run
successfully.
Story: 2004008
Task: 30916
Change-Id: I993a52aa97ee8a21a8d04697b690f371d2a3bad5
Signed-off-by: Sun Austin <austin.sun@intel.com>
The jobs are working here but stx-nfv devstack fails in fm-rest-api,
what fun it is to find a transient dependency! fm-rest-api has
apparently never started correctly but we did not properly detect
that until getting the stx-nfv plugin up.
Add requirements.txt and test-requirements.txt, only
test-requirements.txt is directly used by DevStack but WSME
is not a test-only dependency...
Cleanup to follow:
* properly get the dependencies of the service documented for pip
* properly detect when fm-rest-api fails to start under systemd
Change-Id: I91236d2bbb785c75264e8c4772d4ad6dfa45562c
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
Issue:
for the alarm_show/alarm_delete/event_show api, if the input UUID
string is not valid, http server side will return error code 500
(Internal Server Error), due to the error code is not set correctly.
Solution:
Correct the error code in server, and add client code to handle
"HTTP 404 - Not Found" error code correctly.
Test:
Run "fm alarm-show/alarm-delete/event-show" with valid/invalid
UUID, and the response is correct. For invalid UUID, response
string will like below. "xxx" for the invalid UUID string.
"Alarm not found: xxx"
"Event log not found: xxx"
Closes-Bug: 1806927
Change-Id: I8d17c5bc55733f269d875ae835ab6295fed4d899
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Include the optional “expand” option to the get list for alarms and
events so the entire set of data can be returned and not just a
subset. “expand” is and optional parameter and defaults to False.
Story: 2004818
Task: 29096
Change-Id: Ife3b900d90c56564e9aeaa820e57f69d15194eb3
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
Closes-Bug: 1789979
fm-rest-api returns json format message for the correct api
access. For the invalid api access, the message is with html
format, need add wrap_app to convert it to json format.
Fix:
The implementation code is copied from upstream Ironic API,
and minor modification to pass tox pep8 check.
Also remove "x" attribute for the py file.
Test:
Pass build and multi node deploy test. Confirmed the return
message is json format.
Change-Id: I36fa89b82377d52008a467316c42c06caa65cd90
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
The fm-rest-api setup.py and fm-mgr spec file have incorrect
License.
This update changes the license to Apache-2.0 in setup.py and
and the spec file.
Change-Id: Id923baac0c54fb1feb38069dad73dfd13dbf1d39
Signed-off-by: Tao Liu <tao.liu@windriver.com>
Create fault management REST API service
Create fault management client and CLI shell
Add a python extension for fault management application APIs
Update fault management python APIs to use the python extension
Update fault manager to retrieve the SNMP configuration from the config file
Story: 2002828
Task: 22747
Depends-On: https://review.openstack.org/#/c/592176/
Change-Id: I888d8d23edf75d05d51594ccca55570ae366c848
Signed-off-by: Tao Liu <tao.liu@windriver.com>
Joao Victor Portal