Instead of
`<manila.db.sqlalchemy.models.ShareServer object at 0x7f603645bf40>`
we now get
```
ShareServer(created_at=datetime.datetime(2023, 4, 18, 15, 44, 43, 741302),
updated_at=datetime.datetime(2023, 4, 19, 10, 18, 59, 614018),
deleted_at=datetime.datetime(2023, 4, 19, 10, 18, 59, 613577),
id='3363dcd4-fcdd-4cfa-b02d-5899fba3d413',
deleted='3363dcd4-fcdd-4cfa-b02d-5899fba3d413',
share_network_subnet_id='8d4365dc-0869-4896-8049-6455f84ad08f', ...)
```
e.g. in error messages of failed purges or in interactive shell sessions.
Change-Id: I5e6513e7ca7b4a9d4d7261f873cbf3f5bf3c83ab
Add the privsep module to Manila, as well as its initialization.
All changes to other Manila calls will be reusing this new module
and the new root context brought by it.
Partially-Implements: bp privsep-migration
Change-Id: I35bd548894d96ed66faab4ede7c16f28e9755663
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project. Tenacity is a fork of retrying, but has improved the
interface and extensibility (see [1] for more details). Our end
goal here is removing the retrying package from our requirements.
Tenacity provides the same functionality as retrying, but has the
following major differences to account for:
- Tenacity uses seconds rather than ms as retrying did
(the retry interface in manila exposed time in seconds as well)
- Tenacity has different kwargs for the decorator and
Retrying class itself.
- Tenacity has a different approach for retrying args by
using classes for its stop/wait/retry kwargs.
- By default tenacity raises a RetryError if a retried callable
times out; retrying raises the last exception from the callable.
Tenacity provides backwards compatibility here by offering
the 'reraise' kwarg - we are going to set this in the retry interface
by default.
- For retries that check a result, tenacity will raise if the
retried function raises, whereas retrying retried on all
exceptions - we haven't exposed this in the retry interface.
This patch updates all usages of retrying with tenacity.
Unit tests are added where applicable.
[1] https://github.com/jd/tenacity
Co-Authored-By: boden <bodenvmw@gmail.com>
Co-Authored-By: Goutham Pacha Ravi <gouthampravi@gmail.com>
Closes-Bug: #1635393
Change-Id: Ia0c3fa5cd82356a33becbf57444f3db5ffbb0dd0
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Oslo policy moved "deprecated_*" parameters to the
DeprecatedRule object in [1] and deprecated it in
DocumentedRuleDefault object. Also bump oslo_policy
to the 3.7.0.
Similar change in neutron: https://review.opendev.org/781561
[1] https://review.opendev.org/766628
Change-Id: I14b215cdcd6458d67622360e4c910b3da1ae9848
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
oslo policy handles the mapping of
credentials from a context object to values
that oslo policy cares about. This mapping
includes some deprecations and compatibility
handling code that we must take advantage of [1].
So, stop mapping context to policy values
on our end, and rely on oslo.policy handling
this for us.
enforce and authorize methods in policy.py
do the same thing, but with a subtle
difference. The "enforce" method doesn't
raise errors when unregistered policies are
invoked - this shouldn't ever be the case
for any policies written/maintained within
manila - however, we support API extensions
and don't dictate what must be done there. So
add docstrings to clarify that we shouldn't
invoke enforce, ever.
Also handle InvalidPolicyScope exceptions
and raise the oslo.policy library version
since some test enhancements have been
committed in the latest version.
[1] d3185debdb/oslo_policy/policy.py (L1077-L1096)
Change-Id: I069bf7143d6ff66b3dcdc34c9b52d48f5808481b
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
*) Add osprofiler wsgi middleware
This middleware is used for 2 things:
1) It checks that person who want to trace is trusted and knows
secret HMAC key.
2) It start tracing in case of proper trace headers and add
first wsgi trace point, with info about HTTP request
*) Add initialization of osprofiler at start of serivce.
You should use python-manilaclient with this patch:
https://review.opendev.org/#/c/769731
Run any command with --profile SECRET_KEY
$ manila --profile SECRET_KEY create NFS 1 --name <share> \
--description <share_description> --share-network <network> \
--share-type default
# it will print <Trace ID>
Get pretty HTML with traces:
$ osprofiler trace show --html <Trace ID> --connection-string \
<connection_string> --out <output.html>
e.g. --connection-string can be redis://localhost:6379
Note that osprofiler should be run from admin user name & tenant.
Implements: blueprint manila-os-profiler
Change-Id: I3bce1f04d1cfebfacd78ed135a949a068c78987d
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:
1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.
2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.
Also convert manila/tests/policy.json to manila/tests/policy.yaml
using oslopolicy-convert-json-to-yaml tool and replace
policy.json to policy.yaml ref from doc and tests.
[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I3748313912b2527c43c9b16a6ba3e3ccd4cf5221
pip 20.3 brings in a strict dependency resolver which
is enabled by default. This causes our lower-constraints
tests to fail, because the requirement files were out
of date from reality - they had conflicting requirements
which previous versions of pip were ignoring. Let's catch
up package versions to newer ones that are supported in
the python runtimes that the Wallaby release will be
deployed to.
[1] http://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html
[2] https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020
Change-Id: I5a31b561654aa368bb85a56f4dd38276cfdbb91a
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
As per victoria cycle testing runtime and community goal[1]
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).
Fixing:
- bug#1886298
Bump the lower constraints for required deps which added python3.8 support
in their later version.
Story: #2007865
Task: #40194
Depends-On: https://review.opendev.org/#/c/738328/
Closes-Bug: #1886298
Change-Id: Ib0f84e8c3815be48bca0ae8d4fe15a56b5256c20
When the image count is over 25, there might not get
manila-service-image, because current manila shares
creation is using novaclient to get image info, but
novaclient can only get 25 images due to pagination
of glance server, So this change is to switch to use
glanceclient instead of novaclient to get image info,
because glanceclient can iter all image info, while
novaclient is rarely maintained with stuff of image
API.
Change-Id: Id1715d0b9cb3a4aeedeb23d9b1d9924a78d18dc6
Closes-Bug: #1741425
Use the default value provided by tox:
https://tox.readthedocs.io/en/latest/config.html#conf-install_command
See discussion on the openstack-discuss ML [1]
for the complete context.
To get this fix in, we'll need to raise
a few requirements:
- eventlet because of the ssl issues with python3.6 in
older packages
(https://github.com/eventlet/eventlet/issues/371)
- oslo.messaging to fix the error "ACCESS_REFUSED - Login
was refused using authentication mechanism AMQPLAIN.".
- psycopg2-binary since 2.6.2 is blacklisted
- oslo.service was bumped to 2.1.0, to pick up its support
for eventlet>=0.22
- oslo.utils was bumped to match the lower constraint from
oslo.service at 2.1.0
[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014237.html
Change-Id: I6344f44fb955d631a4f0a13fa53ecc19826c7ea6
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
OpenStack is dropping the py2.7 support in ussuri cycle.
Make a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
know about the requirement
- Remove section from setup.cfg: Wheel is not needed for python
3 only repo
- Update requirements
Change-Id: Ibca03f9ebaeaca595e027be9cb5900c368c8de6d
This release of the Cinder client broke support for the v3
volume-transfer APIs unless microversion 3.55 or higher was requested.
depend on https://review.opendev.org/#/c/587877/
Depends-On: Ibbc3a3f04ea6253139112944cf7e0f6c265d0f26
Change-Id: Icb4c948e9f8ea879f5c2bc5d854df6696ee8e6a7
This adds basic framework for manila-status upgrade
check commands. For now it has only "check_placeholder"
check implemented.
Real checks can be added to this tool in the future.
Change-Id: Id809535d0a01617916a8e29f151ca4e61f738fad
Story: 2003657
Task: 26139
We will manage the eventlet version using constraints now. See the
thread starting at
http://lists.openstack.org/pipermail/openstack-dev/2018-April/129096.html
for more details.
Also adjust lower_contstraints.txt to match requirements.txt and
test_requirements.txt so that the requirements job will pass.
Change-Id: Ib1d74405c239f0a7d62f327a3aad6a9c8fd549e7
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
When the image count is over 25, there might not get
manila-service-image, because current manila shares
creation is using novaclient to get image info, but
novaclient can only get 25 images due to pagination
of glance server, So this change is to switch to use
glanceclient instead of novaclient to get image info,
because glanceclient can iter all image info, while
novaclient is rarely maintained with stuff of image
API.
Change-Id: Id905d47600bda9923cebae617749c8286552ec94
Closes-Bug: #1741425