Initail commit

This commit is contained in:
Andy Wu 2023-01-09 15:51:38 +00:00
commit 6d602a8bbb
21 changed files with 721 additions and 0 deletions

3
.stestr.conf Normal file
View File

@ -0,0 +1,3 @@
[DEFAULT]
test_path=./unit_tests
top_dir=./

4
.zuul.yaml Normal file
View File

@ -0,0 +1,4 @@
- project:
templates:
- openstack-python3-ussuri-jobs
- openstack-cover-jobs

65
README.md Normal file
View File

@ -0,0 +1,65 @@
# Overview
The cinder charm is the Openstack block storage (i.e: Volume) service, whereas the cinder-nfs charm works as a subordinate of cinder, implementing a NFS backend.
# Usage
## Configuration
This section covers common and/or important configuration options. See file `config.yaml` for the full list of options, along with their descriptions and default values.
### `nfs-shares`
A list of nfs shares that NFS driver should attempt to provision new Cinder volumes into
Multiple nfs shares can be provided, each on its own line, in a format of `<host>:<share path>`
```
192.168.1.200:/storage
192.168.1.201:/storage
```
The content will be written to /etc/cinder/nfs_shares by default or the file specified in nfs-shares-config option
### `nfs-shares-config`
The file that contain a list of NFS shares. Cinder-volume will read this file to get its NFS backend detail
### `nfs-mount-options`
Specify mount options. See section of the NFS man page for details.
## Deployment
This charm's primary use is as a backend for the cinder charm. To do so, add a relation betweeen both charms:
```
juju add-relation cinder-nfs:storage-backend cinder:storage-backend
```
# Developing
Create and activate a virtualenv with the development requirements:
```
virtualenv -p python3 venv
source venv/bin/activate
pip3 install -r requirements.txt
pip3 install -r test-requirements.txt
```
# Documentation
The OpenStack Charms project maintains two documentation guides:
* [OpenStack Charm Guide][cg]: for project information, including development
and support notes
* [OpenStack Charms Deployment Guide][cdg]: for charm usage information
# Bugs
Please report bugs on [Launchpad][lp-bugs-charm-cinder-netapp].
[cg]: https://docs.openstack.org/charm-guide
[cdg]: https://docs.openstack.org/project-deploy-guide/charm-deployment-guide
[lp-bugs-charm-cinder-netapp]: https://bugs.launchpad.net/charm-cinder-nfs/+filebug

7
build-requirements.txt Normal file
View File

@ -0,0 +1,7 @@
# NOTES(lourot):
# * We don't install charmcraft via pip anymore because it anyway spins up a
# container and scp the system's charmcraft snap inside it. So the charmcraft
# snap is necessary on the system anyway.
# * `tox -e build` successfully validated with charmcraft 1.2.1
cffi==1.14.6; python_version < '3.6' # cffi 1.15.0 drops support for py35.

32
charmcraft.yaml Normal file
View File

@ -0,0 +1,32 @@
type: charm
parts:
charm:
after:
- update-certificates
charm-python-packages:
# NOTE(lourot): see
# * https://github.com/canonical/charmcraft/issues/551
# * https://github.com/canonical/charmcraft/issues/632
- setuptools < 58
build-packages:
- git
update-certificates:
plugin: nil
# See https://github.com/canonical/charmcraft/issues/658
override-build: |
apt update
apt install -y ca-certificates
update-ca-certificates
bases:
- build-on:
- name: ubuntu
channel: "20.04"
architectures:
- amd64
run-on:
- name: ubuntu
channel: "20.04"
architectures: [amd64, s390x, ppc64el, arm64]
- name: ubuntu
channel: "22.04"
architectures: [amd64, s390x, ppc64el, arm64]

68
config.yaml Normal file
View File

@ -0,0 +1,68 @@
options:
volume-backend-name:
type: string
default: 'cinder-nfs'
description: |-
Service backend name to present to Cinder
If left empty, application's name will be used as backend name
nfs-shares:
type: string
default: ''
description: |-
'A list of nfs shares in format of <host>:<share>, each on their
own line, to which the driver should attempt to provision new
Cinder volumes into.
# example
192.168.1.200:/storage
192.168.1.201:/storage
Above content will be writtent to /etc/cinder/nfs_shares or the file
specified in nfs-shares-config option'
nfs-shares-config:
type: string
default: "/etc/cinder/nfs_shares"
description: |
The file name that contain a a list of nfs-shares
nfs-mount-point-base:
type: string
default: "/var/lib/cinder/nfs"
description: Directory where cinder-volume mounts all NFS shares.
nfs-mount-options:
type: string
default: ''
description: |-
Mount options passed to NFS client.
See NFS man page for available mount options
nfs-mount-attempts:
type: int
default: 3
description: |-
Number of attempts to mount NFS shares before raising an error.
At least one attempt will be made to mount an NFS share, regardless
of the value specified.
nfs-snapshot-support:
type: boolean
default: False
description: |
Enable support for snapshots on the NFS driver.
Platforms using libvirt <1.2.7 will encounter issues with this feature.
nfs-qcow2-volumes:
type: boolean
default: False
description: |
Create volumes as QCOW2 files rather than raw files.
nfs-sparsed-volumes:
type: boolean
default: True
description: |
'Create volumes as sparsed files which take no space.
If set to False volume is created as regular file. In such case volume
creation takes a lot of time.'

16
copyright Normal file
View File

@ -0,0 +1,16 @@
Format: http://dep.debian.net/deps/dep5/
Files: *
Copyright: Copyright 2021-2022, Canonical Ltd., All Rights Reserved.
License: Apache License 2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
http://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

24
metadata.yaml Normal file
View File

@ -0,0 +1,24 @@
name: cinder-nfs
summary: NFS integration for OpenStack Block Storage
maintainer: OpenStack Charmers <openstack-charmers@lists.ubuntu.com>
description: |
Cinder is the block storage service for the Openstack project.
.
This charm provides a NFS backend for Cinder
tags:
- openstack
- storage
- file-servers
- misc
series:
- focal
- jammy
subordinate: true
provides:
storage-backend:
interface: cinder-backend
scope: container
requires:
juju-info:
interface: juju-info
scope: container

7
osci.yaml Normal file
View File

@ -0,0 +1,7 @@
- project:
templates:
- charm-unit-jobs
vars:
needs_charm_build: true
build_type: charmcraft
charm_build_name: cinder-nfs

18
pip.sh Executable file
View File

@ -0,0 +1,18 @@
#!/usr/bin/env bash
#
# This file is managed centrally by release-tools and should not be modified
# within individual charm repos. See the 'global' dir contents for available
# choices of tox.ini for OpenStack Charms:
# https://github.com/openstack-charmers/release-tools
#
# setuptools 58.0 dropped the support for use_2to3=true which is needed to
# install blessings (an indirect dependency of charm-tools).
#
# More details on the beahvior of tox and virtualenv creation can be found at
# https://github.com/tox-dev/tox/issues/448
#
# This script is wrapper to force the use of the pinned versions early in the
# process when the virtualenv was created and upgraded before installing the
# depedencies declared in the target.
pip install 'pip<20.3' 'setuptools<50.0.0'
pip "$@"

13
rename.sh Executable file
View File

@ -0,0 +1,13 @@
#!/bin/bash
charm=$(grep "charm_build_name" osci.yaml | awk '{print $2}')
echo "renaming ${charm}_*.charm to ${charm}.charm"
echo -n "pwd: "
pwd
ls -al
echo "Removing bad downloaded charm maybe?"
if [[ -e "${charm}.charm" ]];
then
rm "${charm}.charm"
fi
echo "Renaming charm here."
mv ${charm}_*.charm ${charm}.charm

5
requirements.txt Normal file
View File

@ -0,0 +1,5 @@
# requirements
ops
git+https://github.com/juju/charm-helpers.git#egg=charmhelpers
git+https://opendev.org/openstack/charm-ops-openstack#egg=ops_openstack
git+https://opendev.org/openstack/charm-ops-interface-tls-certificates#egg=interface_tls_certificates

92
src/charm.py Executable file
View File

@ -0,0 +1,92 @@
#!/usr/bin/env python3
# Copyright 2022 Canonical Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Charm for deploying and maintaining the Cinder NFS backend driver."""
from ops.main import main
from ops.model import ActiveStatus, BlockedStatus
from ops_openstack.plugins.classes import CinderStoragePluginCharm
import os
import io
import shutil
def _check_config(charm_config):
"""
These checks are in addition to the parent class checks
for MANDATORY_CONFIG.
"""
if not charm_config["nfs-shares"]:
return BlockedStatus("NFS shares not configured")
return ActiveStatus("Unit is ready")
class CharmCinderNFSCharm(CinderStoragePluginCharm):
"""Charm the Cinder NFS driver."""
PACKAGES = ["cinder-common"]
MANDATORY_CONFIG = [
"nfs-shares",
"nfs-shares-config",
"volume-backend-name",
"nfs-mount-point-base",
"nfs-mount-attempts",
"nfs-snapshot-support",
"nfs-qcow2-volumes",
"nfs-sparsed-volumes",
]
def on_config(self, event):
status = _check_config(self.framework.model.config)
if not isinstance(status, ActiveStatus):
self.unit.status = status
return
super().on_config(event)
def cinder_configuration(self, charm_config):
options = []
nfs_shares = ""
volumedriver = "cinder.volume.drivers.nfs.NfsDriver"
options.append(("volume_driver", volumedriver))
for key, value in charm_config.items():
# if volume-backend-name is empty, set to application name
if key == "volume-backend-name" and not value:
value = self.framework.model.app.name
if key == "nfs-shares":
nfs_shares = os.linesep.join([s for s in value.splitlines() if s])
buff = io.StringIO(nfs_shares)
continue
if key == "nfs-shares-config":
path = value
with open(path, "w+") as f:
print(buff.getvalue(), file=f)
os.chmod(path, 0o640)
shutil.chown(path, user="root", group="cinder")
options.append((key.replace("-", "_"), value))
return options
if __name__ == "__main__":
main(CharmCinderNFSCharm)

16
test-requirements.txt Normal file
View File

@ -0,0 +1,16 @@
# This file is managed centrally. If you find the need to modify this as a
# one-off, please don't. Intead, consult #openstack-charms and ask about
# requirements management in charms via bot-control. Thank you.
charm-tools>=2.4.4
coverage>=3.6
mock>=1.2
flake8>=4.0.1
stestr>=2.2.0
requests>=2.18.4
psutil
# oslo.i18n dropped py35 support
oslo.i18n<4.0.0
git+https://github.com/openstack-charmers/zaza.git#egg=zaza
git+https://github.com/openstack-charmers/zaza-openstack-tests.git#egg=zaza.openstack
pytz # workaround for 14.04 pip/tox
pyudev # for ceph-* charm unit tests (not mocked?)

0
tests/__init__.py Normal file
View File

View File

@ -0,0 +1,35 @@
series: focal
variables:
openstack-origin: &openstack-origin distro
machines:
'0':
local_overlay_enabled: false
relations:
- - cinder:storage-backend
- cinder-three-par:storage-backend
applications:
cinder:
charm: ch:cinder
num_units: 1
options:
openstack-origin: *openstack-origin
block-device: None
overwrite: "true"
ephemeral-unmount: /mnt
to:
- '0'
cinder-three-par:
charm: ../../cinder-three-par.charm
options:
hpe3par-debug: False
driver-type: fc
san-ip: 127.0.0.1
san-login: admin
san-password: password
hpe3par-username: admin
hpe3par-password: password
hpe3par-api-url: https://127.0.0.1:8080/api/v1/
hpe3par-cpg: cpgname
hpe3par_cpg_snap: cpgname
use-multipath-for-image-xfer: True
enforce-multipath-for-image-xfer: True

17
tests/tests.yaml Normal file
View File

@ -0,0 +1,17 @@
charm_name: cinder-three-par
target_deploy_status:
cinder:
workload-status: blocked
workload-status-message-prefix: "Missing relations:"
tests:
- tests.tests_cinder_three_par.CinderThreeParTest
configure: []
gate_bundles:
- focal-ussuri
smoke_bundles:
- focal-ussuri
dev_bundles:
- focal-ussuri

45
tests/tests_cinder_nfs.py Normal file
View File

@ -0,0 +1,45 @@
#!/usr/bin/env python3
# Copyright 2021 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import zaza.model
from zaza.openstack.charm_tests.test_utils import BaseCharmTest
class CinderNFSTest(BaseCharmTest):
"""Encapsulate Cinder NFS tests."""
def test_cinder_config(self):
"""Test that configuration options match our expectations."""
zaza.model.run_on_leader(
"cinder",
"sudo cp /etc/cinder/cinder.conf /tmp/",
)
zaza.model.block_until_oslo_config_entries_match(
"cinder",
"/tmp/cinder.conf",
{
"cinder-nfs": {
# sanity test a few common params
"volume_backend_name": ["cinder-nfs"],
"nfs_share_config": ["/etc/cinder/nfs_shares"],
"nfs_mount_point_base": ["/var/lib/cinder/nfs"],
"nfs_mount_options": [""],
"nfs_mount_attempt": [3],
}
},
timeout=2,
)

134
tox.ini Normal file
View File

@ -0,0 +1,134 @@
# Operator charm (with zaza): tox.ini
[tox]
envlist = pep8,py3
skipsdist = True
# NOTE: Avoid build/test env pollution by not enabling sitepackages.
sitepackages = False
# NOTE: Avoid false positives by not skipping missing interpreters.
skip_missing_interpreters = False
# NOTES:
# * We avoid the new dependency resolver by pinning pip < 20.3, see
# https://github.com/pypa/pip/issues/9187
# * Pinning dependencies requires tox >= 3.2.0, see
# https://tox.readthedocs.io/en/latest/config.html#conf-requires
# * It is also necessary to pin virtualenv as a newer virtualenv would still
# lead to fetching the latest pip in the func* tox targets, see
# https://stackoverflow.com/a/38133283
# * It is necessary to declare setuptools as a dependency otherwise tox will
# fail very early at not being able to load it. The version pinning is in
# line with `pip.sh`.
requires = pip < 20.3
virtualenv < 20.0
setuptools < 50.0.0
tox < 4.0.0
# NOTE: https://wiki.canonical.com/engineering/OpenStack/InstallLatestToxOnOsci
minversion = 3.2.0
[testenv]
setenv = VIRTUAL_ENV={envdir}
PYTHONHASHSEED=0
CHARM_DIR={envdir}
install_command =
pip install {opts} {packages}
commands = stestr run --slowest {posargs}
allowlist_externals =
git
add-to-archive.py
bash
charmcraft
rename.sh
passenv = HOME TERM CS_* OS_* TEST_*
deps = -r{toxinidir}/test-requirements.txt
[testenv:py35]
deps =
basepython = python3.5
# python3.5 is irrelevant on a focal+ charm.
commands = /bin/true
[testenv:py36]
basepython = python3.6
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
[testenv:py37]
basepython = python3.7
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
[testenv:py38]
basepython = python3.8
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
[testenv:py39]
basepython = python3.9
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
[testenv:py310]
basepython = python3.10
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
[testenv:py3]
basepython = python3
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
[testenv:pep8]
basepython = python3
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
commands = flake8 {posargs} src unit_tests tests
[testenv:cover]
# Technique based heavily upon
# https://github.com/openstack/nova/blob/master/tox.ini
basepython = python3
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
setenv =
{[testenv]setenv}
PYTHON=coverage run
commands =
coverage erase
stestr run --slowest {posargs}
coverage combine
coverage html -d cover
coverage xml -o cover/coverage.xml
coverage report
[coverage:run]
branch = True
concurrency = multiprocessing
parallel = True
source =
.
omit =
.tox/*
*/charmhelpers/*
unit_tests/*
[testenv:venv]
basepython = python3
commands = {posargs}
[testenv:build]
basepython = python3
deps = -r{toxinidir}/build-requirements.txt
commands =
charmcraft clean
charmcraft -v pack
{toxinidir}/rename.sh
[testenv:func-noop]
basepython = python3
commands =
functest-run-suite --help
[testenv:func]
basepython = python3
commands =
functest-run-suite --keep-model
[testenv:func-smoke]
basepython = python3
commands =
functest-run-suite --keep-model --smoke
[testenv:func-dev]
basepython = python3
commands =
functest-run-suite --keep-model --dev
[testenv:func-target]
basepython = python3
commands =
functest-run-suite --keep-model --bundle {posargs}
[flake8]
# Ignore E902 because the unit_tests directory is missing in the built charm.
ignore = E402,E226,E902

16
unit_tests/__init__.py Normal file
View File

@ -0,0 +1,16 @@
# Copyright 2022 Canonical Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import ops.testing
ops.testing.SIMULATE_CAN_CONNECT = True

104
unit_tests/test_charm.py Normal file
View File

@ -0,0 +1,104 @@
# Copyright 2022 Canonical Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import unittest
import json
from ops.model import Relation, BlockedStatus, ActiveStatus
from ops.testing import Harness
from src.charm import CharmCinderNFSCharm
class TestCharm(unittest.TestCase):
maxDiff = None
def setUp(self):
self.harness = Harness(CharmCinderNFSCharm)
self.addCleanup(self.harness.cleanup)
self.harness.begin()
self.harness.set_leader(True)
self.model = self.harness.model
self.storage_backend = self.harness.add_relation("storage-backend", "cinder")
self.harness.add_relation_unit(self.storage_backend, "cinder/0")
self.harness.update_config(
{
"volume-backend-name": "cinder-nfs",
"nfs-shares": "172.18.18.61:/srv/test",
"nfs-mount-point-base": "/var/lib/cinder/nfs",
"nfs-mount-options": "vers=3",
"nfs-mount-attempts": 3,
}
)
def _get_sub_conf(self):
rel = self.model.get_relation("storage-backend", 0)
self.assertIsInstance(rel, Relation)
rdata = rel.data[self.model.unit]
rdata = json.loads(rdata["subordinate_configuration"])
return dict(
rdata["cinder"]["/etc/cinder/cinder.conf"]["sections"]["cinder-nfs"]
)
def test_backend_name_in_data(self):
rel = self.model.get_relation("storage-backend", 0)
rdata = rel.data[self.model.unit]
self.assertEqual(rdata["backend_name"], "cinder-nfs")
def test_config_changed(self):
self.harness.update_config(
{
"nfs-mount-point-base": "/var/lib/cinder/nfsmount",
"nfs-mount-options": "vers=4.1,proto=tcp,retry=4,",
"nfs-mount-attempts": 4,
}
)
self.assertEqual(
self._get_sub_conf(),
{
"volume_driver": "cinder.volume.drivers.nfs.NfsDriver",
"nfs_mount_point_base": "/var/lib/cinder/nfsmount",
"nfs_mount_options": "vers=4.1,proto=tcp,retry=4,",
"nfs_mount_attempts": 4,
},
)
def test_blocked_status(self):
self.harness.update_config(unset=["nfs-shares"])
self.assertIsInstance(self.harness.charm.unit.status, BlockedStatus)
message = self.harness.charm.unit.status.message
self.assertIn("NFS shares not configured", message)
def test_status_with_mandatory_config(self):
self.assertEqual(self.harness.charm.unit.status.message, "Unit is ready")
self.assertIsInstance(self.harness.charm.unit.status, ActiveStatus)
self.harness.update_config(
unset=["nfs-shares-config"],
)
self.assertEqual(
self.harness.charm.unit.status.message,
"Missing option(s): nfs-shares-config",
)
self.assertIsInstance(self.harness.charm.unit.status, BlockedStatus)
def test_volume_backend_name_config(self):
self.assertEqual(self._get_sub_conf().get("volume_backend_name"), "cinder-nfs")
self.harness.update_config(
{
"volume-backend-name": "test-backend",
}
)
self.assertEqual(
self._get_sub_conf().get("volume_backend_name"), "test-backend"
)