starlingx/distcloud
Code Issues Proposed changes

Decontainerizing rvmc.py and Modular Integration in DC Repo

Browse Source 
Make the following changes for decontainerizing rvmc.py:
1. Copy rvmc.py from the Metal repo to the DC repo. Convert it into a
   common module that can be imported either from an independent script
   (rvmc_install.py), which is invoked from the Ansible installation
   playbook, or from other DC Python modules, such as the
   subcloud_install module. This module is used to shut down the host
   when subcloud deployment is aborted.
2. Create a Redfish Debian package called python3-redfish to install
   the Redfish Python library for the rvmc module.
3. Install rvmc_install.py to /usr/local/bin using the
   distributedcloud-dccommon package.

Make the following changes in the rvmc module/script:
1. Create LoggingUtil and ExitHandler classes to differentiate the way
   of logging and exit handling when called from the module or an
   independent script.
2. Implement HTTP request logging when the debug log level is set to 0.
3. Introduce a timeout mechanism to restrict the total execution time
   of the script during installation.
4. Add a signal handler to ensure the script exits properly when
   aborting a subcloud installation. Implement process ID tracking to
   allow only one RVMC process to run at a time for a subcloud
   installation, regardless of the termination status of previous
   installations.
5. Incorporate a power-off function to be called from the DC
   subcloud_install module, which is used in "subcloud deploy abort."
6. Include "subcloud_name" and "config_file" parameters/arguments to
   enhance functionality and flexibility.

Test Plan:

PASS: Verify successful subcloud installation.
PASS: Verify successful "subcloud deploy abort/resume".
PASS: Verify the playbook execution is halted, and the subcloud is
      shut down when an abort is triggered during subcloud installation
PASS: Verify the successful generation of the Redfish Debian package
      using the 'build-pkgs' command.
PASS: Verify that the image is successfully generated, along with
      the required Redfish package and its dependencies,
      using the 'build-image' command.
PASS: Verify that 'rvmc_install.py' is located in '/usr/local/bin'
      after the successful installation of the system controller with
      the new image.
PASS: Verify HTTP request and response logging are generated regardless
      of debug log level for failure conditions.
PASS: Verify the retry attempts to be applied only when the response
      status code is 5XX.
PASS: Verify that the subcloud installation command fails with
      the expected error message when the value of 'rvmc_debug_level'
      is set outside of the range [0..4].
PASS: Verify the success of the subcloud installation command by setting
      the 'rvmc_debug_level' value within the range [0..4].
PASS: Verify that the RVMC script exits when the timeout is reached.
PASS: Verify that only one RVMC process is running at a time for
      a subcloud installation, regardless of whether the previous
      subcloud installation terminated properly or not.

Story: 2010144
Task: 48897

Depends-On: https://review.opendev.org/c/starlingx/tools/+/900206

Change-Id: Ic35da91a35594ede668cef6875c752b93a19d6d5
Signed-off-by: lzhu1 <li.zhu@windriver.com>
This commit is contained in:
Li Zhu 2023-10-05 16:04:43 -04:00
parent 28ca4fc97f
commit 4b876fa82c
22 changed files with 2190 additions and 173 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
debian_iso_image.inc
View File

@ -6,4 +6,4 @@ distributedcloud-dcdbsync
distributedcloud-dcmanager
distributedcloud-dcorch
python3-redfish

1
debian_pkg_dirs
View File

@ -1 +1,2 @@
distributedcloud
python/python3-redfish

13
distributedcloud/dccommon/exceptions.py
View File

@ -121,10 +121,6 @@ class ImageNotInLocalRegistry(NotFound):
"system registry-image-tags %(image_name)s")
class SubcloudShutdownError(PlaybookExecutionFailed):
message = _("Subcloud %(subcloud_name)s could not be shut down.")
class ApiException(DCCommonException):
message = _("%(endpoint)s failed with status code: %(rc)d")
@ -140,3 +136,12 @@ class SubcloudPeerGroupNotFound(NotFound):
class SubcloudPeerGroupDeleteFailedAssociated(DCCommonException):
message = _("Subcloud Peer Group %(peer_group_ref)s delete failed "
"cause it is associated with a system peer.")
class RvmcException(Exception):
def __init__(self, message=None):
super(RvmcException, self).__init__(message)
class RvmcExit(DCCommonException):
message = _("Rvmc failed with status code: %(rc)d")

1692
distributedcloud/dccommon/rvmc.py Executable file
View File

File diff suppressed because it is too large Load Diff

146
distributedcloud/dccommon/subcloud_install.py
View File

@ -13,41 +13,31 @@
# limitations under the License.
#
import json
import os
import shutil
import socket
import ssl
import tempfile
import time
from eventlet.green import subprocess
import netaddr
from oslo_concurrency import lockutils
from oslo_log import log as logging
from six.moves.urllib import error as urllib_error
from six.moves.urllib import parse
from six.moves.urllib import request
import yaml
from dccommon import consts
from dccommon.drivers.openstack.keystone_v3 import KeystoneClient
from dccommon.drivers.openstack.sysinv_v1 import SysinvClient
from dccommon import exceptions
from dccommon import kubeoperator
from dccommon import utils as dccommon_utils
from dcmanager.common import consts as dcmanager_consts
from dcmanager.common import utils
LOG = logging.getLogger(__name__)
BOOT_MENU_TIMEOUT = '5'
# The RVMC_IMAGE_NAME:RVMC_IMAGE_TAG must align with the one specified
# in system images in the ansible install/upgrade playbook
RVMC_IMAGE_NAME = 'docker.io/starlingx/rvmc'
RVMC_IMAGE_TAG = 'stx.8.0-v1.0.2'
SUBCLOUD_ISO_PATH = '/opt/platform/iso'
SUBCLOUD_ISO_DOWNLOAD_PATH = '/var/www/pages/iso'
SUBCLOUD_FEED_PATH = '/var/www/pages/feed'
@ -60,104 +50,6 @@ NETWORK_INTERFACE_PREFIX = 'ifcfg'
NETWORK_ROUTE_PREFIX = 'route'
LOCAL_REGISTRY_PREFIX = 'registry.local:9001/'
# Redfish constants
ACTION_URL = '/Actions/ComputerSystem.Reset'
POWER_OFF_PAYLOAD = {'Action': 'Reset', 'ResetType': 'ForceOff'}
REDFISH_HEADER = {'Content-Type': 'application/json',
'Accept': 'application/json'}
REDFISH_SYSTEMS_URL = '/redfish/v1/Systems'
SUCCESSFUL_STATUS_CODES = [200, 202, 204]
RVMC_LOCK_NAME = 'dc-rvmc-install'
RVMC_NAMESPACE = 'rvmc'
KUBE_SYSTEM_NAMESPACE = 'kube-system'
DEFAULT_REGISTRY_KEY = 'default-registry-key'
class SubcloudShutdown(object):
"""Sends a shutdown signal to a Redfish controlled subcloud
Approach:
To shutdown a Redfish controlled subcloud, it's needed to first
send a GET request to find the @odata.id of the member, and then
send a POST request with the shutdown signal. Since this is
intended as a way to turn off the subcloud during the deploy abort
process, only the ForceOff option is considered.
"""
def __init__(self, subcloud_name):
self.target = subcloud_name
self.rvmc_data = self._get_subcloud_data()
def _get_subcloud_data(self):
rvmc_config_file_path = os.path.join(consts.ANSIBLE_OVERRIDES_PATH,
self.target, consts.RVMC_CONFIG_FILE_NAME)
if not os.path.isfile(rvmc_config_file_path):
raise Exception('Missing rvmc files for %s' % self.target)
with open(os.path.abspath(rvmc_config_file_path), 'r') as f:
rvmc_data = f.read()
rvmc_config_values = yaml.load(rvmc_data, Loader=yaml.SafeLoader)
base_url = "https://" + rvmc_config_values['bmc_address']
bmc_username = rvmc_config_values['bmc_username']
bmc_password = rvmc_config_values['bmc_password']
credentials = ("%s:%s" % (bmc_username.rstrip(), bmc_password)).encode("utf-8")
return {'base_url': base_url, 'credentials': credentials}
def _make_request(self, url, credentials, method, retry=5):
if method == 'get':
payload = None
else:
payload = json.dumps(POWER_OFF_PAYLOAD).encode('utf-8')
try:
context = ssl._create_unverified_context()
req = request.Request(url, headers=REDFISH_HEADER, method=method)
req.add_header('Authorization', 'Basic %s' % credentials)
response = request.urlopen(req, data=payload, context=context)
status_code = response.getcode()
if status_code not in SUCCESSFUL_STATUS_CODES:
if retry <= 0:
raise exceptions.SubcloudShutdownError(
subcloud_name=self.target)
retry -= retry
time.sleep(2)
self._make_request(url, credentials, method, retry=retry)
except urllib_error.URLError:
# This occurs when the BMC is not available anymore,
# so we just ignore it.
return None
except Exception as ex:
raise ex
return response
def _get_data_id(self):
base_url = self.rvmc_data['base_url']
credentials = self.rvmc_data['credentials']
url = base_url + REDFISH_SYSTEMS_URL
response = self._make_request(url, credentials, method='GET')
if not response:
return None
r = json.loads(response.read().decode())
for member in r['Members']:
if member.get('@odata.id'):
url_with_id = member['@odata.id']
break
return url_with_id
def send_shutdown_signal(self):
base_url = self.rvmc_data['base_url']
credentials = self.rvmc_data['credentials']
url_with_id = self._get_data_id()
if not url_with_id:
return None
url = base_url + url_with_id + ACTION_URL
response = self._make_request(url, credentials, method='POST')
return response
class SubcloudInstall(object):
"""Class to encapsulate the subcloud install operations"""
@ -256,17 +148,6 @@ class SubcloudInstall(object):
return "%s://%s:%s" % (protocol, self.get_oam_address(), port)
def check_image_exists(self, image_name, image_tag):
tags = self.sysinv_client.get_registry_image_tags(image_name)
if tags:
if any(getattr(tag, 'tag') == image_tag for tag in tags):
return
msg = "Error: Image %s:%s not found in the local registry." % (
image_name, image_tag)
LOG.error(msg)
raise exceptions.ImageNotInLocalRegistry(image_name=image_name,
image_tag=image_tag)
@staticmethod
def create_rvmc_config_file(override_path, payload):
@ -279,39 +160,17 @@ class SubcloudInstall(object):
if k in consts.BMC_INSTALL_VALUES or k == 'image':
f_out_rvmc_config_file.write(k + ': ' + v + '\n')
@lockutils.synchronized(RVMC_LOCK_NAME)
def copy_default_registry_key(self):
"""Copy default-registry-key secret for pulling rvmc image."""
kube = kubeoperator.KubeOperator()
try:
if kube.kube_get_secret(DEFAULT_REGISTRY_KEY, RVMC_NAMESPACE) is None:
if not kube.kube_get_namespace(RVMC_NAMESPACE):
LOG.info("Creating rvmc namespace")
kube.kube_create_namespace(RVMC_NAMESPACE)
LOG.info("Copying default-registry-key secret to rvmc namespace")
kube.kube_copy_secret(
DEFAULT_REGISTRY_KEY, KUBE_SYSTEM_NAMESPACE, RVMC_NAMESPACE)
except Exception as e:
LOG.exception("Failed to copy default-registry-key secret")
raise e
def create_install_override_file(self, override_path, payload):
LOG.debug("create install override file")
self.check_image_exists(RVMC_IMAGE_NAME, RVMC_IMAGE_TAG)
rvmc_image = LOCAL_REGISTRY_PREFIX + RVMC_IMAGE_NAME + ':' +\
RVMC_IMAGE_TAG
install_override_file = os.path.join(override_path,
'install_values.yml')
rvmc_name = "%s-%s" % (consts.RVMC_NAME_PREFIX, self.name)
host_name = socket.gethostname()
with open(install_override_file, 'w') as f_out_override_file:
f_out_override_file.write(
'---'
'\npassword_change: true'
'\nrvmc_image: ' + rvmc_image +
'\nrvmc_name: ' + rvmc_name +
'\nhost_name: ' + host_name +
'\nrvmc_config_dir: ' + override_path
+ '\n'
@ -687,9 +546,6 @@ class SubcloudInstall(object):
# create the rvmc config file
self.create_rvmc_config_file(override_path, payload)
# copy the default_registry_key secret to rvmc namespace
self.copy_default_registry_key()
# remove the bmc values from the payload
for k in consts.BMC_INSTALL_VALUES:
if k in payload:

17
distributedcloud/dccommon/utils.py
View File

@ -30,6 +30,7 @@ from dccommon import consts
from dccommon import exceptions
from dccommon.exceptions import PlaybookExecutionFailed
from dccommon.exceptions import PlaybookExecutionTimeout
from dccommon import rvmc
from dccommon.subprocess_cleanup import kill_subprocess_group
from dccommon.subprocess_cleanup import SubprocessCleanup
from dcorch.common.i18n import _
@ -356,3 +357,19 @@ def get_ssl_cert_ca_file():
return os.path.join(
consts.SSL_CERT_CA_DIR,
consts.CERT_CA_FILE_DEBIAN if is_debian() else consts.CERT_CA_FILE_CENTOS)
def send_subcloud_shutdown_signal(subcloud_name):
"""Sends a shutdown signal to a Redfish controlled subcloud.
:param subcloud_name: the name of the subcloud to be shut down
:type subcloud_name: str
"""
# All logs are expected to originate from the rvmc module,
# so the log churn from the 'redfish.rest.v1' module is disabled.
logging.getLogger('redfish.rest.v1').setLevel(logging.CRITICAL)
rvmc_config_file = os.path.join(consts.ANSIBLE_OVERRIDES_PATH,
subcloud_name,
consts.RVMC_CONFIG_FILE_NAME)
rvmc.power_off(subcloud_name, rvmc_config_file, LOG)

4
distributedcloud/dcmanager/common/consts.py
View File

@ -263,6 +263,10 @@ ERR_MSG_DICT = {
"ping_bmc": "Check reachability to the BMC: ping <<BMC_URL>>",
"rvmc_process": "Ensure the previous RVMC process is terminated.",
"rvmc_timeout": "Please check the dcmanager ansible log for details.",
"dm_pod_failed": """- Ensure you are using the correct tarball that \
corresponds to the image.
- Check helm overrides files, ensure the deployment manager images exist in \

10
distributedcloud/dcmanager/common/phased_subcloud_deploy.py
View File

@ -638,6 +638,16 @@ def validate_install_values(payload, subcloud=None):
LOG.exception(e)
pecan.abort(400, _("rd.net.timeout.ipv6dad invalid: %s") % e)
if 'rvmc_debug_level' in install_values:
try:
rvmc_debug_level = int(install_values['rvmc_debug_level'])
if rvmc_debug_level < 0 or rvmc_debug_level > 4:
pecan.abort(400, _("rvmc_debug_level must be an integer "
"between 0 and 4."))
except ValueError as e:
LOG.exception(e)
pecan.abort(400, _("Invalid value of rvmc_debug_level: %s") % e)
def validate_k8s_version(payload):
"""Validate k8s version.

29
distributedcloud/dcmanager/manager/subcloud_manager.py
View File

@ -46,9 +46,9 @@ from dccommon.exceptions import PlaybookExecutionFailed
from dccommon.exceptions import SubcloudNotFound
from dccommon import kubeoperator
from dccommon.subcloud_install import SubcloudInstall
from dccommon.subcloud_install import SubcloudShutdown
from dccommon.utils import AnsiblePlaybook
from dccommon.utils import LAST_SW_VERSION_IN_CENTOS
from dccommon.utils import send_subcloud_shutdown_signal
from dcmanager.audit import rpcapi as dcmanager_audit_rpc_client
from dcmanager.common import consts
from dcmanager.common.consts import INVENTORY_FILE_POSTFIX
@ -291,7 +291,11 @@ class SubcloudManager(manager.Manager):
"-e", "@%s" % dccommon_consts.ANSIBLE_OVERRIDES_PATH + "/" +
subcloud_name + '/' + "install_values.yml",
"-e", "install_release_version=%s" %
software_version if software_version else SW_VERSION]
software_version if software_version else SW_VERSION,
"-e", "rvmc_config_file=%s" %
os.path.join(dccommon_consts.ANSIBLE_OVERRIDES_PATH,
subcloud_name,
dccommon_consts.RVMC_CONFIG_FILE_NAME)]
return install_command
def compose_bootstrap_command(self, subcloud_name,
@ -1107,24 +1111,11 @@ class SubcloudManager(manager.Manager):
return
if subcloud.deploy_status == consts.DEPLOY_STATE_ABORTING_INSTALL:
# First delete the k8s job and pod, stopping the current
# installation attempt if exists
# Then send shutdown signal to subcloud
kube = kubeoperator.KubeOperator()
shutdown_subcloud = SubcloudShutdown(subcloud.name)
namespace = dccommon_consts.RVMC_NAME_PREFIX
jobname = '%s-%s' % (namespace, subcloud.name)
pod_basename = '%s-' % jobname
all_pods = kube.get_pods_by_namespace(namespace)
desired_pod = next((s for s in all_pods if pod_basename in s), None)
if desired_pod:
kube.kube_delete_job(jobname, namespace)
kube.kube_delete_pod(desired_pod, namespace)
while kube.pod_exists(desired_pod, namespace):
time.sleep(2)
shutdown_subcloud.send_shutdown_signal()
# Send shutdown signal to subcloud
send_subcloud_shutdown_signal(subcloud.name)
except Exception as ex:
LOG.error("Subcloud deploy abort failed for subcloud %s" % subcloud.name)
LOG.error("Subcloud deploy abort failed for subcloud %s: %s" %
(subcloud.name, str(ex)))
utils.update_abort_status(context, subcloud.id, subcloud.deploy_status,
abort_failed=True)
# exception is logged above

7
distributedcloud/dcmanager/orchestrator/states/upgrade/upgrading_simplex.py
View File

@ -356,12 +356,17 @@ class UpgradingSimplexState(BaseState):
utils.create_subcloud_inventory(install_values,
ansible_subcloud_inventory_file)
rvmc_config_file = os.path.join(dccommon_consts.ANSIBLE_OVERRIDES_PATH,
strategy_step.subcloud.name,
dccommon_consts.RVMC_CONFIG_FILE_NAME)
# SubcloudInstall.prep creates data_install.yml (install overrides)
install_command = [
"ansible-playbook", dccommon_consts.ANSIBLE_SUBCLOUD_INSTALL_PLAYBOOK,
"-i", ansible_subcloud_inventory_file,
"-e", "@%s" % dccommon_consts.ANSIBLE_OVERRIDES_PATH + "/" +
strategy_step.subcloud.name + '/' + "install_values.yml"
strategy_step.subcloud.name + '/' + "install_values.yml",
"-e", "rvmc_config_file=%s" % rvmc_config_file
]
# Run the remote install playbook

6
distributedcloud/dcmanager/tests/unit/manager/test_subcloud_manager.py
View File

@ -1762,7 +1762,11 @@ class TestSubcloudManager(base.DCManagerTestCase):
'-i', f'{dccommon_consts.ANSIBLE_OVERRIDES_PATH}/subcloud1_inventory.yml',
'--limit', 'subcloud1',
'-e', f"@{dccommon_consts.ANSIBLE_OVERRIDES_PATH}/subcloud1/install_values.yml",
'-e', "install_release_version=%s" % FAKE_PREVIOUS_SW_VERSION
'-e', "install_release_version=%s" % FAKE_PREVIOUS_SW_VERSION,
'-e', "rvmc_config_file=%s" %
os.path.join(dccommon_consts.ANSIBLE_OVERRIDES_PATH,
'subcloud1',
dccommon_consts.RVMC_CONFIG_FILE_NAME)
]
)

1
distributedcloud/debian/deb_folder/distributedcloud-dccommon.install
View File

@ -2,3 +2,4 @@ etc/logrotate.d/distcloud.conf
etc/syslog-ng/conf.d/distcloud.conf
usr/lib/python3/dist-packages/dccommon/*
usr/lib/python3/dist-packages/distributedcloud-*.egg-info
usr/local/bin/rvmc_install.py

7
distributedcloud/debian/deb_folder/rules
View File

@ -14,6 +14,9 @@ BIN_DIR = $(ROOT)/usr/bin
%:
dh $@ --with python3 --buildsystem=pybuild
override_dh_usrlocal:
# skip running dh_usrlocal
override_dh_install:
python3 setup.py install -f --install-layout=deb \
--root=$(CURDIR)/debian/tmp
@ -77,6 +80,10 @@ override_dh_install:
--output-file ./dcdbsync/dcdbsync.conf.sample
install -p -D -m 640 ./dcdbsync/dcdbsync.conf.sample $(SYS_CONF_DIR)/dcdbsync/dcdbsync.conf
# install rvmc_install.py script
install -d $(ROOT)/usr/local/bin/
install -p -D -m 700 scripts/rvmc_install.py $(ROOT)/usr/local/bin
rm -rf $(ROOT)/usr/lib/python3/dist-packages/dcmanagerclient/tests
rm -rf $(ROOT)/usr/lib/python3/dist-packages/dccommon/tests
rm -rf $(ROOT)/usr/lib/python3/dist-packages/dcmanager/tests

3
distributedcloud/requirements.txt
View File

@ -33,10 +33,11 @@ python-neutronclient>=6.3.0 # Apache-2.0
python-cinderclient>=2.1.0 # Apache-2.0
python-novaclient>=7.1.0 # Apache-2.0
python-keystoneclient>=3.8.0 # Apache-2.0
routes>=2.3.1 # MIT
redfish # BSD
requests!=2.12.2,!=2.13.0,>=2.10.0 # Apache-2.0
requests_toolbelt # Apache-2.0
retrying!=1.3.0,>=1.2.3 # Apache-2.0
routes>=2.3.1 # MIT
six>=1.9.0 # MIT
sqlalchemy!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8,>=1.0.10 # MIT
sqlalchemy-migrate>=0.11.0 # Apache-2.0

311
distributedcloud/scripts/rvmc_install.py Normal file
View File

@ -0,0 +1,311 @@
#!/usr/bin/python3
###############################################################################
#
# Copyright (c) 2019-2023 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# Redfish Virtual Media Controller Installer Script"""
#
###############################################################################
#
# This Redfish Virtual Media Controller forces an install of a specified BMC's
# host using the Redfish Platform management protocol.
#
# To do so the following Redfish operations are performed
#
# Step 1: Client Connect ... Establish a client connection to the BMC
# Step 2: Root Query ... Learn Redfish Services offered by the BMC
# Step 3: Find CD/DVD ... Locate the virtual media CD/DVD device
# Step 4: Power Off Host ... Host power needs to be off
# Step 5: Eject Iso ... Eject iso if needed
# Step 6: Inject Iso ... Inject the URL based ISO image into CD/DVD
# Step 7: Force DVD Boot ... Set Net boot device to be CD/DVD
# Step 8: Power On Host ... Host will boot and install from DVD
#
# Note: All server starting state conditions such as the server running or
# being stuck in POST, say at the grub prompt due to previous host boot
# failure, the host needs to be in the powered off state for the ISO
# Insertion and Set DVD for Next Boot steps.
#
# Mandatory Arguments:
#
# --config_file <A yaml file with BMC information>
#
# Config file is assumed to be a yaml file with the following format.
#
# bmc_address: <bmc ip address>
# bmc_username: <the bmc username>
# bmc_password: <base64 encoded password>
# image: http://<ip>:<port>/<path>/bootimage.iso
#
# Real Example:
#
# bmc_address: 123.45.67.89
# bmc_username: root
# bmc_password: TGk2OW51eCo=
# image: http://[2620:10a:a001:a103::81]:8080/iso/sub_cloud_bootimage.iso
#
# > rvmc_install.py --config_file
# "/opt/dc-vault/ansible/subcloud1/rvmc-config.yaml"
#
# --subcloud_name <subcloud name>
#
# The subcloud name is used as the bmc target name, and used for tracking
# the RVMC installation process ID. The process ID file is generated under
# '/var/run/rvmc', eg. '/var/run/rvmc/subcloud1_rvmc.pid'.
#
# > rvmc_install.py --subcloud_name subcloud1 --config_file <config_file>
#
# Optional Argument:
#
# --debug <0 .. 4>
#
# Note: 0 no debug info (default value)
# 1 = execution stage
# 2 + http request logs
# 3 + headers and payloads and misc other
# 4 + json output of all command responses
#
# > rvmc_install.py --debug 4 --subcloud_name <subcloud name>
# --config_file <config_file>
###############################################################################
#
# Code structure: Note: any error causes error log, session close and exit.
#
# parse command line arguments
#
# create target control object
#
# execute:
# _redfish_client_connect ... connect to bmc
# _redfish_root_query ... get base url tree
# _redfish_create_session ... authenticated session
# _redfish_get_managers ... get managers urls
# _redfish_get_systems_members .. get systems members info
# _redfish_get_vm_url ... get cd/dvd vm url
# _redfish_load_vm_actions ... get eject/insert action urls/info
# _redfish_poweroff_host ... tell bmc to power-off the host
# _redfish_eject_image ... eject current media if present
# _redfish_insert_image ... insert and verify insertion of iso
# _redfish_set_boot_override ... set boot from cd/dvd on next reset
# _redfish_poweron_host ... tell bmc to power-on the host
#
# exit code:
# 0 - Success
# 1 - Retryable failures:
# - Config file not found/opened
# - BNC IP address ping failure
# - Redfish GET query failures (managers, system members, URLs, status,
# virtual media group, etc.)
# - Power command not found
# - Host power on/off failure
# - Virtual media not supported by BMC
# - CD/DVD virtual media type not found
# - Eject target not found
# - VM state query failure
# - Image insertion/ejection timeout
# - Media insertion failure
# - Image verification failure
# - Boot override set/verification failure
# - ...
# 2 - Non-retryable failures:
# - Invalid credentials
# - Script execution time out
# - Failed to terminate the previous process
###############################################################################
import argparse
import eventlet
import os
import signal
import sys
import time
from dccommon import rvmc
# Constants
# ---------
FEATURE_NAME = 'Redfish Virtual Media Controller'
VERSION_MAJOR = 3
VERSION_MINOR = 1
# The path for RVMC PID file
RVMC_PID_FILE_PATH = '/var/run/rvmc/'
RVMC_PID_FILENAME_POSTFIX = '_rvmc.pid'
# The signals to be caught for abnormal termination
EXIT_SIGNALS = [signal.SIGTERM, signal.SIGABRT, signal.SIGINT]
# Global variables
# ------------------
# The logging utility
logging_util = None
# The exit handler
exit_handler = None
def parse_arguments():
"""Parse command line arguments.
:returns argparse.Namespace: the arguments with name and value
"""
parser = argparse.ArgumentParser(description=FEATURE_NAME)
parser.add_argument("--debug", type=int, required=False, default=0,
help="Optional debug level ; 0..4")
parser.add_argument("--subcloud_name", type=str, required=False,
help="Subcloud name")
parser.add_argument("--config_file", type=str, required=True,
help="RVMC config file")
return parser.parse_args()
def prepare_execution(rvmc_pid_file):
"""Terminate the previous RMVC process if it's still running.
:param rvmc_pid_file: The RVMC PID file
:param type: str
"""
if not rvmc_pid_file:
return
if not os.path.exists(RVMC_PID_FILE_PATH):
os.makedirs(RVMC_PID_FILE_PATH)
# Check if the PID file exists.
# Usually, it exists only when the parent process was manually killed.
if os.path.exists(rvmc_pid_file):
with open(rvmc_pid_file, 'r') as pid_file:
pid = pid_file.read()
# Attempt to kill the previous RVMC process using SIGTERM (15)
if pid:
try:
os.kill(int(pid), 15)
except ProcessLookupError:
# Ignore the error if the process with this PID doesn't exit
logging_util.ilog(
"Process %s not found or already terminated." % pid)
except Exception:
logging_util.elog(
"Failed to terminate the previous process %s," % pid)
logging_util.alog(
"Please terminate the previous process %s "
"before running the RVMC script again." % pid)
exit_handler.exit(2)
# Give some time between reading and writing to the same PID file
time.sleep(3)
# Get the current process ID
current_pid = os.getpid()
# Write the PID to the file
logging_util.dlog1("Save process ID %d to the file %s." %
(current_pid, rvmc_pid_file))
with open(rvmc_pid_file, 'w') as pid_file:
pid_file.write(str(current_pid))
def signal_handler():
"""This function handles signals received by the script"""
logging_util.elog("Received exit signal.")
exit_handler.exit(1)
class ExitHandler(rvmc.ExitHandler):
"""A utility class for handling different exit scenarios in a process.
Provides methods to manage the process exit in various situations.
"""
def __init__(self, rvmc_pid_file):
"""Handler object constructor.
:param rvmc_pid_file: the RVMC PID file.
:type rvmc_pid_file: str.
"""
self.rvmc_pid_file = rvmc_pid_file
def exit(self, code):
"""Early fault handling.
:param code: the exit status code
:type code: int.
"""
if self.rvmc_pid_file and os.path.exists(self.rvmc_pid_file):
os.remove(self.rvmc_pid_file)
sys.stdout.write("\n\n")
sys.exit(code)
##############################################################################
#
# Main steps:
# 1. Parse script arguments.
# 2. Register the signal handler.
# 3. Load BMC target info from config file.
# 4. Insert BMC iso for the target through self.execute
#
##############################################################################
if __name__ == "__main__":
args = parse_arguments()
# get debug level
debug = args.debug
# get subcloud name
subcloud_name = args.subcloud_name
# get config file
config_file = args.config_file
# RVMC PID file
rvmc_pid_file = os.path.join(
RVMC_PID_FILE_PATH, subcloud_name + RVMC_PID_FILENAME_POSTFIX)
# Set logging utility and exit handler
logging_util = rvmc.LoggingUtil(debug_level=debug)
exit_handler = ExitHandler(rvmc_pid_file)
logging_util.ilog("%s version %d.%d\n" %
(FEATURE_NAME, VERSION_MAJOR, VERSION_MINOR))
# Register the signal handler
for sig in EXIT_SIGNALS:
signal.signal(sig, signal_handler)
config, target_object = rvmc.parse_config_file(
subcloud_name, config_file, logging_util, exit_handler)
if target_object:
prepare_execution(rvmc_pid_file)
# TODO(lzhu1): support --timeout <value> option
script_timeout = eventlet.timeout.Timeout(
int(os.environ.get('RVMC_SCRIPT_TIMEOUT', 1800)))
try:
# Load the Iso for the target
logging_util.ilog("BMC Target : %s" % target_object.target)
logging_util.ilog("BMC IP Addr : %s" % target_object.ip)
logging_util.ilog("Host Image : %s" % target_object.img)
target_object.execute()
except eventlet.timeout.Timeout as e:
if e is not script_timeout:
raise
logging_util.elog("RVMC script execution timed out.")
exit_handler.exit(2)
except Exception as e:
logging_util.elog("Got exception: %s" % e)
exit_handler.exit(1)
finally:
script_timeout.cancel()
else:
logging_util.elog("Operation aborted ; no valid bmc information found")
if config_file and config:
logging_util.ilog("Config File :\n%s" % config)
exit_handler.exit(1)
exit_handler.exit(0)

1
distributedcloud/test-requirements.txt
View File

@ -7,6 +7,7 @@ coverage!=4.4,>=4.0 # Apache-2.0
fixtures>=3.0.0 # Apache-2.0/BSD
mock>=2.0 # BSD
python-subunit>=0.0.18 # Apache-2.0/BSD
redfish # BSD
requests-mock>=1.1 # Apache-2.0
testrepository>=0.0.18 # Apache-2.0/BSD
testtools>=1.4.0 # MIT

5
python/python3-redfish/debian/deb_folder/changelog Normal file
View File

@ -0,0 +1,5 @@
redfish (3.2.1) unstable; urgency=medium
* Initial release.
-- Li Zhu <li.zhu@windriver.com> Fri, 20 Oct 2023 10:14:40 -0300

18
python/python3-redfish/debian/deb_folder/control Normal file
View File

@ -0,0 +1,18 @@
Source: redfish
Section: admin
Priority: optional
Maintainer: StarlingX Developers <starlingx-discuss@lists.starlingx.io>
Build-Depends: debhelper-compat (= 13),
dh-python,
flake8,
python3-setuptools,
python3-all,
python3-jsonpath-rw,
python3-requests-unixsocket
Standards-Version: 4.5.1
Package: python3-redfish
Architecture: all
Depends: ${python3:Depends}, ${misc:Depends}
Description: Python library for Redfish operations
This package provides the Redfish Python library.

48
python/python3-redfish/debian/deb_folder/copyright Normal file
View File

@ -0,0 +1,48 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: python3-redfish
Source: https://opendev.org/starlingx/distcloud/python/python3-redfish
Files: *
Copyright: Copyright 2016-2022 DMTF. All rights reserved.
License: BSD-3-clause
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of Distributed Management Task Force (DMTF) nor the names
of its contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
Files: debian/*
Copyright: (c) 2023 Wind River Systems, Inc
License: Apache-2
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
https://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian-based systems the full text of the Apache version 2.0 license
can be found in `/usr/share/common-licenses/Apache-2.0'.

17
python/python3-redfish/debian/deb_folder/rules Executable file
View File

@ -0,0 +1,17 @@
#!/usr/bin/make -f
export DH_VERBOSE=1
export PYBUILD_NAME=python3-redfish
%:
dh $@ --with python3 --buildsystem=pybuild
override_dh_auto_clean:
python3 setup.py clean -a
find . -name \*.pyc -exec rm {} \;
override_dh_auto_build:
python3 setup.py build --force
override_dh_auto_install:
python3 setup.py install --force --root=debian/python3-redfish --no-compile -O0 --install-layout=deb

14
python/python3-redfish/debian/meta_data.yaml Normal file
View File

@ -0,0 +1,14 @@
---
debname: redfish
debver: 3.2.1
dl_path:
name: redfish-3.2.1.tar.gz
url: >
https://files.pythonhosted.org/packages/ea/04/
0e08ec3ad8af6de811edc864c7f3c505e22af14a7da735c45c512dfe9e9c/
redfish-3.2.1.tar.gz
md5sum: 8203bb777f2559e65302e988eee9f1cc
sha256sum: c10b11ff5a4e6cab0888b577c51facae689f4c6002b7ce34b1741707290aca78
revision:
dist: $STX_DIST
PKG_GITREVCOUNT: true

11
tox.ini
View File

@ -51,10 +51,19 @@ commands =
rm -rf api-ref/build
sphinx-build -W -b html -d api-ref/build/doctrees api-ref/source api-ref/build/html
[bandit]
# The following bandit tests are being skipped:
# B605: Test for starting a process with a shell
#
# Note: 'skips' entry cannot be split across multiple lines
#
skips = B605
exclude = tests
[testenv:bandit]
description = Bandit code scan for *.py files under config folder
deps = bandit
commands = bandit -r {toxinidir}/ -x '**/.tox/**,**/.eggs/**' -lll
commands = bandit --ini tox.ini -r {toxinidir}/ -x '**/.tox/**,**/.eggs/**' -lll
[testenv:linters]
allowlist_externals = bash