starlingx
/
nfv
Code Issues Proposed changes

Ignore host audits with potentially outdated info 

There are scenarios (e.g. host lock) where the VIM and the
maintenance component process aspects of the operation in
parallel. The VIM also periodically audits the host state
from maintenance. This opens up a window where the VIM can
get outdated state information from maintenance and then
reverse an operation that is in progress.

The solution is to update the VIM to ignore any audit
responses for hosts that have recently changed state. The
host audit is a non-critical audit that runs approximately
once a minute to ensure the VIM's state matches the
maintenance state and there will be no harm in ignoring
audits that occur shortly after a state change.

As added robustness, the VIM is also updated to apply the
a NoExecute taint to locked hosts when the host is
disabled outside an actual lock operation. This will
catch any cases where the host flips between the enabled
and disabled states while it is locked (e.g. due to the
scenario above).

Change-Id: I228a72956a4057c768c7cea12b63d967233e556d
Closes-Bug: 1820105
Signed-off-by: Bart Wensley <barton.wensley@windriver.com>
This commit is contained in:
Bart Wensley 2019-03-19 13:14:18 -05:00
parent 258a61c81d
commit 940826c80c
2 changed files with 14 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
nfv/nfv-vim/nfv_vim/audits/_vim_nfvi_audits.py
View File

@ -138,7 +138,16 @@ def _audit_nfvi_hosts_callback(timer_id):
if not host.is_deleted():
deletable_host_groups.remove(host.name)
host.nfvi_host_update(nfvi_host)
if 30 <= host.elapsed_time_in_state:
# Only process the audited host information if the host has
# been in the current state for at least 30 seconds. This is
# necessary because the host state information comes from
# maintenance and the maintenance states may lag the states in
# the VIM because the VIM and maintenance process some actions
# at the same time (e.g. when a host is locked).
host.nfvi_host_update(nfvi_host)
else:
DLOG.info("Ignoring audit reply for host %s" % nfvi_host.name)
for host_name in deletable_host_groups:
host = host_table[host_name]

8
nfv/nfv-vim/nfv_vim/host_fsm/_host_tasks.py
View File

@ -247,10 +247,10 @@ class DisableHostTask(state_machine.StateTask):
task_work_list.append(NotifyInstancesHostDisabledTaskWork(self, host))
if host.host_service_configured(objects.HOST_SERVICES.CONTAINER):
# Only disable the container services if the host is being locked
# and we are not running in a single controller configuration. In
# a single controller configuration we keep the container services
# running.
if self._host.is_locking():
# (or is already locked) and we are not running in a single
# controller configuration. In a single controller configuration we
# keep the container services running.
if self._host.is_locking() or self._host.is_locked():
from nfv_vim import directors
sw_mgmt_director = directors.get_sw_mgmt_director()
if not sw_mgmt_director.single_controller: