distcloud/distributedcloud/dcorch/tests
John Kung 181bd0c86d Enable multiple dcorch-engine worker processes
Multiple worker processes are created for distributing the dcorch
sync and audit work of the dcorch-engine on the active platform
controller host.

The database sync lock is introduced as a synchronization mechanism
for dcorch-engine workers and stores subcloud, endpoint
and action locks.

The subcloud sync status table is introduced to track subcloud/endpoint
sync request, audit status, and report status.  In-memory caching of
subcloud objects and tracking info such as last audit and reported sync
are removed and the dcorch DB is referenced instead.

The GenericSyncManager and sync_threads are restructured and use
thread pools for work to be performed on subclouds.  The infrastructure
allows for sync_threads to be allocated to a dcorch-engine worker.
In this commit, each worker process can handle each subcloud.

Robustness audits ensures no stale sync_lock and dcorch-engine
worker process file limits are increased.

Tests Performed with multiple subclouds:
  Bootstrap, Installation and Deployment
  Initial manage and sync
  Resource update and sync via proxy and audit
  Unmanage and manage subclouds
  Verify sync_lock on controller swact and process termination
  Dead Office Recovery

Co-Authored-By: Tao Liu <tao.liu@windriver.com>
Co-Authored-By: Teresa Ho <teresa.ho@windriver.com>

Change-Id: I179279dea5babaf57084b28adea86a03b330924a
Story: 2007267
Task: 41374
Task: 38662
Signed-off-by: John Kung <john.kung@windriver.com>
2020-12-17 17:27:46 -05:00
..
unit Enable multiple dcorch-engine worker processes 2020-12-17 17:27:46 -05:00
__init__.py Move content to subdir to support relocated packaging 2019-11-04 13:57:02 -05:00
base.py Fix subcloud manage/unmanage issues caused by identity sync 2020-02-18 07:23:08 -06:00
utils.py Move content to subdir to support relocated packaging 2019-11-04 13:57:02 -05:00