25 Commits

Author SHA1 Message Date
Mihnea Saracin
6c3eeee22f Add a script to migrate releases from helm2 to helm3
This commit introduces a custom script that will help
migrating helm releases/installs from helm2 to helm3.

The procedure is:

# install the /helm-2to3 plugin

$ mkdir plugin
$ wget https://github.com/helm/helm-2to3/releases/download/v0.9.0/helm-2to3_0.9.0_linux_amd64.tar.gz
$ pushd plugin/
$ tar -xvzf ../helm-2to3_0.9.0_linux_amd64.tar.gz
$ mkdir bin
$ cp 2to3 bin
$ popd
$ export HELM_LINTER_PLUGIN_NO_INSTALL_HOOK=true
$ helm plugin install ./plugin

# move the helm2 config to helm3

$ helm 2to3 move config

# chose a helmv2 release to migrate:

$ helmv2-cli -- helm list -a

# migrate a helm2 release e.g deployment manager

$ ./migrate_helm_release.py deployment-manager

# check if it migrated successfully:

$ helm list -A -a

# it should not be present anymore in helm2:

helmv2-cli -- helm list -a

Change-Id: I9a3370ddc10a307ba87d72837f8e284fbe2594bd
Signed-off-by: Mihnea Saracin <Mihnea.Saracin@windriver.com>
Closes-Bug: 1943834
2021-09-16 14:35:21 +00:00
Andy Ning
61dfb4c0e6 Enhance upgrade script 85 to handle empty data
Upgrade script 85-update-sc-admin-endpoint-cert.py extracts admin
endpoint certificate and private key from cert-manager. But
intermittently the certificate data returned is empty, causing an
empty /etc/ssl/private/admin-ep-cert.pem generated and outage
of many services. Eventually the upgrade is failed.

This update enhanced the script to detect invalid data returned
and retry to extract the data from cert-manager.

Change-Id: Ib2c7da9147f28bf10dcb8a053412a8f94af42353
Closes-Bug: 1916279
Signed-off-by: Andy Ning <andy.ning@windriver.com>
2021-02-25 13:12:27 -05:00
Mihnea Saracin
516369277d B&R: Fix openstack helper scripts for B&R
The helper script will be used to backup and
restore the glance images. This script was previously
deleted by https://review.opendev.org/#/c/631312/.
In order to be able to access the containerized openstack,
the tidy storage script was modified to use the configuration
data from the '/etc/openstack/clouds.yaml' file.
Also, the numpy dependency was removed because all
the numpy code was easily rewritten in plain python code.
Moreover, the v1 glance client was removed and the code
was updated accordingly.

Depends-on: https://review.opendev.org/#/c/698478/
Story: 2006770
Task: 37286
Change-Id: I2918b4220b1bd3aafde20263e6cef756ac06c6db
Signed-off-by: Mihnea Saracin <Mihnea.Saracin@windriver.com>
2020-11-04 14:53:03 +02:00
Tee Ngo
7149c60059 Add subcloud status migration script
In this commit, a new directory upgrade-scripts is added
and pre-populated with the dcmanager subcloud status
migration script. This directory is intended for all
database related scripts that are executed during data
migration phase of StarlingX upgrade.

Story: 2007403
Task: 39913
Change-Id: Iab732c75e89fc9be386d1d24f011136e3cf7d32e
Signed-off-by: Tee Ngo <Tee.Ngo@windriver.com>
2020-06-01 17:27:16 -04:00
Don Penney
320cc40de8 Add auto-versioning to starlingx/config packages
This update makes use of the PKG_GITREVCOUNT variable to auto-version
the packages in this repo.

Change-Id: I3a2c8caeb4b4647608978b1f2ccfcf0661508803
Depends-On: https://review.opendev.org/727837
Story: 2006166
Task: 39766
Signed-off-by: Don Penney <don.penney@windriver.com>
2020-05-20 14:11:21 +00:00
John Kung
4e0b2acdfe Enable duplex platform upgrades
Enable the mechanism to upgrade the platform components on
a running StarlingX system with duplex controllers.

This includes upgrade updates for:
  o generation of kubernetes join_cmd to enable the N+1 controller
    to join the cluster
  o migrate of kubernetes config
  o migrate etcd on host-swact
  o migrate of DistributedCloud dcmanager and dcorch databases

A maintenance release for stx3.x is required to upgrade to stx4.0

Tests Performed with duplex controller: AIO-DX and Standard
- system load-import
- system health-query-upgrade
- system upgrade-start
- system host-upgrade controller-0
- system host-lock/unlock host N while controller N, N+1
- system host-lock/unlock controller-0 while controller N+1
- system host-upgrade controller-1
- system host-upgrade storage
- system host-upgrade worker
- system upgrade-activate
- system upgrade-abort
- system host-downgrade
- system upgrade-complete
- verified application (e.g. stx-monitor) over upgrade

Change-Id: I4267c7b32b2e7b59b5ffdd8146288698962da1e0
Story: 2007403
Task: 39243
Task: 39244
Task: 39245
Signed-off-by: John Kung <john.kung@windriver.com>
2020-04-20 15:20:19 -04:00
Carmen Rata
01c8b191d1 Config updates for stx3.0 upgrades
Update controllerconfig to remove non-platform openstack components
and fix db barbican migration.
Create RPC call to allow to touch /etc/platform/.upgrade_controller_1
Remove not needed upgrade-scripts.
Obsolete software version related fixes.

Story: 2007403
Task: 39086
Task: 39087
Task: 39182
Task: 39183
Task: 39226

Change-Id: I28e746f3d267c322f59402beaf25c271138a124d
Signed-off-by: Carmen Rata <carmen.rata@windriver.com>
2020-04-01 23:12:30 -04:00
Angie Wang
f5e43997e3 Clean up dead code in controllerconfig
This commit updates to clean up the obsolete code in
controllerconfig as a result of switch to Ansible.

The cleanup includes:
  - the whole config_controller with its relevant files
    (ie..initial configuration, backup/restore, image clone...)
  - config_subcloud
  - config_region

Tests conducted:
 - fresh installation of AIO-SX, AIO-DX, standard
   and storage labs
 - platform sanity performed on AIO-SX, AIO-DX,
   standard and storage labs
 - stx-openstack app apply/remove performed on AIO-SX
   and AIO-DX
 - stx-monitor app apply/remove on standard and
   storage labs
 - backup&restore performed on standard lab
 - distributed cloud was deployed

Change-Id: I0d1eb91188dfb24dca68a19a497b9e4edb0b0ef0
Partial-Bug: 1834218
Depends-On: https://review.opendev.org/#/c/703517/
Signed-off-by: Angie Wang <angie.wang@windriver.com>
2020-01-21 15:32:41 -05:00
Erich Cordoba
f212a7ee5f Fix typo in dependency package.
Misspelled "ruamal" caused a runtime failure in testing.

Story: 2005542
Task: 30712

Change-Id: I6cbc6b4e0e057754f91d12b814dd58ace4004a38
Signed-off-by: Erich Cordoba <erich.cordoba.malibran@intel.com>
2019-05-03 15:17:27 -05:00
Erich Cordoba
7f2c5e9f3f Add missing dependencies on controllerconfig spec file.
Some of the build time dependencies are missing from the list,
mainly other components from StarlingX.

Story: 2005542
Task: 30712

Change-Id: I299e2562ab0fcd692fef2e044e671de933b531a9
Signed-off-by: Erich Cordoba <erich.cordoba.malibran@intel.com>
2019-05-03 10:21:49 -05:00
Tyler Smith
43381a8748 Renaming deprecated options and updating spec requirements
- Renaming idle_timeout to connection_recycle_time since it was
  deprecated in Stein
- Explicitly including required packages in the sysinv spec file to fix
  DC

Change-Id: Ief055d26f3a1eb43b8cf144952a49e7e0f3ff939
Story: 2004765
Task: 28883
Depends-On: https://review.openstack.org/#/c/653086
Signed-off-by: Tyler Smith <tyler.smith@windriver.com>
2019-04-16 20:21:36 +00:00
Kristine Bujold
a1e2d1e183 Remove wrs-configutilities SDK Module
Remove configutilities and move what is being used in other components
to controllerconfig.

Tested with a clean install on AIO-DX and running config_controller.

With the StarlingX move to supporting pure upstream OpenStack, the
majority of the SDK Modules are related to functionality no longer
supported. The remaining SDK Modules will be moved to StarlingX
documentation.

Story: 2005275
Task: 30262

Change-Id: Ie496548dfc6efee677a501c98c227c586df0a7d6
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
2019-04-02 11:50:23 -04:00
Tao Liu
0b4f304be9 Configurable Host HTTP/HTTPS Port Binding
In order to avoid conflicts with containerized services
binding to standard HTTP (80) / HTTPS (443) port numbers,
the default port numbers are changed to 8080 and 8443.
Furthermore, CLI commands are provided to allow binding
to alternate port numbers.

List of changes:
. Add service parameters for HTTP and HTTPS port
. Configure the lighttpd ports via puppet and use port
  8008 for platform horizon
. Add http port to platform.conf for the config scripts
. Support helm repo URL update
. Add helm-toolkit plugin for location override
. Override Armada manifest location
. Add installer base URL option to pxeboot-update
  script
. Add a patching run time class to restart patch-agent
  when the port config is changed
. Add a semantic check to block port config when a
  patching operation is in progress or a host is not
  in unlocked/enabled state

CLI commands for viewing and updating port numbers are:
system service-parameter-list --service http
system service-parameter-modify lighttpd port http=8090
system service-parameter-apply  lighttpd

Tests Performed:
Non-containerized deployment installation and sanity
AIO-DX: Sanity and Nightly automated test suite
2+2 System: Sanity and Nightly automated test suite
2+4+6 System: Sanity and Nightly automated test suite

Kubernetes deployment on VBox:
AIO-SX: application apply and launch instance
AIO-DX: application apply and launch instance
2+2 System: application apply and launch instance

HTTP/HTTPS port configuration
Enable/Disable https

Story: 2004642
Task: 28592

Change-Id: I65029e0c15aaf626acb56ab71e7bbde64c7e76a8
Signed-off-by: Tao Liu <tao.liu@windriver.com>
2019-02-06 12:47:00 -06:00
Alex Kozyrev
ebc7ac888d Barbican configuration thru Puppet and SysInv.
1. Add the new barbican DB and barbican user.
2. Support DB backup/restore and upgrades for barbican.
3. Configure barbican user and password in region config.
4. Provide Barbican configuration with appropriate data via SysInv.
5. Setup Barbican thru puppet manifests.
There are three main services that need to be configured:
- Barbican API: a RESTful API for managing secrets.
- Barbican Worker: a RPC interface for Barbican API.
- Barbican Keystone Listener: a service for Keystone changes.
Also, HA Proxy and Firewall need to be updated with Barbican port (9311)
as well as Remote Logging manifest to allow Barbican log collection.

Change-Id: I6b0b0c90456627bebde2b834b339bc968100b6f9
Story: 2003108
Task: 27700
Depends-On: I2667d56a71b7d3881c03b6a5c1e5ed61d4f0b902
Signed-off-by: Alex Kozyrev <alex.kozyrev@windriver.com>
2018-12-03 15:49:37 -05:00
Wei Zhou
2b6f92da1b Support an Alternative Backup Restore Procedure
This code change is to support system backup and restore
without reinstalling storage nodes.

The config_controller command is modified as follows:

  A new option is added to system restore command
  "config_controller --restore-system <name>" with default set
  to exclude-storage-reinstall.

  config_controller --restore-system <include-storage-reinstall |
      exclude-storage-reinstall> <name>

Add a new command "tidy_storage_post_restore" to be run after
the restore. It scans Cinder/Glance and rbd backend for any
discrepancy between the two and generates a user action log file.

This code change has been tested on a virtual box as well as
hardware lab.

Story: 2004184
Task: 27672
Change-Id: I5e64fde70b977ea4bc3a5927bcbd852a393caec2
Signed-off-by: Wei Zhou <wei.zhou@windriver.com>
2018-11-08 18:36:46 -05:00
slin14
af2f75a17c remove lshell dependency
lshell is removed from system for security reason. So delete the
lshell requirement.

Partial-Bug: 1795451
Depends-On: https://review.openstack.org/610761

Change-Id: I0b279cc0bb7d3de5dc853a202f04bf71fdb8b2f3
Signed-off-by: slin14 <shuicheng.lin@intel.com>
2018-10-30 19:17:10 +08:00
Don Penney
4095ac7005 Add hooks for python wheel generation
This update adds hooks to the spec files for the following packages
to generate wheels for the python modules:
- configutilities
- controllerconfig
- cgts-client
- sysinv

Change-Id: I340db4c6516c9bf8badbdd0ecdcf61670e168aea
Story: 2003907
Task: 27524
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-10-23 10:34:33 -04:00
Kristine Bujold
7d16a8a9d9 Enhance warning for mgmt network configuration
This fix enhance the warning regarding the size of the address pool
when configuring the management network. A subnet of /29 will only
allow 8 addresses. If a LVM/Ceph image backend is added (e.g. system
storage-backend-add lvm -s cinder –confirmed, system
storage-backend-add ceph -s cinder,glance –confirmed), the command
will fail with “Address pool management has no available addresses”.
The user will now be asked to confirm this configuration.

Closes-Bug: 1796306

Change-Id: Ib33efff1e929e9754c241d6c93398be5c4d83db8
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
2018-10-05 14:33:09 -04:00
Alex Kozyrev
deab0cad95 Include reference to PTP in config_controller prompt for system date and time
Change-Id: I9db26d9bf0aa0b463480983af00418fea9f5c6f7
Story: 2002935
Task: 22922
Signed-off-by: Alex Kozyrev <alex.kozyrev@windriver.com>
2018-08-09 13:43:07 -04:00
Teresa Ho
0e68f13e27 Upgrades: Handle upgrades for qat-vf devices
This update provides a data migration script for converting the
pci-passthrough alias containing Coleto Creek qat-vf device to the
more explicit name qat-dh895xcc-vf.

Change-Id: Ia543bfc05f4f536cae7a0eb7098233d46fb7d66a
Story: 2002886
Task: 22847
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-06 09:10:09 -04:00
David Sullivan
183ac04745 AIO-DX: Controller-1 unlock failed on first attempt after upgrade
The sysinv data is not migrated to the N+1 version until after the
sysinv agent is run.
This prevents the sysinv agent from adding the UUID to the platform
conf, which prevents compute huge from setting the correct Kernel boot
arguments.

We set the UUID during the data migration so compute huge can update the
boot arguments when the node is unlocked which will prevent the extra
reboot.

Change-Id: Iecf03dd9e30b37419939f50677305066841a2fa8
Story: 2002886
Task: 22847
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-06 09:10:08 -04:00
David Sullivan
94bc1b03fd Data migration failure during upgrade
CONFIG_SYSTEM_MODE may not be present in pre-upgrade installs.
All older installs are DX so assume the value is duplex if missing.

Change-Id: Ic9f9c7b8551862225f6a563009ed930a88536af8
Story: 2002886
Task: 22847
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-06 09:10:08 -04:00
David Sullivan
81a98656bf Check version during sx upgrades
Ensure the running version matches what is expected during an
sx upgrade

Change-Id: Ice34ce26c33fc18e4b0d858a3d712c766868e442
Story: 2002886
Task: 22847
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-06 09:10:08 -04:00
Bart Wensley
4b2873a312 Allow large IP address ranges
Fix inefficient IP address validation code in config_controller so
large IP address ranges can be used. Also fix INI file validation
to prevent the configuration of an IPv6 management network without
a separate PXEBOOT network.

Change-Id: I87e1801092a35d74354b666e8ecd76cbf4d1e14c
Signed-off-by: Bart Wensley <barton.wensley@windriver.com>
2018-06-25 09:13:41 -05:00
Dean Troyer
9b95aa0a35 StarlingX open source release updates
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-05-31 07:35:52 -07:00