27 Commits

Author SHA1 Message Date
Eric MacDonald
44b74e4066 Implement collectd startup in manifest apply post stage
Starting collectd too early in the manifest apply is seen
to occasionally fail due to a dependency configuration on
hostname resolution in FQDNLookup not being complete.

Since influxdb is used by collectd and is a controller
only service this update moves it to the manifest apply
post stage as well and is filtered out from non
controller load types.

This issue is fixed by the following multi-git changes.

stx-metal:
   Filter influxdb out of storage and compute only loads.
   No real inter git merge dependency

stx-integ: This update.
   Add startup Before=pmond dependency

stx-config:
   Move collectd config and startup to manifest apply post stage
   Move influxdb config and startup to manifest apply post stage

Test Plan:
PASS: Build iso
PASS: verify install storage system and collectd startup
PASS: Verify Storage system DOR
PASS: Verify influxdb and extensions excluded in non-controller loads
PASS: Verify collectd starts properly on all nodes (CC,DOR,UNLOCK)
PASS: Verify influxdb starts properly on controller nodes (CC,DOR,UNLOCK)
PASS: Verify collectd pmond process monitoring and recovery
PASS: Verify influxdb pmond process monitoring and recovery

PEND: Verify collectd statistics storage and fetch to/from influxdb
PEND: Install AIO DX and verify collectd and influxdb startup

Change-Id: I47d70b05bdbdd22f8fce2f56fcc287fac7371ace
Closes-Bug: 1797909
Signed-off-by: Eric MacDonald <eric.macdonald@windriver.com>
2019-01-02 10:21:06 -05:00
Sun Austin
eb8ba2c3ab Fix: "dict" issue for Python 2/3 compatible code
Replace dict.iteritems() with dict.items()
Replace dict.iterkeys() with dict.keys()
Change dict.keys() to list(dict.keys())

Story: 2002909
Task: 24564

Change-Id: Icf5ef6cc9947a87ee8e013d1225b3f36914e6678
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-12-25 16:52:30 +08:00
Zuul
d1e9a2fe73 Merge "Change compute node to worker node personality" 2018-12-14 22:40:53 +00:00
Zuul
0716301e98 Merge "Fix collectd Memory plugin Strict Mode learning" 2018-12-13 20:22:44 +00:00
Eric MacDonald
b9b8116efd Fix collectd Memory plugin Strict Mode learning
Existing code sets overcommit strict mode to True
if any non-zero value is returned from a read
of /proc/sys/vm/overcommit_memory.

This is incorrect.

Strict mode should only be set when the returned
value is 2.

Change-Id: I2c5328624571bb3b2f478d5a79615650bb92cbd2
Closes-Bug: 1808225
Signed-off-by: Eric MacDonald <eric.macdonald@windriver.com>
2018-12-13 09:31:03 -05:00
Tao Liu
797b3fc16d Change compute node to worker node personality
The compute personality & subfunction has been changed to
worker, and compute_reserved.conf has been rename to
worker_reserved.conf. Compute configuration flags have
been updated to worker flags.

This update changes misc dependencies to compute
personality, compute_reserved.conf and configuration
flag files.

It aslo removed puppet-nova dependencies to
compute_reserved.conf.

Tests Performed:
Non-containerized deployment
AIO-SX: Sanity and Nightly automated test suite
AIO-DX: Sanity and Nightly automated test suite
2+2 System: Sanity and Nightly automated test suite
2+2 System: Horizon Patch Orchestration

Kubernetes deployment:
AIO-SX: Create, delete, reboot and rebuild instances
2+2+2 System: worker nodes are unlock enable and no alarms

Story: 2004022
Task: 27013

Depends-On: https://review.openstack.org/#/c/624452/

Change-Id: Iccf5584058a2154f1c4ffdb061938e76b9965861
Signed-off-by: Tao Liu <tao.liu@windriver.com>
2018-12-12 15:09:04 -05:00
Zuul
647cd6ea5a Merge "Refactor patches for initscripts package" 2018-11-21 19:21:22 +00:00
zhipengl
c38ea82ac6 Refactor patches for initscripts package
Use initscripts-config package to package script and service file for
initscripts package.
Refactor 0001-Disable-zeroconf-route.patch,
spec-add-mountnfs-init-script.patch and
spec-include-TiS-changes.patch, let initscripts-config to be
responsible for the installation of config/script/service files.
Merged some meta patches that just includes adding source code patch to
spec-include-Tis-changes.patch.
Removed base/procps since it just includes one file, sysctl.conf. And
move this file to initscripts-config folder.The monitor-tools package
in stx-integ has a %post script that is adding an entry to sysctl.conf,
so add "Requires: initscripts-config" in monitor-tools.spec, to ensure
it is installed after this package replaces the file.

Deployment test and ping test between VMs pass
Service, config and script files check pass.

Story: 2003768
Task: 27585

Change-Id: I2ea3bd05bdc5bca5658d157e6f40f7380e922500
Signed-off-by: zhipengl <zhipengs.liu@intel.com>
2018-11-21 01:26:32 +00:00
Zuul
437ad7b5a4 Merge "Python 3 compatibility: use print as a function." 2018-11-02 18:38:13 +00:00
zhangyangyang
9594a8e391 Python 3 compatibility: use print as a function.
In Python 3 print is a function.
Especially for multiple string print, need to import
print_function from __future__.

Story: 2002909

Task: 24560

Signed-off-by: zhangyangyang <zhangyangyang@unionpay.com>

Change-Id: Ie31eb59368af57776eb9785dba494432111cd250
2018-10-29 07:40:56 +00:00
Zuul
26f48ef96a Merge "Add hooks for python wheel generation" 2018-10-23 21:59:43 +00:00
Don Penney
48298a97a6 Add hooks for python wheel generation
This update adds hooks to the spec files for the following packages
to generate wheels for the python modules:
- ceph-manager
- libvirt-python
- logmgmt
- platform-util
- python-3parclient
- python-cephclient
- python-lefthandclient
- python-ryu
- vm-topology

Change-Id: Ia63291e686818d19d0df52ff26b5f0bb3812b8ce
Story: 2003907
Task: 26787
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-10-23 10:35:32 -04:00
Eric MacDonald
1dc6c1595f Make collectd alarm notifier retry alarm clear attempts that fail
The Starling-X collectd alarm notification handler Fault Manager (FM)
call to clear an alarm can lead to a stuck alarm if that FM request
fails, say due to a concurrent swact operation, and the clear is not
retried.

The alarm will remain stuck until there is another same alarm assertion,
followed by deassertion that leads to a successful clear.

The fix is to execute a 'return' in the alarm clear failure path so
that the alarm notifier's alarm manager control structure is not
updated with the clear state so that the clear will be automatically
retried on the next audit interval.

Change-Id: Iddf4e0e7b99eab0bf0748230a25851419e7c06fa
Closes-Bug: 1793314
Signed-off-by: Eric MacDonald <eric.macdonald@windriver.com>
2018-09-20 14:21:32 -04:00
Zuul
4c866ab0aa Merge "Merge remote-tracking branch 'origin/f/centos75'" 2018-09-20 04:44:47 +00:00
Dean Troyer
d7ecc895dc Merge remote-tracking branch 'origin/f/centos75'
Change-Id: I93512a81ac4cc3e883c680bbb8366bf6d6e78437
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-09-19 14:36:01 -05:00
Dean Troyer
c320b8668b Merge remote-tracking branch 'origin/master' into f/centos75
Change-Id: I166b99c4ce2ed458ac79ce686064ab05ad3ca198
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-09-15 01:18:37 -05:00
Zuul
f8c4583d89 Merge "Python 3 compatibility: fix xrange/range issues" 2018-09-04 17:34:35 +00:00
Dean Troyer
98bac0a0a2 Merge remote-tracking branch 'origin/master' into f/centos75-merge
Change-Id: Idf04b651ea1301745ad429a423b09d93543012e3
2018-09-02 15:42:07 -05:00
zhangyangyang
d52c7d1783 Python 3 compatibility: fix xrange/range issues
xrange is not defined in python3.
Rename xrange() to range().

Story: 2002909

Task: 24567

Change-Id: I1cf9b77339fee79fd4750ca873a952a39974c23c
Signed-off-by: zhangyangyang <zhangyangyang@unionpay.com>
2018-09-01 17:42:12 +08:00
Zuul
038a084f20 Merge "Remove the duplicated word" 2018-08-20 20:47:59 +00:00
melissaml
637e9ac132 Remove the duplicated word
Change-Id: I68dc653708a33536b69ede4f032457ab951c24dd
2018-08-17 15:34:51 +08:00
Tee Ngo
de2f57ce6b Schedtop not displaying delay stats
Enable kernel schedstats which is disabled by default since switching to CentOS.
Adapt to format change.

Change-Id: I70da51eaca0518a8bb09d4b11d6f98b00c857059
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-08-01 12:41:51 -04:00
Scott Little
1e75d19c9a Relocate monitor-tools to stx-integ/tools/monitor-tools
Move content from stx-utils into stx-integ or stx-update

Packages will be relocated to

stx-update:
    enable-dev-patch
    extras

stx-integ:
    config-files/
        io-scheduler

    filesystem/
        filesystem-scripts

    grub/
        grubby

    logging/
        logmgmt

    tools/
        collector
        monitor-tools

    tools/engtools/
        hostdata-collectors
        parsers

    utilities/
        build-info
        branding   (formerly wrs-branding)
        platform-util

Change-Id: Id181ee00015b41c50c75250384e7fbf36a300c1c
Story: 2002801
Task: 22687
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-08-01 12:28:38 -04:00
Scott Little
7ca5359f3e Internal restructuring of stx-integ
Create new directories:
   ceph
   config
   config-files
   filesystem
   kernel
   kernel/kernel-modules
   ldap
   logging
   strorage-drivers
   tools
   utilities
   virt

Retire directories:
   connectivity
   core
   devtools
   support
   extended

Delete two packages:
   tgt
   irqbalance

Relocated packages:
   base/
      dhcp
      initscripts
      libevent
      lighttpd
      linuxptp
      memcached
      net-snmp
      novnc
      ntp
      openssh
      pam
      procps
      sanlock
      shadow
      sudo
      systemd
      util-linux
      vim
      watchdog

   ceph/
      python-cephclient

   config/
      facter
      puppet-4.8.2
      puppet-modules

   filesystem/
      e2fsprogs
      nfs-utils
      nfscheck

   kernel/
      kernel-std
      kernel-rt

   kernel/kernel-modules/
      mlnx-ofa_kernel

   ldap/
      nss-pam-ldapd
      openldap

   logging/
      syslog-ng
      logrotate

   networking/
      lldpd
      iproute
      mellanox
      python-ryu
      mlx4-config

   python/
      python-2.7.5
      python-django
      python-gunicorn
      python-setuptools
      python-smartpm
      python-voluptuous

   security/
      shim-signed
      shim-unsigned
      tboot

   strorage-drivers/
      python-3parclient
      python-lefthandclient

   virt/
      cloud-init
      libvirt
      libvirt-python
      qemu

   tools/
      storage-topology
      vm-topology

   utilities/
      tis-extensions
      namespace-utils
      nova-utils
      update-motd

Change-Id: I37ade764d873c701b35eac5881eb40412ba64a86
Story: 2002801
Task: 22687
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-08-01 10:06:31 -04:00
Eric MacDonald
a960a5ff88 Recreate /var/run/influxdb dir upon recovery
This update fixes an issue where the /var/run/influxdb directory
is not being re-created over a DOR because the controller manifest that
creates it is not being run in that recovery mode.

The fix is to enhance the influxdb service file to ensure this directory
is created whenever the service is started.

Story: 2002823
Task: 22740

Change-Id: Iecd81969ae1611b963fae5595f60c3eb2d2da851
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-20 10:53:56 -04:00
Eric MacDonald
b8c0ce1903 Collectd+InfluxDb-RMON Replacement(ALL METRICS) P1
This is the primary update that introduces collectd monitoring and
sample storage into the influxdb database.
Two new packages are introduced by this update
 - collectd-extensions package which includes
   - newly developed collectd platform memory, cpu and filesystem
     plugins
     - note that the example, ntpq and interface plugins are not
       complete and are not enabled by this update.
   - pmond process monitoring / recovery support for collectd
   - updated service file for pidfile management ; needed by pmond
 - influxdb-extensions package which includes
   - pmond process monitoring / recovery support for influxdb
   - updated service file for pidfile management ; needed by pmond
   - log rotate support for influxdb

Change-Id: I06511fecb781781ed5491c926ad4b1273a1bc23b
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-03 11:06:24 -04:00
Dean Troyer
948d42b03b Add .gitreview
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-05-31 07:36:35 -07:00