180 Commits

Author SHA1 Message Date
Jack Ding
07d87406ee Add dcorch-identity-api-proxy ocf script
Add dcorch-identity-api-proxy ocf script as part of Distributed Keystone
for Distributed Cloud feature.

Story: 2002842
Task: 22785

Change-Id: Icd215916e27db785af145c68c72d13abdaf577ba
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-31 18:16:37 -04:00
Jack Ding
c0ab6ee9f1 Restore Gnocchi dispatcher to publisher
The review https://review.openstack.org/587417 requires
an upstream change from ceilometer that has not been
integrated into StarlingX project.

Use Gnocchi dispatcher for now until openstack ceilometer is rebased to
include the following change:
https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=83ffaffcb2cee6a0b19601bc7cefd863685601e2

Story: 2002825
Task: 22871

Change-Id: Icc71a9a179e6bc85e585a062f60d7b3dc0876233
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-31 18:16:37 -04:00
Angie Wang
ad20476a8c Intergrate gnocchi storage backend
Changes included in the patch:
- Add missing init files
- Update MANIFEST.in to include non-python files
- Add the ability to launch gnocchi api with gunicorn

Story: 2002825
Task: 22871

Change-Id: Iad1ed6e4d6cd4c4debfd8ed7845d4de69af3ea14
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-31 18:16:37 -04:00
Angie Wang
da76605405 Update heat templates and openstack-aodh for gnocchi
- Support new gnocchiclient interface in openstack-aodh
  Ported fix from openstack-aodh 5.0.0.
- Update heat templates to create gnocchi resources type of alarm
  instead of ceilometer
- Remove all the heat templates using cfn-push-stats tool to push
  metrics from instance, as heat CloudWatch API/cfn-push-stats is
  deprecated for a long time and it was removed in openstack queens.
  references:http://lists.openstack.org/pipermail/openstack-dev/2017-October/123104.html
             https://bugs.launchpad.net/heat/+bug/1743707
- updates the scaling interval to 5 minutes in heat templates as the
  granularity of the archive policy we are using in gnocchi is 5
  minutes.

Story: 2002825
Task: 22871
Depends-On: https://review.openstack.org/587279

Change-Id: I4872c67d15065c0e7a16632390488305649f8f37
Signed-off-by: Don Penney <don.penney@windriver.com>
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-31 18:16:37 -04:00
Angie Wang
71aaf7b58c Integrate gnocchi for OpenStack Telemetry
Add python-gnocchi and python-gnocchiclient.
Update in python-openstackclient:
 Remove the support of celiometer CLI extensions.
Update in python-ceilometer:
 Install ceilometer publisher instead of dispatcher.

Story: 2002825
Task: 22871
Depends-On: https://review.openstack.org/587279

Change-Id: I3b0dde2c8668f7e623bcf128a13010b26667d802
Signed-off-by: Don Penney <don.penney@windriver.com>
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-31 18:15:07 -04:00
Kam Nasim
1a7e61ed51 Distributed Keystone for Distributed Cloud: Horizon
In Distributed Cloud, Keystone is now running on each Subcloud.
Switching to Subcloud region now requires Openstack Auth to retrieve an
Unscoped token from the switched Region and reinitialize the django
session and cookie data with token data retrieved from the Subcloud.

Since Subcloud's Keystone doesn't contain the Identity endpoint for the
Central Region, there was no way to go back in Horizon from a subcloud
region to the SystemController region. We achieve this by caching the
SystemController endpoint in the Django Session at the time of login.

Story: 2002842
Task: 22785

Change-Id: I274e8b0756e0f24321a108c6c1a0a5d6178e0c7a
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-31 17:02:22 -04:00
Kam Nasim
8fbb9bb335 Add service file for dorch-identity-api-proxy
This is part of Distributed Keystone for Distributed Cloud feature.

Story: 2002842
Task: 22785

Change-Id: Ia8d2e3371e74ce8994a8c23ff926e5b6816b4f51
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-31 17:00:58 -04:00
Jack Ding
e886f12c5a Correct typo: caph should be ceph
Change-Id: Ib00bf2179eac0bdaa0e9b5a462ecb83fde005498
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-23 15:36:38 -04:00
Andy Ning
7b520d4f36 Add image download support to DC config
This is part of Distributed Cloud keystone scalability.

Signed-off-by: Andy Ning <andy.ning@windriver.com>

Change-Id: Ia39bf0b7a53d28188bb0cf9585d0b15a0ea66539
Story: 2002842
Task: 22785
Signed-off-by: Don Penney <don.penney@windriver.com>
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-23 12:34:01 -04:00
Robert Church
c1c51db08d Update upgrade code for removing Ceph Cache Tiering
Story: 2002884
Task: 22846

Change-Id: I31cf3eb7de935676790cf7e4c1b40307d7110390
Signed-off-by: Don Penney <don.penney@windriver.com>
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-23 12:34:01 -04:00
Elena Taivan
8a8aaeecde Fix poor volume creation performance from glance images
Fixed glance uploads on poor write performing systems: give other
greenthreads a chance to schedule.

Story: 2002893
Task: 22856

Change-Id: I70c192f1c72d8bb1a9f9b7a0fd2545c49c3acb46
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-23 12:34:01 -04:00
Jack Ding
7a8612dab8 Cleanup internal references
Story: 2002971
Task: 22979

Change-Id: I2e182478e993df8a2120f0181d8142839d6e7c1a
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-07-23 12:34:01 -04:00
Zuul
e7b54ab43d Merge "Remove non-voting gate job" 2018-06-29 23:26:25 +00:00
Dean Troyer
da8196a181 Remove non-voting gate job
Change-Id: I47da9309533677a85fe03867771fb28a27ad39c8
2018-06-29 14:35:23 -05:00
Zuul
3fb3dabc92 Merge "Remove Ceph Cache Tiering" 2018-06-29 16:17:41 +00:00
Zuul
9b3a811e20 Merge "AIO Reduce number of workers" 2018-06-29 16:17:40 +00:00
Robert Church
df6003ebf9 Remove Ceph Cache Tiering
Ceph Cache Tiering feature is not supported anymore. This commit removes
all the code changes associated with the no longer supported Ceph Cache
Tiering.
This implies:
a. cache tiering cannot be configured on system
b. no ceph-caching host could be added
c. no ceph-backing host could be added
d. ceph-caching/ceph-backing personality sub-type won't show up
when 'system host-show'/system host-add command is issued
e. ceph-caching/ceph-backing personality sub-type won't show up when host
is added/listed from horizon

Change-Id: Idfe9a1672d17ecb996069311e2ad9377921b3df0
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-06-28 21:57:23 -04:00
Tee Ngo
e0e3b52400 AIO Reduce number of workers
TIS_WORKERS_FACTOR for keystone service is reduced to 1 for all AIO
variants.

Tests conducted:
- Bulk VM launch/delete via heat stack create/delete - duplex (20 VMs) &
  simplex (10 VMs)
- Swact, failover, parallel live migration - duplex
- Lock/unlock with max VMs, DOR - both duplex & simplex
- Sanity, nightly sanity - both duplex & simplex
- Full nova regression - duplex
- Bulk VM launch/delete stress test - duplex (60 VMs)
- Platform CPU adjustments via system host-cpu-modify
- Ceilometer message queues verification
- Standard system installation

Change-Id: Ia455f1492ed3950ec937ad6d3a2e08b2364d9665
Signed-off-by: Jack Ding <jack.ding@windriver.com>
2018-06-28 21:56:19 -04:00
Ovidiu Poncea
ac42d3f022 Fix periodic thread that monitors Ceph
A timeout between ceph-manager and sysinv is causing ceph-manager to
stop responding. When Ceph Manager detects that 'require_jewel_osds'
flag needs to be set it queries sysinv which for whatever reason
(most likely dealing with some final operations) fails to respond in
a reasonable amount of time. This causes an exception in ceph-manager
which breaks one of the periodic threads execution.

Change-Id: Iac5a4cd171cd702d84053fba7860bea244b31f4e
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
2018-06-28 14:44:52 -04:00
Tee Ngo
bc71b1437d AIO-SX Regression: Memory usage exceed alarms appeared during test run
- Reduce both CPU and memory footprint for AIO simplex by reducing the number of workers across the board.
- In addition:
     a. Reduce ceilometer workers to 1 as the traffic generated from max number of VMs in simplex (20) does
        not warrant more workers than 1. Increase batch timeout to reduce load.
     b. Change keystone workers factor from 1.5 to 1 which further reduces the number of keystone workers
        for simplex. The number of requests do not warrrant more workers than other services.
     c. Reduce the number of nova api metadata workers to half of nova api compute workers for both AIO and
        standard configs based on LO data.
  AIO installations, simplex sanity & sanity nightly all passed. Sanity, parallel VM/volume launch timings
  also improved. Ceilometer messages were cleared timely.

Change-Id: I86455e64b48fdc543cd9e99cf95f5ddb2295f71f
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-06-26 11:35:45 -04:00
Zuul
e7dc49a0b6 Merge "Update sdk to include distributed cloud client." 2018-06-25 15:41:36 +00:00
Kevin Smith
89afe7500b Update sdk to include distributed cloud client.
Change-Id: Ieca2ef5dfab45817176442cf3e497c5e5018d311
Signed-off-by: Tyler Smith <tyler.smith@windriver.com>
2018-06-22 21:08:53 +00:00
Kam Nasim
bda6517c97 Distributed Keystone for Distributed Cloud - Endpoint filters
Bringing in the Endpoint Filter Group blueprint into Openstack
client. Endpoint filter groups have been around in Keystone since the
days of KILO but only picked up by Openstack client in v3.15.

In the absence of Endpoint filter groups, clients are responsible for
filtering the endpoint list, instead of Keystone sending a tailored
list. For e.g. in Distributed Cloud, local Openstack services in the
Central Region should not be seeing endpoints for subclouds, as they
don't directly communicate with subclouds.

As part of this blueprint we also had to bring in an important
commit that Optimizes the command "openstack endpoint list". The current
behavior is to make a Service call for each Endpoint in the Endpoint
list, so for N endpoints yo have N + 1 calls to Keystone. The new
behavior makes 1 call for Endpoint list, and 1 call for Service list and
works with the two lists... so you do down from (N+1) calls to 2

Change-Id: I9c03938c25b56d64b59ce42cae5026f2830f02b7
Signed-off-by: Tyler Smith <tyler.smith@windriver.com>
2018-06-22 14:11:33 -05:00
Scott Little
bc7abb4399 Split centos-pkg-dirs along git boundaries.
Problem:
The centos-pkg-dirs files should only reference packages with
compilation instructions hosted in the same git.

Solution:
Create centos-pkg-dirs files in other stx-* gits, and relocate
the relevant entries from the stx-utils centos-pkg-dirs into
the appropriate destination git.

Change-Id: I31c38ced2227aa494060c0ce333b569a64cc0424
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-06-20 16:26:22 -04:00
Erich Cordoba
c542488ff2 Update TIS_BASE_SRCREV missing references
Change-Id: I030482dd37ea4158f04af14f2af5e610d38ca14e
Signed-off-by: Erich Cordoba <erich.cordoba.malibran@intel.com>
2018-06-18 11:34:49 -05:00
Zuul
78ecaa97cb Merge "Open vSwitch integration with host and configuration framework" 2018-06-18 14:48:17 +00:00
Matt Peters
3a375fde71 Open vSwitch integration with host and configuration framework
Integrates the latest Open vSwitch with DPDK into the host management
and configuration framework and configures the default system
vswitch type to be ovs-dpdk.

Change-Id: I943f7668d9c94508706fa1611444898fc28ad4c7
Signed-off-by: Matt Peters <matt.peters@windriver.com>
2018-06-12 12:13:34 -05:00
Dean Troyer
24c92163e4 Add default test framework
Change-Id: I63420abc27ebd4ee33ae39112d8cbdf15e138200
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-06-12 10:26:12 -05:00
Dean Troyer
9d3ca49387 StarlingX open source release updates
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-05-31 07:37:19 -07:00
Dean Troyer
48eda4de1e Add .gitreview
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-05-31 07:37:19 -07:00