368 Commits

Author SHA1 Message Date
Shilpa Devharakar
596b3d142e Reorganize masakari documentation
Updated operator guide documentation for
1. segregated sections appropriately
2. referred masakariclient for CLI section
3. added detailed 'Install and configure for Ubuntu' section
4. added detailed 'Verify operation' section
5. used sphinx-build instead of the pbr sphinx extention for building docs
   as instructed by the PTI[1]
6. cleaned up build_sphinx related entries in setup.cfg

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html#documentation

Change-Id: Iec93bebdbc7ffe1ccd27bb11f474384ff06a559c
2019-10-11 01:32:08 +00:00
Zuul
a61319ecea Merge "Support noauth2 auth_strategy" 2019-10-08 04:29:33 +00:00
Zuul
127f403576 Merge "Cleanup instances test data" 2019-10-08 04:26:37 +00:00
Arthur Dayne
3bc725eb7a Support noauth2 auth_strategy
Currently, noauth2 auth_strategy is not supported. If you specify
[DEFAULT]auth_strategy=noauth2 in `/etc/masakari/ masakari.conf`
and start the masakari api service, it gives ImportError.

Added NoAuthMiddleware to support noauth2 auth_strategy.

Closes-Bug: #1836354
Change-Id: I288cd5838b09d3e6c31b6408fff85e6bb5f529a6
2019-10-03 04:42:03 +00:00
3a6f76b562 Update master for stable/train
Add file to the reno documentation build to show release notes for
stable/train.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/train.

Change-Id: I1a3bd45beb70175765afc8082729047e3f35ef3b
Sem-Ver: feature
2019-09-27 17:44:10 +00:00
Shilpa Devharakar
edf41d1833 Cleanup instances test data
Instances are not cleanup properly after functional tests finished
execution. This patch addresses this issue.

Change-Id: I77456ef967addb42c31033f4e08adb2b868a7323
Closes-bug: #1838418
2019-09-06 18:10:00 +05:30
Shilpa Devharakar
1b861d8b56 Notification functional tests fails randomly
During notification functional tests, get_notification call fails
randomly with KeyError during progress details retrieval, this patch
fixes this issue.

Change-Id: I89f47232eb6d106217717155441ba5bc1dfbaf98
Closes-bug: #1838417
2019-09-06 18:00:01 +05:30
Shilpa Devharakar
24f678cd29 Remove references of novaclient.exceptions.EndpointNotFound
Removing references 'EndpointNotFound' since removed from novaclient [1]

[1]: https://review.opendev.org/#/c/667762/

Change-Id: I02f96ed851af28c8458db63663d0231ee208a38e
2019-09-06 17:52:40 +05:30
Corey Bryant
85fa0c0058 Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Co-Authored-By: gujin <gu.jin@99cloud.net>
Change-Id: Ib98f78131050be2c38a0128d2e2a792639c03c1c
Story: #2005924
Task: #34221
2019-08-26 14:08:28 +05:30
Shilpa Devharakar
118381fcb4 add libpq-dev and python-dev to build psycopg2
This patch adds build prerequisites for psycopg2-2.8.3 as per [1]

[1]: http://initd.org/psycopg/docs/install.html

Change-Id: I591776f5e356152655b9c453644a79b397559f4f
2019-08-26 13:36:36 +05:30
Zuul
89cc782a58 Merge "Update api-ref location" 2019-08-26 06:15:23 +00:00
caoyuan
50938d0636 Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I3976f7c19b4813a40a800cea58d3dc8e343e4ac8
2019-08-11 15:00:14 +00:00
Andreas Jaeger
7f16e79c49 Update api-ref location
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.

Note that redirects will be set up as well but let's point now to the
new location.

For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html

Change-Id: Ib3bf99b1fcb81d70aae3125734b3fe21e6b1aa5b
2019-08-01 16:43:01 +02:00
Shilpa Devharakar
9a2ce34e64 Resolve functional gate job failure
With switch to resource layer in openstacksdk [1], a wrong hypervisor
attribute 'hypervisor_hostname' disappeared. Used a proper 'name'
attribute instead.

[1]: 1044776140

Change-Id: I69bcd33e76d4847229abcf55f659d7218fa4c6c2
2019-07-30 17:48:16 +05:30
Shilpa Devharakar
204fa9c849 Resolve gate job failure
This patch resolves below gate job issues:
* openstack-tox-py27: Failing for double word hacking test, resolved
  by taking reference for nova fixes [1]
* masakari-functional-devstack-multinode: Failing since 'Split
  OpenStackCloud at SDK [1]', resolved by calling proper SDK connection.
* Devstack installation add more than one networks if neutron service is
  enabled [3] and hence server creation fails with error 'ERROR
  (Conflict): Multiple possible networks. Fixed this issue by passing
  the network during server creation.

[1]: f545a25cc4
[2]: 1a4ed826b9 (diff-a948cab0aca56c2f4bf6569f8dc8ac66)
[3]: 8f7216290a

Change-Id: Ie477ddf4516712f8fd145ec78ca2335762972053
2019-07-18 13:07:02 +00:00
OpenDev Sysadmins
e8a18eee83 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:36:34 +00:00
Sean McGinnis
0e100304a7
Switch from oslosphinx to openstackdocstheme
The oslosphinx package is not compatible with the current Sphinx 2.0
release. Migration has been happening to openstackdocstheme for some
time. To fix build failures for docs and releasenotes jobs, this
switches masakari jobs to use openstackdocstheme.

Uncapping jsonschema to pass requirements check so it matches global
requirements.

Change-Id: Ic74e8b075aa67d410617dac3346bf1c903995da7
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-04-10 09:07:04 -05:00
Ian Wienand
d067deaa61 Merge "Replace openstack.org git:// URLs with https://" 2019-04-10 00:42:35 +00:00
Ian Wienand
e2f9a3df2b Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I4596e5c35143c0e7757cdf2acabdd6eac98159ea
2019-03-24 20:34:10 +00:00
d73a88a9c4 Update master for stable/stein
Add file to the reno documentation build to show release notes for
stable/stein.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/stein.

Change-Id: Ic8a9836acad68b95652b42d004d623337569ab3d
Sem-Ver: feature
2019-03-22 16:45:15 +00:00
Zuul
bc93a673f8 Merge "Migrate multinode jobs to Ubuntu Bionic" 2019-03-22 10:35:32 +00:00
openstack
2e57fa0198 Add devstack support for masakari-monitors
Added devstack support to install and run below masakari-monitors
services:
1. instancemonitor
2. processmonitor
3. introspectiveinstancemonitor

Change-Id: If4c793e316f98f14748300fc0d9003faffefeea3
2019-03-22 04:41:41 +00:00
Zuul
d7d785ba96 Merge "Functional tests to check recovery workflow details" 7.0.0.0rc1 7.0.0 2019-03-22 04:07:39 +00:00
shilpa
5e037db22e Updated rh host workflow for recovery workflow details
After merge of [1] host recovery for reserved host is getting failed
as it attempts to persists Host object.

This patch addressed this issue by sending list of names of reserved host
instead of host object to recovery flow.

[1]: https://review.openstack.org/#/c/640755/

Change-Id: Ifd008b89ff639e2e3bd8229830b5a20dced1c31b
Closes-Bug: #1819578
2019-03-21 10:45:29 +05:30
Zuul
ef378b1c04 Merge "Improve logging error messages" 2019-03-19 04:44:54 +00:00
Zuul
2b0b637a1e Merge "fix oslo-config-generator conf" 2019-03-14 13:41:55 +00:00
shilpa
f2343ef09c Handle KeyError: 'progress_details'
If user tries to get notification details before masakari-engine taskflow
driver add progress details of the first task of the recovery workflow,
it's giving Keyerror: 'progress_details'.

Fixed the issue by checking progress_details is available in the meta.
If it's not present, then get notification details will return empty list
of recovery_workflow_details.

Change-Id: I79af240d7715718b424695253ee452dc9552607d
Closes-Bug: #1819422
2019-03-13 14:50:00 +05:30
ghanshyam
5f9f9bab0a Migrate multinode jobs to Ubuntu Bionic
We have migrated the zuulv3 job to Bionic during Dec/Jan month.
 - http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.html
 - https://etherpad.openstack.org/p/devstack-bionic
But that effort does not move all gate job to Bionic as there are
large amount of jobs are still legacy jobs. All the legacy jobs still
use Xenial as nodeset.

As per the decided runtime for Stein, we need to test everything on openstack
CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html

Below patch move the legacy base jobs to bionic which will move the derived jobs
automatically to bionic. These jobs are modified with branch variant so that they will use
Bionic node from stein onwards and xenial for all other stable branches
until stable/rocky.
- https://review.openstack.org/#/c/639096

This commit modify the nodeset of multinode job to bionic

More Details: 
- https://etherpad.openstack.org/p/legacy-job-bionic
- http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003614.html

Depends-On: https://review.openstack.org/#/c/639096
Change-Id: I4c7398a3f9b6f3f948007e4ae2467598f0c29818
2019-03-13 04:23:08 +00:00
binhong.hua
c22c1d8081 fix oslo-config-generator conf
1) oslo.config should not in conf
2) namespace masakari.api and masakari.engine not exist in entry_point

Change-Id: Iad749cd8467f67a1aef64fb35a654ce221b172b4
2019-03-12 15:53:17 +08:00
tpatil
ab45ee1538 Functional tests to check recovery workflow details
Added functional tests for microversion 1.1 in which
notification returns recovery workflow details.

Change-Id: I6db5b1623672a2b9cba27277ddb7fb0e2534764b
2019-03-12 05:27:58 +00:00
Zuul
41814ab45d Merge "Configure taskflow connection" 2019-03-09 09:17:02 +00:00
jayashri bidwe
5a86d3090d Functional tests for notification APIs
This patch adds functional tests to check the behaviour
of notification APIs for notification types:-

a) Process
b) VM

Change-Id: I06b17e388773db5fb9ade31d70ee4b2ad8aefabf
2019-03-09 16:29:56 +09:00
tpatil
76a9242929 Configure taskflow connection
In patch [1], a new config option `connection` is introduced
in a new section `taskflow`  to save the recovery workflow
details of the notification.

This patch configures this new config option during creation
of config files in devstack setup.

[1] : https://review.openstack.org/#/c/640798/

Change-Id: I2383aaefaeb522e1783e2769e78438410b8033f3
2019-03-09 16:28:44 +09:00
shilpa.devharakar
7321ee32fd Add progress details for recovery workflows
This patch uses taskflow's persistence feature to store recovery
workflows details into database.

Added a new microversion to return progress_details of the
notification in `GET /notifications/<notification_uuid>` API.

APIImpact:
GET /notifications/<notification_uuid> API includes details
of recovery workflow

Change-Id: I93c1b7d88823e02d9a02855cabb8b22c9e40a7d5
Implements: bp progress-details-recovery-workflows
2019-03-08 10:52:51 +05:30
tpatil
8e77f48d1a Improve logging error messages
There are couple of places where the logging level used
was incorrect and few of other places errors are not logged.

This patch improves logging error messages.

Change-Id: I6857a5d03c27057170739174173ab1fd8c0c51c2
2019-03-07 10:05:01 +00:00
Zuul
0cbe4a3f7c Merge "Send notifications for all API changes" 2019-03-07 09:33:07 +00:00
Zuul
8284a82c00 Merge "Recover resized instance(power_state=SHUTDOWN) with vm_state=stopped" 2019-03-06 10:58:01 +00:00
jayashri bidwe
1b04778308 Functional tests for host APIs
This patch adds functional tests to check the behaviour
of host APIs.

Change-Id: Iea2769bce4ceaea34d6d52e97c7e29cb8606ca71
2019-03-06 11:13:49 +05:30
tpatil
0fa59ab7ed Add functional CI job and tests for segments
Added multi-node CI job to run functional tests.
This patch adds functional tests to check the behaviour
of Segment APIs.

Co-Authored-By: jayashri bidwe <jayashri.bidwe@nttdata.com>

Change-Id: I787b26b29fcd9556bc6b24b75ffe8e70e9f44610
2019-03-05 12:22:07 +05:30
Kengo Takahara
0616b4bd99 Send notifications for all API changes
This patch send notifications for all API changes.
1. Segment: create, update, delete
2. Host: create, update, delete
3. Notifications: create, update

Co-Authored-By: Shilpa Devharakar <Shilpa.Devharakar@nttdata.com>

Change-Id: I02411db0b3563f6a24e164dbaf50bb30744c2fda
Implements: bp notifications-in-masakari
2019-03-05 10:46:23 +05:30
Kengo Takahara
da942db7a6 Add common methods to use for sending notification
This patch adds the methods that masakari uses when sending
notifications.

Co-Authored-By: Shilpa Devharakar <Shilpa.Devharakar@nttdata.com>

Change-Id: Iee57249b5bc2659e41589b5c6c98eb7eec0bc1b3
Partial-Implements: bp notifications-in-masakari
2019-03-03 19:13:12 +05:30
Kengo Takahara
00d11056ba Define objects used for notification
This patch adds the object definition used for notification.

Co-Authored-By: Shilpa Devharakar <Shilpa.Devharakar@nttdata.com>

Change-Id: If6e67b67fa30b41a1dffac374f8a1b395b33ada3
Partial-Implements: bp notifications-in-masakari
2019-03-03 19:12:53 +05:30
Kengo Takahara
3253b13eb9 Add necessary constants and methods for RPC notification
As first step of adding notify functionality to masakari, this patch
adds the necessary constants and methods for RPC notification.

Co-Authored-By: Shilpa Devharakar <Shilpa.Devharakar@nttdata.com>

Change-Id: Iad3f487ef14effb41484014e5426117129543f5c
Partial-Implements: bp notifications-in-masakari
2019-03-03 19:11:39 +05:30
openstack
68b9c42bef Run all jobs by default using python3
This patch implements the community wide goal to run all jobs by
default using python3.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: Ie235cbffcfc69eee96c4b34358467009ca34ef9f
2019-03-03 19:11:19 +05:30
Corey Bryant
041af276d7 Py3: Ensure wsgi headers are not bytes type
In Python 3, running the Masakari API as a wsgi application was
resulting in "TypeError: expected unicode object, value of type
bytes found". It looks like headerlist should not include bytes
type objects so let's ensure they are str objects for Python 3.

Change-Id: Ica030c540970474cd2511ffe0ba653d7bb057849
Partial-Bug: #1815657
2019-03-03 19:10:39 +05:30
Zuul
c6a79ff315 Merge "Fix parsing api version with correct service_type" 2019-02-26 04:17:31 +00:00
Zuul
dcf400ff41 Merge "Update hacking version >1.1.0" 2019-02-26 04:01:04 +00:00
Kengo Takahara
c3d12e37b0 Recover resized instance(power_state=SHUTDOWN) with vm_state=stopped
The instance which is vm_state=resized and power_state=SHUTDOWN
should not be recovered with vm_state=error. It should be recovered
with vm_state=stopped. The instances such as paused or suspended
(which are other than vm_state=active,stopped,error) are also the same.

This patch modified that masakari-engine will recover these instances
with vm_state=stopped.

Change-Id: Ibdea705f69d139834c9b2294ae11bc9344c259bd
Closes-Bug: #1782518
2019-02-25 05:30:38 +00:00
shilpa.devharakar
324e1794f9 Fix parsing api version with correct service_type
Masakari project service_type is `instance-ha` but it reads
the api_version from `OpenStack-API-Version` request header
using service_type `masakari`.

This patch fixes this issue by changing service_type from
`masakari` to `instance-ha`.

Closes-Bug: #1816941
Change-Id: Ife141f9eab438e21f02966f3d27a6f044d4aa4ac
2019-02-21 07:15:35 +00:00
tpatil
d471b824ea Update hacking version >1.1.0
Updated hacking version to >1.1.0 and <1.2.0.

Change-Id: I77ff4723fd420fc2f9afe0ca6c3d85d7ae1a28ba
2019-02-18 18:29:45 +09:00