Framework for debugging nsx-openstack setup

Purpose of this script is to build a framework which can be leveraged
to build utilities to help the on-field ops in system debugging. README
contains all the instructions on how to use it and extend the framework
by adding new hooks.

Change-Id: I7eabb3afcb1491888445297f33b55bb8d77af87b
This commit is contained in:
Akash Gangil 2015-10-24 10:30:36 -07:00
parent 699cee3f3c
commit c96d12ccab
15 changed files with 763 additions and 0 deletions

View File

@ -0,0 +1,163 @@
Admin Utility
=============
Introduction
------------
Purpose of this script is to build a framework which can be leveraged to build
utilities to help the on-field ops in system debugging.
Adding custom functions
-----------------------
Refer to the security groups example for reference implementation under,
admin/plugins/nsx_v3/resources/securitygroups.py
Adding new functions is fairly straightforward:
* Define the function under appropriate package. We use neutron callbacks to provide hooks.
So your function definition should be like,
::
def function(resource, event, trigger, **kwargs)
* Add the Resources and Operations enums if they don't exist.
::
class Operations(object):
NEUTRON_CLEAN = 'neutron_clean'
::
nsxv3_resources = {
constants.SECURITY_GROUPS: Resource(constants.SECURITY_GROUPS, ops)
}
* In resource.py, add the function to the callback registry.
::
registry.subscribe(neutron_clean_security_groups,
Resources.SECURITY_GROUPS.value,
Operations.NEUTRON_CLEAN.value)
* To test, do
::
cd python-nsxadmin/
sudo pip install -e .
nsxadmin -r <resource_name_you_added> -o <operation_you_added>
TODO
----
* Use Cliff
* Auto complete command line args.
Directory Structure
-------------------
admin/
plugins/
common/
Contains code specific to different plugin versions.
nsx_v3/
resources/
Contains modules for various resources supported by the
admin utility. These modules contains methods to perform
operations on these resources.
Installation
------------
::
sudo pip install -e .
Usage
-----
::
nsxadmin -r <resource> -o <operation>
Example
-------
::
$ nsxadmin -r security-groups -o list
==== [NSX] List Security Groups ====
Firewall Sections
+------------------------------------------------+--------------------------------------+
| display_name | id |
|------------------------------------------------+--------------------------------------|
| default - 261343f8-4f35-4e57-9cc7-6c4fc7723b72 | 91a05fbd-054a-48b6-8e60-3b5d445be8c7 |
| default - 823247b6-bdb3-47be-8bac-0d1114fc1ad7 | 78116d4a-de77-4a8f-b3e5-e76f458840ea |
| OS default section for security-groups | 10a2fc6c-29c9-4d8d-ac2c-b24aafa15c79 |
| Default Layer3 Section | e479e404-e712-4adb-879c-e432d510c056 |
+------------------------------------------------+--------------------------------------+
Firewall NS Groups
+------------------------------------------------+--------------------------------------+
| display_name | id |
|------------------------------------------------+--------------------------------------|
| NSGroup Container | c0b26e82-d49b-49f0-b68e-7449a59366e9 |
| default - 261343f8-4f35-4e57-9cc7-6c4fc7723b72 | 2e5b5ca1-f687-4556-8130-9524b313474b |
| default - 823247b6-bdb3-47be-8bac-0d1114fc1ad7 | b5cd9ae4-42b5-47a7-a1bf-9767ac62466e |
+------------------------------------------------+--------------------------------------+
==== [NEUTRON] List Security Groups ====
Security Groups
+--------+------+
| name | id |
|--------+------|
+--------+------+
$ nsxadmin -r security-groups -o list -f json
==== [NSX] List Security Groups ====
{
"Firewall Sections": [
{
"display_name": "default - 261343f8-4f35-4e57-9cc7-6c4fc7723b72",
"id": "91a05fbd-054a-48b6-8e60-3b5d445be8c7"
},
{
"display_name": "default - 823247b6-bdb3-47be-8bac-0d1114fc1ad7",
"id": "78116d4a-de77-4a8f-b3e5-e76f458840ea"
},
{
"display_name": "OS default section for security-groups",
"id": "10a2fc6c-29c9-4d8d-ac2c-b24aafa15c79"
},
{
"display_name": "Default Layer3 Section",
"id": "e479e404-e712-4adb-879c-e432d510c056"
}
]
}
{
"Firewall NS Groups": [
{
"display_name": "NSGroup Container",
"id": "c0b26e82-d49b-49f0-b68e-7449a59366e9"
},
{
"display_name": "default - 261343f8-4f35-4e57-9cc7-6c4fc7723b72",
"id": "2e5b5ca1-f687-4556-8130-9524b313474b"
},
{
"display_name": "default - 823247b6-bdb3-47be-8bac-0d1114fc1ad7",
"id": "b5cd9ae4-42b5-47a7-a1bf-9767ac62466e"
}
]
}
==== [NEUTRON] List Security Groups ====
{
"Security Groups": []
}
Help
----
::
$ nsxadmin --help

View File

@ -0,0 +1,18 @@
# Copyright 2015 VMware, Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import logging
logging.basicConfig(format='%(message)s', level=logging.INFO)
logging.getLogger('requests').setLevel(logging.WARNING)

View File

@ -0,0 +1,26 @@
# Copyright 2015 VMware, Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# NSX Plugin Constants
NSXV3_PLUGIN = 'vmware_nsx.plugin.NsxV3Plugin'
NSXV_PLUGIN = 'vmware_nsx.plugin.NsxVPlugin'
# NSXV3 Resource Constants
FIREWALL_SECTIONS = 'Firewall Sections'
FIREWALL_NSX_GROUPS = 'Firewall NS Groups'
SECURITY_GROUPS = 'security-groups'
# NSXV Resource Constants
EDGES = 'edges'
SPOOFGUARD_POLICY = 'spoofguard-policy'

View File

@ -0,0 +1,53 @@
# Copyright 2015 VMware, Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import json
import logging
from oslo_config import cfg
from tabulate import tabulate
LOG = logging.getLogger(__name__)
def output_formatter(resource_name, resources_list, attrs):
"""Method to format the output response from NSX/Neutron.
Depending on the --fmt cli option we format the output as
JSON or as a table.
"""
LOG.info('%(resource_name)s', {'resource_name': resource_name})
if not resources_list:
LOG.info('No resources found')
return ''
fmt = cfg.CONF.fmt
if fmt == 'psql':
resource_attr_values = []
for resource in resources_list:
resource_list = []
for attr in attrs:
resource_list.append(resource.get(attr))
resource_attr_values.append(resource_list)
return tabulate(resource_attr_values, attrs, tablefmt=fmt)
elif fmt == 'json':
js_output = {}
js_output[resource_name] = []
for resource in resources_list:
result = {}
for attr in attrs:
result[attr] = resource[attr]
js_output[resource_name].append(result)
return json.dumps(js_output, sort_keys=True, indent=4)

View File

@ -0,0 +1,67 @@
# Copyright 2015 VMware, Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import logging
import sys
LOG = logging.getLogger(__name__)
def output_header(func):
"""Decorator to demarcate the output of various hooks.
Based on the callback function name we add a header to the
cli output. Callback name's should follow the convention of
component_operation_it_does to leverage the decorator
"""
def func_desc(*args, **kwargs):
component = '[%s]' % func.func_name.split('_')[0].upper()
op_desc = [n.capitalize() for n in func.func_name.split('_')[1:]]
LOG.info('==== %s %s ====', component, ' '.join(op_desc))
return func(*args, **kwargs)
func_desc.__name__ = func.func_name
return func_desc
def query_yes_no(question, default="yes"):
"""Ask a yes/no question via raw_input() and return their answer.
"question" is a string that is presented to the user.
"default" is the presumed answer if the user just hits <Enter>.
It must be "yes" (the default), "no" or None (meaning
an answer is required of the user).
The "answer" return value is True for "yes" or False for "no".
"""
valid = {"yes": True, "y": True, "ye": True,
"no": False, "n": False}
if default is None:
prompt = " [y/n] "
elif default == "yes":
prompt = " [Y/n] "
elif default == "no":
prompt = " [y/N] "
else:
raise ValueError("invalid default answer: '%s'" % default)
while True:
sys.stdout.write(question + prompt)
choice = raw_input().lower()
if default is not None and choice == '':
return valid[default]
elif choice in valid:
return valid[choice]
else:
sys.stdout.write("Please respond with 'yes' or 'no' "
"(or 'y' or 'n').\n")

View File

@ -0,0 +1,155 @@
# Copyright 2015 VMware, Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import logging
from admin.plugins.common import constants
from admin.plugins.common import formatters
from admin.plugins.common.utils import output_header
from admin.plugins.common.utils import query_yes_no
from admin.shell import Operations
from neutron.callbacks import registry
from neutron import context as neutron_context
from neutron.db import common_db_mixin as common_db
from neutron.db import securitygroups_db as sg_db
from vmware_nsx.nsxlib.v3 import dfw_api as firewall
LOG = logging.getLogger(__name__)
class NeutronSecurityGroupApi(sg_db.SecurityGroupDbMixin,
common_db.CommonDbMixin):
def __init__(self):
self.sg_api = super(NeutronSecurityGroupApi, self)
self.neutron_admin_context = neutron_context.get_admin_context()
def get_security_groups(self):
self.sg_api.get_security_groups(self.neutron_admin_context)
def delete_security_group(self, sg_id):
self.sg_api.delete_security_group(self.neutron_admin_context,
sg_id)
neutron_sg = NeutronSecurityGroupApi()
@output_header
def nsx_list_security_groups(resource, event, trigger, **kwargs):
sections = firewall.list_sections()
LOG.info(formatters.output_formatter(constants.FIREWALL_SECTIONS,
sections, ['display_name', 'id']))
nsgroups = firewall.list_nsgroups()
LOG.info(formatters.output_formatter(constants.FIREWALL_NSX_GROUPS,
nsgroups, ['display_name', 'id']))
return bool(sections) or bool(nsgroups)
@output_header
def nsx_delete_security_groups(resource, event, trigger, **kwargs):
if kwargs['force'] is False:
if nsx_list_security_groups(resource, event, trigger, **kwargs):
user_confirm = query_yes_no('Do you want to delete the following '
'NSX firewall sections/nsgroups?',
default='no')
if user_confirm is False:
LOG.info('NSX security groups cleanup aborted by user')
return
sections = firewall.list_sections()
# NOTE(gangila): We use -1 indexing because we trying to delete default
# security group on NSX Manager raises an exception.
if sections:
NON_DEFAULT_SECURITY_GROUPS = -1
for section in sections[:NON_DEFAULT_SECURITY_GROUPS]:
LOG.info("Deleting firewall section %(display_name)s, "
"section id %(id)s",
{'display_name': section['display_name'],
'id': section['id']})
firewall.delete_section(section['id'])
nsgroups = firewall.list_nsgroups()
if nsgroups:
for nsgroup in nsgroups:
LOG.info("Deleting ns-group %(display_name)s, "
"ns-group id %(id)s",
{'display_name': nsgroup['display_name'],
'id': nsgroup['id']})
firewall.delete_nsgroup(nsgroup['id'])
@output_header
def neutron_list_security_groups(resource, event, trigger, **kwargs):
security_groups = neutron_sg.get_security_groups()
LOG.info(formatters.output_formatter(constants.SECURITY_GROUPS,
security_groups, ['name', 'id']))
return bool(security_groups)
@output_header
def neutron_delete_security_groups(resource, event, trigger, **kwargs):
if kwargs['force'] is False:
if neutron_list_security_groups(resource, event, trigger, **kwargs):
user_confirm = query_yes_no('Do you want to delete the followin '
'neutron security groups?',
default='no')
if user_confirm is False:
LOG.info('Neutron security groups cleanup aborted by user')
return
security_groups = neutron_sg.get_security_groups()
if not security_groups:
return
for security_group in security_groups:
try:
LOG.info('Trying to delete %(sg_id)s',
{'sg_id': security_group['id']})
neutron_sg.delete_security_group(security_group['id'])
LOG.info("Deleted security group name: %(name)s id: %(id)s",
{'name': security_group['name'],
'id': security_group['id']})
except Exception as e:
LOG.warning(str(e))
registry.subscribe(nsx_list_security_groups,
constants.SECURITY_GROUPS,
Operations.LIST.value)
registry.subscribe(nsx_list_security_groups,
constants.SECURITY_GROUPS,
Operations.NSX_LIST.value)
registry.subscribe(neutron_list_security_groups,
constants.SECURITY_GROUPS,
Operations.LIST.value)
registry.subscribe(neutron_list_security_groups,
constants.SECURITY_GROUPS,
Operations.NEUTRON_LIST.value)
registry.subscribe(nsx_delete_security_groups,
constants.SECURITY_GROUPS,
Operations.CLEAN.value)
registry.subscribe(nsx_delete_security_groups,
constants.SECURITY_GROUPS,
Operations.NSX_CLEAN.value)
registry.subscribe(neutron_delete_security_groups,
constants.SECURITY_GROUPS,
Operations.CLEAN.value)
registry.subscribe(neutron_delete_security_groups,
constants.SECURITY_GROUPS,
Operations.NEUTRON_CLEAN.value)

View File

@ -0,0 +1,203 @@
# Copyright 2015 VMware, Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Purpose of this script is to build a framework which can be leveraged
to build utilities to help the on-field ops in system debugging.
TODO: Use Cliff https://pypi.python.org/pypi/cliff
TODO: Define commands instead of -r -o like get-security-groups,
delete-security-groups, nsx neutron nsxv3 can be options
TODO: Add support for other resources, ports, logical switches etc.
TODO: Autocomplete command line args
TODO: Error handling, print only options which are supported
"""
from enum import Enum
import glob
import importlib
import logging
import os
from os.path import basename
import requests
import sys
from neutron.callbacks import registry
from neutron.common import config as neutron_config
from vmware_nsx.common import config # noqa
from oslo_config import cfg
from oslo_log import _options
from admin.plugins.common import constants
from admin import version
# Suppress the Insecure request warning
requests.packages.urllib3.disable_warnings()
LOG = logging.getLogger(__name__)
class Operations(Enum):
LIST = 'list'
CLEAN = 'clean'
NEUTRON_LIST = 'neutron_list'
NEUTRON_CLEAN = 'neutron_clean'
NSX_LIST = 'nsx_list'
NSX_CLEAN = 'nsx_clean'
ops = [op.value for op in Operations]
class Resource(object):
def __init__(self, name, ops):
self.name = name
self.supported_ops = ops
# Add supported NSX-V3 resources in this dictionary
nsxv3_resources = {
constants.SECURITY_GROUPS: Resource(constants.SECURITY_GROUPS, ops)
}
# Add supported NSX-V resources in this dictionary
nsxv_resources = {
constants.EDGES: Resource(constants.EDGES, [Operations.LIST.name,
Operations.CLEAN.name]),
constants.SPOOFGUARD_POLICY: Resource(constants.SPOOFGUARD_POLICY,
[Operations.LIST.name])
}
nsxv3_resources_names = map(lambda res: res.name, nsxv3_resources.itervalues())
nsxv_resources_names = map(lambda res: res.name, nsxv_resources.itervalues())
def _get_plugin():
plugin = cfg.CONF.core_plugin
plugin_name = ''
if plugin == constants.NSXV3_PLUGIN:
plugin_name = 'nsxv3'
elif plugin == constants.NSXV_PLUGIN:
plugin_name = 'nsxv'
return plugin_name
def _get_plugin_dir():
return 'admin/plugins/{}/resources'.format(_get_plugin())
def _get_resources():
modules = glob.glob(_get_plugin_dir() + "/*.py")
return map(lambda module: os.path.splitext(basename(module))[0], modules)
cli_opts = [cfg.StrOpt('neutron-conf',
default='/etc/neutron/neutron.conf',
help='Neutron configuration file'),
cfg.StrOpt('nsx-conf',
default='/etc/neutron/plugins/vmware/nsx.ini',
help='NSX configuration file'),
cfg.StrOpt('fmt',
short='f',
default='psql',
choices=['psql', 'json'],
help='Supported output formats: json, psql'),
cfg.StrOpt('resource',
short='r',
choices=nsxv_resources_names + nsxv3_resources_names,
help='Supported list of resources: NSX-V3: %s '
'NSX-V: %s' % (', '.join(nsxv3_resources_names),
', '.join(nsxv_resources_names))),
cfg.StrOpt('operation',
short='o',
choices=ops,
help='Supported list of operations: {}'
.format(', '.join(ops))),
cfg.BoolOpt('force',
default=False,
help='Enables \'force\' mode. No confirmations will '
'be made before deletions.')
]
def _init_resource_plugin():
resources = _get_resources()
for resource in resources:
if resource != '__init__':
importlib.import_module("." + resource,
_get_plugin_dir().replace("/", "."))
def _init_cfg():
cfg.CONF.register_cli_opts(cli_opts)
# NOTE(gangila): neutron.common.config registers some options by default
# which are then shown in the help message. We don't need them
# so we unregister these options
cfg.CONF.unregister_opts(_options.common_cli_opts)
cfg.CONF.unregister_opts(_options.logging_cli_opts)
cfg.CONF.unregister_opts(neutron_config.core_cli_opts)
cfg.CONF(args=sys.argv[1:], project='NSX',
prog='Admin Utility',
version=version.__version__,
usage='nsxadmin -r <resources> -o <operation>',
default_config_files=[cfg.CONF.neutron_conf,
cfg.CONF.nsx_conf])
def validate_resource_choice(resource, nsx_plugin):
if nsx_plugin == 'nsxv' and resource not in nsxv_resources:
LOG.error('Supported list of NSX-V resources: %s',
nsxv_resources_names)
sys.exit(1)
elif nsx_plugin == 'nsxv3'and resource not in nsxv3_resources:
LOG.error('Supported list of NSX-V3 resources: %s',
nsxv3_resources_names)
sys.exit(1)
def validate_op_choice(choice, nsx_plugin):
if choice is None and nsx_plugin == 'nsxv':
LOG.error('Supported list of operations for the NSX-V resource %s',
nsxv_resources[cfg.CONF.resource].supported_ops)
exit(1)
elif choice is None and nsx_plugin == 'nsxv3':
LOG.error('Supported list of operations for the NSX-V resource %s',
nsxv3_resources[cfg.CONF.resource].supported_ops)
sys.exit(1)
def main(argv=sys.argv[1:]):
_init_cfg()
_init_resource_plugin()
nsx_plugin_in_use = _get_plugin()
LOG.info('NSX Plugin in use: %s', nsx_plugin_in_use)
validate_resource_choice(cfg.CONF.resource, nsx_plugin_in_use)
validate_op_choice(cfg.CONF.operation, nsx_plugin_in_use)
registry.notify(cfg.CONF.resource, cfg.CONF.operation,
'nsxadmin', force=cfg.CONF.force)
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

View File

@ -0,0 +1,15 @@
# Copyright 2015 VMware, Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
__version__ = '0.1'

View File

@ -0,0 +1,2 @@
tabulate>=0.7.5
enum>=0.4.4

View File

@ -0,0 +1,42 @@
[metadata]
name = python-nsxadmin
summary = CLI and Client Library for VMware-NSX
description-file =
README.rst
author = VMware-NSX Project
author-email = neutron-team@vmware.com
home-page = https://launchpad.net/vmware-nsx
classifier =
Environment :: OpenStack
Intended Audience :: Developers
Intended Audience :: Information Technology
Intended Audience :: System Administrators
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Programming Language :: Python :: 2.6
Programming Language :: Python :: 3
Programming Language :: Python :: 3.3
Programming Language :: Python :: 3.4
[files]
packages =
nsxadmin
[global]
setup-hooks =
pbr.hooks.setup_hook
[entry_points]
console_scripts =
nsxadmin = admin.shell:main
[build_sphinx]
all_files = 1
build-dir = doc/build
source-dir = doc/source
[wheel]
universal = 1

View File

@ -0,0 +1,19 @@
# Copyright 2015 VMware, Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import setuptools
setuptools.setup(
setup_requires=['pbr>=1.8'],
pbr=True)