starlingx/metal
Code Issues Proposed changes

SysInv Decoupling: Create Inventory Service

Browse Source 
Create host inventory services (api, conductor and agent) and
python-inventoryclient.

The inventory service collects the host resources and provides a
REST API and client to expose the host resources.

Create plugin for integration with system configuration (sysinv)
service.

This is the initial inventory service infratructure commit.
Puppet configuration, SM integration and host integration with
sysinv(systemconfig) changes are pending and planned to be
delivered in future commits.

Tests Performed:
 Verify the changes are inert on config_controller installation
 and provisioning.
     Puppet and spec changes are required in order to create keystone,
     database and activate inventory services.

 Unit tests performed (when puppet configuration for keystone, database
 is applied):
     Trigger host configure_check, configure signals into
         systemconfig(sysinv).

     Verify python-inventoryclient and api service:
         Disks and related storage resources are pending.
         inventory host-cpu-list/show
         inventory host-device-list/show/modify
         inventory host-ethernetport-list/show
         inventory host-lldp-neighbor-list
         inventory host-lldp-agent-list/show
         inventory host-memory-list/show
         inventory host-node-list/show
         inventory host-port-list/show

     Tox Unit tests:
         inventory: pep8
         python-inventoryclient: py27, pep8, cover, pylint

Change-Id: I744ac0de098608c55b9356abf180cc36601cfb8d
Story: 2002950
Task: 22952
Signed-off-by: John Kung <john.kung@windriver.com>
This commit is contained in:
John Kung 2018-11-23 14:05:44 -05:00
parent a92c543fd5
commit bd998017d5
249 changed files with 40028 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
centos_iso_image.inc
View File

@ -23,3 +23,7 @@ pxe-network-installer
# platform-kickstarts
platform-kickstarts
# inventory
inventory
python-inventoryclient

2
centos_pkg_dirs
View File

@ -5,3 +5,5 @@ mtce-control
mtce-storage
installer/pxe-network-installer
kickstart
inventory
python-inventoryclient

13
inventory/PKG-INFO Normal file
View File

@ -0,0 +1,13 @@
Metadata-Version: 1.1
Name: inventory
Version: 1.0
Summary: Inventory
Home-page: https://wiki.openstack.org/wiki/StarlingX
Author: StarlingX
Author-email: starlingx-discuss@lists.starlingx.io
License: Apache-2.0
Description: Inventory Service
Platform: UNKNOWN

2
inventory/centos/build_srpm.data Normal file
View File

@ -0,0 +1,2 @@
SRC_DIR="inventory"
TIS_PATCH_VER=1

195
inventory/centos/inventory.spec Normal file
View File

@ -0,0 +1,195 @@
Summary: Inventory
Name: inventory
Version: 1.0
Release: %{tis_patch_ver}%{?_tis_dist}
License: Apache-2.0
Group: base
Packager: Wind River <info@windriver.com>
URL: unknown
Source0: %{name}-%{version}.tar.gz
BuildRequires: cgts-client
BuildRequires: python-setuptools
BuildRequires: python-jsonpatch
BuildRequires: python-keystoneauth1
BuildRequires: python-keystonemiddleware
BuildRequires: python-mock
BuildRequires: python-neutronclient
BuildRequires: python-oslo-concurrency
BuildRequires: python-oslo-config
BuildRequires: python-oslo-context
BuildRequires: python-oslo-db
BuildRequires: python-oslo-db-tests
BuildRequires: python-oslo-i18n
BuildRequires: python-oslo-log
BuildRequires: python-oslo-messaging
BuildRequires: python-oslo-middleware
BuildRequires: python-oslo-policy
BuildRequires: python-oslo-rootwrap
BuildRequires: python-oslo-serialization
BuildRequires: python-oslo-service
BuildRequires: python-oslo-utils
BuildRequires: python-oslo-versionedobjects
BuildRequires: python-oslotest
BuildRequires: python-osprofiler
BuildRequires: python-os-testr
BuildRequires: python-pbr
BuildRequires: python-pecan
BuildRequires: python-psutil
BuildRequires: python-requests
BuildRequires: python-retrying
BuildRequires: python-six
BuildRequires: python-sqlalchemy
BuildRequires: python-stevedore
BuildRequires: python-webob
BuildRequires: python-wsme
BuildRequires: systemd
BuildRequires: systemd-devel
Requires: python-pyudev
Requires: pyparted
Requires: python-ipaddr
Requires: python-paste
Requires: python-eventlet
Requires: python-futurist >= 0.11.0
Requires: python-jsonpatch
Requires: python-keystoneauth1 >= 3.1.0
Requires: python-keystonemiddleware >= 4.12.0
Requires: python-neutronclient >= 6.3.0
Requires: python-oslo-concurrency >= 3.8.0
Requires: python-oslo-config >= 2:4.0.0
Requires: python-oslo-context >= 2.14.0
Requires: python-oslo-db >= 4.24.0
Requires: python-oslo-i18n >= 2.1.0
Requires: python-oslo-log >= 3.22.0
Requires: python-oslo-messaging >= 5.24.2
Requires: python-oslo-middleware >= 3.27.0
Requires: python-oslo-policy >= 1.23.0
Requires: python-oslo-rootwrap >= 5.0.0
Requires: python-oslo-serialization >= 1.10.0
Requires: python-oslo-service >= 1.10.0
Requires: python-oslo-utils >= 3.20.0
Requires: python-oslo-versionedobjects >= 1.17.0
Requires: python-osprofiler >= 1.4.0
Requires: python-pbr
Requires: python-pecan
Requires: python-psutil
Requires: python-requests
Requires: python-retrying
Requires: python-six
Requires: python-sqlalchemy
Requires: python-stevedore >= 1.20.0
Requires: python-webob >= 1.7.1
Requires: python-wsme
%description
Inventory Service
%define local_bindir /usr/bin/
%define local_etc_goenabledd /etc/goenabled.d/
%define local_etc_inventory /etc/inventory/
%define local_etc_motdd /etc/motd.d/
%define pythonroot /usr/lib64/python2.7/site-packages
%define ocf_resourced /usr/lib/ocf/resource.d
%define local_etc_initd /etc/init.d/
%define local_etc_pmond /etc/pmon.d/
%define debug_package %{nil}
%prep
%setup
# Remove bundled egg-info
rm -rf *.egg-info
%build
echo "Start inventory build"
export PBR_VERSION=%{version}
%{__python} setup.py build
PYTHONPATH=. oslo-config-generator --config-file=inventory/config-generator.conf
%install
echo "Start inventory install"
export PBR_VERSION=%{version}
%{__python} setup.py install --root=%{buildroot} \
--install-lib=%{pythonroot} \
--prefix=/usr \
--install-data=/usr/share \
--single-version-externally-managed
install -d -m 755 %{buildroot}%{local_etc_goenabledd}
install -p -D -m 755 etc/inventory/inventory_goenabled_check.sh %{buildroot}%{local_etc_goenabledd}/inventory_goenabled_check.sh
install -d -m 755 %{buildroot}%{local_etc_inventory}
install -p -D -m 755 etc/inventory/policy.json %{buildroot}%{local_etc_inventory}/policy.json
install -d -m 755 %{buildroot}%{local_etc_motdd}
install -p -D -m 755 etc/inventory/motd-system %{buildroot}%{local_etc_motdd}/10-system-config
install -m 755 -p -D scripts/inventory-api %{buildroot}/usr/lib/ocf/resource.d/platform/inventory-api
install -m 755 -p -D scripts/inventory-conductor %{buildroot}/usr/lib/ocf/resource.d/platform/inventory-conductor
install -m 644 -p -D scripts/inventory-api.service %{buildroot}%{_unitdir}/inventory-api.service
install -m 644 -p -D scripts/inventory-conductor.service %{buildroot}%{_unitdir}/inventory-conductor.service
# TODO(jkung) activate inventory-agent with puppet integration)
# install -d -m 755 %{buildroot}%{local_etc_initd}
# install -p -D -m 755 scripts/inventory-agent-initd %{buildroot}%{local_etc_initd}/inventory-agent
# install -d -m 755 %{buildroot}%{local_etc_pmond}
# install -p -D -m 644 etc/inventory/inventory-agent-pmond.conf %{buildroot}%{local_etc_pmond}/inventory-agent-pmond.conf
# install -p -D -m 644 scripts/inventory-agent.service %{buildroot}%{_unitdir}/inventory-agent.service
# Install sql migration
install -m 644 inventory/db/sqlalchemy/migrate_repo/migrate.cfg %{buildroot}%{pythonroot}/inventory/db/sqlalchemy/migrate_repo/migrate.cfg
# install default config files
cd %{_builddir}/%{name}-%{version} && oslo-config-generator --config-file inventory/config-generator.conf --output-file %{_builddir}/%{name}-%{version}/inventory.conf.sample
# install -p -D -m 644 %{_builddir}/%{name}-%{version}/inventory.conf.sample %{buildroot}%{_sysconfdir}/inventory/inventory.conf
# TODO(jkung) activate inventory-agent
# %post
# /usr/bin/systemctl enable inventory-agent.service >/dev/null 2>&1
%clean
echo "CLEAN CALLED"
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
%doc LICENSE
%{local_bindir}/*
%{pythonroot}/%{name}
%{pythonroot}/%{name}-%{version}*.egg-info
%{local_etc_goenabledd}/*
%{local_etc_inventory}/*
%{local_etc_motdd}/*
# SM OCF Start/Stop/Monitor Scripts
%{ocf_resourced}/platform/inventory-api
%{ocf_resourced}/platform/inventory-conductor
# systemctl service files
%{_unitdir}/inventory-api.service
%{_unitdir}/inventory-conductor.service
# %{_bindir}/inventory-agent
%{_bindir}/inventory-api
%{_bindir}/inventory-conductor
%{_bindir}/inventory-dbsync
%{_bindir}/inventory-dnsmasq-lease-update
# inventory-agent files
# %{local_etc_initd}/inventory-agent
# %{local_etc_pmond}/inventory-agent-pmond.conf
# %{_unitdir}/inventory-agent.service

6
inventory/inventory/.coveragerc Normal file
View File

@ -0,0 +1,6 @@
[run]
branch = True
source = inventory
[report]
ignore_errors = True

59
inventory/inventory/.gitignore vendored Normal file
View File

@ -0,0 +1,59 @@
*.py[cod]
# C extensions
*.so
# Packages
*.egg*
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
lib
lib64
# Installer logs
pip-log.txt
# Unit test / coverage reports
cover/
.coverage*
!.coveragerc
.tox
nosetests.xml
.testrepository
.stestr
.venv
# Translations
*.mo
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
# Complexity
output/*.html
output/*/index.html
# Sphinx
doc/build
# pbr generates these
AUTHORS
ChangeLog
# Editors
*~
.*.swp
.*sw?
# Files created by releasenotes build
releasenotes/build

3
inventory/inventory/.mailmap Normal file
View File

@ -0,0 +1,3 @@
# Format is:
# <preferred e-mail> <other e-mail 1>
# <preferred e-mail> <other e-mail 2>

3
inventory/inventory/.stestr.conf Normal file
View File

@ -0,0 +1,3 @@
[DEFAULT]
test_path=./inventory/tests
top_dir=./

19
inventory/inventory/CONTRIBUTING.rst Normal file
View File

@ -0,0 +1,19 @@
If you would like to contribute to the development of StarlingX, you must
follow the steps in this page:
https://wiki.openstack.org/wiki/StarlingX/Contribution_Guidelines
If you already have a good understanding of how the system works and your
StarlingX accounts are set up, you can skip to the development workflow
section of this documentation to learn how changes to StarlingX should be
submitted for review via the Gerrit tool:
http://docs.openstack.org/infra/manual/developers.html#development-workflow
Pull requests submitted through GitHub will be ignored.
Bugs should be filed on Launchpad:
https://bugs.launchpad.net/starlingx
Storyboard:
https://storyboard.openstack.org/#!/story/2002950

4
inventory/inventory/HACKING.rst Normal file
View File

@ -0,0 +1,4 @@
inventory Style Commandments
============================
Read the OpenStack Style Commandments https://docs.openstack.org/hacking/latest/

176
inventory/inventory/LICENSE Normal file
View File

@ -0,0 +1,176 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

3
inventory/inventory/README.rst Normal file
View File

@ -0,0 +1,3 @@
Placeholder to allow setup.py to work.
Removing this requires modifying the
setup.py manifest.

2
inventory/inventory/babel.cfg Normal file
View File

@ -0,0 +1,2 @@
[python: **.py]

4
inventory/inventory/doc/requirements.txt Normal file
View File

@ -0,0 +1,4 @@
sphinx!=1.6.6,!=1.6.7,>=1.6.2 # BSD
openstackdocstheme>=1.18.1 # Apache-2.0
# releasenotes
reno>=2.5.0 # Apache-2.0

5
inventory/inventory/doc/source/admin/index.rst Normal file
View File

@ -0,0 +1,5 @@
====================
Administrators guide
====================
Administrators guide of inventory.

5
inventory/inventory/doc/source/cli/index.rst Normal file
View File

@ -0,0 +1,5 @@
================================
Command line interface reference
================================
CLI reference of inventory.

82
inventory/inventory/doc/source/conf.py Executable file
View File

@ -0,0 +1,82 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
sys.path.insert(0, os.path.abspath('../..'))
# -- General configuration ----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
'sphinx.ext.autodoc',
'openstackdocstheme',
#'sphinx.ext.intersphinx',
]
# autodoc generation is a bit aggressive and a nuisance when doing heavy
# text edit cycles.
# execute "export SPHINX_DEBUG=1" in your terminal to disable
# The suffix of source filenames.
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'inventory'
copyright = u'2018, StarlingX'
# openstackdocstheme options
repository_name = 'stx-metal'
bug_project = '22952'
bug_tag = ''
html_last_updated_fmt = '%Y-%m-%d %H:%M'
# If true, '()' will be appended to :func: etc. cross-reference text.
add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
add_module_names = True
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# -- Options for HTML output --------------------------------------------------
# The theme to use for HTML and HTML Help pages. Major themes that come with
# Sphinx are currently 'default' and 'sphinxdoc'.
# html_theme_path = ["."]
# html_theme = '_theme'
# html_static_path = ['static']
html_theme = 'starlingxdocs'
# Output file base name for HTML help builder.
htmlhelp_basename = '%sdoc' % project
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass
# [howto/manual]).
latex_documents = [
('index',
'%s.tex' % project,
u'%s Documentation' % project,
u'OpenStack Developers', 'manual'),
]
# Example configuration for intersphinx: refer to the Python standard library.
#intersphinx_mapping = {'http://docs.python.org/': None}

5
inventory/inventory/doc/source/configuration/index.rst Normal file
View File

@ -0,0 +1,5 @@
=============
Configuration
=============
Configuration of inventory.

4
inventory/inventory/doc/source/contributor/contributing.rst Normal file
View File

@ -0,0 +1,4 @@
============
Contributing
============
.. include:: ../../../CONTRIBUTING.rst

9
inventory/inventory/doc/source/contributor/index.rst Normal file
View File

@ -0,0 +1,9 @@
=========================
Contributor Documentation
=========================
.. toctree::
:maxdepth: 2
contributing

30
inventory/inventory/doc/source/index.rst Normal file
View File

@ -0,0 +1,30 @@
.. inventory documentation master file, created by
sphinx-quickstart on Tue Jul 9 22:26:36 2013.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
=========================================
Welcome to the documentation of inventory
=========================================
Contents:
.. toctree::
:maxdepth: 2
readme
install/index
library/index
contributor/index
configuration/index
cli/index
user/index
admin/index
reference/index
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

10
inventory/inventory/doc/source/install/common_configure.rst Normal file
View File

@ -0,0 +1,10 @@
2. Edit the ``/etc/inventory/inventory.conf`` file and complete the following
actions:
* In the ``[database]`` section, configure database access:
.. code-block:: ini
[database]
...
connection = mysql+pymysql://inventory:INVENTORY_DBPASS@controller/inventory

75
inventory/inventory/doc/source/install/common_prerequisites.rst Normal file
View File

@ -0,0 +1,75 @@
Prerequisites
-------------
Before you install and configure the inventory service,
you must create a database, service credentials, and API endpoints.
#. To create the database, complete these steps:
* Use the database access client to connect to the database
server as the ``root`` user:
.. code-block:: console
$ mysql -u root -p
* Create the ``inventory`` database:
.. code-block:: none
CREATE DATABASE inventory;
* Grant proper access to the ``inventory`` database:
.. code-block:: none
GRANT ALL PRIVILEGES ON inventory.* TO 'inventory'@'localhost' \
IDENTIFIED BY 'INVENTORY_DBPASS';
GRANT ALL PRIVILEGES ON inventory.* TO 'inventory'@'%' \
IDENTIFIED BY 'INVENTORY_DBPASS';
Replace ``INVENTORY_DBPASS`` with a suitable password.
* Exit the database access client.
.. code-block:: none
exit;
#. Source the ``admin`` credentials to gain access to
admin-only CLI commands:
.. code-block:: console
$ . admin-openrc
#. To create the service credentials, complete these steps:
* Create the ``inventory`` user:
.. code-block:: console
$ openstack user create --domain default --password-prompt inventory
* Add the ``admin`` role to the ``inventory`` user:
.. code-block:: console
$ openstack role add --project service --user inventory admin
* Create the inventory service entities:
.. code-block:: console
$ openstack service create --name inventory --description "inventory" inventory
#. Create the inventory service API endpoints:
.. code-block:: console
$ openstack endpoint create --region RegionOne \
inventory public http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
inventory internal http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
inventory admin http://controller:XXXX/vY/%\(tenant_id\)s

9
inventory/inventory/doc/source/install/get_started.rst Normal file
View File

@ -0,0 +1,9 @@
==========================
inventory service overview
==========================
The inventory service provides host inventory of resources on the host.
The inventory service consists of the following components:
``inventory-api`` service
Accepts and responds to end user API calls...

17
inventory/inventory/doc/source/install/index.rst Normal file
View File

@ -0,0 +1,17 @@
====================================
inventory service installation guide
====================================
.. toctree::
:maxdepth: 2
get_started.rst
install.rst
verify.rst
next-steps.rst
The inventory service (inventory) provides...
This chapter assumes a working setup of StarlingX following the
`StarlingX Installation Guide
<https://docs.starlingx.io/installation_guide/index.html>`_.

34
inventory/inventory/doc/source/install/install-obs.rst Normal file
View File

@ -0,0 +1,34 @@
.. _install-obs:
Install and configure for openSUSE and SUSE Linux Enterprise
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This section describes how to install and configure the inventory service
for openSUSE Leap 42.1 and SUSE Linux Enterprise Server 12 SP1.
.. include:: common_prerequisites.rst
Install and configure components
--------------------------------
#. Install the packages:
.. code-block:: console
# zypper --quiet --non-interactive install
.. include:: common_configure.rst
Finalize installation
---------------------
Start the inventory services and configure them to start when
the system boots:
.. code-block:: console
# systemctl enable openstack-inventory-api.service
# systemctl start openstack-inventory-api.service

33
inventory/inventory/doc/source/install/install-rdo.rst Normal file
View File

@ -0,0 +1,33 @@
.. _install-rdo:
Install and configure for Red Hat Enterprise Linux and CentOS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This section describes how to install and configure the inventory service
for Red Hat Enterprise Linux 7 and CentOS 7.
.. include:: common_prerequisites.rst
Install and configure components
--------------------------------
#. Install the packages:
.. code-block:: console
# yum install
.. include:: common_configure.rst
Finalize installation
---------------------
Start the inventory services and configure them to start when
the system boots:
.. code-block:: console
# systemctl enable openstack-inventory-api.service
# systemctl start openstack-inventory-api.service

31
inventory/inventory/doc/source/install/install-ubuntu.rst Normal file
View File

@ -0,0 +1,31 @@
.. _install-ubuntu:
Install and configure for Ubuntu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This section describes how to install and configure the inventory
service for Ubuntu 14.04 (LTS).
.. include:: common_prerequisites.rst
Install and configure components
--------------------------------
#. Install the packages:
.. code-block:: console
# apt-get update
# apt-get install
.. include:: common_configure.rst
Finalize installation
---------------------
Restart the inventory services:
.. code-block:: console
# service openstack-inventory-api restart

20
inventory/inventory/doc/source/install/install.rst Normal file
View File

@ -0,0 +1,20 @@
.. _install:
Install and configure
~~~~~~~~~~~~~~~~~~~~~
This section describes how to install and configure the
inventory service, code-named inventory, on the controller node.
This section assumes that you already have a working OpenStack
environment with at least the following components installed:
.. (add the appropriate services here and further notes)
Note that installation and configuration vary by distribution.
.. toctree::
:maxdepth: 2
install-obs.rst
install-rdo.rst
install-ubuntu.rst

9
inventory/inventory/doc/source/install/next-steps.rst Normal file
View File

@ -0,0 +1,9 @@
.. _next-steps:
Next steps
~~~~~~~~~~
Your OpenStack environment now includes the inventory service.
To add additional services, see
https://docs.openstack.org/project-install-guide/ocata/.

24
inventory/inventory/doc/source/install/verify.rst Normal file
View File

@ -0,0 +1,24 @@
.. _verify:
Verify operation
~~~~~~~~~~~~~~~~
Verify operation of the inventory service.
.. note::
Perform these commands on the controller node.
#. Source the ``admin`` project credentials to gain access to
admin-only CLI commands:
.. code-block:: console
$ . admin-openrc
#. List service components to verify successful launch and registration
of each process:
.. code-block:: console
$ openstack inventory service list

7
inventory/inventory/doc/source/library/index.rst Normal file
View File

@ -0,0 +1,7 @@
=====
Usage
=====
To use inventory in a project:
import inventory

1
inventory/inventory/doc/source/readme.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../README.rst

5
inventory/inventory/doc/source/reference/index.rst Normal file
View File

@ -0,0 +1,5 @@
==========
References
==========
References of inventory.

5
inventory/inventory/doc/source/user/index.rst Normal file
View File

@ -0,0 +1,5 @@
===========
Users guide
===========
Users guide of inventory.

20
inventory/inventory/etc/inventory/delete_load.sh Normal file
View File

@ -0,0 +1,20 @@
#!/bin/bash
# Copyright (c) 2015-2018 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# This script removes a load from a controller.
# The load version is passed in as the first variable.
: ${1?"Usage $0 VERSION"}
VERSION=$1
FEED_DIR=/www/pages/feed/rel-$VERSION
rm -f /pxeboot/pxelinux.cfg.files/*-$VERSION
rm -rf /pxeboot/rel-$VERSION
rm -f /usr/sbin/pxeboot-update-$VERSION.sh
rm -rf $FEED_DIR

9
inventory/inventory/etc/inventory/inventory-agent-pmond.conf Normal file
View File

@ -0,0 +1,9 @@
[process]
process = inventory-agent
pidfile = /var/run/inventory-agent.pid
script = /etc/init.d/inventory-agent
style = lsb ; ocf or lsb
severity = major ; minor, major, critical
restarts = 3 ; restarts before error assertion
interval = 5 ; number of seconds to wait between restarts
debounce = 20 ; number of seconds to wait before degrade clear

36
inventory/inventory/etc/inventory/inventory_goenabled_check.sh Normal file
View File

@ -0,0 +1,36 @@
#!/bin/bash
#
# Copyright (c) 2013-2018 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# Inventory "goenabled" check.
# Wait for inventory information to be posted prior to allowing goenabled.
NAME=$(basename $0)
INVENTORY_READY_FLAG=/var/run/.inventory_ready
# logfile=/var/log/platform.log
function LOG {
logger "$NAME: $*"
# echo "`date "+%FT%T"`: $NAME: $*" >> $logfile
}
count=0
while [ $count -le 45 ]; do
if [ -f $INVENTORY_READY_FLAG ]; then
LOG "Inventory is ready. Passing goenabled check."
echo "Inventory goenabled iterations PASS $count"
LOG "Inventory goenabled iterations PASS $count"
exit 0
fi
sleep 1
count=$(($count+1))
done
echo "Inventory goenabled iterations FAIL $count"
LOG "Inventory is not ready. Continue."
exit 0

10
inventory/inventory/etc/inventory/motd-system Normal file
View File

@ -0,0 +1,10 @@
#!/bin/bash
#
# Copyright (c) 2013-2018 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# update inventory MOTD if motd.system content present
[ -f /etc/inventory/motd.system ] && cat /etc/inventory/motd.system || true

5
inventory/inventory/etc/inventory/policy.json Normal file
View File

@ -0,0 +1,5 @@
{
"admin": "role:admin or role:administrator",
"admin_api": "is_admin:True",
"default": "rule:admin_api"
}

11
inventory/inventory/inventory/__init__.py Normal file
View File

@ -0,0 +1,11 @@
#
# Copyright (c) 2018 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
import pbr.version
__version__ = pbr.version.VersionInfo(
'inventory').version_string()

0
inventory/inventory/inventory/agent/__init__.py Normal file
View File

114
inventory/inventory/inventory/agent/base_manager.py Normal file
View File

@ -0,0 +1,114 @@
#
# Copyright (c) 2018 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
"""Base agent manager functionality."""
import futurist
from futurist import periodics
from futurist import rejection
import inspect
from inventory.common import exception
from inventory.common.i18n import _
from oslo_config import cfg
from oslo_log import log
LOG = log.getLogger(__name__)
class BaseAgentManager(object):
def __init__(self, host, topic):
super(BaseAgentManager, self).__init__()
if not host:
host = cfg.CONF.host
self.host = host
self.topic = topic
self._started = False
def init_host(self, admin_context=None):
"""Initialize the agent host.
:param admin_context: the admin context to pass to periodic tasks.
:raises: RuntimeError when agent is already running.
"""
if self._started:
raise RuntimeError(_('Attempt to start an already running '
'agent manager'))
rejection_func = rejection.reject_when_reached(64)
# CONF.conductor.workers_pool_size)
self._executor = futurist.GreenThreadPoolExecutor(
64, check_and_reject=rejection_func)
# JK max_workers=CONF.conductor.workers_pool_size,
"""Executor for performing tasks async."""
# Collect driver-specific periodic tasks.
# Conductor periodic tasks accept context argument,
LOG.info('Collecting periodic tasks')
self._periodic_task_callables = []
self._collect_periodic_tasks(self, (admin_context,))
self._periodic_tasks = periodics.PeriodicWorker(
self._periodic_task_callables,
executor_factory=periodics.ExistingExecutor(self._executor))
# Start periodic tasks
self._periodic_tasks_worker = self._executor.submit(
self._periodic_tasks.start, allow_empty=True)
self._periodic_tasks_worker.add_done_callback(
self._on_periodic_tasks_stop)
self._started = True
def del_host(self, deregister=True):
# Conductor deregistration fails if called on non-initialized
# agent (e.g. when rpc server is unreachable).
if not hasattr(self, 'agent'):
return
self._periodic_tasks.stop()
self._periodic_tasks.wait()
self._executor.shutdown(wait=True)
self._started = False
def _collect_periodic_tasks(self, obj, args):
"""Collect periodic tasks from a given object.
Populates self._periodic_task_callables with tuples
(callable, args, kwargs).
:param obj: object containing periodic tasks as methods
:param args: tuple with arguments to pass to every task
"""
for name, member in inspect.getmembers(obj):
if periodics.is_periodic(member):
LOG.debug('Found periodic task %(owner)s.%(member)s',
{'owner': obj.__class__.__name__,
'member': name})
self._periodic_task_callables.append((member, args, {}))
def _on_periodic_tasks_stop(self, fut):
try:
fut.result()
except Exception as exc:
LOG.critical('Periodic tasks worker has failed: %s', exc)
else:
LOG.info('Successfully shut down periodic tasks')
def _spawn_worker(self, func, *args, **kwargs):
"""Create a greenthread to run func(*args, **kwargs).
Spawns a greenthread if there are free slots in pool, otherwise raises
exception. Execution control returns immediately to the caller.
:returns: Future object.
:raises: NoFreeConductorWorker if worker pool is currently full.
"""
try:
return self._executor.submit(func, *args, **kwargs)
except futurist.RejectedSubmission:
raise exception.NoFreeConductorWorker()

369
inventory/inventory/inventory/agent/disk.py Normal file
View File

@ -0,0 +1,369 @@
#
# Copyright (c) 2013-2018 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# All Rights Reserved.
#
""" inventory idisk Utilities and helper functions."""
import os
import pyudev
import re
import subprocess
import sys
from inventory.common import constants
from inventory.common import context
from inventory.common import utils
from inventory.conductor import rpcapi as conductor_rpcapi
from oslo_log import log
LOG = log.getLogger(__name__)
class DiskOperator(object):
'''Class to encapsulate Disk operations for System Inventory'''
def __init__(self):
self.num_cpus = 0
self.num_nodes = 0
self.float_cpuset = 0
self.default_hugepage_size_kB = 0
self.total_memory_MiB = 0
self.free_memory_MiB = 0
self.total_memory_nodes_MiB = []
self.free_memory_nodes_MiB = []
self.topology = {}
def convert_range_string_to_list(self, s):
olist = []
s = s.strip()
if s:
for part in s.split(','):
if '-' in part:
a, b = part.split('-')
a, b = int(a), int(b)
olist.extend(range(a, b + 1))
else:
a = int(part)
olist.append(a)
olist.sort()
return olist
def get_rootfs_node(self):
cmdline_file = '/proc/cmdline'
device = None
with open(cmdline_file, 'r') as f:
for line in f:
for param in line.split():
params = param.split("=", 1)
if params[0] == "root":
if "UUID=" in params[1]:
key, uuid = params[1].split("=")
symlink = "/dev/disk/by-uuid/%s" % uuid
device = os.path.basename(os.readlink(symlink))
else:
device = os.path.basename(params[1])
if device is not None:
if constants.DEVICE_NAME_NVME in device:
re_line = re.compile(r'^(nvme[0-9]*n[0-9]*)')
else:
re_line = re.compile(r'^(\D*)')
match = re_line.search(device)
if match:
return os.path.join("/dev", match.group(1))
return
@utils.skip_udev_partition_probe
def get_disk_available_mib(self, device_node):
# Check that partition table format is GPT.
# Return 0 if not.
if not utils.disk_is_gpt(device_node=device_node):
LOG.debug("Format of disk node %s is not GPT." % device_node)
return 0
pvs_command = '{} {}'.format('pvs | grep -w ', device_node)
pvs_process = subprocess.Popen(pvs_command, stdout=subprocess.PIPE,
shell=True)
pvs_output = pvs_process.stdout.read()
if pvs_output:
LOG.debug("Disk %s is completely used by a PV => 0 available mib."
% device_node)
return 0
# Get sector size command.
sector_size_bytes_cmd = '{} {}'.format('blockdev --getss', device_node)
# Get total free space in sectors command.
avail_space_sectors_cmd = '{} {} {}'.format(
'sgdisk -p', device_node, "| grep \"Total free space\"")
# Get the sector size.
sector_size_bytes_process = subprocess.Popen(
sector_size_bytes_cmd, stdout=subprocess.PIPE, shell=True)
sector_size_bytes = sector_size_bytes_process.stdout.read().rstrip()
# Get the free space.
avail_space_sectors_process = subprocess.Popen(
avail_space_sectors_cmd, stdout=subprocess.PIPE, shell=True)
avail_space_sectors_output = avail_space_sectors_process.stdout.read()
avail_space_sectors = re.findall(
'\d+', avail_space_sectors_output)[0].rstrip()
# Free space in MiB.
avail_space_mib = (int(sector_size_bytes) * int(avail_space_sectors) /
(1024 ** 2))
# Keep 2 MiB for partition table.
if avail_space_mib >= 2:
avail_space_mib = avail_space_mib - 2
else:
avail_space_mib = 0
return avail_space_mib
def disk_format_gpt(self, host_uuid, idisk_dict, is_cinder_device):
disk_node = idisk_dict.get('device_path')
utils.disk_wipe(disk_node)
utils.execute('parted', disk_node, 'mklabel', 'gpt')
if is_cinder_device:
LOG.debug("Removing .node_cinder_lvm_config_complete_file")
try:
os.remove(constants.NODE_CINDER_LVM_CONFIG_COMPLETE_FILE)
except OSError:
LOG.error(".node_cinder_lvm_config_complete_file not present.")
pass
# On SX ensure wipe succeeds before DB is updated.
# Flag file is used to mark wiping in progress.
try:
os.remove(constants.DISK_WIPE_IN_PROGRESS_FLAG)
except OSError:
# it's ok if file is not present.
pass
# We need to send the updated info about the host disks back to
# the conductor.
idisk_update = self.idisk_get()
ctxt = context.get_admin_context()
rpcapi = conductor_rpcapi.ConductorAPI(
topic=conductor_rpcapi.MANAGER_TOPIC)
rpcapi.idisk_update_by_ihost(ctxt,
host_uuid,
idisk_update)
def handle_exception(self, e):
traceback = sys.exc_info()[-1]
LOG.error("%s @ %s:%s" % (
e, traceback.tb_frame.f_code.co_filename, traceback.tb_lineno))
def is_rotational(self, device_name):
"""Find out if a certain disk is rotational or not. Mostly used for
determining if disk is HDD or SSD.
"""
# Obtain the path to the rotational file for the current device.
device = device_name['DEVNAME'].split('/')[-1]
rotational_path = "/sys/block/{device}/queue/rotational"\
.format(device=device)
rotational = None
# Read file and remove trailing whitespaces.
if os.path.isfile(rotational_path):
with open(rotational_path, 'r') as rot_file:
rotational = rot_file.read()
rotational = rotational.rstrip()
return rotational
def get_device_id_wwn(self, device):
"""Determine the ID and WWN of a disk from the value of the DEVLINKS
attribute.
Note: This data is not currently being used for anything. We are
gathering this information so conductor can store for future use.
"""
# The ID and WWN default to None.
device_id = None
device_wwn = None
# If there is no DEVLINKS attribute, return None.
if 'DEVLINKS' not in device:
return device_id, device_wwn
# Extract the ID and the WWN.
LOG.debug("[DiskEnum] get_device_id_wwn: devlinks= %s" %
device['DEVLINKS'])
devlinks = device['DEVLINKS'].split()
for devlink in devlinks:
if "by-id" in devlink:
if "wwn" not in devlink:
device_id = devlink.split('/')[-1]
LOG.debug("[DiskEnum] by-id: %s id: %s" % (devlink,
device_id))
else:
device_wwn = devlink.split('/')[-1]
LOG.debug("[DiskEnum] by-wwn: %s wwn: %s" % (devlink,
device_wwn))
return device_id, device_wwn
def idisk_get(self):
"""Enumerate disk topology based on:
:param self
:returns list of disk and attributes
"""
idisk = []
context = pyudev.Context()
for device in context.list_devices(DEVTYPE='disk'):
if not utils.is_system_usable_block_device(device):
continue
if device['MAJOR'] in constants.VALID_MAJOR_LIST:
if 'ID_PATH' in device:
device_path = "/dev/disk/by-path/" + device['ID_PATH']
LOG.debug("[DiskEnum] device_path: %s ", device_path)
else:
# We should always have a udev supplied /dev/disk/by-path
# value as a matter of normal operation. We do not expect
# this to occur, thus the error.
#
# The kickstart files for the host install require the
# by-path value also to be present or the host install will
# fail. Since the installer and the runtime share the same
# kernel/udev we should not see this message on an
# installed system.
device_path = None
LOG.error("Device %s does not have an ID_PATH value "
"provided by udev" % device.device_node)
size_mib = 0
available_mib = 0
model_num = ''
serial_id = ''
# Can merge all try/except in one block but this allows
# at least attributes with no exception to be filled
try:
size_mib = utils.get_disk_capacity_mib(device.device_node)
except Exception as e:
self.handle_exception("Could not retrieve disk size - %s "
% e)
try:
available_mib = self.get_disk_available_mib(
device_node=device.device_node)
except Exception as e:
self.handle_exception(
"Could not retrieve disk %s free space" % e)
try:
# ID_MODEL received from udev is not correct for disks that
# are used entirely for LVM. LVM replaced the model ID with
# its own identifier that starts with "LVM PV".For this
# reason we will attempt to retrieve the correct model ID
# by using 2 different commands: hdparm and lsblk and
# hdparm. If one of them fails, the other one can attempt
# to retrieve the information. Else we use udev.
# try hdparm command first
hdparm_command = 'hdparm -I %s |grep Model' % (