x/fuel-ccp-specs
Code Issues Proposed changes

Retire repository

Browse Source 
Fuel (from openstack namespace) and fuel-ccp (in x namespace)
repositories are unused and ready to retire.

This change removes all content from the repository and adds the usual
README file to point out that the repository is retired following the
process from
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project

See also
http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011647.html

Depends-On: https://review.opendev.org/699362
Change-Id: Icb7b079c113ff7ddb658e680a5f4b07a3761b114
This commit is contained in:
Andreas Jaeger 2019-12-18 09:54:02 +01:00
parent 9524ec52ff
commit 192b54150c
35 changed files with 8 additions and 3248 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
.gitignore vendored
View File

@ -1,10 +0,0 @@
AUTHORS
ChangeLog
build
.tox
.venv
*.egg*
*.swp
*.swo
*.pyc
.testrepository

4
.testr.conf
View File

@ -1,4 +0,0 @@
[DEFAULT]
test_command=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_TEST_TIMEOUT=60 ${PYTHON:-python} -m subunit.run discover -t ./ . $LISTOPT $IDOPTION
test_id_option=--load-list $IDFILE
test_list_option=--list

3
LICENSE
View File

@ -1,3 +0,0 @@
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
http://creativecommons.org/licenses/by/3.0/legalcode

158
README.rst
View File

@ -1,152 +1,10 @@
=======
README
=======
This project is no longer maintained.
NextGen Specifications
======================
The contents of this repository are still available in the Git
source code management system. To see the contents of this
repository before it reached its end of life, please check out the
previous commit with "git checkout HEAD^1".
This git repository is used to hold approved design specifications for the
NextGen project. Reviews of the specs are done in gerrit, using a similar
workflow to how we review and merge changes to the code itself. For specific
policies around specification review, refer to the end of this document.
The layout of this repository is::
specs/<group>/
Where there are two sub-directories:
specs/<release>/approved: specifications approved but not yet implemented
specs/<release>/implemented: implemented specifications
The lifecycle of a specification
--------------------------------
TODO(slukjanov): Cleanup the readme for NextGen case.
Developers proposing a specification should propose a new file in the
``approved`` directory. nextgen-specs-core will review the change in the usual
manner for the OpenStack project, and eventually it will get merged if a
consensus is reached. At this time the Launchpad blueprint is also approved.
The developer is then free to propose code reviews to implement their
specification. These reviews should be sure to reference the Launchpad
blueprint in their commit message for tracking purposes.
Once all code for the feature is merged into Nova, the Launchpad blueprint is
marked complete. As the developer of an approved specification it is your
responsibility to mark your blueprint complete when all of the required
patches have merged.
Periodically, someone from nextgen-specs-core will move implemented specifications
from the ``approved`` directory to the ``implemented`` directory. Whilst
individual developers are welcome to propose this move for their implemented
specifications, we have generally just done this in a batch at the end of the
release cycle. It is important to create redirects when this is done so that
existing links to the approved specification are not broken. Redirects aren't
symbolic links, they are defined in a file which sphinx consumes. An example
is at ``specs/kilo/redirects``.
This directory structure allows you to see what we thought about doing,
decided to do, and actually got done. Users interested in functionality in a
given release should only refer to the ``implemented`` directory.
Example specifications
----------------------
You can find an example spec in ``specs/template.rst``.
Backlog specifications
----------------------
Additionally, we allow the proposal of specifications that do not have a
developer assigned to them. These are proposed for review in the same manner as
above, but are added to::
specs/backlog/approved
Specifications in this directory indicate the original author has either
become unavailable, or has indicated that they are not going to implement the
specification. The specifications found here are available as projects for
people looking to get involved with Nova. If you are interested in
claiming a spec, start by posting a review for the specification that moves it
from this directory to the next active release. Please set yourself as the new
`primary assignee` and maintain the original author in the `other contributors`
list.
Design documents for releases prior to Juno
-------------------------------------------
Prior to the Juno development cycle, this repository was not used for spec
reviews. Reviews prior to Juno were completed entirely through `Launchpad
blueprints <http://blueprints.launchpad.net/nova>`_
Please note, Launchpad blueprints are still used for tracking the
current status of blueprints. For more information, see
https://wiki.openstack.org/wiki/Blueprints
Working with gerrit and specification unit tests
------------------------------------------------
For more information about working with gerrit, see
http://docs.openstack.org/infra/manual/developers.html#development-workflow
To validate that the specification is syntactically correct (i.e. get more
confidence in the Jenkins result), please execute the following command::
$ tox
After running ``tox``, the documentation will be available for viewing in HTML
format in the ``doc/build/`` directory.
Specification review policies
=============================
There are a number of review policies which nextgen-specs-core will apply when
reviewing proposed specifications. They are:
Trivial specifications
----------------------
Proposed changes which are trivial (very small amounts of code) and don't
change any of our public APIs are sometimes not required to provide a
specification. In these cases a Launchpad blueprint is considered sufficient.
These proposals are approved during the `Open Discussion` portion of the
weekly nova IRC meeting. If you think your proposed feature is trivial and
meets these requirements, we recommend you bring it up for discussion there
before writing a full specification.
Previously approved specifications
----------------------------------
`Specifications are only approved for a single release`. If your specification
was previously approved but not implemented (or not completely implemented),
then you must seek re-approval for the specification. You can re-propose your
specification by doing the following:
* Copy (not move) your specification to the right directory for the current release.
* Update the document to comply with the new template.
* If there are no functional changes to the specification (only template changes) then add the `Previously-approved: <release>` tag to your commit message.
* Send for review.
* nextgen-specs-core will merge specifications which meet these requirements with a single +2.
Specifications which depend on merging code in other OpenStack projects
-----------------------------------------------------------------------
For specifications `that depend on code in other OpenStack projects merging`
we will not approve the nova specification until the code in that other project
has merged. The best example of this is Cinder and Neutron drivers. To
indicate your specification is in this state, please use the Depends-On git
commit message tag. The correct format is `Depends-On: <change id of other
work>`. nextgen-specs-core can approve the specification at any time, but it wont
merge until the code we need to land in the other project has merged as well.
New libvirt image backends
--------------------------
There are some cases where an author might propose adding a new libvirt
driver image storage backend which does not require code in other OpenStack
projects. An example was the ceph image storage backend, if we treat that as
separate from the ceph volume support code. Implementing a new image storage
backend in the libvirt drive always requires a specification because of our
historical concerns around adequate CI testing.
For any further questions, please email
openstack-discuss@lists.openstack.org or join #openstack-dev on
Freenode.

276
doc/source/conf.py
View File

@ -1,276 +0,0 @@
# -*- coding: utf-8 -*-
#
# Tempest documentation build configuration file, created by
# sphinx-quickstart on Tue May 21 17:43:32 2013.
#
# This file is execfile()d with the current directory set to its containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import datetime
import sys
import os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0, os.path.abspath('.'))
# -- General configuration -----------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = ['redirect',
'sphinx.ext.autodoc',
'sphinx.ext.todo',
'sphinx.ext.viewcode',
'oslosphinx',
'yasfb',
]
# Feed configuration for yasfb
# feed_base_url = 'TBD'
# feed_author = 'NextGen Team'
todo_include_todos = True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'NextGen Specs'
copyright = u'%s, NextGen Team' % datetime.date.today().year
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = [
'_build',
]
# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
add_module_names = False
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
modindex_common_prefix = ['nextgen-specs.']
# -- Options for man page output ----------------------------------------------
man_pages = []
# -- Options for HTML output ---------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'nature'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
git_cmd = "git log --pretty=format:'%ad, commit %h' --date=local -n1"
html_last_updated_fmt = os.popen(git_cmd).read()
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
html_domain_indices = False
# If false, no index is generated.
html_use_index = False
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'NextGen-Specsdoc'
# -- Options for LaTeX output --------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'NextGen-specs.tex', u'NextGen Specs',
u'NextGen Team', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for Texinfo output ------------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'NextGen-specs', u'NextGen Design Specs',
u'NextGen Team', 'nextgen-specs', 'Design specifications for the NextGen project.',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# -- Options for Epub output ---------------------------------------------------
# Bibliographic Dublin Core info.
epub_title = u'NextGen Specs'
epub_author = u'NextGen Team'
epub_publisher = u'NextGen Team'
epub_copyright = u'2014, NextGen Team'
# The language of the text. It defaults to the language option
# or en if the language is not set.
#epub_language = ''
# The scheme of the identifier. Typical schemes are ISBN or URL.
#epub_scheme = ''
# The unique identifier of the text. This can be a ISBN number
# or the project homepage.
#epub_identifier = ''
# A unique identification for the text.
#epub_uid = ''
# A tuple containing the cover image and cover page html template filenames.
#epub_cover = ()
# HTML files that should be inserted before the pages created by sphinx.
# The format is a list of tuples containing the path and title.
#epub_pre_files = []
# HTML files shat should be inserted after the pages created by sphinx.
# The format is a list of tuples containing the path and title.
#epub_post_files = []
# A list of files that should not be packed into the epub file.
#epub_exclude_files = []
# The depth of the table of contents in toc.ncx.
#epub_tocdepth = 3
# Allow duplicate toc entries.
#epub_tocdup = True

53
doc/source/index.rst
View File

@ -1,53 +0,0 @@
.. nextgen-specs documentation master file
==============================
NextGen Project Specifications
==============================
Priorities
==========
Lists of the most priority specs:
.. toctree::
:glob:
:maxdepth: 1
priorities/initial-priorities
Specifications
==============
Here you can find the specs, and spec templates:
.. toctree::
:glob:
:maxdepth: 1
specs/initial/approved/*
There are also some approved backlog specifications that are looking for owners:
.. toctree::
:glob:
:maxdepth: 1
specs/backlog/approved/*
Process
=======
Documentation for nextgen-specs process:
.. toctree::
:maxdepth: 1
How to submit a spec <readme>
For more details, look at spec template for the specific release, and see the
wiki page on Blueprints: https://wiki.openstack.org/wiki/Blueprints
Indices and tables
==================
* :ref:`search`

1
doc/source/priorities
View File

@ -1 +0,0 @@
../../priorities

1
doc/source/readme.rst
View File

@ -1 +0,0 @@
.. include:: ../../README.rst

52
doc/source/redirect.py
View File

@ -1,52 +0,0 @@
# Copied from openstack/nova-specs
# A simple sphinx plugin which creates HTML redirections from old names
# to new names. It does this by looking for files named "redirect" in
# the documentation source and using the contents to create simple HTML
# redirection pages for changed filenames.
import os.path
from sphinx.application import ENV_PICKLE_FILENAME
from sphinx.util.console import bold
def setup(app):
from sphinx.application import Sphinx
if not isinstance(app, Sphinx):
return
app.connect('build-finished', emit_redirects)
def process_redirect_file(app, path, ent):
parent_path = path.replace(app.builder.srcdir, app.builder.outdir)
with open(os.path.join(path, ent)) as redirects:
for line in redirects.readlines():
from_path, to_path = line.rstrip().split(' ')
from_path = from_path.replace('.rst', '.html')
to_path = to_path.replace('.rst', '.html')
redirected_filename = os.path.join(parent_path, from_path)
redirected_directory = os.path.dirname(redirected_filename)
if not os.path.exists(redirected_directory):
os.makedirs(redirected_directory)
with open(redirected_filename, 'w') as f:
f.write('<html><head><meta http-equiv="refresh" content="0; '
'url=%s" /></head></html>'
% to_path)
def emit_redirects(app, exc):
app.builder.info(bold('scanning %s for redirects...') % app.builder.srcdir)
def process_directory(path):
for ent in os.listdir(path):
p = os.path.join(path, ent)
if os.path.isdir(p):
process_directory(p)
elif ent == 'redirects':
app.builder.info(' found redirects at %s' % p)
process_redirect_file(app, path, ent)
process_directory(app.builder.srcdir)
app.builder.info('...done')

1
doc/source/redirects
View File

@ -1 +0,0 @@
../../specs/redirects

1
doc/source/specs
View File

@ -1 +0,0 @@
../../specs

195
priorities/initial-priorities.rst
View File

@ -1,195 +0,0 @@
.. _intitial-priorities:
==========================
NextGen Initial Priorities
==========================
List of themes and corresponding specs that should be covered as the first
step.
+------------------------------+---------------------------+
| Priority | Spec Driver |
+==============================+===========================+
| `Dev env`_ | `Artur Zarzycki`_ |
+------------------------------+---------------------------+
| `CI/CD pipeline arch`_ | `TBD`_ |
+------------------------------+---------------------------+
| `CI jobs`_ | `TBD`_ |
+------------------------------+---------------------------+
| `Repos break down`_ | `Michal Rostecki`_ |
+------------------------------+---------------------------+
| `Docker layers & versions`_ | `Kirill Proskurin`_ |
+------------------------------+---------------------------+
| `Docker build arch`_ | `Oleksandr Mogylchenko`_ |
+------------------------------+---------------------------+
| `Multiple envs per Mesos`_ | `TBD`_ |
+------------------------------+---------------------------+
| `LMA integration`_ | `Eric Lemoine`_ |
+------------------------------+---------------------------+
.. _Artur Zarzycki: https://directory.mirantis.com/#/profile/azarzycki
.. _Michal Rostecki: https://directory.mirantis.com/#/profile/mrostecki
.. _Kirill Proskurin: https://directory.mirantis.com/#/profile/kproskurin
.. _Oleksandr Mogylchenko: https://directory.mirantis.com/#/profile/amogylchenko
.. _Eric Lemoine: https://directory.mirantis.com/#/profile/elemoine
.. _Ruslan Kamaldinov: https://directory.mirantis.com/#/profile/rkamaldinov
.. _Sergey Lukjanov: https://directory.mirantis.com/#/profile/slukjanov
.. _TBD: #
Dev env
-------
We need to have a spec to cover the following aspects of the Dev Env:
* Installer - we should have a single reusable installer that will be used for
both dev env and at least small labs installation.
* Should be deisgned to work together with Vagrant while supporting manually
created VMs and/or OpenStack cloud to provision them.
Current vision of how we are going to approach is desribed in `NextGen CI/CD
Proposal`_ document. So, it should be used as at least as a base for spec(s).
Here we need to have very clear short-term and long-term plans that will
include intention to have both single node (AIO) and multinode dev envs.
Additional info re installer could be found in `NextGen Installation Proposal`.
Contact persons for more details:
* `Ruslan Kamaldinov`_
CI/CD pipeline arch
-------------------
CI/CD pipeline should be described in terms of used components (Zuul, Jenkins,
Nodepool, Artifactory, Statis files, Logs, ELK, etc.) and approache of building
everything from source code (project-config repo). The repo structure, used
URLs, Mirantis clouds and HW planned to use should be described, etc.
Current vision of how we are going to approach is desribed in `NextGen CI/CD
Proposal`_ document. So, it should be used as at least as a base for spec(s).
More info about CI architecture could be found in `Wiki - CI Design`.
Contact persons for more details:
* `Ruslan Kamaldinov`_
CI jobs
-------
List of CI jobs to run for all components and their dependencies that will
cover our needs to run style checks, unit tests, functional and integration
tests for all system components and OpenStack components.
Current vision of how we are going to approach is desribed in `NextGen CI/CD
Proposal`_ document. So, it should be used as at least as a base for spec(s).
Contact persons for more details:
* `Ruslan Kamaldinov`_
Repos break down
----------------
We need to cover the repositories breakdown from a two repos we have now
(Kolla and Kolla-Mesos) to component specific repositories with initial
structure for each of this repositories. It should additionally include
prerequisites and requirements for separation.
Current vision of how we are going to approach is desribed in `NextGen CI/CD
Proposal`_ document. So, it should be used as at least as a base for spec(s).
Contact persons for more details:
* `Ruslan Kamaldinov`_
* `Sergey Lukjanov`_
Docker layers & versions
------------------------
We need to describe which layering strategy we'll use for the NextGen project
as well as naming and versioning strategies for the generated docker images.
We need to keep in mind that docker could be replaced with some other
containerization tooling, so, when possible, the approaches should be more
generic that specific to docker itself.
Current vision of how we are going to approach is desribed in `NextGen CI/CD
Proposal`_ document. So, it should be used as at least as a base for spec(s).
Contact persons for more details:
* `Ruslan Kamaldinov`_
* `Sergey Lukjanov`_
Docker build arch
-----------------
We need to describe how we'll be storing or generating Dockerfiles for building
container images instead of just mixing different configs and types in a single
Jinja2 template. There are some existing tools like Packer for doint it.
We need to keep in mind that docker could be replaced with some other
containerization tooling, so, when possible, the approaches should be more
generic that specific to docker itself. Because of that, we need to think about
possibility to implement own yaml-based DSL to store lists of the packages to
be installed and generating Dockerfile from them to build docker images and
make it pluggable to support rkt images generating as well.
Current vision of how we are going to approach is desribed in `NextGen CI/CD
Proposal`_ document. So, it should be used as at least as a base for spec(s).
Contact persons for more details:
* `Ruslan Kamaldinov`_
* `Sergey Lukjanov`_
Multiple envs per Mesos
-----------------------
We need to have an ability to deploy multiple OpenStack clouds per single
Mesos stack cluster. The main reason for this requirement is to enable more
effective HW usage for CI/CD and for the development environments.
There is already bunch of potential issues and ideas collected in different
google docs and etherpads, so, all of them should be aggregated into a single
spec that will answer the question - what should be fixed / implemented to
enable multiple OpenStack clusters deployment on top of single Mesos cluster.
So far, it sounds like we need to:
* switch to port mapping + LB / service discovery
* impl somehow compute / network node pinning
LMA integration
---------------
We need to have a short-term plan to integrate minimal logging and monitoring
into the NextGen project as well as the long-term plan to integrate
Stacklight for providing the full logging, monitoring and alerting stack.
The main task in this theme is to cover the short-term plans with concrete
solution and implementation design.
The first thing to cover now is to collect logs across the NextGen cluster
including how we'll be pushing/pulling logs from containers and which
storage will be used for storing and accessing them.
Plan is to have two specs:
* general logging/monitoring/alerting spec discussing the whole architecture.
This will provide the highlevel picture.
* spec focusing on central logging (with Heka and Elasticsearch). This will be
a rewrite of the `Upstream LMA spec for Kolla`.
.. _NextGen CI/CD Proposal: https://docs.google.com/document/d/1rtuINpfTvUFc1lJLuMK5LIzGKWwdh8uRUTvOLKnXuPM/edit
.. _NextGen Installation Proposal: https://docs.google.com/document/d/1TZ2X3QhmJ4bjl4bcFyLxBBbDMpl3hV6mZdq3gmD-rPo/edit
.. _Wiki - CI Design: https://mirantis.jira.com/wiki/display/NG/CI+design
.. _Upstream LMA spec for Kolla: https://github.com/openstack/kolla/blob/master/specs/logging-with-heka.rst

14
requirements.txt
View File

@ -1,14 +0,0 @@
docutils
jinja2>=2.8
python-subunit
markupsafe
extras
pyrsistent
six
hacking>=0.10.0,<0.11
oslosphinx
pbr>=1.6 # Apache-2.0
sphinx>=1.1.2,<1.2
testrepository>=0.0.18
testtools>=0.9.34
yasfb>=0.5.1

23
setup.cfg
View File

@ -1,23 +0,0 @@
[metadata]
name = nextgen-specs
summary = NextGen Project Development Specs
description-file =
README.rst
author = NextGen
author-email = nextgen-all@mirantis.com
home-page = http://mirantis.jira.com/wiki/display/NG/
classifier =
Intended Audience :: Developers
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
[build_sphinx]
all_files = 1
build-dir = doc/build
source-dir = doc/source
[pbr]
warnerrors = True
[wheel]
universal = 1

22
setup.py
View File

@ -1,22 +0,0 @@
#!/usr/bin/env python
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT
import setuptools
setuptools.setup(
setup_requires=['pbr'],
pbr=True)

169
specs/ansible_k8s_installer.rst
View File

@ -1,169 +0,0 @@
=====================================
Ansible-based Kubernetes Installer
=====================================
https://mirantis.jira.com/browse/MCP-614
We have decided to pursue deployment of Kubernetes with an Ansible-based
tool that scales well.
Ansible has a large community behind it and is simple to interact with. This
document proposes to use an established Kubernetes deployment utility, called
Kargo, as the basis of the installer.
Problem description
===================
This document highlights what is important for the installer to have and its
expected outcomes.
This document does not define the underlay which will provision bare metal
hosts (or VMs). It will merely set assumptions for how a prepared host should
be presented.
All installers require either input data or discovery capabilities.
In this case, there will be an underlay system to provision and configure
networking for k8s nodes. The underlay needs to expose metadata to a deployer
in order to proceed with a Kubernetes installation that is adapted for the
given environment.
Use Cases
---------
The installer will be used at all levels, by developers and deployers. CI
testing of Kubernetes and related components will run this installer tool to
test for regressions. QA will use this tool to ensure full deployment scenarios
are bug-free.
All levels of deployment should be the resulting case, but full HA is not in
scope of this spec implementation. Eventually this will apply to live
production scenarios and testing for scale/load/longevity.
It needs to work cleanly and avoid fault scenarios.
Kargo itself can be run with Vagrant and kargo-cli will deploy a heat stack on
OpenStack and also to AWS/GCE environments. Vagrant deployment from a given
metadata YAML, however, is out of scope for this spec.
Proposed change
===============
Kargo has been selected as an initial deployment utility. It requires very
little input for the Ansible inventory at this stage, although our
requirements will likely become more complex in the future.
Kargo is a community-supported, Apache-licensed, open source tool with Ansible
playbooks for deploying Kubernetes. It supports a variety of operating systems
and can source its Kubernetes components from an arbitrary source (but only in
tarball format).
A simple Kargo deploy wrapper will be put in place for installing k8s on top of
Ubuntu Xenial (and Debian) nodes. The initial tool will be a shell script that
performs simple SSH commands to prepare hosts and call Kargo deployment.
Provisioned nodes should contain the following conditions:
* Nodes contain at least 1 configured network interface.
* If calico requires a separate bridge/interface, it will be set up by the
underlay.
* Nodes have /etc/hosts configured and corresponds to provided inventory data
(Kargo will write if it doesn't exist already)
* SSH access is available via key or password
* Effective partitioning on the system to segregate Docker and logs from root
and /var
Additionally, the node performing deployment must be provided a YAML or set of
ENV variables for node IPs and node hostnames must be present to generate
Ansible inventory. Mandatory parameters include:
* Node IPs
* Node hostnames
* Node SSH credentials
Optionally, the provided YAML may contain the following configurable items:
* kube password
* kube network CIDR
* kube network plugin
* kube proxy type
* Docker daemon options
* upstream DNS list
* DNS domain
* component versions
* binary download URLs (and checksums) for kubernetes, etcd, calico
.. note:: See http://bit.ly/1Y4nhPT for all download options.
Alternatives
------------
We could invest in sophisticated SSL setups, external load balancers, or
customizing which network plugin to use. Additionally, external Ceph
configuration may be necessary. These are out of scope, but should be
considered in the underlay story.
There are smaller Ansible-based Kubernetes installers available, but they have
much smaller communities and typically are geared toward Fedora/CentOS. This
includes the in-tree kubernetes Ansible playbook. We could and probably should
contribute here, but it will have a high initial cost for us.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Matthew Mosesohn <raytrac3r>
Work Items
----------
* Kargo deploy wrapper (initially written in Bash, later in Python and/or
Ansible)
* Implement Kargo deployment in CI
* Requirements specification for CCP underlay
* Add support for deploying custom k8s/etcd/calico binaries
* Obtain QA acceptance criteria
Dependencies
============
* This project requires an underlay tool, but we will rely on fuel-devops and
packer-built images until underlay installer is ready.
Testing
=======
While Ansible does not support unit testing, it is possible to include
assertions to test functionality after deployment.
There will be CI coverage for all commits to ensure a "standardized" deployment
works.
Documentation Impact
====================
TBD
References
==========
https://github.com/kubespray/kargo
https://github.com/kubespray/kargo-cli
https://github.com/openstack/fuel-devops
History
=======
.. list-table:: Revisions
:header-rows: 1
* - Iteration 12-1
- Introduced

1
specs/backlog-template.rst
View File

@ -1 +0,0 @@
template.rst

1
specs/backlog/approved/backlog-template.rst
View File

@ -1 +0,0 @@
../../backlog-template.rst

142
specs/ci_infrastructure.rst
View File

@ -1,142 +0,0 @@
==================
CI Infrastructure
==================
This spec captures the work necessary to provision CI infrastructure for the
project. This spec covers following topics:
* Underlying hardware and virtualized infrastructure
* Source code repositories
* CI tools - Jenkins, Zuul, Nodepool, etc
* Methods of deployment of CI tools
* Configuration of CI infrastructure
Problem description
===================
There are several goals for CI infrastructure:
* Reproducibility - it should be a matter of a few actions to roll out new
infrastructure if the old one is gone.
* Re-use existing tools - reuse whatever is available from MOS Infra and
upstream OpenStack Infra.
* Dogfooding - we should run every infrastructure service on top of OpenStack
in the same way upstream OpenStack Infra is running on top of public clouds.
Possibly we should use MCP, then MOS and Mirantis cloud.
Use cases
---------
Proposed change
===============
Services
--------
Setup infrastructure and automation for provisioning of the following
components:
1. Jenkins
Jenkins will run CI jobs
2. Nodepool and VM-based workers
Nodepool will provision single-use worker machines for Jenkins on infra OpenStack
cloud. Following types of workers will be provisioned:
* Build machine
** With Docker engine pre-installed
** Python dependencies pre-installed
** Everything else needed to build and publish Docker images
3. Hardware Jenkins workers
These servers will be used for deployment jobs (each job can spawn multiple
VMs e.g. multinode K8s cluster).
3. Zuul
Zuul will launch Jenkins jobs based on events from Gerrit
4. LMA (Logging, Monitoring, Alerting) services and tools:
The LMA subsystem will consist of various components:
4.1. Static log storage
Will be used to publish logs from automated CI jobs
4.2. Advanced logging and searching services:
This part of LMA is very important but not critical for operation of CI.
It will be covered under separate research including the following technologies:
* ELK stack (ElasticSearch, Logstash, Kibana), if possible integrated with
Jenkins using existing solution. The idea is to send logs from CI jobs to ELK
for further analysis.
* ElasticRecheck: if possible to be based on upstream solution
http://docs.openstack.org/infra/system-config/elastic-recheck.html
* Stacklight
https://launchpad.net/lma-toolchain
5. Artifact repository.
This repository will serve several purposes:
* Storage for Docker images
* Local cache for DEB packages
* frozen repositories (pypi, Deb, Docker) to prevent dynamic updates from
public repos
For internal development any repository can be used (e.g. simple Docker
registry) in the beginning of development, later on more complex solutions
will be evaluated (e.g. Artifactory, Nexus).
Infrastructure
--------------
* https://mirantis.jira.com/wiki/display/NG/Lab+environment
Alternatives
------------
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Mateusz Matuszkowiak
Work items
----------
* deploy HW workers
* create VMs
* configure networking
* deploy all services from Puppet manifest
* configure related repositories
* create sample jobs
Services are documented here:
https://mirantis.jira.com/wiki/display/NG/CI+design
Dependencies
============
Testing
=======
Basic tests of CI infrastructure should be done by running sample jobs
that execute "hello-world" type of shell scripts on selected nodes (bare metal,
VM or on K8s cluster).
Documentation impact
====================
References
==========
.. _Jenkins: http://docs.openstack.org/infra/system-config/jenkins.html
.. _Zuul: http://docs.openstack.org/infra/zuul/
.. _Nodepool: http://docs.openstack.org/infra/nodepool/
History
=======

267
specs/container_build_arch.rst
View File

@ -1,267 +0,0 @@
=====================================
Container build architecture
=====================================
We need to describe how we’ll be storing or generating container specifications
Problem description
===================
Docker proved itself to be production ready and is widely accepted as a ready-to-use
Linux container toolbox.
There are at least 3 different container specifications at the moment, all of
them at different state of production readiness, community acceptance and
interoperability:
* DockerSpec_;
* ACI_;
* OCI_;
It should also be noted that Docker is a ready to use tool, where specification
is provided more as a documentation, rather than a motivation for future changes.
Both ACI and OCF are specifications:
* RKT is an implementation of ACI on Linux, and Jetpack implements ACI
specification on FreeBSD;
* runC_ is an implementation of OCI_ specification on Linux;
Docker, on the other hand, relies on compatibility layer to run on FreeBSD.
Current efforts to bring ACI support to Docker have not yet succeeded (see
https://github.com/docker/docker/issues/9538,
https://github.com/docker/docker/pull/10776,
and https://github.com/docker/docker/issues/10777)
Starting from `1.11 release
<https://github.com/docker/docker/releases/tag/v1.11.0>_` Docker now support
OCI_ runtime specification and allows natively running containers defined in
that format. The drawback here is that OCI_ image format is not defined yet,
and runC_ does not support Docker layers.
Use Cases
---------
Containers are especially useful in micro-services architectures, where full
isolation of application environment is usually an overkill. Containers
provide an easy and declarative way to maintain environment for services.
Proposed change
===============
All containers should use Docker as target platform, and Debian as base OS.
Container configuration should be shipped in Jinja2 template format, which is
made in accordance to Docker format (e.g. Dockerfile). For example:
.. code::
FROM {{ namespace }}/{{ image_prefix }}base:{{ tag }}
MAINTAINER {{ maintainer }}
RUN apt-get install -y --no-install-recommends \
cron \
logrotate \
&& apt-get clean
{% endif %}
COPY extend_start.sh /usr/local/bin/extend_start
RUN chmod 755 /usr/local/bin/extend_start
{{ include_footer }}
Full documentation about templating engine can be found here:
http://jinja.pocoo.org/docs/dev/
Dockerfiles should not contain exact package versions:
* this simplifies maintenance tasks and removed the burden from the
development team;
* avoids breaking dependencies which are maintained by upstream developers;
* brings the risk of fetching broken upstream packages - this should be
mitigated either by having local mirrors, or by additional tests or by
conbination of those two;
See Dependencies_ for full list of specs which define exact layering scheme and
format of the repositories.
Each commit entering build system will flow according to the sheme below:
+-------------+ +------------------------------------------------+
| GIT | | Jenkins |
| | | |
|Microservices| | |
| repo +--+--->Monitor Changes+------------>New commit |
| | ^ | + |
| Application | | | | |
| repo | | | | |
+-------------+ | | | |
| | v |
+-------------+ | | Success<--------------------+Run spec |
| Artifactory | | | + tests |
| | | | | |
| Images of +--+ | | |
| lower layers| | | |
| | | v |
+-------------+ | Build Docker<-------------------+ |
| image | |
| | |
| + |
| Run integration Trigger rebuild |
| tests of all dependent images |
| ^ |
| | |
| Push | |
| to registry+-------------------+ |
| |
+------------------------------------------------+
CI/CD build artifacts should follow certain rules:
* artifacts built from master branch should be kept for a long period of time
for investigations;
* artifacts built from other branches should be rotated on a weekly basis with
a possibility to mark certain build as an exception and keep it for a longer
period of time;
Alternatives
------------
* implement own DSL with compiler capable of producing Dockefile/ACI/OCI spec
* additional burden on the team;
* Docker, ACI and OCI are not 100% compatible in terms of features, so DSL
will only implement some subset of features;
* incorporate `Packer.io <https://www.packer.io/>`_, which allows building
VM images with any available provisioner like Puppet or Salt
* this approach moves the logic of VM configuration from declarative
Dockerfile to Puppet/Salt/whatever, making it yet another logic element;
* does not really provide a way to move to ACI because of
https://github.com/mitchellh/packer/issues/1905
* migrate to OCI runtime spec:
* allows to reuse existing tools for testing configurations, like
https://github.com/huawei-openlab/oct
* OCI image format is not formalized yet, and runC_ does not support layers;
* keep container image configuration in Docker format, but migrate runtime to
runC_:
* allows to keep well known configuration format and layers;
* requires additional convertion before starting containers;
* Docker immutable images feature will not be preserved;
* use Ubuntu as Debian-compatible OS as a base:
* Ubuntu's copyright policy is unsuitable not only for Mirantis, but for
rest of the world as well, see:
https://mjg59.dreamwidth.org/39913.html
Implementation
==============
Concrete implementation may differ in details. For example, Zuul might be used
for launching new jobs instead of Jenkins events. This does not affect the spec
and depends on the CI/CD engineer.
Assignee(s)
-----------
Primary assignee:
TBD
Work Items
----------
#. Configure jenkins using the above schema for workflow
#. Setup spec tests using rspec
#. Configure integration tests and write tests
Dependencies
============
#. `Docker layers & versions`
#. `Repositories split`
#. Jenkins
#. Docker registry
Testing
=======
Tests should be executed on each commit to ensure green CI/CD pipeline. Minimal
set of tests should ensure that:
* Docker logic stays valid;
* integration with other components is not broken;
Exact tooling is out of scope of this spec, but I believe the following
projects might be good candidates:
* https://github.com/mizzy/serverspec (with
https://github.com/swipely/docker-api);
* https://github.com/huawei-openlab/oct (Docker support is under development,
but project's aim is to provide integrated test pipeline for any container
specification available);
In addition to spec testing (where we do not have image yet), integration
testing should be implemented as well. This layer will run tests on environment
with running images, e.g. it should ensure that all applications inside
containers are configured correctly:
* application is listening on the port according to the configuration;
* if TLS/SSL is configured, application correctly negotiates secured
connections;
* application correctly loads configuration and applies it (exact
configuration management is out of scope of this spec though);
* application replies on API requests according to specification;
* regression testing, which will guarantee that new changes do not break old
behavior;
* application is able to work in k8s environment according to its pod
definition;
Documentation Impact
====================
Jenkins workflow should be described in details after all specs are finalized
and merged, and resulting system is implemented.
References
==========
.. _DockerSpec: https://github.com/docker/docker/blob/master/image/spec/v1.md
.. _ACI: https://github.com/appc/spec/blob/master/spec/aci.md
.. _OCI: https://github.com/opencontainers/specs
.. _runC: https://github.com/opencontainers/runc
History
=======
.. list-table:: Revisions
:header-rows: 1
* - Release Name
- Description
* - Mitaka
- Introduced

250
specs/docker-layers.rst
View File

@ -1,250 +0,0 @@
=====================================
Docker layers & versions
=====================================
We need to define a way to layer docker images and how to tag them.
Right now we decided what we should be focused on manageability of the layers,
not on the size.
Problem description
===================
Layers
------
Each Docker image references a list of read-only layers that represent
filesystem differences. Layers are stacked on top of each other to form a
the base for a container’s root filesystem.
When you create a new container, you add a new, thin, writable layer on top of
the underlying stack. This layer is often called the “container layer”. All
changes made to the running container - such as writing new files, modifying
existing files, and deleting files - are written to this thin writable
container layer.
More details `here <https://docs.docker.com/engine/userguide/storagedriver/
imagesandcontainers/>`__
So, our goal it to create good layering system which will allow us to re-use
same(common) layers for most containers. This allows us to save space and fix
common problems for all containers in one place.
Versions
--------
The container itself should be considered as an immutable package of some kind.
And so we need to mark them with some versions, to be able to understand what
code this container contains.
Use Cases
---------
Both layers and versions are base objects for creating and running containers,
so without this, we can't even create the container, so this will affect every
user involved.
Proposed change
===============
Layers
------
My proposal architecture for a layering is based on Nextgen one, with few
differences:
====================== ====================================
Container type Description
====================== ====================================
base Linux OS base
tools Common Linux packages and DSAs
common-libs Common Python and modules
dependencies-[service] All dependencies for a Service
[service] Component executable code only
====================== ====================================
So if we take Keystone as an example:
::
debian
debian-mos
debian-mos-ms
debian-mos-ms-keystone-base
debian-mos-ms-keystone
.. Note:: "ms" stands for "micro services"
And if we take a mysql as an example:
::
debian
debian-mos
debian-mos-mysql
This layering system has several benefits:
- We could edit each type of layer independently from each other.
- We keep the good balance between manageability and size of the layers.
- Each layer serves to one simple purpose which is easy to understand.
Differences from the `NG document <https://docs.google.com/document/d/
1rtuINpfTvUFc1lJLuMK5LIzGKWwdh8uRUTvOLKnXuPM>`_:
- NG document proposes more complex layering for OpenStack services, like
creating that kind of layers
::
nova-base
nova-api-base
nova-api
Versions
--------
Full image name in docker constructed from 3 objects:
1. Registry address
2. Image name
3. Image tag
So it could look like:
`172.20.8.52:6555/kolla-team/centos-binary-base:latest`
============================= =====================
172.20.8.52:6555 Registry address
kolla-team Organisation\\username
centos-binary-base Image name
latest Image tag
============================= =====================
Organisation\\username part is a bit confusing. It makes sense in terms of
private repository, but if the repository is public, it's more like a "group"
in which images are contained.
So, my proposal is to use this Organisation\\username part as a group. So it
will be constructed from this parts:
- `registry-addr`
- `<os_release-version>-<mos-version>-<additional-version>`
- `image-name`
- `<major-version>.<minor-version>-p<patch-version>`
For keystone container it will be:
`1.1.1.1:5000/debian8.3-mos9.0-stable/debian-mos-ms-keystone:1.0-p5`
======================= =====================
1.1.1.1:5000 Registry address
debian8.3-mos9.0-stable Image group
debian-mos-ms-keystone Image name
1.0 Major & minor version
p5 Patch level 5
======================= =====================
Version for CI builds
---------------------
For the CI builds we need to add something uniq to version. When developers
make changes that trigger rebuilding of container image(s) in CI (e.g. change
in keystone source code) we need to be able to match the resulting image with
these changes (e.g. to run deployment tests using this exact set of images).
To achieve this we propose to add Gerrit "change number" (GCR) to the version
string.
Unlike git commit hash that is only unique within 1 project, GCR is unique
within Gerrit instance that can host multiple projects and is also
human-readable (and stores useful information, e.g. it's trivial to build
Gerrit URL from GCR and review related change).
To make this versioning even more granular we can additionally attach Gerrit
"patchset number" (GPR) to distinquish multiple builds for the same CR.
Examples:
- `debian-mos-ms-keystone:1.0-p5-19026` (just the GCR)
- `debian-mos-ms-keystone:1.0-p5-19026_2` (GCR_GPR)
Alternatives
------------
Currently, we have 2 alternatives for layering:
1. **Kolla layering**
Pros:
- Compatibility with upstream.
- Ability to re-use some of their work.
Cons:
- Kolla layers a bit different from our layering vision here at Mirantis. For
example we want to use additional layer for different usefull tools, which is
separated from the base layer and the common libs layer.
- Could be really hard or impossible to convince community to change something
in layering if we will need to.
2. **NG document proposal**
Pros:
- A bit more fine tuned layering system.
Cons:
- I think this will lead to confusion and not really help us to solve any
problems, since it doesn't really help us to maintain size\managability
balance, since this layers are tiny and used only for one type of service
right now.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
`Proskurin Kirill <https://launchpad.net/~kproskurin>`__
Other contributors:
`Proskurin Kirill <https://launchpad.net/~kproskurin>`__
`Zawadzki Marek <https://launchpad.net/~mzawadzki-f>`__
Work Items
----------
None
Dependencies
============
None
Testing
=======
None
Documentation Impact
====================
This policy needs to be added to some kind of future documentation about
container creation.
References
==========
None
History
=======
.. list-table:: Revisions
:header-rows: 1
* - Release Name
- Description
* - Mitaka
- Introduced

95
specs/dsl_versioning_spec.rst
View File

@ -1,95 +0,0 @@
==============
DSL Versioning
==============
The goal of this spec is creating versioning mechanism for DSL
Problem description
===================
The problem is that our DSL may be changed. Some modifications are
backward compatible, some of them are not. And we may have situation when
service file written for old DSL version will be deployed using new fuel-ccp
tool which does not support that old version of DSL.
Use Cases
---------
#. User tries to deploy services with newer version than fuel-ccp DSL version,
services files are incompatible with fuel-ccp tool.
#. User tries to deploy services with older or equal version than fuel-ccp DSL
version, services files are compatible with fuel-ccp tool
#. User tries to deploy services with older version than fuel-ccp DSL
version, services files are incompatible with fuel-ccp tool
Proposed change
===============
#. Introduce semver-based DSL versioning in fuel-ccp tool. Version is a number
MAJOR.MINOR.PATCH. This version will be defined somewhere in
`fuel-ccp repo`. Each service definition will contain minimal DSL
version that required for successful deploy.
You should increment:
#. MAJOR version when you make incompatible changes in DSL;
#. MINOR version when you make backwards-compatible changes in DSL;
#. PATCH version when you make fixes that do not change DSL, but affect
processing flow (will be rarely used).
# Check version compatibility during "fuel-ccp deploy" call according next
rules:
#. if fuel-ccp's DSL version is less than service's DSL version -
they are incompatible - error will be printed, deployment will be
aborted;
#. if MAJOR part of these versions are different - they are incompatible -
error will be printed, deployment will be aborted;
#. otherwise they are compatible and deploy should be continued.
# The initial version for all services and fuel-ccp will be 0.1.0
Alternatives
------------
#. Leave as is, without versioning
#. Use some existing solution like https://www.python.org/dev/peps/pep-0440/
#. Invent some another versioning approach
Implementation
==============
#. Add versions to all service's repos
#. Add version to fuel-ccp repo
#. Add logic to fuel-ccp deploy for handling version validation
Assignee(s)
-----------
apavlov
Work Items
----------
#. implement patch for fuel-ccp
#. implement patches for service's repos
Dependencies
============
None
Testing
=======
Unit testing for fuel-ccp version validation logic
Documentation Impact
====================
Describe version bump rules
References
==========
None
History
=======
None

297
specs/general-lma-spec.rst
View File

@ -1,297 +0,0 @@
==========================================================
General Logging, Monitoring, Alerting architecture for MCP
==========================================================
[No Jira Epic for this spec]
This specification describes the general Logging, Monitoring, Alerting
architecture for MCP (Mirantis Cloud Platform).
Problem description
===================
Logging, Monitoring and Alerting are key aspects which need to be taken into
account from the very beginning of the MCP project.
This specification just describes the general architecture for Logging,
Monitoring and Alerting. Details on the different parts will be provided with
more specific specifications.
In the rest of the document we will use LMA to refer to Logging, Monitoring and
Alerting.
Use Cases
---------
The final goal is to provide tools to help OpenStack Operator diagnose and
troubleshoot problems.
Proposed change
===============
We propose to add LMA components to MCP. The proposed software and architecture
are based on the current Fuel StackLight product (composed of four Fuel
plugins), with adjustements and improvements to meet the requirement of MCP
(Mirantis Cloud Platform).
General Architecture
--------------------
The following diagram describes the general architecture::
OpenStack nodes
+-------------------+
| +-------------------+
| | +----+ |
| | Logs+-+ +-+Snap| | +-------------+
| | | | +----+ | | |
| | +v--v+ | +------>Elasticsearch|
| | |Heka+--------------+ | |
+-+ +----+ | | +-------------+
+-------------------+ |
| +-------------+
| | |
+------+InfluxDB |
k8s master node | | |
+-------------------+ | +-------------+
| +-------------------+ |
| | +----+ | | +-------------+
| | Logs+-+ +-+Snap| | +--+ | |
| | | | +----+ | | +------>Nagios |
| | +v--v+ | | | |
| | |Heka+-----------+ +-------------+
+-+ +----+ |
+-------------------+
The boxes on the top-left corner of the diagram represent the nodes where the
OpenStack services run. The boxes on the bottom-left corner of the diagram
represent the the nodes where the Kubernetes infrastructure services run. The
boxes on the right of the diagram represent the nodes where the LMA backends
are run.
Each node runs two services: Heka and Snap. Although it is not depicted in the
diagram Heka and Snap also run on the backend nodes, where we also want to
collect logs and telemetry data.
`Snap`_ is the telemetry framework created by Intel that we will use in MCP for
collecting telemetry data (CPU usage, etc.). The current StackLight product
uses Collectd instead of Snap, so this is an area where StackLight and MCP will
differ. The telemetry data collected by Snap will be sent to Heka.
`Heka`_ is a stream processing software created and maintained by Mozilla. We
will use Heka for collecting logs and notifications, deriving new metrics from
the telemetry data received from Snap, and sending the results to
Elasticsearch, InfluxDB and Nagios.
`Elasticsearch`_ will be used for indexing logs and notifications. And
`Kibana`_ will be used for visualizing the data indexed in Elasticsearch.
Default Kibana dashboards will be shipped in MCP.
`InfluxDB`_ is a database optimized for time-series. It will be used for
storing the telemetry data. And `Grafana`_ will be used for visualizing the
telemetry data stored in InfluxDB. Default Grafana dashboards will be shipped
in MCP.
`Nagios`_ is a feature-full monitoring software. In MCP we may use it for
handling status messages sent by Heka and reporting on the current status of
nodes and services. For that Nagios's `Passive Checks`_ would be used. We've
been looking at alternatives such as `Sensu`_ and `Icinga`_ , but until now we
haven't found something with the level of functionality of Nagios. Another
alternative is to just rely on Heka's `Alert module`_ and `SMTPOutput plugin`_
for notifications. Whether Nagios will be used or not in MCP will be discussed
with a more specific specification. It is also to be noted that Alerting should
be an optional part of the monitoring sytem in MCP.
.. _Snap: https://github.com/intelsdi-x/snap
.. _Heka: http://hekad.readthedocs.org/
.. _Elasticsearch: https://www.elastic.co/products/elasticsearch
.. _Kibana: https://www.elastic.co/products/kibana
.. _InfluxDB: https://influxdata.com/
.. _Grafana: http://grafana.org/
.. _Nagios: https://www.nagios.org/
.. _Passive Checks: https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/3/en/passivechecks.html
.. _Sensu: https://sensuapp.org/
.. _Icinga: https://www.icinga.org/
.. _Alert module: http://hekad.readthedocs.io/en/latest/sandbox/index.html#alert-module
.. _SMTPOutput plugin: http://hekad.readthedocs.io/en/latest/config/outputs/smtp.html
Kubernetes Logging and Monitoring
---------------------------------
Kubernetes comes with its own monitoring and logging stack, so the question of
what we will use and not use of this stack should be raised. This section
discusses that.
Monitoring
~~~~~~~~~~
`Kubernetes Monitoring`_ uses `Heapster`_. Heapster runs as a pod on a node of
the Kubernetes cluster. Heapster gets container statistics by querying the
cluster nodes' Kubelets. The Kubelet itself fetches the data from cAdvisor.
Heapster groups the information by pod and sends the data to a backend for
storage and visualization (InfluxDB is supported).
Collecting container and pod statistics is necessary for MCP, but it's not
sufficient. For example, we also want to collect OpenStack services, to be able
to report on the health of the OpenStack services that run on the cluster.
Also, Heapster does not currently support any form of alerting.
The proposal is to use Snap en each node (see the previous section). Snap
already includes `plugins for OpenStack`_. For container statistics the `Docker
plugin`_ may be used, and, if necessary, a Kubernetes/Kubelet-specific Snap
plugin may be developed.
Relying on Snap on each node, instead of a centralized Heapster instance, will
also result in a more scalable solution.
However, it is to be noted that `Kubernetes Autoscaling`_ currently requires
Heapster. This means that Heapster must be used if the Autoscaling
functionality is required for MCP. But in that case, no storage backend should
be set in the Heapster configuration, as Heapster will just be used for the
Autoscaling functionality.
.. _Kubernetes Monitoring: http://kubernetes.io/docs/user-guide/monitoring/
.. _Heapster: https://github.com/kubernetes/heapster
.. _plugins for OpenStack: https://github.com/intelsdi-x?utf8=%E2%9C%93&query=snap-plugin-collector
.. _Docker plugin: https://github.com/intelsdi-x/snap-plugin-collector-docker
.. _Kubernetes Autoscaling: http://kubernetes.io/docs/user-guide/horizontal-pod-autoscaling/
Logging
~~~~~~~
`Kubernetes Logging`_ relies on Fluentd, with a Fluentd agent running on each
node. That agent collects container logs (through the Docker Engine running on
the node) and sends them to Google Cloud Logging or Elasticsearch (the backend
used is pecified through the ``KUBE_LOGGING_DESTINATION`` variable).
The main problem with this solution is our inability to act on the logs before
they're stored into Elasticsearch. For instance we want to be able to monitor
tho logs, to be able to detect spikes of errors. We also want to be able to
derive metrics from logs, such as HTTP response time metrics. Also, we may want
to use Kafka in the future (see below). In summary, Kubernetes Logging does not
provide us with the flexibility we need.
Our proposal is to use Heka instead of Fluentd. The benefits are:
* Flexibility (e.g. use Kafka between Heka and Elasticsearch in the future).
* Be able to collect logs from services that can't log to stdout.
* Team's experience on using Heka and running it in production.
* Re-use all the Heka plugins we've developed (parsers for OpenStack logs, log
monitoring filters, etc.).
.. _Kubernetes Logging: http://kubernetes.io/docs/getting-started-guides/logging/
Use Kafka
---------
Another component that we're considering introducing is `Apache Kafka`_. Kafka
will sit between Heka and the backends, and it will be used as a robust and
scalable messaging system for the communications between the Heka instances and
the backends. Heka has the capability of buffering messaging, but we believe
that Kafka would allow for a more robust and resilient system. We may make
Kafka optional, but highly recommended for medium and large clusters.
The following diagram depicts the architecture when Kafka is used:
OpenStack nodes
+-------------------+
| +-------------------+ Kafka cluster
| | +----+ | +-------+ +-------------+
| | Logs+-+ +-+Snap| | | | | |
| | | | +----+ | +--+Kafka +--+ +----->Elasticsearch|
| | +v--v+ | | | | | | | |
| | |Heka+--------------> +-------+ +----+ +-------------+
+-+ +----+ | | |
+-------------------+ | +-------+ | +-------------+
| | | | | |
+--+Kafka +--+ +--->InfluxDB |
| | | | | | |
k8s master nodes | +-------+ +------+ +-------------+
+-------------------+ | |
| +-------------------+ +--> +-------+ +----+ +-------------+
| | +----+ | | | | | | | | |
| | Logs+-+ +-+Snap| | | +--+Kafka +--+ +----->Nagios |
| | | | +----+ | | | | | |
| | +v--v+ | | +-------+ +-------------+
| | |Heka+-----------+
+-+ +----+ |
+-------------------+
The Heka instances running on the OpenStack and Kubernetes nodes are Kafka
producers. Although not depicted on the diagram Heka instances will also
probably be used as Kafka consumers between the Kafka cluster and the backends.
We will need to run performance tests to determine if Heka will be able to keep
up with the load when used as a Kafka consumer.
A specific specification will be written for the introduction of Kafka.
.. _Apache Kafka: https://kafka.apache.org/
Packaging and deployment
------------------------
All the services participating to the LMA architecture will run in Docker
containers, following the MCP approach to packaging and service execution.
Relying on `Kubernetes Daemon Sets`_ for deploying Heka and Snap on all the
cluster nodes sounds like a good approach. The Kubernetes doc even mentions
logstash and collectd as a good candidates for running as Daemon Sets.
.. _Kubernetes Daemon Sets: http://kubernetes.io/docs/admin/daemons/
Alternatives
------------
The possible alternatives will be discussed in more specific specifications.
Implementation
==============
The implementation will be described in more specific specifications.
Assignee(s)
-----------
Primary assignee:
elemoine (elemoine@mirantis.com)
Other contributors:
obourdon (obourdon@mirantis.com)
Work Items
----------
Other specification documents will be written:
* Logging with Heka
* Logs storage and analytics with Elasticsearch and Kibana
* Monitoring with Snap
* Metrics storage and analytics with InfluxDB and Grafana
* Alerting in MCP
* Introducing Kafka to the MCP Monitoring stack
Dependencies
============
None.
Testing
=======
The testing strategy will be described in more specific specifications.
Documentation Impact
====================
The MCP monitoring system will be documented.
References
==========
None.
History
=======
None.

1
specs/initial-template.rst
View File

@ -1 +0,0 @@
template.rst

1
specs/initial/approved/initial-template.rst
View File

@ -1 +0,0 @@
../../initial-template.rst

277
specs/introduce-monitoring.rst
View File

@ -1,277 +0,0 @@
=====================
Add Monitoring to MCP
=====================
This specification describes the initial work that will be done for adding
monitoring capabilities to MCP. The goal is to lay out the architecture for
collecting, storing and visualizing basic system metrics (CPU, memory, etc.).
Problem description
===================
Monitoring is a core aspect of MCP. The goal of the work discussed in this
specification is to introduce first elements of Monitoring in MCP.
Use Cases
---------
This specification covers the following:
* Collecting data with Snap
* Processing the data with Hindsight
* Storing the data into InfluxDB
* Visualizing the data with Grafana
This specification focuses on Monitoring, aspects related to Alarming are out
of scope.
Proposed change
===============
Collecting data with Snap
-------------------------
We will use `Snap`_ for collecting monitoring data. Snap will run on every
cluster node. At this stage only system statistics will be collected, i.e. CPU,
memory, disk, network, etc. The list of collected metrics will depend on what
is available in Snap.
The data collected by Snap will then published to Hindsight (described in the
next section). For that we will use Snap's `Heka publisher plugin`_.
.. _Snap: http://intelsdi-x.github.io/snap/
.. _Heka publisher plugin: https://github.com/intelsdi-x/snap-plugin-publisher-heka
Snap in Docker
--------------
Snap will run in a Docker container, so a ``Dockerfile`` will be written for
Snap in MCP.
Some Snap plugins aren't currently compatible with running into Docker
containers. We've created GitHub Issues for these incompatibilities:
* https://github.com/intelsdi-x/snap-plugin-collector-cpu/issues/14
* https://github.com/intelsdi-x/snap-plugin-collector-processes/issues/11
* https://github.com/intelsdi-x/snap-plugin-collector-meminfo/issues/11
* https://github.com/intelsdi-x/snap-plugin-collector-disk/issues/6
* https://github.com/intelsdi-x/snap-plugin-collector-df/issues/8
* https://github.com/intelsdi-x/snap-plugin-collector-load/issues/8
* https://github.com/intelsdi-x/snap-plugin-collector-iostat/issues/9
* https://github.com/intelsdi-x/snap-plugin-collector-swap/issues/7
We will address these imcompatibilities with Pull Requests to Snap. Plugins
like the `df` and `iostat` use Unix commands internally (the `iostat` plugin
uses the `iostat` command for example). And in most cases these Unix commands
cannot be parameterized to read from another directory than `/proc`. This means
that these plugins will need to be eventually rewritten.
Snap is in active development. We ourselves need to change Snap and create Pull
Requests. So it is important that we are able to build Snap from sources, as
opposed to depending on binary releases created by the Snap team.
Docker Hub includes `official images for Go (golang)`_, which are based on
`buildpack-deps`_ images. We're not going to rely of these images. Instead we
will rely on MCP's ``ms-debian-base`` image, and we will ourselves install the
build tools we need for building Snap. In this way we can remove the build
tools, and thereby minimize the weight of Docker images. The final Snap image
will just include the Snap binaries required for running Snap. The Go compiler
will for example not be present in the final image.
.. _official images for Go (golang): https://hub.docker.com/r/library/golang/
.. _buildpack-deps: https://hub.docker.com/_/buildpack-deps/
Processing the data with Hindsight
----------------------------------
The data collected with Snap will be published to `Hindsight`_. Like Snap,
Hindsight will run on each cluster node, and Snap will publish the collected
data to the Hindsight instance running on the same node (in the same ``Pod``).
Hindsight is a rewrite of Heka in C. It was created to address some issues
found in Heka (mainly performance issues). Hindsight is compatible with Heka,
in the sense that Lua plugins that work in Heka also work in Hindsight. And
Hindsight supports Heka's Protobuf format.
For the communication between Snap and Hindsight we will use TCP and Heka's
Protobuf format, which Hindsight supports. On the Snap side the Heka publisher
plugin will be used. On the Hindsight side we will use the `Heka TCP Input
Plugin`_. The plugin will listen on port ``5565``, and decode the Protobuf
messages sent by Snap. The resulting messages injected into the Hindsight
pipeline will then be grouped and sent to InfluxDB by batch. For that an
Hindsight Output Plugin will be developed. That plugin will reuse some `Lua
code`_ from StackLight.
It is to be noted that Hindsight just acts as a "passthru" here. In other
words, for the basic needs described in this specification, we could do without
Hindsight and have Snap directly publish the metrics to InfluxDB (through
Snap's InfluxDB publisher plugin). But in the future we will use Hindsight for
evaluating alarms, deriving metrics from logs, etc. So it is important to use
Hindsight from the beginning.
.. _Hindsight: https://github.com/trink/hindsight/
.. _Heka TCP Input Plugin: https://github.com/mozilla-services/lua_sandbox/blob/master/sandboxes/heka/input/heka_tcp.lua
.. _Lua code: https://github.com/openstack/fuel-plugin-lma-collector/blob/master/deployment_scripts/puppet/modules/lma_collector/files/plugins/filters/influxdb_accumulator.lua
Snap and Hindsight deployment
-----------------------------
Snap and Hindsight will be deployed by Kubernetes on every Kubernetes minion.
We will rely on a ``DaemonSet`` for that. In this way, Kubernetes will start
Snap and Hindsight on any new node added to the cluster.
Snap and Hindsight will run in separate Containers part of the same ``Pod``.
Only one ``DaemonSet`` will be required.
At this stage we won't use `Snap Tribes`_. We indeed don't need that level of
sophistication for now, because all the Snap instances will be identical: they
will run the same plugins and tasks.
The Snap configuration will be stored in ``ConfigMap``, and we will rely on
Snap's plugin/task `auto_discovery_path`_ functionality for Snap to load the
plugins and create the tasks at start-up time. Currently the
``auto_discovery_path`` functionality only works for the loading of plugins. We
will extend it to also work for the creation of tasks.
Likewise, the Hindsight configuration will also be stored in ``ConfigMap``.
.. _Snap Tribes: https://github.com/intelsdi-x/snap/blob/master/docs/TRIBE.md
.. _auto_discovery_path: https://github.com/intelsdi-x/snap/blob/master/docs/SNAPD_CONFIGURATION.md#snapd-control-configurations
Git repositories
----------------
As discussed previously Snap and Hindsight will run in the same ``Pod``, and
they will be deployed by the same ``DaemonSet``. This means that the
``DaemonSet`` spec file will be common to Snap and Hindsight. For that reason
we will use just one Git repository: ``ms-lma`` or ``ms-stacklight``.
Storing the data into InfluxDB
------------------------------
As described in the previous section Hindsight will send the data to InfluxDB
for storing.
.. note:: We will investigate using Cassandra in the future. But we will start
with InfluxDB, because we know how to run and operate InfluxDB.
InfluxDB deployment
-------------------
InfluxDB will be deployed by Kubernetes. At this point we will not run InfluxDB
in cluster mode. We will use a ``ReplicaSet`` (in a ``Deployment``) with one
replica.
Storage is another issue. To simplify the problem we will dedicate a node to
InfluxDB (using a node label). InfluxDB will run on that node and it will not
run on any other node. At this point we will use an ``emptyDir`` or
``hostPath`` Kubernetes volume on a local disk for the data storage. In the
future we may use LVM, depending on the outcome of our `LVM driver for
Volumes`_ work.
For Hindsight and Grafana to be able to access InfluxDB a Kubernetes Service
will be created for InfluxDB. The ``ClusterIP`` service type will be used.
.. note:: Using Ceph RDB for the data storage is not an option. However we know
from experience that a local SSD is required to get decent performances.
.. note:: In the future we will need to support the case of a remote InfluxDB
backend deployed outside the Kubernetes cluster. This means that it will be
possible to configure Hindsight to use a different InfluxDB endpoint.
.. note:: For deploying InfluxDB/Grafana with Kubernetes we can get inspiration
from Heapster. See https://github.com/kubernetes/heapster/tree/master/deploy/kube-config/influxdb.
.. _LVM driver for Volumes: https://mirantis.jira.com/browse/MCP-692
Grafana
-------
Grafana will run in a Docker container. And Grafana will be deployed by
Kubernetes, through a dedicated ``Pod`` and a dedicated ``ReplicaSet``.
The number of replicas will be set to one, ensuring that there will be at most
one running Grafana instance at a given time. In the future we will be able to
scale Grafana by using more replicas, but we don't need that level of
sophistication for the moment.
Grafana needs a database to store users and dashboards. By default Grafana uses
SQLite. To simplify the deployment we will use SQLite and an ``emptyDir``
Kubernetes volume. This means that any custom settings will be lost if Grafana
is restarted on another node. In the future we could rely on an RDB or LVM
volume to avoid that problem. We may also consider using another DBMS than
SQLite.
Grafana will be pre-configured with a default data source (connected to the
InfluxDB instance) and default dashboards. Adding data sources and dashboards
to Grafana is done using the Grafana HTTP API. The configuration operations
will be done in the ``start.sh`` script that starts the Grafana service. See
https://github.com/grafana/grafana-docker/blob/master/run.sh for an example.
The operator will need to access the Grafana UI. For that we will create
a Kubernetes Service for Grafana. We will use a `NodePort Service`_ for the
moment, but we will probably need to rely on a Load Balancer in the future.
This depends on what will be used in MCP.
.. _NodePort Service_: http://kubernetes.io/docs/user-guide/services/#type-nodeport
Users and Passwords
-------------------
Both InfluxDB and Grafana require creating users and passwords. We will use
`Kubernetes Secrets`_ for that.
.. _Kubernetes Secrets: http://kubernetes.io/docs/user-guide/secrets/
Alternatives
------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Éric Lemoine
Other contributors:
Olivier Bourdon
Work Items
----------
* Create a ``Dockerfile`` for Snap
* Create a ``Dockerfile`` for Heka
* Create a ``Dockerfile`` for InfluxDB
* Create a ``Dockerfile`` for Grafana
* Create ``DaemonSet`` and ``Pod`` definitions for Snap/Heka
* Create ``ReplicaSet`` for InfluxDB/Grafana
* Create ``ConfigMap`` for Snap and Heka configurations
* Extend Snap to support the auto-discovery of tasks
* Make Snap plugins compatible with Docker
Dependencies
============
* Working developement environment
* Working CI
Testing
=======
We will develop functional tests to verify that our Snap/Heka pipeline
works as expected. This can be done with a mock Snap collector plugin, and
checking that the output of the pipeline is as expected.
Documentation Impact
====================
We will document how to set up Monitoring in MCP.
References
==========
None.

229
specs/logging-with-heka.rst
View File

@ -1,229 +0,0 @@
=================
Logging with Heka
=================
[No Jira Epic for this spec]
This specification describes the logging system to implement in Mirantis Cloud
Platform (MCP). It is complementary to the *General Logging, Monitoring,
Alerting architecture for MCP* specification.
That system is based on `Heka`_, `Elasticsearch`_ and `Kibana`_.
.. _Heka: http://hekad.readthedocs.org/
.. _Elasticsearch: https://www.elastic.co/products/elasticsearch
.. _Kibana: https://www.elastic.co/products/kibana
Problem description
===================
It is important that MCP comes with a robust and scalable logging system. This
specification describes the logging system we want to implement in MCP. It is
based on StackLight's logging solution for MOS.
Use Cases
---------
The target for the logging system is the Operator of Kubernetes cluster. The
Operator will use Kibana to view and search logs, with dashboards providing
statistics views of the logs.
We also want to be able to derive metrics from logs and monitor logs to detect
spike of errors for example. The solution described in this specification makes
this possible, but the details of log monitoring will be covered with
a separate specification.
The deployment and configuration of Elasticsearch and Kibana through Kubernetes
will also be described with a separate specification.
Proposed change
===============
Architecture
------------
This is the architecture::
Cluster nodes
+---------------+ +----------------+
| +---------------+ | +----------------+
| | +---------------+ | | +----------------+
| | | | | | | |
| | | Logs+-+ | | | | |
| | | | | | | | |
| | | | | | | | Elasticsearch |
| | | +--v-+ | | | | |
| | | |Heka+--------------------------> | | |
+-+ | +----+ | +-+ | |
+-+ | +-+ |
+---------------+ +----------------+
In this architecture Heka runs on every node of the Kubernetes cluster. It runs
in a dedicated container, referred to as the *Heka container* in the rest of
this document.
Each Heka instance reads and processes the logs local to the node it runs on,
and sends these logs to Elasticsearch for indexing. Elasticsearch may be
distributed on multiple nodes for resiliency and scalability, but this topic is
outside the scope of that specification.
Heka, written in Go, is fast and has a small footprint, making it possible to
run it on every node of the cluster and effectively distribute the log
processing load.
Another important aspect is flow control and avoiding the loss of log messages
in case of overload. Heka’s filter and output plugins, and the Elasticsearch
output plugin in particular, support the use of a disk based message queue.
This message queue allows plugins to reprocess messages from the queue when
downstream servers (Elasticsearch) are down or cannot keep up with the data
flow.
Rely on Docker Logging
----------------------
Based on `discussions`_ with the Mirantis architects and experience gained with
the Kolla project the plan is to rely on `Docker Logging`_ and Heka's
`DockerLogInput plugin`_.
Since the `Kolla logging specification`_ was written the support for Docker
Logging has improved in Heka. More specifically Heka is now able to collect
logs that were created while Heka wasn't running.
Things to note:
* When ``DockerLogInput`` is used there is no way to differentiate log messages
for containers producing multiple log streams – containers running multiple
processes/agents for example. So some other technique will have to be used
for containers producing multiple log streams. One technique involves using
log files and Docker volumes, which is the technique currently used in Kolla.
Another technique involves having services use Syslog and have Heka act as
a Syslog server for these services.
* We will also probably encounter services that cannot be configured to log to
``stdout``. So again, we will have to resort to using some other technique
for these services. Log files or Syslog can be used, as described previously.
* Past experiments have shown that the OpenStack logs written to ``stdout`` are
visible to neither Heka nor ``docker logs``. This problem does not exist
when ``stderr`` is used rather than ``stdout``. The cause of this problem is
currently unknown.
* ``DockerLogInput`` relies on Docker's `Get container logs endpoint`_, which
works only for containers with the ``json-file`` or ``journald`` logging
drivers. This means the Docker daemon cannot be configured with another
logging driver than ``json-file`` or ``journald``.
* If the ``json-file`` logging driver is used then the ``max-size`` and
``max-file`` options should be set, for containers logs to be rolled over as
appropriate. These options are not set by default in Ubuntu (in neither 14.04
nor 16.04).
.. _discussions: https://docs.google.com/document/d/15QYIX_cggbDH2wAJ6-7xUfmyZ3Izy_MOasVACutwqkE
.. _Docker Logging: https://docs.docker.com/engine/admin/logging/overview/
.. _DockerLogInput plugin: http://hekad.readthedocs.org/en/v0.10.0/config/inputs/docker_log.html
.. _Kolla logging specification: https://github.com/openstack/kolla/blob/master/specs/logging-with-heka.rst
.. _Get container logs endpoint: https://docs.docker.com/engine/reference/api/docker_remote_api_v1.20/#get-container-logs
Read Python Tracebacks
----------------------
In case of exceptions the OpenStack services log Python Tracebacks as multiple
log messages. If no special care is taken then the Python Tracebacks will be
indexed as separate documents in Elasticsearch, and displayed as distinct log
entries in Kibana, making them hard to read. To address that issue we will use
a custom Heka decoder, which will be responsible for coalescing the log lines
making up a Python Traceback into one message.
Collect system logs
-------------------
In addition to container logs it is important to collect system logs as well.
For that we propose to mount the host's ``/var/log`` directory into the Heka
container (as ``/var/log-host/``), and configure Heka to get logs from standard
log files located in that directory (e.g. ``kern.log``, ``auth.log``,
``messages``).
Create an ``heka`` user
-----------------------
For security reasons an ``heka`` user will be created in the Heka container and
the ``hekad`` daemon will run under that user.
Deployment
----------
Following the MCP approach to packaging and service execution the Heka daemon
will run in a container. We plan to rely on Kubernetes's `Daemon Sets`_
functionality for deploying Heka on all the Kubernetes nodes.
We also want Heka to be deployed on the Kubernetes master node. For that the
Kubernetes master node should also be a minion server, where Kubernetes may
deploy containers.
.. _Daemon Sets: http://kubernetes.io/docs/admin/daemons/
Security impact
---------------
The security impact is minor, as Heka will not expose any network port to the
outside. Also, Heka's "dynamic sandboxes" functionality will be disabled,
eliminating the risk of injecting malicious code into the Heka pipeline.
Performance Impact
------------------
The ``hekad`` daemon will run in a container on each cluster node. And we have
assessed that Heka is lightweight enough to run on every node. See the
`Introduction of Heka in Kolla`_ email sent to the openstack-dev mailing list
for a discussion on comparison between Heka and Logstash. Also, a possible
option would be to constrain the resources associated to the Heka container.
.. _Introduction of Heka in Kolla: http://lists.openstack.org/pipermail/openstack-dev/2016-January/083751.html
Alternatives
------------
An alternative to this proposal involves relying on Kubernetes Logging, i.e.
use Kubernetes's native logging system. Some `research`_ has been done on
Kubernetes Logging. The conclusion to this research is that Kubernetes Logging
is not flexible enough, making it impossible to implement features such as
log monitoring in the future.
.. _research: https://mirantis.jira.com/wiki/display/NG/k8s+LMA+approaches
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Éric Lemoine (elemoine)
Work Items
----------
1. Create an Heka Docker image
2. Create some general Heka configuration
3. Deploy Heka through Kubernetes
4. Collect OpenStack logs
5. Collect other services' logs (RabbitMQ, MySQL...)
6. Collect Kubernetes logs
7. Send logs to Elasticsearch
Testing
=======
We will add functional tests that verify that the Heka chain works for all the
service and system logs Heka collects. These tests will be executed as part of
the gating process.
Documentation Impact
====================
None.
References
==========
None.

0
specs/redirects
View File

324
specs/repositories-split.rst
View File

@ -1,324 +0,0 @@
==========================================
Repositories split
==========================================
MCP project is going to use splitted repositories for Dockerfiles and
config files per each OpenStack service and infra service.
Problem description
===================
MCP project wants to move out from using Kolla images and it's aiming to
provide its own repositories for Dockerfiles and configuration files
The layout of Kolla repository, which contains all the Dockerfiles and config
files is not satysfying NextGen needs.
Pros for splitting repositories:
* Possibility to engage engineers from teams to take some responsibility on
containerizing each component.
* Similarity to the Puppet, Chef and Ansible repositories in upstream.
Cons:
* CI will have to handle cross-repository dependencies. That may be hard
especially in case of fixing the issues which have to be fixed in the
configuration of multiple OpenStack components at once, because CI will become
green after accepting a stream of patches in many repos.
* Release management and stable branch maintenance will be harder, because
the whole Microservices project will contain ~38 repositories.
Use Cases
---------
Proposed change
===============
The "nextgen" organization should be created on Gerrit with following git
repositories:
* nextgen/microservices - the builder of Docker images and CLI tool for pushing
configuration in etcd and deploying OpenStack on Kubernetes
* nextgen/ms-ext-config - start script for containers which fetches config
from etcd/ZooKeeper
* nextgen/ms-debian-base - the layer for base Linux packages
* nextgen/ms-openstack-base - the base layer for OpenStack services, containing
common Python modules
* nextgen/ms-aodh-api
* nextgen/ms-aodh-base
* nextgen/ms-aodh-evaluator
* nextgen/ms-aodh-expirer
* nextgen/ms-aodh-listener
* nextgen/ms-aodh-notifier
* nextgen/ms-ceilometer-api
* nextgen/ms-ceilometer-base
* nextgen/ms-ceilometer-central
* nextgen/ms-ceilometer-collector
* nextgen/ms-ceilometer-compute
* nextgen/ms-ceilometer-notification
* nextgen/ms-ceph-base
* nextgen/ms-ceph-mon
* nextgen/ms-ceph-osd
* nextgen/ms-ceph-rgw
* nextgen/ms-cinder-api
* nextgen/ms-cinder-backup
* nextgen/ms-cinder-base
* nextgen/ms-cinder-rpcbind
* nextgen/ms-cinder-scheduler
* nextgen/ms-cinder-volume
* nextgen/ms-designate-api
* nextgen/ms-designate-backend-bind9
* nextgen/ms-designate-base
* nextgen/ms-designate-central
* nextgen/ms-designate-mdns
* nextgen/ms-designate-poolmanager
* nextgen/ms-designate-sink
* nextgen/ms-elasticsearch
* nextgen/ms-glance-api
* nextgen/ms-glance-base
* nextgen/ms-glance-registry
* nextgen/ms-heat-api
* nextgen/ms-heat-api-cfn
* nextgen/ms-heat-base
* nextgen/ms-heat-engine
* nextgen/ms-heka
* nextgen/ms-horizon
* nextgen/ms-ironic-api
* nextgen/ms-ironic-base
* nextgen/ms-ironic-conductor
* nextgen/ms-ironic-inspector
* nextgen/ms-ironic-pxe
* nextgen/ms-keystone
* nextgen/ms-kibana
* nextgen/ms-toolbox
* nextgen/ms-magnum-api
* nextgen/ms-magnum-base
* nextgen/ms-magnum-conductor
* nextgen/ms-manila-api
* nextgen/ms-manila-base
* nextgen/ms-manila-scheduler
* nextgen/ms-manila-share
* nextgen/ms-mariadb
* nextgen/ms-memcached
* nextgen/ms-mistral-api
* nextgen/ms-mistral-base
* nextgen/ms-mistral-engine
* nextgen/ms-mistral-executor
* nextgen/ms-mongodb
* nextgen/ms-murano-api
* nextgen/ms-murano-base
* nextgen/ms-murano-engine
* nextgen/ms-neutron-base
* nextgen/ms-neutron-dhcp-agent
* nextgen/ms-neutron-l3-agent
* nextgen/ms-neutron-linuxbridge-agent
* nextgen/ms-neutron-metadata-agent
* nextgen/ms-neutron-openvswitch-agent
* nextgen/ms-neutron-server
* nextgen/ms-nova-api
* nextgen/ms-nova-base
* nextgen/ms-nova-compute-ironic
* nextgen/ms-nova-compute
* nextgen/ms-nova-conductor
* nextgen/ms-nova-consoleauth
* nextgen/ms-nova-libvirt
* nextgen/ms-nova-network
* nextgen/ms-nova-novncproxy
* nextgen/ms-nova-scheduler
* nextgen/ms-nova-spicehtml5proxy
* nextgen/ms-openvswitch-base
* nextgen/ms-openvswitch-db-server
* nextgen/ms-openvswitch-vswitchd
* nextgen/ms-rabbitmq
* nextgen/ms-sahara-api
* nextgen/ms-sahara-base
* nextgen/ms-sahara-engine
* nextgen/ms-swift-account
* nextgen/ms-swift-base
* nextgen/ms-swift-container
* nextgen/ms-swift-object
* nextgen/ms-swift-proxy-server
* nextgen/ms-swift-rsyncd
* nextgen/ms-tempest
* nextgen/ms-trove-api
* nextgen/ms-trove-base
* nextgen/ms-trove-conductor
* nextgen/ms-trove-guestagent
* nextgen/ms-trove-taskmanager
* nextgen/ms-zaqar
* nextgen/dev-env - Vagrantfile for creating the development environment
* nextgen/puppet-deployment - Puppet mianifests for deploying Kubernetes and all
infra aroud it
* nextgen/project-config - config of NextGen/Microservices repositories
Repositories structure
----------------------
nextgen/microservices
~~~~~~~~~~~~~~~
This repository will be an usual Python package, made from the OpenStack
cookiecutter template[1].
nextgen/ms-ext-config
~~~~~~~~~~~~~~~~~~~~~
This repository will be an usual Python package as well.
nextgen/ms-*
~~~~~~~~~~~~
The structure of this repository will look like:
* *docker/*
* *image.json.j2 / Dockerfile.j2* - template of appc definition or Dockerfile
(TBD), rendered and consumed by microservices CLI
* *config/*
* *<service_name>.conf.j2* - template of config file
* *<service_name>-some-other-config-file.conf.j2* - template of config file
* *service/*
* *<service_name>.yaml.j2* - service definition for Kubernetes
nextgen/dev-env
~~~~~~~~~~~~~~~
The structure of this repository will look like:
* *Vagrantfile* - main Vagrantfile
* *Vagrantfile.custom.example* - example config file for Vagrant
* *provision.sh* - provisioning script
Alternatives
------------
Kolla community is working on spec about running the kolla-kubernetes project.[2]
The only difference between this project and our planned Microservices project is
a single git repository for all Dockerfiles.
Data model impact
-----------------
There is no data model impact.
REST API impact
---------------
There is no REST API impact.
Security impact
---------------
There is no security impact.
Notifications impact
--------------------
There is no notification impact.
Other end user impact
---------------------
There shoudn't be impact on the end user. The experience which currently is
comming from kolla-build and kolla-mesos utilities shouldn't be changed from
user's point of view and we can even achieve a compatibility of configuration
and usage if we want.
However, in addition to the backwards compatibility, we are going to provide
all needed tooling to be able to easy deploy and plug-in any set of services
just by using nextgen/microservies tool.
Performance Impact
------------------
There will be two options regarding the service repositories.
First option is to clone all needed repositories. That will decrease performance,
but not in critical way.
Second option is to specify already cloned repositories. That will not change
anything in performance.
Other deployer impact
---------------------
Deployer will have to be aware of using the multiple git repositories when
deploying NG OpenStack and Kubernetes infra.
Developer impact
----------------
There will be an impact on developer's experience:
* CI will have to use many repositories and check whether the deployment from
the current Code Review + the trunk of the other NextGen repositories is
working.
* In case of cross-repository failures, developers should care about making
appropriate dependencies by Depends-On clause in commits as well as handling
and contributing the changes in the other repositories if needed.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Michal Rostecki
Work Items
----------
* Create the repositories for all OpenStack big tent services, infra services
around them and base layers in fuel-infra Gerrit. The name of these
repositories should follow the following pattern: "ms-<service_name>".
* Move kolla-build tool from openstack/kolla repository to ng-builder repository.
* Move kolla-mesos tool from openstack/kolla-mesos repository to nextgen/microservices
repository.
* Make changes to build and mesos deployment tools to fetch the external git
repositories.
* Move the Dockerfiles from openstack/kolla repository to the appropriate
NextGen repositories.
* Move the configuration files from openstack/kolla and/or openstack/kolla-mesos
to the appropriate NextGen repositories.
Dependencies
============
* Specs about Docker build architecture [3] and layers [4].
Testing
=======
The testing scenarios for NextGen will be similar to what was planned
during our work on kolla-mesos. But of course it should include change
from Mesos to Kubernetes and repositories splitup.
Testing scenarios for kolla-mesos were described on Mirantis wiki[5].
We need to re-write them when implementing this bleuprint.
However, there will be some infrastructure impact due to running tests
separately for each repo.
Documentation Impact
====================
Only parts of documentation describing the kolla-build and kolla-mesos CLI-s
can be preserved with the small changes. Every other part of documentation
should be rewritten from scratch.
References
==========
* https://docs.google.com/document/d/1rtuINpfTvUFc1lJLuMK5LIzGKWwdh8uRUTvOLKnXuPM/edit
* [1] https://github.com/openstack-dev/cookiecutter
* [2] https://review.openstack.org/#/c/304182/
* [3] https://review.fuel-infra.org/#/c/18867/
* [4] https://review.fuel-infra.org/#/c/19028/
* [5] https://mirantis.jira.com/wiki/display/KOLLA/CI+jobs

179
specs/template.rst
View File

@ -1,179 +0,0 @@
=====================================
Example Spec - The title of your epic
=====================================
Include the URL of your Jira Epic.
Introduction paragraph -- why are we doing anything? A single paragraph of
prose that operators can understand. The title and this first paragraph
should be used as the subject line and body of the commit message
respectively.
TODO(slukjanov): detailed notes about spec process.
Some notes about using this template:
* Your spec should be in ReSTructured text, like this template.
* Please wrap text at 79 columns.
.. * The filename in the git repository should match the Epic short name
* Please do not delete any of the sections in this template. If you have
nothing to say for a whole section, just write: None
* For help with syntax, see http://sphinx-doc.org/rest.html
* To test out your formatting, build the docs using tox and see the generated
HTML file in doc/build/html/specs/<path_of_your_file>
.. * If you would like to provide a diagram with your spec, ascii diagrams are
required. http://asciiflow.com/ is a very nice tool to assist with making
ascii diagrams. The reason for this is that the tool used to review specs is
based purely on plain text. Plain text will allow review to proceed without
having to look at additional files which can not be viewed in gerrit. It
will also allow inline feedback on the diagram itself.
Problem description
===================
A detailed description of the problem. What problem is this blueprint
addressing?
Use Cases
---------
What use cases does this address? What impact on actors does this change have?
Ensure you are clear about the actors in each use case: Developer, End User,
Deployer etc.
Proposed change
===============
Here is where you cover the change you propose to make in detail. How do you
propose to solve this problem?
If this is one part of a larger effort make it clear where this piece ends. In
other words, what's the scope of this effort?
At this point, if you would like to just get feedback on if the problem and
proposed change fit in project, you can stop here and post this for review to
get preliminary feedback. If so please say:
"Posting to get preliminary feedback on the scope of this spec."
Alternatives
------------
What other ways could we do this thing? Why aren't we using those? This doesn't
have to be a full literature review, but it should demonstrate that thought has
been put into why the proposed solution is an appropriate one.
TODO(slukjanov) impacts
-----------------------
Implementation
==============
Assignee(s)
-----------
Who is leading the writing of the code? Or is this a blueprint where you're
throwing it out there to see who picks it up?
If more than one person is working on the implementation, please designate the
primary author and contact.
Primary assignee:
<launchpad-id or None>
Other contributors:
<launchpad-id or None>
Work Items
----------
Work items or tasks -- break the feature up into the things that need to be
done to implement it. Those parts might end up being done by different people,
but we're mostly trying to understand the timeline for implementation.
Dependencies
============
* Include specific references to specs and/or blueprints in NextGen, or in
other projects, that this one either depends on or is related to.
.. * If this requires functionality of another project that is not currently
.. used by NextGen (such as the glance v2 API when we previously only
.. required v1), document that fact.
* Does this feature require any new library dependencies or code otherwise not
included in OpenStack? Or does it depend on a specific version of library?
Testing
=======
Please discuss the important scenarios needed to test here, as well as
specific edge cases we should be ensuring work correctly. For each
scenario please specify if this requires specialized hardware, a full
openstack environment, or can be simulated inside the NextGen tree.
Please discuss how the change will be tested. We especially want to know what
tempest tests will be added. It is assumed that unit test coverage will be
added so that doesn't need to be mentioned explicitly, but discussion of why
you think unit tests are sufficient and we don't need to add more tempest
tests would need to be included.
Is this untestable in gate given current limitations (specific hardware /
software configurations available)? If so, are there mitigation plans (3rd
party testing, gate enhancements, etc).
Documentation Impact
====================
Which audiences are affected most by this change, and which documentation
titles on docs.openstack.org should be updated because of this change? Don't
repeat details discussed above, but reference them here in the context of
documentation for multiple audiences. For example, the Operations Guide targets
cloud operators, and the End User Guide would need to be updated if the change
offers a new feature available through the CLI or dashboard. If a config option
changes or is deprecated, note here that the documentation needs to be updated
to reflect this specification's change.
References
==========
Please add any useful references here. You are not required to have any
reference. Moreover, this specification should still make sense when your
references are unavailable. Examples of what you could include are:
* Links to mailing list or IRC discussions
* Links to notes from a summit session
* Links to relevant research, if appropriate
* Related specifications as appropriate (e.g. if it's an EC2 thing, link the
EC2 docs)
* Anything else you feel it is worthwhile to refer to
History
=======
Optional section for Mitaka intended to be used each time the spec
is updated to describe new design, API or any database schema
updated. Useful to let reader understand what's happened along the
time.
.. list-table:: Revisions
:header-rows: 1
* - Release Name
- Description
* - Mitaka
- Introduced

8
test-requirements.txt
View File

@ -1,8 +0,0 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
docutils
hacking>=0.10.0,<0.11
jinja2>=2.8
python-subunit

0
tests/__init__.py
View File

34
tests/test_directories.py
View File

@ -1,34 +0,0 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Copied from openstack/nova-specs
import glob
import os
import testtools
class TestDirectories(testtools.TestCase):
def test_directories(self):
releases = [x.split('/')[1] for x in glob.glob('specs/*/')]
for release in releases:
files = os.listdir("specs/%s/" % release)
valid = ['redirects', 'implemented', 'approved']
for name in files:
if name.startswith('.'):
continue
self.assertIn(name, valid,
"Found unexpected file in "
"'specs/%s', specs should be submitted to "
"'specs/%s/approved'" % (release, release))

129
tests/test_titles.py
View File

@ -1,129 +0,0 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Copied from openstack/nova-specs
import glob
import re
import docutils.core
import testtools
# Used for new sections introduced during a release.
# - "History" introduced in Liberty should be
# mandatory for M.
OPTIONAL_SECTIONS = ("History",)
class TestTitles(testtools.TestCase):
def _get_title(self, section_tree):
section = {
'subtitles': [],
}
for node in section_tree:
if node.tagname == 'title':
section['name'] = node.rawsource
elif node.tagname == 'section':
subsection = self._get_title(node)
section['subtitles'].append(subsection['name'])
return section
def _get_titles(self, spec):
titles = {}
for node in spec:
if node.tagname == 'section':
# Note subsection subtitles are thrown away
section = self._get_title(node)
titles[section['name']] = section['subtitles']
return titles
def _check_titles(self, filename, expect, actual):
missing_sections = [x for x in expect.keys() if (
x not in actual.keys()) and (x not in OPTIONAL_SECTIONS)]
extra_sections = [x for x in actual.keys() if x not in expect.keys()]
msgs = []
if len(missing_sections) > 0:
msgs.append("Missing sections: %s" % missing_sections)
if len(extra_sections) > 0:
msgs.append("Extra sections: %s" % extra_sections)
for section in expect.keys():
missing_subsections = [x for x in expect[section]
if x not in actual.get(section, {})]
# extra subsections are allowed
if len(missing_subsections) > 0:
msgs.append("Section '%s' is missing subsections: %s"
% (section, missing_subsections))
if len(msgs) > 0:
self.fail("While checking '%s':\n %s"
% (filename, "\n ".join(msgs)))
def _check_lines_wrapping(self, tpl, raw):
code_block = False
for i, line in enumerate(raw.split("\n")):
# NOTE(ndipanov): Allow code block lines to be longer than 79 ch
if code_block:
if not line or line.startswith(" "):
continue
else:
code_block = False
if "::" in line:
code_block = True
if "http://" in line or "https://" in line:
continue
# Allow lines which do not contain any whitespace
if re.match("\s*[^\s]+$", line):
continue
self.assertTrue(
len(line) < 80,
msg="%s:%d: Line limited to a maximum of 79 characters." %
(tpl, i + 1))
def _check_no_cr(self, tpl, raw):
matches = re.findall('\r', raw)
self.assertEqual(
len(matches), 0,
"Found %s literal carriage returns in file %s" %
(len(matches), tpl))
def _check_trailing_spaces(self, tpl, raw):
for i, line in enumerate(raw.split("\n")):
trailing_spaces = re.findall(" +$", line)
self.assertEqual(len(trailing_spaces), 0,
"Found trailing spaces on line %s of %s" % (i + 1, tpl))
def test_template(self):
components = [x.split('/')[1] for x in glob.glob('specs/*/')]
self.assertTrue(len(components), "Not able to find spec directories")
for component in components:
with open("specs/%s-template.rst" % component) as f:
template = f.read()
spec = docutils.core.publish_doctree(template)
template_titles = self._get_titles(spec)
files = glob.glob("specs/%s/*/*" % component)
for filename in files:
self.assertTrue(filename.endswith(".rst"),
"spec %s must use 'rst' extension."
% filename)
with open(filename) as f:
data = f.read()
spec = docutils.core.publish_doctree(data)
titles = self._get_titles(spec)
self._check_titles(filename, template_titles, titles)
self._check_lines_wrapping(filename, data)
self._check_no_cr(filename, data)
self._check_trailing_spaces(filename, data)

38
tox.ini
View File

@ -1,38 +0,0 @@
[tox]
minversion = 1.6
envlist = docs,py27,pep8,linters
skipsdist = True
[testenv]
usedevelop = True
setenv = VIRTUAL_ENV={envdir}
install_command = pip install -U {opts} {packages}
deps = -r{toxinidir}/test-requirements.txt
whitelist_externals = find
commands =
find . -type f -name "*.pyc" -delete
python setup.py testr --slowest --testr-args='{posargs}'
[testenv:venv]
commands = {posargs}
[testenv:docs]
commands =
find . -type f -name "*.pyc" -delete
python setup.py build_sphinx
[testenv:pep8]
commands =
flake8 {posargs}
[testenv:linters]
commands =
find . -type f -name "*.pyc" -delete
python setup.py build_sphinx
flake8 {posargs}
python setup.py testr --slowest --testr-args='{posargs}'
[flake8]
ignore = E128
exclude = .venv,.git,.tox,doc,.eggs