Retire the Searchlight project
As announced in openstack-discuss ML[1], Searchlight project is retiring in Wallaby cycle. This commit retires this repository as per process deinfed in project-guide[2]. Anyone would like to maintain it again, please revert back this commit and propose the re-adding of Searchlight to governance. The community wishes to express our thanks and appreciation to all of those who have contributed to the Searchlight project over the years. Depends-On: https://review.opendev.org/c/openstack/project-config/+/764519 Needed-By: https://review.opendev.org/c/openstack/governance/+/764530 [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018637.html [2] https://docs.openstack.org/project-team-guide/repository.html#retiring-a-repository Change-Id: I0c5c9a961cca19113f4d59e37cb16e3bc0ccf8e1
This commit is contained in:
parent
ae76091a3b
commit
7331642231
@ -1,7 +0,0 @@
|
||||
[run]
|
||||
branch = True
|
||||
source = searchlight
|
||||
omit = searchlight/tests/*
|
||||
|
||||
[report]
|
||||
ignore_errors = True
|
44
.gitignore
vendored
44
.gitignore
vendored
@ -1,44 +0,0 @@
|
||||
*.pyc
|
||||
*.log
|
||||
.searchlight-venv
|
||||
.venv
|
||||
.stestr/
|
||||
.tox
|
||||
.coverage*
|
||||
cover/*
|
||||
covhtml
|
||||
nosetests.xml
|
||||
coverage.xml
|
||||
searchlight.sqlite
|
||||
AUTHORS
|
||||
ChangeLog
|
||||
build
|
||||
.DS_Store
|
||||
dist
|
||||
*.egg
|
||||
searchlight.egg-info
|
||||
tests.sqlite
|
||||
searchlight/versioninfo
|
||||
etc/searchlight.conf.sample
|
||||
!/.coveragerc
|
||||
|
||||
#Files created by releasenotes build
|
||||
releasenotes/build
|
||||
# Swap files range from .saa to .swp
|
||||
*.s[a-w][a-p]
|
||||
|
||||
# generated policy file
|
||||
etc/searchlight/policy.yaml.sample
|
||||
|
||||
# Files created by doc build
|
||||
doc/source/api
|
||||
_static/
|
||||
|
||||
# IDE files
|
||||
.project
|
||||
.pydevproject
|
||||
.idea
|
||||
.e4p
|
||||
.eric5project/
|
||||
.issues/
|
||||
.ropeproject
|
@ -1,4 +0,0 @@
|
||||
[DEFAULT]
|
||||
test_path=./searchlight/tests
|
||||
top_dir=.
|
||||
|
@ -1,8 +0,0 @@
|
||||
- project:
|
||||
templates:
|
||||
- openstack-python3-victoria-jobs
|
||||
- openstack-cover-jobs
|
||||
- publish-openstack-docs-pti
|
||||
- check-requirements
|
||||
- openstack-lower-constraints-jobs
|
||||
- release-notes-jobs-python3
|
@ -1,21 +0,0 @@
|
||||
If you would like to contribute to the development of OpenStack,
|
||||
you must follow the steps documented at:
|
||||
|
||||
https://docs.openstack.org/infra/manual/developers.html#development-workflow
|
||||
|
||||
Once those steps have been completed, changes to OpenStack
|
||||
should be submitted for review via the Gerrit tool, following
|
||||
the workflow documented at:
|
||||
|
||||
https://docs.openstack.org/infra/manual/developers.html#development-workflow
|
||||
|
||||
Pull requests submitted through GitHub will be ignored.
|
||||
|
||||
Bugs should be filed on Storyboard, not GitHub:
|
||||
|
||||
https://storyboard.openstack.org/#!/project_group/searchlight
|
||||
|
||||
Further documentation on feature request and bug report processes may be
|
||||
found here:
|
||||
|
||||
https://docs.openstack.org/searchlight/latest/contributor/feature-requests-bugs.html
|
19
HACKING.rst
19
HACKING.rst
@ -1,19 +0,0 @@
|
||||
Searchlight Style Commandments
|
||||
==============================
|
||||
|
||||
- Step 1: Read the OpenStack Style Commandments
|
||||
https://docs.openstack.org/hacking/latest
|
||||
- Step 2: Read on
|
||||
|
||||
Searchlight Specific Commandments
|
||||
---------------------------------
|
||||
|
||||
- [SL316] Change assertTrue(isinstance(A, B)) by optimal assert like
|
||||
assertIsInstance(A, B)
|
||||
- [SL317] Change assertEqual(type(A), B) by optimal assert like
|
||||
assertIsInstance(A, B)
|
||||
- [SL318] Change assertEqual(A, None) or assertEqual(None, A) by optimal assert like
|
||||
assertIsNone(A)
|
||||
- [SL319] Validate that logs are not translated
|
||||
- [SL327] Prevent use of deprecated contextlib.nested
|
||||
- [SL343] Check for common double word typos
|
176
LICENSE
176
LICENSE
@ -1,176 +0,0 @@
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
72
README.rst
72
README.rst
@ -1,66 +1,10 @@
|
||||
========================
|
||||
Team and repository tags
|
||||
========================
|
||||
This project is no longer maintained.
|
||||
|
||||
.. image:: https://governance.openstack.org/tc/badges/searchlight.svg
|
||||
:target: https://governance.openstack.org/tc/reference/tags/index.html
|
||||
:alt: Searchlight is an official OpenStack project as indicated by
|
||||
the "project:official" badge
|
||||
.. NOTE(rosmaita): the alt text above will have to be updated when
|
||||
additional tags are asserted for Searchlight. (The SVG in the
|
||||
governance repo is updated automatically.)
|
||||
The contents of this repository are still available in the Git
|
||||
source code management system. To see the contents of this
|
||||
repository before it reached its end of life, please check out the
|
||||
previous commit with "git checkout HEAD^1".
|
||||
|
||||
.. Change things from this point on
|
||||
|
||||
========
|
||||
Welcome!
|
||||
========
|
||||
|
||||
The Searchlight project provides indexing and search capabilities across
|
||||
OpenStack resources. Its goal is to achieve high performance and flexible
|
||||
querying combined with near real-time indexing. It uses Elasticsearch, a
|
||||
real-time distributed indexing and search engine built on Apache Lucene, but
|
||||
adds OpenStack authentication and Role Based Access Control to provide
|
||||
appropriate protection of data.
|
||||
|
||||
Documentations
|
||||
==============
|
||||
|
||||
* Installation instruction and user guides:
|
||||
https://docs.openstack.org/searchlight/latest
|
||||
* Searchlight wiki:
|
||||
https://wiki.openstack.org/wiki/Searchlight
|
||||
* Release notes: https://docs.openstack.org/releasenotes/searchlight/
|
||||
* Specs: https://specs.openstack.org/openstack/searchlight-specs/
|
||||
|
||||
Code
|
||||
====
|
||||
|
||||
Searchlight code is available in the following repositories:
|
||||
|
||||
* Searchlight API and Listener:
|
||||
https://opendev.org/openstack/searchlight
|
||||
* Searchlight Python client:
|
||||
https://opendev.org/openstack/python-searchlightclient
|
||||
* Searchlight Horizon UI:
|
||||
https://opendev.org/openstack/searchlight-ui
|
||||
|
||||
Bugs
|
||||
====
|
||||
|
||||
Please report bugs at https://storyboard.openstack.org/#!/project_group/searchlight
|
||||
|
||||
License
|
||||
=======
|
||||
|
||||
Apache Licence 2.0
|
||||
|
||||
Other Resources
|
||||
===============
|
||||
|
||||
Use the following resources to learn more:
|
||||
|
||||
* Storyboard project: https://storyboard.openstack.org/#!/project_group/searchlight
|
||||
* Send mail to openstack-discuss@lists.openstack.org with [Searchlight]
|
||||
tag for help and hacking of Searchlight
|
||||
* IRC channel: #openstack-searchlight
|
||||
For any further questions, please email
|
||||
openstack-discuss@lists.openstack.org or join #openstack-dev on
|
||||
Freenode.
|
||||
|
@ -1,209 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
#
|
||||
# searchlight api-ref build config file, copied from:
|
||||
# nova documentation build configuration file, created by
|
||||
# sphinx-quickstart on Sat May 1 15:17:47 2010.
|
||||
#
|
||||
# This file is execfile()d with the current directory set to
|
||||
# its containing dir.
|
||||
#
|
||||
# Note that not all possible configuration values are present in this
|
||||
# autogenerated file.
|
||||
#
|
||||
# All configuration values have a default; values that are commented out
|
||||
# serve to show the default.
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
html_theme = 'openstackdocs'
|
||||
html_theme_options = {
|
||||
"sidebar_mode": "toc",
|
||||
}
|
||||
|
||||
extensions = [
|
||||
'os_api_ref',
|
||||
'openstackdocstheme'
|
||||
]
|
||||
|
||||
# openstackdocstheme options
|
||||
openstackdocs_repo_name = 'openstack/searchlight'
|
||||
openstackdocs_bug_project = 'searchlight'
|
||||
openstackdocs_bug_tag = ''
|
||||
openstackdocs_auto_name = False
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
sys.path.insert(0, os.path.abspath('../../'))
|
||||
sys.path.insert(0, os.path.abspath('../'))
|
||||
sys.path.insert(0, os.path.abspath('./'))
|
||||
|
||||
# -- General configuration ----------------------------------------------------
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
|
||||
|
||||
# The suffix of source filenames.
|
||||
source_suffix = '.rst'
|
||||
|
||||
# The encoding of source files.
|
||||
#
|
||||
# source_encoding = 'utf-8'
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = 'index'
|
||||
|
||||
# General information about the project.
|
||||
project = u'Search Service API Reference'
|
||||
copyright = u'2015-present, OpenStack Foundation'
|
||||
|
||||
# for a list of supported languages.
|
||||
#
|
||||
# language = None
|
||||
|
||||
# There are two options for replacing |today|: either, you set today to some
|
||||
# non-false value, then it is used:
|
||||
# today = ''
|
||||
# Else, today_fmt is used as the format for a strftime call.
|
||||
# today_fmt = '%B %d, %Y'
|
||||
|
||||
# The reST default role (used for this markup: `text`) to use
|
||||
# for all documents.
|
||||
# default_role = None
|
||||
|
||||
# If true, '()' will be appended to :func: etc. cross-reference text.
|
||||
# add_function_parentheses = True
|
||||
|
||||
# If true, the current module name will be prepended to all description
|
||||
# unit titles (such as .. function::).
|
||||
add_module_names = False
|
||||
|
||||
# If true, sectionauthor and moduleauthor directives will be shown in the
|
||||
# output. They are ignored by default.
|
||||
show_authors = False
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = 'native'
|
||||
|
||||
# -- Options for man page output ----------------------------------------------
|
||||
|
||||
# Grouping the document tree for man pages.
|
||||
# List of tuples 'sourcefile', 'target', u'title', u'Authors name', 'manual'
|
||||
|
||||
|
||||
# -- Options for HTML output --------------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. Major themes that come with
|
||||
# Sphinx are currently 'default' and 'sphinxdoc'.
|
||||
# html_theme_path = ["."]
|
||||
# html_theme = '_theme'
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
# html_theme_options = {}
|
||||
|
||||
# Add any paths that contain custom themes here, relative to this directory.
|
||||
# html_theme_path = []
|
||||
|
||||
# The name for this set of Sphinx documents. If None, it defaults to
|
||||
# "<project> v<release> documentation".
|
||||
# html_title = None
|
||||
|
||||
# A shorter title for the navigation bar. Default is the same as html_title.
|
||||
# html_short_title = None
|
||||
|
||||
# The name of an image file (relative to this directory) to place at the top
|
||||
# of the sidebar.
|
||||
# html_logo = None
|
||||
|
||||
# The name of an image file (within the static path) to use as favicon of the
|
||||
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
|
||||
# pixels large.
|
||||
# html_favicon = None
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
# html_static_path = ['_static']
|
||||
|
||||
# If true, SmartyPants will be used to convert quotes and dashes to
|
||||
# typographically correct entities.
|
||||
# html_use_smartypants = True
|
||||
|
||||
# Custom sidebar templates, maps document names to template names.
|
||||
# html_sidebars = {}
|
||||
|
||||
# Additional templates that should be rendered to pages, maps page names to
|
||||
# template names.
|
||||
# html_additional_pages = {}
|
||||
|
||||
# If false, no module index is generated.
|
||||
# html_use_modindex = True
|
||||
|
||||
# If false, no index is generated.
|
||||
# html_use_index = True
|
||||
|
||||
# If true, the index is split into individual pages for each letter.
|
||||
# html_split_index = False
|
||||
|
||||
# If true, links to the reST sources are added to the pages.
|
||||
# html_show_sourcelink = True
|
||||
|
||||
# If true, an OpenSearch description file will be output, and all pages will
|
||||
# contain a <link> tag referring to it. The value of this option must be the
|
||||
# base URL from which the finished HTML is served.
|
||||
# html_use_opensearch = ''
|
||||
|
||||
# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
|
||||
# html_file_suffix = ''
|
||||
|
||||
# Output file base name for HTML help builder.
|
||||
htmlhelp_basename = 'searchlightdoc'
|
||||
|
||||
|
||||
# -- Options for LaTeX output -------------------------------------------------
|
||||
|
||||
# The paper size ('letter' or 'a4').
|
||||
# latex_paper_size = 'letter'
|
||||
|
||||
# The font size ('10pt', '11pt' or '12pt').
|
||||
# latex_font_size = '10pt'
|
||||
|
||||
# Grouping the document tree into LaTeX files. List of tuples
|
||||
# (source start file, target name, title, author, documentclass
|
||||
# [howto/manual]).
|
||||
latex_documents = [
|
||||
('index', 'Searchlight.tex', u'OpenStack Search Service API Documentation',
|
||||
u'OpenStack Foundation', 'manual'),
|
||||
]
|
||||
|
||||
# The name of an image file (relative to this directory) to place at the top of
|
||||
# the title page.
|
||||
# latex_logo = None
|
||||
|
||||
# For "manual" documents, if this is true, then toplevel headings are parts,
|
||||
# not chapters.
|
||||
# latex_use_parts = False
|
||||
|
||||
# Additional stuff for the LaTeX preamble.
|
||||
# latex_preamble = ''
|
||||
|
||||
# Documents to append as an appendix to all manuals.
|
||||
# latex_appendices = []
|
||||
|
||||
# If false, no module index is generated.
|
||||
# latex_use_modindex = True
|
@ -1,55 +0,0 @@
|
||||
200:
|
||||
default: |
|
||||
The response contains an entity corresponding to the requested resource.
|
||||
201:
|
||||
default: |
|
||||
The request has been fullfilled and a new resource created.
|
||||
202:
|
||||
default: |
|
||||
The request has been accepted for processing, but the processing has not
|
||||
been completed.
|
||||
204:
|
||||
default: |
|
||||
The request has been processed successfully.
|
||||
400:
|
||||
default: |
|
||||
The server cannot (or will not) process the request due to a client error.
|
||||
informal: |
|
||||
Your bad ... something is wrong with your request.
|
||||
401:
|
||||
default: |
|
||||
You have either forgotten to provide credentials, or the credentials you
|
||||
provided are malformed or expired.
|
||||
403:
|
||||
default: |
|
||||
You do not have sufficient permissions to perform the requested action.
|
||||
404:
|
||||
default: |
|
||||
The resource you have requested cannot be found.
|
||||
409:
|
||||
default: |
|
||||
The request you have made cannot be processed because it conflicts with
|
||||
the current state of the resource.
|
||||
410:
|
||||
default: |
|
||||
The requested resource is no longer available at the server and no
|
||||
forwarding address is known.
|
||||
413:
|
||||
default: |
|
||||
The payload you have submitted is too large given the configuration of the
|
||||
server.
|
||||
415:
|
||||
default: |
|
||||
The server is refusing to service the request because the entity of the
|
||||
request is in a format not supported by the requested resource for the
|
||||
requested method.
|
||||
500:
|
||||
default: |
|
||||
The server encountered an unexpected condition which prevented it from
|
||||
fulfilling the request.
|
||||
informal: |
|
||||
Our bad ... something unfortunate happened on the server.
|
||||
503:
|
||||
default: |
|
||||
The server is currently unable to handle the request due to a temporary
|
||||
overloading or maintenance of the server.
|
@ -1,25 +0,0 @@
|
||||
..
|
||||
Copyright 2016 OpenStack Foundation
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
:tocdepth: 3
|
||||
|
||||
==================
|
||||
Search Service API
|
||||
==================
|
||||
|
||||
.. rest_expand_all::
|
||||
|
||||
.. include:: searchlight-v1.inc
|
@ -1,6 +0,0 @@
|
||||
{
|
||||
"query": {
|
||||
"match_all": {}
|
||||
},
|
||||
"all_projects": true
|
||||
}
|
@ -1,11 +0,0 @@
|
||||
{
|
||||
"query": {
|
||||
"match_all": {}
|
||||
},
|
||||
"type": ["OS::Glance::Image"],
|
||||
"limit": 0,
|
||||
"aggregations": {
|
||||
"name": {"terms": {"field": "name"}},
|
||||
"container_format": {"terms": {"field": "container_format"}}
|
||||
}
|
||||
}
|
@ -1,64 +0,0 @@
|
||||
{
|
||||
"_shards": {
|
||||
"successful": 1,
|
||||
"failed": 0,
|
||||
"total": 1
|
||||
},
|
||||
"aggregations": {
|
||||
"container_format": {
|
||||
"buckets": [
|
||||
{
|
||||
"doc_count": 101,
|
||||
"key": "bare"
|
||||
},
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "aki"
|
||||
},
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "ami"
|
||||
},
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "ari"
|
||||
}
|
||||
],
|
||||
"doc_count_error_upper_bound": 0,
|
||||
"sum_other_doc_count": 0
|
||||
},
|
||||
"name": {
|
||||
"buckets": [
|
||||
{
|
||||
"doc_count": 100,
|
||||
"key": "image"
|
||||
},
|
||||
{
|
||||
"doc_count": 3,
|
||||
"key": "0.3.4"
|
||||
},
|
||||
{
|
||||
"doc_count": 3,
|
||||
"key": "cirros"
|
||||
},
|
||||
{
|
||||
"doc_count": 3,
|
||||
"key": "uec"
|
||||
},
|
||||
{
|
||||
"doc_count": 3,
|
||||
"key": "x86_64"
|
||||
}
|
||||
],
|
||||
"doc_count_error_upper_bound": 0,
|
||||
"sum_other_doc_count": 0
|
||||
}
|
||||
},
|
||||
"hits": {
|
||||
"hits": [],
|
||||
"max_score": 0.0,
|
||||
"total": 104
|
||||
},
|
||||
"took": 5,
|
||||
"timed_out": false
|
||||
}
|
@ -1,5 +0,0 @@
|
||||
{
|
||||
"query": {
|
||||
"match_all": {}
|
||||
}
|
||||
}
|
@ -1,61 +0,0 @@
|
||||
{
|
||||
"_shards": {
|
||||
"failed": 0,
|
||||
"successful": 2,
|
||||
"total": 2
|
||||
},
|
||||
"hits": {
|
||||
"hits": [
|
||||
{
|
||||
"_id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"_index": "searchlight",
|
||||
"_type": "OS::Glance::Image",
|
||||
"_score": 1.0,
|
||||
"_source": {
|
||||
"id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"members": [],
|
||||
"name": "cirros-0.3.2-x86_64-uec",
|
||||
"owner": "d95b27da6e9f4acc9a8031918e443e04",
|
||||
"visibility": "public",
|
||||
"...": "..."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "OS::Software::DBMS",
|
||||
"_index": "searchlight",
|
||||
"_type": "OS::Glance::Metadef",
|
||||
"_score": 1.0,
|
||||
"_source": {
|
||||
"description": "A database is an ...",
|
||||
"display_name": "Database Software",
|
||||
"namespace": "OS::Software::DBMS",
|
||||
"objects": [
|
||||
{
|
||||
"description": "PostgreSQL, often simply 'Postgres' ...",
|
||||
"name": "PostgreSQL",
|
||||
"properties": [
|
||||
{
|
||||
"default": "5432",
|
||||
"description": "Specifies the TCP/IP port...",
|
||||
"property": "sw_database_postgresql_listen_port",
|
||||
"...": "..."
|
||||
},
|
||||
{ "...": "..." }
|
||||
]
|
||||
}
|
||||
],
|
||||
"tags": [
|
||||
{
|
||||
"name": "Database"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{ "...": "..." }
|
||||
],
|
||||
"max_score": 1.0,
|
||||
"total": 8
|
||||
},
|
||||
"timed_out": false,
|
||||
"took": 1
|
||||
}
|
@ -1,25 +0,0 @@
|
||||
{
|
||||
"type": "OS::Glance::Image",
|
||||
"query": {
|
||||
"bool" : {
|
||||
"must" : [
|
||||
{ "term" : { "status" : "active" } },
|
||||
{ "term" : { "visibility" : "public" } }
|
||||
],
|
||||
"must_not" : [
|
||||
{ "range" : { "min_ram" : { "gt" : 4096 } } },
|
||||
{ "terms" : { "disk_format" : ["aki", "ari"] } }
|
||||
],
|
||||
"should" : [
|
||||
{ "match" : { "name" : "cirros" } },
|
||||
{ "match" : { "tag" : "cirros" } }
|
||||
],
|
||||
"minimum_should_match" : 1
|
||||
}
|
||||
},
|
||||
"highlight": {
|
||||
"fields": {
|
||||
"*":{}
|
||||
}
|
||||
}
|
||||
}
|
@ -1,93 +0,0 @@
|
||||
{
|
||||
"hits": {
|
||||
"hits": [
|
||||
{
|
||||
"_type": "OS::Glance::Image",
|
||||
"_source": {
|
||||
"status": "active",
|
||||
"created_at": "2016-08-25T17:32:41Z",
|
||||
"project_id": "98c05407d80c405aa6efa01b279385ce",
|
||||
"name": "cirros-0.3.4-x86_64-uec",
|
||||
"tags": [],
|
||||
"image_type": "image",
|
||||
"checksum": "eb9139e4942121f22bbc2afc0400b2a4",
|
||||
"min_ram": 0,
|
||||
"ramdisk_id": "f310fb51-29dc-47bf-8cbd-3ea1170fb4e3",
|
||||
"disk_format": "ami",
|
||||
"updated_at": "2016-08-25T17:32:42Z",
|
||||
"visibility": "public",
|
||||
"kernel_id": "34ddf1d2-b588-4a9a-a3d3-344768c04bfd",
|
||||
"protected": false,
|
||||
"members": [],
|
||||
"container_format": "ami",
|
||||
"min_disk": 0,
|
||||
"owner": "98c05407d80c405aa6efa01b279385ce",
|
||||
"virtual_size": null,
|
||||
"id": "23fa3971-3ed8-435a-a3c6-8addea5153aa",
|
||||
"size": 25165824
|
||||
},
|
||||
"_score": 3.3642778,
|
||||
"_index": "searchlight-2016_08_25_18_48_30",
|
||||
"highlight": {
|
||||
"status": [
|
||||
"<em>active</em>"
|
||||
],
|
||||
"name": [
|
||||
"<em>cirros</em>-0.3.4-x86_64-uec"
|
||||
],
|
||||
"visibility": [
|
||||
"<em>public</em>"
|
||||
]
|
||||
},
|
||||
"_id": "23fa3971-3ed8-435a-a3c6-8addea5153aa"
|
||||
},
|
||||
{
|
||||
"_type": "OS::Glance::Image",
|
||||
"_source": {
|
||||
"status": "active",
|
||||
"created_at": "2016-08-25T17:32:34Z",
|
||||
"project_id": "98c05407d80c405aa6efa01b279385ce",
|
||||
"name": "cirros-0.3.4-x86_64-disk",
|
||||
"tags": [],
|
||||
"image_type": "image",
|
||||
"checksum": "ee1eca47dc88f4879d8a229cc70a07c6",
|
||||
"min_ram": 0,
|
||||
"disk_format": "qcow2",
|
||||
"updated_at": "2016-08-25T17:32:35Z",
|
||||
"visibility": "public",
|
||||
"owner": "98c05407d80c405aa6efa01b279385ce",
|
||||
"protected": false,
|
||||
"members": [],
|
||||
"container_format": "bare",
|
||||
"min_disk": 0,
|
||||
"virtual_size": null,
|
||||
"id": "f34de38c-7a5c-4b00-8015-70a7a63480e8",
|
||||
"size": 13287936
|
||||
},
|
||||
"_score": 3.3642778,
|
||||
"_index": "searchlight-2016_08_25_18_48_30",
|
||||
"highlight": {
|
||||
"status": [
|
||||
"<em>active</em>"
|
||||
],
|
||||
"name": [
|
||||
"<em>cirros</em>-0.3.4-x86_64-disk"
|
||||
],
|
||||
"visibility": [
|
||||
"<em>public</em>"
|
||||
]
|
||||
},
|
||||
"_id": "f34de38c-7a5c-4b00-8015-70a7a63480e8"
|
||||
}
|
||||
],
|
||||
"total": 2,
|
||||
"max_score": 3.3642778
|
||||
},
|
||||
"_shards": {
|
||||
"successful": 1,
|
||||
"failed": 0,
|
||||
"total": 1
|
||||
},
|
||||
"took": 1,
|
||||
"timed_out": false
|
||||
}
|
@ -1,9 +0,0 @@
|
||||
{
|
||||
"type": "OS::Glance::Image",
|
||||
"query": {
|
||||
"multi_match": {
|
||||
"query":"cirros",
|
||||
"fields": ["_all"]
|
||||
}
|
||||
}
|
||||
}
|
@ -1,45 +0,0 @@
|
||||
{
|
||||
"_shards": {
|
||||
"failed": 0,
|
||||
"successful": 2,
|
||||
"total": 2
|
||||
},
|
||||
"hits": {
|
||||
"hits": [
|
||||
{
|
||||
"_id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"_index": "searchlight",
|
||||
"_type": "OS::Glance::Image",
|
||||
"_score": 1.0,
|
||||
"_source": {
|
||||
"id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"members": [],
|
||||
"name": "cirros-0.3.2-x86_64-uec",
|
||||
"owner": "d95b27da6e9f4acc9a8031918e443e04",
|
||||
"visibility": "public",
|
||||
"...": "..."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "15c2501d-07bd-4eb1-b15f-77ef739e5419",
|
||||
"_index": "searchlight",
|
||||
"_type": "OS::Glance::Image",
|
||||
"_score": 1.0,
|
||||
"_source": {
|
||||
"id": "15c2501d-07bd-4eb1-b15f-77ef739e5419",
|
||||
"description": "Extra special cirros image",
|
||||
"members": [],
|
||||
"name": "Simply Sensational",
|
||||
"owner": "d95b27da6e9f4acc9a8031918e443e04",
|
||||
"visibility": "public",
|
||||
"...": "..."
|
||||
}
|
||||
},
|
||||
{ "...": "..." }
|
||||
],
|
||||
"max_score": 1.0,
|
||||
"total": 8
|
||||
},
|
||||
"timed_out": false,
|
||||
"took": 1
|
||||
}
|
@ -1,6 +0,0 @@
|
||||
{
|
||||
"query": {
|
||||
"match_all": {}
|
||||
},
|
||||
"type": ["OS::Glance::Image", "OS::Glance::Metadef"]
|
||||
}
|
@ -1,8 +0,0 @@
|
||||
{
|
||||
"type": "OS::Glance::Image",
|
||||
"query": {
|
||||
"match_phrase": {
|
||||
"description": "Preconfigured with a schmaltz to optimize the foobar"
|
||||
}
|
||||
}
|
||||
}
|
@ -1,45 +0,0 @@
|
||||
{
|
||||
"hits": {
|
||||
"hits": [
|
||||
{
|
||||
"_score": 12.945373,
|
||||
"_type": "OS::Glance::Image",
|
||||
"_id": "7a99b58d-0b70-4138-8086-ab0cc4bd1ba3",
|
||||
"_source": {
|
||||
"container_format": "bare",
|
||||
"min_ram": 0,
|
||||
"updated_at": "2016-08-25T17:54:02Z",
|
||||
"cim_pasd_processorarchitecture": "Power",
|
||||
"owner": "158776c9face41a7a3026d6c1cae686a",
|
||||
"id": "7a99b58d-0b70-4138-8086-ab0cc4bd1ba3",
|
||||
"size": 5969,
|
||||
"image_type": "image",
|
||||
"disk_format": "raw",
|
||||
"project_id": "158776c9face41a7a3026d6c1cae686a",
|
||||
"status": "active",
|
||||
"description": "Preconfigured with a schmaltz to optimize the foobar",
|
||||
"tags": [],
|
||||
"hw_cpu_policy": "shared",
|
||||
"visibility": "public",
|
||||
"members": [],
|
||||
"min_disk": 0,
|
||||
"virtual_size": null,
|
||||
"name": "My Awesome Image",
|
||||
"checksum": null,
|
||||
"created_at": "2016-08-25T17:54:02Z",
|
||||
"protected": false
|
||||
},
|
||||
"_index": "searchlight-2016_08_25_18_48_30"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"max_score": 12.945373
|
||||
},
|
||||
"_shards": {
|
||||
"successful": 1,
|
||||
"failed": 0,
|
||||
"total": 1
|
||||
},
|
||||
"took": 4,
|
||||
"timed_out": false
|
||||
}
|
@ -1,6 +0,0 @@
|
||||
{
|
||||
"query": {
|
||||
"match_all": {}
|
||||
},
|
||||
"type": "OS::Glance::Image"
|
||||
}
|
@ -1,44 +0,0 @@
|
||||
{
|
||||
"_shards": {
|
||||
"failed": 0,
|
||||
"successful": 2,
|
||||
"total": 2
|
||||
},
|
||||
"hits": {
|
||||
"hits": [
|
||||
{
|
||||
"_id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"_index": "searchlight",
|
||||
"_type": "OS::Glance::Image",
|
||||
"_score": 1.0,
|
||||
"_source": {
|
||||
"id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"members": [],
|
||||
"name": "cirros-0.3.2-x86_64-uec",
|
||||
"owner": "d95b27da6e9f4acc9a8031918e443e04",
|
||||
"visibility": "public",
|
||||
"...": "..."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "a701f610-b162-45ac-a5a0-69e08aba8687",
|
||||
"_index": "searchlight",
|
||||
"_type": "OS::Glance::Image",
|
||||
"_score": 1.0,
|
||||
"_source": {
|
||||
"id": "a701f610-b162-45ac-a5a0-69e08aba8687",
|
||||
"members": [],
|
||||
"name": "Docker container image",
|
||||
"owner": "d95b27da6e9f4acc9a8031918e443e04",
|
||||
"visibility": "public",
|
||||
"...": "..."
|
||||
}
|
||||
},
|
||||
{ "...": "..." }
|
||||
],
|
||||
"max_score": 1.0,
|
||||
"total": 8
|
||||
},
|
||||
"timed_out": false,
|
||||
"took": 1
|
||||
}
|
@ -1,78 +0,0 @@
|
||||
{
|
||||
"OS::Glance::Image": {
|
||||
"doc_count": 50,
|
||||
"facets": [
|
||||
{
|
||||
"name": "status",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "created_at",
|
||||
"type": "date"
|
||||
},
|
||||
{
|
||||
"name": "virtual_size",
|
||||
"type": "long"
|
||||
},
|
||||
{
|
||||
"name": "name",
|
||||
"type": "string",
|
||||
"facet_field": "name.raw"
|
||||
},
|
||||
{ "...": "..." }
|
||||
]
|
||||
},
|
||||
"OS::Glance::Metadef": {
|
||||
"doc_count": 25,
|
||||
"facets": [
|
||||
{
|
||||
"name": "objects.description",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "objects.properties.description",
|
||||
"type": "string"
|
||||
},
|
||||
{ "...": "..." }
|
||||
]
|
||||
},
|
||||
"OS::Nova::Server": {
|
||||
"doc_count": 100,
|
||||
"facets": [
|
||||
{
|
||||
"name": "status",
|
||||
"options": [
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "ACTIVE"
|
||||
}
|
||||
],
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "OS-EXT-SRV-ATTR:host",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "name",
|
||||
"type": "string",
|
||||
"facet_field": "name.raw"
|
||||
},
|
||||
{
|
||||
"name": "image.id",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "OS-EXT-AZ:availability_zone",
|
||||
"options": [
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "nova"
|
||||
}
|
||||
],
|
||||
"type": "string"
|
||||
},
|
||||
{ "...": "..." }
|
||||
]
|
||||
}
|
||||
}
|
@ -1,90 +0,0 @@
|
||||
{
|
||||
"OS::Glance::Image": {
|
||||
"doc_count": 50,
|
||||
"facets": [
|
||||
{
|
||||
"name": "status",
|
||||
"type": "string",
|
||||
"options": [
|
||||
{
|
||||
"key": "queued",
|
||||
"doc_count": 47
|
||||
},
|
||||
{
|
||||
"key": "active",
|
||||
"doc_count": 3
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "created_at",
|
||||
"type": "date"
|
||||
},
|
||||
{
|
||||
"name": "virtual_size",
|
||||
"type": "long"
|
||||
},
|
||||
{
|
||||
"name": "name",
|
||||
"type": "string",
|
||||
"facet_field": "name.raw"
|
||||
},
|
||||
{ "...": "..." }
|
||||
]
|
||||
},
|
||||
"OS::Glance::Metadef": {
|
||||
"doc_count": 25,
|
||||
"facets": [
|
||||
{
|
||||
"name": "objects.description",
|
||||
"nested": true,
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "objects.properties.description",
|
||||
"nested": true,
|
||||
"type": "string"
|
||||
},
|
||||
{ "...": "..." }
|
||||
]
|
||||
},
|
||||
"OS::Nova::Server": {
|
||||
"doc_count": 100,
|
||||
"facets": [
|
||||
{
|
||||
"name": "status",
|
||||
"options": [
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "ACTIVE"
|
||||
}
|
||||
],
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "OS-EXT-SRV-ATTR:host",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "name",
|
||||
"type": "string",
|
||||
"facet_field": "name.raw"
|
||||
},
|
||||
{
|
||||
"name": "image.id",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "OS-EXT-AZ:availability_zone",
|
||||
"options": [
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "nova"
|
||||
}
|
||||
],
|
||||
"type": "string"
|
||||
},
|
||||
{ "...": "..." }
|
||||
]
|
||||
}
|
||||
}
|
@ -1,11 +0,0 @@
|
||||
{
|
||||
"OS::Nova::Server": {
|
||||
"doc_count": 7
|
||||
},
|
||||
"OS::Neutron::Net": {
|
||||
"doc_count": 19
|
||||
},
|
||||
"OS::Glance::Image": {
|
||||
"doc_count": 68
|
||||
}
|
||||
}
|
@ -1,27 +0,0 @@
|
||||
{
|
||||
"OS::Nova::Server": {
|
||||
"doc_count": 100,
|
||||
"facets": [
|
||||
{
|
||||
"name": "status",
|
||||
"options": [
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "ACTIVE"
|
||||
}
|
||||
],
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "OS-EXT-SRV-ATTR:host",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "name",
|
||||
"type": "string",
|
||||
"facet_field": "name.raw"
|
||||
},
|
||||
{ "...": "..." }
|
||||
]
|
||||
}
|
||||
}
|
@ -1,14 +0,0 @@
|
||||
{
|
||||
"plugins": [
|
||||
{
|
||||
"type": "OS::Glance::Image",
|
||||
"alias-indexing": "searchlight-listener",
|
||||
"alias-searching": "searchlight-search"
|
||||
},
|
||||
{
|
||||
"type": "OS::Glance::Metadef",
|
||||
"alias-indexing": "searchlight-listener",
|
||||
"alias-searching": "searchlight-search"
|
||||
}
|
||||
]
|
||||
}
|
@ -1,170 +0,0 @@
|
||||
# template for entries
|
||||
#name:
|
||||
# description: |
|
||||
# yada yada yada
|
||||
# in: {header, path, query, body}
|
||||
# required: {true, false}
|
||||
# type: {JSON datatype: number, string, boolean, array, object, null}
|
||||
#
|
||||
# Note: entries must be in alphabetical order within each section
|
||||
|
||||
# variables in header
|
||||
Content-type-json:
|
||||
description: |
|
||||
Mime type of the request body. Must be ``application/json``.
|
||||
in: header
|
||||
required: true
|
||||
type: string
|
||||
|
||||
# variables in path
|
||||
|
||||
# variables in query
|
||||
include-in-query:
|
||||
description: |
|
||||
Disable showing the facet fields. Display count only.
|
||||
The default value is ``true``.
|
||||
in: query
|
||||
required: false
|
||||
type: boolean
|
||||
options-in-query:
|
||||
description: |
|
||||
Disable aggregating facet fields. Provides a performance
|
||||
improvement. The default value is ``false``.
|
||||
in: query
|
||||
required: false
|
||||
type: boolean
|
||||
type-in-query:
|
||||
description: |
|
||||
The resource type whose facets you'd like to see listed.
|
||||
in: query
|
||||
required: false
|
||||
type: string
|
||||
|
||||
# variables in body
|
||||
all-projects:
|
||||
description: |
|
||||
Indicates that an administator wants to include all resources for
|
||||
all projects in the search. The default is ``false``.
|
||||
in: body
|
||||
required: false
|
||||
type: boolean
|
||||
es-aggregate-obj:
|
||||
description: |
|
||||
A JSON object containing the aggregation results. It contains the following
|
||||
fields:
|
||||
|
||||
- ``buckets`` - An array of aggregated results (defaults to the top ten
|
||||
terms). There is a bucket for each term. Each element of the array
|
||||
contains the following fields:
|
||||
|
||||
- ``doc_count`` - Number of times this term appears.
|
||||
- ``key`` - The value of this term.
|
||||
|
||||
- ``doc_count_error_upper_bound`` - The upper bound on the error when
|
||||
compiling the document count, needed since the count is approximate.
|
||||
- ``sum_other_doc_count`` - Since the number of elements in the bucket
|
||||
array may be capped (default is ten), the number of all other document
|
||||
counts not included in buckets.
|
||||
in: body
|
||||
required: true
|
||||
type: object
|
||||
es-hits-obj:
|
||||
description: |
|
||||
A JSON object containing the search results. It contains the following
|
||||
fields:
|
||||
|
||||
- ``hits`` - An array of search results (defaults to the first 25
|
||||
documents). Each element of the array contains the following fields:
|
||||
|
||||
- ``_id`` - Uniquely identifies the resource within its OpenStack context
|
||||
(for instance, Glance images use their Image ID)
|
||||
- ``_index`` - The service to which the resource belongs (for example,
|
||||
searchlight)
|
||||
- ``_score`` - The relevance of a given hit. By default, this is the
|
||||
field upon which results are sorted
|
||||
- ``_source`` - The document originally indexed, containing all data
|
||||
associated with the OpenStack resource. This field is a map,
|
||||
where each key is a field whose value may be a scalar value, a list,
|
||||
a nested object, or a list of nested objects.
|
||||
- ``_type`` - The OpenStack resource type of the document (for example,
|
||||
"OS::Neutron::Net", "OS::Glance::Image")
|
||||
- ``_parent`` - *(optional)* When the resource type has a child
|
||||
relationship to another resource type, the id of the specific parent
|
||||
resource will appear here.
|
||||
- ``_routing`` - *(optional)* Included when an Elasticsearch routing
|
||||
for the document is explicitly specified.
|
||||
|
||||
- ``max_score`` - The highest ``_score`` of any document that matches the
|
||||
query
|
||||
- ``total`` - The total number of documents matching the search criteria
|
||||
in: body
|
||||
required: true
|
||||
type: object
|
||||
es-query:
|
||||
description: |
|
||||
A JSON object containing the requested query, expressed in the
|
||||
`Elasticsearch Query DSL <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html>`_.
|
||||
in: body
|
||||
required: true
|
||||
type: object
|
||||
es-shards-obj:
|
||||
description: |
|
||||
A JSON object containing the following fields:
|
||||
|
||||
- ``failed`` - Number of shards in which the search could not be performed
|
||||
- ``successful`` - Number of shards in which the search was successful
|
||||
- ``total`` - The total number of shards involved in processing the query
|
||||
in: body
|
||||
required: true
|
||||
type: object
|
||||
es-timed-out-field:
|
||||
description: |
|
||||
Indicates whether the search timed out or not.
|
||||
in: body
|
||||
required: true
|
||||
type: boolean
|
||||
es-took-field:
|
||||
description: |
|
||||
The time in milliseconds it took for Elasticsearch to execute the search.
|
||||
in: body
|
||||
required: true
|
||||
type: number
|
||||
facet-list:
|
||||
description: |
|
||||
A JSON object pairing resource type names with a list of facets
|
||||
supported for that resource type. Each facet object in the list
|
||||
contains the following fields:
|
||||
|
||||
- name - the name of the facet
|
||||
- type - the datatype of the facet
|
||||
- facet_field - (optional, may be available for some string fields)
|
||||
may be used to do an exact term match
|
||||
in: body
|
||||
required: true
|
||||
type: object
|
||||
multi_match-in-body:
|
||||
description: |
|
||||
A JSON object containing a ``query`` field whose value is an Elasticsearch
|
||||
query string.
|
||||
in: body
|
||||
required: true
|
||||
type: string or array of strings
|
||||
plugin-list:
|
||||
description: |
|
||||
List of plugins. Each element of the list contains the following fields:
|
||||
|
||||
- ``alias-indexing`` - the alias this plugin uses to refer to the
|
||||
Elasticsearch index in which documents are created
|
||||
- ``alias-searching`` - the alias this plugin uses to refer to the
|
||||
Elasticsearch index used for querying documents created by this plugin
|
||||
- ``type`` - the document type associated with this plugin
|
||||
in: body
|
||||
required: true
|
||||
type: array
|
||||
type-in-body:
|
||||
description: |
|
||||
The name of the resource type or types to which the search should be
|
||||
restricted.
|
||||
in: body
|
||||
required: true
|
||||
type: string or array of strings
|
@ -1,699 +0,0 @@
|
||||
.. -*- rst -*-
|
||||
|
||||
Search
|
||||
******
|
||||
|
||||
Provides indexing and search capabilities across OpenStack resources.
|
||||
|
||||
General information
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The OpenStack Searchlight project provides a Search API that presents an
|
||||
interface for querying about the various resources available in an OpenStack
|
||||
cloud. Searchlight creates an index (using open source Elasticsearch
|
||||
technology) and keeps it updated. Additionally, Searchlight's Search API
|
||||
requires authentication and respects the Role Based Access Control defined by
|
||||
each OpenStack service. This gives end users access to the powerful searching
|
||||
facilities provided by Elasticsearch, but operators can rest assured that end
|
||||
users won't have access to information they wouldn't be able to find in the
|
||||
APIs exposed by each of the OpenStack services.
|
||||
|
||||
The Search API allows you to construct detailed queries using the
|
||||
`Elasticsearch Query Domain Specific Language
|
||||
<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html>`_.
|
||||
That's great if you are an Elasticsearch expert (or want to become one), but is
|
||||
probably beyond what most end users need in order to use the Search API
|
||||
effectively. So we'll provide a wide range of useful query examples below (see
|
||||
`Create a general search`_).
|
||||
|
||||
To make effective use of the Search API, you need to know a few things about
|
||||
OpenStack resource types, how Searchlight stores data about OpenStack
|
||||
resources, and something about Searchlight plugins.
|
||||
|
||||
OpenStack resource types
|
||||
------------------------
|
||||
|
||||
An OpenStack cloud offers users many resources, for example, servers, images,
|
||||
networks. In order to eliminate ambiguity (for example, sometimes a resource
|
||||
you create with Nova is called a "server", sometimes it's called an
|
||||
"instance"), a standard vocabulary has developed based around that used by the
|
||||
OpenStack `Heat <http://docs.openstack.org/developer/heat/>`_ project.
|
||||
|
||||
Heat identifies specific *resource types* that are assigned identifiers of the
|
||||
form
|
||||
|
||||
``OS::{project_name}::{resource}``
|
||||
|
||||
where ``{project_name}`` and ``{resource}`` are replaced by an actual project
|
||||
name and resource name. For example, a Nova server (or "instance") has the
|
||||
resource type identifier ``OS::Nova::Server``.
|
||||
|
||||
A list of OpenStack resource type identifiers is maintained by the Heat project
|
||||
at `OpenStack Resource Types
|
||||
<http://docs.openstack.org/developer/heat/template_guide/openstack.html>`_.
|
||||
|
||||
To summarize, you need to know the identifier for each OpenStack resource type
|
||||
you're interested in so that you can efficiently search for those resources.
|
||||
|
||||
Searchlight data
|
||||
----------------
|
||||
|
||||
Searchlight creates a *document* to represent each individual item (server,
|
||||
image, network) in an OpenStack cloud. These documents are stored in an
|
||||
Elasticsearch *index*. When you make a query using the Search API, the
|
||||
documents appropriate to your query (and your RBAC roles) are returned to you.
|
||||
|
||||
To facilitate searching, each *document* has a *document type*. Searchlight
|
||||
uses the OpenStack *resource type* (described in the previous section) as the
|
||||
*document type*. For example, a virtual machine image stored in Glance would
|
||||
be represented by a *document* with the *document type* ``OS::Glance::Image``.
|
||||
|
||||
The particular *document* that corresponds to an individual OpenStack resource
|
||||
requires a unique identifier. It just so happens that each OpenStack service
|
||||
already assigns a unique identifier to each individual resource it controls, so
|
||||
that's what Searchlight assigns as the *document ID*. For example, if you have
|
||||
an image in Glance with ID ``997ec7e8-a46b-4ab7-a1d4-788e06f52abe``, then that
|
||||
image will be represented in the *index* by a *document* with a *document ID*
|
||||
of ``997ec7e8-a46b-4ab7-a1d4-788e06f52abe``.
|
||||
|
||||
The information in the document will be the same content that would be
|
||||
available to you if you queried the native API that handles that particular
|
||||
resource type. For example, a document with *document type*
|
||||
``OS::Glance::Image`` will contain the same information about an image as you
|
||||
would receive if you queried the Images API for information about that image.
|
||||
The advantage to using Searchlight, of course, is that you have access to
|
||||
advanced searching facilities provided by Elasticsearch.
|
||||
|
||||
Searchlight index
|
||||
-----------------
|
||||
|
||||
In the previous section, you learned that each document stored in Elasticsearch
|
||||
is uniquely identified by (a) what *index* it's in, (b) what *document type* it
|
||||
has, and (c) what its *document ID* is. You already know how to determine the
|
||||
*document type* (it's the resource type of the OpenStack resource you're
|
||||
interested in) and the *document ID* (it's the same as the identifier assigned
|
||||
to the resource by the OpenStack service that manages that resource). So
|
||||
what's left is to figure out what index to use.
|
||||
|
||||
Due to careful engineering by top professionals, that's pretty easy.
|
||||
Searchlight maintains its own index named 'searchlight'. Further, the Search
|
||||
API automatically uses this index, so you don't need to specify it. We only
|
||||
mention it here because you'll see its name in some Elasticsearch responses
|
||||
that you'll receive from the Search API, so it's worth knowing what it is.
|
||||
|
||||
One final point. Searchlight accomplishes its magic by using *plugins* that
|
||||
understand and index particular OpenStack resource types. So in order for a
|
||||
resource type to be accessible via the Search API, there must be a Searchlight
|
||||
plugin installed to index that kind of resource. Not all OpenStack resources
|
||||
currently have Searchlight plugins associated with them, and not all OpenStack
|
||||
deployers may have installed all available plugins. You can determine what's
|
||||
available in a particular cloud by using the `List plugins`_ call.
|
||||
|
||||
.. TODO(rosmaita): include authentication section? what are other api-ref
|
||||
doing?
|
||||
|
||||
Authentication
|
||||
--------------
|
||||
|
||||
Like other OpenStack APIs, the Search API depends on Keystone and the
|
||||
OpenStack Identity API to handle authentication. Hence, you must obtain an
|
||||
authentication token from Keystone and pass it to Searchlight in API
|
||||
requests with the ``X-Auth-Token`` header.
|
||||
|
||||
List plugins
|
||||
~~~~~~~~~~~~
|
||||
|
||||
.. rest_method:: GET /v1/search/plugins
|
||||
|
||||
List the supported plugins.
|
||||
|
||||
In order to index OpenStack resources, Searchlight requires a *plugin* specific
|
||||
to each resource type. Hence, the only resources that can be indexed in a
|
||||
particular OpenStack cloud are those for which (a) a plugin exists, and (b) the
|
||||
plugin is enabled by the cloud operator. You use this call to determine what
|
||||
document types you can expect to be able to search in a particular cloud.
|
||||
|
||||
.. rest_status_code:: success http-codes.yaml
|
||||
|
||||
- 200
|
||||
|
||||
.. rest_status_code:: error http-codes.yaml
|
||||
|
||||
- 400
|
||||
- 401
|
||||
- 403
|
||||
- 500
|
||||
|
||||
Request
|
||||
-------
|
||||
|
||||
There are no request parameters, and this call does not take a request body.
|
||||
|
||||
Response
|
||||
--------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- plugins: plugin-list
|
||||
|
||||
Response Example
|
||||
----------------
|
||||
|
||||
.. literalinclude:: samples/list-plugins-response.json
|
||||
:language: json
|
||||
|
||||
List facets
|
||||
~~~~~~~~~~~
|
||||
|
||||
.. rest_method:: GET /v1/search/facets
|
||||
|
||||
List the supported facets.
|
||||
|
||||
For each registered resource type, Searchlight can provide a list of *field
|
||||
names* and *values* present for those fields.
|
||||
|
||||
- Which fields are returned and whether values are listed is determined by
|
||||
each plugin.
|
||||
- Some fields or values may only be listed for administrative users.
|
||||
- For some string fields, ``facet_field`` may be included in the result. If
|
||||
present, it can be used to do an exact term match against facet options.
|
||||
|
||||
.. rest_status_code:: success http-codes.yaml
|
||||
|
||||
- 200
|
||||
|
||||
.. rest_status_code:: error http-codes.yaml
|
||||
|
||||
- 400
|
||||
- 401
|
||||
- 403
|
||||
- 500
|
||||
|
||||
Request
|
||||
-------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- exclude_options: options-in-query
|
||||
- include_fields: include-in-query
|
||||
- type: type-in-query
|
||||
|
||||
.. NOTE(rosmaita): we can force extra space below the parameters
|
||||
table by inserting raw html here, but it would be better to
|
||||
put up a patch on the os-api-ref project to modify the style in CSS
|
||||
|
||||
.. TODO(rosmaita): remove this example
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<p></p>
|
||||
|
||||
This call does not take a request body.
|
||||
|
||||
Response
|
||||
--------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- facets: facet-list
|
||||
|
||||
Response Example
|
||||
----------------
|
||||
|
||||
.. TODO(rosmaita): decide what to do about admin-only field in
|
||||
the example response
|
||||
|
||||
.. literalinclude:: samples/list-facets-all-response.json
|
||||
:language: json
|
||||
|
||||
Response Example (including ``exclude_options`` query parameter)
|
||||
----------------------------------------------------------------
|
||||
|
||||
Assume this request: ``GET /v1/search/facets?exclude_options=true``
|
||||
|
||||
.. literalinclude:: samples/list-facets-all-response-exclude-options.json
|
||||
:language: json
|
||||
|
||||
Response Example (including ``type`` query parameter)
|
||||
-----------------------------------------------------
|
||||
|
||||
Assume this request: ``GET /v1/search/facets?type=OS::Nova::Server``
|
||||
|
||||
.. literalinclude:: samples/list-facets-type-response.json
|
||||
:language: json
|
||||
|
||||
Response Example (including ``include_fields`` query parameter)
|
||||
---------------------------------------------------------------
|
||||
|
||||
Assume this request: ``GET /v1/search/facets?include_fields=false``
|
||||
|
||||
.. literalinclude:: samples/list-facets-include-fields-response.json
|
||||
:language: json
|
||||
|
||||
.. _general-search:
|
||||
|
||||
Create a general search
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. rest_method:: POST /v1/search
|
||||
|
||||
Submit a search request.
|
||||
|
||||
.. rest_status_code:: success http-codes.yaml
|
||||
|
||||
- 200
|
||||
|
||||
.. rest_status_code:: error http-codes.yaml
|
||||
|
||||
- 400
|
||||
- 401
|
||||
- 403
|
||||
- 500
|
||||
|
||||
Request
|
||||
-------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- Content-type: Content-type-json
|
||||
- query: es-query
|
||||
|
||||
Request Example
|
||||
---------------
|
||||
|
||||
.. literalinclude:: samples/create-search-all-request.json
|
||||
:language: json
|
||||
|
||||
Response
|
||||
--------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- _shards: es-shards-obj
|
||||
- hits: es-hits-obj
|
||||
- timed_out: es-timed-out-field
|
||||
- took: es-took-field
|
||||
|
||||
Response Example
|
||||
----------------
|
||||
|
||||
.. literalinclude:: samples/create-search-all-response.json
|
||||
:language: json
|
||||
|
||||
Create a type restricted search
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. rest_method:: POST /v1/search
|
||||
|
||||
Request a type restricted search.
|
||||
|
||||
By including a ``type`` field in your request, you can restrict
|
||||
the results to a particular resource type or types.
|
||||
|
||||
The ``type`` field can take either a string value (the name of
|
||||
a particular resource type) or an array of values (where each
|
||||
value is the name of a resource type).
|
||||
|
||||
.. rest_status_code:: success http-codes.yaml
|
||||
|
||||
- 200
|
||||
|
||||
.. rest_status_code:: error http-codes.yaml
|
||||
|
||||
- 400
|
||||
- 401
|
||||
- 403
|
||||
- 500
|
||||
|
||||
Request
|
||||
-------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- Content-type: Content-type-json
|
||||
- query: es-query
|
||||
- type: type-in-body
|
||||
|
||||
Request Example (single resource type)
|
||||
--------------------------------------
|
||||
|
||||
This example illustrates a request for a single resource type, Glance images.
|
||||
|
||||
.. literalinclude:: samples/create-search-single-type-request.json
|
||||
:language: json
|
||||
|
||||
Request Example (multiple resource types)
|
||||
-----------------------------------------
|
||||
|
||||
This example illustrates a request for multiple resource types, Glance images
|
||||
and Glance metadefs.
|
||||
|
||||
.. literalinclude:: samples/create-search-multiple-type-request.json
|
||||
:language: json
|
||||
|
||||
Response
|
||||
--------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- _shards: es-shards-obj
|
||||
- hits: es-hits-obj
|
||||
- timed_out: es-timed-out-field
|
||||
- took: es-took-field
|
||||
|
||||
Response Example (single resource type)
|
||||
---------------------------------------
|
||||
|
||||
This example illustrates the response to a request for a single resource type,
|
||||
Glance images (``OS::Glance::Image``).
|
||||
|
||||
.. literalinclude:: samples/create-search-single-type-response.json
|
||||
:language: json
|
||||
|
||||
Response Example (multiple resource types)
|
||||
------------------------------------------
|
||||
|
||||
This example illustrates the resonse to a request for multiple resource types,
|
||||
Glance images (``OS::Glance::Image``) and Glance metadefs
|
||||
(``OS::Glance::Metadef``).
|
||||
|
||||
.. literalinclude:: samples/create-search-all-response.json
|
||||
:language: json
|
||||
|
||||
Create an administrative search
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. rest_method:: POST /v1/search
|
||||
|
||||
Submit an administrative search request.
|
||||
|
||||
By default, the results a user receives in a search response are limited
|
||||
to public resources and those resources owned by (or shared with) the user's
|
||||
project (or "tenant").
|
||||
|
||||
Administrators have the option to view all resources in the index by passing
|
||||
the ``all_projects`` field in the search request body.
|
||||
|
||||
.. rest_status_code:: success http-codes.yaml
|
||||
|
||||
- 200
|
||||
|
||||
.. rest_status_code:: error http-codes.yaml
|
||||
|
||||
- 400
|
||||
- 401
|
||||
- 403
|
||||
- 500
|
||||
|
||||
Request
|
||||
-------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- Content-type: Content-type-json
|
||||
- query: es-query
|
||||
- all_projects: all-projects
|
||||
|
||||
Request Example
|
||||
---------------
|
||||
|
||||
.. literalinclude:: samples/create-search-admin-request.json
|
||||
:language: json
|
||||
|
||||
Response
|
||||
--------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- _shards: es-shards-obj
|
||||
- hits: es-hits-obj
|
||||
- timed_out: es-timed-out-field
|
||||
- took: es-took-field
|
||||
|
||||
Response Example
|
||||
----------------
|
||||
|
||||
.. literalinclude:: samples/create-search-all-response.json
|
||||
:language: json
|
||||
|
||||
Create a free text search
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. rest_method:: POST /v1/search
|
||||
|
||||
Submit a free text search request.
|
||||
|
||||
.. rest_status_code:: success http-codes.yaml
|
||||
|
||||
- 200
|
||||
|
||||
.. rest_status_code:: error http-codes.yaml
|
||||
|
||||
- 400
|
||||
- 401
|
||||
- 403
|
||||
- 500
|
||||
|
||||
Request
|
||||
-------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- Content-type: Content-type-json
|
||||
- type: type-in-body
|
||||
- multi_match: multi_match-in-body
|
||||
|
||||
Request Example
|
||||
---------------
|
||||
|
||||
Suppose you want to find all images that have the word "cirros" in some field,
|
||||
whether the in the name, tags, or any other image properties.
|
||||
|
||||
.. literalinclude:: samples/create-search-free-text-request.json
|
||||
:language: json
|
||||
|
||||
Response
|
||||
--------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- _shards: es-shards-obj
|
||||
- hits: es-hits-obj
|
||||
- timed_out: es-timed-out-field
|
||||
- took: es-took-field
|
||||
|
||||
Response Example
|
||||
----------------
|
||||
|
||||
.. literalinclude:: samples/create-search-free-text-response.json
|
||||
:language: json
|
||||
|
||||
Create a phrase search
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. rest_method:: POST /v1/search
|
||||
|
||||
Submit a search request that looks for a phrase in a particular field.
|
||||
|
||||
A common scenario is that you have custom metadata on some resources describing
|
||||
the resource in natural language. For example, you may have set a property
|
||||
named ``description`` on your images, where the value is a natural language
|
||||
sentence describing the recommended uses for that image. You can query the
|
||||
Search API for responses that match that phrase in that field.
|
||||
|
||||
.. rest_status_code:: success http-codes.yaml
|
||||
|
||||
- 200
|
||||
|
||||
.. rest_status_code:: error http-codes.yaml
|
||||
|
||||
- 400
|
||||
- 401
|
||||
- 403
|
||||
- 500
|
||||
|
||||
Request
|
||||
-------
|
||||
|
||||
Suppose you use a ``description`` property on images as described above, and
|
||||
you are interested in finding any images that are described as "Preconfigured
|
||||
with a schmaltz to optimize the foobar."
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- Content-type: Content-type-json
|
||||
- query: es-query
|
||||
|
||||
Request Example
|
||||
---------------
|
||||
|
||||
.. literalinclude:: samples/create-search-phrase-field-request.json
|
||||
:language: json
|
||||
|
||||
Response
|
||||
--------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- _shards: es-shards-obj
|
||||
- hits: es-hits-obj
|
||||
- timed_out: es-timed-out-field
|
||||
- took: es-took-field
|
||||
|
||||
|
||||
Response Example
|
||||
----------------
|
||||
|
||||
.. literalinclude:: samples/create-search-phrase-field-response.json
|
||||
|
||||
|
||||
Create a compound boolean search
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. rest_method:: POST /v1/search
|
||||
|
||||
Submit a compound boolean search request.
|
||||
|
||||
.. rest_status_code:: success http-codes.yaml
|
||||
|
||||
- 200
|
||||
|
||||
.. rest_status_code:: error http-codes.yaml
|
||||
|
||||
- 400
|
||||
- 401
|
||||
- 403
|
||||
- 500
|
||||
|
||||
Request
|
||||
-------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- Content-type: Content-type-json
|
||||
- query: es-query
|
||||
|
||||
Request Example
|
||||
---------------
|
||||
|
||||
Suppose you want to find all the images that meet the following set of
|
||||
conditions:
|
||||
|
||||
- The image must be *public* and *active*. (This is a *must clause*.)
|
||||
- The image *cannot require more than 4096 RAM*. (This is a *must_not clause
|
||||
with a range query*.)
|
||||
- The image must *not* be ``aki`` or ``ari``. (This is a *must_not clause*)
|
||||
- Either the *name* or the *tags* should contain "cirros". (This is a *should
|
||||
clause with minimum match of 1*.)
|
||||
- In the results, indicate ("highlight") why these results were included in the
|
||||
request.
|
||||
|
||||
.. literalinclude:: samples/create-search-complex-boolean-request.json
|
||||
:language: json
|
||||
|
||||
Response
|
||||
--------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- _shards: es-shards-obj
|
||||
- hits: es-hits-obj
|
||||
- timed_out: es-timed-out-field
|
||||
- took: es-took-field
|
||||
|
||||
Response Example
|
||||
----------------
|
||||
|
||||
.. literalinclude:: samples/create-search-complex-boolean-response.json
|
||||
|
||||
|
||||
Create an aggregated search
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. rest_method:: POST /v1/search
|
||||
|
||||
Submit an aggregated search request.
|
||||
|
||||
.. rest_status_code:: success http-codes.yaml
|
||||
|
||||
- 200
|
||||
|
||||
.. rest_status_code:: error http-codes.yaml
|
||||
|
||||
- 400
|
||||
- 401
|
||||
- 403
|
||||
- 500
|
||||
|
||||
Request
|
||||
-------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- Content-type: Content-type-json
|
||||
- query: es-query
|
||||
|
||||
Request Example
|
||||
---------------
|
||||
|
||||
Suppose you want to find a count of the different names and container_formats
|
||||
in all images.
|
||||
|
||||
.. literalinclude:: samples/create-search-aggregation-request.json
|
||||
:language: json
|
||||
|
||||
Response
|
||||
--------
|
||||
|
||||
.. rest_parameters:: search-parameters.yaml
|
||||
|
||||
- _shards: es-shards-obj
|
||||
- hits: es-hits-obj
|
||||
- aggregations: es-aggregate-obj
|
||||
- timed_out: es-timed-out-field
|
||||
- took: es-took-field
|
||||
|
||||
Response Example
|
||||
----------------
|
||||
|
||||
.. literalinclude:: samples/create-search-aggregation-response.json
|
||||
|
||||
|
||||
Notes
|
||||
~~~~~
|
||||
|
||||
The index you see in responses is not a literal ElasticSearch index.
|
||||
Here are a few points to keep in mind:
|
||||
|
||||
- For architectural reasons (primarily to allow zero-downtime re-indexing),
|
||||
Searchlight doesn't refer to the index directly, but instead uses *index
|
||||
aliases*. The key impact this has on you as a consumer of the API is that
|
||||
you'll see references to aliases, instead of directly to the index, in
|
||||
some Search API responses.
|
||||
- All accesses to ElasticSearch are done through aliases and not directly
|
||||
through an index. For more details on aliases see the `List plugins`_
|
||||
call. The actual indices behind the aliases are transient, since a new
|
||||
one is created whenever Searchlight is re-indexed.
|
||||
- By default, Searchlight uses a single index named 'searchlight'. An
|
||||
operator, however, may choose to use a different name. Further, an
|
||||
operator may choose to have different resource types indexed into
|
||||
different indices so that, for example, "OS::Cinder::Volume" can have one
|
||||
Elasticsearch index, "OS::Neutron::Net" can have a different Elasticsearch
|
||||
index and "OS::Neutron::SecurityGroup" can have a third separate
|
||||
ElasticSearch index. When this is done, the aliases exposed via the
|
||||
Searchlight API will also change accordingly.
|
||||
|
||||
|
||||
References
|
||||
~~~~~~~~~~
|
||||
|
||||
- `Bool Query
|
||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html>`_
|
||||
- `Range Query
|
||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html>`_
|
||||
- `Highlighting
|
||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-highlighting.html>`_
|
||||
- `Compound Queries
|
||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/compound-queries.html>`_
|
||||
- `Aggregate Queries
|
||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html>`_
|
@ -1,2 +0,0 @@
|
||||
default-jre [!platform:ubuntu-bionic]
|
||||
openjdk-8-jre [platform:ubuntu-bionic]
|
@ -1,30 +0,0 @@
|
||||
FROM ubuntu:18.04
|
||||
MAINTAINER saphi070@gmail.com
|
||||
|
||||
RUN apt update
|
||||
RUN apt install -y python-dev libssl-dev python-pip libxml2-dev \
|
||||
libxslt-dev git git-review libffi-dev gettext \
|
||||
python-tox python-memcache crudini
|
||||
|
||||
### Clone searchlight source code
|
||||
RUN apt install wget -y
|
||||
WORKDIR /home
|
||||
RUN git clone https://opendev.org/openstack/searchlight
|
||||
WORKDIR /home/searchlight
|
||||
|
||||
### Install Searchlight
|
||||
RUN pip install -r requirements.txt
|
||||
RUN pip install -r elasticsearch5.txt
|
||||
RUN python setup.py install
|
||||
RUN oslo-config-generator --config-file etc/oslo-config-generator/searchlight.conf
|
||||
RUN cp etc/searchlight.conf.sample /etc/searchlight/searchlight.conf
|
||||
RUN mkdir /etc/searchlight/
|
||||
|
||||
### Copy Configure files
|
||||
RUN cp etc/api-paste.ini /etc/searchlight/
|
||||
# COPY searchlight.conf /etc/searchlight/
|
||||
COPY entrypoint.sh /home/
|
||||
RUN chmod 775 /home/entrypoint.sh
|
||||
EXPOSE 9393
|
||||
WORKDIR /home
|
||||
ENTRYPOINT ["./entrypoint.sh"]
|
@ -1,76 +0,0 @@
|
||||
Running Searchlight in containers
|
||||
=================================
|
||||
|
||||
This contrib includes Dockerfile, docker-compose file and some configurations to run
|
||||
Searchlight in Docker containers.
|
||||
|
||||
Requirements
|
||||
------------
|
||||
|
||||
Follow guides in [1], [2] to install docker and docker-compose
|
||||
|
||||
Usage
|
||||
-----
|
||||
|
||||
- Build docker image
|
||||
|
||||
::
|
||||
|
||||
docker-compose -f docker-compose.example.yml build
|
||||
|
||||
- Adjust the authentication variables in your `docker-compose.example.yml`.
|
||||
|
||||
Example
|
||||
|
||||
::
|
||||
|
||||
version: "3"
|
||||
services:
|
||||
searchlight-api:
|
||||
build: .
|
||||
ports:
|
||||
- 9393:9393
|
||||
environment:
|
||||
PROCESS: api
|
||||
AUTH_URL: http://192.168.53.31:5000/v3
|
||||
TRANSPORT_URL: rabbitmq://openstack:openstack@192.168.53.31
|
||||
SEARCHLIGHT_PASS: openstack
|
||||
ELASTICSEARCH_HOST: elasticsearch:9200
|
||||
|
||||
searchlight-listener:
|
||||
build: .
|
||||
environment:
|
||||
PROCESS: listener
|
||||
AUTH_URL: http://192.168.53.31:5000/v3
|
||||
TRANSPORT_URL: rabbitmq://openstack:openstack@192.168.53.31
|
||||
SEARCHLIGHT_PASS: openstack
|
||||
ELASTICSEARCH_HOST: elasticsearch:9200
|
||||
|
||||
elasticsearch:
|
||||
image: elasticsearch:5.6
|
||||
|
||||
|
||||
- Running
|
||||
|
||||
::
|
||||
|
||||
docker-compose -f docker-compose.example.yml up -
|
||||
|
||||
Environment Variables
|
||||
---------------------
|
||||
|
||||
- TRANSPORT_URL: RabbitMQ URL. Example: rabbitmq://openstack:openstack@192.168.53.31
|
||||
- REGION_NAME: Openstack Region Name. Example: RegionOne
|
||||
- AUTH_URL: Keystone Auth URL. Example: http://192.168.53.31:5000
|
||||
- SEARCHLIGHT_USER: Username of searchlight. Example: searchlight
|
||||
- SEARCHLIGHT_PASS: Password of searchlight. Example: openstack
|
||||
- PROJECT_NAME: Name of project for searchlight. Example: service
|
||||
- DOMAIN_NAME: Name of domain for searchlight user and project: Example: default
|
||||
- ELASTICSEARCH_HOST: Host and port of Elasticsearch. Example: 192.168.53.31:9200
|
||||
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
- [1] Install docker on Ubuntu: https://docs.docker.com/install/linux/docker-ce/ubuntu/
|
||||
- [2] Install docker-compose: https://docs.docker.com/compose/install/
|
@ -1,24 +0,0 @@
|
||||
version: "3"
|
||||
services:
|
||||
searchlight-api:
|
||||
build: .
|
||||
ports:
|
||||
- 9393:9393
|
||||
environment:
|
||||
PROCESS: api
|
||||
AUTH_URL: http://192.168.53.31:5000/v3
|
||||
TRANSPORT_URL: rabbitmq://openstack:openstack@192.168.53.31
|
||||
SEARCHLIGHT_PASS: openstack
|
||||
ELASTICSEARCH_HOST: elasticsearch:9200
|
||||
|
||||
searchlight-listener:
|
||||
build: .
|
||||
environment:
|
||||
PROCESS: listener
|
||||
AUTH_URL: http://192.168.53.31:5000/v3
|
||||
TRANSPORT_URL: rabbitmq://openstack:openstack@192.168.53.31
|
||||
SEARCHLIGHT_PASS: openstack
|
||||
ELASTICSEARCH_HOST: elasticsearch:9200
|
||||
|
||||
elasticsearch:
|
||||
image: elasticsearch:5.6
|
@ -1,52 +0,0 @@
|
||||
#!/bin/bash
|
||||
echo "Configure Searhclight"
|
||||
|
||||
searchlight_conf=/etc/searchlight/searchlight.conf
|
||||
|
||||
DOMAIN_NAME=${DOMAIN_NAME:-default}
|
||||
PROJECT_NAME=${PROJECT_NAME:-service}
|
||||
SEARCHLIGHT_USER=${SEARCHLIGHT_USER:-searchlight}
|
||||
SEARCHLIGHT_PASS=${SEARCHLIGHT_PASS:-openstack}
|
||||
|
||||
|
||||
crudini --set $searchlight_conf DEFAULT transport_url $TRANSPORT_URL
|
||||
|
||||
crudini --set $searchlight_conf service_credentials os_region_name $REGION_NAME
|
||||
crudini --set $searchlight_conf service_credentials auth_url $AUTH_URL
|
||||
crudini --set $searchlight_conf service_credentials username $SEARCHLIGHT_USER
|
||||
crudini --set $searchlight_conf service_credentials password $SEARCHLIGHT_PASS
|
||||
crudini --set $searchlight_conf service_credentials project_name $PROJECT_NAME
|
||||
crudini --set $searchlight_conf service_credentials user_domain_name $DOMAIN_NAME
|
||||
crudini --set $searchlight_conf service_credentials project_domain_name $DOMAIN_NAME
|
||||
|
||||
crudini --set $searchlight_conf keystone_authtoken auth_url $AUTH_URL
|
||||
crudini --set $searchlight_conf keystone_authtoken auth_type password
|
||||
crudini --set $searchlight_conf keystone_authtoken project_domain_name $DOMAIN_NAME
|
||||
crudini --set $searchlight_conf keystone_authtoken user_domain_name $DOMAIN_NAME
|
||||
crudini --set $searchlight_conf keystone_authtoken project_name $PROJECT_NAME
|
||||
crudini --set $searchlight_conf keystone_authtoken username $SEARCHLIGHT_USER
|
||||
crudini --set $searchlight_conf keystone_authtoken password $SEARCHLIGHT_PASS
|
||||
|
||||
crudini --set $searchlight_conf elasticsearch hosts $ELASTICSEARCH_HOST
|
||||
|
||||
|
||||
if [ ! -f /opt/searchlight-synced ];then
|
||||
echo "Sync index to elasticsearch"
|
||||
while true;
|
||||
do
|
||||
curl -s $ELASTICSEARCH_HOST > /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "Starting sync index to elasticsearch"
|
||||
searchlight-manage index sync --force
|
||||
break
|
||||
fi
|
||||
done
|
||||
touch /opt/searchlight-synced
|
||||
fi
|
||||
|
||||
echo "Starting searchlight"
|
||||
|
||||
if [ ! -d /var/log/searchlight ]; then
|
||||
mkdir /var/log/searchlight
|
||||
fi
|
||||
searchlight-$PROCESS --log-file=/var/log/searchlight/searchlight-$PROCESS --config-file $searchlight_conf
|
@ -1,70 +0,0 @@
|
||||
[DEFAULT]
|
||||
verbose = true
|
||||
transport_url = rabbit://openstack:openstack@rabbitmq
|
||||
|
||||
[service_credentials]
|
||||
os_region_name = RegionOne
|
||||
auth_url = https://keystone:35357
|
||||
password = openstack
|
||||
project_name = service
|
||||
user_domain_name = Default
|
||||
project_domain_name = Default
|
||||
username = searchlight
|
||||
auth_type = password
|
||||
|
||||
[paste_deploy]
|
||||
flavor = keystone
|
||||
|
||||
[keystone_authtoken]
|
||||
auth_version = 3
|
||||
auth_url = https://keystone:5000
|
||||
auth_type = password
|
||||
project_domain_name = default
|
||||
user_domain_name = default
|
||||
project_name = service
|
||||
username = searchlight
|
||||
password = openstack
|
||||
|
||||
[elasticsearch]
|
||||
index_settings = number_of_shards:1,number_of_replicas:0
|
||||
hosts = elasticsearch:9200
|
||||
|
||||
[service_credentials:nova]
|
||||
compute_api_version = 2.1
|
||||
|
||||
[resource_plugin:os_nova_server]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_nova_hypervisor]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_nova_flavor]
|
||||
notifications_topics_exchanges = versioned_notifications,nova
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_nova_servergroup]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_glance_image]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_glance_metadef]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_searchlight_volume]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_searchlight_snapshot]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_net]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_port]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_floatingip]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_security_group]
|
||||
enabled = True
|
77
contrib/vagrant/Vagrantfile
vendored
77
contrib/vagrant/Vagrantfile
vendored
@ -1,77 +0,0 @@
|
||||
VAGRANTFILE_API_VERSION = "2"
|
||||
GITCONFIG = `cat $HOME/.gitconfig`
|
||||
|
||||
Vagrant.require_version ">= 1.5"
|
||||
|
||||
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
|
||||
config.ssh.forward_agent = true
|
||||
|
||||
config.vm.provider "virtualbox" do |vb, override|
|
||||
vb.customize ["modifyvm", :id, "--memory", "6144"]
|
||||
if not RUBY_PLATFORM.downcase.include?("mswin")
|
||||
vb.customize ["modifyvm", :id, "--cpus", `awk "/^processor/ {++n} END {print n}" /proc/cpuinfo 2> /dev/null || sh -c 'sysctl hw.logicalcpu 2> /dev/null || echo ": 2"' | awk \'{print \$2}\' `.chomp ]
|
||||
end
|
||||
vb.customize ["modifyvm", :id, "--nicpromisc3", "allow-all"]
|
||||
|
||||
override.vm.synced_folder "../..", "/opt/stack/searchlight"
|
||||
end
|
||||
|
||||
config.vm.provider :libvirt do |lv, override|
|
||||
lv.graphics_ip = '0.0.0.0'
|
||||
lv.nested = true
|
||||
lv.memory = 8192
|
||||
if not RUBY_PLATFORM.downcase.include?("mswin")
|
||||
lv.cpus = `awk "/^processor/ {++n} END {print n}" /proc/cpuinfo 2> /dev/null || sh -c 'sysctl hw.logicalcpu 2> /dev/null || echo ": 2"' | awk \'{print \$2}\' `.chomp
|
||||
end
|
||||
|
||||
override.vm.synced_folder ".", "/vagrant", type: "nfs"
|
||||
override.vm.synced_folder "../..", "/opt/stack/searchlight", type: "nfs"
|
||||
end
|
||||
|
||||
$script = <<SCRIPT
|
||||
set -e
|
||||
|
||||
# Fixup permissions on /opt/stack/
|
||||
sudo chown vagrant:vagrant /opt/stack/
|
||||
|
||||
# Copy over git config
|
||||
cat << EOF > /home/vagrant/.gitconfig
|
||||
#{GITCONFIG}
|
||||
EOF
|
||||
|
||||
# Clone DevStack
|
||||
if [ ! -d "/home/vagrant/devstack" ]; then
|
||||
git clone https://opendev.org/openstack/devstack.git /home/vagrant/devstack
|
||||
fi
|
||||
|
||||
# Install Vagrant localrc sample
|
||||
cd /opt/stack/searchlight/contrib/vagrant
|
||||
|
||||
if [ ! -f "/home/vagrant/devstack/local.conf" ]; then
|
||||
cp local.conf /home/vagrant/devstack/local.conf
|
||||
fi
|
||||
|
||||
SCRIPT
|
||||
|
||||
config.vm.define "searchlight-ubuntu" do |ubuntu|
|
||||
ubuntu.vm.box = "ubuntu/xenial64"
|
||||
|
||||
ubuntu.vm.network :private_network, ip: "192.168.27.100"
|
||||
|
||||
ubuntu.vm.provision :shell, :privileged => true, :inline => "DEBIAN_FRONTEND=noninteractive apt-get update"
|
||||
ubuntu.vm.provision :shell, :privileged => true, :inline => "DEBIAN_FRONTEND=noninteractive apt-get install --yes git lvm2"
|
||||
|
||||
ubuntu.vm.provision :shell, :privileged => false, :inline => $script
|
||||
end
|
||||
|
||||
config.vm.define "searchlight-fedora" do |fedora|
|
||||
fedora.vm.box = "box-cutter/fedora20"
|
||||
|
||||
fedora.vm.network :private_network, ip: "192.168.27.101"
|
||||
|
||||
fedora.vm.provision :shell, :privileged => true, :inline => "yum update -y vim-minimal" # RH Bug 1066983
|
||||
fedora.vm.provision :shell, :privileged => true, :inline => "yum install -y git-core MySQL-python"
|
||||
|
||||
fedora.vm.provision :shell, :privileged => false, :inline => $script
|
||||
end
|
||||
end
|
@ -1,81 +0,0 @@
|
||||
[[local|localrc]]
|
||||
#########################
|
||||
# General DevStack Config
|
||||
# =======================
|
||||
# IP Address for services to bind to (Should match IP from Vagrantfile)
|
||||
HOST_IP=192.168.27.101
|
||||
SERVICE_HOST=$HOST_IP
|
||||
|
||||
ADMIN_PASSWORD=devstack
|
||||
MYSQL_PASSWORD=$ADMIN_PASSWORD
|
||||
RABBIT_PASSWORD=$ADMIN_PASSWORD
|
||||
SERVICE_PASSWORD=$ADMIN_PASSWORD
|
||||
SERVICE_TOKEN=$ADMIN_PASSWORD
|
||||
|
||||
# Logging
|
||||
LOGFILE=$DEST/logs/stack.sh.log
|
||||
VERBOSE=True
|
||||
ENABLE_DEBUG_LOG_LEVEL=True
|
||||
ENABLE_VERBOSE_LOG_LEVEL=True
|
||||
|
||||
# ElasticSearch version
|
||||
# Currently, only ElasticSearch 2.x and 5.x are supported
|
||||
ELASTICSEARCH_VERSION=5.6.11
|
||||
|
||||
# Enable Searchlight plugin
|
||||
enable_plugin searchlight https://opendev.org/openstack/searchlight master
|
||||
|
||||
# Enable the basic services we require
|
||||
enable_service rabbit mysql key
|
||||
|
||||
#############################
|
||||
# Searchlight Devstack Config
|
||||
# ===========================
|
||||
# Enable Searchlight services
|
||||
enable_service searchlight-api
|
||||
enable_service searchlight-listener
|
||||
|
||||
#######################
|
||||
# Other Devstack Config
|
||||
# =====================
|
||||
# Optional TLS Proxy
|
||||
#enable_service tls-proxy
|
||||
|
||||
# Optional Tempest
|
||||
#enable_service tempest
|
||||
|
||||
# Optional Rally
|
||||
#enable_service rally
|
||||
|
||||
# Optional Horizon
|
||||
#enable_service horizon
|
||||
|
||||
# Optional Glance
|
||||
enable_service g-api g-reg
|
||||
|
||||
# Optional Nova
|
||||
#enable_service n-api n-cpu n-net n-cond n-sch n-novnc
|
||||
|
||||
# Optional Neutron
|
||||
#disable_service n-net
|
||||
#enable_service q-svc q-agt q-dhcp q-l3 q-meta
|
||||
|
||||
[[post-config|$NOVA_CONF]]
|
||||
[DEFAULT]
|
||||
notification_driver=messagingv2
|
||||
notification_topics=searchlight_indexer
|
||||
notify_on_state_change=vm_and_task_state
|
||||
notify_on_any_change=True
|
||||
instance_usage_audit=True
|
||||
instance_usage_audit_period=hour
|
||||
|
||||
[[post-config|$NEUTRON_CONF]]
|
||||
[DEFAULT]
|
||||
notification_driver=messagingv2
|
||||
notification_topics=searchlight_indexer
|
||||
|
||||
[[post-config|$GLANCE_API_CONF]]
|
||||
[DEFAULT]
|
||||
notification_driver=messagingv2
|
||||
notification_topics=searchlight_indexer
|
||||
rpc_backend = 'rabbit'
|
@ -1,80 +0,0 @@
|
||||
..
|
||||
c) Copyright 2015 Hewlett-Packard Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
=================================
|
||||
Enabling Searchlight in Devstack
|
||||
=================================
|
||||
|
||||
1. Download DevStack (git clone)
|
||||
|
||||
2. Update local.conf
|
||||
|
||||
You may follow the customization instructions below or use the example
|
||||
local.conf.
|
||||
|
||||
3. Run ``stack.sh``
|
||||
|
||||
.. note::
|
||||
This installs a headless JRE. If you are working on a desktop based OS
|
||||
(such as Ubuntu 14.04), this may cause tools like pycharms to no longer
|
||||
launch. You can switch between JREs and back: to a headed JRE version using:
|
||||
"sudo update-alternatives --config java".
|
||||
|
||||
|
||||
Full example local.conf
|
||||
=======================
|
||||
|
||||
The example `local.conf.example <local.conf.example>`_ MAY not be up to date
|
||||
with the rest of devstack.
|
||||
|
||||
.. note::
|
||||
You will need to look through the settings and potentially customize it to
|
||||
your environment, especially ``HOST_IP``.
|
||||
|
||||
Existing local.conf customization
|
||||
=================================
|
||||
|
||||
1. Add this repo as an external repository::
|
||||
|
||||
> cat local.conf
|
||||
[[local|localrc]]
|
||||
enable_plugin searchlight https://opendev.org/openstack/searchlight
|
||||
enable_service searchlight-api
|
||||
enable_service searchlight-listener
|
||||
|
||||
To use stable branches, make sure devstack is on that branch, and specify
|
||||
the branch name to enable_plugin, for example::
|
||||
|
||||
enable_plugin searchlight https://opendev.org/openstack/searchlight stable/mitaka
|
||||
|
||||
2. Configure desired searchlight plugins
|
||||
|
||||
The search service is driven using a plugin mechanism for integrating to other
|
||||
services. Each integrated service may need to be specifically enabled
|
||||
in devstack and may require additional configuration settings to work with
|
||||
searchlight. For example, you typically will need to set the notifications
|
||||
driver in each service's configuration.
|
||||
|
||||
Please review the plugin documentation and add configuration appropriately:
|
||||
|
||||
* `Searchlight Plugins <https://docs.openstack.org/searchlight/latest/configuration/plugins.html>`_
|
||||
|
||||
3. Customize searchlight configuration
|
||||
|
||||
Searchlight documentation talks about settings in ``searchlight.conf``.
|
||||
To customize searchlight.conf settings, add them under the following
|
||||
section in ``local.conf``::
|
||||
|
||||
[[post-config|$SEARCHLIGHT_CONF]]
|
@ -1,16 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -ex
|
||||
|
||||
pushd $BASE/new/devstack
|
||||
|
||||
export KEEP_LOCALRC=1
|
||||
export ENABLED_SERVICES=mysql,key,searchlight,searchlight-api
|
||||
|
||||
# Pass through any SEARCHLIGHT_ env vars to the localrc file
|
||||
env | grep -E "^SEARCHLIGHT_" >> $BASE/new/devstack/local.conf || :
|
||||
|
||||
popd
|
||||
|
||||
# Run DevStack Gate
|
||||
$BASE/new/devstack-gate/devstack-vm-gate.sh
|
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
# This script is executed inside post_test_hook function in devstack gate.
|
||||
|
||||
# source $BASE/new/devstack/openrc admin admin
|
||||
|
||||
function generate_test_results {
|
||||
if [ -f .testrepository/0 ]; then
|
||||
sudo .tox/py27-gate/bin/testr last --subunit > $WORKSPACE/testrepository.subunit
|
||||
sudo mv $WORKSPACE/testrepository.subunit $BASE/logs/testrepository.subunit
|
||||
sudo .tox/py27-gate/bin/python /usr/local/jenkins/slave_scripts/subunit2html.py $BASE/logs/testrepository.subunit $BASE/logs/testr_results.html
|
||||
sudo gzip -9 $BASE/logs/testrepository.subunit
|
||||
sudo gzip -9 $BASE/logs/testr_results.html
|
||||
sudo chown $USER:$USER $BASE/logs/testrepository.subunit.gz $BASE/logs/testr_results.html.gz
|
||||
sudo chmod a+r $BASE/logs/testrepository.subunit.gz $BASE/logs/testr_results.html.gz
|
||||
fi
|
||||
}
|
||||
|
||||
set -x
|
||||
|
||||
export SEARCHLIGHT_DIR="$BASE/new/searchlight"
|
||||
sudo chown -R stack:stack $SEARCHLIGHT_DIR
|
||||
cd $SEARCHLIGHT_DIR
|
||||
|
||||
# Collect and parse result
|
||||
generate_test_results
|
||||
exit $EXIT_CODE
|
@ -1,259 +0,0 @@
|
||||
[[local|localrc]]
|
||||
|
||||
LOGFILE=$DEST/logs/stack.sh.log
|
||||
VERBOSE=TRUE
|
||||
USE_PYTHON3=True
|
||||
|
||||
### NETWORK SETTINGS ###
|
||||
#
|
||||
# Change the FLOATING_RANGE to whatever IPs VM is working in.
|
||||
# In NAT mode it is subnet VMWare Fusion provides, in bridged mode it is your local network.
|
||||
# But only use the top end of the network by using a /27 and starting at the 224 octet.
|
||||
#FLOATING_RANGE=192.168.1.224/27
|
||||
#FIXED_RANGE=10.0.0.1/24
|
||||
#FIXED_NETWORK_SIZE=256
|
||||
#FLAT_INTERFACE=eth0
|
||||
#EXT_GW_IP=192.168.122.1
|
||||
HOST_IP=127.0.0.1
|
||||
SERVICE_HOST=$HOST_IP
|
||||
IMAGE_HOST=$HOST_IP
|
||||
IDENTITY_HOST=$HOST_IP
|
||||
|
||||
### SET PASSWORDS FOR SERVICES AND USERS ###
|
||||
#
|
||||
ADMIN_PASSWORD=devstack
|
||||
MYSQL_PASSWORD=$ADMIN_PASSWORD
|
||||
RABBIT_PASSWORD=$ADMIN_PASSWORD
|
||||
SERVICE_PASSWORD=$ADMIN_PASSWORD
|
||||
SERVICE_TOKEN=$ADMIN_PASSWORD
|
||||
|
||||
|
||||
### SET SOME INSTALL OPTIONS ###
|
||||
#
|
||||
# run already-installed devstack in offline mode
|
||||
# Set ``OFFLINE`` to ``True`` to configure ``stack.sh`` to run cleanly without
|
||||
# Internet access. ``stack.sh`` must have been previously run with Internet
|
||||
# access to install prerequisites and fetch repositories.
|
||||
#OFFLINE=True
|
||||
|
||||
GIT_BASE=${GIT_BASE:-https://opendev.org}
|
||||
|
||||
# Reclone will ensure any repos already present are not re-cloned
|
||||
#
|
||||
# To review a patch to devstack itself, set this to yes, then:
|
||||
# cd /opt/stack/searchlight
|
||||
# git review -d <gerrit-id>
|
||||
# <devstack>/unstack.sh
|
||||
# <devstack>/stack.sh
|
||||
RECLONE=no
|
||||
|
||||
# always upgrade all Python dependencies
|
||||
PIP_UPGRADE=False
|
||||
|
||||
# Set libraries that will be installed from git and not PyPI
|
||||
# e.g. python-searchlightclient (not yet supported)
|
||||
#LIBS_FROM_GIT=python-searchlightclient
|
||||
|
||||
### ADD SOME IMAGES TO GLANCE ###
|
||||
#
|
||||
# latest Ubuntu Xenial amd64 cloud image
|
||||
IMAGE_URLS+=",https://cloud-images.ubuntu.com/releases/xenial/release/ubuntu-16.04-server-cloudimg-amd64-disk1.img"
|
||||
|
||||
# Fedora 21 cloud image (e.g. for AWS LoadBalancer resource in Heat)
|
||||
#IMAGE_URLS+=",http://download.fedoraproject.org/pub/fedora/linux/releases/21/Cloud/Images/x86_64/Fedora-Cloud-Base-20141203-21.x86_64.qcow2"
|
||||
|
||||
# ElasticSearch version
|
||||
# Currently, only ElasticSearch 2.x and 5.x are supported
|
||||
ELASTICSEARCH_VERSION=5.6.11
|
||||
|
||||
### CONFIGURE INSTALLED SERVICES ###
|
||||
#
|
||||
# Default set of components installed (as of DevStack Juno) is:
|
||||
# - Nova with Nova-network
|
||||
# - Keystone
|
||||
# - Glance
|
||||
# - Cinder with LVM backend
|
||||
# - Horizon
|
||||
# The below changes it.
|
||||
|
||||
|
||||
### HEAT ###
|
||||
#
|
||||
# enable Heat services
|
||||
enable_service h-eng h-api h-api-cfn h-api-cw heat
|
||||
|
||||
# (ON REVIEW) Install image for Heat's integration tests
|
||||
HEAT_TEST_IMAGE=True
|
||||
|
||||
### CINDER ###
|
||||
#
|
||||
# Do not securely rewrite cinder volumes on delete
|
||||
#CINDER_SECURE_DELETE=False
|
||||
|
||||
### HORIZON ###
|
||||
#
|
||||
# disable Horizon (Dashboard)
|
||||
#disable_service horizon
|
||||
HORIZON_BRANCH=master
|
||||
|
||||
### NEUTRON ###
|
||||
#
|
||||
# disable Nova-network and enable Neutron
|
||||
disable_service n-net
|
||||
enable_service q-svc q-agt q-dhcp q-l3 q-meta q-metering q-lbaas q-fwaas neutron
|
||||
|
||||
### CEILOMETER ###
|
||||
#
|
||||
# metering services
|
||||
#enable_service ceilometer-acompute ceilometer-acentral ceilometer-anotification ceilometer-collector
|
||||
|
||||
# alarming services
|
||||
#enable_service ceilometer-alarm-evaluator,ceilometer-alarm-notifier
|
||||
|
||||
# api services
|
||||
# enable_service ceilometer-api
|
||||
|
||||
# set shorter sample collection interval (default is 600)
|
||||
#CEILOMETER_PIPELINE_INTERVAL=60
|
||||
|
||||
### SWIFT ###
|
||||
#
|
||||
enable_service s-proxy s-object s-container s-account
|
||||
|
||||
# set swift hash - the hash below is result of
|
||||
# echo "SWIFT_HASH" | md5sum | awk '{print $1}'
|
||||
SWIFT_HASH=096d08da4f8d4cce3a724c5f6c18f055
|
||||
SWIFT_REPLICAS=1
|
||||
SWIFT_DATA_DIR=$DEST/data/swift
|
||||
|
||||
### DESIGNATE ###
|
||||
#
|
||||
enable_plugin designate https://opendev.org/openstack/designate
|
||||
|
||||
# Searchlight
|
||||
#
|
||||
enable_plugin searchlight https://opendev.org/openstack/searchlight
|
||||
enable_service searchlight-api
|
||||
enable_service searchlight-listener
|
||||
|
||||
enable_plugin searchlight-ui https://opendev.org/openstack/searchlight-ui
|
||||
|
||||
### POST CONFIG STAGE SETTINGS ###
|
||||
#
|
||||
# fix to allow access to instance vnc console when accessing Horizon via tunnel
|
||||
[[post-config|$NOVA_CONF]]
|
||||
[DEFAULT]
|
||||
#novncproxy_base_url = http://172.18.200.23:6127/vnc_auto.html
|
||||
|
||||
[notifications]
|
||||
notify_on_state_change = vm_and_task_state
|
||||
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
[[post-config|$NEUTRON_CONF]]
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
# decrease number of Heat engine workers, when too much for devstack
|
||||
[[post-config|$HEAT_CONF]]
|
||||
[DEFAULT]
|
||||
num_engine_workers = 2
|
||||
|
||||
[[post-config|$GLANCE_API_CONF]]
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
[[post-config|$CINDER_CONF]]
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
[[post-config|$DESIGNATE_CONF]]
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
### Searchlight Configuration ###
|
||||
|
||||
[[post-config|$SEARCHLIGHT_CONF]]
|
||||
|
||||
[elasticsearch]
|
||||
index_settings = number_of_shards:1,number_of_replicas:0
|
||||
|
||||
[manage]
|
||||
#workers = 3
|
||||
|
||||
[listener]
|
||||
#notifications_pool = searchlight-listener
|
||||
|
||||
[resource_plugin]
|
||||
#notifications_topic = notifications
|
||||
#resource_group_name = searchlight
|
||||
|
||||
[service_credentials:nova]
|
||||
compute_api_version = 2.1
|
||||
|
||||
[resource_plugin:os_nova_server]
|
||||
enabled = True
|
||||
#admin_only_fields = OS-EXT-STS:vm_state
|
||||
|
||||
[resource_plugin:os_nova_hypervisor]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_nova_flavor]
|
||||
notifications_topics_exchanges = versioned_notifications,nova
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_nova_servergroup]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_glance_image]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_glance_metadef]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_cinder_volume]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_cinder_snapshot]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_designate_zone]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_designate_recordset]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_net]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_port]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_floatingip]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_security_group]
|
||||
enabled = True
|
||||
|
||||
## SWIFT SETTINGS ###
|
||||
# At this time we recommend that you manually enable the Searchlight plugins
|
||||
# for Swift after devstack has completed stacking. Please follow the
|
||||
# instructions in the docs:
|
||||
#
|
||||
# https://docs.openstack.org/searchlight/latest/admin/plugins/swift.html
|
||||
#
|
||||
# Notifications must be configured properly for searchlight to process
|
||||
# incremental updates. There is a middleware patch to provide Swift updates.
|
||||
# Please see the plugin guide for swift for more information.
|
||||
|
||||
[resource_plugin:os_swift_account]
|
||||
enabled = False
|
||||
#Specify same value as in swift proxy config for reseller_prefix
|
||||
reseller_prefix = AUTH_
|
||||
|
||||
[resource_plugin:os_swift_container]
|
||||
enabled = False
|
||||
|
||||
[resource_plugin:os_swift_object]
|
||||
enabled = False
|
@ -1,417 +0,0 @@
|
||||
# Install and start **Searchlight** service
|
||||
|
||||
# To enable Searchlight services, add the following to localrc
|
||||
# enable_plugin searchlight https://opendev.org/openstack/searchlight
|
||||
# enable_service searchlight-api
|
||||
# enable_service searchlight-listener
|
||||
|
||||
# stack.sh
|
||||
# ---------
|
||||
# install_searchlight
|
||||
# configure_searchlight
|
||||
# init_searchlight
|
||||
# start_searchlight
|
||||
# stop_searchlight
|
||||
# cleanup_searchlight
|
||||
|
||||
# Save trace setting
|
||||
XTRACE=$(set +o | grep xtrace)
|
||||
set +o xtrace
|
||||
|
||||
|
||||
# Defaults
|
||||
# --------
|
||||
# Set up default repos
|
||||
SEARCHLIGHT_REPO=${SEARCHLIGHT_REPO:-${GIT_BASE}/openstack/searchlight.git}
|
||||
SEARCHLIGHT_BRANCH=${SEARCHLIGHT_BRANCH:-master}
|
||||
|
||||
# Set up default paths
|
||||
SEARCHLIGHT_BIN_DIR=$(get_python_exec_prefix)
|
||||
SEARCHLIGHT_DIR=$DEST/searchlight
|
||||
SEARCHLIGHT_CONF_DIR=/etc/searchlight
|
||||
SEARCHLIGHT_STATE_PATH=${SEARCHLIGHT_STATE_PATH:=$DATA_DIR/searchlight}
|
||||
SEARCHLIGHT_CONF=$SEARCHLIGHT_CONF_DIR/searchlight.conf
|
||||
SEARCHLIGHT_LOG_DIR=/var/log/searchlight
|
||||
SEARCHLIGHT_AUTH_CACHE_DIR=${SEARCHLIGHT_AUTH_CACHE_DIR:-/var/cache/searchlight}
|
||||
SEARCHLIGHT_APIPASTE_CONF=$SEARCHLIGHT_CONF_DIR/api-paste.ini
|
||||
SEARCHLIGHT_UWSGI_CONF=$SEARCHLIGHT_CONF_DIR/searchlight-uwsgi.ini
|
||||
SEARCHLIGHT_UWSGI_APP=$SEARCHLIGHT_BIN_DIR/searchlight-api-wsgi
|
||||
|
||||
if is_service_enabled tls-proxy; then
|
||||
SEARCHLIGHT_SERVICE_PROTOCOL="https"
|
||||
fi
|
||||
|
||||
# Public IP/Port Settings
|
||||
SEARCHLIGHT_SERVICE_PROTOCOL=${SEARCHLIGHT_SERVICE_PROTOCOL:-$SERVICE_PROTOCOL}
|
||||
SEARCHLIGHT_SERVICE_HOST=${SEARCHLIGHT_SERVICE_HOST:-$SERVICE_HOST}
|
||||
SEARCHLIGHT_SERVICE_PORT=${SEARCHLIGHT_SERVICE_PORT:-9393}
|
||||
SEARCHLIGHT_SERVICE_PORT_INT=${SEARCHLIGHT_SERVICE_PORT_INT:-19393}
|
||||
|
||||
ELASTICSEARCH_VERSION=${ELASTICSEARCH_VERSION:-2.3.4}
|
||||
# Base URL for ElasticSearch 5.x and 6.x
|
||||
ELASTICSEARCH_BASEURL=https://artifacts.elastic.co/downloads/elasticsearch
|
||||
# Base URL for ElasticSearch 2.x
|
||||
ELASTICSEARCH_BASEURL_LEGACY=https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution
|
||||
|
||||
# Helper Functions
|
||||
# ----------------
|
||||
function setup_colorized_logging_searchlight {
|
||||
local conf_file=$1
|
||||
local conf_section=$2
|
||||
local project_var=${3:-"project_name"}
|
||||
local user_var=${4:-"user_name"}
|
||||
|
||||
setup_colorized_logging $conf_file $conf_section $project_var $user_var
|
||||
|
||||
# Override the logging_context_format_string value chosen by
|
||||
# setup_colorized_logging.
|
||||
iniset $conf_file $conf_section logging_context_format_string "%(asctime)s.%(msecs)03d %(color)s%(levelname)s %(name)s [[01;36m%(request_id)s [00;36m%(user_identity)s%(color)s] [01;35m%(instance)s%(color)s%(message)s[00m"
|
||||
}
|
||||
|
||||
# DevStack Plugin
|
||||
# ---------------
|
||||
|
||||
# cleanup_searchlight - Remove residual data files, anything left over from previous
|
||||
# runs that a clean run would need to clean up
|
||||
function cleanup_searchlight {
|
||||
_stop_elasticsearch
|
||||
sudo rm -rf $SEARCHLIGHT_STATE_PATH $SEARCHLIGHT_AUTH_CACHE_DIR
|
||||
sudo rm -f $(apache_site_config_for searchlight_api)
|
||||
}
|
||||
|
||||
# configure_searchlight - Set config files, create data dirs, etc
|
||||
function configure_searchlight {
|
||||
[ ! -d $SEARCHLIGHT_CONF_DIR ] && sudo mkdir -m 755 -p $SEARCHLIGHT_CONF_DIR
|
||||
sudo chown $STACK_USER $SEARCHLIGHT_CONF_DIR
|
||||
|
||||
[ ! -d $SEARCHLIGHT_LOG_DIR ] && sudo mkdir -m 755 -p $SEARCHLIGHT_LOG_DIR
|
||||
sudo chown $STACK_USER $SEARCHLIGHT_LOG_DIR
|
||||
|
||||
# (Re)create ``searchlight.conf``
|
||||
rm -f $SEARCHLIGHT_CONF
|
||||
|
||||
# General Configuration
|
||||
iniset_rpc_backend searchlight $SEARCHLIGHT_CONF DEFAULT
|
||||
|
||||
iniset $SEARCHLIGHT_CONF DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL
|
||||
iniset $SEARCHLIGHT_CONF DEFAULT state_path $SEARCHLIGHT_STATE_PATH
|
||||
|
||||
# API Configuration
|
||||
sudo cp $SEARCHLIGHT_DIR/etc/api-paste.ini $SEARCHLIGHT_APIPASTE_CONF
|
||||
iniset $SEARCHLIGHT_CONF api public_endpoint $SEARCHLIGHT_SERVICE_PROTOCOL://$SEARCHLIGHT_SERVICE_HOST/search
|
||||
|
||||
# OpenStack users
|
||||
iniset $SEARCHLIGHT_CONF service_credentials auth_type password
|
||||
iniset $SEARCHLIGHT_CONF service_credentials username searchlight
|
||||
iniset $SEARCHLIGHT_CONF service_credentials user_domain_id default
|
||||
iniset $SEARCHLIGHT_CONF service_credentials project_domain_id default
|
||||
iniset $SEARCHLIGHT_CONF service_credentials password $SERVICE_PASSWORD
|
||||
iniset $SEARCHLIGHT_CONF service_credentials project_name $SERVICE_PROJECT_NAME
|
||||
iniset $SEARCHLIGHT_CONF service_credentials auth_url $KEYSTONE_SERVICE_URI
|
||||
iniset $SEARCHLIGHT_CONF service_credentials os_region_name $REGION_NAME
|
||||
|
||||
# Keystone Middleware
|
||||
iniset $SEARCHLIGHT_CONF paste_deploy flavor keystone
|
||||
configure_auth_token_middleware $SEARCHLIGHT_CONF searchlight $SEARCHLIGHT_AUTH_CACHE_DIR
|
||||
|
||||
# Oslo Concurrency
|
||||
iniset $SEARCHLIGHT_CONF oslo_concurrency lock_path "$SEARCHLIGHT_STATE_PATH"
|
||||
|
||||
# TLS Proxy Configuration
|
||||
if is_service_enabled tls-proxy; then
|
||||
# Set the service port for a proxy to take the original
|
||||
iniset $SEARCHLIGHT_CONF api bind_port $SEARCHLIGHT_SERVICE_PORT_INT
|
||||
else
|
||||
iniset $SEARCHLIGHT_CONF api bind_port $SEARCHLIGHT_SERVICE_PORT
|
||||
fi
|
||||
|
||||
# Logging Configuration
|
||||
if [ "$SYSLOG" != "False" ]; then
|
||||
iniset $SEARCHLIGHT_CONF DEFAULT use_syslog True
|
||||
fi
|
||||
|
||||
# Format logging
|
||||
if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ]; then
|
||||
setup_colorized_logging_searchlight $SEARCHLIGHT_CONF DEFAULT "tenant" "user"
|
||||
fi
|
||||
|
||||
# Plugin config - disable designate by default since it's not typically installed
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_designate_zone enabled False
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_designate_recordset enabled False
|
||||
|
||||
# Plugin config - disable ironic by default since it's not typically installed
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_ironic_chassis enabled False
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_ironic_chassis notifications_topics_exchanges ironic_versioned_notifications,ironic
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_ironic_node enabled False
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_ironic_node notifications_topics_exchanges ironic_versioned_notifications,ironic
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_ironic_port enabled False
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_ironic_port notifications_topics_exchanges ironic_versioned_notifications,ironic
|
||||
|
||||
# Plugin config - enable versioned notifications for flavor
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_nova_flavor enabled True
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_nova_flavor notifications_topics_exchanges versioned_notifications,nova
|
||||
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_nova_server enabled True
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_nova_server notifications_topics_exchanges versioned_notifications,nova
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_nova_server use_versioned_notifications True
|
||||
|
||||
# Plugin config - disable swift by default since it's not typically installed
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_swift_account enabled False
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_swift_container enabled False
|
||||
iniset $SEARCHLIGHT_CONF resource_plugin:os_swift_object enabled False
|
||||
|
||||
# uWSGI configuration
|
||||
write_uwsgi_config "$SEARCHLIGHT_UWSGI_CONF" "$SEARCHLIGHT_UWSGI_APP" "/search"
|
||||
}
|
||||
|
||||
# create_searchlight_accounts - Set up common required searchlight accounts
|
||||
|
||||
# Tenant User Roles
|
||||
# ------------------------------------------------------------------
|
||||
# service searchlight admin # if enabled
|
||||
function create_searchlight_accounts {
|
||||
if [[ "$ENABLED_SERVICES" =~ "searchlight-" ]]; then
|
||||
create_service_user "searchlight" "admin"
|
||||
|
||||
if is_service_enabled searchlight-api; then
|
||||
get_or_create_service "searchlight" "search" "Searchlight Service"
|
||||
get_or_create_endpoint "search" \
|
||||
"$REGION_NAME" \
|
||||
"$SEARCHLIGHT_SERVICE_PROTOCOL://$SEARCHLIGHT_SERVICE_HOST/search" \
|
||||
"$SEARCHLIGHT_SERVICE_PROTOCOL://$SEARCHLIGHT_SERVICE_HOST/search" \
|
||||
"$SEARCHLIGHT_SERVICE_PROTOCOL://$SEARCHLIGHT_SERVICE_HOST/search"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# init_searchlight - Initialize etc.
|
||||
function init_searchlight {
|
||||
# Create cache dir
|
||||
sudo mkdir -p $SEARCHLIGHT_AUTH_CACHE_DIR
|
||||
sudo chown $STACK_USER $SEARCHLIGHT_AUTH_CACHE_DIR
|
||||
rm -f $SEARCHLIGHT_AUTH_CACHE_DIR/*
|
||||
|
||||
_start_elasticsearch
|
||||
|
||||
$SEARCHLIGHT_BIN_DIR/searchlight-manage --config-file $SEARCHLIGHT_CONF index sync --force
|
||||
}
|
||||
|
||||
# Install Searchlight's requirements
|
||||
# See https://elasticsearch-py.readthedocs.io/en/master/#compatibility
|
||||
function _setup_searchlight_dev {
|
||||
setup_develop $SEARCHLIGHT_DIR
|
||||
if [[ $ELASTICSEARCH_VERSION =~ ^5 ]]; then
|
||||
echo "Installing python elasticsearch for ES 5.x"
|
||||
$REQUIREMENTS_DIR/.venv/bin/edit-constraints $REQUIREMENTS_DIR/upper-constraints.txt elasticsearch
|
||||
pip_install -U -r $SEARCHLIGHT_DIR/elasticsearch5.txt
|
||||
elif [[ $ELASTICSEARCH_VERSION =~ ^6 ]]; then
|
||||
echo "WARNING - Searchlight is not tested with ES 6.x!!!"
|
||||
# echo "Installing python elasticsearch for ES 6.x"
|
||||
# $REQUIREMENTS_DIR/.venv/bin/edit-constraints $REQUIREMENTS_DIR/upper-constraints.txt eleasticsearch
|
||||
# pip install -c $REQUIREMENTS_DIR/upper-constraints.txt -U -r $SEARCHLIGHT_DIR/elasticsearch6.txt
|
||||
fi
|
||||
}
|
||||
|
||||
# install_searchlight - Collect source and prepare
|
||||
function install_searchlight {
|
||||
git_clone $SEARCHLIGHT_REPO $SEARCHLIGHT_DIR $SEARCHLIGHT_BRANCH
|
||||
_setup_searchlight_dev
|
||||
_download_elasticsearch
|
||||
_install_elasticsearch
|
||||
pip_install uwsgi
|
||||
}
|
||||
|
||||
# install_searchlightclient - Collect source and prepare
|
||||
function install_searchlightclient {
|
||||
git_clone $SEARCHLIGHTCLIENT_REPO $SEARCHLIGHTCLIENT_DIR $SEARCHLIGHTCLIENT_BRANCH
|
||||
setup_develop $SEARCHLIGHTCLIENT_DIR
|
||||
}
|
||||
|
||||
# start_searchlight - Start running processes, including screen
|
||||
function start_searchlight {
|
||||
if is_service_enabled searchlight-api; then
|
||||
run_process searchlight-api "$SEARCHLIGHT_BIN_DIR/uwsgi --ini $SEARCHLIGHT_UWSGI_CONF"
|
||||
fi
|
||||
if is_service_enabled searchlight-listener; then
|
||||
run_process searchlight-listener "$SEARCHLIGHT_BIN_DIR/searchlight-listener --config-file $SEARCHLIGHT_CONF"
|
||||
fi
|
||||
}
|
||||
|
||||
# stop_searchlight - Stop running processes
|
||||
function stop_searchlight {
|
||||
# Kill the searchlight screen windows
|
||||
stop_process searchlight-api
|
||||
stop_process searchlight-listener
|
||||
remove_uwsgi_config "$SEARCHLIGHT_UWSGI_CONF" "$SEARCHLIGHT_UWSGI_APP"
|
||||
}
|
||||
|
||||
|
||||
###############
|
||||
# ELASTICSEARCH
|
||||
# Moving this here because the devstack team has determined that only
|
||||
# services supporting devstack core projects should live in devstack
|
||||
|
||||
function _wget_elasticsearch {
|
||||
local baseurl=${1}
|
||||
local file=${2}
|
||||
if [ ! -f ${FILES}/${file} ]; then
|
||||
wget ${baseurl}/${file} -O ${FILES}/${file}
|
||||
fi
|
||||
|
||||
if [ ! -f ${FILES}/${file}.sha1 ]; then
|
||||
# Starting with 2.0.0, sha1 files dropped the .txt extension and changed
|
||||
# the format slightly; need the leading spaces to comply with sha1sum
|
||||
( wget ${baseurl}/${file}.sha1 -O ${FILES}/${file}.sha1 &&
|
||||
echo " ${file}" >> ${FILES}/${file}.sha1 )
|
||||
fi
|
||||
|
||||
pushd ${FILES}; sha1sum ${file} > ${file}.sha1.gen; popd
|
||||
|
||||
if ! diff ${FILES}/${file}.sha1.gen ${FILES}/${file}.sha1; then
|
||||
echo "Invalid elasticsearch download. Could not install."
|
||||
return 1
|
||||
else
|
||||
echo "SHA1 for ${file} matches downloaded ${FILES}/${file}.sha1"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
function _download_elasticsearch {
|
||||
if is_ubuntu; then
|
||||
arch="deb"
|
||||
elif is_fedora; then
|
||||
arch="rpm"
|
||||
else
|
||||
echo "Unknown architecture; can't download ElasticSearch"
|
||||
fi
|
||||
ELASTICSEARCH_FILENAME=elasticsearch-${ELASTICSEARCH_VERSION}.${arch}
|
||||
|
||||
if [[ $ELASTICSEARCH_VERSION =~ ^2 ]]; then
|
||||
ELASTICSEARCH_URL=${ELASTICSEARCH_BASEURL_LEGACY}/${arch}/elasticsearch/${ELASTICSEARCH_VERSION}
|
||||
elif [[ $ELASTICSEARCH_VERSION =~ ^5 ]]; then
|
||||
ELASTICSEARCH_URL=${ELASTICSEARCH_BASEURL}
|
||||
else
|
||||
echo "Current Searchlight only supports ElasticSearch 2.x and 5.x"
|
||||
fi
|
||||
echo "Downloading ElasticSearch $ELASTICSEARCH_VERSION"
|
||||
echo "ElasticSearch URL is $ELASTICSEARCH_URL"
|
||||
_wget_elasticsearch $ELASTICSEARCH_URL $ELASTICSEARCH_FILENAME
|
||||
}
|
||||
|
||||
function _check_elasticsearch_ready {
|
||||
# poll elasticsearch to see if it's started
|
||||
if ! wait_for_service 30 http://localhost:9200; then
|
||||
die $LINENO "Maximum timeout reached. Could not connect to ElasticSearch"
|
||||
fi
|
||||
}
|
||||
|
||||
function _start_elasticsearch {
|
||||
echo "Starting elasticsearch"
|
||||
if is_ubuntu; then
|
||||
sudo /etc/init.d/elasticsearch start
|
||||
_check_elasticsearch_ready
|
||||
elif is_fedora; then
|
||||
sudo /bin/systemctl start elasticsearch.service
|
||||
_check_elasticsearch_ready
|
||||
else
|
||||
echo "Unsupported architecture... Can not start elasticsearch."
|
||||
fi
|
||||
}
|
||||
|
||||
function _stop_elasticsearch {
|
||||
echo "Stopping elasticsearch"
|
||||
if is_ubuntu; then
|
||||
sudo /etc/init.d/elasticsearch stop
|
||||
elif is_fedora; then
|
||||
sudo /bin/systemctl stop elasticsearch.service
|
||||
else
|
||||
echo "Unsupported architecture...can not stop elasticsearch."
|
||||
fi
|
||||
}
|
||||
|
||||
function _install_elasticsearch {
|
||||
# echo "Installing elasticsearch"
|
||||
# pip_install_gr elasticsearch
|
||||
if is_package_installed elasticsearch; then
|
||||
echo "Note: elasticsearch was already installed."
|
||||
return
|
||||
fi
|
||||
if is_ubuntu; then
|
||||
if [[ ${DISTRO} == "bionic" ]]; then
|
||||
is_package_installed openjdk-8-jdk-headless || install_package openjdk-8-jdk-headless
|
||||
else
|
||||
is_package_installed default-jdk-headless || install_package default-jdk-headless
|
||||
fi
|
||||
sudo dpkg -i ${FILES}/elasticsearch-${ELASTICSEARCH_VERSION}.deb
|
||||
sudo update-rc.d elasticsearch defaults 95 10
|
||||
elif is_fedora; then
|
||||
is_package_installed java-1.8.0-openjdk-headless || install_package java-1.8.0-openjdk-headless
|
||||
yum_install ${FILES}/elasticsearch-${ELASTICSEARCH_VERSION}.rpm
|
||||
sudo /bin/systemctl daemon-reload
|
||||
sudo /bin/systemctl enable elasticsearch.service
|
||||
else
|
||||
echo "Unsupported install of elasticsearch on this architecture."
|
||||
fi
|
||||
}
|
||||
|
||||
function _uninstall_elasticsearch {
|
||||
echo "Uninstalling elasticsearch"
|
||||
if is_package_installed elasticsearch; then
|
||||
if is_ubuntu; then
|
||||
sudo apt-get purge elasticsearch
|
||||
elif is_fedora; then
|
||||
sudo yum remove elasticsearch
|
||||
else
|
||||
echo "Unsupported install of elasticsearch on this architecture."
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
#
|
||||
# END OF ELASTICSEARCH
|
||||
######################
|
||||
|
||||
|
||||
# check for service enabled
|
||||
if is_service_enabled searchlight; then
|
||||
|
||||
if [[ "$1" == "source" ]]; then
|
||||
# Initial source of lib script
|
||||
source $TOP_DIR/lib/searchlight
|
||||
fi
|
||||
|
||||
if [[ "$1" == "stack" && "$2" == "install" ]]; then
|
||||
echo_summary "Installing Searchlight"
|
||||
install_searchlight
|
||||
|
||||
echo_summary "Installing Searchlight client"
|
||||
install_searchlightclient
|
||||
elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
|
||||
echo_summary "Configuring Searchlight"
|
||||
configure_searchlight
|
||||
|
||||
if is_service_enabled key; then
|
||||
echo_summary "Creating Searchlight Keystone Accounts"
|
||||
create_searchlight_accounts
|
||||
fi
|
||||
|
||||
elif [[ "$1" == "stack" && "$2" == "extra" ]]; then
|
||||
echo_summary "Initializing Searchlight"
|
||||
init_searchlight
|
||||
|
||||
echo_summary "Starting Searchlight"
|
||||
start_searchlight
|
||||
fi
|
||||
|
||||
if [[ "$1" == "unstack" ]]; then
|
||||
stop_searchlight
|
||||
fi
|
||||
|
||||
if [[ "$1" == "clean" ]]; then
|
||||
echo_summary "Cleaning Searchlight"
|
||||
cleanup_searchlight
|
||||
fi
|
||||
fi
|
||||
|
||||
# Restore xtrace
|
||||
$XTRACE
|
@ -1,9 +0,0 @@
|
||||
# Devstack settings
|
||||
|
||||
#Searchlight client settings
|
||||
SEARCHLIGHTCLIENT_DIR=$DEST/python-searchlightclient
|
||||
SEARCHLIGHTCLIENT_REPO=${SEARCHLIGHTCLIENT_REPO:-${GIT_BASE}/openstack/python-searchlightclient.git}
|
||||
SEARCHLIGHTCLIENT_BRANCH=${SEARCHLIGHTCLIENT_BRANCH:-master}
|
||||
|
||||
# we have to add searchlight to enabled services for screen_it to work
|
||||
enable_service searchlight
|
@ -1,6 +0,0 @@
|
||||
# Documentation
|
||||
sphinx>=2.0.0,!=2.1.0 # BSD
|
||||
openstackdocstheme>=2.2.1 # Apache-2.0
|
||||
os-api-ref>=1.4.0 # Apache-2.0
|
||||
reno>=3.1.0 # Apache-2.0
|
||||
sphinxcontrib-svg2pdfconverter>=0.1.0 # BSD
|
@ -1,347 +0,0 @@
|
||||
..
|
||||
Copyright (c) 2016 Hewlett Packard Enterprise Development LP
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
Searchlight Architecture
|
||||
========================
|
||||
|
||||
The OpenStack Search Service
|
||||
----------------------------
|
||||
|
||||
Searchlight is a microservice based architecture that provides advanced and
|
||||
multi-tenant searching capabilities across multiple cloud services through
|
||||
indexing into Elasticsearch. Once indexed, users are able to search through
|
||||
all internal data associated with an OpenStack service through a single API
|
||||
with a consistent interface. These searches can be quite complex, are blazingly
|
||||
fast, and easily paginated. Without Searchlight it can be necessary to make
|
||||
calls to multiple services to retrieve resource information.
|
||||
|
||||
All documents in Elasticsearch are protected by Searchlight with role-based
|
||||
access control (RBAC) for data security and protection. RBAC protection
|
||||
prevents unauthorized users from accessing data that they are not permitted to
|
||||
view. Searchlight uses RBAC in three ways to protect the data:
|
||||
|
||||
1. **Network Restriction:** Elasticsearch is not directly accessible. It
|
||||
resides on a restricted network behind Searchlight. The only way to access
|
||||
the data within Elasticsearch is indirectly through the Searchlight.
|
||||
API endpoints. Users need to be authenticated to access these endpoints.
|
||||
2. **Document Separation:** Each resource is indexed twice in Elasticsearch.
|
||||
Once for the admin user and once for all others. The document associated
|
||||
with the admin has all fields present. The document associated with
|
||||
non-admins has all admin-only fields removed. This model prevents a
|
||||
non-admin from viewing any admin-only fields since these fields are not
|
||||
present in the document at all.
|
||||
3. **Pre-Query Protection:** Searchlight will inject RBAC-based protection
|
||||
into all queries before they get sent to Elasticsearch. In general the
|
||||
pre-query RBAC injection will provide gross level protection. For example,
|
||||
entire projects that the user does not have access to. See Figure 2.
|
||||
4. **Post-Query Protection:** Searchlight uses the RBAC-based protection to
|
||||
filter the response to each query in order to remove any sensitive data
|
||||
before returning the results to the eagerly waiting user. In general the
|
||||
post-query RBAC filtering will provide fine level protection. For example,
|
||||
specific fields within a document. See Figure 2.
|
||||
|
||||
The functionality provided by Searchlight differs greatly from the logging
|
||||
capabilities of OpenStack. The logging services store predetermined messages
|
||||
from the OpenStack services, as opposed to the dynamic, internal state of the
|
||||
OpenStack services.
|
||||
|
||||
Primary Users
|
||||
-------------
|
||||
|
||||
The primary users are Horizon-based. This includes users accessing their
|
||||
cloud projects ("tenants") and cloud administrators (IT operations).
|
||||
|
||||
Searchlight Components
|
||||
----------------------
|
||||
|
||||
Searchlight consists of multiple components. The components live in one of two
|
||||
environments: externally on the end-user's system or internally on an
|
||||
OpenStack deployed system. Figure 1 illustrates how all of these components
|
||||
interact.
|
||||
|
||||
**Figure 1: Architecture Diagram**
|
||||
|
||||
.. image:: ../../../images/Figure1.png
|
||||
|
||||
Figure 1 shows the relationship between all of the Searchlight components. The
|
||||
external Searchlight components are colored light-grey. The internal
|
||||
Searchlight components are colored peach. The internal non-Searchlight
|
||||
components are brightly colored, as are the arrows indicating the interfaces
|
||||
for these components.
|
||||
|
||||
* **Searchlight UI:** A plugin for Horizon that provides search support. Lives
|
||||
externally on the end-user's system.
|
||||
* **Searchlight User CLI (OpenStack Client plugin):** Command-line interface
|
||||
to the Searchlight API. Lives externally on the end-user's system.
|
||||
* **Searchlight API:** OpenStack Searchlight RESTful API. A separate process
|
||||
running a WSGI web server on top of the python Searchlight framework. All
|
||||
external access (for example, Horizon, browser, and CLI) go through the
|
||||
Searchlight API. The Searchlight API performs read-only accesses of
|
||||
Elasticsearch.
|
||||
* **HAProxy:** High availability load balancer and proxy server for the
|
||||
Searchlight API component. HAProxy enables multiple instances of the
|
||||
Searchlight API to run and allows for better performance, scalability, and
|
||||
reliability of the Searchlight service.
|
||||
* **Searchlight Listener:** A separate Searchlight process that is plugged
|
||||
into the notification MQ for the OpenStack services. Whenever there is a
|
||||
change in state for a resource type in an OpenStack service, a notification
|
||||
is sent to the Searchlight Listener. This results in the OpenStack service
|
||||
state being indexed into Elasticsearch as a document related to that
|
||||
resource type (for example, OS::Nova::Server). The listener is never accessed
|
||||
directly by any of the other Searchlight components.
|
||||
* **Searchlight Admin CLI (searchlight-manage):** A separate process that
|
||||
provides admin support for Searchlight. The main functionality is
|
||||
re-indexing support. Connects to various OpenStack services and loads data
|
||||
from them into the Elasticsearch indices. The admin needs to log in directly
|
||||
to a Searchlight node and run the Searchlight Admin CLI from that node. The
|
||||
Searchlight Admin CLI performs read/write accesses to Elasticsearch. Lives
|
||||
internally on an OpenStack deployed system.
|
||||
* **Elasticsearch:** A database service used to index information about
|
||||
selected services. Elasticsearch is meant to be shared by multiple OpenStack
|
||||
projects, not just Searchlight. This implies that there may be multiple
|
||||
indices within Elasticsearch. All Elasticsearch indices used by Searchlight
|
||||
are created in a way so as to be unique to Searchlight. This will prevent
|
||||
any duplication between Searchlight and other OpenStack projects that may
|
||||
be sharing Elasticsearch. Lives internally on an OpenStack deployed system.
|
||||
* **RabbitMQ:** Messaging Queue used for accessing the OpenStack's OSLO
|
||||
notification MQ. Each OpenStack service will place notifications for any
|
||||
state change in RabbitMQ. The Searchlight listener will pull this
|
||||
notifications from RabbitMQ and process them. Lives internally on an
|
||||
OpenStack deployed system. All access of the notification MQ is done
|
||||
through OSLO messaging.
|
||||
* **OpenStack Services:** Various OpenStack Services from which Searchlight
|
||||
gathers information through internal plugins. The current list of supported
|
||||
OpenStack services are: Cinder, Designate, Glance, Neutron, Nova, Swift.
|
||||
Searchlight accesses the services as an admin, not as a regular user.
|
||||
This admin access is required for the extra privileges that are needed,
|
||||
for example, to access objects across projects ("tenants"). The credentials
|
||||
for the admin are stored in the config files.
|
||||
* **OpenStack Service Plugins:** Each OpenStack service mentioned above has
|
||||
one or more associated plugins. These plugins live in Searchlight and not in
|
||||
the OpenStack service. The plugins are based on a resource type and not on
|
||||
the OpenStack service. For example, the Cinder Service has two resource types
|
||||
used by Searchlight: OS::Cinder::Volume and OS::Cinder::Snapshot. This
|
||||
results in two plugins for the Cinder service. The architecture is the same
|
||||
for each plugin. The plugins are not a separate process. The plugins live
|
||||
within the Searchlight listener and the Searchlight Admin CLI components.
|
||||
|
||||
The numbered arrows represent the interfaces between the components. These
|
||||
interfaces are fully described in Table 1.
|
||||
|
||||
Searchlight Eco-System
|
||||
----------------------
|
||||
|
||||
The Searchlight Eco-system is illuminated by Figure 2. We will step through
|
||||
each stage of the Searchlight life-cycle to further explore how the various
|
||||
components interact with each other.
|
||||
|
||||
**Deployment Stage:** This stage is when Searchlight gets instantiated by the
|
||||
admin dev ops. Even though the steps are mentioned as if they are manual steps,
|
||||
they are all done within a deployment process. More deployment stage options
|
||||
are discussed in Figure 4.
|
||||
|
||||
1. The OpenStack services (for example, Nova, Neutron) are started. These
|
||||
services are used by Searchlight, but are not a part of the Searchlight
|
||||
Eco-system. The admin will set up the configuration files for all services
|
||||
that are used by Searchlight, including enabling the notifications that will
|
||||
later be consumed by Searchlight.
|
||||
|
||||
2. Elasticsearch is started. This analytics engine is used by Searchlight, but
|
||||
it is not a part of the Searchlight Eco-system. As a note, other services
|
||||
may also be using Elasticsearch.
|
||||
|
||||
3. The admin sets up the Searchlight configuration file.
|
||||
|
||||
4. The initial indexing for Searchlight occurs (component 2 in Figure 2). This
|
||||
happens when the admin runs the command ``searchlight-manage index sync`` on
|
||||
a Searchlight node. Searchlight will pull the data from the OpenStack
|
||||
service's API and index it into Elasticsearch.
|
||||
|
||||
5. The Searchlight listeners are started (component 3 in Figure 2). This
|
||||
happens when the admin runs the command ``searchlight-listener`` At this
|
||||
point, Searchlight is notified of any state changes that occurs in the
|
||||
services. The OpenStack services will push the data to Searchlight, which
|
||||
in turn will index that data into Elasticsearch.
|
||||
|
||||
6. The Searchlight API service is started (component 1 in Figure 2). This
|
||||
happens when the admin runs the command ``searchlight-api`` At this point,
|
||||
the Searchlight API is live and anyone can start making queries to
|
||||
Searchlight.
|
||||
|
||||
**Running Stage (automatic):** This stage consumes the vast majority of the
|
||||
Searchlight life cycle. The running stage is further highlighted in Figure 3.
|
||||
|
||||
1. The Searchlight listeners are actively receiving notifications from the
|
||||
OpenStack services whenever there is a state change.
|
||||
|
||||
2. The Searchlight API service is actively receiving queries from Horizon apps
|
||||
and users.
|
||||
|
||||
**Maintenance Stage (manual):** This stage occurs when the admin needs to
|
||||
analyze the state of Searchlight Eco-system or make corrections to it.
|
||||
|
||||
1. To verify there are no orphaned Elasticsearch indices, the admin can view
|
||||
the Elasticsearch indices being used by Searchlight using the
|
||||
``searchlight-manage`` command. Any orphaned indices will be taken
|
||||
care of through Elasticsearch and not through Searchlight.
|
||||
See :ref:`ES-Index-Cleanup`.
|
||||
|
||||
2. If the Elasticsearch data is no longer coherent with the OpenStack
|
||||
services, the admin will need to re-index Elasticsearch. This is done by
|
||||
running the command ``searchlight-manage`` similarly to step 4 in
|
||||
the deployment stage. See :ref:`ES-Bulk-Indexing`.
|
||||
|
||||
**Figure 2: Searchlight Eco-System**
|
||||
|
||||
.. image:: ../../../images/Figure2.png
|
||||
|
||||
Searchlight Flow
|
||||
----------------
|
||||
|
||||
The main portion of the Eco-system, the running stage, is showcased in Figure
|
||||
3. The diagram is from the viewpoint of the users of the Eco-system.
|
||||
|
||||
The clients will access the OpenStack services. This could include issuing
|
||||
commands to create a new server, delete a volume or modify a networking subnet.
|
||||
The commands can be sent directly to the OpenStack service's API or indirectly
|
||||
through the Horizon dashboard. Once these commands are executed by the
|
||||
OpenStack services, the OpenStack services will notify Searchlight of the
|
||||
resulting state changes.
|
||||
|
||||
Simultaneously, the user can make queries of Searchlight to better understand
|
||||
the entire OpenStack Eco-system. These queries can be made directly through the
|
||||
Searchlight API or indirectly through the Horizon dashboard.
|
||||
|
||||
**Figure 3: Searchlight Flow**
|
||||
|
||||
.. image:: ../../../images/Figure3.png
|
||||
|
||||
Searchlight Scaling
|
||||
-------------------
|
||||
|
||||
When deploying Searchlight, the Eco-system is scaled in multiple ways. This
|
||||
scaling is done for both performance and reliability. Both the Searchlight API
|
||||
and the Searchlight listener are stateless microservices. This makes it easier
|
||||
to scale them. Figure 4 shows this in glorious color.
|
||||
|
||||
Searchlight API Services:
|
||||
|
||||
* Multiple instances of the Searchlight API service can be deployed and placed
|
||||
behind a load balancer. The load balancer hides the details of the
|
||||
Searchlight API services from the user. All user API access will go through
|
||||
the load balancer. In addition, the WSGI web service has multiple threads
|
||||
handling all requests.
|
||||
|
||||
Searchlight Listeners:
|
||||
|
||||
* There are multiple notification workers that are receiving messages from the
|
||||
OpenStack services. The number of workers can be statically specified in the
|
||||
Searchlight configuration file. In addition, each type of resource indexed by
|
||||
Searchlight can be scaled independently by spawning different processes with
|
||||
different configuration files.
|
||||
|
||||
Elasticsearch:
|
||||
|
||||
* Elasticsearch is deployed with multiple nodes in a cluster. Each cluster can
|
||||
also have multiple shards and replicas. The number of nodes, shards and
|
||||
replicas are specified in the Elasticsearch configuration file. But the
|
||||
number of shards and replicas can be overwritten on a per-index basis by
|
||||
Searchlight. See :ref:`Indexing-Model`
|
||||
|
||||
**Figure 4: Searchlight Scaling**
|
||||
|
||||
.. image:: ../../../images/Figure4.png
|
||||
|
||||
Component Interfaces
|
||||
--------------------
|
||||
|
||||
Tables 1 and 2 provide more details with the interfaces in Figure 1. These
|
||||
details are geared toward understanding the security model and analyzing
|
||||
potential threats for Searchlight.
|
||||
|
||||
**Table 1: Interfaces**
|
||||
|
||||
========= ================ ===================== ==================== =================== ====================== ================= =================== ==================== =====================================
|
||||
Interface Network protocol Requestor Request Request credentials Request authorization Listener Response Response credentials Description of operation
|
||||
========= ================ ===================== ==================== =================== ====================== ================= =================== ==================== =====================================
|
||||
1 HTTPS Searchlight UI Horizon Searches Session token Admin or User Searchlight API Search results TLS Certificate Search query from Horizon.
|
||||
1 HTTPS Searchlight User CLI User Searches Session token Admin or User Searchlight API Search results TLS Certificate Search query form user.
|
||||
2 HTTP Searchlight API Query None None Elasticsearch Query Results None Elasticsearch access to
|
||||
query data.
|
||||
2 HTTP Searchlight Listener Query/Index None None Elasticsearch Query/Index Results None Elasticsearch access to
|
||||
re-index or query data.
|
||||
2 HTTP Searchlight Admin CLI Query/Index None None Elasticsearch Query/Index Results None Elasticsearch access to
|
||||
re-index or query data.
|
||||
3 HTTPS Searchlight Listener Service State Update Session token Admin OpenStack Service State Changes TLS Certificate Querying Service state
|
||||
changes.
|
||||
3 HTTPS Searchlight Admin CLI Service State Update Session token Admin OpenStack Service State Changes TLS Certificate Querying Service state
|
||||
changes.
|
||||
4 AMQP Service Connect to MQ Service's MQ Service MQ Account RabbitMQ State Changes TLS Certificate Service connects to MQ and
|
||||
pushes state changes.
|
||||
4 AMQP Searchlight Listener Connect to MQ Searchlight's MQ Searchlight MQ Account RabbitMQ State Changes TLS Certificate Searchlight Listener connects
|
||||
to MQ to listen for state
|
||||
changes pushed by OpenStack services.
|
||||
========= ================ ===================== ==================== =================== ====================== ================= =================== ==================== =====================================
|
||||
|
||||
**Table 2: Default Network Ports**
|
||||
|
||||
============ ========= ==========
|
||||
Port / Range Protocol Notes
|
||||
============ ========= ==========
|
||||
5671 AMQPS Messaging protocol
|
||||
9200 HTTP Elasticsearch protocol
|
||||
9393 HTTPS Searchlight API access
|
||||
============ ========= ==========
|
||||
|
||||
Security Controls
|
||||
-----------------
|
||||
|
||||
Searchlight's security and threat analysis can be viewed through multiple control points.
|
||||
|
||||
* **Audit:** Searchlight performs logging and sends logs to the Logstash
|
||||
central logging service.
|
||||
* **Authentication:** Searchlight requires a valid domain, username and
|
||||
password to access the services. Keystone handles authentication and
|
||||
authorization via session tokens. Note: No authentication is used for
|
||||
Elasticsearch.
|
||||
* **Authorization:** Searchlight provides access to user information.
|
||||
Authentication and authorization is handled by Keystone.
|
||||
* **Availability:** Searchlight uses HAProxy. The users can deploy
|
||||
Elasticsearch in a high-availability mode if needed.
|
||||
* **Confidentiality:** Searchlight API is configured to use TLS, traffic
|
||||
between hosts is communicated using TLS. Data and config files protected
|
||||
via filesystem controls. Note: No TLS between Searchlight and Elasticsearch.
|
||||
* **Integrity:** Data is stored in Elasticsearch, but not backed up. The data
|
||||
is always accessible from the OpenStack services themselves. If data in
|
||||
Elasticsearch ever gets corrupted or out of sync, a resync will restore the
|
||||
data integrity.
|
||||
|
||||
Dependencies
|
||||
------------
|
||||
|
||||
Searchlight has dependencies on other components. The OpenStack services are
|
||||
"soft" dependencies. The dependency exists only if the plugin for that particular
|
||||
service is enabled.
|
||||
|
||||
* Elasticsearch
|
||||
* RabbitMQ
|
||||
* HAProxy
|
||||
* Cinder Service
|
||||
* Designate Service
|
||||
* Glance Service
|
||||
* Neutron Service
|
||||
* Nova Service
|
||||
* Swift Service
|
@ -1,11 +0,0 @@
|
||||
======================
|
||||
Administration guide
|
||||
======================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:glob:
|
||||
|
||||
architecture
|
||||
indexingservice
|
||||
plugins/*
|
@ -1,330 +0,0 @@
|
||||
..
|
||||
Copyright (c) 2015 Hewlett-Packard Development Company, L.P.
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
Searchlight Indexing
|
||||
====================
|
||||
In order for the Searchlight API service to return results, information
|
||||
must be indexed. The two primary mechanisms by which this happens are indexing
|
||||
from the source (which allows a complete index rebuild) and incrementally
|
||||
updating the index based on information received via notifications.
|
||||
The information indexed is determined by a plugin model.
|
||||
|
||||
Search plugins
|
||||
--------------
|
||||
|
||||
The search service determines the type of information that is indexed and
|
||||
searchable via a plugin mechanism.
|
||||
|
||||
See :ref:`searchlight-plugins` for plugin installation and general
|
||||
configuration information.
|
||||
|
||||
See each plugin below for detailed information about specific plugins:
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
plugins/*
|
||||
|
||||
.. _Indexing-Model:
|
||||
|
||||
Indexing model
|
||||
--------------
|
||||
The Mitaka Searchlight release introduced the ability to continue executing
|
||||
search requests while reindexing operations are running. This feature is called
|
||||
*zero-downtime reindexing*. In order to implement zero-downtime indexing, the
|
||||
concept of a *resource group* was introduced.
|
||||
|
||||
A *resource group* is a collection of plugins that share an Elasticsearch
|
||||
index. Since each plugin represents a *resource type*, you can think of a
|
||||
resource group as a collection of resource types.
|
||||
|
||||
For each resource group, Searchlight creates an index whose name consists of
|
||||
the resource group name appended with a timestamp. Each resource group is
|
||||
referred to by a pair of Elasticsearch aliases_. One alias is used for
|
||||
searching by the API (the *search alias*), and the other (the *listener alias*)
|
||||
is used to index incoming events.
|
||||
|
||||
During reindexing, a new index is created, and the listener alias is pointed at
|
||||
both the old and new indices. Incoming events are therefore indexed into both
|
||||
indices. The search alias is left pointing at the old index. Once indexing is
|
||||
finished, both aliases are pointed solely at the new index and the old index
|
||||
is deleted.
|
||||
|
||||
In order to improve the performance of reindexing, index refresh of the new
|
||||
index is disabled during reindexing, and turned on after reindexing is done.
|
||||
As a consequence, Documents synced to the new index are not searchable until
|
||||
index is refreshed, but document retrieval by IDs still works, because GET
|
||||
operation in Elasticsearch is realtime.
|
||||
|
||||
It is important to note that zero-downtime reindexing requires that **all**
|
||||
plugins in a resource group are indexed together. When it's desired to index an
|
||||
individual resource type, an optimization copies existing data directly from
|
||||
the old index to the new one to avoid re-harvesting the data from each service
|
||||
API.
|
||||
|
||||
.. note::
|
||||
Due to some limitations discovered during the Mitaka release, indexing into
|
||||
multiple indices (multiple plugin resource groups) is disabled. The newton release
|
||||
implemented full support for specifying different resource groups for different
|
||||
resource types.
|
||||
|
||||
.. _using-resource-groups:
|
||||
|
||||
Sorting across resource groups
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Using multiple resource groups will impact sort behavior when sorting on fields
|
||||
across resource types when all the resource types don't have the requested 'sort-by field'.
|
||||
Follow the guidelines below to avoid errors:
|
||||
|
||||
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_ignoring_unmapped_fields
|
||||
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_missing_values
|
||||
|
||||
.. _aliases: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html
|
||||
|
||||
.. _ES-Bulk-Indexing:
|
||||
|
||||
Bulk indexing
|
||||
-------------
|
||||
To initially create the catalog index (or add new resource typs to it later),
|
||||
run the following command::
|
||||
|
||||
$ searchlight-manage index sync
|
||||
|
||||
This will iterate through all registered and enabled search plugins and
|
||||
request that they perform a full indexing of all data that's available to them.
|
||||
|
||||
It is also possible to index just a single resource, or all resources
|
||||
belonging to a resource group. For instance, to index all glance images::
|
||||
|
||||
$ searchlight-manage index sync --type OS::Glance::Image
|
||||
|
||||
As described above, this will create a new index for all plugins that share a
|
||||
resource group with OS::Glance::Image. The management command will retrieve
|
||||
up-to-date information from the Glance API. Data for other plugins will be
|
||||
bulk-copied from a preexisting index into the new one using the scroll_ and
|
||||
bulk_ features of Elasticsearch.
|
||||
|
||||
You can use the wildcard character * at the *end* of the ``type`` argument.
|
||||
For instance, the following will match all cinder plugins::
|
||||
|
||||
$ searchlight-manage index sync --type OS::Cinder::*
|
||||
|
||||
Wildcard characters are only allowed at the end of the argument; they will not
|
||||
be matched anywhere else.
|
||||
|
||||
To index all resources in the 'searchlight' resource group::
|
||||
|
||||
$ searchlight-manage index sync --index searchlight
|
||||
|
||||
You will be prompted to confirm unless ``--force`` is provided.
|
||||
|
||||
To reindex resources only without notification::
|
||||
|
||||
$ searchlight-manage index sync --notification-less
|
||||
|
||||
.. note::
|
||||
|
||||
We *strongly* recommend putting the notification-less plugins in their own
|
||||
resource group and scheduling a cron_
|
||||
job to periodically re-sync the notification plugins to keep the documents
|
||||
up to date.
|
||||
|
||||
The ``searchlight-manage index sync`` command may be re-run at any time to
|
||||
perform a full re-index of the data. As described above, there should be no
|
||||
or very little impact on search requests during this process.
|
||||
|
||||
.. _scroll: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html
|
||||
.. _bulk: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html
|
||||
.. _cron: https://en.wikipedia.org/wiki/Cron
|
||||
|
||||
Parent/child relationships
|
||||
--------------------------
|
||||
Some plugins contain multiple resources with parent/child relationships;
|
||||
the Designate plugins are an example. Because reindexing parent data independent
|
||||
of child documents does not logically make sense (without orphaning them), it
|
||||
is not possible to request indexing of a child resource type::
|
||||
|
||||
$ searchlight-manage index sync --type OS::Designate::RecordSet
|
||||
|
||||
'OS::Designate::RecordSet' is a child of 'OS::Designate::Zone' and cannot be indexed separately.
|
||||
Indexing 'OS::Designate::Zone' will re-index all child resource types.
|
||||
|
||||
You can see parent/child relationships in the list of resources presented prior
|
||||
to indexing::
|
||||
|
||||
$ searchlight-manage index sync --type OS::Designate::Zone
|
||||
|
||||
Resource types (and indices) matching selection:
|
||||
OS::Designate::Zone (designate)
|
||||
----> OS::Designate::RecordSet
|
||||
|
||||
Child plugins will inherit their resource group name from their parent. Any
|
||||
child configuration setting for resource_group_name will be ignored.
|
||||
|
||||
Incremental Updates
|
||||
-------------------
|
||||
|
||||
Once a resource has been indexed, typically you will only need to consume
|
||||
incremental updates rather than re-index the entire data set again. The
|
||||
preferred methodology is to set up notification listening.
|
||||
|
||||
Notifications
|
||||
^^^^^^^^^^^^^
|
||||
Many services publish notifications when there are changes to the resources
|
||||
they own. The searchlight listener consumes these notifications and will
|
||||
perform incremental updates to the index based on those notifications.
|
||||
|
||||
To start this service, run the following command::
|
||||
|
||||
$ searchlight-listener
|
||||
|
||||
Note, this will typically require that you have configured notifications
|
||||
properly for the service which owns the resource. For example, the glance
|
||||
service owns images and metadata definitions. Please check the plugin
|
||||
documentation for each service's specific configuration requirements.
|
||||
|
||||
Multi-Thread Support
|
||||
--------------------
|
||||
The Newton Searchlight release introduced multiple thread support for
|
||||
indexing. Previously when the ``searchlight-manage index sync``
|
||||
command was executed, all indexing occurred in a single thread. To boost
|
||||
the performance of the indexing functionality, each resource type
|
||||
will now index in its own thread. Multiple indexing threads will run
|
||||
concurrently.
|
||||
|
||||
By default, the maximum number of simultaneous threads is 3. This limit
|
||||
can be modified in the Searchlight configuration file. The setting is
|
||||
called ``workers`` and lives under ``[manage]``. For example, to
|
||||
increase the maximum number of threads to 6, the following can be added
|
||||
to the Searchlight configuration file::
|
||||
|
||||
[manage]
|
||||
workers=6
|
||||
|
||||
The use of threads can also affect the parsing of the log files. The
|
||||
default formatting of the log messages include only the process ID,
|
||||
but no thread-specific information. This can be changed by modifying
|
||||
the formatting string settings in the Searchlight configuration file.
|
||||
To add the thread ID for a message, add ``%(thread)d``. To add the thread
|
||||
name, add ``%(threadName)s``. For example, to add the thread ID and the
|
||||
thread name after the process ID to the logging message, the following
|
||||
setting can be added to the Searchlight configuration file::
|
||||
|
||||
logging_default_format_string = %(asctime)s.%(msecs)03d %(process)d %(thread)d %(threadName)s %(levelname)s %(name)s [-] %(instance)s%(message)s
|
||||
|
||||
Force Elasticsearch indexing
|
||||
----------------------------
|
||||
The Newton Searchlight release introduced the ability to reindex
|
||||
from Elasticsearch only, bypassing the plugin APIs altogether.
|
||||
This option is useful if there has been a change to the mapping
|
||||
definitions or the index settings. This functionality is enabled
|
||||
with the option ``--apply-mapping-changes`` for the ``index`` command.
|
||||
|
||||
A sample usage would be::
|
||||
|
||||
$ searchlight-manage index aliases --apply-mapping-changes
|
||||
|
||||
The ``--type`` option is not compatible with the ``--apply-mapping-changes``
|
||||
option. Specifying both options on the command line will result in an error.
|
||||
|
||||
.. warning::
|
||||
|
||||
The resource group cannot be changed when using this option.
|
||||
If you do change the resource group, the underlying index will
|
||||
be changed and will result in an empty index.
|
||||
|
||||
.. _ES-Index-Cleanup:
|
||||
|
||||
Elasticsearch Index Cleanup
|
||||
---------------------------
|
||||
|
||||
In some cases, there may be orphaned Searchlight indices in Elasticsearch.
|
||||
An orphaned index is one that is no longer used by Searchlight, either
|
||||
directly or through an alias.
|
||||
|
||||
To help detect which Searchlight-related indices may be orphaned in
|
||||
Elasticsearch, the ``searchlight-manage`` command will display all indices
|
||||
that are currently being used by Searchlight. This is the ``aliases``
|
||||
option to the ``index`` command::
|
||||
|
||||
$ searchlight-manage index aliases
|
||||
|
||||
This command outputs a listing of all indices that are used by
|
||||
Searchlight (based on the current configuration file). The aliases
|
||||
associated with each index is also shown. A sample output will look
|
||||
like this::
|
||||
|
||||
$ searchlight-manage index aliases
|
||||
List of Elasticsearch indices (and their associated aliases) used by Searchlight.
|
||||
|
||||
Note:
|
||||
The indices are based on the current config file.
|
||||
To view indices used by other Searchlight config files, use the --config-file option.
|
||||
|
||||
Indices are denoted with a '*'
|
||||
Aliases are denoted with a '+'
|
||||
|
||||
* searchlight-2016_07_13_17_09_27
|
||||
+ searchlight-listener
|
||||
+ searchlight-search
|
||||
* sl-swift-2016_07_13_17_09_26
|
||||
+ sl-swift-listener
|
||||
+ sl-swift-search
|
||||
|
||||
The example shows that Searchlight is using two indices in Elasticsearch:
|
||||
``searchlight-2016_07_13_17_09_27`` and ``sl-swift-2016_07_13_17_09_26``.
|
||||
The index ``searchlight-2016_07_13_17_09_27`` has two aliases: ``searchlight-listener``
|
||||
and ``searchlight-search``. The index ``sl-swift-2016_07_13_17_09_26`` has
|
||||
two aliases: ``sl-swift-listener`` and ``sl-swift-search``.
|
||||
|
||||
Any other indices or aliases in Elasticsearch are not used by this specific
|
||||
Searchlight configuration. NOTE: If there are other Searchlight
|
||||
instances running with a different configuration, their indices and aliases
|
||||
will not be displayed by this command. The user will need to rerun the
|
||||
``index aliases`` command using other configuration files.
|
||||
|
||||
.. _Notifications:
|
||||
|
||||
Notifications
|
||||
=============
|
||||
Aside from periodic reindexing, Searchlight can index in close to real-time
|
||||
by subscribing to oslo.messaging notifications_. To configure services to send
|
||||
them, it's typically only necessary to enable a driver in their configuration
|
||||
files::
|
||||
|
||||
[oslo_messaging_notifications]
|
||||
driver = messaging
|
||||
|
||||
Most notifications are emitted by a service's API but some services also emit
|
||||
them in scheduler or conductor components.
|
||||
|
||||
Searchlight subscribes to notifications using a listener pool_ so it will not
|
||||
steal notifications from e.g. telemetry projects.
|
||||
|
||||
.. _oslo-notifications: https://wiki.openstack.org/wiki/Oslo/Messaging#Emitting_Notifications
|
||||
.. _pool: https://specs.openstack.org/openstack/oslo-specs/specs/kilo/notification-listener-pools.html
|
||||
|
||||
RabbitMQ
|
||||
--------
|
||||
The rabbitmq driver is the most commonly used. The permissions_ required for
|
||||
searchlight are::
|
||||
|
||||
* configure: '^(searchlight-listener|openstack|nova|neutron|cinder|glance)$'
|
||||
* write: '^(searchlight-listener)$'
|
||||
* read: '^(searchlight-listener|nova|glance|cinder|neutron|openstack)$'
|
||||
|
||||
.. _permissions: https://www.rabbitmq.com/access-control.html
|
@ -1,117 +0,0 @@
|
||||
..
|
||||
c) Copyright 2016 Hewlett-Packard Enterprise Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
*******************
|
||||
Cinder Plugin Guide
|
||||
*******************
|
||||
|
||||
Integration is provided via a plugin. There are multiple configuration
|
||||
settings required for proper indexing and incremental updates. Some of the
|
||||
settings are specified in Searchlight configuration files. Others are
|
||||
provided in other service configuration files.
|
||||
|
||||
Searchlight Configuration
|
||||
=========================
|
||||
|
||||
Searchlight resource configuration options are shown below with their
|
||||
configuration file and default values.
|
||||
|
||||
See :ref:`searchlight-plugins` for common options with their default values,
|
||||
general configuration information, and an example complete configuration.
|
||||
|
||||
.. note::
|
||||
|
||||
Unless you are changing to a non-default value, you do not need to
|
||||
specify any of the following configuration options.
|
||||
|
||||
searchlight.conf
|
||||
----------------
|
||||
|
||||
Plugin: OS::Cinder::Volume
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_cinder_volume]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
Plugin: OS::Cinder::Snapshot
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_cinder_snapshot]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
Cinder Configuration
|
||||
====================
|
||||
|
||||
Cinder sends notifications on create/update/delete actions on the
|
||||
resources that it implements. Currently Searchlight supports indexing
|
||||
for volumes and snapshots of volumes. Backup support will be added but
|
||||
some changes to the cinder backup API is required first.
|
||||
|
||||
cinder.conf
|
||||
-----------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Enable notifications using the following::
|
||||
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
.. note::
|
||||
|
||||
Restart the Cinder api service (c-api) after making changes.
|
||||
See :ref:`plugin_notifications` for more information on
|
||||
notification topics.
|
||||
|
||||
local.conf (devstack)
|
||||
---------------------
|
||||
|
||||
The settings above may be automatically configured by ``stack.sh``
|
||||
by adding them to the following post config section in devstack.
|
||||
Just place the following in local.conf and copy the above settings
|
||||
underneath it.::
|
||||
|
||||
[[post-config|$CINDER_CONF]]
|
||||
[DEFAULT]
|
||||
|
||||
Release Notes
|
||||
=============
|
||||
|
||||
0.2.0.0 (Mitaka)
|
||||
----------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Searchlight must use its own topic. Use the following::
|
||||
|
||||
notification_driver = messaging
|
||||
notification_topics = searchlight_indexer
|
||||
|
||||
The following fields are exposed to administrators only for cinder volumes:
|
||||
* os-vol-mig-status-attr:*
|
||||
* os-vol-host-attr:*
|
||||
* migration
|
||||
|
||||
Additional properties can be similarly protected with the `admin_only_fields`
|
||||
under each plugin's configuration section. Glob-like patterns are supported.
|
||||
For instance::
|
||||
|
||||
[resource_plugin:os_cinder_volume]
|
||||
admin_only_fields=status,bootable
|
||||
|
||||
See: ADMIN_ONLY_FIELDS in:
|
||||
* searchlight/elasticsearch/plugins/cinder/volumes.py
|
@ -1,150 +0,0 @@
|
||||
..
|
||||
c) Copyright 2015 Hewlett-Packard Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
**********************
|
||||
Designate Plugin Guide
|
||||
**********************
|
||||
|
||||
Integration is provided via a plugin. There are multiple configuration
|
||||
settings required for proper indexing and incremental updates. Some of the
|
||||
settings are specified in Searchlight configuration files. Others are
|
||||
provided in other service configuration files.
|
||||
|
||||
Searchlight Configuration
|
||||
=========================
|
||||
|
||||
Searchlight resource configuration options are shown below with their
|
||||
configuration file and default values.
|
||||
|
||||
See :ref:`searchlight-plugins` for common options with their default values,
|
||||
general configuration information, and an example complete configuration.
|
||||
|
||||
.. note::
|
||||
|
||||
Unless you are changing to a non-default value, you do not need to
|
||||
specify any of the following configuration options.
|
||||
|
||||
searchlight.conf
|
||||
----------------
|
||||
|
||||
Plugin: OS::Designate::Zone
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_designate_zone]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
Plugin: OS::Designate::RecordSet
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_designate_recordset]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
.. warning::
|
||||
|
||||
*OS::Designate::Zone* documents have a parent relationship to
|
||||
*OS::Designate::RecordSet* documents. Because of this you must have
|
||||
both *os_designate_zone* and *os_designate_recordset* plugin
|
||||
configurations enabled or disabled together.
|
||||
|
||||
Designate Configuration
|
||||
=======================
|
||||
|
||||
The Designate services must be configured properly to work with searchlight.
|
||||
|
||||
designate.conf
|
||||
--------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Enable notifications using the following::
|
||||
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
.. note::
|
||||
|
||||
Restart ``designate-central``, ``designate-pool-manager``, and
|
||||
``designate-zone-manager`` after making changes.
|
||||
See :ref:`plugin_notifications` for more information on
|
||||
notification topics.
|
||||
|
||||
local.conf (devstack)
|
||||
---------------------
|
||||
|
||||
.. note::
|
||||
|
||||
Designate resource types are *not* enabled by default (``enabled = false``)
|
||||
in the Searchlight devstack script because Designate is not
|
||||
installed by default in devstack. If you have Designate installed in
|
||||
devstack, you have two options for enabling designate resource types in
|
||||
Searchlight:
|
||||
|
||||
1. Prior to stacking: modify the searchlight post config section in
|
||||
``local.conf`` by adding a ``[[post-config|$SEARCHLIGHT_CONF]]`` section.
|
||||
|
||||
2. After stacking: manually edit the ``searchlight.conf`` file.
|
||||
|
||||
The Designate plugin must be enabled and run with devstack to include Designate
|
||||
with your devstack deployment. Follow the instructions here:
|
||||
https://opendev.org/openstack/designate/src/branch/master/devstack
|
||||
|
||||
The settings above may be automatically configured by ``stack.sh``
|
||||
by adding them to the following post config section in devstack.
|
||||
Just place the following in local.conf and copy the above settings
|
||||
underneath it.::
|
||||
|
||||
[[post-config|$DESIGNATE_CONF]]
|
||||
[DEFAULT]
|
||||
|
||||
Release Notes
|
||||
=============
|
||||
|
||||
0.2.0.0 (Mitaka)
|
||||
----------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Searchlight must use its own topic. Use the following::
|
||||
|
||||
notification_driver = messaging
|
||||
notification_topics = searchlight_indexer
|
||||
|
||||
The Designate notification limitations mentioned in Liberty still apply.
|
||||
|
||||
You no longer need to use the --no-delete option mentioned below. Zero
|
||||
downtime reindexing implemented in Mitaka handles all re-indexing
|
||||
transparently.
|
||||
|
||||
0.1.0.0 (Liberty)
|
||||
-----------------
|
||||
|
||||
For best results, use the v2 Designate API. Using the Designate v1 API to
|
||||
create domains results in the Designate service not sending all possible
|
||||
status change notifications. This causes Designate record set documents to
|
||||
stay in the ``Pending`` status in the search index.
|
||||
|
||||
The Horizon UI uses the v1 API and causes the above issue to be seen.
|
||||
So in order to ensure the search index contains the correct status values
|
||||
for record sets when using Horizon, you may set up a cron job to
|
||||
re-index Designate data.
|
||||
|
||||
You should use the ``--no-delete`` option to prevent the index from
|
||||
temporarily not containing any data (which otherwise would happen with a full
|
||||
bulk indexing job)::
|
||||
|
||||
searchlight-manage index sync --type OS::Designate::Zone,OS::Designate::RecordSet --force --no-delete
|
||||
|
@ -1,143 +0,0 @@
|
||||
..
|
||||
c) Copyright 2015 Hewlett-Packard Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
*******************
|
||||
Glance Plugin Guide
|
||||
*******************
|
||||
|
||||
Integration is provided via a plugin. There are multiple configuration
|
||||
settings required for proper indexing and incremental updates. Some of the
|
||||
settings are specified in Searchlight configuration files. Others are
|
||||
provided in other service configuration files.
|
||||
|
||||
Searchlight Configuration
|
||||
=========================
|
||||
|
||||
Searchlight resource configuration options are shown below with their
|
||||
configuration file and default values.
|
||||
|
||||
See :ref:`searchlight-plugins` for common options with their default values,
|
||||
general configuration information, and an example complete configuration.
|
||||
|
||||
.. note::
|
||||
|
||||
Unless you are changing to a non-default value, you do not need to
|
||||
specify any of the following configuration options.
|
||||
|
||||
searchlight.conf
|
||||
----------------
|
||||
|
||||
Plugin: OS::Glance::Image
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_glance_image]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
**Glance Image Property Protections**
|
||||
|
||||
Glance uses a property protections mechanism to ensure that certain
|
||||
properties are limited to only people with the appropriate permissions.
|
||||
Searchlight includes the same functionality and must be deployed with
|
||||
the same property protections files and configured to use that file. A
|
||||
sample configuration file is included in the repo and may be used for testing.
|
||||
|
||||
To configure it, add a ``property_protection_file`` property with a path
|
||||
to the file in ``searchlight.conf``. For example::
|
||||
|
||||
property_protection_file = /etc/searchlight/property-protections-roles.conf
|
||||
|
||||
See also: `Glance Property Protections <https://docs.openstack.org/glance/latest/admin/property-protections.html>`_
|
||||
|
||||
Plugin: OS::Glance::Metadef
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_glance_metadef]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
See also: `Metadata Definitions Catalog <https://docs.openstack.org/glance/latest/user/metadefs-concepts.html>`_
|
||||
|
||||
Glance Configuration
|
||||
====================
|
||||
|
||||
The Glance services must be configured properly to work with searchlight.
|
||||
|
||||
glance-api.conf
|
||||
---------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Enable notifications using the following::
|
||||
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
.. note::
|
||||
|
||||
Restart the Glance api service (g-api) after making changes.
|
||||
See :ref:`plugin_notifications` for more information on
|
||||
notification topics.
|
||||
|
||||
local.conf (devstack)
|
||||
---------------------
|
||||
|
||||
The settings above may be automatically configured by ``stack.sh``
|
||||
by adding them to the following post config section in devstack.
|
||||
Just place the following in local.conf and copy the above settings
|
||||
underneath it.::
|
||||
|
||||
[[post-config|$GLANCE_API_CONF]]
|
||||
[DEFAULT]
|
||||
|
||||
Release Notes
|
||||
=============
|
||||
|
||||
0.2.0.0 (Mitaka)
|
||||
----------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Searchlight must use its own topic. Use the following::
|
||||
|
||||
notification_driver = messaging
|
||||
notification_topics = searchlight_indexer
|
||||
|
||||
0.1.0.0 (Liberty)
|
||||
-----------------
|
||||
|
||||
Glance did not generate notifications for Image Member updates up to and
|
||||
including the Liberty release.
|
||||
|
||||
This means that search results will include correct results when the image
|
||||
visibility is ``public`` or ``private``, but ``shared`` images will only be
|
||||
included in search results for the owning project without additional deployment
|
||||
configuration.
|
||||
|
||||
The patch (https://review.opendev.org/221307) implements this feature and
|
||||
will be included/merged in the Glance Mitaka release.
|
||||
|
||||
Searchlight developers/installers should apply the above patch in Glance when
|
||||
using Searchlight with the Glance Liberty release.
|
||||
|
||||
Alternatively, you may set up a cron job to re-index glance images
|
||||
periodically to get updated membership information.
|
||||
|
||||
You should use the ``--no-delete`` option to prevent the index from
|
||||
temporarily not containing any data (which otherwise would happen with a full
|
||||
bulk indexing job)::
|
||||
|
||||
searchlight-manage index sync --type OS::Glance::Image --force --no-delete
|
||||
|
@ -1,86 +0,0 @@
|
||||
..
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
*******************
|
||||
Ironic Plugin Guide
|
||||
*******************
|
||||
|
||||
Integration is provided via a plugin. There are multiple configuration
|
||||
settings required for proper indexing and incremental updates. Some of the
|
||||
settings are specified in Searchlight configuration files. Others are
|
||||
provided in other service configuration files.
|
||||
|
||||
Searchlight Configuration
|
||||
=========================
|
||||
|
||||
Searchlight resource configuration options are shown below with their
|
||||
configuration file.
|
||||
|
||||
See :ref:`searchlight-plugins` for common options with their default values,
|
||||
general configuration information, and an example complete configuration.
|
||||
|
||||
searchlight.conf
|
||||
----------------
|
||||
|
||||
Plugin: OS::Ironic::Chassis
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_ironic_chassis]
|
||||
notifications_topics_exchanges = ironic_versioned_notifications,ironic
|
||||
|
||||
.. note::
|
||||
|
||||
Chassis is not parent resource for node because chassis is not mandatory
|
||||
for a node.
|
||||
|
||||
Plugin: OS::Ironic::Node
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_ironic_node]
|
||||
notifications_topics_exchanges = ironic_versioned_notifications,ironic
|
||||
|
||||
Plugin: OS::Ironic::Port
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_ironic_port]
|
||||
notifications_topics_exchanges = ironic_versioned_notifications,ironic
|
||||
|
||||
Ironic Configuration
|
||||
====================
|
||||
|
||||
The ironic services must be configured properly to work with searchlight.
|
||||
|
||||
ironic.conf
|
||||
-----------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Enable notifications using the following::
|
||||
|
||||
[DEFAULT]
|
||||
notification_level = info
|
||||
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
.. note::
|
||||
|
||||
Restart ironic api and conductor services after making changes.
|
||||
|
||||
Release Notes
|
||||
=============
|
||||
|
||||
``properties`` node's field mapped to ``node_properties`` due to limitation of
|
||||
older elasticsearch versions.
|
@ -1,196 +0,0 @@
|
||||
..
|
||||
c) Copyright 2016 Hewlett-Packard Enterprise Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
********************
|
||||
Neutron Plugin Guide
|
||||
********************
|
||||
|
||||
Integration is provided via a plugin. There are multiple configuration
|
||||
settings required for proper indexing and incremental updates. Some of the
|
||||
settings are specified in Searchlight configuration files. Others are
|
||||
provided in other service configuration files.
|
||||
|
||||
Searchlight Configuration
|
||||
=========================
|
||||
|
||||
Searchlight resource configuration options are shown below with their
|
||||
configuration file and default values.
|
||||
|
||||
See :ref:`searchlight-plugins` for common options with their default values,
|
||||
general configuration information, and an example complete configuration.
|
||||
|
||||
.. note::
|
||||
|
||||
Unless you are changing to a non-default value, you do not need to
|
||||
specify any of the following configuration options.
|
||||
|
||||
searchlight.conf
|
||||
----------------
|
||||
|
||||
Plugin: OS::Neutron::Net
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_neutron_net]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
Plugin: OS::Neutron::Port
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_neutron_port]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
Plugin: OS::Neutron::Subnet
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_neutron_subnet]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
Plugin: OS::Neutron::Router
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_neutron_router]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
Plugin: OS::Neutron::FloatingIP
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_neutron_floatingip]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
Plugin: OS::Neutron::SecurityGroup
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_neutron_security_group]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
Neutron Configuration
|
||||
=====================
|
||||
|
||||
Neutron sends notifications on create/update/delete actions on the
|
||||
concepts that it implements. Currently Searchlight supports indexing
|
||||
for networks, subnets, ports, routers, floating IPs and security groups.
|
||||
|
||||
neutron.conf
|
||||
------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Enable notifications using the following::
|
||||
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
.. note::
|
||||
|
||||
Restart the Neutron api service (q-svc) after making changes.
|
||||
See :ref:`plugin_notifications` for more information on
|
||||
notification topics.
|
||||
|
||||
local.conf (devstack)
|
||||
---------------------
|
||||
|
||||
The settings above may be automatically configured by ``stack.sh``
|
||||
by adding them to the following post config section in devstack.
|
||||
Just place the following in local.conf and copy the above settings
|
||||
underneath it.::
|
||||
|
||||
[[post-config|$NEUTRON_CONF]]
|
||||
[DEFAULT]
|
||||
|
||||
Neutron RBAC Reference
|
||||
======================
|
||||
|
||||
RBAC Filters
|
||||
------------
|
||||
RBAC in searchlight neutron plugin is implemented as:
|
||||
|
||||
Networks are visible within a tenant OR if they are shared OR if they are external.
|
||||
Subnets are visible within a tenant OR if their network is shared (OR for admins if their network is external).
|
||||
Ports are visible within a tenant (OR for admins if their network is shared or external).
|
||||
Routers are visible within a tenant.
|
||||
Floating IPs are visible within a tenant.
|
||||
Security groups are visible within a tenant.
|
||||
|
||||
Neutron Ports Notifications
|
||||
===========================
|
||||
|
||||
Not all Neutron ports send notifications when created/updated. Depending on the port's
|
||||
``device_owner``, the notifications will be sent. The ``device_owner``'s that will
|
||||
send a notification are:
|
||||
|
||||
* compute:*
|
||||
* baremetal:*
|
||||
|
||||
We want the initial indexing (and subsequent re-indexings) to match the state that
|
||||
Searchlight receives from the Neutron notifications. Having this mismatch will lead
|
||||
to the Searchlight state being out of sync with the Neutron state. To prevent this
|
||||
from happening, ``searchlight-manage`` will index only the Neutron ports that have
|
||||
``device_owner`` defined above. All other ports with ``device_owner`` not listed
|
||||
above will be ignored when indexing.
|
||||
|
||||
Release Notes
|
||||
=============
|
||||
|
||||
1.0.0.0 (Newton)
|
||||
----------------
|
||||
|
||||
The Neutron tenant RBAC policy functionality is supported as
|
||||
part of the OS::Neutron::Net resource type.
|
||||
|
||||
0.2.0.0 (Mitaka)
|
||||
----------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Searchlight must use its own topic. Use the following::
|
||||
|
||||
notification_driver = messaging
|
||||
notification_topics = searchlight_indexer
|
||||
|
||||
DHCP ports are *not* indexed. Neutron doesn't provide a reliable way for
|
||||
Searchlight to index these ports since they are created and modified
|
||||
asynchronously from the subnets that they're attached to.
|
||||
|
||||
All provider:* properties of networks are exposed to administrators only.
|
||||
All binding:* properties of ports are also visible only to administrators.
|
||||
The 'distributed' and 'ha' router properties are available only to
|
||||
administrators.
|
||||
|
||||
Additional properties can be protected similarly with the `admin_only_fields`
|
||||
under each plugin's configuration section. Glob-like patterns are supported.
|
||||
For instance::
|
||||
|
||||
[resource_plugin:os_neutron_net]
|
||||
admin_only_fields=admin_state_up,status
|
||||
|
||||
See: ADMIN_ONLY_FIELDS in:
|
||||
* searchlight/elasticsearch/plugins/neutron/networks.py
|
||||
* searchlight/elasticsearch/plugins/neutron/ports.py
|
||||
* searchlight/elasticsearch/plugins/neutron/routers.py
|
||||
|
||||
Floating IP addresses are mapped as type 'ip' which only supports ipv4 in
|
||||
Elasticsearch prior to the 5.0 release. Neutron doesn't seem to support ipv6
|
||||
floating IPs since translating ipv4 FIPs to ipv6 internal addresses isn't
|
||||
supported and mapping ipv6->ipv6 is deemed unnecessary.
|
@ -1,231 +0,0 @@
|
||||
..
|
||||
c) Copyright 2015 Hewlett-Packard Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
*****************
|
||||
Nova Plugin Guide
|
||||
*****************
|
||||
|
||||
Integration is provided via a plugin. There are multiple configuration
|
||||
settings required for proper indexing and incremental updates. Some of the
|
||||
settings are specified in Searchlight configuration files. Others are
|
||||
provided in other service configuration files.
|
||||
|
||||
Searchlight Configuration
|
||||
=========================
|
||||
|
||||
Searchlight resource configuration options are shown below with their
|
||||
configuration file and default values.
|
||||
|
||||
See :ref:`searchlight-plugins` for common options with their default values,
|
||||
general configuration information, and an example complete configuration.
|
||||
|
||||
.. note::
|
||||
|
||||
Unless you are changing to a non-default value, you do not need to
|
||||
specify any of the following configuration options.
|
||||
|
||||
searchlight.conf
|
||||
----------------
|
||||
|
||||
Nova microversions
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[service_credentials:nova]
|
||||
compute_api_version = 2.1
|
||||
|
||||
.. note::
|
||||
|
||||
Nova adds/removes fields using microversion mechanism, check
|
||||
https://docs.openstack.org/nova/latest/api_microversion_history.html
|
||||
for detailed Nova microversion history.
|
||||
|
||||
Plugin: OS::Nova::Server
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_nova_server]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
notifications_topics_exchanges = versioned_notifications,nova
|
||||
use_versioned_notifications = true
|
||||
|
||||
Plugin: OS::Nova::Hypervisor
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
This plugin represents each of the Nova compute nodes.
|
||||
|
||||
::
|
||||
|
||||
[resource_plugin:os_nova_hypervisor]
|
||||
enabled = true
|
||||
resource_group_name = sl_without_notification
|
||||
|
||||
.. note::
|
||||
|
||||
There are no notifications from the compute nodes ("hypervisors") from
|
||||
nova yet, so we recommend putting it in its own resource group and
|
||||
scheduling a cron job to periodically re-sync. This will create a very
|
||||
low overhead way to keep the index up to date. The index latency will be
|
||||
dependent on how often you re-sync the data.
|
||||
|
||||
Plugin: OS::Nova::Flavor
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_nova_flavor]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
notifications_topics_exchanges = versioned_notifications,nova
|
||||
|
||||
.. note::
|
||||
|
||||
The notifications topic for flavors is versioned_notifications, so we
|
||||
need to config notifications_topics_exchanges with value
|
||||
'versioned_notifications,nova' in order to get the related versioned
|
||||
notifications from nova.
|
||||
|
||||
Plugin: OS::Nova::SeverGroup
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_nova_servergroup]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
.. note::
|
||||
|
||||
The return value of os-server-groups API from nova doesn't contain
|
||||
project and user information before nova API microversion v2.13,
|
||||
thus the index cannot been searched by particular project.
|
||||
|
||||
Nova Configuration
|
||||
==================
|
||||
|
||||
The nova services must be configured properly to work with searchlight.
|
||||
|
||||
nova.conf
|
||||
---------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Enable notifications using the following::
|
||||
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
[notifications]
|
||||
notify_on_state_change = vm_and_task_state
|
||||
# notification_format = versioned
|
||||
|
||||
.. note::
|
||||
|
||||
Restart Nova API and Nova scheduler (n-api, n-sch) after making changes.
|
||||
See :ref:`plugin_notifications` for more information on
|
||||
notification topics.
|
||||
|
||||
The default setting for notification_format is 'both' which sends both
|
||||
versioned and unversioned notifications. Searchlight uses
|
||||
'use_versioned_notifications' to decide which to use.
|
||||
|
||||
local.conf (devstack)
|
||||
---------------------
|
||||
|
||||
The settings above may be automatically configured by ``stack.sh``
|
||||
by adding them to the following post config section in devstack.
|
||||
Just place the following in local.conf and copy the above settings
|
||||
underneath it.::
|
||||
|
||||
[[post-config|$NOVA_CONF]]
|
||||
[DEFAULT]
|
||||
|
||||
Neutron Configuration
|
||||
=====================
|
||||
|
||||
Since changes to Neutron can affect Nova instances you may optionally turn on
|
||||
notifications for Neutron. If you do not, networking changes will only be
|
||||
picked up by Searchlight when notifications are received from Nova.
|
||||
|
||||
neutron.conf
|
||||
------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Enable notifications using the following::
|
||||
|
||||
[oslo_messaging_notifications]
|
||||
driver = messagingv2
|
||||
|
||||
.. note::
|
||||
|
||||
Restart the Neutron api service (q-svc) after making changes.
|
||||
See :ref:`plugin_notifications` for more information on
|
||||
notification topics.
|
||||
|
||||
local.conf (devstack)
|
||||
---------------------
|
||||
|
||||
The settings above may be automatically configured by ``stack.sh``
|
||||
by adding them to the following post config section in devstack.
|
||||
Just place the following in local.conf and copy the above settings
|
||||
underneath it.::
|
||||
|
||||
[[post-config|$NEUTRON_CONF]]
|
||||
[DEFAULT]
|
||||
|
||||
Release Notes
|
||||
=============
|
||||
|
||||
1.0.0.0 (Newton)
|
||||
----------------
|
||||
In order to reduce the impact on the nova API, changes have been made to the
|
||||
way notifications are processed. Currently searchlight has to retrieve nova
|
||||
server information from nova because the notifications alone are missing
|
||||
several pieces of information. Prior to Newton this meant up to 7 API requests
|
||||
during a server boot. During Newton this was changed. There will now be one
|
||||
initial nova request prior to the scheduler, one when the
|
||||
``instance.create.start`` notification is received, one when networking is
|
||||
established and one after the instance has booted and run any init scripts.
|
||||
Other notifications during boot will update only the server status.
|
||||
|
||||
0.2.0.0 (Mitaka)
|
||||
----------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Searchlight must use its own topic. Use the following::
|
||||
|
||||
notification_driver = messaging
|
||||
notification_topics = searchlight_indexer
|
||||
|
||||
The following fields are exposed to administrators only for nova instances:
|
||||
* OS-EXT-SRV-ATTR:*
|
||||
|
||||
Additional properties can be similarly protected with the `admin_only_fields`
|
||||
under each plugin's configuration section. Glob-like patterns are supported.
|
||||
For instance::
|
||||
|
||||
[resource_plugin:os_nova_server]
|
||||
admin_only_fields=OS-EXT-STS:vm_state
|
||||
|
||||
See: ADMIN_ONLY_FIELDS in:
|
||||
* searchlight/elasticsearch/plugins/nova/servers.py
|
||||
|
||||
0.1.0.0 (Liberty)
|
||||
-----------------
|
||||
|
||||
All OS-EXT-SRV-ATTR:.* properties are filtered out from search results
|
||||
for non-admin users. This is not a configuration option in this release.
|
||||
To change this or filter out additional properties, you must change the
|
||||
plugin code to add additional properties.
|
||||
|
||||
See: ADMIN_ONLY_PROPERTIES in searchlight/elasticsearch/plugins/nova/servers.py
|
@ -1,223 +0,0 @@
|
||||
..
|
||||
(c) Copyright 2016 Hewlett-Packard Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
******************
|
||||
Swift Plugin Guide
|
||||
******************
|
||||
|
||||
WARNING: The Swift plugin is currently EXPERIMENTAL as incremental indexing is
|
||||
only provided via an experimental OSLO messaging middleware patch while
|
||||
other indexing methodologies are explored for Swift.
|
||||
See :ref:`proxy-server.conf` for additional information.
|
||||
|
||||
Integration is provided via a plugin. There are multiple configuration
|
||||
settings required for proper indexing and incremental updates. Some of the
|
||||
settings are specified in Searchlight configuration files. Others are
|
||||
provided in Swift configuration files.
|
||||
|
||||
Swift Configuration
|
||||
====================
|
||||
|
||||
reseller_admin_role
|
||||
-------------------
|
||||
|
||||
Users with the Keystone role defined in `reseller_admin_role` (`ResellerAdmin`
|
||||
by default) can operate on any account. The auth system sets the request
|
||||
context variable `reseller_request` to True if a request is coming from a user
|
||||
with this role.
|
||||
|
||||
Searchlight needs this role for its service user to access all of the swift
|
||||
accounts during initial indexing. The `searchlight` user and `service` project
|
||||
being referred to here are defined in the `service_credentials` section of
|
||||
`searchlight.conf` file. If any of the Swift plugins are enabled, this
|
||||
role must be added prior to running `searchlight-manage index sync`.
|
||||
|
||||
::
|
||||
|
||||
openstack role add --user searchlight --project service ResellerAdmin
|
||||
|
||||
|
||||
.. _proxy-server.conf:
|
||||
|
||||
proxy-server.conf
|
||||
-----------------
|
||||
|
||||
Incremental indexing for searchlight is typically provided via OSLO
|
||||
messaging. The Swift service currently doesn't send notifications, but
|
||||
work has been started to investigate more performant ways to achieve
|
||||
indexing. In the meantime, experimental support for providing notifications
|
||||
via middleware is provided in the following patch:
|
||||
|
||||
* https://review.opendev.org/#/c/249471
|
||||
|
||||
#. Apply the patch to the Swift proxy server
|
||||
#. Make the below configuration changes to `proxy-server.conf`
|
||||
#. Restart the Swift proxy server
|
||||
|
||||
::
|
||||
|
||||
# Add the following new section
|
||||
[filter:oslomiddleware]
|
||||
paste.filter_factory = swift.common.middleware.oslo_notifications:filter_factory
|
||||
publisher_id = swift.localhost
|
||||
#Replace <user>,<password>,<rabbitip> and <rabbitport> for your environment values
|
||||
transport_url = rabbit://<user>:<password>@<rabbitip>:<rabbitport>/
|
||||
notification_driver = messaging
|
||||
notification_topics = notifications
|
||||
|
||||
# Add oslomiddleware to pipeline:main
|
||||
# see example below.
|
||||
[pipeline:main]
|
||||
pipeline = catch_errors gatekeeper ...<other>... oslomiddleware proxy-logging proxy-server
|
||||
|
||||
.. note::
|
||||
|
||||
Restart the swift proxy API service (s-proxy) after making changes.
|
||||
Starting in Newton, Searchlight can share the same notification topic as
|
||||
other services, because it uses a messaging pool.
|
||||
|
||||
Searchlight Configuration
|
||||
=========================
|
||||
|
||||
Searchlight resource configuration options are shown below with their
|
||||
configuration file and default values.
|
||||
|
||||
See :ref:`searchlight-plugins` for common options with their default values,
|
||||
general configuration information, and an example complete configuration.
|
||||
|
||||
.. note::
|
||||
|
||||
Unless you are changing to a non-default value, you do not need to
|
||||
specify any of the following configuration options. After enabling or
|
||||
disabling a plugin you do need to restart the searchlight services
|
||||
(`searchlight-api` and `searchlight-listener`).
|
||||
After enabling a Swift plugin, you will also need to run the sync job:
|
||||
`searchlight-manage index sync --type OS::Swift::Account`
|
||||
|
||||
searchlight.conf
|
||||
----------------
|
||||
|
||||
Plugin: OS::Swift::Account
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_swift_account]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
# Specify same value as in swift proxy-server.conf for reseller_prefix
|
||||
reseller_prefix = AUTH_
|
||||
|
||||
.. note::
|
||||
|
||||
`os_swift_account` is disabled by default. You need to explicitly
|
||||
set enabled = True as shown above.
|
||||
|
||||
Plugin: OS::Swift::Container
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_swift_container]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
.. note::
|
||||
|
||||
`os_swift_container` is disabled by default. You need to explicitly
|
||||
set enabled = True as shown above.
|
||||
|
||||
Plugin: OS::Swift::Object
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
::
|
||||
|
||||
[resource_plugin:os_swift_object]
|
||||
enabled = true
|
||||
resource_group_name = searchlight
|
||||
|
||||
.. note::
|
||||
|
||||
`os_swift_object` is disabled by default. You need to explicitly
|
||||
set enabled = True as shown above.
|
||||
|
||||
|
||||
local.conf (devstack)
|
||||
---------------------
|
||||
|
||||
At this time we recommend that you manually enable the Searchlight plugins
|
||||
and middleware for Swift after devstack has completed stacking. Please
|
||||
follow the instructions above.
|
||||
|
||||
Release Notes
|
||||
=============
|
||||
|
||||
0.2.0.0 (Mitaka)
|
||||
----------------
|
||||
|
||||
Notifications must be configured properly for searchlight to process
|
||||
incremental updates. Searchlight must use its own topic. Use the following::
|
||||
|
||||
notification_driver = messaging
|
||||
notification_topics = searchlight_indexer
|
||||
|
||||
Large scale swift cluster support is targeted at a future release, but
|
||||
we encourage trial deployments to help us address issues as soon as possible.
|
||||
|
||||
Swift did not generate notifications for account/container/object CRUD
|
||||
during the Mitaka release. This means that search results will not include
|
||||
incremental updates after the initial indexing. However, there is a patch
|
||||
available to enable notifications via oslo messaging for the Mitaka release.
|
||||
|
||||
* https://review.opendev.org/#/c/249471
|
||||
|
||||
For devstack, the easiest way to test is::
|
||||
|
||||
cd /opt/stack/swift
|
||||
git review -x 249471
|
||||
<restart swift api>
|
||||
|
||||
Searchlight developers/installers should apply the above patch in Swift when
|
||||
using Searchlight with the Swift Mitaka release. We are working with the
|
||||
Swift team to create a supported incremental indexing methodology for future
|
||||
releases.
|
||||
|
||||
Alternatively, you may set up a cron job to re-index swift
|
||||
account/container/objects periodically to get updated information. The
|
||||
recommendation is to use the notifications, because a full re-indexing will
|
||||
not be performant in large installations.
|
||||
::
|
||||
|
||||
searchlight-manage index sync --type OS::Swift::Account
|
||||
|
||||
The Searchlight Swift plugin resource types follow the hierarchy similar to
|
||||
Swift concepts
|
||||
::
|
||||
|
||||
OS::Swift:Account(Parent)
|
||||
-> OS:Swift::Container(Child)
|
||||
-> OS::Swift::Object(Grand Child)
|
||||
|
||||
which means indexing is initiated by specifying only the top parent
|
||||
(OS::Swift::Account) and that will in-turn index all the child
|
||||
plugins(Container and Object)
|
||||
|
||||
Searchlight is adding indexing isolation in the Newton release via a concept
|
||||
called resource group isolation. This will better support re-indexing
|
||||
scalability.
|
||||
|
||||
Additional properties can be similarly protected with the `admin_only_fields`
|
||||
under each plugin's configuration section. Glob-like patterns are supported.
|
||||
For instance::
|
||||
|
||||
[resource_plugin:os_swift_object]
|
||||
admin_only_fields=x-meta-admin*
|
@ -1,7 +0,0 @@
|
||||
CLI Reference
|
||||
=============
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
searchlight-status
|
@ -1,78 +0,0 @@
|
||||
==================
|
||||
searchlight-status
|
||||
==================
|
||||
|
||||
Synopsis
|
||||
========
|
||||
|
||||
::
|
||||
|
||||
searchlight-status <category> <command> [<args>]
|
||||
|
||||
Description
|
||||
===========
|
||||
|
||||
:program:`searchlight-status` is a tool that provides routines for checking the
|
||||
status of a Searchlight deployment.
|
||||
|
||||
Options
|
||||
=======
|
||||
|
||||
The standard pattern for executing a :program:`searchlight-status` command is::
|
||||
|
||||
searchlight-status <category> <command> [<args>]
|
||||
|
||||
Run without arguments to see a list of available command categories::
|
||||
|
||||
searchlight-status
|
||||
|
||||
Categories are:
|
||||
|
||||
* ``upgrade``
|
||||
|
||||
Detailed descriptions are below.
|
||||
|
||||
You can also run with a category argument such as ``upgrade`` to see a list of
|
||||
all commands in that category::
|
||||
|
||||
searchlight-status upgrade
|
||||
|
||||
These sections describe the available categories and arguments for
|
||||
:program:`searchlight-status`.
|
||||
|
||||
Upgrade
|
||||
~~~~~~~
|
||||
|
||||
.. _searchlight-status-checks:
|
||||
|
||||
``searchlight-status upgrade check``
|
||||
Performs a release-specific readiness check before restarting services with
|
||||
new code. This command expects to have complete configuration and access
|
||||
to databases and services.
|
||||
|
||||
**Return Codes**
|
||||
|
||||
.. list-table::
|
||||
:widths: 20 80
|
||||
:header-rows: 1
|
||||
|
||||
* - Return code
|
||||
- Description
|
||||
* - 0
|
||||
- All upgrade readiness checks passed successfully and there is nothing
|
||||
to do.
|
||||
* - 1
|
||||
- At least one check encountered an issue and requires further
|
||||
investigation. This is considered a warning but the upgrade may be OK.
|
||||
* - 2
|
||||
- There was an upgrade status check failure that needs to be
|
||||
investigated. This should be considered something that stops an
|
||||
upgrade.
|
||||
* - 255
|
||||
- An unexpected error occurred.
|
||||
|
||||
**History of Checks**
|
||||
|
||||
**5.0.0 (Stein)**
|
||||
|
||||
* Placeholder to be filled in with checks as they are added in Stein.
|
@ -1,253 +0,0 @@
|
||||
# Copyright (c) 2010 OpenStack Foundation.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
# implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
#
|
||||
# Searchlight documentation build configuration file, created by
|
||||
# sphinx-quickstart on Tue May 26 13:50:15 2015.
|
||||
#
|
||||
# This file is execfile()'d with the current directory set to its containing
|
||||
# dir.
|
||||
#
|
||||
# Note that not all possible configuration values are present in this
|
||||
# autogenerated file.
|
||||
#
|
||||
# All configuration values have a default; values that are commented out
|
||||
# serve to show the default.
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
sys.path = [
|
||||
os.path.abspath('../..'),
|
||||
os.path.abspath('../../bin')
|
||||
] + sys.path
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
|
||||
extensions = ['sphinx.ext.coverage',
|
||||
'sphinx.ext.ifconfig',
|
||||
'sphinx.ext.graphviz',
|
||||
'openstackdocstheme',
|
||||
'oslo_policy.sphinxext',
|
||||
'oslo_policy.sphinxpolicygen',
|
||||
'sphinxcontrib.rsvgconverter',
|
||||
]
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
# templates_path = []
|
||||
|
||||
# The suffix of source filenames.
|
||||
source_suffix = '.rst'
|
||||
|
||||
# The encoding of source files.
|
||||
#source_encoding = 'utf-8'
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = 'index'
|
||||
|
||||
# General information about the project.
|
||||
copyright = u'2010-2015, OpenStack Foundation.'
|
||||
|
||||
policy_generator_config_file = \
|
||||
'../../etc/oslo-policy-generator/searchlight.conf'
|
||||
sample_policy_basename = '_static/searchlight'
|
||||
|
||||
# openstackdocstheme options
|
||||
openstackdocs_repo_name = 'openstack/searchlight'
|
||||
openstackdocs_pdf_link = True
|
||||
openstackdocs_bug_project = 'searchlight'
|
||||
openstackdocs_bug_tag = ''
|
||||
|
||||
# The short X.Y version.
|
||||
from searchlight.version import version_info as searchlight_version
|
||||
# The full version, including alpha/beta/rc tags.
|
||||
release = searchlight_version.version_string_with_vcs()
|
||||
# The short X.Y version.
|
||||
version = searchlight_version.canonical_version_string()
|
||||
|
||||
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||
# for a list of supported languages.
|
||||
#language = None
|
||||
|
||||
# There are two options for replacing |today|: either, you set today to some
|
||||
# non-false value, then it is used:
|
||||
#today = ''
|
||||
# Else, today_fmt is used as the format for a strftime call.
|
||||
#today_fmt = '%B %d, %Y'
|
||||
|
||||
# List of documents that shouldn't be included in the build.
|
||||
#unused_docs = []
|
||||
|
||||
# List of directories, relative to source directory, that shouldn't be searched
|
||||
# for source files.
|
||||
exclude_trees = ['api']
|
||||
|
||||
# The reST default role (for this markup: `text`) to use for all documents.
|
||||
#default_role = None
|
||||
|
||||
# If true, '()' will be appended to :func: etc. cross-reference text.
|
||||
#add_function_parentheses = True
|
||||
|
||||
# If true, the current module name will be prepended to all description
|
||||
# unit titles (such as .. function::).
|
||||
#add_module_names = True
|
||||
|
||||
# If true, sectionauthor and moduleauthor directives will be shown in the
|
||||
# output. They are ignored by default.
|
||||
show_authors = True
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = 'native'
|
||||
|
||||
# A list of ignored prefixes for module index sorting.
|
||||
modindex_common_prefix = ['searchlight.']
|
||||
|
||||
# -- Options for man page output --------------------------------------------
|
||||
|
||||
# Grouping the document tree for man pages.
|
||||
# List of tuples 'sourcefile', 'target', u'title', u'Authors name', 'manual'
|
||||
|
||||
#man_pages = [
|
||||
# ('man/searchlightapi', 'searchlight-api', u'Searchlight API Server',
|
||||
# [u'OpenStack'], 1),
|
||||
# ('man/searchlightlistener', 'searchlight-listener', u'Searchlight indexing agent',
|
||||
# [u'OpenStack'], 1),
|
||||
#]
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. Major themes that come with
|
||||
# Sphinx are currently 'default' and 'sphinxdoc'.
|
||||
# html_theme_path = ["."]
|
||||
html_theme = 'openstackdocs'
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
#html_theme_options = {}
|
||||
|
||||
# Add any paths that contain custom themes here, relative to this directory.
|
||||
#html_theme_path = []
|
||||
|
||||
# The name for this set of Sphinx documents. If None, it defaults to
|
||||
# "<project> v<release> documentation".
|
||||
#html_title = None
|
||||
|
||||
# A shorter title for the navigation bar. Default is the same as html_title.
|
||||
#html_short_title = None
|
||||
|
||||
# The name of an image file (relative to this directory) to place at the top
|
||||
# of the sidebar.
|
||||
#html_logo = None
|
||||
|
||||
# The name of an image file (within the static path) to use as favicon of the
|
||||
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
|
||||
# pixels large.
|
||||
#html_favicon = None
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
#html_static_path = ['_static']
|
||||
|
||||
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
|
||||
# using the given strftime format.
|
||||
html_last_updated_fmt = '%Y-%m-%d %H:%M'
|
||||
|
||||
# If true, SmartyPants will be used to convert quotes and dashes to
|
||||
# typographically correct entities.
|
||||
#html_use_smartypants = True
|
||||
|
||||
# Custom sidebar templates, maps document names to template names.
|
||||
#html_sidebars = {}
|
||||
|
||||
# Additional templates that should be rendered to pages, maps page names to
|
||||
# template names.
|
||||
#html_additional_pages = {}
|
||||
|
||||
# If false, no module index is generated.
|
||||
html_use_modindex = False
|
||||
|
||||
# If false, no index is generated.
|
||||
html_use_index = False
|
||||
|
||||
# If true, the index is split into individual pages for each letter.
|
||||
#html_split_index = False
|
||||
|
||||
# If true, links to the reST sources are added to the pages.
|
||||
#html_show_sourcelink = True
|
||||
|
||||
# If true, an OpenSearch description file will be output, and all pages will
|
||||
# contain a <link> tag referring to it. The value of this option must be the
|
||||
# base URL from which the finished HTML is served.
|
||||
#html_use_opensearch = ''
|
||||
|
||||
# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
|
||||
#html_file_suffix = ''
|
||||
|
||||
# Output file base name for HTML help builder.
|
||||
htmlhelp_basename = 'searchlightdoc'
|
||||
|
||||
|
||||
# -- Options for LaTeX output ------------------------------------------------
|
||||
|
||||
# The paper size ('letter' or 'a4').
|
||||
#latex_paper_size = 'letter'
|
||||
|
||||
# The font size ('10pt', '11pt' or '12pt').
|
||||
#latex_font_size = '10pt'
|
||||
|
||||
latex_use_xindy = False
|
||||
|
||||
latex_domain_indices = False
|
||||
|
||||
latex_elements = {
|
||||
'makeindex': '',
|
||||
'printindex': '',
|
||||
'preamble': r'\setcounter{tocdepth}{3}',
|
||||
}
|
||||
|
||||
# Grouping the document tree into LaTeX files. List of tuples
|
||||
# (source start file, target name, title, author,
|
||||
# documentclass [howto/manual]).
|
||||
latex_documents = [
|
||||
('index',
|
||||
'doc-Searchlight.tex',
|
||||
u'Searchlight Documentation',
|
||||
u'Searchlight Team', 'manual', True),
|
||||
]
|
||||
|
||||
# The name of an image file (relative to this directory) to place at the top of
|
||||
# the title page.
|
||||
#latex_logo = None
|
||||
|
||||
# For "manual" documents, if this is true, then toplevel headings are parts,
|
||||
# not chapters.
|
||||
#latex_use_parts = False
|
||||
|
||||
# Additional stuff for the LaTeX preamble.
|
||||
#latex_preamble = ''
|
||||
|
||||
# Documents to append as an appendix to all manuals.
|
||||
#latex_appendices = []
|
||||
|
||||
# If false, no module index is generated.
|
||||
#latex_use_modindex = True
|
@ -1,250 +0,0 @@
|
||||
..
|
||||
Copyright 2010 OpenStack Foundation
|
||||
All Rights Reserved.
|
||||
c) Copyright 2015 Hewlett-Packard Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
Keystone Authentication
|
||||
=======================
|
||||
|
||||
Searchlight should be integrated with keystone. Setting this up is
|
||||
relatively straightforward, as the keystone distribution includes the
|
||||
necessary middleware. Once you have installed keystone and edited your
|
||||
configuration files, users will need to have an authenticated keystone token
|
||||
in all API requests. The keystone integration will allow both active denial
|
||||
of requests from unauthenticated users and will also allow proper search
|
||||
result filtering.
|
||||
|
||||
.. DANGER::
|
||||
If the API is not configured with keystone, all data indexed by
|
||||
searchlight is at risk of being accessed by unauthorized users.
|
||||
|
||||
|
||||
Configuring the searchlight services to use keystone
|
||||
----------------------------------------------------
|
||||
|
||||
Keystone is integrated with searchlight through the use of middleware.
|
||||
The default configuration files for the Searchlight API use a single piece of
|
||||
middleware called ``unauthenticated-context``, which generates a request
|
||||
context containing blank authentication information. In order to configure
|
||||
Searchlight to use Keystone, the ``authtoken`` and ``context`` middleware
|
||||
must be deployed in place of the ``unauthenticated-context`` middleware.
|
||||
The ``authtoken`` middleware performs the authentication token validation
|
||||
and retrieves actual user authentication information. It can be found in
|
||||
the keystone distribution. For more information, please refer to the Keystone
|
||||
documentation on the ``auth_token`` middleware:
|
||||
https://docs.openstack.org/keystonemiddleware/latest/middlewarearchitecture.html
|
||||
|
||||
api-paste.ini
|
||||
`````````````
|
||||
|
||||
First, ensure that declarations for the middleware exist in the
|
||||
``api-paste.ini`` file. Here is an example for ``authtoken``::
|
||||
|
||||
[pipeline:searchlight-keystone]
|
||||
pipeline = authtoken context rootapp
|
||||
|
||||
[filter:authtoken]
|
||||
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
|
||||
delay_auth_decision = true
|
||||
|
||||
searchlight.conf
|
||||
````````````````
|
||||
|
||||
You must then update the main ``searchlight.conf`` configuration file
|
||||
to enable the keystone application pipeline.
|
||||
|
||||
Set ``flavor`` to ``keystone`` in the ``paste_deploy`` group::
|
||||
|
||||
[paste_deploy]
|
||||
flavor = keystone
|
||||
|
||||
Set ``keystone_authtoken`` options. The following sets the searchlight
|
||||
service user as the user for performing policy API authentication checks.
|
||||
The actual options and values in this section will need to be set according
|
||||
to your environment::
|
||||
|
||||
[keystone_authtoken]
|
||||
auth_url = http://127.0.0.1:5000
|
||||
auth_type = password
|
||||
project_domain_id = default
|
||||
project_name = service
|
||||
user_domain_id = default
|
||||
password = <SERVICE_PASSWORD>
|
||||
username = searchlight
|
||||
|
||||
.. note::
|
||||
For development and unit testing, it is recommended to also set
|
||||
``revocation_cache_timeout = 10`` under the ``keystone_authtoken`` group.
|
||||
|
||||
Set ``service_credentials`` options. Searchlight plugins may make API calls
|
||||
to other services to index their data. Prior to doing this, it will get a
|
||||
valid token based on the integration account credentials::
|
||||
|
||||
[service_credentials]
|
||||
# These are needed to make API calls to other services when indexing
|
||||
auth_type = password
|
||||
username = searchlight
|
||||
password = <SERVICE_PASSWORD>
|
||||
user_domain_id = default
|
||||
project_domain_id = default
|
||||
project_name = service
|
||||
auth_url = http://127.0.0.1:5000
|
||||
|
||||
# If resource_plugin.include_region_name is set, this value will be
|
||||
# the default value for the 'region_name' field on all documents
|
||||
# os_region_name =
|
||||
|
||||
For keystone v2 development::
|
||||
|
||||
[service_credentials]
|
||||
auth_type = v2password
|
||||
username = searchlight
|
||||
tenant_name = service
|
||||
password = <SERVICE_PASSWORD>
|
||||
auth_url = http://127.0.0.1:35357/v2.0
|
||||
|
||||
# If resource_plugin.include_region_name is set, this value will be
|
||||
# the default value for the 'region_name' field on all documents
|
||||
# os_region_name =
|
||||
|
||||
|
||||
Service integration account
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Some of the above configuration implicitly uses a ``searchlight`` service user.
|
||||
If you intend to use this user, it must have been created and registered with
|
||||
keystone. Typically, this is done with the following commands (v3 keystone)::
|
||||
|
||||
$ openstack project create --or-show service --property domain=default
|
||||
$ openstack user create searchlight --password <SERVICE_PASSWORD> --project service
|
||||
$ openstack role add admin --project service --user searchlight
|
||||
|
||||
For more information on keystone service accounts, see:
|
||||
|
||||
https://docs.openstack.org/keystone/latest/admin/manage-services.html#create-service-users
|
||||
|
||||
Policy restriction
|
||||
==================
|
||||
|
||||
Searchlight uses the oslo policy library to allow control over the level of
|
||||
access a user has based on their authenticated roles. Policy rules are defined
|
||||
in a configuration file (by default, `etc/policy.json`). By default, all
|
||||
operations are allowed.
|
||||
|
||||
https://docs.openstack.org/oslo.policy/latest/reference/index.html
|
||||
rule formatting.
|
||||
|
||||
During the last few cycles concerns were raised about the scope of the
|
||||
``admin`` role within OpenStack. Many services consider any token scoped with
|
||||
the ``admin`` role to have access to resources within any project. With the
|
||||
introduction of keystone v3 it is possible to create users with the admin role
|
||||
on a particular project, but not with the intention of them seeing resources in
|
||||
other projects.
|
||||
|
||||
Keystone added two configuration options called ``admin_project_name`` and
|
||||
``admin_project_domain_name`` to attempt to address this. If a request is
|
||||
authenticated against a the project whose name is ``admin_project_name``
|
||||
in the ``admin_project_domain_name`` domain, a flag is set on the
|
||||
authentication response headers indicating that the user is authenticated
|
||||
against the administrative project. This can then be supported by the policy
|
||||
rule (in Searchlight's ``policy.json``)::
|
||||
|
||||
"is_admin_context": "role:admin and is_admin_project:True"
|
||||
|
||||
Since devstack configures keystone to support those options, this is the
|
||||
default in Searchlight. To maintain backwards compatibility, if your keystone
|
||||
is *not* configured to set these options, any token with the ``admin`` role
|
||||
will be assumed to have administrative powers (this approach has been taken
|
||||
by other OpenStack services).
|
||||
|
||||
For more history see https://bugs.launchpad.net/keystone/+bug/968696.
|
||||
|
||||
Access to operations
|
||||
--------------------
|
||||
|
||||
It is possible to restrict access to functionality by setting rules for
|
||||
``query``, ``facets`` or ``plugins_info``. For instance, to restrict facet
|
||||
listing to administrators and disable plugin information for all users::
|
||||
|
||||
"facets": "role:admin",
|
||||
"plugins_info": "!"
|
||||
|
||||
Where a request is disallowed on this basis, the user will receive a
|
||||
403 Forbidden response.
|
||||
|
||||
Note that policy rules are applied on the fly; no server restart is required.
|
||||
Policy rules denying access to operations take precedence over the per-resource
|
||||
access described below.
|
||||
|
||||
Access to resources
|
||||
-------------------
|
||||
|
||||
It is possible to disable access to individual plugins. For instance, the
|
||||
following restricts access to Nova servers to admins, and disables access
|
||||
entirely to Glance images::
|
||||
|
||||
"resource:OS::Nova::Server": "role:admin",
|
||||
"resource:OS::Glance::Image": "!",
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
At current plugins still apply RBAC separately from policy rules. We
|
||||
aim to bring the two closer together in a later patch.
|
||||
|
||||
When resources are restricted in this way resources will be excluded
|
||||
from the search (which may result in empty search results). No Forbidden
|
||||
response will be returned.
|
||||
|
||||
.. _service-policy-controls:
|
||||
|
||||
Service policy controls
|
||||
-----------------------
|
||||
|
||||
If configured, Searchlight can consult service policy files (e.g. that used
|
||||
to configure the nova API). Each resource is configured with a policy target
|
||||
it will check if possible. Policy file paths can either be absolute or relative
|
||||
to `service_policy_path` (which itself can be relative to the current working
|
||||
directory or left blank). The actual filepath used will be determined by
|
||||
oslo.config using the same `logic`_ as for other config files (for logging,
|
||||
searchlight's policy file etc). With the following configuration
|
||||
stanza::
|
||||
|
||||
[service_policies]
|
||||
service_policy_files=compute:nova-policy.json
|
||||
service_policy_path=/etc/searchlight/
|
||||
|
||||
And with the following contents in nova-policy.json (which might be a symlink
|
||||
to an existing nova policy file, a copy or a separate file)::
|
||||
|
||||
{
|
||||
"is_admin": "role: admin",
|
||||
"os_compute_api:servers:index": "rule:is_admin"
|
||||
}
|
||||
|
||||
Only requests with the admin role assigned will be allowed to search or facet
|
||||
Nova servers.
|
||||
|
||||
Policy files are configured per *service*, not per resource type. If files
|
||||
are in different directories absolute paths should be used, and
|
||||
``service_policy_path`` left unset.
|
||||
|
||||
.. note::
|
||||
|
||||
Policy rules are always *more* restrictive. If a rule in Searchlight's
|
||||
``policy.json`` would allow access but a service policy file would disallow
|
||||
it (or vice versa), the more restrictive rule will be used.
|
||||
|
||||
.. _logic: https://docs.openstack.org/oslo.config/latest/reference/configopts.html
|
@ -1,11 +0,0 @@
|
||||
=====================
|
||||
Configuration guide
|
||||
=====================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
authentication
|
||||
plugins
|
||||
policy
|
||||
sample-policy-yaml
|
@ -1,384 +0,0 @@
|
||||
..
|
||||
Copyright 2015 Hewlett-Packard Development Company, L.P.
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
.. _searchlight-plugins:
|
||||
|
||||
Searchlight Plugin Documentation
|
||||
================================
|
||||
|
||||
The search service determines the types of information that is searchable
|
||||
via a plugin mechanism.
|
||||
|
||||
.. _installing-plugins:
|
||||
|
||||
Installing Plugins
|
||||
------------------
|
||||
|
||||
Plugins must be registered in ``setup.cfg``.
|
||||
|
||||
Within ``setup.cfg`` the setting within ``[entry_points]`` named
|
||||
``searchlight.index_backend`` should list the plugin for each available
|
||||
indexable type. After making a change, it's necessary to re-install the
|
||||
python package (for instance with ``pip install -e .``).
|
||||
|
||||
Each plugin registered in ``setup.cfg`` is enabled by default. Typically it
|
||||
should only be necessary to modify ``setup.cfg`` if you are installing a new
|
||||
plugin. It is not necessary to modify ``[entry_points]`` to temporarily
|
||||
enable or disable installed plugins. Once they are installed, they can be
|
||||
disabled, enabled and configured in the ``searchlight.conf`` file.
|
||||
|
||||
Configuring Plugins
|
||||
-------------------
|
||||
|
||||
After installation, plugins are configured in ``searchlight.conf``.
|
||||
|
||||
.. note::
|
||||
|
||||
After making changes to ``searchlight.conf`` you must perform the
|
||||
actions indicated in the tables below.
|
||||
|
||||
1. ``Restart services``: Restart all running ``searchlight-api`` *and*
|
||||
``searchlight-listener`` processes.
|
||||
|
||||
2. ``Re-index affected types``: You will need to re-index any resource
|
||||
types affected by the change. (See :doc:`../admin/indexingservice`).
|
||||
|
||||
.. note::
|
||||
|
||||
Unless you are changing to a non-default value, you do not need to
|
||||
specify any of the following configuration options.
|
||||
|
||||
.. _end-to-end-plugin-configuration-example:
|
||||
|
||||
End to End Configuration Example
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The following shows a sampling of various configuration options in
|
||||
``searchlight.conf``. These are **NOT** necessarily recommended
|
||||
or default configuration values. They are intended for exemplary purposes only.
|
||||
Please read the rest of the guide for detailed information.::
|
||||
|
||||
[listener]
|
||||
notifications_pool = searchlight
|
||||
|
||||
[resource_plugin]
|
||||
resource_group_name = searchlight
|
||||
# include_region_name = True
|
||||
|
||||
[service_credentials:nova]
|
||||
compute_api_version = 2.1
|
||||
|
||||
[resource_plugin:os_nova_server]
|
||||
enabled = True
|
||||
admin_only_fields = OS-EXT-SRV*,OS-EXT-STS:vm_state
|
||||
|
||||
[resource_plugin:os_nova_hypervisor]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_nova_flavor]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_nova_servergroup]
|
||||
enabled = False
|
||||
|
||||
[resource_plugin:os_glance_image]
|
||||
enabled = True
|
||||
# override_region_name = Region1,Region2
|
||||
|
||||
[resource_plugin:os_glance_metadef]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_cinder_volume]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_cinder_snapshot]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_net]
|
||||
enabled = True
|
||||
admin_only_fields=admin_state_up,status
|
||||
|
||||
[resource_plugin:os_neutron_port]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_neutron_security_group]
|
||||
enabled = True
|
||||
|
||||
[resource_plugin:os_designate_zone]
|
||||
enabled = False
|
||||
|
||||
[resource_plugin:os_designate_recordset]
|
||||
enabled = False
|
||||
|
||||
[resource_plugin:os_swift_account]
|
||||
enabled = False
|
||||
|
||||
[resource_plugin:os_swift_container]
|
||||
enabled = False
|
||||
|
||||
[resource_plugin:os_swift_object]
|
||||
enabled = False
|
||||
|
||||
Common Plugin Configuration Options
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
There are common configuration options that all plugins honor. They are split
|
||||
between *global*, *inheritable* and *non-inheritable* options.
|
||||
|
||||
**Global** plugin configuration options apply to all plugins and cannot be
|
||||
overridden by an individual plugin.
|
||||
|
||||
**Inheritable** common configuration options may be specified in a default
|
||||
configuration group of ``[resource_plugin]`` in ``searchlight.conf`` and
|
||||
optionally overridden in a specific plugin's configuration. For example::
|
||||
|
||||
[resource_plugin]
|
||||
resource_group_name = searchlight
|
||||
|
||||
[resource_plugin:os_nova_server]
|
||||
resource_group_name = searchlight-nova-servers
|
||||
|
||||
**Non-Inheritable** common configuration options are honored by all plugins,
|
||||
but must be specified directly in that plugin's configuration group. They
|
||||
are not inherited from the ``[resource_plugin]`` configuration group. For
|
||||
example::
|
||||
|
||||
[resource_plugin:os_glance_image]
|
||||
enabled = false
|
||||
|
||||
.. _plugin_notifications:
|
||||
|
||||
Notifications
|
||||
.............
|
||||
|
||||
There are two ways to configure services to send notifications that
|
||||
Searchlight can receive. The recommended method is to configure
|
||||
Searchlight to use the notification topic that each service is already
|
||||
configured to use and then to allow Searchlight to consume messages from
|
||||
that topic using a pool, touched on in the `messaging documentation`_.
|
||||
Searchlight uses this configuration by default.
|
||||
|
||||
.. _`messaging documentation`: https://docs.openstack.org/oslo.messaging/latest/configuration/opts.html
|
||||
|
||||
**Topics**
|
||||
|
||||
Searchlight defaults to using the oslo notification topic of
|
||||
``notifications``. This is the oslo default topic which most services also
|
||||
use to broadcast their notifications. You will need to change the topic in both
|
||||
``searchlight.conf`` and the various service configuration files if you want
|
||||
to modify the topic used by Searchlight. Each plugin can use a different topic.
|
||||
|
||||
Notification topics are a special case. It is possible to override
|
||||
the notification ``topic`` as a shared setting; it is also possible to
|
||||
override ``<topic>,<exchange>`` pairs per-plugin in the case where some
|
||||
services are using different topics. For instance, in a setup where (for
|
||||
example) neutron is using a separate notification topic::
|
||||
|
||||
[resource_plugin]
|
||||
notifications_topic = searchlight_indexer
|
||||
|
||||
[resource_plugin:os_nova_server]
|
||||
notifications_topics_exchanges = searchlight_indexer,nova
|
||||
notifications_topics_exchanges = another-topic,neutron
|
||||
|
||||
If you override one service topic, you must provide topic,exchange pairs
|
||||
for all service notifications a plugin supports.
|
||||
|
||||
**Pools**
|
||||
|
||||
In addition, Searchlight uses a notification pool. This allows Searchlight
|
||||
to listen on the same topic to which other services are listening while
|
||||
ensuring that Searchlight still gets its own copy of each notification. The
|
||||
default notification pool is set to ``searchlight``. This is set using the
|
||||
``notifications_pool`` setting in the ``[listener]`` configuration group.
|
||||
Example::
|
||||
|
||||
[listener]
|
||||
notification_pools = searchlight
|
||||
|
||||
See :ref:`individual-plugin-configuration` for more information and examples
|
||||
on individual plugin configuration.
|
||||
|
||||
Publishers
|
||||
..........
|
||||
|
||||
Searchlight supports configuration of publishers which push enriched
|
||||
notification info to other external systems. To use this feature, you must first
|
||||
register publishers in ``setup.cfg``. This is similar to
|
||||
:ref:`installing-plugins` action. Within ``setup.cfg`` the setting within
|
||||
``[entry_points]`` named ``searchlight.publishers`` should list publishers for
|
||||
plugins to use. Don't forget to re-install the python package(`pip install -e`)
|
||||
if you have made changes to publisher entry points.
|
||||
Example::
|
||||
|
||||
[entry_points]
|
||||
searchlight.publisher =
|
||||
log_publisher = searchlight.publisher.log.LogPublisher
|
||||
|
||||
Publishers can be specified in a default configuration group of `[resource_plugin]`
|
||||
in `searchlight.conf` or overridden in a specific plugin's configuration.
|
||||
Example::
|
||||
|
||||
[resource_plugin]
|
||||
publishers = log_publisher
|
||||
|
||||
Currently publishers only work for incremental updates. Bulk api updates are not
|
||||
supported.
|
||||
|
||||
Global Configuration Options
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
| Option | Default value | Description | Action(s) Required |
|
||||
+=====================+===============+=====================================+===========================+
|
||||
| resource_group_name | searchlight | Determines the ElasticSearch index | |
|
||||
| | | and alias where documents will be | | Restart services |
|
||||
| | | stored. Index names will be | | Re-index all types |
|
||||
| | | suffixed with a timestamp. Group | |
|
||||
| | | name must consist of only lowercase | |
|
||||
| | | alphanumeric characters and | |
|
||||
| | | underscores. The first character | |
|
||||
| | | cannot be an underscore. | |
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
| include_region_name | | Defined for all plugins. Controls | | Restart services |
|
||||
| | | whether or not to include | | Reindex all |
|
||||
| | | region_name as a mapping field and | |
|
||||
| | | in each document. Defaults to off. | |
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
|
||||
.. note::
|
||||
|
||||
Sorting on fields across resource types(plugins), with each plugin specifying a different resource group
|
||||
name will cause errors if sort-by fields are not defined in each resource type.
|
||||
See :ref:`using-resource-groups` for more information on how to sort across different resource groups
|
||||
|
||||
Inheritable Common Configuration Options
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
| Option | Default value | Description | Action(s) Required |
|
||||
+=====================+===============+=====================================+===========================+
|
||||
| mapping_use\_ | | Use doc_values to store documents | |
|
||||
| doc_values | true | rather than fieldata. doc_values | | Full re-index |
|
||||
| | | has some advantages, particularly | |
|
||||
| | | around memory usage. | |
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
| notifications_topic | notifications | The oslo.messaging topic on which | Restart listener |
|
||||
| | | services send notifications. Each | |
|
||||
| | | plugin defines a list of exchanges | |
|
||||
| | | to which it will subscribe. | |
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
| publishers | | Plugin can have multiple publishers | Restart listener |
|
||||
| | | separated by commas. Each publisher | |
|
||||
| | | will receive enriched notifications | |
|
||||
| | | once plugin subscribed events come. | |
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
|
||||
Non-Inheritable Common Configuration Options
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
| Option | Default value | Description | Action(s) Required |
|
||||
+=====================+===============+=====================================+===========================+
|
||||
| enabled | true | An installed plugin may be enabled | | Restart services |
|
||||
| | | (true) or disabled (false). When | | Re-index affected types |
|
||||
| | | disabled, it will not be available | |
|
||||
| | | for bulk indexing, notification | |
|
||||
| | | listening, or searching. | |
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
| admin_only_fields | <none> | A comma separated list of fields | | Restart services |
|
||||
| | | (wildcards allowed) that are only | | Re-index affected types |
|
||||
| | | visible to administrators, and only | |
|
||||
| | | searchable by administrators. Non- | |
|
||||
| | | administrative users will not be | |
|
||||
| | | able to see or search on these | |
|
||||
| | | fields. | |
|
||||
| | | These fields are typically | |
|
||||
| | | specified for search performance, | |
|
||||
| | | search accuracy, or security | |
|
||||
| | | reasons. | |
|
||||
| | | or security reasons. | |
|
||||
| | | If a plugin has a hard-coded | |
|
||||
| | | mapping for a specific field, it | |
|
||||
| | | will take precedence over this | |
|
||||
| | | configuration option. | |
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
| notifications\_ | <none> | Override topic,exchange pairs (see | | Restart services |
|
||||
| topics_exchanges | | note above). Use when services | |
|
||||
| | | output notifications on dissimilar | |
|
||||
| | | topics. | |
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
| override_region_name| <none> | Specifies a region_name to be used | | Restart services |
|
||||
| | | instead of one configured in | |
|
||||
| | | service_credentials.os_region_name. | |
|
||||
| | | Useful for multi-region deployments | |
|
||||
| | | where a service is shared between | |
|
||||
| | | regions. E.g. RegionOne,RegionTwo | |
|
||||
+---------------------+---------------+-------------------------------------+---------------------------+
|
||||
|
||||
.. _individual-plugin-configuration:
|
||||
|
||||
Individual Plugin Configuration
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Individual plugins may also be configured in ``searchlight.conf``.
|
||||
|
||||
.. note::
|
||||
|
||||
Plugin configurations are typically named based on their resource type.
|
||||
The configuration name uses the following naming pattern:
|
||||
|
||||
* The resource type name changed to all lower case
|
||||
|
||||
* All ``::`` (colons) converted into ``_`` (underscores).
|
||||
|
||||
For example: OS::Glance::Image --> [resource_plugin:os_glance_image]
|
||||
|
||||
To override a default configuration option on a specific plugin, you must
|
||||
specify a configuration group for that plugin with the option(s) that you
|
||||
want to override. For example, if you wanted to **just** disable the Glance
|
||||
image plugin, you would add the following configuration group::
|
||||
|
||||
[resource_plugin:os_glance_image]
|
||||
enabled = false
|
||||
|
||||
Each plugin may have additional configuration options specific to it.
|
||||
Information about those configuration options will be found in documentation
|
||||
for that plugin.
|
||||
|
||||
Finally, each integrated service (Glance, Nova, etc) may require
|
||||
additional configuration settings. For example, typically, you will need
|
||||
to add the ``searchlight_indexer`` notification topic to each service's
|
||||
configuration in order for Searchlight to receive incremental updates from
|
||||
that service.
|
||||
|
||||
.. note:: In Newton, notification messaging pools will become the default
|
||||
recommended configuration, which does not require changing any
|
||||
service configurations beyond enabling notifications.
|
||||
|
||||
To enable the use of notification pools instead of a separate
|
||||
topic, add the ``notifications_pool`` option in the ``listener``
|
||||
section of ``searchlight.conf``. There is no need in this case
|
||||
to add an additional topic. Messages will begin to be delivered
|
||||
to the pool after ``searchlight-listener`` has started.
|
||||
|
||||
Please review each plugin's documentation for more information:
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:glob:
|
||||
|
||||
../admin/plugins/*
|
@ -1,12 +0,0 @@
|
||||
================================
|
||||
Searchlight Policy Configuration
|
||||
================================
|
||||
|
||||
Configuration
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
The following is an overview of all available policies in Searchlight.
|
||||
For a sample configuration file, refer to :doc:`sample-policy-yaml`.
|
||||
|
||||
.. show-policy::
|
||||
:config-file: ../../etc/oslo-policy-generator/searchlight.conf
|
@ -1,8 +0,0 @@
|
||||
===========
|
||||
policy.yaml
|
||||
===========
|
||||
|
||||
Use the ``policy.yaml`` file to define additional access controls that will be
|
||||
applied to Searchlight:
|
||||
|
||||
.. literalinclude:: ../_static/searchlight.policy.yaml.sample
|
@ -1,539 +0,0 @@
|
||||
..
|
||||
Copyright 2016 Hewlett-Packard Enterprise Development Company, L.P.
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
.. _searchlight-plugin-authoring:
|
||||
|
||||
Authoring Searchlight Plugins
|
||||
=============================
|
||||
|
||||
At a bare minimum, a plugin must consist of an elasticsearch mapping, and a
|
||||
method by which it can provide data to be indexed. Many plugins also require a
|
||||
way to receive updates in order to keep the index up to date. For Openstack
|
||||
resources, typically the service API is used for initial indexing and
|
||||
notifications are received via oslo.messaging.
|
||||
|
||||
This documentation will use as an example the Neutron network plugin as a
|
||||
reasonably complete and complex example.
|
||||
|
||||
Getting some data
|
||||
-----------------
|
||||
The very first thing you should do is figure out exactly what you're trying to
|
||||
index. When I've developed plugins I've found it helpful to generate test data
|
||||
both for initial indexing and for notifications.
|
||||
|
||||
Initial indexing
|
||||
^^^^^^^^^^^^^^^^
|
||||
In the case of neutron networks, the initial data will come from
|
||||
``neutronclient``. Some browsing of the API documentation reveals that the
|
||||
call I want is ``list_networks``::
|
||||
|
||||
import os
|
||||
|
||||
from oslo_serialization import jsonutils
|
||||
|
||||
from keystoneclient.auth.identity import v3
|
||||
from keystoneclient import session
|
||||
from neutronclient.v2_0 import client as nc_20
|
||||
|
||||
def get_session():
|
||||
username = os.environ['OS_USERNAME']
|
||||
password = os.environ['OS_PASSWORD']
|
||||
auth_url = os.environ['OS_AUTH_URL']
|
||||
tenant_name = os.environ['OS_TENANT_NAME']
|
||||
auth = v3.Password(**locals())
|
||||
return session.Session(auth=auth)
|
||||
|
||||
|
||||
nc = nc_20.Client(session=get_session())
|
||||
networks = nc.list_networks()
|
||||
|
||||
print(jsonutils.dumps(networks, indent=4, sort_keys=True))
|
||||
|
||||
This outputs::
|
||||
|
||||
{
|
||||
"networks": [
|
||||
{
|
||||
"admin_state_up": true,
|
||||
"availability_zone_hints": [],
|
||||
"availability_zones": [
|
||||
"nova"
|
||||
],
|
||||
"created_at": "2016-04-08T16:44:17",
|
||||
"description": "",
|
||||
"id": "4d73d257-35d5-4f4e-bc71-f7f629f21904",
|
||||
"ipv4_address_scope": null,
|
||||
"ipv6_address_scope": null,
|
||||
"is_default": true,
|
||||
"mtu": 1450,
|
||||
"name": "public",
|
||||
"port_security_enabled": true,
|
||||
"provider:network_type": "vxlan",
|
||||
"provider:physical_network": null,
|
||||
"provider:segmentation_id": 1053,
|
||||
"router:external": true,
|
||||
"shared": false,
|
||||
"status": "ACTIVE",
|
||||
"subnets": [
|
||||
"abcc5896-4844-4870-a5d8-6ae4b8edd42e",
|
||||
"ea47304e-bd54-4337-901a-1eb5196ea18e"
|
||||
],
|
||||
"tags": [],
|
||||
"tenant_id": "fa1537e9bda9405891d004ef9c08d0d1",
|
||||
"updated_at": "2016-04-08T16:44:17"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Since that's the output from neutron client, that's what should go in
|
||||
``searchlight/tests/functional/data/load/networks.json``, though you might
|
||||
also want more examples to test different things.
|
||||
|
||||
Notifications
|
||||
^^^^^^^^^^^^^
|
||||
Openstack documents some of the notifications_ sent by some services. It's
|
||||
also possible to eavesdrop on notifications sent by running services. Taking
|
||||
neutron as an example (though all services are slightly different), we can
|
||||
make it output notifications by editing ``/etc/neutron/neutron.conf`` and
|
||||
adding under the ``[oslo_messaging_notifications]`` section::
|
||||
|
||||
driver = messagingv2
|
||||
|
||||
There are then two ways to configure the service to send notifications that
|
||||
Searchlight can receive. The recommended method is to use notification pools,
|
||||
touched on in the `messaging documentation`_.
|
||||
|
||||
.. _`messaging documentation`:
|
||||
|
||||
https://docs.openstack.org/oslo.messaging/latest/reference/notification_listener.html
|
||||
|
||||
Notification pools
|
||||
##################
|
||||
|
||||
A notification messaging pool allows additional listeners to receive
|
||||
messages on an existing topic. By default, Openstack services send notification
|
||||
messages to an oslo.messaging 'topic' named `notifications`. To view these
|
||||
notifications while still allowing ``searchlight-listener`` or Ceilometer's
|
||||
agent to continue to receive them, you may use the utility script in
|
||||
``test-scripts/listener.py``::
|
||||
|
||||
. ~/devstack/openrc admin admin
|
||||
# If your rabbitmq user/pass are not the same as for devstack, you
|
||||
# can set RABBIT_PASSWORD and/or RABBIT_USER
|
||||
./test-scripts/listener.py neutron test-notifications
|
||||
|
||||
Adding a separate topic
|
||||
#######################
|
||||
|
||||
In the same config file (``/etc/neutron/neutron.conf``) the following line
|
||||
(again, under the ``[DEFAULT]`` section) will cause neutron to output
|
||||
notifications to a topic named ``searchlight_indexer``::
|
||||
|
||||
notification_topics = searchlight_indexer
|
||||
|
||||
.. note::
|
||||
|
||||
``searchlight-listener`` also listens on the ``searchlight_indexer``
|
||||
topic, so if you have ``searchlight-listener`` running, it will receive
|
||||
and process some or all of the notifications you're trying to look at.
|
||||
Thus, you should either stop the ``searchlight-listener`` or add another
|
||||
topic (comma-separated) for the specific notifications you want to see.
|
||||
For example::
|
||||
|
||||
notification_topics = searchlight_indexer,my_test_topic
|
||||
|
||||
After restarting the ``q-svc`` service notifications will be output to the
|
||||
message bus (rabbitmq by default). They can be viewed in any RMQ management
|
||||
tool; there is also a utility script in ``test-scripts/listener.py`` that
|
||||
will listen for notifications::
|
||||
|
||||
. ~/devstack/openrc admin admin
|
||||
# If your rabbitmq user/pass are not the same as for devstack, you
|
||||
# can set RABBIT_PASSWORD and/or RABBIT_USER
|
||||
./test-scripts/listener.py neutron
|
||||
|
||||
.. note::
|
||||
|
||||
If you added a custom topic as described above, you'll need to edit
|
||||
``listener.py`` to use your custom topic::
|
||||
|
||||
# Change this line
|
||||
topic = 'searchlight_indexer'
|
||||
# to
|
||||
topic = 'my_test_topic'
|
||||
|
||||
Using the results
|
||||
#################
|
||||
|
||||
Issuing various commands (``neutron net-create``, ``neutron net-update``,
|
||||
``neutron net-delete``) will cause ``listener.py`` to receive notifications.
|
||||
Usually the notifications with ``event_type`` ending ``.end`` are the ones of
|
||||
most interest (many fields omitted for brevity)::
|
||||
|
||||
{"event_type": "network.update.end",
|
||||
"payload": {
|
||||
"network": {
|
||||
"status": "ACTIVE",
|
||||
"router:external": false,
|
||||
"subnets": ["9b6094de-18cb-46e1-8d51-e303ff844c86",
|
||||
"face0b47-40d3-45c0-9b62-5f05311710f5",
|
||||
"7b7bdf5f-8f22-44a3-bec3-1daa78df83c5"],
|
||||
"updated_at": "2016-05-03T19:05:38",
|
||||
"tenant_id": "34518c16d95e40a19b1a95c1916d8335",
|
||||
"id": "abf3a939-4daf-4d05-8395-3ec735aa89fc", "name": "private"}
|
||||
},
|
||||
"publisher_id": "network.devstack",
|
||||
"ctxt": {
|
||||
"read_only": false,
|
||||
"domain": null,
|
||||
"project_name": "demo",
|
||||
"user_id": "c714917a458e428fa5dc9b1b8aa0d4d6"
|
||||
},
|
||||
"metadata": {
|
||||
"timestamp": "2016-05-03 19:05:38.258273",
|
||||
"message_id": "ec9ac6a1-aa17-4ee3-aa6e-ab48c1fb81a8"
|
||||
}
|
||||
}
|
||||
|
||||
The entire message can go into
|
||||
``searchlight/tests/functional/data/events/network.json``. The ``payload``
|
||||
(in addition to the API response) will inform the mapping that should be
|
||||
applied for a given plugin.
|
||||
|
||||
.. _notifications: https://wiki.openstack.org/wiki/SystemUsageData
|
||||
|
||||
File structure
|
||||
--------------
|
||||
Plugins live in ``searchlight/elasticsearch/plugins``. We have tended to create
|
||||
a subpackage named after the service (``neutron``) and within it a module named
|
||||
after the resource type (``networks.py``). Notification handlers can be in a file
|
||||
specific to each resource type but can also be in a single file together
|
||||
(existing ones use ``notification_handlers.py``).
|
||||
|
||||
``networks.py`` contains a class named ``NetworkIndex`` that implements the base
|
||||
class ``IndexBase`` found in ``searchlight.elasticsearch.plugins.base``.
|
||||
|
||||
.. note::
|
||||
|
||||
If there are plugins for multiple resources within the same Openstack
|
||||
service (for example, Glance images and meta definitions) those plugins
|
||||
can exist in the same subpackage ('glance') in different modules, each
|
||||
implementing an IndexBase.
|
||||
|
||||
Enabling plugins
|
||||
----------------
|
||||
Searchlight plugins are loaded by Stevedore_. In order for a plugin to be
|
||||
enabled for indexing and searching, it's necessary to add an entry to the
|
||||
``entry_points`` list in Searchlight's configuration in ``setup.cfg``. The
|
||||
name should be the plugin resource name (typically the name used to represent
|
||||
it in Heat_)::
|
||||
|
||||
[entry_points]
|
||||
searchlight.index_backend =
|
||||
os_neutron_net = searchlight.elasticsearch.plugins.neutron.networks:NetworkIndex
|
||||
|
||||
.. note::
|
||||
|
||||
After modifying entrypoints, you'll need to reinstall the searchlight
|
||||
package to register them (you may need to activate your virtual environment;
|
||||
see :ref:`Installation Instructions`)::
|
||||
|
||||
python setup.py develop
|
||||
|
||||
.. _Stevedore: https://docs.openstack.org/stevedore/latest/
|
||||
.. _Heat: https://docs.openstack.org/heat/latest/template_guide/openstack.html
|
||||
|
||||
Writing some code
|
||||
-----------------
|
||||
At this point you're probably about ready to start filling in the code. My
|
||||
usual approach is to create the unit test file first, and copy some of the
|
||||
more boilerplate functionality from one of the other plugins.
|
||||
|
||||
You can run an individual test file with::
|
||||
|
||||
tox -epy34 searchlight.tests.unit.<your test module>
|
||||
|
||||
This has the advantage of running just your tests and executing them very
|
||||
quickly. It can be easier to start from a full set of failing unit tests
|
||||
and build up the actual code from there. Functional tests I've tended to add
|
||||
later. Again, you can run an individual functional test file:
|
||||
|
||||
tox -epy34 searchlight.tests.functional.<your test module>
|
||||
|
||||
Required plugin functions
|
||||
-------------------------
|
||||
This section describes some of the functionality from ``IndexBase`` you will
|
||||
need to override.
|
||||
|
||||
Document type
|
||||
^^^^^^^^^^^^^
|
||||
As a convention, plugins define their document type (which will map to an
|
||||
ElasticSearch document type) as the `resource name`_ Heat uses to identify it::
|
||||
|
||||
@classmethod
|
||||
def get_document_type(self):
|
||||
return "OS::Neutron::Net"
|
||||
|
||||
.. _`resource name`: https://docs.openstack.org/heat/latest/template_guide/openstack.html
|
||||
|
||||
Retrieving object for initial indexing
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Plugins must implement ``get_objects`` which in many cases will go to the
|
||||
API of the service it's indexing. It should return an iterable that will be
|
||||
passed to a function (also required) named ``serialize``, which in turn must
|
||||
return a dictionary suitable for Elasticsearch to index. In the example for
|
||||
Neutron networks, this would be a call to ``list_networks`` on an instance of
|
||||
``neutronclient``::
|
||||
|
||||
def get_objects(self):
|
||||
"""Generator that lists all networks owned by all tenants."""
|
||||
# Neutronclient handles pagination itself; list_networks is a generator
|
||||
neutron_client = openstack_clients.get_neutronclient()
|
||||
for network in neutron_client.list_networks()['networks']:
|
||||
yield network
|
||||
|
||||
Mapping
|
||||
^^^^^^^
|
||||
|
||||
``get_mapping`` is also required. It must return a dictionary that tells
|
||||
Elasticsearch how to map documents for the plugin (see the documentation for
|
||||
mapping_).
|
||||
|
||||
At a minimum a plugin should define an ``id`` field and an ``updated_at`` field
|
||||
because consumers will generally rely on those being present; a ``name`` field
|
||||
is highly advisable. If the resource doesn"t contain these values your
|
||||
``serialize`` function can map to them. In particular, if your resource does
|
||||
not have a native ``id`` value, you must override ``get_document_id_field``
|
||||
so that the indexing code can retrieve the correct value when indexing.
|
||||
|
||||
It is worth understanding how Elasticsearch indexes various field types,
|
||||
particularly strings. String fields are typically broken down into tokens to
|
||||
allow searching::
|
||||
|
||||
"The quick brown fox" -> ["The", "quick", "brown", "fox"]
|
||||
|
||||
This works well for full-text type documents but less well, for example,
|
||||
for UUIDS::
|
||||
|
||||
"aaab-bbbb-55555555" -> ["aaab", "bbbb", "55555555"]
|
||||
|
||||
In the second example, a search for the full UUID will not match. As a result,
|
||||
we tend to mark these kinds of fields as ``not_analyzed`` as with the example
|
||||
to follow.
|
||||
|
||||
Where field types are not specified, Elasticsearch will make a best guess from
|
||||
the first document that's indexed.
|
||||
|
||||
Some notes (expressed below as comments starting with #)::
|
||||
|
||||
{
|
||||
# This allows indexing of fields not specified in the mapping doc
|
||||
"dynamic": true,
|
||||
"properties": {
|
||||
|
||||
# not_analyzed is important for id fields; it prevents Elasticsearch
|
||||
# tokenizing the field, allowing for exact matches
|
||||
"id": {"type": "string", "index": "not_analyzed"},
|
||||
|
||||
# This allows name to be tokenized for searching, but Searchlight will
|
||||
# attempt to use the 'raw' (untokenized) field for sorting which gives
|
||||
# more consistent results
|
||||
"name": {
|
||||
"type": "string",
|
||||
"fields": {
|
||||
"raw": {"type": "string", "index": "not_analyzed"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
If you are mapping a field which is a reference id to other plugin type, you
|
||||
should add a _meta mapping for that field. This will enable Searchlight(SL) to
|
||||
provide more information to CLI/UI. The reference id and the plugin resource
|
||||
type can be used by CLI/UI to issue a ``GET`` request to fetch more information
|
||||
from SL. See below for an example on nova server plugin mapping::
|
||||
|
||||
def get_mapping(self):
|
||||
return {
|
||||
'dynamic': True,
|
||||
'properties': {
|
||||
'id': {'type': 'string', 'index': 'not_analyzed'},
|
||||
'name': {
|
||||
'type': 'string',
|
||||
'fields': {
|
||||
'raw': {'type': 'string', 'index': 'not_analyzed'}
|
||||
}
|
||||
}
|
||||
'image': {
|
||||
'type': 'nested',
|
||||
'properties': {
|
||||
'id': {'type': 'string', 'index': 'not_analyzed'}
|
||||
}
|
||||
}
|
||||
},
|
||||
"_meta": {
|
||||
"image.id": {
|
||||
"resource_type": resource_types.GLANCE_IMAGE
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
.. note:: Parent plugin id field(when available) is automatically linked to the
|
||||
parent resource type.
|
||||
|
||||
Doc values
|
||||
^^^^^^^^^^
|
||||
|
||||
For many field types Searchlight will alter the mapping to change the format in
|
||||
which field data is stored. Prior to Elasticsearch 2.x field values by default
|
||||
were stored in 'fielddata' format, which could result in high memory usage under
|
||||
some sort and aggregation operations. An alternative format, called ``doc_values``
|
||||
trades slightly increased disk usage for better memory efficiency. In Elasticsearch
|
||||
2.x ``doc_values`` is the default, and Searchlight uses this option as the default
|
||||
regardless of Elasticsearch version. For more information see the Elasticsearch
|
||||
documentation_.
|
||||
|
||||
.. _documentation: https://www.elastic.co/guide/en/elasticsearch/reference/2.1/doc-values.html
|
||||
|
||||
Generally this default will be fine. However, there are several ways in which
|
||||
the default can be overridden:
|
||||
|
||||
* Globally in plugin configuration; in ``searchlight.conf``::
|
||||
|
||||
[resource_plugin]
|
||||
mapping_use_doc_values = false
|
||||
|
||||
* For an individual plugin in ``searchlight.conf``::
|
||||
|
||||
[resource_plugin:os_neutron_net]
|
||||
mapping_use_doc_values = false
|
||||
|
||||
* For a plugin's entire mapping; in code, override the ``mapping_use_doc_values``
|
||||
property (and thus ignoring any configuration property)::
|
||||
|
||||
@property
|
||||
def mapping_use_doc_values(self):
|
||||
return False
|
||||
|
||||
* For individual fields in a mapping, by setting ``doc_values`` to False::
|
||||
|
||||
{
|
||||
"properties": {
|
||||
"some_field": {"type": "date", "doc_values": False}
|
||||
}
|
||||
}
|
||||
|
||||
Access control
|
||||
^^^^^^^^^^^^^^
|
||||
Plugins must define how they are access controlled. Typically this is a
|
||||
restriction matching the user's project/tenant::
|
||||
|
||||
def _get_rbac_field_filters(self, request_context):
|
||||
return [
|
||||
{'term': {'tenant_id': request_context.owner}}
|
||||
]
|
||||
|
||||
Any filters listed will be applied to queries against the plugin's document
|
||||
type. A document will match the RBAC filters if any of the clauses match.
|
||||
Administrative users can specify ``all_projects`` in searches to bypass
|
||||
these filters. This default behavior can be overridden for a plugin by setting
|
||||
the ``allow_admin_ignore_rbac`` property to ``False`` on the plugin (currently
|
||||
only in code). ``all_projects`` will be ignore for that plugin.
|
||||
|
||||
Policy
|
||||
^^^^^^
|
||||
Related to access control is policy. Most services control API access with
|
||||
policy files that define rules enforced with `oslo.policy`_. Searchlight has
|
||||
its own policy file that configures access to its own API and resources, but
|
||||
it also supports reading other services' policy files. In the future this will
|
||||
be expanded to define RBAC rules, but at present external policy files are
|
||||
only used to determine whether a resource should be available to a user.
|
||||
|
||||
To support this in your plugin, you must define two properties. The first
|
||||
is ``service_type`` which must correspond to the service 'type' as seen
|
||||
in the keystone catalog (e.g. nova's service 'type' is 'compute'). The
|
||||
second property is ``resource_allowed_policy_target`` which identifies the
|
||||
rule name in the service's policy files. If either of these properties are
|
||||
'None' no rule will be enforced.
|
||||
|
||||
For example::
|
||||
|
||||
@property
|
||||
def resource_allowed_policy_target(self):
|
||||
return 'os_compute_api:servers:index'
|
||||
|
||||
@property
|
||||
def service_type(self):
|
||||
return 'compute'
|
||||
|
||||
See :ref:`service-policy-controls` for configuration information.
|
||||
|
||||
.. _oslo.policy: https://docs.openstack.org/oslo.policy/latest/
|
||||
|
||||
Faceting
|
||||
^^^^^^^^
|
||||
Any fields defined in the mapping document are eligible to be identified as
|
||||
facets, which allows a UI to let users search on specific fields. Many plugins
|
||||
define ``facets_excluded`` which exclude specified fields. Many also define
|
||||
``facets_with_options`` which should return fields with low cardinality where
|
||||
it makes sense to return valid options for those fields.
|
||||
|
||||
Protected fields
|
||||
^^^^^^^^^^^^^^^^
|
||||
``admin_only_fields`` determines fields which only administrators should be
|
||||
able to see or search. For instance, this will mark any fields beginning with
|
||||
``provider:`` as well as any defined in the plugin configuration::
|
||||
|
||||
@property
|
||||
def admin_only_fields(self):
|
||||
from_conf = super(NetworkIndex, self).admin_only_fields
|
||||
return ['provider:*'] + from_conf
|
||||
|
||||
These fields end up getting indexed in separate admin-only documents.
|
||||
|
||||
Parent/child relationships
|
||||
--------------------------
|
||||
In some cases there is a strong ownership implied between plugins. In these
|
||||
cases the child plugin can define ``parent_plugin_type`` and
|
||||
``get_parent_id_field`` (which determines a field on the child that refers
|
||||
to its parent). See the Neutron ``Port`` plugin for an example.
|
||||
|
||||
Remember that Elasticsearch is not a relational database and it doesn't do
|
||||
joins, per se, but this linkage does allow running queries referencing children
|
||||
(or parents).
|
||||
|
||||
.. _mapping: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html
|
||||
|
||||
Pipeline architecture
|
||||
---------------------
|
||||
Notification handlers can emit enriched resource data into pipeline, configured
|
||||
publishers could use these data to notify external systems. To use this feature,
|
||||
each event handler should return one or a sequence of pipeline items. These items
|
||||
will be passed to subscribed publshers::
|
||||
|
||||
def create_or_update(self, event_type, payload, timestamp):
|
||||
network_id = payload['network']['id']
|
||||
LOG.debug("Updating network information for %s", network_id)
|
||||
|
||||
network = serialize_network(payload['network'])
|
||||
version = self.get_version(network, timestamp)
|
||||
|
||||
self.index_helper.save_document(network, version=version)
|
||||
return pipeline.IndexItem(self.index_helper.plugin,
|
||||
event_type,
|
||||
payload,
|
@ -1,88 +0,0 @@
|
||||
============================
|
||||
So You Want to Contribute...
|
||||
============================
|
||||
|
||||
For general information on contributing to OpenStack, please check out the
|
||||
`contributor guide <https://docs.openstack.org/contributors/>`_ to get started.
|
||||
It covers all the basics that are common to all OpenStack projects: the accounts
|
||||
you need, the basics of interacting with our Gerrit review system, how we
|
||||
communicate as a community, etc.
|
||||
|
||||
Below will cover the more project specific information you need to get started
|
||||
with Searchlight.
|
||||
|
||||
Communication
|
||||
~~~~~~~~~~~~~~
|
||||
.. This would be a good place to put the channel you chat in as a project; when/
|
||||
where your meeting is, the tags you prepend to your ML threads, etc.
|
||||
|
||||
- IRC channel: #openstack-searchlight
|
||||
- Mailing list's prefix: [searchlight]
|
||||
- Meeting Times: http://eavesdrop.openstack.org/#Searchlight_Team_Meeting
|
||||
- Meeting Agenda: https://etherpad.openstack.org/p/search-team-meeting-agenda
|
||||
- Meeting Logs: http://eavesdrop.openstack.org/meetings/openstack_search
|
||||
|
||||
Contacting the Core Team
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
.. This section should list the core team, their irc nicks, emails, timezones etc. If
|
||||
all this info is maintained elsewhere (i.e. a wiki), you can link to that instead of
|
||||
enumerating everyone here.
|
||||
|
||||
- Trinh Nguyen <dangtrinhnt@gmail.com> (PTL) - dangtrinhnt - GMT+9
|
||||
- Thuy Dang <thuydang.de@gmail.com> - thuydang - GMT+1
|
||||
|
||||
New Feature Planning
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
.. This section is for talking about the process to get a new feature in. Some
|
||||
projects use blueprints, some want specs, some want both! Some projects
|
||||
stick to a strict schedule when selecting what new features will be reviewed
|
||||
for a release.
|
||||
|
||||
1. Talk to the team via IRC (meeting) or ML (with [searchlight] prefix) about
|
||||
the feature you want to add. We will discuss and figure out what is the best
|
||||
way forward considering our plan for the development cycle.
|
||||
2. After the team has discussed and everyone agreed to have your feature
|
||||
landed in the current cycle, you can propose a blueprint document describing your
|
||||
feature in details (e.g., architecture, components, usage, etc.)
|
||||
3. The team will review your blueprint and make comments until it is good enough
|
||||
to start implementing the feature.
|
||||
4. You implement the feature based on the design in the blueprint.
|
||||
5. Searchlight team will again review your code and approve it.
|
||||
|
||||
Task Tracking
|
||||
~~~~~~~~~~~~~~
|
||||
.. This section is about where you track tasks- launchpad? storyboard? is there more
|
||||
than one launchpad project? what's the name of the project group in storyboard?
|
||||
|
||||
We track our tasks in `Storyboard <https://storyboard.openstack.org/#!/project/openstack/searchlight>`_
|
||||
If you're looking for some smaller, easier work item to pick up and get started
|
||||
on, search for the 'low-hanging-fruit' tag.
|
||||
|
||||
.. NOTE: If your tag is not 'low-hanging-fruit' please change the text above.
|
||||
|
||||
Reporting a Bug
|
||||
~~~~~~~~~~~~~~~
|
||||
.. Pretty self explanatory section, link directly to where people should report bugs for
|
||||
your project.
|
||||
|
||||
You found an issue and want to make sure we are aware of it? You can do so
|
||||
`HERE <https://storyboard.openstack.org/#!/project/openstack/searchlight>`_.
|
||||
|
||||
Getting Your Patch Merged
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
.. This section should have info about what it takes to get something merged. Do
|
||||
you require one or two +2's before +W? Do some of your repos require unit test
|
||||
changes with all patches? etc.
|
||||
|
||||
Due to the small number of core reviewers of the Searchlight project, we only need
|
||||
one +2 before +W (merge). All patches excepting for documentation or typos fixes
|
||||
must have unit test.
|
||||
|
||||
|
||||
Project Team Lead Duties
|
||||
------------------------
|
||||
.. this section is where you can put PTL specific duties not already listed in
|
||||
the common PTL guide (linked below) or if you already have them written
|
||||
up elsewhere, you can link to that doc here.
|
||||
|
||||
All common PTL duties are enumerated here in the `PTL guide <https://docs.openstack.org/project-team-guide/ptl.html>`_.
|
@ -1,186 +0,0 @@
|
||||
..
|
||||
c) Copyright 2015 Hewlett-Packard Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
================================
|
||||
Feature Requests and Bug Reports
|
||||
================================
|
||||
|
||||
Searchlight is an open project and we encourage contribution from everybody.
|
||||
|
||||
We support both developers and non-developers who want to provide input,
|
||||
requests for features, and bug fixes. We want to be able to move quickly
|
||||
without getting too bogged down in process, but still provide a rich mechanism
|
||||
for feature reviews as needed.
|
||||
|
||||
|
||||
Workflow
|
||||
========
|
||||
|
||||
Our process is meant to allow users, developers, and operators to express
|
||||
their desires for new features using Storyboard blueprints. A review of
|
||||
blueprints is done regularly. These may turn directly into features, or
|
||||
for complex requests, additional specifications ("specs") may be needed.
|
||||
|
||||
The workflow is very simple:
|
||||
|
||||
* If something is clearly broken, submit a `bug report`_ in Storyboard.
|
||||
* If you want to change or add a feature, submit a `blueprint`_ in Storyboard.
|
||||
* Searchlight drivers may request that you submit a `specification`_ to gerrit to elaborate on the feature request
|
||||
* Significant features require `Release Notes`_ to be included when the code is merged
|
||||
|
||||
.. note::
|
||||
|
||||
If you already have code to submit, go ahead and submit a gerrit review!
|
||||
We encourage early code sharing. Just be aware that it needs to be cross
|
||||
linked with a bug or blueprint and may not be accepted without approval
|
||||
of the blueprint or bug. We also use the blueprint and bug priorities
|
||||
to guide our code review priorities.
|
||||
|
||||
We will review incoming bugs and blueprints on an ongoing basis. It is
|
||||
always okay to ask for feedback on a bug or blueprint that has been submitted.
|
||||
We always welcome you in the IRC channel, the mailing list, and the weekly
|
||||
meeting. We just ask that you understand that there may be many reviews
|
||||
happening at any point in time and it may take a little time for reviews to be
|
||||
completed.
|
||||
|
||||
.. _bug report:
|
||||
|
||||
Bug Reports
|
||||
-----------
|
||||
|
||||
Current Bugs are found here:
|
||||
|
||||
* https://storyboard.openstack.org/#!/project_group/93
|
||||
|
||||
A bug may be filed by adding a story.
|
||||
|
||||
Please provide information on what the problem is, how to replicate it,
|
||||
any suggestions for fixing it, and a recommendation of the priority.
|
||||
|
||||
Security Bugs
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Reporting bugs referencing security related vulnerabilities in released
|
||||
versions of Searchlight requires additional consideration. A bug with security
|
||||
implications should be filed as a private security bug. This prevents public
|
||||
disclosure of potential security issues before they can be addressed.
|
||||
|
||||
To mark a bug as a private security bug, set the value for the field "This bug
|
||||
contains information that is:" from "Public" to "Private Security". If you have
|
||||
questions, please contact either of the following groups:
|
||||
|
||||
* `Searchlight Core Security Team <https://launchpad.net/~searchlight-coresec>`_
|
||||
* `OpenStack Vulnerability Management Process <https://security.openstack.org/vmt-process.html>`_
|
||||
|
||||
In the event that a bug filed as a private security bug is determined not to
|
||||
have security implications, the bug will be moved to a public bug report.
|
||||
|
||||
.. _blueprint:
|
||||
|
||||
Blueprints
|
||||
----------
|
||||
|
||||
Current blueprints are found here:
|
||||
|
||||
* https://storyboard.openstack.org/#!/project_group/93
|
||||
|
||||
A blueprint may be filed by adding a story.
|
||||
|
||||
The initial blueprint primarily needs to express the intent of the idea with
|
||||
enough details that the idea can be evaluated for compatibility with the
|
||||
Searchlight mission and whether or not the change requires a
|
||||
`specification`_ for change tracked reviews. It is *not*
|
||||
expected to contain all of the implementation details. If the feature
|
||||
is very simple and well understood by the team, then describe it simply.
|
||||
Searchlight team members will request more information as needed.
|
||||
|
||||
If the blueprint starts to seem non-trivial, or seem like it will benefit
|
||||
from a better tool for comments and change tracking, then you should
|
||||
submit a `specification`_ to gerrit proactively and simply
|
||||
link to it from your blueprint to accommodate better reviews.
|
||||
|
||||
|
||||
.. _specification:
|
||||
|
||||
Specifications
|
||||
--------------
|
||||
|
||||
We use the `searchlight-specs
|
||||
<https://opendev.org/openstack/searchlight-specs>`_ repository for
|
||||
specification reviews. Specifications:
|
||||
|
||||
* Provide a review tool for collaborating on feedback and reviews for complex features
|
||||
* Serve as the basis for documenting the feature once implemented
|
||||
* Ensure that the overall impact on the system is considered
|
||||
|
||||
The Searchlight team does not enforce deadlines for specs. These can be submitted
|
||||
throughout the release cycle. The drivers team will review this on a regular
|
||||
basis throughout the release, and based on the load for the milestones, will
|
||||
assign these into milestones or move them to the backlog for selection into
|
||||
a future release.
|
||||
|
||||
Please note that we use a `template
|
||||
<https://opendev.org/openstack/searchlight-specs/src/branch/master/specs/template.rst>`_
|
||||
for spec submissions. It is not required to fill out all sections in the
|
||||
template. Review of the spec may require filling in information left out by
|
||||
the submitter.
|
||||
|
||||
The review system will run a few tests to check the basic format and
|
||||
syntax of your spec. You will just need to run ``tox`` locally from within
|
||||
the checked out spec repository to replicate the review tests.
|
||||
|
||||
Release Notes
|
||||
=============
|
||||
|
||||
The release notes for a patch should be included in the patch. If not, the
|
||||
release notes should be in a follow-on review.
|
||||
|
||||
If any of the following applies to the patch, a release note is required:
|
||||
|
||||
* The deployer needs to take an action when upgrading
|
||||
* A new feature is implemented
|
||||
* Plugin API function was removed or changed
|
||||
* Current behavior is changed
|
||||
* A new config option is added that the deployer should consider changing from
|
||||
the default
|
||||
* A security bug is fixed
|
||||
* Change may break previous versions of the client library(ies)
|
||||
* Requirement changes are introduced for important libraries like oslo, six
|
||||
requests, etc.
|
||||
* Deprecation period starts or code is purged
|
||||
|
||||
A release note is suggested if a long-standing or important bug is fixed.
|
||||
Otherwise, a release note is not required.
|
||||
|
||||
Searchlight uses `reno <https://docs.openstack.org/reno/latest/user/usage.html>`_ to
|
||||
generate release notes. Please read the docs for details. In summary, use
|
||||
the following:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ tox -e venv -- reno new <bug-,bp-,whatever>
|
||||
|
||||
Then edit the sample file that was created and push it with your change.
|
||||
|
||||
To see the results:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ git commit # Commit the change because reno scans git log.
|
||||
|
||||
$ tox -e releasenotes
|
||||
|
||||
Then look at the generated release notes files in releasenotes/build/html in
|
||||
your favorite browser.
|
@ -1,12 +0,0 @@
|
||||
==================
|
||||
Contribution guide
|
||||
==================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
contributing
|
||||
searchlight-vision
|
||||
vision-reflection
|
||||
authoring-plugins
|
||||
feature-requests-bugs
|
@ -1,52 +0,0 @@
|
||||
..
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
|
||||
Searchlight Team Vision
|
||||
=======================
|
||||
|
||||
Searchlight was originally developed and released in the Kilo release of
|
||||
Glance as the Catalog Index Service [#]_. At the Liberty Summit, we decided
|
||||
to broaden the scope to provide advanced and scalable search across
|
||||
multi-tenant cloud resources. And in the Stein cycle, we have developed a
|
||||
grand vision for Searchlight to help sustain the project in alignment with
|
||||
the OpenStack Cloud Vision [#]_.
|
||||
|
||||
|
||||
Our Vision
|
||||
----------
|
||||
|
||||
With the modular architecture of Searchlight and based on the discussions of
|
||||
the Searchlight team, we envisioned making Searchlight a universal search
|
||||
interface not only for OpenStack but also other cloud platforms such as
|
||||
Microsoft Azure [#]_, AWS [#]_, or even Kubernetes [#]_, etc.. The final
|
||||
product of this vision could be building a multi-cloud management application
|
||||
and a unified API for multi-cloud resource discovery, which serves as a
|
||||
cloud information base for automation application, e.g., VIM management,
|
||||
NFV MANO. This requires new designs of additional data models, APIs,
|
||||
communication, and features which are analyzed further in specific use
|
||||
case analysis & design documents. References will be provided when available.
|
||||
|
||||
.. figure:: ../../../images/SeaaS.png
|
||||
:width: 100%
|
||||
:alt: Search as a Service
|
||||
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
.. [#] http://specs.openstack.org/openstack/glance-specs/specs/kilo/catalog-index-service.html
|
||||
.. [#] https://governance.openstack.org/tc/reference/technical-vision.html
|
||||
.. [#] https://storyboard.openstack.org/#!/story/2004718
|
||||
.. [#] https://storyboard.openstack.org/#!/story/2004719
|
||||
.. [#] https://storyboard.openstack.org/#!/story/2004382
|
@ -1,77 +0,0 @@
|
||||
===========================================================
|
||||
Searchlight reflects on the 2018 OpenStack Technical Vision
|
||||
===========================================================
|
||||
|
||||
In late-2018, the OpenStack Technical Committee composed a technical vision
|
||||
[#]_ of what OpenStack clouds should look like. This document compares the
|
||||
state of Searchlight relative to that vision to provide some guidance on
|
||||
broad-stroke ways in which Searchlight may need to change to match the vision.
|
||||
Note that there is also a Searchlight Vision document [#]_.
|
||||
|
||||
The TC vision document is divided into three sections, which this document
|
||||
mirrors. This should be a living document which evolves as Searchlight itself
|
||||
evolves.
|
||||
|
||||
|
||||
The Pillars of Cloud
|
||||
====================
|
||||
|
||||
While the Searchlight API performs read-only accesses of Elasticsearch for
|
||||
user-facing features (i.e., search), the Searchlight Amin CLI provides
|
||||
read-write access to Elasticsearch for administrative tasks (e.g., re-index
|
||||
data, etc.), so basically, anything can talk to it, enabling the self-service
|
||||
and application control features that define a cloud.
|
||||
|
||||
|
||||
OpenStack-specific Considerations
|
||||
=================================
|
||||
|
||||
Interoperability
|
||||
----------------
|
||||
|
||||
Because Searchlight only has Keystone as a hard-dependency, Searchlight
|
||||
deployment can be ported from one OpenStack cloud to another easily with
|
||||
minimal modification of the settings depends on the services Searchlight want
|
||||
to index.
|
||||
|
||||
Bidirectional Compatibility
|
||||
---------------------------
|
||||
|
||||
Searchlight APIs is versioned and designed to allow client introspection of
|
||||
the available versions and features.
|
||||
|
||||
Cross-Project Dependencies
|
||||
--------------------------
|
||||
|
||||
Searchlight depends on Keystone to have universal access to other OpenStack
|
||||
services and user authentication. Other services that Searchlight supports
|
||||
(e.g., Nova, Cinder, Glance, etc.) are soft dependencies.
|
||||
|
||||
Partitioning
|
||||
------------
|
||||
|
||||
At the current state, Searchlight only supports indexing resource information
|
||||
of one single OpenStack instance where Searchlight deployed. In the Stein
|
||||
cycle, we have designed a vision that is to make Searchlight a multi-cloud
|
||||
solution. It means that Searchlight will be able to search resource
|
||||
information across different cloud platforms and tenants. Moreover, by going
|
||||
in that direction, Searchlight would offer features such as tagging and or
|
||||
partitioning resource information arbitrarily.
|
||||
|
||||
|
||||
Design Goals
|
||||
============
|
||||
|
||||
Searchlight already maps well to most of the design goals in the TC vision
|
||||
document such as scalability, reliability, customization, and flexible
|
||||
utilization models. Searchlight also offers a graphical user interface to
|
||||
query the indexed resources. We should strive to keep this. Details of how we
|
||||
plan to do so should be maintained in the Searchlight Vision document.
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
.. [#] https://governance.openstack.org/tc/reference/technical-vision.html
|
||||
|
||||
.. [#] https://docs.openstack.org/searchlight/latest/contributor/searchlight-vision.html
|
@ -1,113 +0,0 @@
|
||||
..
|
||||
Copyright 2010 OpenStack Foundation
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
Welcome to Searchlight's documentation!
|
||||
=======================================
|
||||
|
||||
The Searchlight project provides indexing and search capabilities across
|
||||
OpenStack resources. Its goal is to achieve high performance and flexible
|
||||
querying combined with near real-time indexing. It uses
|
||||
`Elasticsearch <http://http://www.elasticsearch.org/>`_, a real-time
|
||||
distributed indexing and search engine built on Apache Lucene, but adds
|
||||
OpenStack authentication and Role Based Access Control to provide appropriate
|
||||
protection of data.
|
||||
|
||||
Searchlight, as with all OpenStack projects, is written with the following design
|
||||
guidelines in mind:
|
||||
|
||||
* **Microservice based architecture**: Quickly add new behaviors
|
||||
* **Highly available**: Scale to very serious workloads
|
||||
* **Fault tolerant**: Isolated processes avoid cascading failures
|
||||
* **Recoverable**: Failures should be easy to diagnose, debug, and rectify
|
||||
* **Open standards**: Be a reference implementation for a community-driven api
|
||||
|
||||
This documentation is generated by the Sphinx toolkit using `tox -e docs` and
|
||||
lives in the source tree.
|
||||
|
||||
Additional documentation on Searchlight can be found on the `OpenStack wiki`_.
|
||||
|
||||
.. _`OpenStack wiki`: https://wiki.openstack.org/wiki/Searchlight
|
||||
|
||||
Indexes
|
||||
-------
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
user/index
|
||||
admin/index
|
||||
install/index
|
||||
configuration/index
|
||||
contributor/index
|
||||
cli/index
|
||||
|
||||
|
||||
Contributing
|
||||
------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
contributor/feature-requests-bugs
|
||||
contributor/authoring-plugins
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
admin/architecture
|
||||
user/searchlightapi
|
||||
admin/indexingservice
|
||||
configuration/authentication
|
||||
|
||||
Client and UI
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
* `Horizon UI Plugin`_
|
||||
* `OpenStack Client Plugin`_
|
||||
|
||||
.. _`Horizon UI Plugin`: https://opendev.org/openstack/searchlight-ui
|
||||
.. _`OpenStack Client Plugin`: https://opendev.org/openstack/python-searchlightclient
|
||||
|
||||
Search plugins
|
||||
--------------
|
||||
|
||||
The search service determines the type of information that is indexed and
|
||||
searchable via a plugin mechanism.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
configuration/plugins
|
||||
contributor/authoring-plugins
|
||||
|
||||
See each plugin below for detailed information about specific plugins:
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:glob:
|
||||
|
||||
admin/plugins/*
|
||||
|
||||
Installation and Setup
|
||||
----------------------
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
install/dev-environment
|
||||
install/elasticsearch
|
||||
install/uwsgi
|
@ -1,394 +0,0 @@
|
||||
..
|
||||
c) Copyright 2015 Hewlett-Packard Development Company, L.P.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
***********************
|
||||
Development Environment
|
||||
***********************
|
||||
|
||||
This guide will walk you through setting up a typical development
|
||||
environment. You may set it up using devstack or manually.
|
||||
|
||||
.. _Devstack Development Environment:
|
||||
|
||||
Devstack Development Environment
|
||||
++++++++++++++++++++++++++++++++
|
||||
|
||||
Please see: https://opendev.org/openstack/searchlight/src/branch/master/devstack
|
||||
|
||||
.. _Manual Development Environment:
|
||||
|
||||
Manual Development Environment
|
||||
++++++++++++++++++++++++++++++
|
||||
|
||||
Prerequisites
|
||||
=============
|
||||
|
||||
Install prerequisites::
|
||||
|
||||
# Ubuntu/Debian:
|
||||
sudo apt-get update
|
||||
sudo apt-get install python-dev libssl-dev python-pip libxml2-dev \
|
||||
libxslt-dev git git-review libffi-dev gettext \
|
||||
python-tox
|
||||
|
||||
# Fedora/RHEL:
|
||||
sudo yum install python-devel openssl-devel python-pip libxml2-devel \
|
||||
libxslt-devel git git-review libffi-devel gettext
|
||||
|
||||
# openSUSE/SLE 12:
|
||||
sudo zypper install git git-review libffi-devel \
|
||||
libopenssl-devel libxml2-devel libxslt-devel \
|
||||
python-devel python-flake8 \
|
||||
python-nose python-pip python-setuptools-git \
|
||||
python-testrepository python-tox python-virtualenv \
|
||||
gettext-runtime
|
||||
|
||||
sudo easy_install nose
|
||||
sudo pip install virtualenv setuptools-git flake8 tox testrepository
|
||||
|
||||
If using RHEL and yum reports "No package python-pip available" and "No
|
||||
package git-review available", use the EPEL software repository. Instructions
|
||||
can be found at `<http://fedoraproject.org/wiki/EPEL/FAQ#howtouse>`_.
|
||||
|
||||
You may need to explicitly upgrade virtualenv if you've installed the one
|
||||
from your OS distribution and it is too old (tox will complain). You can
|
||||
upgrade it individually, if you need to::
|
||||
|
||||
sudo pip install -U virtualenv
|
||||
|
||||
This guide assumes that you also have the following services
|
||||
minimally available:
|
||||
|
||||
* Elasticsearch (See :doc:`elasticsearch` for install information).
|
||||
* Keystone
|
||||
|
||||
.. note::
|
||||
Some Elasticsearch installation methods may call for installing a
|
||||
headless JRE. If you are working on a desktop based OS (such as Ubuntu
|
||||
14.04), this may cause tools like pycharms to no longer launch. You can
|
||||
switch between JREs and back: to a headed JRE version using:
|
||||
"sudo update-alternatives --config java".
|
||||
|
||||
Additional services required to be installed will depend on the plugins
|
||||
activated in searchlight.
|
||||
|
||||
.. _Installation Instructions:
|
||||
|
||||
Installing Searchlight
|
||||
======================
|
||||
|
||||
.. index::
|
||||
double: install; searchlight
|
||||
|
||||
1. Clone the Searchlight repo from opendev.org
|
||||
|
||||
::
|
||||
|
||||
$ mkdir openstack
|
||||
$ cd openstack
|
||||
$ git clone https://opendev.org/openstack/searchlight.git
|
||||
$ cd searchlight
|
||||
|
||||
|
||||
2. Setup a virtualenv
|
||||
|
||||
.. note::
|
||||
This is an optional step, but will allow Searchlight's dependencies
|
||||
to be installed in a contained environment that can be easily deleted
|
||||
if you choose to start over or uninstall Searchlight.
|
||||
|
||||
::
|
||||
|
||||
$ tox -evenv --notest
|
||||
|
||||
Activate the virtual environment whenever you want to work in it.
|
||||
All further commands in this section should be run with the venv active:
|
||||
|
||||
::
|
||||
|
||||
$ . .tox/venv/bin/activate
|
||||
|
||||
.. note::
|
||||
When ALL steps are complete, deactivate the virtualenv: $ deactivate
|
||||
|
||||
4. Install Searchlight and its dependencies
|
||||
|
||||
::
|
||||
|
||||
(venv) $ python setup.py develop
|
||||
|
||||
5. Generate sample config.
|
||||
|
||||
::
|
||||
|
||||
(venv) $ oslo-config-generator --config-file
|
||||
etc/oslo-config-generator/searchlight.conf
|
||||
|
||||
6. Create Searchlight's config files by copying the sample config files
|
||||
|
||||
::
|
||||
|
||||
$ cd etc/
|
||||
$ ls *.sample | while read f; do cp $f $(echo $f | sed "s/.sample$//g"); done
|
||||
|
||||
7. Make the directory for Searchlight's log files
|
||||
|
||||
::
|
||||
|
||||
$ mkdir -p ../log
|
||||
|
||||
8. Make the directory for Searchlight's state files
|
||||
|
||||
::
|
||||
|
||||
$ mkdir -p ../state
|
||||
|
||||
Configuring Searchlight
|
||||
=======================
|
||||
|
||||
.. index::
|
||||
double: configure; searchlight
|
||||
|
||||
Searchlight has several configuration files. The following are the basic
|
||||
configuration suggestions.
|
||||
|
||||
Keystone integration
|
||||
--------------------
|
||||
|
||||
Keystone integration should be set up for proper authentication and service
|
||||
integration
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
../configuration/authentication
|
||||
|
||||
Other development environment configuration
|
||||
-------------------------------------------
|
||||
|
||||
Additional development environment configuration items are specified below.
|
||||
|
||||
searchlight.conf
|
||||
````````````````
|
||||
::
|
||||
|
||||
[DEFAULT]
|
||||
debug = true
|
||||
log_file = log/searchlight.log
|
||||
|
||||
Plugin Configuration
|
||||
--------------------
|
||||
|
||||
The search service is driven using a plugin mechanism for integrating to other
|
||||
services. Each integrated service may require additional configuration
|
||||
settings. For example, you typically will need to enable the
|
||||
``oslo_messaging_notifications`` messaging driver and may need to add
|
||||
the ``notifications`` topic to each service's configuration.
|
||||
|
||||
Searchlight uses notification messaging pools. This usually does not
|
||||
require changing any service configurations beyond enabling the notifications
|
||||
driver. Searchlight uses sensible defaults for most deployments, but if you
|
||||
want to customize the settings see :ref:`searchlight-plugins` for plugin
|
||||
installation and general configuration information.
|
||||
|
||||
See each plugin below for detailed information about specific plugins:
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:glob:
|
||||
|
||||
../admin/plugins/*
|
||||
|
||||
Initialize the Elasticsearch Index
|
||||
==================================
|
||||
|
||||
.. index::
|
||||
double: install; sync
|
||||
|
||||
|
||||
Perform initial sync of the resources to Elasticsearch. All plugins for
|
||||
searchlight must have their services installed and available at an endpoint
|
||||
registered in keystone (e.g. glance).
|
||||
|
||||
::
|
||||
|
||||
$ cd <install dir. eg: openstack/searchlight>
|
||||
|
||||
# Make sure your virtualenv is sourced
|
||||
$ . .tox/venv/bin/activate
|
||||
|
||||
# Run the index operation.
|
||||
(venv) $ searchlight-manage index sync
|
||||
|
||||
# Alternatively, you can directly invoke the command using the following.
|
||||
(venv) $ python searchlight/cmd/manage.py --config-file etc/searchlight.conf index sync
|
||||
|
||||
This command may be re-run at any time to perform a full re-index.
|
||||
|
||||
.. IMPORTANT:: You must perform initial sync to create a new index in Elasticsearch,
|
||||
even if there are no resources to sync in your environment.
|
||||
|
||||
The V2 Elasticsearch Client
|
||||
---------------------------
|
||||
|
||||
.. IMPORTANT:: Please read if you receive a warning about using the
|
||||
elasticsearch v2 client when running `index sync`
|
||||
|
||||
The v2 elasticsearch client removed functionality Searchlight uses to
|
||||
clear existing data. If the v2 client is installed, you will receive a
|
||||
warning when attempting to index data. A workaround is to run the
|
||||
`index sync` command below with the `--no-delete` flag. If you have
|
||||
existing data indexed, at present it is necessary to delete the entire
|
||||
index and reindex. You can do this by running
|
||||
|
||||
::
|
||||
|
||||
# Assume elasticsearch is running on localhost, and the default
|
||||
# 'searchlight' index is in use
|
||||
curl -X DELETE localhost:9200/searchlight
|
||||
|
||||
(venv) $ searchlight-manage --config-file etc/searchlight.conf index sync --no-delete
|
||||
|
||||
Alternatively, install the Elasticsearch 1.9.0 client by editing
|
||||
`requirements.txt` and making the following change
|
||||
|
||||
::
|
||||
|
||||
# Change THIS
|
||||
elasticsearch>=1.3.0
|
||||
# To THIS
|
||||
elasticsearch>=1.3.0,<2.0.0
|
||||
|
||||
Then re-install requirements and index
|
||||
|
||||
::
|
||||
|
||||
(venv) $ pip install -r requirements.txt
|
||||
(venv) $ searchlight-manage --config-file etc/searchlight.conf index sync
|
||||
|
||||
Note that if you are running a version 2 elasticsearch *server*, the 1.x
|
||||
*client* will not work and you must follow the workaround above.
|
||||
|
||||
Start Index Update Monitoring
|
||||
=============================
|
||||
|
||||
The index is updated continually based on updates to the source resource
|
||||
data. Start this service to start update monitoring. Note, depending on the
|
||||
resource type, this will typically require that you have configured
|
||||
notifications properly for the service which owns the resource (e.g. Glance
|
||||
images).
|
||||
|
||||
::
|
||||
|
||||
$ cd <install dir. eg: openstack/searchlight>
|
||||
|
||||
# Make sure your virtualenv is sourced
|
||||
$ . .tox/venv/bin/activate
|
||||
|
||||
# Start the index update monitoring.
|
||||
(venv) $ searchlight-listener --config-file etc/searchlight.conf
|
||||
|
||||
# Alternatively, you can directly invoke the command using the following.
|
||||
(venv) $ python searchlight/cmd/listener.py --config-file
|
||||
etc/searchlight.conf
|
||||
|
||||
Initialize & Start the API Service
|
||||
==================================
|
||||
|
||||
.. index::
|
||||
double: install; api
|
||||
|
||||
Open up a new ssh window and log in to your server (or however you're
|
||||
communicating with your server).
|
||||
|
||||
::
|
||||
|
||||
$ cd <install dir. eg: openstack/searchlight>
|
||||
|
||||
# Make sure your virtualenv is sourced
|
||||
$ . .tox/venv/bin/activate
|
||||
|
||||
# Start the API Service.
|
||||
(venv) $ searchlight-api --config-file etc/searchlight.conf
|
||||
|
||||
# Alternatively, you can directly invoke the command using the following.
|
||||
(venv) $ python searchlight/cmd/api.py --config-file etc/searchlight.conf
|
||||
|
||||
You should now see the log from the API service.
|
||||
|
||||
|
||||
Exercising the API
|
||||
==================
|
||||
|
||||
.. note:: If you have a firewall enabled, make sure to open port 9393.
|
||||
|
||||
Using a web browser, curl statement, or a REST client, calls can be made to the
|
||||
Searchlight API using the following format where "api_version" is v1
|
||||
and "command" is any of the commands listed under the :doc:`../user/searchlightapi`
|
||||
|
||||
::
|
||||
|
||||
http://IP.Address:9393/api_version/command
|
||||
|
||||
Example: List plugins::
|
||||
|
||||
$ curl http://localhost:9393/v1/search/plugins
|
||||
|
||||
Example: List all data::
|
||||
|
||||
# Prerequisite Setup:
|
||||
$ token=<insert token>
|
||||
$ touch search.json
|
||||
# Paste content
|
||||
{
|
||||
"query": {
|
||||
"match_all": {}
|
||||
},
|
||||
"limit": 1000
|
||||
}
|
||||
|
||||
# Execute query
|
||||
$ curl -X POST http://localhost:9393/v1/search -H "X-Auth-Token:$token" \
|
||||
-d @search.json -H "Content-Type: application/json" | python -mjson.tool
|
||||
|
||||
You can find the IP Address of your server by running
|
||||
|
||||
::
|
||||
|
||||
curl -s checkip.dyndns.org | sed -e 's/.*Current IP Address: //' -e 's/<.*$//'
|
||||
|
||||
Troubleshooting
|
||||
===============
|
||||
|
||||
Elasticsearch:
|
||||
|
||||
You can directly connect to the Elasticsearch server and examine
|
||||
its contents. We recommend using the Sense extension for google chrome.
|
||||
|
||||
Notifications:
|
||||
|
||||
Use rabbitmqctl to examine unconsumed notifications::
|
||||
|
||||
sudo rabbitmqctl list_queues | grep info
|
||||
|
||||
There are also a number of management tools available to help with
|
||||
troubleshooting.
|
||||
|
||||
Please see: https://www.rabbitmq.com/management.html
|
||||
|
||||
If you have searchlight setup to share a notification topic, but do
|
||||
not have a notification pool configured, then notifications may be consumed
|
||||
by other listeners and searchlight will not receive the notifications.
|
@ -1,246 +0,0 @@
|
||||
..
|
||||
Copyright (c) 2015 Hewlett-Packard Development Company, L.P.
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
|
||||
Installing and Configuring Elasticsearch
|
||||
========================================
|
||||
|
||||
The Searchlight indexing service is responsible for indexing data in
|
||||
Elasticsearch [1]_; Elasticsearch has very good documentation on
|
||||
installation but some pointers are provided here.
|
||||
|
||||
.. IMPORTANT:: We *strongly* recommend using ElasticSearch 5.x and the
|
||||
accompanying python client version [2]_. ElasticSearch 2.x is still
|
||||
supported and will be removed after Stein cycle.
|
||||
|
||||
Installation
|
||||
~~~~~~~~~~~~
|
||||
|
||||
Elasticsearch requires a Java Runtime Environment (or Java Development Kit). OpenJDK
|
||||
and Oracle's Java are supported. Information on the current recommended version can
|
||||
be found in the installation instructions [3]_.
|
||||
|
||||
Installing from a download package
|
||||
##################################
|
||||
|
||||
Links to various formats and also older versions of Elasticsearch can be found
|
||||
on the download page [4]_. Once downloaded and extracted, you can start
|
||||
Elasticsearch with:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
bin/elasticsearch
|
||||
|
||||
For more details see the installation instructions [5]_
|
||||
|
||||
Quick command line example with 5.6.11:
|
||||
|
||||
.. note::
|
||||
|
||||
Do the following commands as "root" or via sudo <command>
|
||||
|
||||
Download the ES package:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
cd ~
|
||||
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.11.deb
|
||||
sudo dpkg -i elasticsearch-5.6.11.deb
|
||||
sudo update-rc.d elasticsearch defaults 95 10
|
||||
sudo /etc/init.d/elasticsearch start
|
||||
|
||||
.. note::
|
||||
|
||||
If you install ElasticSearch 5.x, You should install
|
||||
python-elasticsearch>=5.0.0. global-requirements currently constrains it to v2.x
|
||||
|
||||
Configuration
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Elasticsearch comes with very a very sensible default configuration that
|
||||
allows for clustering and high performance out of the box. There are some
|
||||
settings, both general and specific to Searchlight's indexing service, that might
|
||||
be of interest depending on your scenario.
|
||||
|
||||
Elasticsearch's configuration is in $ES_HOME/config/elasticsearch.yaml, where
|
||||
ES_HOME is the directory in which Elasticsearch is installed. It uses YAML,
|
||||
a superset of JSON.
|
||||
|
||||
Indices
|
||||
#######
|
||||
|
||||
Elasticsearch (and Lucene) store information in indices. Within an index can
|
||||
be one or more document types. Searchlight's indexing service uses an index
|
||||
per service that has a plugin available, and each plugin generally will have
|
||||
its own document type. For instance, the glance plugin has *glance.image* and
|
||||
*glance.metadef*. Since the volume of data is lower than a typical use case for
|
||||
Elasticsearch it may make sense to change the default sharing and replication
|
||||
mechanism. We also recommend disabling implicit index creation, though if you
|
||||
are sharing an Elasticsearch installation this may be inadvisable. The
|
||||
following options control indexing behavior:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
# Number of shards for each index (performance)
|
||||
index.number_of_shards: 5
|
||||
# Number of replicas per shard (redundancy and recovering)
|
||||
index.number_of_replicas: 1
|
||||
|
||||
# Disable automatic index creation so that index creation
|
||||
# is an explicit action
|
||||
action.auto_create_index: false
|
||||
|
||||
Index settings
|
||||
**************
|
||||
|
||||
In addition to server-wide index settings it's possible to configure
|
||||
Searchlight to apply settings to indices it creates with
|
||||
``searchlight-manage``. Index settings can be specified as follows in
|
||||
``searchlight.conf``:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
[elasticsearch]
|
||||
index_settings = refresh_interval:2s,number_of_replicas:1
|
||||
|
||||
The ``index.`` prefix for settings is optional; Searchlight will prepend it if
|
||||
it's not given (e.g. ``index.refresh_interval`` is also acceptable).
|
||||
|
||||
Index settings are applied at creation time and so are not limited to the
|
||||
'dynamic' index settings. They are applied to all indices at the time they
|
||||
are created. If you wish to update settings for an existing index, you
|
||||
should use the Elasticsearch API to do so or reindex.
|
||||
|
||||
See also [9]_, [10]_.
|
||||
|
||||
Scripts
|
||||
#######
|
||||
|
||||
The scripting module allows to use scripts in order to evaluate custom
|
||||
expressions. Scripting is turned off by default in elasticsearch latest
|
||||
versions. Searchlight doesn't allow scripts in the search api but requires
|
||||
scripts to sync Index updates from notifications. For security purpose index
|
||||
updates are allowed only for admin role.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
script.engine.groovy.inline.update: on
|
||||
|
||||
See also [8]_.
|
||||
|
||||
Development
|
||||
###########
|
||||
|
||||
For development, Elasticsearch's default configuration is overkill. It's
|
||||
possible to run Elasticsearch with a much lower memory footprint than by
|
||||
default, and you may wish to disable clustering behavior.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
# Configures elasticsearch as a single node (no discovery)
|
||||
node.local: true
|
||||
|
||||
# Disable sharding and replication
|
||||
index.number_of_shards: 1
|
||||
index.number_of_replicas: 0
|
||||
|
||||
JVM settings
|
||||
************
|
||||
|
||||
Setting the ES_HEAP_SIZE environment variable will restrict how much memory
|
||||
Elasticsearch uses, equivalent to setting -Xmx and -Xms to the same value for
|
||||
the Java runtime. For development you can set it as low as a few tens of MB:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
export ES_HEAP_SIZE=40m
|
||||
|
||||
Memory usage will be somewhat higher than that figure, because Java itself
|
||||
requires memory on top of that.
|
||||
|
||||
Production
|
||||
##########
|
||||
|
||||
Some settings you may wish to change for production:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
# Cluster name is used by cluster discovery; it's important to ensure
|
||||
# this is set across all nodes you wish to be in the cluster
|
||||
cluster.name: searchlight
|
||||
|
||||
# By default elasticsearch picks a random name from a list of Marvel
|
||||
# comic characters. If you specify this, make sure it's different on
|
||||
# each node in the cluster
|
||||
node.name: This Node Name
|
||||
|
||||
# Bind to a non-standard address
|
||||
network.host: 0.0.0.0
|
||||
|
||||
# Bind to a non-standard port
|
||||
http.port: 9200
|
||||
|
||||
# Configure the default data and log directories. By default, these
|
||||
# directories will be created in $ES_HOME.
|
||||
path:
|
||||
logs: /var/log/elasticsearch
|
||||
data: /var/data/elasticsearch
|
||||
|
||||
# This setting locks the Elasticsearch process address space into RAM
|
||||
# (preventing locking). If you set this, ensure that you've configured
|
||||
# ES_HEAP_SIZE appropriately (see below). Linux only.
|
||||
bootstrap.mlockall: true
|
||||
|
||||
For more details see Elasticsearch's configuration information [6]_.
|
||||
|
||||
Specifying nodes in a cluster
|
||||
*****************************
|
||||
|
||||
Elasticsearch's default discovery relies on multicast requests. If this isn't
|
||||
a good fit, you can use unicast discovery:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
discovery.zen.ping.multicast.enabled: false
|
||||
discovery.zen.ping.unicast.hosts: ['w.x.y.z', 'w.x.y.z']
|
||||
|
||||
|
||||
See [7]_ for more details.
|
||||
|
||||
JVM settings
|
||||
************
|
||||
|
||||
For production, Elasticsearch recommends setting the ES_HEAP_SIZE environment
|
||||
variable to a value around 60% of a dedicated machine's memory:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
export ES_HEAP_SIZE=2g
|
||||
|
||||
|
||||
References
|
||||
~~~~~~~~~~
|
||||
|
||||
.. [1] https://www.elastic.co/
|
||||
.. [2] https://elasticsearch-py.readthedocs.io/en/master/#compatibility
|
||||
.. [3] https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html
|
||||
.. [4] https://www.elastic.co/downloads/elasticsearch
|
||||
.. [5] https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html
|
||||
.. [6] https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html
|
||||
.. [7] https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-zen.html
|
||||
.. [8] https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html#modules-scripting
|
||||
.. [9] https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index\_.html
|
||||
.. [10] https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html
|
@ -1,10 +0,0 @@
|
||||
====================
|
||||
Installation guide
|
||||
====================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
dev-environment
|
||||
elasticsearch
|
||||
uwsgi
|
@ -1,71 +0,0 @@
|
||||
..
|
||||
Copyright (c) 2017 Intel Corporation
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
|
||||
Running Searchlight API using uWSGI
|
||||
===================================
|
||||
The recommended way to deploy Searchlight is have a web server such as Apache
|
||||
or nginx to handle http requests and proxy these requests to Searchlight WSGI
|
||||
app running in uWSGI. Searchlight comes with some configuration templates on
|
||||
how to deploy the api service with Apache and uWSGI.
|
||||
|
||||
app.wsgi
|
||||
********
|
||||
The ``searchlight/api/app.wsgi`` file contains a WSGI application of
|
||||
Searchlight API service. This file is installed with Searchlight application
|
||||
code.
|
||||
|
||||
apache-searchlight.template
|
||||
***************************
|
||||
The ``searchlight/etc/apache-searchlight.template`` file contains a copy
|
||||
of Apache configuration file for Searchlight API used by devstack.
|
||||
|
||||
searchlight-uwsgi.ini.sample
|
||||
****************************
|
||||
The ``searchlight/etc/searchlight-uwsgi.ini.sample`` file is a sample
|
||||
configuration file for uWSGI server. Update the file to match your
|
||||
system configuration.
|
||||
|
||||
Steps to use these sample configuration files:
|
||||
|
||||
1. Enable mod_proxy_uwsgi module
|
||||
|
||||
* On Ubuntu install required uwsgi package
|
||||
``sudo apt-get install libapache2-mod-proxy-uwsgi``; enable using
|
||||
``sudo a2enmod proxy``, ``sudo a2enmod proxy_uwsgi``.
|
||||
* On Fedora the required package is mod_proxy_uwsgi; enable by creating a file
|
||||
``/etc/httpd/conf.modules.d/11-proxy_uwsgi.conf`` containing
|
||||
``LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so``
|
||||
|
||||
2. On deb-based systems copy or symlink the file to
|
||||
``/etc/apache2/sites-available/searchlight.conf``. For rpm-based systems the file should go into
|
||||
``/etc/httpd/conf.d/searchlight.conf``.
|
||||
|
||||
3. Enable Searchlight site. On deb-based systems::
|
||||
|
||||
$ a2ensite searchlight
|
||||
$ service apache2 reload
|
||||
|
||||
On rpm-based systems::
|
||||
|
||||
$ service httpd reload
|
||||
|
||||
4. Copy searchlight/etc/searchlight-uwsgi.ini.sample to /etc/searchlight/searchlight-uwsgi.ini.
|
||||
|
||||
5. Start Searchlight api using uWSGI::
|
||||
|
||||
$ sudo pip install uwsgi
|
||||
$ uwsgi /etc/searchlight/searchlight-uwsgi.ini
|
@ -1,9 +0,0 @@
|
||||
============
|
||||
User guide
|
||||
============
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
usecases
|
||||
searchlightapi
|
@ -1,827 +0,0 @@
|
||||
..
|
||||
Copyright (c) 2015 Hewlett-Packard Development Company, L.P.
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
Searchlight API
|
||||
===============
|
||||
|
||||
Searchlight's API adds authentication and Role Based Access Control in front
|
||||
of Elasticsearch's query API.
|
||||
|
||||
Authentication
|
||||
--------------
|
||||
|
||||
Searchlight, like other OpenStack APIs, depends on Keystone and the
|
||||
OpenStack Identity API to handle authentication. You must obtain an
|
||||
authentication token from Keystone and pass it to Searchlight in API requests
|
||||
with the ``X-Auth-Token`` header.
|
||||
|
||||
See :doc:`../configuration/authentication` for more information on integrating with Keystone.
|
||||
|
||||
Using v1
|
||||
--------
|
||||
|
||||
For the purposes of examples, assume a Searchlight server is running
|
||||
at the URL ``http://searchlight.example.com`` on HTTP port 80. All
|
||||
queries are assumed to include an ``X-Auth-Token`` header. Where request
|
||||
bodies are present, it is assumed that an appropriate ``Content-Type``
|
||||
header is present (usually ``application/json``).
|
||||
|
||||
Searches use Elasticsearch's
|
||||
`query DSL <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html>`_.
|
||||
|
||||
Elasticsearch stores each 'document' in an 'index', which has one or more
|
||||
'types'. Searchlight's indexing service stores all resource
|
||||
types in their own document type, grouped by service into indices. For
|
||||
instance, the ``OS::Glance::Image`` and ``OS::Glance::Metadef`` types both
|
||||
reside in the ``searchlight`` index. ``type`` is unique to a resource type.
|
||||
|
||||
Document access is defined by each document type, for instance for glance
|
||||
images:
|
||||
|
||||
* If the current user is the resource owner OR
|
||||
* If the resource is marked public
|
||||
|
||||
Some resources may have additional rules. Administrators have access to all resources,
|
||||
though by default searches are restricted to the current tenant unless ``all_projects``
|
||||
is set in the search request body.
|
||||
|
||||
Querying available plugins
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Searchlight indexes OpenStack resources as defined by installed plugins. In
|
||||
general, a plugin maps directly to an OpenStack resource type. For instance, a
|
||||
plugin might index nova instances, or glance images. There may be multiple
|
||||
plugins related to a given OpenStack project (an example being glance images
|
||||
and metadefs).
|
||||
|
||||
A given deployment may not necessarily expose all available plugins.
|
||||
Searchlight provides a REST endpoint to request a list of installed plugins.
|
||||
A ``GET`` request to ``http://searchlight.example.com/v1/search/plugins``
|
||||
might yield::
|
||||
|
||||
{
|
||||
"plugins": [
|
||||
{
|
||||
"type": "OS::Glance::Image",
|
||||
"alias-searching": "searchlight-search"
|
||||
"alias-indexing": "searchlight-listener"
|
||||
},
|
||||
{
|
||||
"type": "OS::Glance::Metadef",
|
||||
"alias-searching": "searchlight-search"
|
||||
"alias-indexing": "searchlight-listener"
|
||||
}
|
||||
|
||||
]
|
||||
}
|
||||
|
||||
This response shows the plugin information associated with the Glance image
|
||||
and metadef resources.
|
||||
|
||||
* **type**: the resource group, which is used as the document type in
|
||||
Elasticsearch.
|
||||
* **alias-searching**: the Elasticsearch alias used for querying.
|
||||
* **alias-indexing**: the Elasticsearch alias used for indexing.
|
||||
|
||||
If desired, all indexed Glance images can be queried directly from
|
||||
Elasticsearch, rather than using Searchlight. Assuming an Elasticsearch
|
||||
server running on localhost, the following request can be made::
|
||||
|
||||
curl http://localhost:9200/searchlight-search/OS::Glance::Image/_search
|
||||
|
||||
Running a search
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
The simplest query is to ask for everything we have access to. We issue a
|
||||
``POST`` request to ``http://searchlight.example.com/v1/search`` with the
|
||||
following body::
|
||||
|
||||
{
|
||||
"query": {
|
||||
"match_all": {}
|
||||
}
|
||||
}
|
||||
|
||||
The data is returned as a JSON-encoded mapping from Elasticsearch::
|
||||
|
||||
{
|
||||
"_shards": {
|
||||
"failed": 0,
|
||||
"successful": 2,
|
||||
"total": 2
|
||||
},
|
||||
"hits": {
|
||||
"hits": [
|
||||
{
|
||||
"_id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"_index": "searchlight",
|
||||
"_type": "OS::Glance::Image"
|
||||
"_score": 1.0,
|
||||
"_source": {
|
||||
"id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"members": [],
|
||||
"name": "cirros-0.3.2-x86_64-uec",
|
||||
"owner": "d95b27da6e9f4acc9a8031918e443e04",
|
||||
"visibility": "public",
|
||||
...
|
||||
}
|
||||
},
|
||||
{
|
||||
"_id": "OS::Software::DBMS",
|
||||
"_index": "searchlight",
|
||||
"_type": "metadef",
|
||||
"_score": 1.0,
|
||||
"_source": {
|
||||
"description": "A database is an ...",
|
||||
"display_name": "Database Software",
|
||||
"namespace": "OS::Software::DBMS",
|
||||
"objects": [
|
||||
{
|
||||
"description": "PostgreSQL, often simply 'Postgres' ...",
|
||||
"name": "PostgreSQL",
|
||||
"properties": [
|
||||
{
|
||||
"default": "5432",
|
||||
"description": "Specifies the TCP/IP port...",
|
||||
"property": "sw_database_postgresql_listen_port",
|
||||
...
|
||||
},
|
||||
...
|
||||
]
|
||||
}
|
||||
],
|
||||
"tags": [
|
||||
{
|
||||
"name": "Database"
|
||||
},
|
||||
]
|
||||
}
|
||||
},
|
||||
...
|
||||
],
|
||||
"max_score": 1.0,
|
||||
"total": 8
|
||||
},
|
||||
"timed_out": false,
|
||||
"took": 1
|
||||
}
|
||||
|
||||
Each ``hit`` is a document in Elasticsearch, representing an OpenStack
|
||||
resource. the fields in the root of each hit are:
|
||||
|
||||
* ``_id``
|
||||
|
||||
Uniquely identifies the resource within its OpenStack context (for
|
||||
instance, Glance images use their GUID).
|
||||
|
||||
* ``_index``
|
||||
|
||||
The service to which the resource belongs (e.g. ``searchlight``).
|
||||
|
||||
* ``_type``
|
||||
|
||||
The document type within the service (e.g. ``image``, ``metadef``)
|
||||
|
||||
* ``_score``
|
||||
|
||||
Where applicable the relevancy of a given ``hit``. By default,
|
||||
the field upon which results are sorted.
|
||||
|
||||
* ``_source``
|
||||
|
||||
The document originally indexed. The ``_source`` is a map, where each key
|
||||
is a ``field`` whose value may be a scalar value, a list, a nested object
|
||||
or a list of nested objects.
|
||||
|
||||
More example searches
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Results are shown here only where it would help illustrate the example. The
|
||||
``query`` parameter supports anything that Elasticsearch exposes via its
|
||||
`query DSL`_. There are normally multiple ways to represent the same query,
|
||||
often with some subtle differences, but some common examples are shown here.
|
||||
|
||||
.. _`query DLS`: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-queries.html
|
||||
|
||||
Administrators - search all resources
|
||||
*************************************
|
||||
By default, all users see search results restricted by access control; in
|
||||
practice, this is a combination of resources belonging to the user's current
|
||||
tenant/project, and any fields that are restricted to administrators.
|
||||
|
||||
Administrators also have the option to view all resources, by passing
|
||||
``all_projects`` in the search request body. For instance, a ``POST`` to
|
||||
``http://searchlight.example.com/searchlight/v1/search``::
|
||||
|
||||
{
|
||||
"query": {
|
||||
"match_all": {}
|
||||
},
|
||||
"all_projects": true
|
||||
}
|
||||
|
||||
|
||||
|
||||
Restricting document index or type
|
||||
**********************************
|
||||
To restrict a query to Glance image and metadef information only (both
|
||||
``index`` and ``type`` can be arrays or a single string)::
|
||||
|
||||
{
|
||||
"query": {
|
||||
"match_all": {}
|
||||
},
|
||||
"type": ["OS::Glance::Image", "OS::Glance::Metadef"]
|
||||
}
|
||||
|
||||
If ``index`` or ``type`` are not provided they will default to covering as
|
||||
wide a range of results as possible. Be aware that it is possible to specify
|
||||
combinations of ``index`` and ``type`` that can return no results. In general
|
||||
``type`` is preferred since ``type`` is unique to a resource.
|
||||
|
||||
Retrieving an item by id
|
||||
************************
|
||||
To retrieve a resource by its OpenStack ID (e.g. a glance image), we can use
|
||||
Elasticsearch's `term query <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-query.html>`_::
|
||||
|
||||
{
|
||||
"index": "searchlight",
|
||||
"query": {
|
||||
"term": {
|
||||
"id": "79fa243d-e05d-4848-8a9e-27a01e83ceba"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Limiting and paging results
|
||||
***************************
|
||||
Elasticsearch (and Searchlight) support paging_ through the
|
||||
``size`` and ``from`` parameters (Searchlight also accepts
|
||||
``limit`` and ``offset`` respectively as synonyms). ``from`` is
|
||||
zero-indexed. If ``size`` is zero, no results will be returned. This
|
||||
can be useful for retrieving the total number of hits for a query without
|
||||
being interested in the results themselves, or for `aggregations`_::
|
||||
|
||||
{
|
||||
"query": {"match_all": {}},
|
||||
"size": 0
|
||||
}
|
||||
|
||||
Gives::
|
||||
|
||||
{
|
||||
"hits": {
|
||||
"hits": [],
|
||||
"max_score": 0.0,
|
||||
"total": 40
|
||||
}
|
||||
}
|
||||
|
||||
.. _paging: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html
|
||||
|
||||
Limiting the fields returned
|
||||
****************************
|
||||
To restrict the ``source`` to include only certain fields using Elasticsearch's
|
||||
`source filtering <https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-source-filtering.html>`_::
|
||||
|
||||
{
|
||||
"type": "OS::Glance::Image",
|
||||
"_source": ["name", "size"]
|
||||
}
|
||||
|
||||
Gives::
|
||||
|
||||
{
|
||||
"_shards": {
|
||||
"failed": 0,
|
||||
"successful": 1,
|
||||
"total": 1
|
||||
},
|
||||
"hits": {
|
||||
"hits": [
|
||||
{
|
||||
"_id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"_index": "searchlight",
|
||||
"_score": 1.0,
|
||||
"_source": {
|
||||
"name": "cirros-0.3.2-x86_64-uec",
|
||||
"size": 3723817
|
||||
},
|
||||
"_type": "OS::Glance::Image"
|
||||
},
|
||||
...
|
||||
],
|
||||
"max_score": 1.0,
|
||||
"total": 4
|
||||
},
|
||||
"timed_out": false,
|
||||
"took": 1
|
||||
}
|
||||
|
||||
Versioning
|
||||
**********
|
||||
Internally an always-incrementing value is stored with search results to
|
||||
ensure that out of order notifications don't lead to inconsistencies with
|
||||
search results. Normally this value is not exposed in search results, but
|
||||
including a search parameter ``version: true`` in requests will result in
|
||||
a field named ``_version`` (note the underscore) being present in each result::
|
||||
|
||||
{
|
||||
"index": "searchlight",
|
||||
"query": {"match_all": {}},
|
||||
"version": true
|
||||
}
|
||||
|
||||
{
|
||||
"hits": {
|
||||
"hits": [
|
||||
{
|
||||
"_id": "76580e9d-f83d-49d8-b428-1fb90c5d8e95",
|
||||
"_index": "searchlight",
|
||||
"_version": 462198730000000000,
|
||||
....
|
||||
},
|
||||
....
|
||||
]
|
||||
},
|
||||
...
|
||||
}
|
||||
|
||||
Sorting
|
||||
*******
|
||||
Elasticsearch allows sorting by single or multiple fields. See Elasticsearch's
|
||||
`sort <https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html>`_
|
||||
documentation for details of the allowed syntax. Sort fields can be included as a top
|
||||
level field in the request body. For instance::
|
||||
|
||||
{
|
||||
"query": {"match_all": {}},
|
||||
"sort": {"name": "desc"}
|
||||
}
|
||||
|
||||
You will see in the search results a ``sort`` field for each result::
|
||||
|
||||
...
|
||||
{
|
||||
"_id": "7741fbcc-3fa9-4ace-adff-593304b6e629",
|
||||
"_index": "glance",
|
||||
"_score": null,
|
||||
"_source": {
|
||||
"name": "cirros-0.3.4-x86_64-uec",
|
||||
"size": 25165824
|
||||
},
|
||||
"_type": "image",
|
||||
"sort": [
|
||||
"cirros-0.3.4-x86_64-uec",
|
||||
25165824
|
||||
]
|
||||
},
|
||||
...
|
||||
|
||||
Wildcards
|
||||
*********
|
||||
Elasticsearch supports regular expression searches but often wildcards within
|
||||
``query_string`` elements are sufficient, using ``*`` to represent one or more
|
||||
characters or ``?`` to represent a single character. Note that *starting* a
|
||||
search term with a wildcard can lead to *extremely* slow queries::
|
||||
|
||||
{
|
||||
"query": {
|
||||
"query_string": {
|
||||
"query": "name: ubun?u AND mysql_version: 5.*"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Highlighting
|
||||
************
|
||||
A common requirement is to highlight search terms in results::
|
||||
|
||||
|
||||
{
|
||||
"type": "OS::Glance::Metadef",
|
||||
"query": {
|
||||
"query_string": {
|
||||
"query": "database"
|
||||
}
|
||||
},
|
||||
"_source": ["namespace", "description"],
|
||||
"highlight": {
|
||||
"fields": {
|
||||
"namespace": {},
|
||||
"description": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Results::
|
||||
|
||||
{
|
||||
"hits": {
|
||||
"hits": [
|
||||
{
|
||||
"_id": "OS::Software::DBMS",
|
||||
"_index": "searchlight",
|
||||
"_type": "OS::Glance::Metadef",
|
||||
"_score": 0.56079304,
|
||||
"_source": {
|
||||
"description": "A database is an organized collection of data. The data is typically organized to model aspects of reality in a way that supports processes requiring information. Database management systems are computer software applications that interact with the user, other applications, and the database itself to capture and analyze data. (http://en.wikipedia.org/wiki/Database)"
|
||||
},
|
||||
"highlight": {
|
||||
"description": [
|
||||
"A <em>database</em> is an organized collection of data. The data is typically organized to model aspects of",
|
||||
" reality in a way that supports processes requiring information. <em>Database</em> management systems are",
|
||||
" computer software applications that interact with the user, other applications, and the <em>database</em> itself",
|
||||
" to capture and analyze data. (http://en.wikipedia.org/wiki/<em>Database</em>)"
|
||||
],
|
||||
"display_name": [
|
||||
"<em>Database</em> Software"
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"max_score": 0.56079304,
|
||||
"total": 1
|
||||
},
|
||||
"timed_out": false,
|
||||
"took": 3
|
||||
}
|
||||
|
||||
Faceting
|
||||
********
|
||||
Searchlight can provide a list of field names and values present for those
|
||||
fields for each registered resource type. Exactly which fields are returned
|
||||
and whether values are listed is up to each plugin. Some fields or values may
|
||||
only be listed for administrative users. For some string fields, 'facet_field'
|
||||
may be included in the result and can be used to do an exact term
|
||||
match against facet options.
|
||||
|
||||
To list supported facets, issue a ``GET`` to
|
||||
``http://searchlight.example.com/v1/search/facets``::
|
||||
|
||||
{
|
||||
"OS::Glance::Image": [
|
||||
{
|
||||
"name": "status",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "created_at",
|
||||
"type": "date"
|
||||
},
|
||||
{
|
||||
"name": "virtual_size",
|
||||
"type": "long"
|
||||
},
|
||||
{
|
||||
"name": "name",
|
||||
"type": "string",
|
||||
"facet_field": "name.raw"
|
||||
},
|
||||
...
|
||||
],
|
||||
"OS::Glance::Metadef": [
|
||||
{
|
||||
"name": "objects.description",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "objects.properties.description",
|
||||
"type": "string",
|
||||
"nested": true
|
||||
},
|
||||
...
|
||||
],
|
||||
"OS::Nova::Server": [
|
||||
{
|
||||
"name": "status",
|
||||
"options": [
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "ACTIVE"
|
||||
}
|
||||
],
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "OS-EXT-SRV-ATTR:host",
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"name": "name",
|
||||
"type": "string",
|
||||
"facet_field": "name.raw"
|
||||
},
|
||||
{
|
||||
"name": "image.id",
|
||||
"type": "string",
|
||||
"nested": false
|
||||
},
|
||||
{
|
||||
"name": "OS-EXT-AZ:availability_zone",
|
||||
"options": [
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "nova"
|
||||
}
|
||||
],
|
||||
"type": "string"
|
||||
}
|
||||
...
|
||||
]
|
||||
}
|
||||
|
||||
Facet fields containing the 'nested' (boolean) attribute indicate that the
|
||||
field mapping type is either 'nested' or 'object'. This can influence how a
|
||||
field should be queried. In general 'object' types are queried as any other
|
||||
field; 'nested' types require some `additional complexity`_.
|
||||
|
||||
It's also possible to request facets for a particular type by adding a
|
||||
``type`` query parameter. For instance, a ``GET`` to
|
||||
``http://searchlight.example.com/v1/search/facets?type=OS::Nova::Server``::
|
||||
|
||||
{
|
||||
"OS::Nova::Server": [
|
||||
{
|
||||
"name": "status",
|
||||
"options": [
|
||||
{
|
||||
"doc_count": 1,
|
||||
"key": "ACTIVE"
|
||||
}
|
||||
],
|
||||
"type": "string"
|
||||
},
|
||||
...
|
||||
]
|
||||
}
|
||||
|
||||
As with searches, administrators are able to request facet terms for all
|
||||
projects/tenants. By default, facet terms are limited to the currently scoped
|
||||
project; adding ``all_projects=true`` as a query parameter removes the
|
||||
restriction.
|
||||
|
||||
It is possible to limit the number of ``options`` returned for fields that
|
||||
support facet terms. ``limit_terms`` restricts the number of terms (sorted
|
||||
in order of descending frequency). A value of 0 indicates no limit, and is the
|
||||
default.
|
||||
|
||||
It is possible to not return any options for facets. By default all options
|
||||
are returned for fields that support facet terms. Adding
|
||||
``exclude_options=true`` as a query parameter will return only the facet
|
||||
field and not any of the options. Using this option will avoid an aggregation
|
||||
query being performed on Elasticsearch, providing a performance boost.
|
||||
|
||||
.. _`additional complexity`: https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html
|
||||
|
||||
Aggregations
|
||||
************
|
||||
`Faceting`_ (above) is a more general form of `Elasticsearch aggregation`_.
|
||||
Faceting is an example of 'bucketing'; 'metrics' includes functions like min,
|
||||
max, percentiles.
|
||||
|
||||
Aggregations will be based on the ``query`` provided as well as restrictions
|
||||
on resource type and any RBAC filters.
|
||||
|
||||
To include aggregations in a query, include ``aggs`` or ``aggregations`` in
|
||||
a search request body. ``"size": 0`` prevents Elasticsearch
|
||||
returning any results, just the aggregation, though it is valid to retrieve
|
||||
both search results and aggregations from a single query. For example::
|
||||
|
||||
{
|
||||
"query": {"match_all": {}},
|
||||
"size": 0,
|
||||
"aggregations": {
|
||||
"names": {
|
||||
"terms": {"field": "name"}
|
||||
},
|
||||
"earliest": {
|
||||
"min": {"field": "created_at"}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Response::
|
||||
|
||||
{
|
||||
"hits": {"total": 2, "max_score": 0.0, "hits": []},
|
||||
"aggregations": {
|
||||
"names": {
|
||||
"doc_count_error_upper_bound": 0,
|
||||
"sum_other_doc_count": 0,
|
||||
"buckets": [
|
||||
{"key": "for_instance1", "doc_count": 2},
|
||||
{"key": "instance1", "doc_count": 1}
|
||||
]
|
||||
},
|
||||
"earliest": {
|
||||
"value": 1459946898000.0,
|
||||
"value_as_string": "2016-04-06T12:48:18.000Z"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Note that for some aggregations ``value_as_string`` may be more useful than
|
||||
``value`` - for example, the ``earliest`` aggregation in the example operates
|
||||
on a date field whose internal representation is a timestamp.
|
||||
|
||||
The `global aggregation`_ type is not allowed because unlike other aggregation
|
||||
types it operates outside the search query scope.
|
||||
|
||||
.. _`Elasticsearch aggregation`: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html
|
||||
.. _`global aggregation`: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-global-aggregation.html
|
||||
|
||||
Freeform queries
|
||||
****************
|
||||
Elasticsearch has a flexible query parser that can be used for many kinds of
|
||||
search terms: the `query_string <https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html>`_
|
||||
operator.
|
||||
|
||||
Some things to bear in mind about using ``query_string`` (see the documentation
|
||||
for full options):
|
||||
|
||||
* A query term may be prefixed with a ``field`` name (as seen below). If it
|
||||
is not, by default the entire document will be searched for the term.
|
||||
* The default operator between terms is ``OR``
|
||||
* By default, query terms are case insensitive
|
||||
|
||||
For instance, the following will look for images with a
|
||||
restriction on name and a range query on size::
|
||||
|
||||
{
|
||||
"query": {
|
||||
"query_string": {
|
||||
"query": "name: (Ubuntu OR Fedora) AND size: [3000000 TO 5000000]"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Within the query string query, you may perform a number of interesting
|
||||
queries. Below are some examples.
|
||||
|
||||
Phrases
|
||||
"""""""
|
||||
::
|
||||
|
||||
\"i love openstack\"
|
||||
|
||||
By default, each word you type will be searched for
|
||||
individually. You may also try to search an exact phrase by
|
||||
using quotes ("my phrase") to surround a phrase. The search
|
||||
service may allow a certain amount of phrase slop - meaning that
|
||||
if you have some words out of order in the phrase it may still
|
||||
match.
|
||||
|
||||
Wildcards
|
||||
"""""""""
|
||||
::
|
||||
|
||||
python3.?
|
||||
10.0.0.*
|
||||
172.*.4.*
|
||||
|
||||
By default, each word you type will match full words
|
||||
only. You may also use wildcards to match parts of words. Wildcard
|
||||
searches can be run on individual terms, using ? to replace a
|
||||
single character, and * to replace zero or more character. 'demo'
|
||||
will match the full word 'demo' only. However, 'de*'
|
||||
will match anything that starts with 'de', such as 'demo_1'.
|
||||
'de*1' will match anything that starts with 'de' and ends with '1'.
|
||||
|
||||
.. note:: Wildcard queries place a heavy burden on the search service and
|
||||
may perform poorly.
|
||||
|
||||
Term Operators
|
||||
""""""""""""""
|
||||
::
|
||||
|
||||
+apache
|
||||
-apache
|
||||
web +(apache OR python)
|
||||
|
||||
Add a '+' or a '-' to indicate terms that must or must
|
||||
not appear. For example '+python -apache web' would find
|
||||
everything that has 'python' does NOT have 'apache' and should have
|
||||
'web'. This may also be used with grouping. For example,
|
||||
'web -(apache AND python)' would find anything with 'web', but does
|
||||
not have either 'apache' or 'python'.
|
||||
|
||||
Boolean Operators
|
||||
"""""""""""""""""
|
||||
::
|
||||
|
||||
python AND apache
|
||||
nginx OR apache
|
||||
web && !apache
|
||||
|
||||
You can separate search terms and groups with
|
||||
AND, OR and NOT (also written &&, || and !). For example,
|
||||
'python OR javascript' will find anything with either term
|
||||
(OR is used by default, so does not need to be specified).
|
||||
However, 'python AND javascript' will find things that only have
|
||||
both terms. You can do this with as many terms as you'd like (e.g.
|
||||
'django AND javascript AND !unholy'). It is important to use all
|
||||
caps or the alternate syntax (&&, ||), because 'and' will be
|
||||
treated as another search term, but 'AND' will be treated as a
|
||||
logical operator.
|
||||
|
||||
Grouping
|
||||
""""""""
|
||||
::
|
||||
|
||||
python AND (2.7 OR 3.4)
|
||||
web && (apache !python)
|
||||
|
||||
Use parenthesis to group different aspects of your
|
||||
query to form sub-queries. For example, 'web OR (python AND
|
||||
apache)' will return anything that either has 'web' OR has both
|
||||
'python' AND 'apache'.
|
||||
|
||||
Facets
|
||||
""""""
|
||||
::
|
||||
|
||||
name:cirros
|
||||
name:cirros && protected:false
|
||||
|
||||
|
||||
You may decide to only look in a certain field for a
|
||||
search term by setting a specific facet. This is accomplished by
|
||||
either selecting a facet from the drop down or by typing the facet
|
||||
manually. For example, if you are looking for an image, you
|
||||
may choose to only look at the name field by adding 'name:foo'.
|
||||
You may group facets and use logical operators.
|
||||
|
||||
Range Queries
|
||||
"""""""""""""
|
||||
::
|
||||
|
||||
size:[1 TO 1000]
|
||||
size:[1 TO *]
|
||||
size:>=1
|
||||
size:<1000
|
||||
|
||||
Date, numeric or string fields can use range queries.
|
||||
Use square brackets [min TO max] for inclusive ranges and curly
|
||||
brackets {min TO max} for exclusive ranges.
|
||||
|
||||
IP Addresses
|
||||
""""""""""""
|
||||
::
|
||||
|
||||
172.24.4.0/16
|
||||
[10.0.0.1 TO 10.0.0.4]
|
||||
|
||||
IPv4 addresses may be searched based on ranges and with CIDR notation.
|
||||
|
||||
Boosting
|
||||
""""""""
|
||||
::
|
||||
|
||||
web javascript^2 python^0.1
|
||||
|
||||
You can increase or decrease the relevance of a search
|
||||
term by boosting different terms, phrases, or groups. Boost one of
|
||||
these by adding ^n to the term, phrase, or group where n is a
|
||||
number greater than 1 to increase relevance and between 0 and 1 to
|
||||
decrease relevance. For example 'web^4 python^0.1' would find
|
||||
anything with both web and python, but would increase the
|
||||
relevance for anything with 'web' in the result and decrease the
|
||||
relevance for anything with 'python' in the result.
|
||||
|
||||
Reserved Characters
|
||||
"""""""""""""""""""
|
||||
::
|
||||
|
||||
python \(3.4\)
|
||||
|
||||
|
||||
The following characters are reserved and must be
|
||||
escaped with a leading \ (backslash)::
|
||||
|
||||
+ - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \
|
||||
|
||||
Advanced Features
|
||||
-----------------
|
||||
|
||||
CORS - Accessing Searchlight from the browser
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Searchlight can be configured to permit access directly from the browser. For
|
||||
details on this configuration, please refer to the
|
||||
`OpenStack Cloud Admin Guide`_.
|
||||
|
||||
.. _`OpenStack Cloud Admin Guide`: http://docs.openstack.org/admin-guide-cloud/cross_project_cors.html
|
@ -1,47 +0,0 @@
|
||||
..
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
|
||||
Searchlight Use Cases
|
||||
=====================
|
||||
|
||||
Below are the use cases of Searchlight:
|
||||
|
||||
* **Cloud resources lookup:** whenever the user needs to query some
|
||||
information of the desired OpenStack services (e.g. Nova, Neutron, etc.),
|
||||
instead of using the service's APIs individually, she only needs to use
|
||||
one single Searchlight web interface to perform all the queries. Currently,
|
||||
Searchlight supports indexing OpenStack services such as Cinder, Designate,
|
||||
Glance, Ironic, Neutron, Nova, and Swift. Searchlight also provides a
|
||||
context menu for quick operations on resource search results, e.g., creating
|
||||
an instance from an image.
|
||||
* **Cloud resource repository:** because Searchlight keeps records of
|
||||
real-time notification data from other OpenStack's components, it is
|
||||
aware of the committed, available cloud resources, and other operational
|
||||
events. This capacity of Searchlight can be leveraged by virtual infrastructure
|
||||
management (VIM) applications, e.g., self-healing, self-configuration, RCA, etc.
|
||||
This is a work-in-progress [#]_.
|
||||
* **Multi-cloud discovery:** Searchlight resource repositories can be connected
|
||||
forming a distributed resource discovery infrastructure across multi-cloud
|
||||
deployment, i.e., cross OpenStack domain, OpenStack-Azure, AWS, or Containers
|
||||
on OpenStack VMs. This new role of Searchlight can take advantages of the
|
||||
existing user-friendly interface for multi-cloud resource administrators [#]_
|
||||
while also providing a unified API for the automation of the resource
|
||||
management operation.
|
||||
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
.. [#] https://storyboard.openstack.org/#!/story/2004721
|
||||
.. [#] https://www.slideshare.net/vietstack/building-a-universal-search-interface-for-the-cloud
|
@ -1,2 +0,0 @@
|
||||
# Elasticsearch 5.x
|
||||
elasticsearch>=5.0.0,<6.0.0 # Apache-2.0
|
@ -1,2 +0,0 @@
|
||||
# Elasticsearch 6.x
|
||||
elasticsearch>=6.0.0,<7.0.0 # Apache-2.0
|
@ -1 +0,0 @@
|
||||
ProxyPass "/search" "unix:/var/run/uwsgi/searchlight-api-wsgi.socket" retry=0
|
@ -1,38 +0,0 @@
|
||||
# Use this pipeline for no auth - DEFAULT
|
||||
[pipeline:searchlight]
|
||||
pipeline = cors http_proxy_to_wsgi versionnegotiation unauthenticated-context rootapp
|
||||
|
||||
[pipeline:searchlight-keystone]
|
||||
pipeline = cors versionnegotiation authtoken context rootapp
|
||||
|
||||
[composite:rootapp]
|
||||
paste.composite_factory = searchlight.api:root_app_factory
|
||||
/: apiversions
|
||||
/v1: apiv1app
|
||||
|
||||
[app:apiversions]
|
||||
paste.app_factory = searchlight.api.versions:create_resource
|
||||
|
||||
[app:apiv1app]
|
||||
paste.app_factory = searchlight.api.v1.router:API.factory
|
||||
|
||||
[filter:versionnegotiation]
|
||||
paste.filter_factory = searchlight.api.middleware.version_negotiation:VersionNegotiationFilter.factory
|
||||
|
||||
[filter:unauthenticated-context]
|
||||
paste.filter_factory = searchlight.api.middleware.context:UnauthenticatedContextMiddleware.factory
|
||||
|
||||
[filter:authtoken]
|
||||
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
|
||||
delay_auth_decision = true
|
||||
|
||||
[filter:context]
|
||||
paste.filter_factory = searchlight.api.middleware.context:ContextMiddleware.factory
|
||||
|
||||
[filter:cors]
|
||||
paste.filter_factory = oslo_middleware.cors:filter_factory
|
||||
oslo_config_project = searchlight
|
||||
|
||||
[filter:http_proxy_to_wsgi]
|
||||
paste.filter_factory = oslo_middleware.http_proxy_to_wsgi:HTTPProxyToWSGI.factory
|
||||
oslo_config_project = searchlight
|
@ -1,9 +0,0 @@
|
||||
[DEFAULT]
|
||||
output_file = etc/searchlight.conf.sample
|
||||
namespace = searchlight
|
||||
namespace = oslo.policy
|
||||
namespace = keystonemiddleware.auth_token
|
||||
namespace = oslo.log
|
||||
namespace = oslo.service.service
|
||||
namespace = oslo.middleware.cors
|
||||
namespace = oslo.messaging
|
@ -1,3 +0,0 @@
|
||||
[DEFAULT]
|
||||
output_file = etc/policy.yaml.sample
|
||||
namespace = searchlight
|
@ -1,34 +0,0 @@
|
||||
# property-protections-policies.conf.sample
|
||||
#
|
||||
# This file is an example config file for when
|
||||
# property_protection_rule_format=policies is enabled.
|
||||
#
|
||||
# Specify regular expression for which properties will be protected in []
|
||||
# For each section, specify CRUD permissions. You may refer to policies defined
|
||||
# in policy.json or policy.yaml.
|
||||
# The property rules will be applied in the order specified. Once
|
||||
# a match is found the remaining property rules will not be applied.
|
||||
#
|
||||
# WARNING:
|
||||
# * If the reg ex specified below does not compile, then
|
||||
# the glance-api service fails to start. (Guide for reg ex python compiler
|
||||
# used:
|
||||
# http://docs.python.org/2/library/re.html#regular-expression-syntax)
|
||||
# * If an operation(create, read, update, delete) is not specified or misspelt
|
||||
# then the glance-api service fails to start.
|
||||
# So, remember, with GREAT POWER comes GREAT RESPONSIBILITY!
|
||||
#
|
||||
# NOTE: Only one policy can be specified per action. If multiple policies are
|
||||
# specified, then the glance-api service fails to start.
|
||||
|
||||
[^x_.*]
|
||||
create = default
|
||||
read = default
|
||||
update = default
|
||||
delete = default
|
||||
|
||||
[.*]
|
||||
create = context_is_admin
|
||||
read = context_is_admin
|
||||
update = context_is_admin
|
||||
delete = context_is_admin
|
@ -1,32 +0,0 @@
|
||||
# property-protections-roles.conf.sample
|
||||
#
|
||||
# This file is an example config file for when
|
||||
# property_protection_rule_format=roles is enabled.
|
||||
#
|
||||
# Specify regular expression for which properties will be protected in []
|
||||
# For each section, specify CRUD permissions.
|
||||
# The property rules will be applied in the order specified. Once
|
||||
# a match is found the remaining property rules will not be applied.
|
||||
#
|
||||
# WARNING:
|
||||
# * If the reg ex specified below does not compile, then
|
||||
# glance-api service will not start. (Guide for reg ex python compiler used:
|
||||
# http://docs.python.org/2/library/re.html#regular-expression-syntax)
|
||||
# * If an operation(create, read, update, delete) is not specified or misspelt
|
||||
# then the glance-api service will not start.
|
||||
# So, remember, with GREAT POWER comes GREAT RESPONSIBILITY!
|
||||
#
|
||||
# NOTE: Multiple roles can be specified for a given operation. These roles must
|
||||
# be comma separated.
|
||||
|
||||
[^x_.*]
|
||||
create = admin,member,_member_
|
||||
read = admin,member,_member_
|
||||
update = admin,member,_member_
|
||||
delete = admin,member,_member_
|
||||
|
||||
[.*]
|
||||
create = admin
|
||||
read = admin
|
||||
update = admin
|
||||
delete = admin
|
@ -1,25 +0,0 @@
|
||||
[uwsgi]
|
||||
wsgi-file = /usr/local/bin/searchlight-api-wsgi
|
||||
|
||||
chmod-socket = 666
|
||||
|
||||
socket = /var/run/uwsgi/searchlight-api-wsgi.socket
|
||||
|
||||
# Override the default size for headers from the 4k default.
|
||||
buffer-size = 65535
|
||||
|
||||
# This is running standalone
|
||||
master = true
|
||||
|
||||
enable-threads = true
|
||||
|
||||
# Tune this to your environment.
|
||||
processes = 4
|
||||
|
||||
# uwsgi recommends this to prevent thundering herd on accept.
|
||||
thunder-lock = true
|
||||
|
||||
plugins = python
|
||||
|
||||
# This ensures that file descriptors aren't shared between Searchlight processes.
|
||||
lazy-apps = true
|
Binary file not shown.
Before Width: | Height: | Size: 38 KiB |
Binary file not shown.
Before Width: | Height: | Size: 180 KiB |
Binary file not shown.
Before Width: | Height: | Size: 155 KiB |
Binary file not shown.
Before Width: | Height: | Size: 234 KiB |
BIN
images/SeaaS.png
BIN
images/SeaaS.png
Binary file not shown.
Before Width: | Height: | Size: 87 KiB |
@ -1,44 +0,0 @@
|
||||
coverage==4.5.2
|
||||
elasticsearch==2.0.0
|
||||
eventlet==0.18.2
|
||||
fixtures==3.0.0
|
||||
greenlet==0.4.10
|
||||
keystonemiddleware==4.17.0
|
||||
os-api-ref==1.4.0
|
||||
oslo.concurrency==3.26.0
|
||||
oslo.config==5.2.0
|
||||
oslo.context==2.19.2
|
||||
oslo.i18n==3.15.3
|
||||
oslo.log==3.36.0
|
||||
oslo.messaging==5.29.0
|
||||
oslo.middleware==3.31.0
|
||||
oslo.policy==1.30.0
|
||||
oslo.reports==1.18.0
|
||||
oslo.serialization==2.18.0
|
||||
oslo.service==1.33.0
|
||||
oslo.upgradecheck==0.1.0
|
||||
oslo.utils==3.33.0
|
||||
oslotest==3.2.0
|
||||
osprofiler==1.4.0
|
||||
Paste==2.0.2
|
||||
PasteDeploy==1.5.0
|
||||
pbr==2.0.0
|
||||
psutil==3.2.2
|
||||
pyOpenSSL==17.1.0
|
||||
python-cinderclient==3.3.0
|
||||
python-designateclient==2.7.0
|
||||
python-glanceclient==2.8.0
|
||||
python-ironicclient==2.3.0
|
||||
python-keystoneclient==3.8.0
|
||||
python-neutronclient==6.11.0
|
||||
python-novaclient==9.1.0
|
||||
python-swiftclient==3.2.0
|
||||
requests==2.21.0
|
||||
Routes==2.3.1
|
||||
simplejson==3.5.1
|
||||
sphinx==1.6.2
|
||||
stestr==2.0.0
|
||||
stevedore==1.20.0
|
||||
testtools==2.2.0
|
||||
WSME==0.8.0
|
||||
WebOb==1.8.4
|
27
pylintrc
27
pylintrc
@ -1,27 +0,0 @@
|
||||
[Messages Control]
|
||||
# W0511: TODOs in code comments are fine.
|
||||
# W0142: *args and **kwargs are fine.
|
||||
# W0622: Redefining id is fine.
|
||||
disable-msg=W0511,W0142,W0622
|
||||
|
||||
[Basic]
|
||||
# Variable names can be 1 to 31 characters long, with lowercase and underscores
|
||||
variable-rgx=[a-z_][a-z0-9_]{0,30}$
|
||||
|
||||
# Argument names can be 2 to 31 characters long, with lowercase and underscores
|
||||
argument-rgx=[a-z_][a-z0-9_]{1,30}$
|
||||
|
||||
# Method names should be at least 3 characters long
|
||||
# and be lowercased with underscores
|
||||
method-rgx=[a-z_][a-z0-9_]{2,50}$
|
||||
|
||||
# Module names matching nova-* are ok (files in bin/)
|
||||
module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+)|(nova-[a-z0-9_-]+))$
|
||||
|
||||
# Don't require docstrings on tests.
|
||||
no-docstring-rgx=((__.*__)|([tT]est.*)|setUp|tearDown)$
|
||||
|
||||
[Design]
|
||||
max-public-methods=100
|
||||
min-public-methods=0
|
||||
max-args=6
|
@ -1,62 +0,0 @@
|
||||
---
|
||||
prelude: |
|
||||
The initial mitaka release (0.2) did not support
|
||||
ElasticSearch 2.x. ElasticSearch made several changes
|
||||
to internal index data mappings that were not
|
||||
backwards compatible with ElasticSearch 1.x. This
|
||||
release has all the changes required for Searchlight
|
||||
to support both ElasticSearch 1.x and 2.x.
|
||||
|
||||
Searchlight now supports oslo pools instead of requiring
|
||||
a different topic for each listener. This simplifies deployment
|
||||
configuration. Previously, services would have to be specifically
|
||||
configured to send notifications to the Searchlight topic. With
|
||||
this change, Searchlight can share topics with other services
|
||||
such as Ceilometer.
|
||||
upgrade:
|
||||
- To support ElasticSearch 1.x and 2.x, several internal data
|
||||
mappings have been updated. If you previously deployed Searchlight,
|
||||
you will need to reindex your resource data using
|
||||
``searchlight-manage index sync``. You may re-index everything
|
||||
or limit it to ``OS::Nova::Server``, ``OS::Glance::Image``,
|
||||
``OS::Glance::Metadef``, ``OS::Cinder:Volume``,
|
||||
and ``OS::Cinder::Snapshot``.
|
||||
- |
|
||||
To start using a pool with a shared topic instead of separate topics:
|
||||
|
||||
* Update the respective service configuration files for searchlight
|
||||
enabled plugins to only publish to a single topic
|
||||
(e.g. set ``notification_topics = notifications``). If you have
|
||||
Ceilometer enabled, this must be the same topic which Ceilometer uses.
|
||||
You also typically must restart each service for the change to
|
||||
take effect. For example, in ``nova.conf``, ``glance-api.conf``,
|
||||
``cinder.conf``, ``neutron.conf``, and ``designate.conf``:
|
||||
|
||||
::
|
||||
|
||||
notification_topics = notifications
|
||||
|
||||
* Update ``searchlight.conf``. In the ``[resource_plugin]`` section, set
|
||||
``notifications_topic`` to match the shared topic that you set in the
|
||||
service configurations. In the ``[listener]`` sections, set
|
||||
``notifications_pool`` to your desired pool name. The default name
|
||||
starting in Newton is ``searchlight``. The ``notifications_pool`` does
|
||||
not have to match anything from other services configuration files.
|
||||
For example:
|
||||
|
||||
::
|
||||
|
||||
[resource_plugin]
|
||||
notifications_topic = notifications
|
||||
|
||||
[listener]
|
||||
notifications_pool = searchlight
|
||||
fixes:
|
||||
- Bug 1570213 Apply query to highlight query
|
||||
- Bug 1532010 Ensure consistency in mapping field types
|
||||
- Bug 1570674 Fix unicode error when booting instance from volume
|
||||
- Bug 1570199 Fix inconsistent mapping in image plugin
|
||||
- Bug 1568709 Remove port.create.end handler from nova
|
||||
- Bug 1565015 Add volume.retype event
|
||||
- Bug 1583215 Correct Cinder exchange value
|
||||
- Bug 1583215 Enable notification messaging pools
|
@ -1,4 +0,0 @@
|
||||
---
|
||||
upgrade:
|
||||
- |
|
||||
Add support for ElasticSearch 5.x
|
@ -1,6 +0,0 @@
|
||||
---
|
||||
features:
|
||||
- Added microversion support for Nova plugins, the default
|
||||
API version for Nova plugins is 2.1. User can change the
|
||||
API microversion for Nova using compute_api_version config
|
||||
option added in section service_credentials:nova.
|
@ -1,3 +0,0 @@
|
||||
---
|
||||
features:
|
||||
- Adds notification handler for nova server group.
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user