Update Docs for Mitaka Release
A number of docs required updated to reflect changes made during the Mitaka release candidate period. This captures many of them. Change-Id: I980cea6964ba1dce2ea40c9665c6938271cba6ac
This commit is contained in:
parent
d445624e3c
commit
c8bae93d67
|
@ -126,25 +126,6 @@ SWIFT_HASH=096d08da4f8d4cce3a724c5f6c18f055
|
||||||
SWIFT_REPLICAS=1
|
SWIFT_REPLICAS=1
|
||||||
SWIFT_DATA_DIR=$DEST/data/swift
|
SWIFT_DATA_DIR=$DEST/data/swift
|
||||||
|
|
||||||
## SWIFT NOTIFICATIONS ###
|
|
||||||
#Notifications must be configured properly for searchlight to process
|
|
||||||
#incremental updates. Use the following::
|
|
||||||
|
|
||||||
#Add the following new section in swift proxy conf file
|
|
||||||
#[filter:oslomiddleware]
|
|
||||||
#paste.filter_factory = swift.common.middleware.oslo_notifications:filter_factory
|
|
||||||
#publisher_id = swift.localhost
|
|
||||||
#Replace <user>,<password>,<rabbitip> and <rabbitport> for your environment values
|
|
||||||
#transport_url = rabbit://<user>:<password>@<rabbitip>:<rabbitport>/
|
|
||||||
#notification_driver = messaging
|
|
||||||
#notification_topics = searchlight_indexer
|
|
||||||
|
|
||||||
#Add oslomiddleware to pipeline:main see example below.
|
|
||||||
#[pipeline:main]
|
|
||||||
#pipeline = catch_errors gatekeeper healthcheck ... oslomiddleware proxy-logging proxy-server
|
|
||||||
#Restart swift proxy API service (s-proxy) after making changes.
|
|
||||||
|
|
||||||
|
|
||||||
### DESIGNATE ###
|
### DESIGNATE ###
|
||||||
#
|
#
|
||||||
enable_plugin designate https://git.openstack.org/openstack/designate
|
enable_plugin designate https://git.openstack.org/openstack/designate
|
||||||
|
@ -197,11 +178,11 @@ rpc_backend = 'rabbit'
|
||||||
[[post-config|$SEARCHLIGHT_CONF]]
|
[[post-config|$SEARCHLIGHT_CONF]]
|
||||||
|
|
||||||
[resource_plugin]
|
[resource_plugin]
|
||||||
index_name = searchlight
|
#resource_group_name = searchlight
|
||||||
|
|
||||||
[resource_plugin:os_nova_server]
|
[resource_plugin:os_nova_server]
|
||||||
enabled = True
|
enabled = True
|
||||||
admin_only_fields = OS-EXT-SRV*,OS-EXT-STS:vm_state
|
#admin_only_fields = OS-EXT-STS:vm_state
|
||||||
|
|
||||||
[resource_plugin:os_glance_image]
|
[resource_plugin:os_glance_image]
|
||||||
enabled = True
|
enabled = True
|
||||||
|
@ -223,18 +204,28 @@ enabled = True
|
||||||
|
|
||||||
[resource_plugin:os_neutron_net]
|
[resource_plugin:os_neutron_net]
|
||||||
enabled = True
|
enabled = True
|
||||||
admin_only_fields=admin_state_up,status
|
|
||||||
|
|
||||||
[resource_plugin:os_neutron_port]
|
[resource_plugin:os_neutron_port]
|
||||||
enabled = True
|
enabled = True
|
||||||
|
|
||||||
|
## SWIFT SETTINGS ###
|
||||||
|
# At this time we recommend that you manually enable the Searchlight plugins
|
||||||
|
# for Swift after devstack has completed stacking. Please follow the
|
||||||
|
# instructions in the docs:
|
||||||
|
#
|
||||||
|
# http://docs.openstack.org/developer/searchlight/plugins/swift.html
|
||||||
|
#
|
||||||
|
# Notifications must be configured properly for searchlight to process
|
||||||
|
# incremental updates. There is a middleware patch to provide Swift updates.
|
||||||
|
# Please see the plugin guide for swift for more information.
|
||||||
|
|
||||||
[resource_plugin:os_swift_account]
|
[resource_plugin:os_swift_account]
|
||||||
enabled = false
|
enabled = False
|
||||||
#Specify same value as in swift proxy config for reseller_prefix
|
#Specify same value as in swift proxy config for reseller_prefix
|
||||||
reseller_prefix = AUTH_
|
reseller_prefix = AUTH_
|
||||||
|
|
||||||
[resource_plugin:os_swift_container]
|
[resource_plugin:os_swift_container]
|
||||||
enabled = false
|
enabled = False
|
||||||
|
|
||||||
[resource_plugin:os_swift_object]
|
[resource_plugin:os_swift_object]
|
||||||
enabled = false
|
enabled = False
|
||||||
|
|
|
@ -109,6 +109,15 @@ underneath it.::
|
||||||
Release Notes
|
Release Notes
|
||||||
=============
|
=============
|
||||||
|
|
||||||
|
0.2.0.0 (Mitaka)
|
||||||
|
----------------
|
||||||
|
|
||||||
|
The Designate notification limitations mentioned in Liberty still apply.
|
||||||
|
|
||||||
|
You no longer need to use the --no-delete option mentioned below. Zero
|
||||||
|
downtime reindexing implemented in Mitaka handles all re-indexing
|
||||||
|
transparently.
|
||||||
|
|
||||||
0.1.0.0 (Liberty)
|
0.1.0.0 (Liberty)
|
||||||
-----------------
|
-----------------
|
||||||
|
|
||||||
|
|
|
@ -112,6 +112,22 @@ underneath it.::
|
||||||
Release Notes
|
Release Notes
|
||||||
=============
|
=============
|
||||||
|
|
||||||
|
0.2.0.0 (Mitaka)
|
||||||
|
----------------
|
||||||
|
|
||||||
|
The following fields are exposed to adminstrators only for nova instances:
|
||||||
|
* OS-EXT-SRV-ATTR:*
|
||||||
|
|
||||||
|
Additional properties can be similarly protected with the `admin_only_fields`
|
||||||
|
under each plugin's configuration section. Glob-like patterns are supported.
|
||||||
|
For instance::
|
||||||
|
|
||||||
|
[resource_plugin:os_nova_server]
|
||||||
|
admin_only_fields=OS-EXT-STS:vm_state
|
||||||
|
|
||||||
|
See: ADMIN_ONLY_FIELDS in:
|
||||||
|
* searchlight/elasticsearch/plugins/nova/servers.py
|
||||||
|
|
||||||
0.1.0.0 (Liberty)
|
0.1.0.0 (Liberty)
|
||||||
-----------------
|
-----------------
|
||||||
|
|
||||||
|
|
|
@ -17,7 +17,7 @@
|
||||||
Swift Plugin Guide
|
Swift Plugin Guide
|
||||||
******************
|
******************
|
||||||
|
|
||||||
WARNING: Swift plugin is currently EXPERIMENTAL as notifications aren't
|
WARNING: The Swift plugin is currently EXPERIMENTAL as notifications aren't
|
||||||
fully supported. See below on enabling notifications.
|
fully supported. See below on enabling notifications.
|
||||||
|
|
||||||
Integration is provided via a plugin. There are multiple configuration
|
Integration is provided via a plugin. There are multiple configuration
|
||||||
|
@ -25,6 +25,56 @@ settings required for proper indexing and incremental updates. Some of the
|
||||||
settings are specified in Searchlight configuration files. Others are
|
settings are specified in Searchlight configuration files. Others are
|
||||||
provided in other service configuration files.
|
provided in other service configuration files.
|
||||||
|
|
||||||
|
Swift Configuration
|
||||||
|
====================
|
||||||
|
|
||||||
|
The Swift service currently doesn't send notifications.
|
||||||
|
Apply this patch https://review.openstack.org/#/c/249471
|
||||||
|
for adding notification middleware to swift. Please restart swift-api.
|
||||||
|
|
||||||
|
reseller_admin_role
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
Users with the Keystone role defined in reseller_admin_role (ResellerAdmin by default)
|
||||||
|
can operate on any account. The auth system sets the request environ reseller_request
|
||||||
|
to True if a request is coming from a user with this role.
|
||||||
|
|
||||||
|
Searchlight needs this role for its service user to access all the swift accounts
|
||||||
|
for initial indexing. The searchlight user and sevice project being referred here is the
|
||||||
|
one defined in service_credentials section of searchlight conf file. This
|
||||||
|
must be done prior to running `searchlight-manage index sync` when any of the
|
||||||
|
Swift plugins are enabled.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
openstack role add --user searchlight --project service ResellerAdmin
|
||||||
|
|
||||||
|
|
||||||
|
proxy-server.conf
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
Notifications must be configured properly for searchlight to process
|
||||||
|
incremental updates. Use the following::
|
||||||
|
|
||||||
|
# Add the following new section
|
||||||
|
[filter:oslomiddleware]
|
||||||
|
paste.filter_factory = swift.common.middleware.oslo_notifications:filter_factory
|
||||||
|
publisher_id = swift.localhost
|
||||||
|
#Replace <user>,<password>,<rabbitip> and <rabbitport> for your environment values
|
||||||
|
transport_url = rabbit://<user>:<password>@<rabbitip>:<rabbitport>/
|
||||||
|
notification_driver = messaging
|
||||||
|
notification_topics = searchlight_indexer
|
||||||
|
|
||||||
|
# Add oslomiddleware to pipeline:main
|
||||||
|
# see example below.
|
||||||
|
[pipeline:main]
|
||||||
|
pipeline = catch_errors gatekeeper ...<other>... oslomiddleware proxy-logging proxy-server
|
||||||
|
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Restart swift proxy API service (s-proxy) after making changes.
|
||||||
|
|
||||||
Searchlight Configuration
|
Searchlight Configuration
|
||||||
=========================
|
=========================
|
||||||
|
|
||||||
|
@ -37,7 +87,11 @@ general configuration information, and an example complete configuration.
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
Unless you are changing to a non-default value, you do not need to
|
Unless you are changing to a non-default value, you do not need to
|
||||||
specify any of the following configuration options.
|
specify any of the following configuration options. After enabling or
|
||||||
|
disabling a plugin you do need to restart the searchlight services
|
||||||
|
(`searchlight-api` and `searchlight-listener`).
|
||||||
|
After enabling a Swift plugin, you will also need to run the sync job:
|
||||||
|
`searchlight-manage index sync --type OS::Swift::Account`
|
||||||
|
|
||||||
searchlight.conf
|
searchlight.conf
|
||||||
----------------
|
----------------
|
||||||
|
@ -48,13 +102,13 @@ Plugin: OS::Swift::Account
|
||||||
|
|
||||||
[resource_plugin:os_swift_account]
|
[resource_plugin:os_swift_account]
|
||||||
enabled = true
|
enabled = true
|
||||||
index_name = searchlight
|
# Specify same value as in swift proxy-server.conf for reseller_prefix
|
||||||
#Specify same value as in swift proxy config for reseller_prefix
|
|
||||||
reseller_prefix = AUTH_
|
reseller_prefix = AUTH_
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
os_swift_account is disabled by default. You need to explicitly set enabled = True as shown above
|
`os_swift_account` is disabled by default. You need to explicitly
|
||||||
|
set enabled = True as shown above.
|
||||||
|
|
||||||
Plugin: OS::Swift::Container
|
Plugin: OS::Swift::Container
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
@ -62,11 +116,11 @@ Plugin: OS::Swift::Container
|
||||||
|
|
||||||
[resource_plugin:os_swift_container]
|
[resource_plugin:os_swift_container]
|
||||||
enabled = true
|
enabled = true
|
||||||
index_name = searchlight
|
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
os_swift_container is disabled by default. You need to explicitly set enabled = True as shown above
|
`os_swift_container` is disabled by default. You need to explicitly
|
||||||
|
set enabled = True as shown above.
|
||||||
|
|
||||||
Plugin: OS::Swift::Object
|
Plugin: OS::Swift::Object
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
@ -74,69 +128,19 @@ Plugin: OS::Swift::Object
|
||||||
|
|
||||||
[resource_plugin:os_swift_object]
|
[resource_plugin:os_swift_object]
|
||||||
enabled = true
|
enabled = true
|
||||||
index_name = searchlight
|
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
os_swift_object is disabled by default. You need to explicitly set enabled = True as shown above
|
`os_swift_object` is disabled by default. You need to explicitly
|
||||||
|
set enabled = True as shown above.
|
||||||
|
|
||||||
Swift Configuration
|
|
||||||
====================
|
|
||||||
|
|
||||||
The Swift service currently doesn't send notifications.
|
|
||||||
Apply this patch https://review.openstack.org/#/c/249471
|
|
||||||
for adding notification middleware to swift.
|
|
||||||
|
|
||||||
reseller_admin_role
|
|
||||||
-------------------
|
|
||||||
|
|
||||||
Users with the Keystone role defined in reseller_admin_role (ResellerAdmin by default)
|
|
||||||
can operate on any account. The auth system sets the request environ reseller_request
|
|
||||||
to True if a request is coming from a user with this role.
|
|
||||||
|
|
||||||
Searchlight needs this role for its service user to access all the swift accounts
|
|
||||||
for initial indexing. The searchlight user and sevice project being referred here is the
|
|
||||||
one defined in service_credentials section of searchlight conf file.
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
openstack role add --user searchlight --project service ResellerAdmin
|
|
||||||
|
|
||||||
|
|
||||||
proxy-server.conf
|
|
||||||
-----------------
|
|
||||||
|
|
||||||
Notifications must be configured properly for searchlight to process
|
|
||||||
incremental updates. Use the following::
|
|
||||||
|
|
||||||
#Add the following new section
|
|
||||||
[filter:oslomiddleware]
|
|
||||||
paste.filter_factory = swift.common.middleware.oslo_notifications:filter_factory
|
|
||||||
publisher_id = swift.localhost
|
|
||||||
#Replace <user>,<password>,<rabbitip> and <rabbitport> for your environment values
|
|
||||||
transport_url = rabbit://<user>:<password>@<rabbitip>:<rabbitport>/
|
|
||||||
notification_driver = messaging
|
|
||||||
notification_topics = searchlight_indexer
|
|
||||||
|
|
||||||
#Add oslomiddleware to pipeline:main see example below.
|
|
||||||
[pipeline:main]
|
|
||||||
pipeline = catch_errors gatekeeper healthcheck ... oslomiddleware proxy-logging proxy-server
|
|
||||||
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
Restart swift proxy API service (s-proxy) after making changes.
|
|
||||||
|
|
||||||
local.conf (devstack)
|
local.conf (devstack)
|
||||||
---------------------
|
---------------------
|
||||||
|
|
||||||
The settings above may be automatically configured by ``stack.sh``
|
At this time we recommend that you manually enable the Searchlight plugins
|
||||||
by adding them to the following post config section in devstack.
|
and middleware for Swift after devstack has completed stacking. Please
|
||||||
Just place the following in local.conf and copy the above settings
|
follow the instructions above.
|
||||||
underneath it.::
|
|
||||||
|
|
||||||
[[post-config|$SWIFT_PROXY_CONF]]
|
|
||||||
[DEFAULT]
|
|
||||||
|
|
||||||
Release Notes
|
Release Notes
|
||||||
=============
|
=============
|
||||||
|
@ -144,32 +148,38 @@ Release Notes
|
||||||
0.2.0.0 (Mitaka)
|
0.2.0.0 (Mitaka)
|
||||||
----------------
|
----------------
|
||||||
|
|
||||||
|
Large scale swift cluster support is targeted at a future release, but
|
||||||
|
we encourage trial deployments to help us address issues as soon as possible.
|
||||||
|
|
||||||
Swift did not generate notifications for account/container/object CRUD
|
Swift did not generate notifications for account/container/object CRUD
|
||||||
|
during the Mitaka release. This means that search results will not include
|
||||||
|
incremental updates after the initial indexing. However, there is a patch
|
||||||
|
available to enable notifications via oslo messaging for the Mitaka release.
|
||||||
|
|
||||||
This means that search results will not include incremental updates after
|
* https://review.openstack.org/#/c/249471
|
||||||
the initial indexing.
|
|
||||||
|
|
||||||
The patch (https://review.openstack.org/#/c/249471) implements this feature.
|
For devstack, the easiest way to test is::
|
||||||
|
|
||||||
For devstack, the easiest way to test is
|
cd /opt/stack/swift
|
||||||
cd /opt/stack/swift
|
git review -x 249471
|
||||||
git review -x 249471
|
<restart swift api>
|
||||||
|
|
||||||
Searchlight developers/installers should apply the above patch in Swift when
|
Searchlight developers/installers should apply the above patch in Swift when
|
||||||
using Searchlight with the Swift Mitaka release.
|
using Searchlight with the Swift Mitaka release. We are working with the
|
||||||
|
Swift team to create a supported incremental indexing methodology for future
|
||||||
|
releases.
|
||||||
|
|
||||||
Alternatively, you may set up a cron job to re-index swift
|
Alternatively, you may set up a cron job to re-index swift
|
||||||
account/container/objects periodically to get updated information. The
|
account/container/objects periodically to get updated information. The
|
||||||
recommendation is to use the notifications.
|
recommendation is to use the notifications, because a full re-indexing will
|
||||||
|
not be performant in large installations.
|
||||||
|
::
|
||||||
|
|
||||||
You should use the ``--no-delete`` option to prevent the index from
|
searchlight-manage index sync --type OS::Swift::Account
|
||||||
temporarily not containing any data (which otherwise would happen with a full
|
|
||||||
bulk indexing job)::
|
|
||||||
|
|
||||||
searchlight-manage index sync --type OS::Swift::Account --force --no-delete
|
The Searchlight Swift plugin resource types follow the hierarchy similar to
|
||||||
|
|
||||||
Searchlight swift plugin resource types follow the hierarchy similar to
|
|
||||||
Swift concepts
|
Swift concepts
|
||||||
|
::
|
||||||
|
|
||||||
OS::Swift:Acccount(Parent)
|
OS::Swift:Acccount(Parent)
|
||||||
-> OS:Swift::Container(Child)
|
-> OS:Swift::Container(Child)
|
||||||
|
@ -179,3 +189,13 @@ which means indexing is initiated by specifying only the top parent
|
||||||
(OS::Swift::Account) and that will in-turn index all the child
|
(OS::Swift::Account) and that will in-turn index all the child
|
||||||
plugins(Container and Object)
|
plugins(Container and Object)
|
||||||
|
|
||||||
|
Searchlight is adding indexing isolation in the Newton release via a concept
|
||||||
|
called resource group isolation. This will better support re-indexing
|
||||||
|
scalability.
|
||||||
|
|
||||||
|
Additional properties can be similarly protected with the `admin_only_fields`
|
||||||
|
under each plugin's configuration section. Glob-like patterns are supported.
|
||||||
|
For instance::
|
||||||
|
|
||||||
|
[resource_plugin:os_swift_object]
|
||||||
|
admin_only_fields=x-meta-admin*
|
||||||
|
|
Loading…
Reference in New Issue