Browse Source

Fix admin docs deploying under HTTPD

During Pike cycle there was efforts to deploy services under
Apache HTTPD. It became clear very quickly that Glance did not
operate properly when deployed as bare wsgi app under web
server but admin documentation was merged that indicated that
being somehow the preferred method. It was added to the doc
that in Pike release there was issues in these models.

There was never interest nor resources to fix the underlying
issues but the doc stayed in place indicating that those
issues could be only Pike related when they in matter of
fact has got even worse over the time. Due to the fact that
Glance is even more relying on eventlet than back then it's
time to clarify the docs and make clear it's not adviced
deployment model, it won't work and there is no itention
to change that.

Change-Id: I93dc7c999ff7a180e6b3ff760fa65328b9a883f3
Closes-Bug: #1887994
(cherry picked from commit 785eefcf78)
Erno Kuvaja 3 weeks ago
2 changed files with 23 additions and 93 deletions
  1. +17
  2. +6

+ 17
- 93
doc/source/admin/apache-httpd.rst View File

@@ -2,10 +2,18 @@
Running Glance in HTTPD

In short Glance will not operate properly if tried to be ran without eventlet
and introducing another web server into the mix does not make it any better.
This exercise failed without ever having proper interest or resources to fix
the underlying issues.

None of the models deploying Glance as bare wsgi app under some httpd are
currently adviced.

Since the Pike release Glance has packaged a wsgi script entrypoint that
enables you to run it with a real web server like Apache HTTPD or nginx. To
deploy this there are several patterns. This doc shows two common ways of
deploying Glance with Apache HTTPD.
deploy this there are several patterns, which all fail different ways. This doc
mentions three common ways of trying to deploy Glance with Apache HTTPD.

.. warning::
As pointed out in the Pike and Queens release notes (see the "Known Issues"
@@ -19,103 +27,19 @@ deploying Glance with Apache HTTPD.
uWSGI Server HTTP Mode

This is the current recommended way to deploy Glance with Apache HTTP and it is
how we deploy Glance for testing every proposed commit to OpenStack. In this
deployment method we use the uWSGI server as a web server bound to a random
local port. Then we configure apache using mod_proxy to forward all incoming
requests on the specified endpoint to that local webserver. This has the
advantage of letting apache manage all inbound http connections, but letting
uWSGI manage running the python code. It also means when we make changes to
Glance code or configuration we don't need to restart all of apache (which may
be running other services too) and just need to restart the local uWSGI daemon.

The httpd/ directory contains sample files for configuring HTTPD to run Glance
under the uWSGI server in this configuration. To use the sample configs simply
copy `httpd/uwsgi-glance-api.conf` to the appropriate location for your Apache
server. On Debian/Ubuntu systems it is::


On Red Hat based systems it is::


Enable mod_proxy by running ``sudo a2enmod proxy``

Then on Ubuntu/Debian systems enable the site by creating a symlink from the
file in ``sites-available`` to ``sites-enabled``. (This is not required on Red
Hat based systems)::

ln -s /etc/apache2/sites-available/uwsgi-glance-api.conf /etc/apache2/sites-enabled
This has never worked properly nor it has been of any development focus.

Start or restart HTTPD to pick up the new configuration.

.. NOTE::

Be careful when setting up other proxies/endpoints in the same VirtualHost
on Apache HTTPD using. If any are using ``SetEnv proxy-sendcl 1`` then
Apache HTTPD will buffer the incoming request to local disk before sending
it to glance. This will likely cause problems when running in this
configuration and is not necessary. (However, it is necessary if using
mod_proxy_uwsgi.) For more details, see the section on
:ref:`mod_proxy_uwsgi` below.

Now we need to configure and start the uWSGi service. Copy the
`httpd/glance-api-uwsgi.ini` file to `/etc/glance`. Update the file to match
your system configuration (for example, you'll want to set the number of
processes and threads).

Install the uWSGI server and start the glance-api server using uWSGI::

sudo pip install uwsgi
uwsgi --ini /etc/glance/glance-api-uwsgi.ini

.. NOTE::

In the sample configs port 60999 is used, but this doesn't matter and is
just a randomly selected number. This is not a contract on the port used
for the local uwsgi daemon.
The clearest we can say is just don't do it.

.. _mod_proxy_uwsgi:



Running Glance under HTTPD in this configuration will only work on Python 2
if you use ``Transfer-Encoding: chunked``. Also if running with Python 2
Apache will be buffering the chunked encoding before passing the request
on to uWSGI. See bug:

Instead of running uWSGI as a webserver listening on a local port and then
having Apache HTTP proxy all the incoming requests with mod_proxy. The
normally recommended way of deploying the uWSGI server with Apache HTTPD is to
use mod_proxy_uwsgi and set up a local socket file for uWSGI to listen on.
Apache will send the requests using the uwsgi protocol over this local socket
file. However, there are issues with doing this and using chunked-encoding, so
this is not recommended for use with Glance.

You can work around these issues by configuring your Apache proxy to buffer the
chunked data and send the full content length to the uWSGI server. You do this
by adding::

SetEnv proxy-sendcl 1

to the apache config file using mod_proxy_uwsgi. For more details on using
mod_proxy_uwsgi see the official docs:

There are some additional considerations when doing this though. Having Apache
locally buffer the chunked data to disk before passing it to uWSGI means you'll
need to have sufficient disk space in /tmp (or whatever you set TMPDIR to) to
store all the disk files. The other aspect to consider is that this buffering
can take some time to write the images to disk. To prevent random failures
you'll likely have to increase timeout values in the uWSGI configuration file
to ensure uWSGI will wait long enough for this to happen. (Depending on the
uploaded image file sizes it may be necessary to set the timeouts to multiple
This has not been doable since Ussuri as we only support Python 3.

In theory the same applies as mod_wsgi but even without chunked encoding the
code is still broken under uwsgi.

@@ -152,7 +76,7 @@ Glossary

An Apache 2 HTTP server module that supports the Python WSGI
specification. (It is not recommended for use with Glance.)

+ 6
- 0
releasenotes/notes/fix_httpd_docs-3efff0395f96a94d.yaml View File

@@ -0,0 +1,6 @@
- |
Bug 1887994_: Mixed message in admin docs to deploy under httpd

.. _1887994: