8458 Commits

Author SHA1 Message Date
Sean Dague
b2bfe5617a Turn off use_journal because of eventlet concerns
use_journal uses the systemd native path for logging, however there
are concerns that this might be negatively interacting with
eventlet. To be on the safe side fall back to stdout.

This introduces a USE_JOURNAL option which will let folks turn this
back on for testing.

This also adjusts the debug lines. When using the journal the pid
reported by systemd is correct. When using stdout, it will be the
parent process id, so we need to keep it to see which child each thing
is coming from.

Change-Id: Id7891c532bf99c099252e82d511a37a49506fea9
2017-05-03 18:35:54 +00:00
Sean Dague
8b8441f3be Default developer use case to systemd
This moves the developer use case over to systemd, and updates all the
relevant docs to discuss the systemd workflow instead of screen. It
does so by defaulting USE_SCREEN=False, so will not impact people that
set it explicitly.

Change-Id: I6d664612bc2b850eb7f56852afbc841867223ab7
2017-05-02 06:44:22 -04:00
Sean Dague
803acffcf9 Make ./clean.sh work in more situations
When transitioning between different wsgi modes, or service modes, we
should really safely stop and cleanup things that are started in any
service mode, which makes it easier to ensure that we don't leave
things around from past runs.

Change-Id: I33acbee39e1a2da2bfd79a5dd54b84a12a778be1
2017-05-02 06:20:22 -04:00
Jenkins
7fe9e784c6 Merge "try to use unversioned keystone endpoints everywhere" 2017-05-01 23:05:46 +00:00
Jenkins
02e01c1c18 Merge "Try to remove /identity_admin" 2017-05-01 23:05:39 +00:00
Jenkins
bc2af3c6b5 Merge "remove auth_uri from keystonemiddleware config" 2017-05-01 20:38:00 +00:00
Jenkins
7f8df450db Merge "make USE_SCREEN=False imply USE_SYSTEMD=True" 2017-05-01 19:23:10 +00:00
Jenkins
e44c564112 Merge "Differentiate between DEFAULT_IMAGE_NAME and filename" 2017-05-01 19:18:14 +00:00
Sean Dague
c13b8a1f33 try to use unversioned keystone endpoints everywhere
Change-Id: Iad2a3654d8ba181a7ad452d8aba872a8313d4ece
2017-05-01 09:12:20 -04:00
Sean Dague
bb4431126b Try to remove /identity_admin
We should be able to operate without the identity admin endpoint,
given that in v3 it's all the same. This floats that out there to see
if we can or not.

Change-Id: Ic233f6b43dd1e3cfdadff0f18aba4ea78825a996
2017-05-01 09:12:18 -04:00
Sean Dague
38d4782c9f remove auth_uri from keystonemiddleware config
auth_uri is not a keystonemiddleware option, and it's use in config
files is confusing at best. Remove it for clarity.

Change-Id: Ie3a9ab30d81809363444d5f3b41588b3889dc185
2017-05-01 09:09:35 -04:00
Matthew Treinish
fa898f5174 Add external lock dir to glance api config
As we move to enabling glance-api to use a wsgi script that might be run
as multiple processes, there are a couple places where external
synchronization is necessary. To use this we need to set the lock_path
config option from oslo.concurrency so external locks will work.

Change-Id: I9a66a8636d12037ff9aa4fb73cc3f9b9343dd7e9
2017-04-28 10:29:38 -04:00
Sean Dague
c006bbdeb2 make USE_SCREEN=False imply USE_SYSTEMD=True
In order to start making the transition in the gate make
USE_SCREEN=False also mean USE_SYSTEMD=True. We'll never actually
declare USE_SYSTEMD=True in the gate (as that doesn't exist for stable
branches), but this will let us roll over the existing transition.

We also have to install systemd-python 234 because we are recording
exception info in the journal, and all versions before that had a bug
in processing that.

Remove the somewhat pointless screen following journalctl commands. We
really don't want or need those, and they tend to build up over time.

Depends-On: I24513f5cbac2c34cf0130bf812ff2df6ad76657c

Change-Id: I6af6d1857effaf662a9d72bd394864934eacbe70
2017-04-28 13:21:22 +00:00
Jenkins
0c0232f508 Merge "Set OS_AUTH_TYPE to password" 2017-04-28 04:00:20 +00:00
Jenkins
a292c5068c Merge "Always restart apache" 2017-04-27 08:35:01 +00:00
Matthew Treinish
befe0925e7
Differentiate between DEFAULT_IMAGE_NAME and filename
The DEFAULT_IMAGE_NAME variable is used to reference the name of the
default image in glance after it has been uploaded by devstack. It is
used both inside and outside of devstack for that purpose. However, when
configuring tempest there are some tests which also do image uploads and
need a filename for specifying which file they should upload into glance
for testing purposes. Previously we were just using DEFAULT_IMAGE_NAME
for both purposes, but this causes a conflict if the name of the image
we upload into glance does not have a file extension. So instead of
conflating the things this commit differentiates between them and adds a
new DEFAULT_IMAGE_FILE_NAME variable to use for this purpose.

Change-Id: Icf74badcf2093d8c75db538232b10b3ac7b86eb8
2017-04-26 19:24:12 -04:00
Jenkins
a8204752e3 Merge "neutron-legacy: Defer service_plugins configuration" 2017-04-26 21:22:09 +00:00
Ian Wienand
f6a2d2cd4e Always restart apache
As described in [1], it seems that mod_wsgi is not "graceful" reload
safe.  Upon re-init, it can end up in a segfault loop.

The "reload" (not *restart*) after setting up uwsgi was added with
I1d89be1f1b36f26eaf543b99bde6fdc5701474fe but not causing an issue
until uwsgi was enabled.

We do not notice in the gate, because the TLS setup ends up doing a
restart after this setup.  In the period between the
write_uwsgi_config and that restart, Apache is sitting in a segfault
loop, but we never noticed because we don't try talking to it.  Other
jobs that don't do any further apache configuration have started
failing, however.

Looking at the original comments around "reload_apache_server" I'm not
sure if it is still necessary.  [2] shows it is not used outside these
two calls.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1445540
[2] http://codesearch.openstack.org/?q=reload_apache_server&i=nope&files=&repos=

Closes-Bug: #1686210
Change-Id: I5234bae0595efdcd30305a32bf9c121072a3625e
2017-04-26 11:09:59 +10:00
Jenkins
2eb322ab2e Merge "Set fixed-key key manager" 2017-04-25 23:42:18 +00:00
Jenkins
bc8db65da2 Merge "Define a new function for notifications URL" 2017-04-25 17:47:35 +00:00
Thomas Herve
26e431dbd7 Define a new function for notifications URL
This defines a new function get_notification_url, which returns the URL
of RabbitMQ when you want connect to it, and uses in
ceilometermiddleware. This fixes an issue when we try to use AMQP for
RPC, but not for notifications.

Change-Id: I14450b2440806a17a90e5ddefc243868fdbe4f2c
2017-04-25 17:01:21 +02:00
Jenkins
aa444d0d83 Merge "Create custom dir for uwsgi domain sockets" 2017-04-21 03:29:15 +00:00
rabi
aa26baacb8 Create custom dir for uwsgi domain sockets
On Centos, apache has a private view of /tmp and thus can't see this
socket, causing keystone to fail. This happened after
I46294fb24e3c23fa19fcfd7d6c9ee8a932354702.

Move it to /var/run.

Closes-Bug: #1684360
Change-Id: I47f091656802719c259752454ec88bf50760b967
2017-04-21 10:42:27 +10:00
Jenkins
d8863f6c98 Merge "Increase rsyslog buffer sizes." 2017-04-20 20:03:44 +00:00
Sean Dague
1b245cef7f Cleanup duplicate get_or_add_user_domain_role
It turns out that we ended up with duplicate versions of this function
merging on top of each other within 3 days, and gerrit didn't catch
it. Boo gerrit. Boo bash.

Change-Id: Ic6aa2f9bafdec906de2bc51d5929beeec48a6a40
2017-04-20 00:19:20 +00:00
Paul Belanger
37c7843aad
Increase rsyslog buffer sizes.
Swift proxy logs to syslog during the devstack-gate tempest runs. To
better capture the swift logs increase the rsyslog buffer size to 6k
bytes allowing for longer messages like tracebacks.

This was setup by openstack-infra previous during our diskimage
builds.

  I03e42964e14d9f930c07ed047851bdf775639c59

Change-Id: Iaa232335865410600c93f47d4777ed4f1bce08e2
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-04-19 13:20:47 -04:00
Jenkins
e2fb00ebb7 Merge "cinder: wait for cinder-api for wsgi too" 2017-04-19 15:01:07 +00:00
Jenkins
d4df865c07 Merge "Adding placement-client to compute node local.conf for multinode setup " 2017-04-19 12:52:15 +00:00
Jenkins
23756fbef5 Merge "Updated from generate-devstack-plugins-list" 2017-04-19 12:37:22 +00:00
Jenkins
c5a2f1b1f3 Merge "Add cursive to LIBS_FROM_GIT" 2017-04-19 12:37:15 +00:00
Jenkins
0cfc523ff7 Merge "Add castellan to LIBS_FROM_GIT" 2017-04-19 12:33:06 +00:00
Jenkins
f6c86749fd Merge "fix typo" 2017-04-19 12:32:59 +00:00
Jenkins
041a5d9d83 Merge "Send useful auth_port and auth_prefix to swift's test.conf" 2017-04-19 12:30:19 +00:00
OpenStack Proposal Bot
09a08aacf4 Updated from generate-devstack-plugins-list
Change-Id: If6c07fd6b56d776a5548564b72e637f6bd3dfbfc
2017-04-19 09:24:43 +00:00
Jenkins
4b9022ddd7 Merge "Make a2dissite fail softly if the site is not enabled" 2017-04-19 06:34:20 +00:00
Tim Burke
f43ea47766 Send useful auth_port and auth_prefix to swift's test.conf
Until we can test with a version of swiftclient that knows how to eat
auth_uri, swift still needs a working gate.

Change-Id: I09f9ad5c87b542df962a79898e06fbf1e968b1e3
Related-Change: I46294fb24e3c23fa19fcfd7d6c9ee8a932354702
Related-Change: Ie427f3b0b9eb834ff940fa5d52444a5a6cdcab15
2017-04-18 21:58:10 -07:00
Jenkins
cf2846ab64 Merge "Add new configuration option for LM+grenade job" 2017-04-18 21:29:16 +00:00
Jenkins
227661e881 Merge "Remove a TODO that no longer applies" 2017-04-18 18:39:55 +00:00
Jenkins
487777f5c7 Merge "Enable EPEL mirror by default" 2017-04-18 18:14:17 +00:00
Jenkins
ee49497205 Merge "XenAPI: Remove final references to Integration bridge" 2017-04-18 18:08:09 +00:00
Chris Dent
2fcdaac56e Make a2dissite fail softly if the site is not enabled
a2dissite will return a non-zero error code if the site that is being
disabled is not currently enabled (that is, if the conf file for it does
not exist). This can happen during development if you've been messing
with files by hand. Rather than exploding out of a ./stack.sh, accept
the missing file as meaning "it's disabled" and carry one. The rpm
version of disable, which does not use a2dissite, does this already.

Change-Id: Ie5dfd42efdff4bdba5ffaa765af000dd8e1d596e
2017-04-18 16:54:12 +01:00
Chris Dent
e0be9e3a2e Remove a TODO that no longer applies
The removed TODO was talking about USE_SYSTEMD, not WSGI_MODE.

WSGI_MODE makes sense, so the TODO has been done.

Change-Id: Ib574ef123ea4c82d4d88012c990cd1ad660d7879
2017-04-18 16:52:25 +01:00
Prabhuraj Kamaraj
921da2654b Adding placement-client to compute node local.conf for multinode setup
Change-Id: Ie98f908d5a932da259ae13934af45d535fcffb82
Closes-Bug: #1682362
2017-04-18 05:12:52 +00:00
Sean Dague
6ed53156b6 Convert keystone to use uwsgi with the proxy
This makes keystone use the proxy uwsgi module when running in uwsgi
mode. It also introduces a new stackrc variable which is WSGI_MODE
that we can use to control the conditionals in services that current
work with mod_wsgi.

Also update retry timeouts on proxy pass so that workers don't disable
their connections during polling for initial activity.

Change-Id: I46294fb24e3c23fa19fcfd7d6c9ee8a932354702
2017-04-17 16:27:35 -04:00
Sean Dague
64ffff9b7d Convert placement to new uwsgi mode
This converts the placement API to use the new WSGI_MODE variable
(which is not introduced until the next changeset). We do this so that
placement and keystone patches can be reviewed independently, but
there are some hidden coupling of mod_wsgi setup which happens only in
keystone, so if we do keystone first, it breaks placement.

Change-Id: Id5b2c67701bcc7b12c8e3764c7199d10f85df80f
2017-04-17 14:51:44 -04:00
Sean Dague
604e598e2a Install and enable apache uwsgi proxy
The uwsgi proxy version that comes with Ubuntu xenial is too old, so
we have to build it from source. This is a temporary solution until
the next LTS.

This lays the ground work for using it in keystone.

Change-Id: I00fb1759e6988c7df0ce0f3df5ff1ce9fd7cd381
2017-04-17 14:51:44 -04:00
Sean Dague
a1446b960f always retry proxy errors
When an apache worker gets a proxy error, it will not retry talking to
the backend server until the retry timeout expires. We bring up the
proxy server *before* the backend server, and poll it. If we are
running a small number of workers, there is a likely chance that we're
going to hit one that errored before the backend was up, thus failing
for now real reason.

Set this to 0 instead to mean always retry failed connections.

Change-Id: I9e584f087bd375f71ddf0c70f83205c425094a17
Ref: https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass
2017-04-17 14:31:21 -04:00
Sean Dague
f3b2f4c853 Remove USE_SSL support
tls-proxy is the way we're now doing a standard install using https
between services. There is a lot more work to make services directly
handle https, and having python daemons do that directly is a bit of
an anti pattern. Nothing currently tests this in project-config from
my recent grepping, so in the interest of long term maintenance,
delete it all.

Change-Id: I910df4ceab6f24f3d9c484e0433c93b06f17d6e1
2017-04-17 07:27:32 -04:00
Sean Dague
2f8c88e053 Factor out code to write uwsgi config files
Instead of this code all existing in keystone inline, factor out into
a dedicated set of functions, and make keystone use this. This drops
uwsgi supporting https directly, but that's not going to be a
supported model going forward once we get to proxy only anyway.

Change-Id: I1d89be1f1b36f26eaf543b99bde6fdc5701474fe
2017-04-14 15:41:02 -04:00
Sean Dague
2b85cf0f06 Just use normal restart for apache
We're now in a systemd world where systemd is managing the restart
effectively, there is no reason to be tricksy with apache now that
we're not working around weird upstartd issues.

Change-Id: Ifadfd504eb10a90db5177ea9180b9cd8331a2948
2017-04-14 15:41:02 -04:00