1525 Commits

Author SHA1 Message Date
Ian Wienand
ae9c6ab759 Use "pip list" in check_libs_from_git
As described in the change, "pip freeze" has issues with the way
zuulv3 clones repos without a remote.  This is an attempt to use "pip
list" to check for local install

Change-Id: I33d25f86b6afcadb4b190a0f6c53311111c64521
2017-09-29 12:55:45 +10:00
Ian Wienand
5b419ffb1f Don't check_libs_from_git for now
"pip freeze" reports an error when checking because the repos don't
have a remote under zuulv3

---
Error when trying to get requirement for VCS system Command "git
config --get-regexp remote\..*\.url" failed with error code 1 in
/opt/stack/new/keystone, falling back to uneditable format

Could not determine repository location of /opt/stack/new/keystone
Complete output from command git config --get-regexp remote\..*\.url:

---

This means this check fails.  I think we can fix this by looking at
"pip list" which I will propose in a follow-on, but this fixes the
immediate breaking issue.

Depends-On: Ib12ddf768ee20fd7614622179f6842f5d57864ff
Change-Id: I21ff749ab3e7911fa074e6d53056768f42f8aa57
2017-09-29 12:55:36 +10:00
Jenkins
0a97e56186 Merge "Fix comment wording nit" 2017-09-24 10:32:15 +00:00
Jenkins
7b66a11624 Merge "Check that generate-subunit exists before using it" 2017-09-23 15:26:40 +00:00
Monty Taylor
03ae3c485e
Fix comment wording nit
Change-Id: I558522121099f14d380c163a2c397b34335b03df
2017-09-22 16:25:38 -05:00
Monty Taylor
cbd5f4e0ad
Check that generate-subunit exists before using it
If stack.sh fails before os-testr is installed, the generate-subunit
command won't exist.

Change-Id: I7998ed81e419e25d183e5a780df0b4459cca237c
2017-09-10 15:01:12 -06:00
Jenkins
1ad91a7d4b Merge "Remove screen support from devstack completely" 2017-09-07 13:38:03 +00:00
Jenkins
a7623994bc Merge "Wait for compute service to check in" 2017-09-06 18:16:21 +00:00
Sean Dague
cdba1b3716 Remove screen support from devstack completely
This tears out the alternative path of using screen, so that we only
use systemd enabled paths. This simplifies the number of ways that
devstack can be run, and provides a much more reliable process
launcher than the screen based approach.

Change-Id: I8c27182f60b0f5310b3a8bf5feb02beb7ffbb26a
2017-09-01 15:08:17 -04:00
Davanum Srinivas
aa33c878d8 Fix Cleanups for keystone/placement/nova
* Check KEYSTONE_DEPLOY flag and cleanup appropriately
* When we stop process, we should not wipe uwsgi config we should
  remove files only on cleanup
* We should not call cleanup *BEFORE* configure, we are just wiping
  out the uwsgi ini files
* cleanup_placement should be called from clean.sh

Change-Id: I066f5f87ff22d7da2e3814f8c2de75f2af625d2b
2017-08-17 10:40:38 -04:00
zhangbailin
32608da2c4 Modify some spelling mistakes
There are some comment errors, it's modify 'Captial' to 'Capital' in
keystone file, and modify 'possition' to 'position' in openrc file, and
modify 'comming' to 'coming' in stack file, and
modify 'prefered' to 'preferred' in stackrc file.

Change-Id: I0fdd539cbfff842a4ba7fca9100b881443300f9a
2017-08-10 03:03:47 -07:00
Sean Dague
c2fe916fc7 Wait for compute service to check in
With cell v2, on initial bring up, discover hosts can't run unless all
the compute nodes have checked in. The documentation says that you
should run ``nova service-list --binary nova-compute`` and see all
your hosts before running discover hosts. This isn't really viable in
a multinode devstack because of how things are brought up in parts.

We can however know that stack.sh will not complete before the compute
node is up by waiting for the compute node to check in before
completing. This happens quite late in the stack.sh run, so shouldn't
add any extra time in most runs.

Cells v1 and Xenserver don't use real hostnames in the service table
(they encode complex data that is hostname like to provide more
topology information than just hostnames). They are exempted from this
check.

Related-Bug: #1708039

Change-Id: I32eb59b9d6c225a3e93992be3a3b9f4b251d7189
2017-08-02 09:40:25 -04:00
Jenkins
5f0a963cb3 Merge "Fleetify nova conductor for N cells" 2017-07-25 22:32:18 +00:00
Jenkins
0d9c896cdd Merge "Add f26 to the supported distros" 2017-07-24 14:05:19 +00:00
Attila Fazekas
7bbd4e95d0 Add f26 to the supported distros
The only mentionable diff is the kvm alias
does not exists so we will install
qemu-kvm as with rhel7 which also exists
in the older supported fedoras.

kvm also just an alias in suse so
switching to qemu-kvm  in suse as well.

Change-Id: I5c79ad1ef0b11dba30c931a59786f9eb7e7f8587
2017-07-24 07:32:15 +02:00
Takashi NATSUME
fa0077707e Update URLs
Some URLs are broken, so fix them.
The others are redirect to new URLs,
so replace them with new ones.

The config options of nova serial console proxy
have been gathered in nova/conf/serial_console.py.
So the description in doc/source/guides/nova.rst
is fixed.

Change-Id: Ifd81cc09969341fbf8f135a913fc6003b94e0acc
2017-07-22 09:12:59 +09:00
Dirk Mueller
04016a5dd1 List openSUSE 42.3 as a supported distro
openSUSE 42.3 is a relatively minor update over 42.2 and I'd
like to maintain it and keep it passing.

Experimental gates are being proposed, once those are passing
(which should be the case with this patch included), we could
enable it as a non-voting gate.

Change-Id: Ia421ada0ed3751c65a2a93a208e3f4a43edf8b16
2017-07-18 09:33:45 +00:00
Dan Smith
f3d5331572 Fleetify nova conductor for N cells
This makes us start two levels of nova-conductor processes, and one per cell.

Note that this also sets the notification transport_url to the top-level mq
so that we continue to get a unified stream of notifications.

Related-Bug: #1700496
Change-Id: I08d7da843d18b426dda8a8a231039d950a4c0ce5
Depends-On: I64b600b30f6e54db0ec9083c6c176e895c6d0cc2
Depends-On: If59453f1899e99040c554bcb9ad54c8a506adc56
2017-07-07 13:23:17 -04:00
Ian Wienand
07cbc44942 Put devstack-version info into separate file
As a follow on to I4c269a7f3d63ee9a976e7c3636fc3e5e8dab9ae3; the
quoting gets tricky when putting arbitrary command-substitution
strings into saved echo-able strings.  As they say, "the only winning
move is not to play" :)

An alternative proposal is to not write this into a script but just
dump info into a file.  To my mind, this has several advantages --
avoid getting involved in quoting, not dropping a script into the
global environment -- it's just as easy to "cat" -- and the plain-text
file can be collected as an artifact during log collection (also moved
git commit line to separate line for easier parsing during log search,
etc).

Change-Id: Ic7391dd087657c0daf74046e4a052c53f4eb6e1a
2017-07-03 11:56:16 +10:00
Sean Dague
2c0faca038 Add devstack-version script
This will help in collecting information about the devstack version
being used.

Change-Id: I23a8ca023bda6d097051cd57ae45401806451ad5
2017-06-28 13:53:37 -04:00
Jenkins
cb62ef8336 Merge "Revert "Fleetify nova conductor for N cells"" 2017-06-26 15:49:03 +00:00
Matt Riedemann
20eeff813d Revert "Fleetify nova conductor for N cells"
This reverts commit f63aa021cf996b500b6a570c0114c490bb57461a.

Change-Id: I541abe7311e5721d79e300fcb2a0b8ffdcd3c015
Related-Bug: #1700364
2017-06-26 07:53:08 +00:00
Jenkins
a02b9ba1b1 Merge "Fleetify nova conductor for N cells" 2017-06-23 11:36:35 +00:00
Sean Dague
99a6477c49 remove lib/os_brick
Fold this back into lib/libraries

Change-Id: I2c4e3ebf1a1cc65841e4470d5cfe073f9b505d5f
2017-06-21 10:46:12 -04:00
Sean Dague
3ed99c0b27 mv lib/oslo to lib/libraries
There is confusion about where installation of new libraries should
end up, to prevent lots of little files being added make a
lib/libraries which is the old lib/oslo. Put compat functions and
includes in place to help with transition.

Change-Id: Ieeab605d187ef6aec571211ab235ea67fa95a607
2017-06-20 14:09:30 -04:00
Sean Dague
62b56601a9 rework etcd3 install to be like other devstack services
This separates out the install phase early from the start phase to
make this mirror other services in devstack.

Depends-On: I4124dc7e3fd3b4d973979da85209ec991c0f8c4b

Change-Id: I76f8740448b25a48869ee80006e826baa6cafc2b
2017-06-20 10:24:01 +00:00
Jenkins
418bbddc81 Merge "Provide timings for OSC during devstack run" 2017-06-15 19:36:32 +00:00
Sean Dague
85cf2933cc Provide timings for OSC during devstack run
The OSC number remain high, and it's useful to understand how much
time we spend making OSC calls, especially to surface it relative to
other items. The way we embed this in our code makes it hard to
instrument.

This patch creates a wrapper function for OSC which collects the timings
then aliases `openstack` to that function. This means any invocations of
the openstack utility goes through our function while devstack is
running. Because this is an alias it only affects the stack.sh shell and
any subshells.

This also moves the time tracking infrastructure to count in ms,
instead of s, because some of these operations are close enough to a
second that rounding early is losing way to many significant
digits. We divide by 1000 before reporting to the user.

Change-Id: Ic5f1844ce732d447ee980b3c9fdc417f72482609
2017-06-14 16:43:49 -07:00
Clark Boylan
d095e97624 Support unicode via en_US.utf8
Because C.utf8 is not everywhere and is sometimes called C.UTF-8 (just
to confuse people) use en_US.utf8 which is in most places. This isn't
language/region agnostic but gives a consistent unicode aware locale to
devstack.

Change-Id: I67a8c77a5041e9cee740adf0e02fdc9b183c5bc4
fixes-bug: 1697733
2017-06-14 10:13:14 -04:00
Dan Smith
f63aa021cf Fleetify nova conductor for N cells
This makes us start two levels of nova-conductor processes, and one per cell.

Change-Id: Ice4aceac5dc44954db3661480b9365f54e47a4c9
2017-06-12 12:44:22 -07:00
Jenkins
bc8d7ef89e Merge "Start placement before services that might like to use it" 2017-06-06 00:01:06 +00:00
Jenkins
64d20857ed Merge "Enable opensuse-42.2 as a tested distro" 2017-06-05 23:56:45 +00:00
Dirk Mueller
e61e19ee82 Enable opensuse-42.2 as a tested distro
openSUSE 42.2 passes testing on the experimental gate and
in order to add it as continuosly tested target we need to
add it to the positive list of tested distributions.

Change-Id: I46f94cfad828534f324994c3d21bddff40e8f9a2
2017-06-05 19:28:30 +02:00
Chris Dent
7a74c2ab24 Start placement before services that might like to use it
Otherwise those services, notably n-cpu, will try to register
resource providers before placement is ready.

Change-Id: I89fd4fa42baf3d19ee209c59cd85b97adb97c58b
Closes-Bug: #1695634
2017-06-05 16:18:46 +00:00
Jenkins
a718b5ea92 Merge "Use proper python when configuring certs" 2017-05-31 19:59:59 +00:00
Clark Boylan
4baac65725 Use proper python when configuring certs
We have to do silly overrides of cert locations for requests for
reasons. If we are running under python3 then we were previously looking
in the wrong location for the requests certs. Update the cert fixing
function to properly use python3 to find the certs if python3 is
enabled.

Change-Id: Id1369da0d812edcf9b1204e9c567f8bfe77c48b2
2017-05-27 20:57:56 -07:00
Jenkins
4ae18aec88 Merge "stack: Workaround libvirt issue with multiple of 16 byte fixed_key values" 2017-05-26 19:22:30 +00:00
Jenkins
7b230532dc Merge "Get rid of zookeeper from devstack" 2017-05-25 18:46:09 +00:00
Davanum Srinivas
c0d16c279a Get rid of zookeeper from devstack
In Ibbb430fb1dbf66942168e0cb52d990ab6a2eb8d7, we are adding
etcd3 as a new base service. We should drop zookeeper
and use etcd3 as the backend.

Since cinder is the first service for this tooz+etcd3 DLM
scenario and cinder uses eventlet we have cannnot use the
grpc based driver in tooz. So new CINDER_COORDINATION_URL
that defaults to the etcd3's grpc HTTP gateway based
tooz backend.

We need to hold this change until the tooz change (see
Depends-On) is available in a tooz release.

Depends-On: I6184ed193482dad9643ccb2b97133d4957485408
Change-Id: Ia187e1a86413edf25b909b6bb57e84fb4930a696
2017-05-24 14:13:27 +00:00
Andreas Scheuring
94b9fae4e9 Allow disabling etcd3
Etcd3 was enabled recently as new service in devstack [1]. But there's
no way to disable etcd3. This is required on architectures where no etcd
binaries are available (e.g. s390x). The long term goal of course should
be to have those binaries available. The short term circumvention is to
allow disabling the service in local.conf:

   disable_service etcd3

[1] 546656fc05
Change-Id: I6184ed193482dad9643ccb2b97133d4957485408
Partial-Bug: #1693192
2017-05-24 15:32:03 +02:00
Davanum Srinivas
546656fc05 etcd3 as a base service
ETCD_DOWNLOAD_URL is set to github url, in our CI, we can point
ETCD_DOWNLOAD_URL to a url in tarballs.openstack.org possibly
in devstack-gate

Download the etcd binaries and drop them into /opt/stack/bin and
use it from there. Cache the tgz for subsequent use (local workflow)

daemon-reload is called twice once from inside the write_user_unit_file
and then when we adjust the entries with additional things recommended
by the etcd team. We need a better way to do this in the future.

Added a TODO to verify the downloaded artifact later. The etcd team
posts gpg signature, we could verify that or run sha256sum and hard
code that in lib/etcd3 file. We would have to update it whenever we
bump the etcd3 version.

We use the public key "CoreOS Application Signing Key <security@coreos.com>"
with ID FC8A365E to verify the integrity of the downloaded file

Any jobs that need to be run on architectures where v3.1.7 is not available
should rey the v3.2.0-rcX release candidates. We can switch to v3.2.0
when it gets released.

Initial version of this code was borrowed from the dragonflow
repo:
http://git.openstack.org/cgit/openstack/dragonflow/tree/devstack

Change-Id: Ibbb430fb1dbf66942168e0cb52d990ab6a2eb8d7
2017-05-22 21:57:48 +00:00
Lee Yarwood
00ff904b09 stack: Workaround libvirt issue with multiple of 16 byte fixed_key values
Unable to use LUKS passphrase that is exactly 16 bytes long
https://bugzilla.redhat.com/show_bug.cgi?id=1447297

Change-Id: I565339762549b076119ffedb6b83abfa12977f5e
2017-05-19 13:49:35 +01:00
Dan Smith
1f55d38911 Add systemd ulimit override function and set an override for NOFILE
This sets our default ulimit NOFILE to 2048, which is double what we
set things like mysql'd max_connections to.

Change-Id: I5126bed1e6b9f8c64db00eae4151ac61e47b1bf8
2017-05-16 09:23:28 -07:00
Jenkins
ac042d4122 Merge "remove out of date heat references" 2017-05-16 15:15:34 +00:00
Sean Dague
e264b4ab32 disable journald rate limiting
systemd-journald has rate limiting built in, but that's not sufficient
for the level of logging of OpenStack services during test
runs. Disable the rate limiting so that no log messages are lost.

Change-Id: I64599aba74c5a39276bb8f946cd236600b9cc81b
2017-05-04 15:56:37 -04:00
Sean Dague
3336b4bed6 remove out of date heat references
This removes some remnant heat code and references now that heat is
running in a plugin. Before merging this patch the heat team should
verify they got everything they needed into their heat plugin, as
there were more parts left than I was expecting.

Change-Id: I477e3a6e75591aa8ff836c28f7ef56aa1b5f8727
2017-05-03 08:23:29 -04:00
Sean Dague
8b8441f3be Default developer use case to systemd
This moves the developer use case over to systemd, and updates all the
relevant docs to discuss the systemd workflow instead of screen. It
does so by defaulting USE_SCREEN=False, so will not impact people that
set it explicitly.

Change-Id: I6d664612bc2b850eb7f56852afbc841867223ab7
2017-05-02 06:44:22 -04:00
Jenkins
7f8df450db Merge "make USE_SCREEN=False imply USE_SYSTEMD=True" 2017-05-01 19:23:10 +00:00
Sean Dague
c006bbdeb2 make USE_SCREEN=False imply USE_SYSTEMD=True
In order to start making the transition in the gate make
USE_SCREEN=False also mean USE_SYSTEMD=True. We'll never actually
declare USE_SYSTEMD=True in the gate (as that doesn't exist for stable
branches), but this will let us roll over the existing transition.

We also have to install systemd-python 234 because we are recording
exception info in the journal, and all versions before that had a bug
in processing that.

Remove the somewhat pointless screen following journalctl commands. We
really don't want or need those, and they tend to build up over time.

Depends-On: I24513f5cbac2c34cf0130bf812ff2df6ad76657c

Change-Id: I6af6d1857effaf662a9d72bd394864934eacbe70
2017-04-28 13:21:22 +00:00
Jenkins
a8204752e3 Merge "neutron-legacy: Defer service_plugins configuration" 2017-04-26 21:22:09 +00:00