765 Commits

Author SHA1 Message Date
Ian Wienand
3380a16974 Disable firewalld always
We've bike-sheded over this before
(I5252a12223a35f7fb7a4ac3c58aa4a3cd1bc4799) but I have just traced
down further issues to firewalld with neutron+ipv6 (see the bug).

In fact, as mentioned in the comments, RDO disables firewalld and the
neutron guide says to disable it [1].  The force flag is left if
anyone really wants this; but nobody is testing (or, as far as I can
tell, working on) this so bring devstack back into line and disable it
always.  Note we do not remove the package; as has been found in the
puppet scripts this can lead to dependency issues.

[1] http://docs.openstack.org/developer/devstack/guides/neutron.html

Change-Id: Ief7cb33d926a9538f4eb39c74d906ee0c879de35
Partial-Bug: 1455303
2015-05-15 19:14:19 +10:00
Sean Dague
737e94202f dump compute consoles on fail as well
This provides a dump of the compute consoles as well on failure.

Change-Id: Ib253537a54a1b9d83a930bbefa4512e039575fd1
2015-05-13 05:58:41 -04:00
Sean Dague
60a140571e add network info to the worlddump
This adds potentially helpful networking info to the world dump.

It also refactors some of the output mechanisms into reusable
functions for cleanliness in the code.

Change-Id: I39f95bd487c152925f8fadd1799149db35cffd52
2015-05-11 14:56:49 -04:00
Jenkins
db8898c495 Merge "Update default cirros version" 2015-05-08 18:43:38 +00:00
Jenkins
55f063a106 Merge "create a more generic work around for cffi & wheels" 2015-05-08 01:44:57 +00:00
Jens Rosenboom
9fd75f57fd Update default cirros version
Update the default CIRROS_VERSION to 0.3.4, which has better support
for IPv6 and some other bugfixes.

Co-Authored-By: Scott Moser <smoser@ubuntu.com>
Change-Id: I03ee6e1403680fb6c421225a7cadaf8a82edf702
Depends-On: Iac9f108d947ff4a51f99c6e8ad9d1ac5b32c000a
2015-05-07 16:06:02 +02:00
Sean Dague
168b7c226c dump iptables in the worlddump
If we fail during devstack / grenade runs, it would be nice to have
the map of iptables that are currently active as well. This makes it
handy to start figuring out what's going on when test servers don't
ping.

Change-Id: Ia31736ef2cb0221586d30c089473dfdc1db90e23
2015-05-07 08:57:28 -04:00
Sean Dague
5a59ac7d43 create a more generic work around for cffi & wheels
This is an attempt to fix the cffi vs. wheels bug in a more generic
way by just ensuring that pip has installed cffi with a pip understood
version into the venv before we try to do any builds.

Related-Bug: #1451992

Change-Id: Ibc58668c53933033405b40f79b0e9ffc73a01a6f
2015-05-06 15:58:24 +00:00
Jenkins
14d1cf90ed Merge "Add a peak memory tracker to dstat" 2015-04-20 19:16:58 +00:00
Ian Wienand
72a8be60cd Add a peak memory tracker to dstat
We can see at-a-glance memory usage during the run with dstat but we
have no way to break that down into an overview of where memory is
going.

This adds a peer-service to dstat that records snapshots of the system
during peak memory usage.  It checks periodically if there is less
memory available than before and, if so, records the running processes
and vm overview.

The intent is to add logic into the verify-pipeline jobs to use this
report and send statistics on peak memory usage to statsd [1].  We can
then build a picture of memory-usage growth over time.  This type of
report would have allowed better insight into issues such as
introduced by Idf3a3a914b54779172776822710b3e52e751b1d1 where
memory-usage jumped dramatically after switching to pip versions of
libraries.  Tracking details of memory usage is going to be an
important part of future development.

[1] http://graphite.openstack.org/

Change-Id: I4b0a8f382dcaa09331987ab84a68546ec29cbc18
2015-04-20 12:27:32 -04:00
Jenkins
9aad1b99fd Merge "refactor ping_check" 2015-04-20 13:39:33 +00:00
Jenkins
abfa858f79 Merge "Revert "List all CIRROS ARCH images in image_list.sh"" 2015-04-17 15:47:27 +00:00
Sean Dague
af9bf8663b refactor ping_check
Encapsulate all the neutron specific things you have to do ping a
neutron guest into a separate script. Refactor the main ping_check so
all logic is contained within it.

Change-Id: Ic79d8e3a2473b978551a5635a11dba07e1020bb2
2015-04-17 07:39:28 -04:00
Peter Stachowski
9a808922c1 Use openstack CLI instead of keystone
When running './stack.sh' messages are output stating that
the keystone CLI has been deprecated. These calls should be
replaced to ones utilizing the openstack CLI program instead.

Documentation examples were also updated to reflect the new syntax.

Change-Id: Ib20b8940e317d150e5f6febb618e20bd85d13f8b
Closes-Bug: #1441340
2015-04-13 10:09:13 -04:00
Joe Gordon
33ba738b05 Revert "List all CIRROS ARCH images in image_list.sh"
This reverts commit 71e82f52bde99b4bf791ea1558f1abf86019a384.

aioppcu now uses x86_64 so no need to list the i386 images for caching.

Change-Id: If500367c8bf3fdb4590c866e007ecd7de1ab5781
Depends-On: I839e1c724821ba2624beddb5233eda24b50c149f
2015-04-06 10:27:57 -07:00
Jenkins
0f173a4821 Merge "Update libvirt cpu map before starting nova" 2015-04-03 20:21:21 +00:00
Clark Boylan
edd6048168 Update libvirt cpu map before starting nova
We are trying to get a working 64bit qemu cpu model in the gate for nova
live migration testing. It appears that we need to make this change
prior to nova starting.

Make the change in configure_libvirt() to handle this along with the
other libvirt config updates. This allows us to restart the libvirt
service once. This function calls a python tool which parses and updates
the XML if necessary.

Change-Id: I00667713bfba67ab8cedbcb1660ff94d4f4bcc07
2015-03-31 15:06:30 -07:00
Jenkins
43479db910 Merge "Mostly docs cleanups" 2015-03-29 23:23:51 +00:00
Dean Troyer
dc97cb71e8 Mostly docs cleanups
Fix documentation build errors and RST formatting

Change-Id: Id93153400c5b069dd9d772381558c7085f64c207
2015-03-28 14:35:12 -05:00
Dean Troyer
bba4742e8c Add parent id to worlddump output
Helpful in tracking down some process-doesn't-die problems.

Change-Id: I146910403879c9a85d644bd07a53830ea17ca77d
2015-03-28 13:37:41 -05:00
Jenkins
d9a7ae8056 Merge "Allow devstack plugins to specify prereq packages" 2015-03-19 00:35:34 +00:00
Jenkins
a055557705 Merge "Move configuration functions into inc/*" 2015-03-12 15:22:47 +00:00
Adam Gandelman
7ca90cded3 Allow devstack plugins to specify prereq packages
We offer main devstack components the ability to install their own
system package preqreqs via files/{debs, rpms}/$service.  This adds
similar functionality for plugins, who can now do the same in their
own tree at ./devstack/files/{debs, rpms}/$plugin.

Change-Id: I63af8dc54c75a6e80ca4b2a96c76233a0795aabb
2015-03-11 14:46:35 -07:00
Ian Wienand
7c4ce9edba Check for new versions of get-pip.py
People can leave their devstack installs around for a long time, and
in the mean time new versions of pip can be released.

The current check does not download a new version if an old one
exists.  We want to check for new versions, but we also don't want the
gate jobs trying this sometimes unreliable fetch.

So add a flag-file that tells devstack if it downloaded get-pip.py
originally.  If so, on each run check for a new version using curl's
"-z" flag to request only files modified since the file's timestamp.

Change-Id: I91734528f02deafabf3d18d968c3abd749751199
Closes-Bug: #1429943
2015-03-11 09:40:13 +11:00
Dean Troyer
bf2ad7015d Move configuration functions into inc/*
* config/INI functions from functions-common to to inc/ini-config
* local.conf meta-config functions from lib/config to inc/meta-config

Change-Id: I00fab724075a693529273878875cfd292d00b18a
2015-03-09 22:52:19 -05:00
Bob Ball
1340ee72bf XenAPI: Default JEOS VM to only use 1GB RAM
While Devstack needs 4GB RAM (or more!) the JEOS
used as the base for the Devstack VM for XenServer
needs much less.  Allowing the initial install to
use 1GB means we have lower requirements overall

Change-Id: Iecaeeb4db0dffcc43c5532b5d57cb041d47047a6
2015-03-06 21:15:59 +00:00
Ramy Asselin
db29a7cf0c Fix devstack running behind a proxy
As suggested by dtroyer to resolve wheels not buildings
behind a proxy. Issue introduced in I0283b0dff9146b1b63bd821358505a93566270c6

Change-Id: Ib376469aff73a22ac2bc9d7d5a7f90081004aa8d
Closes-bug: 1426146
2015-02-26 14:40:51 -08:00
Jenkins
3ffd5475bf Merge "List all CIRROS ARCH images in image_list.sh" 2015-02-26 18:19:37 +00:00
Joe Gordon
71e82f52bd List all CIRROS ARCH images in image_list.sh
image_list.sh is used to make a list of all possible images. Loop over
both x86_64 and i386 instead of just using the default arch of x86_64

For live migration we are starting to use the i386 arch based images and
don't want to cache them like all other images.

Change-Id: I47da72a0e9da3689cb085bb43ac6536094445112
2015-02-25 11:39:18 -08:00
Vigneshvar.A.S
834b804d3e Upgrades pip under virtualenv
The version of pip supported by python-virtualenv has issues in
environment under proxy, hence uninstalling python-virtualenv package
and installing the latest version using pip.

Change-Id: Id749c37ab7fefa96b35f11816b56b9def5ef4b08
Closes-Bug: 1421541
2015-02-25 08:18:18 +00:00
Dean Troyer
99c463d56d Recent virtualenv review cleanups
This is a follow-on to comments in https://review.openstack.org/156356
and https://review.openstack.org/#/c/151513/

* Remove work-around for /var/cache/pip
* Remove WHEELHOUSE setting in tools/build_wheels.sh and use the pip
  default directory '<cwd>/wheelhouse'
* Remove bogus MySQL-python install
* Removed unused bits and clean up pip commands in from tools/build_venvs.sh

Closes-Bug: #1423720
Change-Id: I0283b0dff9146b1b63bd821358505a93566270c6
2015-02-20 08:56:53 -06:00
Dean Troyer
8c2ce6ea72 Virtual environment groundwork
Introduce the tooling to build virtual environments.

* tools/build_venv.sh: build a venv
* introduce lib/stack to house functionality extracted from stack.sh that
  is needed in other places, such as Grenade; start with stack_install_service
  to wrap the venv install mechanics
* declare PROJECT_VENV array to track where project venvs should be installed
* create a venv for each project defined in PROJECT_VENV in stack_install_service()

Change-Id: I508588c0e2541b976dd94569d44b61dd2c35c01c
2015-02-18 20:19:38 -06:00
Dean Troyer
b1d8e8e274 Build wheel cache for venvs
Building a bunch of virtual envs later is going to be tedious if we do not
pre-cache certain annoying-to-build packages.

* tools/build_wheels.sh: pre-build some wheels for annoying package installs
* list distro package dependencies in files/*/venv
* list packages to pre-build as wheels in files/venv-requirements.txt
* install database Python modules when setting up the database

Change-Id: Idff1ea69a5ca12ba56098e664dbf6924fe6a2e47
2015-02-18 15:13:00 -06:00
Jenkins
2e9d32baf2 Merge "XenAPI: Recommend using xl console rather than vncviewer" 2015-02-18 12:57:32 +00:00
Bob Ball
e356d8cff6 XenAPI: Recommend using xl console rather than vncviewer
Some vncviewers do not support the -via option, so default to suggesting
xl console.  XenCenter continues to be an option for those running
Windows.

Change-Id: I1e53fd33d309e30a60031965e589167dcbacfbbe
2015-02-17 15:05:34 +00:00
Jenkins
9f362d3ab6 Merge "XenAPI: Move where os-vpx is set" 2015-02-16 22:04:23 +00:00
Jenkins
b2b19040d4 Merge "XenAPI: Move some boot-time functions to install-time" 2015-02-16 21:49:17 +00:00
Jenkins
314723962b Merge "Make EC2 compatible with current AWS CLI." 2015-02-12 17:48:54 +00:00
Bob Ball
0df75a7288 XenAPI: Move where os-vpx is set
os-vpx is only true after we know it's not a JEOS we're creating
Move the location we set the os-vpx flag to as we're starting to
prepare the VM for devstack

Change-Id: If736f3a7de171b8090b0bb905e04accb70470a90
2015-02-12 11:59:46 +00:00
Bob Ball
5b9adb60de XenAPI: Move some boot-time functions to install-time
The boot-time script (prepare_guest.sh) is one of the less reliable parts
of the install process.  This change enables SSH into the host as well as
reporting of the IP address.  This significantly helps debugging issues
now and enables moving of all other setup code to being executed over SSH.

Change-Id: I1555f1d91353ba8b75e2de4607df33ee20307a6e
2015-02-11 11:15:02 +00:00
Bob Ball
347e30877b XenAPI: Use round-robin mirror for Ubuntu
ANL has discontinued their Ubuntu mirror.  Update the target
to be the official round-robin mirror which should be more stable long-term

Change-Id: I1271301d96a44888f45664537435e31dd38ff30d
2015-02-09 12:02:51 +00:00
Andrey Pavlov
0ea8b72a20 Make EC2 compatible with current AWS CLI.
Nova listens for EC2 calls now at the URL without path -
http://some.server.com:8773/
I was made in review -
https://review.openstack.org/#/c/152496/

So I suggest to change EC2 urls in keystone catalog.

Change-Id: Ia2975ce0f6a30eed6016733e12c98b5f97648307
Closes-Bug: 1417555
2015-02-06 22:51:07 +03:00
Jenkins
a08590dbf2 Merge "XenAPI: Replace remaining instances of 'devstack.succeeded'" 2015-02-02 15:38:12 +00:00
Jenkins
481d50d80c Merge "Switch on curl retry options for fetching get-pip.py" 2015-01-29 06:51:52 +00:00
Attila Fazekas
1f316beb20 Remove rhel6 and py26 support
el6 is shipped with Python 2.6.x which is not expected
to be supported with the openstack kilo release.

For el6 support we need to do lot of thing differently,
which makes the code more complicated.

This change removes el6 and py26 support from devstack.

This change also removed a discontinued (1 year ago)
openSUSE 12.2 code path, which used a similar codepath as el6.

Several comment related to el6 also removed or modified.

Change-Id: Iea0b0c98a5e11fd85bb5e93c099f740fe05d2f3a
2015-01-27 09:22:52 +01:00
Bob Ball
f35e95706c XenAPI: Replace remaining instances of 'devstack.succeeded'
Commit I05990c7154366350b0f9cc3e6c70d6f34238486f was incomplete
and left two references to the, now replaced, devstack.succeeded.

The impact of this is that the XVA always claims that devstack failed
to run when actually the run may have been successful

Change-Id: Ie4ac673011d77a9edc8923db94d914efe9f45d5d
2015-01-26 14:13:02 +00:00
Davanum Srinivas
f5b4154251 Switch on curl retry options for fetching get-pip.py
Let's retry a few times before giving up.

Related-Bug: #1413034

Change-Id: I7c5c1a3936b5c08c5de43edc569b5a53d11b55a7
2015-01-22 07:57:42 -05:00
Jenkins
7fba9c9912 Merge "XenAPI: Use SCREEN_LOGDIR rather than assuming a static value" 2015-01-21 08:12:15 +00:00
Jenkins
77aca755b1 Merge "XenAPI: Fix another race condition" 2015-01-21 08:12:06 +00:00
Bob Ball
1e3a5d2d6c XenAPI: Use SCREEN_LOGDIR rather than assuming a static value
This value was defined in Citrix's install-devstack-xen script, so
only worked for those using that script.

Change-Id: Iab63389f41760865f2b67f6dccd57d774e889905
2015-01-16 14:50:40 +00:00