8509 Commits

Author SHA1 Message Date
Davanum Srinivas
d8283fde66 Avoid installing etcd3 in subnodes
We need to handle this better, for now, just don't install
etcd in the sub nodes. We need to setup the proper clustering
mechanism if we want to have etcd3 running in multiple nodes

Change-Id: I8dd385e3c993942473e67d04367cdf74495dbeef
2017-05-23 22:12:42 -04:00
Jenkins
08de346d03 Merge "etcd3 as a base service" 2017-05-24 00:08:47 +00:00
Jenkins
23ed6666ed Merge "Always setup libvirt for tap devices when using Neutron" 2017-05-23 01:26:25 +00:00
Davanum Srinivas
546656fc05 etcd3 as a base service
ETCD_DOWNLOAD_URL is set to github url, in our CI, we can point
ETCD_DOWNLOAD_URL to a url in tarballs.openstack.org possibly
in devstack-gate

Download the etcd binaries and drop them into /opt/stack/bin and
use it from there. Cache the tgz for subsequent use (local workflow)

daemon-reload is called twice once from inside the write_user_unit_file
and then when we adjust the entries with additional things recommended
by the etcd team. We need a better way to do this in the future.

Added a TODO to verify the downloaded artifact later. The etcd team
posts gpg signature, we could verify that or run sha256sum and hard
code that in lib/etcd3 file. We would have to update it whenever we
bump the etcd3 version.

We use the public key "CoreOS Application Signing Key <security@coreos.com>"
with ID FC8A365E to verify the integrity of the downloaded file

Any jobs that need to be run on architectures where v3.1.7 is not available
should rey the v3.2.0-rcX release candidates. We can switch to v3.2.0
when it gets released.

Initial version of this code was borrowed from the dragonflow
repo:
http://git.openstack.org/cgit/openstack/dragonflow/tree/devstack

Change-Id: Ibbb430fb1dbf66942168e0cb52d990ab6a2eb8d7
2017-05-22 21:57:48 +00:00
Jenkins
6a62619cbc Merge "add a table of contents" 2017-05-18 14:28:18 +00:00
Jenkins
e4c98aa42a Merge "Add --tcp option to dstat command for connection stats" 2017-05-18 14:19:20 +00:00
Jenkins
e19abda40a Merge "Restore log colors under devstack/systemd" 2017-05-17 20:10:30 +00:00
melanie witt
fc572a5da0 Add --tcp option to dstat command for connection stats
This enables tcp stats (listen, established, syn, time_wait, close) in
dstat to allow us to get a high-level view of performance changes in
the system during gate runs.

Change-Id: Ifbffbed22446e7e6a3b825c18266b63d2f2e7718
2017-05-17 19:05:56 +00:00
Jenkins
5ff1b476e4 Merge "Copy 'resource_filters.json' file to cinder config folder" 2017-05-17 19:02:22 +00:00
Kevin Benton
d1fe0e62e7 Always setup libvirt for tap devices when using Neutron
This logic has been tied to OVS since it was introduced in [1] and
revised in [2]. However, many other backends may use tap devices that
aren't related to OVS, such as Calico[3] and Linux Bridge after [4]
merges.

This patch just removes the dependency on OVS specifically so
/dev/net/tun is added to cgroups whenever any Neutron backend is used.
This is done in other deployment tools like Juju[5] so it's not
unprecedented.

1. Ifab268f739b004db13024633e8abeb17691b9e46
2. Ic1da132fa421f1c70c10a319ee3239831b0f956f
3.
http://docs.projectcalico.org/master/getting-started/openstack/installation/ubuntu#compute-node-install
4. I23c5faaeab69aede1fd038a36f4a0b8f928498ce
5.
2790f81ecd/templates/qemu.conf

Change-Id: I075595158d8f3b5a6811c4794aa7b91912940db5
Partial-Bug: #1675343
2017-05-17 06:07:35 +00:00
Jenkins
00e5a98eea Merge "Use -y with 'pip uninstall'" 2017-05-17 00:41:04 +00:00
Eric Fried
8cd310d763 Restore log colors under devstack/systemd
One of the pending issues with the conversion to systemd was the loss of
log coloring.  It turns out that journalctl by default strips out
characters it considers "unprintable" - including the color codes
emitted by the old-style logging.  However, journalctl can be made to
print them by adding the `-a` flag.

This change makes devstack's log formatter conf settings include color
codes like the old screen-based setup used to

We also remove stackrc's setting of JOURNALCTL_F, whose usage was
removed via I6af6d1857effaf662a9d72bd394864934eacbe70.

Change-Id: I2401e267913a24d18dae355aa933072dbbdab1d8
2017-05-16 14:56:57 -05:00
Brian Haley
954fd1b729 Use -y with 'pip uninstall'
'pip uninstall' will hang running stack.sh if it has to
prompt the user for input, use -y.

Change-Id: Ic94639e444b87fd3538463d5a51c01a0208a2ab2
Closes-bug: #1691172
2017-05-16 12:24:45 -04:00
Dan Smith
1f55d38911 Add systemd ulimit override function and set an override for NOFILE
This sets our default ulimit NOFILE to 2048, which is double what we
set things like mysql'd max_connections to.

Change-Id: I5126bed1e6b9f8c64db00eae4151ac61e47b1bf8
2017-05-16 09:23:28 -07:00
Jenkins
ac042d4122 Merge "remove out of date heat references" 2017-05-16 15:15:34 +00:00
Doug Hellmann
84ce2f1c9b add a table of contents
Add a table of contents help readers find the information they need
without having to read all of the prose on every page. Remove the
site-map file, which doesn't appear to be linked anywhere.

Change-Id: Ib5761c9cfdd5a083df562413d727cb4ac7547c9e
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-05-16 10:47:09 -04:00
Jenkins
1d638a9b52 Merge "Make openrc more robust for grenade scenarios" 2017-05-16 14:47:04 +00:00
Jenkins
3420019d9d Merge "remove some unused *_PROTOCOL from export" 2017-05-16 14:46:50 +00:00
Jenkins
7834f5285f Merge "Document testing of new devstack features" 2017-05-16 12:41:51 +00:00
Jenkins
79911d8b19 Merge "uninstall libvirt-python and reinstall" 2017-05-16 12:41:44 +00:00
Jenkins
972b518148 Merge "Add files/ironic-inspector* to .gitignore" 2017-05-16 07:19:04 +00:00
Clark Boylan
2a6112ea9a Document testing of new devstack features
At the Boston 2017 Summit I had mentioned that the pattern of using non
voting/experimental jobs was not working for getting new features into
Devstack. It is slow and leads people to being too conservative when it
comes to pushing new things in. Instead I suggested that since Devstack
changes are self testing we add the features, have change that enables
the feature, and if that changes passes we move forward with merging
(assuming code review is fine and necessary communication is done).

Document this process in the HACKING file so that we have something we
can point to when people want to add a new experimental job for every
new little thing (ipv6, tls, systemd, etc).

Change-Id: I5190cc3d3de4e81d52748347306133b5034d5531
2017-05-12 10:16:33 -07:00
Angel Noam
f24e29920b Add support for image name end with .raw
Change-Id: If3bd42825a29755fa8e68fa5661d068c6f0019d2
2017-05-11 15:13:29 +03:00
vmud213
db8a199b30 Add files/ironic-inspector* to .gitignore
When ironic-inspector is enabled through devstack, these two files
are created which needs to be masked.

Change-Id: I7a3db6fd6197da20cca1e938727d54195957ac18
Closes-Bug: #1690105
2017-05-11 09:38:25 +00:00
Jenkins
27df725179 Merge "Remove fping requirement" 2017-05-09 16:44:29 +00:00
Jenkins
6fe14a5d93 Merge "Always disable use of libvirt wheel with UCA" 2017-05-09 16:34:33 +00:00
Sean Dague
f28e7ef6ba uninstall libvirt-python and reinstall
libvirt-python compiles against the currently installed libvirt. If
you upgrade that, it needs to rebuild, however it won't change
versions, so pip install just noops. Force an uninstall / reinstall of
it every time to handle potential upgrades of libvirt.

Change-Id: If34541b34aa6d55eedaf6c603fd1fe92eb887308
2017-05-08 07:30:20 -04:00
Jenkins
caad9221e0 Merge "default gateway regex: use exact match for iface name" 2017-05-08 03:15:07 +00:00
Jenkins
2b88b323c9 Merge "Fix systemctl option: removes --unit option in doc" 2017-05-07 17:03:50 +00:00
Jenkins
37a6b0b2d7 Merge "Set SyslogIdentifier for uwsgi processes" 2017-05-05 19:50:54 +00:00
Sean Dague
f41bf2a92f Set SyslogIdentifier for uwsgi processes
Now that we aren't using native journal messages by default, the
syslog identifier of all the uwsgi processes is the same. We should be
more explicit with those.

Change-Id: Id5406d02407b022d4016517c2e18890973876d88
2017-05-05 07:54:26 -04:00
Jenkins
d235ca32f9 Merge "Make devstack fail early for common systemd pitfalls" 2017-05-05 11:41:26 +00:00
Jenkins
06dc8313ca Merge "Always have ./clean.sh run ./unstack.sh" 2017-05-05 05:25:43 +00:00
Jenkins
51a136a22c Merge "disable journald rate limiting" 2017-05-05 04:40:32 +00:00
Jenkins
620ae0339c Merge "Correct various inaccuracies in overview.rst" 2017-05-04 20:33:15 +00:00
Jenkins
bf62c96702 Merge "convert README to rst" 2017-05-04 20:33:08 +00:00
Jenkins
6612f66374 Merge "Fix logging when under systemd" 2017-05-04 20:31:41 +00:00
Sean Dague
eaadffe07b Always have ./clean.sh run ./unstack.sh
There was detection code in clean.sh to only run it if a screen
session was found, but in systemd world, that's obviously not
true. This was causing me (and others) substantial confusion.

Change-Id: I204e94cd86b8c67012aabfca74796e593151c3a4
2017-05-04 16:05:19 -04:00
Sean Dague
e264b4ab32 disable journald rate limiting
systemd-journald has rate limiting built in, but that's not sufficient
for the level of logging of OpenStack services during test
runs. Disable the rate limiting so that no log messages are lost.

Change-Id: I64599aba74c5a39276bb8f946cd236600b9cc81b
2017-05-04 15:56:37 -04:00
Ian Y. Choi
c2200bc880 Fix systemctl option: removes --unit option in doc
--unit option is for journalctl, not systemctl.
Just executing systemctl without "--unit=" works.

Change-Id: I9752561332e62ec6327b17b12d2d868892718041
2017-05-04 21:17:40 +02:00
Sean Dague
148d58c351 Make devstack fail early for common systemd pitfalls
There are a couple of issues that have ended up being hit by devstack
plugin authors which we can detect and error in a much nicer way
instead of them having a cryptic systemd failure.

Change-Id: I45e4ac363aeefb4503015f9e1b57c58f79b58f40
2017-05-04 13:08:25 -04:00
TommyLike
401de4d20c Copy 'resource_filters.json' file to cinder config folder
Cinder use 'resource_filters.json' to config allowed filters,
copy the new added json file when set config files.

Change-Id: I397cb5859e2b3349af3cb07ee02b6463c6eccc35
Depends-On: 27aeba0b5d3cf64286125937e8336ba1d3b26b16
2017-05-04 13:46:29 +00:00
Sean Dague
b2bfe5617a Turn off use_journal because of eventlet concerns
use_journal uses the systemd native path for logging, however there
are concerns that this might be negatively interacting with
eventlet. To be on the safe side fall back to stdout.

This introduces a USE_JOURNAL option which will let folks turn this
back on for testing.

This also adjusts the debug lines. When using the journal the pid
reported by systemd is correct. When using stdout, it will be the
parent process id, so we need to keep it to see which child each thing
is coming from.

Change-Id: Id7891c532bf99c099252e82d511a37a49506fea9
2017-05-03 18:35:54 +00:00
Sean Dague
3336b4bed6 remove out of date heat references
This removes some remnant heat code and references now that heat is
running in a plugin. Before merging this patch the heat team should
verify they got everything they needed into their heat plugin, as
there were more parts left than I was expecting.

Change-Id: I477e3a6e75591aa8ff836c28f7ef56aa1b5f8727
2017-05-03 08:23:29 -04:00
Sean Dague
2686b36c32 Correct various inaccuracies in overview.rst
This still was referencing nova-network

Change-Id: Id584d3fd26482330f42319aee4cb36981066db55
2017-05-02 09:26:28 -04:00
Sean Dague
29c1f981fe convert README to rst
Everything else in our documentation is RST, we should convert the
Readme to RST as well (as github supports this, so it's fine to read
it there)

Change-Id: If2aabf629affc09b5daa570f4ca3bdf268cb53b0
2017-05-02 09:25:29 -04:00
Sean Dague
27f66e9830 Fix logging when under systemd
Both keystone and neutron didn't yet have systemd awareness for
setting up logging (i.e. drop the extra date / time stamps)

Change-Id: Ib442c603c9afb679676976c37c2c6122201ae846
2017-05-02 09:24:45 -04:00
Sean Dague
8b8441f3be Default developer use case to systemd
This moves the developer use case over to systemd, and updates all the
relevant docs to discuss the systemd workflow instead of screen. It
does so by defaulting USE_SCREEN=False, so will not impact people that
set it explicitly.

Change-Id: I6d664612bc2b850eb7f56852afbc841867223ab7
2017-05-02 06:44:22 -04:00
Sean Dague
803acffcf9 Make ./clean.sh work in more situations
When transitioning between different wsgi modes, or service modes, we
should really safely stop and cleanup things that are started in any
service mode, which makes it easier to ensure that we don't leave
things around from past runs.

Change-Id: I33acbee39e1a2da2bfd79a5dd54b84a12a778be1
2017-05-02 06:20:22 -04:00
Jenkins
7fe9e784c6 Merge "try to use unversioned keystone endpoints everywhere" 2017-05-01 23:05:46 +00:00