100 Commits

Author SHA1 Message Date
Jeremy Stanley
02e4eb0fa8 Finish switching to using new ansible inventory
Change I76b1099bf0cf3bfead17f96e456cdce87d0e8a49 altered the name of
the inventory script, so reflect that in the corresponding
subprocess call in launch-node.py and a comment in the
expand-groups.sh script.

Change-Id: I4c2c762716813b5d59dcc1b623f5988c8aa7d490
2018-06-18 18:27:34 +00:00
Monty Taylor
5af7703eee
Add error messages to dns.py about old sdk
dns.py requires the latest shade/openstacksdk. Put in explicit error
messages about that.

Change-Id: Id7eae3b21765a2b5bc1e4d446d6bb5766f922932
2018-04-03 12:49:43 -05:00
Monty Taylor
1f138011f9
Update connection construction in launch_node
The dns.py file uses openstack.connect to make the Connection but
launch_node.py was still using shade.OpenStackCloud, so when the
connection was passed to dns.py it was trying to use an SDK property but
getting a Shade object.

This is because while sdk has been updated with all of the shade objects,
we haven't updated shade yet to provide the sdk version of the object, so
shade objects from sdk have things shade objects from shade don't yet have.

Update launch_node.py to use the same Connection construction that
dns.py does.

Change-Id: I1c6bfe54f94effe0e592280ba179f61a6d983e7a
2018-04-03 12:49:43 -05:00
David Moreau Simard
213975b930
Update launch-node's dns.py to work with openstacksdk
Shade no longer uses novaclient. shade also strips links dicts from the
resources it returns. shade also now depends on openstacksdk, which does
not strip links dicts.

Change-Id: Ifb6a8280e548cb55932cae4a2bba8e1fa5b34c3c
2018-03-23 13:01:40 -05:00
Clark Boylan
7c37d2bffc Launch virtualenv should be python2
virtualenv on the puppetmaster is defaulting to python3 now but all our
dev header files for python are for python2. Force python2 when creating
the virtualenv so that pycrypto can be built. Additionally ansible
likely wants python2 here anyways.

Change-Id: I19bc1985fc4b6a722b10fb0b89a86127e27340fe
2017-12-11 05:50:16 -08:00
Paul Belanger
2ce7da2301
Bump default image to xenial to launch-node.py
We're now launching xenial for control servers, lets update the
defaults.

Change-Id: I14dc26673c290ae37b7a9ef016d7a343d2763efe
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-09-11 18:42:51 -04:00
Ian Wienand
8791e9fafa launch-node: ignore failure to restart eth0
eth0 might not exist, such as on Xenial hosts with interfaced-based
names.  Since this is a bit of platform/provider specific hack, just
ignore failures.

Change-Id: Ie18b7f49ea2f1b72b496c61ac2576ae53f5ad3eb
2017-03-28 13:50:26 +11:00
Jenkins
44f909dc7c Merge "launch: add puppet environment option" 2016-09-16 01:21:15 +00:00
Monty Taylor
a05340b2f6 Use OS_REGION_NAME so that OCC ignore it properly
os-client-config will construct a cloud called "envvars" if the
environment has environment variables that start with OS_ and are not
OS_CLOUD and OS_REGION_NAME (those are singled out because they are
selectors) The convenience variable in our example code snippet here is
an OS_ var that is neither of those, so it causes the environment to
produce an invalid cloud config which then confuses the ansible
inventory which is trying to iterate over the all the clouds that
exist.

Change-Id: I65324bc2f3ca71dd4ada2f39f322ccc5f13d6897
2016-09-16 00:49:11 +00:00
Jenkins
f2ac215342 Merge "launch: stream syslog on remote host" 2016-09-16 00:23:09 +00:00
James E. Blair
e8da64fa94 launch: add puppet environment option
Change-Id: I3e0ce41e399cc5e3fdb8153e5bac5b97fd58f0e1
Depends-On: I1c0b25a838b6fef6487fc3a8e5b78b73a3ae305a
2016-09-15 15:53:08 -07:00
James E. Blair
ccfe5111c8 launch: stream syslog on remote host
So that operators have a clue what's going on during the long
initial puppet run.

Change-Id: If310c3420d942c79c8b04d3e9ce68f39081cc507
2016-09-15 15:46:33 -07:00
Jenkins
502e83dd1a Merge "launch: document using a virtualenv for launch-node" 2016-09-15 22:45:23 +00:00
Jenkins
2d0f60a3dd Merge "launch: Use popen rather than check_output" 2016-09-15 22:41:30 +00:00
James E. Blair
8f46fbc0b5 launch: document using a virtualenv for launch-node
Change-Id: Ib275299f333e0b109c5e2d22b0d593a9ade0783b
2016-09-15 15:17:55 -07:00
James E. Blair
334715f12a launch: Use popen rather than check_output
This way we are able to stream the output from commands as they
are received for better debugging.  We can also move some new
debug statements to inside of the new run() function so they
are more automatic.

Change-Id: I484f5cf70aa15923ea4bb866f3be536b2e8ed4ed
2016-09-15 15:17:50 -07:00
Ian Wienand
20afe1a62f launch-node.py: More verbose logging
One problem with "shell script as python" is that there's no
equivalent of "-x" in shell, which makes it really hard to extract
what's being called and where output came from.

This adds a bit more verbose logging around the ssh calls to try and
help someone parsing the logs.

Change-Id: I85e2415b47e044cfa1c678fc7786b4891fa1f93e
2016-08-24 11:06:57 +10:00
Ian Wienand
4ac715e0f3 launch-node.py: set ansible log path
Avoid a bunch of warnings about unwritable /var/log/ansible.log (the
default) by setting the log path environment variable where we call
ansible.

Note expand-groups.sh is moved inside the JobDir() context so we can
use the environment var there too, as it calls ansible underneath.

Change-Id: I575d633a36db8cfb891c8903a7bfbea73a4cfb29
2016-08-24 11:06:04 +10:00
Ian Wienand
ab376eac7a launch-node.py: save key when failing early
Save the key to a file in /tmp when failing early with --keep.
Although it is put into the JobDir later, if we fail before that we're
locked out of the host.

While we're here, make what just happened in an error case a little
clearer

Change-Id: Ide601e2018302664bc4ad609c4483aa1451b3724
2016-08-24 11:04:45 +10:00
Ian Wienand
4ed43d2754 launch-node.py: restart interface
RAX nodes are exhibiting new behaviour of having ipv6 configured but
not active.  Restart eth0 to pick up the address in
/etc/network/interfaces so the ping6's work

Change-Id: I6b60bde34cc28ca60c5cbbb41de02cd89354cc32
2016-08-24 06:33:06 +10:00
Jenkins
3addf0b5a9 Merge "Correct launch-node.py --config-drive default" 2016-07-08 08:12:37 +00:00
Monty Taylor
3b9eda9359
Prevent launch-node from breaking generated groups
There are potentially two related issues here which can result in
an empty generated groups file. The first is that if there are OS_
environment variables set, then os-client-config can create an 'environ'
cloud. That cloud then, in most cases here, will not be a valid cloud
since it won't be a full config, so iterating over all existing clouds
to get their server will fail, meaning that the inventory will be empty
meaning that generated groups will then be generated empty.

To deal with that, we can consume the newer upstream option that allows
the inventory to not bail out if it has a bad cloud, but instead get all
of the resources from the clouds that do work.

Additionally though, we can do an explicit inventory run so that we can
look to see if the inventory run failed, and if so, avoid running the
expand-groups.sh script, since we'd be fairly assured that it would be
running on top of a bad inventory cache.

Change-Id: Ib18987b3083f6addc61934b435d7ecb14aa1d25a
2016-07-07 11:21:14 -05:00
Jeremy Stanley
d66b8f47c4 Correct launch-node.py --config-drive default
For --config-drive to actually work as advertised in launch-node.py,
it needs to default to False. Otherwise this option is useless.

Change-Id: Ib29fa758779e89d3d25399615fd009b836dda598
2016-07-07 15:04:53 +00:00
Jenkins
1960078a1d Merge "Use an ordinal server naming pattern" 2016-06-30 20:14:40 +00:00
James E. Blair
df81542135 Use global generated groups in launch-node
When launching a server, ansible needs to know what groups the
new host is in so that it can copy the appropriate files.  Figuring
that out is done based on the groups.txt file and the expand-groups
script.  This change runs that script after creating a host, which
will update the global list of expanded groups.  That is then
symlinked into a temporary inventory directory used by launch-node.

The JobDir concept is borrowed from Zuul as a simple way of creating
and deleting at the appropriate time a complex temporary directory.

Change-Id: Icce083ca67a3473b7d77401142f870fd28dd08f5
2016-06-09 09:23:32 -07:00
Jeremy Stanley
3ac0a5eb69 Use an ordinal server naming pattern
As discussed during the "Launch Node, Ansible and Puppet" summit
session in Austin, we're making things unnecessarily hard on
ourselves by insisting on having multiple servers in our inventory
with the same name. In order to make server addition and replacement
automation simpler, start using an ordinal suffix on server short
names to differentiate them (we can still easily rely on DNS for
their non-numbered convenience names).

Change-Id: I040a5c3b5e1abc50c3e4676bcab0bf4eaa550f4b
2016-05-23 19:42:18 +00:00
Clark Boylan
ba4429022b Properly handle volumes in launch node
We can only get the volume attach device if we are attaching a volume.
Check if the volume is being attached and only determine the attachment
location in that case to avoid errors.

Story: 2000569
Change-Id: I4adc5e23abdfc0627a0850f845e2333d3bd25e63
2016-05-02 15:04:57 -07:00
Clark Boylan
6429922e99 Add support to launch-node for cinder attach
Now that we have a shade version of the launch node script adding in
support for attaching a cinder volume is simple. Do this so that
launching mirrors which rely on cinder volumes is simpler.

This updates the mount_volume.sh script to setup the first cinder volume
with lvm and mount it under the specified path. It will also install
lvm2 pacakges since they may not be present on all base images.

This updates the make_swap.sh script to avoid blindly using /dev/vdb as
the location for swap as this may be a cinder volume or config drive.

We add availability zone, device specification, mount path, and
fs label support to shade-launch-node.py as these are all necessary
inputs to properly mount a cinder volume in a VM.

Change-Id: Ie95fd4bd5fca8df4f8046d43d1333935cad567e3
2016-04-19 11:07:23 -07:00
Jeremy Stanley
ce70bde4de Use environment file in sample reverse DNS command
With Ansible-based launch-node.py now using clouds.yaml, we're no
longer setting up the environment variables expected by the rackdns
utility for setting PTR resource records. Add a line to the example
commands sourcing the appropriate variables into the environment
first.

Change-Id: Ia96296a8fac803d514e49eeeedaeebc76585d6fd
2016-03-23 14:37:52 +00:00
Monty Taylor
453051aafb Don't pass OS_CLOUD and OS_REGION_NAME to expand-groups
There is a bug in OCC that causes an envvars cloud to be created when
the only two env vars are the selectors OS_CLOUD and OS_REGION_NAME. So
exclude them from the envionment when running the group creation
command.

Also, there is a bug in the invocation of the hostname playbook, in that
it was passing in the UUID as the target to run against, but we're
writing out a name-based inventory.

Change-Id: I0b524dc43ec96c6645ae82a090744eab463e7fb9
2016-03-22 19:33:42 -05:00
Paul Belanger
72e045ea16 Give 0775 to launch-node.py
Change-Id: If56ca023c6a30735a18c891c6603424c718aeb67
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-03-22 23:59:42 +00:00
Monty Taylor
9e678eeb8e Don't set the host's name as UUID
It looks like we solved the duplicate server problem twice in
conflicting ways. Using uuid in the inventory is not needed, bcause
we're making a specific inventory for the ansible commands and avoiding
the OpenStack inventory. So the ansible run has no idea of any other
servers other than the one we're making right now. With that, we can use
name as the hostname rather than UUID.

Story: 2000520
Change-Id: Idb967e10fc00471923077e4e9caa32fdb4c1cc78
2016-03-22 12:16:10 -05:00
Monty Taylor
8a1d7a5ab6 Remove interpret_puppet_exitcodes
This is dead code now. The equivilent function is now in
openstack-infra/ansible-puppet

Change-Id: I7ba2f0b9da720b72a0a97b27a43a873b29c41ed8
2016-03-10 09:31:50 -06:00
Monty Taylor
3eb4a5a66a Use set-hostnames playbook in launch-node
We have a playbook that does the logic of setting the hostname. Rather
than implementing that logic in launch-node - just use the playbook.

Change-Id: I1a6c0ff12803bdac35631cb3bb2c8fe70cbd1904
2016-03-10 09:48:34 +01:00
Monty Taylor
cdead64021 Zero out the inventory cache instead of removing it
https://github.com/ansible/ansible/pull/14882 landed, so the inventory
will understand that an empty cache means the inventory needs
refetching. Zero out the file, and start consuming inventory from the
master branch of ansible since mordred controls that file anyway.

Change-Id: I2a4f4b21c50bfa94a229dd109e3d21f47552f0a1
2016-03-09 14:23:54 -06:00
Monty Taylor
f94594a98e Retire old launch-node.py
It has served us well, but it's time for the new world order to take
over.

Change-Id: I72a9f1b1c0f1bd331b5f862f5f69966950ffe963
2016-03-09 14:23:54 -06:00
Monty Taylor
e14471dcc6 Update for ansible and puppet apply
Shade supports all of the TODO items in here now, so use it. Also,
we have ansible playbooks that do the work of running puppet - and since
we're on puppet apply now, we can use them.

Change-Id: I6f57e9a31bf835ef2e22db1f5531d92e99806cf4
2016-03-09 14:23:54 -06:00
Jenkins
b487feaf7c Merge "Set hostname in launch node" 2016-02-24 23:24:37 +00:00
James E. Blair
f6e9e389b9 Set hostname in launch node
Since nova does not believe in the existence of hostnames, we need
to set them ourselves when we boot new servers in launch-node.

Change-Id: Ib318224a09c1b0b748ab31e1ed507975b3190784
2016-02-24 10:05:28 -08:00
Clark Boylan
cda04ed170 Support config drive when using shade-launch-node
Some clouds may not have a metadata service and need to retrieve key
info via config drive. Add a flag to specifically request that a config
drive is provided to the instance booted by nova to facilitate this
information injection.

Change-Id: Ic41df5b34ea67ad62949244e064db82410077453
2016-02-23 21:31:31 -08:00
James E. Blair
387d4a2b6a Add a shade launch-node
Created as a new script so that we aren't hosed if this doesn't
work with current providers.

Change-Id: Ia8d35d0acbfb773ca710c9383d9b746e786766bc
Depends-On: I26bc94408441edf067493b7ffd50eebd9dd95e75
2016-01-21 11:45:41 -08:00
Monty Taylor
7699226007 Add script to make ansible groups from patterns
We have a set of hostname patterns which is not a thing that ansible
supports in inventory files. While we can put hostname patterns into
playbooks directly, that does not help us with copying hiera group files
since ansible doesn't know about the groups in site.pp and puppet
doesn't know about the ansible groups.

Instead, do a quick expansion any time the groups.txt file changes and
at the end of launch-node. It will be left to admins to run
expand-groups.sh whenever they delete a node.

Change-Id: I00c60748ddb2d35a3b98f78d828dabebcf065118
2016-01-15 12:13:00 -05:00
Monty Taylor
b23025acaa Fix three nits
These were caught in earlier reviews.

Change-Id: I026678fbbe871ffb138fd2c0dba9e788457da3a0
2015-11-30 22:24:53 -06:00
Monty Taylor
714c934d0c Use OpenStack for inventory instead of puppet
With the puppetmaster not there anymore, we should consume inventory
from OpenStack rather than from puppet.

It turns out that because of the way static and dynamic inventories get
merged, the static file needs to stand alone. SO - if you need to
disable a dynamic host from OpenStack (pretty much all of our hosts) you
need to not only add it to dynamic:children, you need to add an emtpy
group into the static file too, otherwise you'll get an error like:

 root@puppetmaster:~# ansible -i newinv '!disabled' --list-hosts
 ERROR: newinv/static:4: child group is not defined: (jenkins-dev.openstack.org)

Change-Id: Ic6809ed0b7014d7aebd414bf3a342e3a37eb10b6
2015-11-30 22:02:41 -06:00
James E. Blair
747d832dd5 Disable volume option until it is cinderified.
Change-Id: I82597ef84785a570b37bfc0b5e24c129d9a3b3b9
2015-09-17 10:56:34 -07:00
James E. Blair
3fc24b6b7f Add support for 'admin' user for OVH
OVH images have, of course, yet another 'root' user they would like
us to use instead of root.

Change-Id: If8cc3d31862e352eed40fc041a4c6c695a35063a
2015-09-16 10:32:17 -07:00
James E. Blair
e186820ca5 Handle OVH public ips in launch-node
This is a quick hack to handle public ip addrs on OVH.  A better
fix is porting to shade or ansible.

Change-Id: I62bba7215249521307be5b42af1c6f8fc886d982
2015-09-16 10:24:48 -07:00
James E. Blair
7eab5a6f12 Make OS_SERVICE_NAME optional in launch-node
This parameter is usually not necessary, and over-specifying it
can be problematic because the value is not consistent.

Change-Id: I0a90631499294e7a6eb287f24739cf4884a8db7b
2015-09-16 10:08:02 -07:00
Clark Boylan
871cefc65f Handle ssh failures with reboot in launch-node.py
New systemd based distros reboot so quickly that the ssh connection
errors returning 255 (or -1 in python because signed integers). Ignore
return codes of -1 when rebooting over ssh as a result. All other return
codes will be propogated properly.

Change-Id: I272f00e9e07f1ed04f2b97d0e1609c6e8d49caf3
2015-08-28 16:30:39 -07:00
James E. Blair
0f4690f04b Add support for boot from volume to launch-node
This is needed for rax compute and io flavors.

Change-Id: I20a6d7161a62e28c47031041d6e42b25d304fc46
2015-06-19 17:29:00 -07:00