Use groups.txt for disabling hosts for puppet

We already have a dynamic system for managing static group management.
Use it for the disabled group so that the rules for managing the members
are not different.

Also, update the disabled list to match reality.

Also, Update docs because hosts are no longer groups

The upstream OpenStack Inventory in Ansible was fixed to no longer
return each cloud host as its own group unless there are duplicates for
the host in question. This  means it's no longer the right thing to do
to put hosts into disabled:children - disabled is just fine.

Change-Id: I95c83ed64801db15ad99a14547895f3520356f99

doc/source/sysadmin.rst

@@ -301,33 +301,37 @@ is essential for being able to make informed decisions about actions
 to take.
 
 In the case of needing to disable the running of puppet on a node, it's a
-simple matter of adding an entry to the ansible inventory "disabled" group.
+simple matter of adding an entry to the ansible inventory "disabled" group
-There are two inventory files available for this, `/etc/ansible/hosts/static`
+in `:file:modules/openstack_project/files/puppetmaster/groups.txt`. The
-and `/etc/ansible/hosts/emergency`. `/etc/ansible/hosts/static` is intended
+disabled entry is an input to `ansible --list-hosts` so you can check your
-to be managed via git from the system-config repo in
+entry simply by running it with `ansible $hostlist --list-hosts` as root
-`modules/openstack_project/files/puppetmaster/static-inventory`.
+on the puppetmaster host and ensuring that the list of hosts returned is as
+expected. Globs, group names and server UUIDs should all be acceptable input.
+
+If you need to disable a host immediately without waiting for a patch to land
+to `system-config`, there is a file on the puppetmaster host,
+`/etc/ansible/hosts/emergency` that can be edited directly.
+
 `/etc/ansible/hosts/emergency` is a file that should normally be empty, but
 the contents are not managed by puppet. It's purpose is to allow for disabling
 puppet at times when landing a change to the puppet repo would be either
 unreasonable or impossible.
 
-There are two sections in each file, `disabled` and `disabled:children`. Due
+There are two sections in the emergency file, `disabled` and
-to the multi-cloud nature of the ansible inventory, a hostname cannot be counted
+`disabled:children`. To disable a single host, put it in `disabled`. If you
-on to be unique, so each cloud instance is listed in the inventory by its
+want to disable a group of hosts, put it in `disabled:children`. Any hosts we
-UUID with a group created for its hostname. If you want to disable a cloud
+have that have more than one host with the same name (such as in the case of
-instance by name, you need to put its name in `disabled:children`. If you want
+being in the midst of a migration) will show up as a group with the name of
-to refer to a single instance by UUID, or if there are statically defined
+the hostname and the individual servers will be listed by UUID.
-hosts that need to be disabled, you should put those in `disabled`.
 
 Because of the way static and dynamic inventories get merged by ansible, the
-static file needs to stand alone. If you need to disable a dynamic host from
+emergency file needs to stand alone. If you need to disable a group of servers
-OpenStack (pretty much all of our hosts) you need to not only add it to
+from OpenStack you need to not only add it to `disabled:children`, you need to
-disabled:children, you need to add an emtpy group into the inventory file
+add an emtpy group into the emergency file too.
-(either `static` or `emergency` as appropriate) too.
 
 Disabling puppet via ansible inventory does not disable puppet from being
-run directly on the host, it merely prevents the puppetmaster from causing
+able to be run directly on the host, it merely prevents ansible from
-puppet to be run. If you choose to run puppet manually on a host, take care
+attempting to run it. If you choose to run puppet manually on a host, take care
 to ensure that it has not been disabled at the puppetmaster level first.
 
 Examples
@@ -339,9 +343,7 @@ without landing a puppet change, ensure the following is in
 
 ::
 
-  [amazing.openstack.org]
+  [disabled]
-
-  [disabled:children]
   amazing.openstack.org
 
 To disable one of the OpenStack instances called `git.openstack.org`
@@ -353,13 +355,20 @@ find its UUID via OpenStack tools and ensure it's in the emergency file.
   [disabled]
   811c5197-dba7-4d3a-a3f6-68ca5328b9a7
 
-To disable a staticly defined host that is not an OpenStack host, such as
+To disable a group of hosts in the emergency file, such as all of the pypi
-the Infra cloud controller hosts.
+hosts.
 
 ::
 
-  [disabled]
+  [disabled:children]
-  controller.useast.openstack.org
+  pypi
+
+To disable a staticly defined host that is not an OpenStack host, such as
+the Infra cloud controller hosts, put the following in groups.txt.
+
+::
+
+  disabled controller.useast.openstack.org
 
 .. _cinder:
 

modules/openstack_project/files/puppetmaster/expand-groups.sh

@@ -25,7 +25,7 @@ for line in $(</etc/ansible/groups.txt); do
     name=$(echo $line | cut -f1 -d' ')
     pattern=$(echo $line | cut -f2 -d' ')
     echo "[${name}]" >> $outfile
-    ansible "~${pattern}" --list-hosts | egrep -v '^ +hosts \([0-9]+\):' >> $outfile
+    ansible "${pattern}" --list-hosts | egrep -v '^ +hosts \([0-9]+\):' >> $outfile
 done
 
 cp $outfile /etc/ansible/hosts/generated-groups

modules/openstack_project/files/puppetmaster/groups.txt

@@ -1,11 +1,12 @@
-jenkins jenkins.*\.openstack\.org
+jenkins jenkins*.openstack.org
-logstash-worker logstash-worker\d+\.openstack\.org
+logstash-worker ~logstash-worker\d+\.openstack\.org
-subunit-worker subunit-worker\d+\.openstack\.org
+subunit-worker ~subunit-worker\d+\.openstack\.org
-elasticsearch elasticsearch0[1-7]\.openstack\.org
+elasticsearch ~elasticsearch0[1-7]\.openstack\.org
-git-loadbalancer git(-fe\d+)?\.openstack\.org
+git-loadbalancer ~git(-fe\d+)?\.openstack\.org
-git-server git\d+\.openstack\.org
+git-server ~git\d+\.openstack\.org
-pypi pypi\..*\.openstack\.org
+pypi pypi.*.openstack.org
-zuul-merger zm\d+\.openstack\.org
+zuul-merger ~zm\d+\.openstack\.org
-ci-backup ci-backup-.*\.openstack\.org
+ci-backup ci-backup-*.openstack.org
-afsdb afsdb.*\.openstack\.org
+afsdb afsdb*.openstack.org
-afs afs.*\..*\.openstack\.org
+afs afs*.*.openstack.org
+disabled ci-backup-rs-ord.openstack.org:jenkins-dev.openstack.org

modules/openstack_project/files/puppetmaster/static-inventory

@@ -1,9 +0,0 @@
-[disabled]
-ci-backup-rs-ord.openstack.org
-jenkins-dev.openstack.org
-hound.openstack.org
-pypi.bhs1.openstack.org
-pypi.region-b.geo-1.openstack.org
-pypi.sjc1.openstack.org
-review-dev.openstack.org
-subunit-worker01.openstack.org

modules/openstack_project/manifests/puppetmaster.pp

@@ -221,10 +221,7 @@ class openstack_project::puppetmaster (
   }
 
   file { '/etc/ansible/hosts/static':
-    owner   => 'root',
+    ensure => absent,
-    group   => 'root',
-    mode    => '0444',
-    source  => 'puppet:///modules/openstack_project/puppetmaster/static-inventory',
   }
 
   file { '/etc/ansible/hosts/emergency':