Etcd as a base service

There have been a lot of discussions around scenarios like
distributed locks, storing additional configuration information
and coordinating service liveness checks without using a database
etc. Example:
http://lists.openstack.org/pipermail/openstack-dev/2015-November/thread.html#78214
http://lists.openstack.org/pipermail/openstack-dev/2017-March/thread.html#113941
http://lists.openstack.org/pipermail/openstack-dev/2017-March/thread.html#113885

So at the Boston Summit Forum, we had a session on a specific proposal
to consider Etcd 3.x for such scenarios:
https://etherpad.openstack.org/p/BOS-etcd-base-service

There was consensus in the room based on which a bunch of work
was done in tooz, devstack etc after thorough research on the
options available for both eventlet and non-eventlet based services.

The consensus grew around folks using oslo library like tooz or if
they choose, they can directly use etcd using python client
libraries like etcd3 (for non-eventlet scenarios) and etcd3gw (for
eventlet based scenarios). We should be using the v3 APIs using
the etcd 3.1.x versions to avoid migration issues from the older
v2 APIs.

Change-Id: Id6dd2a8fffd50dca8f0de05def20d31bf9c61717

reference/base-services.rst

@@ -25,6 +25,11 @@ Current list of base services
   as an indirection layer. While most OpenStack deployments use RabbitMQ,
   other message queues are supported.
 
+**Etcd**
+  OpenStack components may use Etcd, a distributed reliable key-value store
+  for distributed key locking, storing configuration, keeping track of
+  service live-ness and other scenarios.
+
 **Keystone**
   Keystone handles AuthN/AuthZ for OpenStack components.
   Deployments can assume that Keystone will be present to perform that role.