Fix cinderlib Python package

Setuptools gives 2 errors with the current cinderlib Python package:

- Project's description-file has an unsupported directive.
- test-requirements has a github reference.

We fix the first issue by removing the git repository from the test
requirements and adding it to the Tox file while adding cinder to the
required-projects to allow cross-repo denedencies.

We also add os-brick since we are currently also very dependent on it on
the nos_brick.py code.

For the second issue we create a new DESCRIPTION.rst file that doesn't
include the "raw" directive unsupported by PyPi.

This patch will allow us to add the publish-to-pypi job to the
project-config repository.

Required-By: I9d8d9f7d802a27c3bfe3dbc0ee84cc24eee24ebc
Change-Id: Id029cedef513753700a10bcae761fdc1807964e1

.zuul.yaml

@@ -1,33 +1,58 @@
 - project:
     templates:
-      # TODO(geguileo): Uncomment publish-to-pypi once we release Cinder on
-      # PyPi and we can add it to test-requirements instead of the git repo.
-      # - publish-to-pypi
-      # PEP8 + PY27 + Post branch-tarball
-      - openstack-python-jobs
-      - openstack-python36-jobs
       - publish-openstack-docs-pti
       - release-notes-jobs-python3
     check:
       queue: integrated
       jobs:
+        - openstack-tox-pep8
+        - cinderlib-tox-py27
+        - cinderlib-tox-py36
+
         - cinderlib-lvm-functional
         - cinderlib-ceph-functional
     gate:
       queue: integrated
       jobs:
+        - openstack-tox-pep8
+        - cinderlib-tox-py27
+        - cinderlib-tox-py36
+
         - cinderlib-lvm-functional
         - cinderlib-ceph-functional
+    post:
+      jobs:
+        - publish-openstack-python-branch-tarball
+
+- job:
+    name: cinderlib-tox-py27
+    parent: openstack-tox-py27
+    required-projects:
+      - openstack/os-brick
+      - openstack/cinder
+
+- job:
+    name: cinderlib-tox-py36
+    parent: openstack-tox-py36
+    required-projects:
+      - openstack/os-brick
+      - openstack/cinder
 
 - job:
     name: cinderlib-lvm-functional
     parent: openstack-tox-functional-with-sudo
+    required-projects:
+      - openstack/os-brick
+      - openstack/cinder
     pre-run: playbooks/setup-lvm.yaml
     nodeset: centos-7
 
 - job:
     name: cinderlib-ceph-functional
     parent: openstack-tox-functional-with-sudo
+    required-projects:
+      - openstack/os-brick
+      - openstack/cinder
     pre-run: playbooks/setup-ceph.yaml
     nodeset: centos-7
     vars:

DESCRIPTION.rst

@@ -0,0 +1,49 @@
+The Cinder Library, also known as cinderlib, is a Python library that leverages
+the Cinder project to provide an object oriented abstraction around Cinder's
+storage drivers to allow their usage directly without running any of the Cinder
+services or surrounding services, such as KeyStone, MySQL or RabbitMQ.
+
+* Free software: Apache Software License 2.0
+* Documentation: https://docs.openstack.org/cinderlib/latest/
+
+The library is intended for developers who only need the basic CRUD
+functionality of the drivers and don't care for all the additional features
+Cinder provides such as quotas, replication, multi-tenancy, migrations,
+retyping, scheduling, backups, authorization, authentication, REST API, etc.
+
+The library was originally created as an external project, so it didn't have
+the broad range of backend testing Cinder does, and only a limited number of
+drivers were validated at the time.  Drivers should work out of the box, and
+we'll keep a list of drivers that have added the cinderlib functional tests to
+the driver gates confirming they work and ensuring they will keep working.
+
+Features
+--------
+
+* Use a Cinder driver without running a DBMS, Message broker, or Cinder
+  service.
+* Using multiple simultaneous drivers on the same application.
+* Basic operations support:
+
+  - Create volume
+  - Delete volume
+  - Extend volume
+  - Clone volume
+  - Create snapshot
+  - Delete snapshot
+  - Create volume from snapshot
+  - Connect volume
+  - Disconnect volume
+  - Local attach
+  - Local detach
+  - Validate connector
+  - Extra Specs for specific backend functionality.
+  - Backend QoS
+  - Multi-pool support
+
+* Metadata persistence plugins:
+
+  - Stateless: Caller stores JSON serialization.
+  - Database: Metadata is stored in a database: MySQL, PostgreSQL, SQLite...
+  - Custom plugin: Caller provides module to store Metadata and cinderlib calls
+    it when necessary.

README.rst

@@ -1,5 +1,5 @@
 Cinder Library
-===============================
+==============
 
 .. image:: https://img.shields.io/pypi/v/cinderlib.svg
    :target: https://pypi.python.org/pypi/cinderlib

setup.cfg

@@ -8,7 +8,7 @@ exclude = .git,.venv,.tox,dist,doc,*egg,build
 name = cinderlib
 summary = Direct usage of Cinder Block Storage drivers without the services
 description-file =
-    README.rst
+    DESCRIPTION.rst
 author = OpenStack
 author-email = openstack-discuss@lists.openstack.org
 home-page = https://docs.openstack.org/cinderlib/latest/

test-requirements.txt

@@ -10,5 +10,3 @@ oslotest>=3.2.0 # Apache-2.0
 testscenarios>=0.4 # Apache-2.0/BSD
 testtools>=2.2.0 # MIT
 stestr>=1.0.0 # Apache-2.0
-# There's no current Cinder PyPi package we can use
-git+git://github.com/openstack/cinder.git

tox.ini

@@ -11,8 +11,13 @@ setenv = OS_STDOUT_CAPTURE=1
          OS_TEST_PATH=./cinderlib/tests/unit
 usedevelop=True
 install_command = pip install {opts} {packages}
-deps= -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt}
-      -r{toxinidir}/test-requirements.txt
+# Use cinder from master instead of from PyPi. Defining the egg name we won't
+# overwrite the package installed by Zuul on jobs supporting cross-project
+# dependencies (include Cinder in required-projects).  This allows us to also
+# run local tests against master.
+deps= -r{toxinidir}/test-requirements.txt
+      git+git://git.openstack.org/openstack/os-brick#egg=os-brick
+      git+git://git.openstack.org/openstack/cinder#egg=cinder
 
 commands =
     find . -ignore_readdir_race -type f -name "*.pyc" -delete