Modernize setup.py and add wheel support

* Incremented version to 0.2.2 for pypi release.
  * Removed all post-install hooks.  All initialization and configuration
    should be done through the cafe-config cli tool.
  * Added setup.cfg with support for universal bdist wheel.
  * Updated MANIFEST.in file.
  * Updated README.rst to reflect new install procedures.
  * Modified cafe-config so that initialization command is now just
      "cafe-config init".
  * Since the ".opencafe" directory is no longer initialized at install
    while access to the source code is guaranteed, the plugins are now
    distributed as package_data, and installed as such to site-packages
    under the new "plugins" directory within the "cafe" namespace.
  * The plugins directory is moved to the cafe package directory as
     package_data.
  * The plugin cache is no longer created at initialization, and all
    code relating to it in the cli.py and managers.py file has been
    removed.
  * Removed pip-requires file in favor of including the only requirement,
    'six', in setup.py.  The plan is to refactor so as to remove the
    dependency on six eventually.
  * Renamed test-requirements.txt to test-requires.
  * Added Authors.rst

Change-Id: I28a605f926ae5f2d972a6a36171d0e4eb92cac09

AUTHORS.md

@@ -0,0 +1,21 @@
+Original Authors
+----------------
+Sam Danes
+Jose Idar
+Brandon Logan
+Carlos Martinez
+Daryl Walleck
+
+Contributors
+------------
+Anna Eilering
+Richard Hawkins
+Christopher Hunt
+Michael Jackson
+Melissa Kam
+Malini Kamalambal
+Stephen Lowrie
+Leonardo Maycotte
+Franklin Naval
+Ivo Vasev
+John Vrbanac

MANIFEST.in

@@ -1,2 +1,2 @@
-include README.rst LICENSE pip-requires
-recursive-include plugins *
+include README.rst LICENSE
+recursive-include cafe/plugins *

README.rst

@@ -1,9 +1,7 @@
 Open CAFE Core
-----------------------------
+==============
 
 
-.. code-block::
-
        ( (
           ) )
         .........
@@ -15,38 +13,114 @@ Open CAFE Core
     === CAFE Core ===
 
 
-The Open Common Automation Framework Engine is the core engine/driver used to build an automated testing framework. It is designed to be used as the
-base engine for building an automated framework for API and non-UI resource testing. It is designed to support functional, integration and
-reliability testing. The engine is **NOT** designed to support performance or load testing.
+OpenCAFE, the Open Common Automation Framework Engine, is designed to be used
+as the base for building an automated testing framework for API and other
+(non-UI) testing.
+It is designed to support all kinds of testing methodologies, such as unit,
+functional and integration testing, using a model-based approach.
+Although the engine is not designed with performance or load testing in mind,
+as it prioritizes repeatability and (verbose) logging over performance, it can
+be used to that end.
 
-CAFE core provides a model, a pattern and assorted common tools for building automated tests. It provides its own light weight unittest based
-runner, however, it is designed to be modular. It can be extended to support most test case front ends/runners (nose, pytest, lettuce, testr, etc...)
-through driver plug-ins.
+
+Installation
+============
+>Source code is available at https://github.com/stackforge/opencafe
 
 Supported Operating Systems
 ---------------------------
-Open CAFE Core has been developed primarily in Linux and MAC environments, however, it supports installation and
-execution on Windows
+Open CAFE Core has been developed primarily on and for Linux, but supports
+installation and execution on BSD and other *nix's, as well as OS X and
+modern Windows.  It can be installed from pypi via pip or from source.
 
-Installation
-------------
-Open CAFE Core can be `installed with pip <https://pypi.python.org/pypi/pip>`_ from the git repository after it is cloned to a local machine.
+>**It is recommended that you install OpenCAFE in a python
+virtualenv.**
 
-* Clone this repository to your local machine
-* CD to the root directory in your cloned repository.
-* Run "pip install . --upgrade" and pip will auto install all dependencies.
+via pip
 
-After the CAFE Core is installed you will have command line access to the default unittest runner, the cafe-runner. (See cafe-runner --help for more info)
+    pip install opencafe
 
-Remember, open CAFE is just the core driver/engine. You have to build an implementation and test repository that use it!
+from source
 
-Configuration
+    $ git clone https://github.com/stackforge/opencafe.git
+     $ cd opencafe
+     $ python setup.py install
+
+Post-install Configuration
+==========================
+Post-install, the ``cafe-config`` cli tool will become available.
+It is used for installing
+plugins and initializing the engine's default ``.opencafe`` directory.
+
+Initialization
 --------------
-Open CAFE works out of the box with the cafe-runner (cafe.drivers.unittest). CAFE will auto-generate a base engine.config during installation. This
-base configuration will be installed to: <USER_HOME>/.opencafe/configs/engine.config
+OpenCAFE uses a set of default locations for logging, storing
+test configurations, test data, statistics, and the like; all of which are
+set in, and read from, the ``engine.config`` file (in order to make it easy
+for the end user to override the default behavior).  The ``engine.config``
+file, and the directories it references, can be created on demand by running:
 
-If you wish to modify default installation values you can update the engine.config file after CAFE installation. Keep in mind that the Engine will
-over-write this file on installation/upgrade.
+``cafe-config init``
+
+> This will create a directory named ``.opencafe`` in the user's home
+directory, or in the case of a python virtualenv, in the virtualenv root
+folder.
+
+Installing Plugins
+------------------
+The ``cafe-config plugins`` command is used to list and install plugins.
+
+Example:
+
+    $ cafe-config plugins list
+    =================================
+    * Available Plugins
+      ... http
+      ... mongo
+      ... winrm
+      ... elasticsearch
+      ... soap
+      ... skip_on_issue
+      ... rsyslog
+      ... ssh
+    =================================
+
+    $ cafe-config plugins install http
+    =================================
+    * Installing Plugins
+      ... http
+    =================================
+
+> Note: There is currently no way to reliably have an implementation of
+        OpenCAFE require specific plugins at install time, due to issues
+        with pip=>7.0.0.  We are working on an overhaul of the plugin system
+        that should remedy the situation.
+
+Package Structure Overview
+==========================
+``cafe.common.reporting``
+-------------------------
+Provides tools for logging and reporting.
+This namespace should be used by plugins to add logging and reporting features.
+
+``cafe.configurator``
+---------------------
+Used by the ``cafe-config`` cli tool for setting up new installations of opencafe.
+
+``cafe.drivers``
+----------------
+Houses various test runner wrappers and supporting tools.
+This namespace should be used by plugins to add new test runner support.
+
+``cafe.engine``
+---------------
+Includes the base classes that OpenCAFE implementations will use to create behaviors, configs, clients and models.
+This namespace should be used by plugins to add new clients.
+
+``cafe.resources``
+------------------
+Deprecated.
+Historically contained all modules that reference external resources to OpenCAFE. Currently acts only as a namespace for backward compatability with some plugins.
 
 Terminology
 -----------
@@ -58,7 +132,6 @@ Following are some notes on Open CAFE lingo and concepts.
 * Product
     Anything that's being tested by an implementation of Open CAFE Core. If you would like to see a refernce implementation, there is an `Open Source implementation <https://github.com/stackforge>`_ based on `OpenStack <http://www.openstack.org/>`_.
 
-
 * Client / Client Method
     A **client** is an "at-least-one"-to-"at-most-one" mapping of a product's functionality to a collection of client methods.  Using a `REST API <https://en.wikipedia.org/wiki/Representational_state_transfer>`_ as an example, a client that represents that API in CAFE will contain at least one (but possibly more) method(s) for every function exposed by that API.  Should a call in the API prove to be too difficult or cumbersome to define via a single **client method**, then multiple client methods can be defined such that as a whole they represent the complete set of that API call's functionality. A **client method** should never be a superset of more than one call's functionality.
 
@@ -70,22 +143,3 @@ Following are some notes on Open CAFE lingo and concepts.
 
 * Provider
     This is meant to be a convenience facade that performs configuration of clients and behaviors to provide configuration-based default combinations of different clients and behaviors.
-
-Basic CAFE Package Anatomy
---------------------------
-Below is a short description of the top level CAFE Packages.
-
-* cafe
-    This is the root package. The wellspring from which the CAFE flows...
-
-* common
-    Contains modules common the entire engine. This is the primary namespace for tools, data generators, common reporting classes, etc...
-
-* engine
-    Contains all the base implementations of clients, behaviors, models to be used by a CAFE implementation. It also contains supported generic clients, behaviors and models. For instance, the engine.clients.remote_instance clients are meant to be used directly by an implementation.
-
-* drivers
-    The end result of CAFE is to build an implementation to talk to a particular product or products, and a repository of automated test cases. The drivers package is specifically for building CAFE support for various Python based test runners. There is a default unittest based driver implemented which heavily extends the basic unittest functionality. Driver plug-ins can easily be constructed to add CAFE support for most of the popular ones already available (nose, pytest, lettuce, testr, etc...) or even for 100% custom test case drivers if desired.
-
-* resources
-    Contains all modules that reference external resources to OpenCAFE. One example of an external resource is a Launchpad tracker.

cafe/configurator/cli.py

@@ -18,8 +18,18 @@ from cafe.configurator.managers import (
 
 class EngineActions(object):
 
+    class Init(object):
+        def __init__(self, args):
+            print("=================================")
+            print("* Initializing Engine Install")
+            EngineDirectoryManager.build_engine_directories()
+            EngineConfigManager.build_engine_config()
+            print("=================================")
+
     class InitInstall(argparse.Action):
-        def __call__(self, parser, namespace, values, option_string=None):
+        def __call__(
+                self, parser, namespace, values,
+                option_string=None):
             print("=================================")
             print("* Initializing Engine Install")
             EngineDirectoryManager.build_engine_directories()
@@ -28,22 +38,20 @@ class EngineActions(object):
 
 
 class PluginActions(object):
-    class AddPluginCache(argparse.Action):
-        def __call__(self, parser, namespace, values, option_string=None):
-            print("=================================")
-            print("* Adding Plugin Cache")
-            EnginePluginManager.populate_plugin_cache(values)
-            print("=================================")
 
     class InstallPlugin(argparse.Action):
-        def __call__(self, parser, namespace, values, option_string=None):
+        def __call__(
+                self, parser, namespace, values,
+                option_string=None):
             print("=================================")
             print("* Installing Plugins")
             EnginePluginManager.install_plugins(values)
             print("=================================")
 
     class ListPlugins(argparse.Action):
-        def __call__(self, parser, namespace, values, option_string=None):
+        def __call__(
+                self, parser, namespace, values,
+                option_string=None):
             print("=================================")
             print("* Available Plugins")
             EnginePluginManager.list_plugins()
@@ -51,7 +59,8 @@ class PluginActions(object):
 
 
 class ConfiguratorCLI(object):
-    """CLI for future engine management and configuration options."""
+    """CLI for future engine management and configuration
+    options."""
 
     @classmethod
     def run(cls):
@@ -59,28 +68,42 @@ class ConfiguratorCLI(object):
         subparsers = parser.add_subparsers(dest="subcommand")
 
         # Engine configuration subparser
-        subparser_engine_config = subparsers.add_parser('engine')
+        subparser_engine_config = subparsers.add_parser(
+            'engine')
         subparser_engine_config.add_argument(
-            '--init-install', action=EngineActions.InitInstall, nargs=0)
+            '--init-install',
+            action=EngineActions.InitInstall,
+            nargs=0,
+            help="Deprecated. Please use 'cafe-config init' instead.")
+
+        subparser_init = subparsers.add_parser('init')
+        subparser_init.set_defaults(func=EngineActions.Init)
 
         # Plugin argument subparser
         subparser_plugins = subparsers.add_parser('plugins')
-        plugin_args = subparser_plugins.add_subparsers(dest='plugin_args')
+        plugin_args = subparser_plugins.add_subparsers(
+            dest='plugin_args')
 
-        plugins_add_parser = plugin_args.add_parser('add')
-        plugins_add_parser.add_argument(
-            'plugin_dir', action=PluginActions.AddPluginCache, type=str)
-
-        plugins_add_parser = plugin_args.add_parser('list')
-        plugins_add_parser.add_argument(
-            'list_plugins', action=PluginActions.ListPlugins, nargs=0)
+        plugins_list_parser = plugin_args.add_parser('list')
+        plugins_list_parser.add_argument(
+            'list_plugins',
+            action=PluginActions.ListPlugins,
+            nargs=0)
 
         plugins_install_parser = plugin_args.add_parser('install')
         plugins_install_parser.add_argument(
-            'plugin-name', action=PluginActions.InstallPlugin, type=str,
+            'plugin-name',
+            action=PluginActions.InstallPlugin,
+            type=str,
             nargs='*')
 
-        return parser.parse_args()
+        # parse args and trigger actions
+        args = parser.parse_args()
+
+        try:
+            args.func(args)
+        except AttributeError:
+            return args
 
 
 def entry_point():

cafe/configurator/managers.py

@@ -21,11 +21,10 @@ import textwrap
 import getpass
 import shutil
 from subprocess import Popen, PIPE
-
 from six.moves.configparser import SafeConfigParser
 
-from cafe.engine.config import EngineConfig
 import cafe
+from cafe.engine.config import EngineConfig
 
 if not platform.system().lower() == 'windows':
     import pwd
@@ -408,8 +407,7 @@ class EngineDirectoryManager(object):
         LOG_DIR=os.path.join(OPENCAFE_ROOT_DIR, 'logs'),
         DATA_DIR=os.path.join(OPENCAFE_ROOT_DIR, 'data'),
         TEMP_DIR=os.path.join(OPENCAFE_ROOT_DIR, 'temp'),
-        CONFIG_DIR=os.path.join(OPENCAFE_ROOT_DIR, 'configs'),
-        PLUGIN_CACHE=os.path.join(OPENCAFE_ROOT_DIR, 'plugin_cache'))
+        CONFIG_DIR=os.path.join(OPENCAFE_ROOT_DIR, 'configs'),)
 
     @classmethod
     def create_engine_directories(cls):
@@ -612,38 +610,13 @@ class EngineConfigManager(object):
 
 
 class EnginePluginManager(object):
-
-    @classmethod
-    def copy_plugin_to_cache(
-            cls, plugins_src_dir, plugins_dest_dir, plugin_name):
-        """ Copies an individual plugin to the .opencafe plugin cache """
-        src_plugin_path = os.path.join(plugins_src_dir, plugin_name)
-        dest_plugin_path = os.path.join(plugins_dest_dir, plugin_name)
-
-        if os.path.exists(dest_plugin_path):
-            shutil.rmtree(dest_plugin_path)
-
-        shutil.copytree(src_plugin_path, dest_plugin_path)
-
-    @classmethod
-    def populate_plugin_cache(cls, plugins_src_dir):
-        """ Handles moving all plugin src data from package into the user's
-        .opencafe folder for installation by the cafe-config tool.
-        """
-
-        default_dest = EngineDirectoryManager.OPENCAFE_SUB_DIRS.PLUGIN_CACHE
-        plugins = next(os.walk(plugins_src_dir))[1]
-
-        for plugin_name in plugins:
-            cls.copy_plugin_to_cache(
-                plugins_src_dir, default_dest, plugin_name)
+    _PLUGIN_DIR = os.path.join(os.path.dirname(cafe.__file__), 'plugins')
 
     @classmethod
     def list_plugins(cls):
         """ Lists all plugins currently available in user's .opencafe cache"""
 
-        plugin_cache = EngineDirectoryManager.OPENCAFE_SUB_DIRS.PLUGIN_CACHE
-        plugin_folders = os.walk(plugin_cache).next()[1]
+        plugin_folders = os.walk(cls._PLUGIN_DIR).next()[1]
         wrap = textwrap.TextWrapper(initial_indent="  ",
                                     subsequent_indent="  ",
                                     break_long_words=False).fill
@@ -664,8 +637,7 @@ class EnginePluginManager(object):
     def install_plugin(cls, plugin_name):
         """ Install a single plugin by name into the current environment"""
 
-        plugin_cache = EngineDirectoryManager.OPENCAFE_SUB_DIRS.PLUGIN_CACHE
-        plugin_dir = os.path.join(plugin_cache, plugin_name)
+        plugin_dir = os.path.join(cls._PLUGIN_DIR, plugin_name)
         wrap = textwrap.TextWrapper(initial_indent="  ",
                                     subsequent_indent="  ",
                                     break_long_words=False).fill

pip-requires

@@ -1 +0,0 @@
-six

setup.cfg

@@ -0,0 +1,2 @@
+[bdist_wheel]
+universal=1

setup.py

@@ -14,62 +14,12 @@ See the License for the specific language governing permissions and
 limitations under the License.
 """
 
+import os
 import sys
-from subprocess import call
 from setuptools import setup, find_packages
-from setuptools.command import easy_install as _easy_install
-from setuptools.command.install import install as _install
 from setuptools.command.test import test as TestCommand
 
 
-# Post-install engine configuration
-def _post_install(dir):
-    call(['cafe-config', 'engine', '--init-install'])
-    call(['cafe-config', 'plugins', 'add', 'plugins'])
-    call(['cafe-config', 'plugins', 'install', 'http'])
-    print(
-        """
-         ( (
-            ) )
-        ........
-        |      |___
-        |      |_  |
-        |  :-) |_| |
-        |      |___|
-        |______|
-    === OpenCAFE ===
-
-    -----------------------------------------------------------------
-    If you wish to install additional plugins, you can do so through
-    the cafe-config tool.
-
-    Example:
-    $ cafe-config plugins install mongo
-    -----------------------------------------------------------------
-    """)
-
-# Read in requirements
-requires = open('pip-requires').readlines()
-
-# Add additional requires for Python 2.6 support
-if sys.version_info < (2, 7):
-    oldpy_requires = ['argparse>=1.2.1', 'unittest2>=0.5.1', 'ordereddict']
-    requires.extend(oldpy_requires)
-
-
-# cmdclass hook allows setup to make post install call
-class install(_install):
-    def run(self):
-
-        # Workaround for problem in six that prevents installation when part of
-        # of some package requirements
-        _easy_install.main(['-U', 'six'])
-        _install.run(self)
-        self.execute(
-            _post_install, (self.install_lib,),
-            msg="\nRunning post install tasks...")
-
-
 # tox integration
 class Tox(TestCommand):
     def finalize_options(self):
@@ -83,20 +33,29 @@ class Tox(TestCommand):
         errno = tox.cmdline(self.test_args)
         sys.exit(errno)
 
-# Normal setup stuff
+# Package the plugin cache as package data
+plugins = []
+dir_path = os.path.join(os.path.dirname(__file__), 'cafe', 'plugins')
+for dirpath, directories, filenames in os.walk(dir_path):
+    dirpath = dirpath.lstrip('cafe/')
+    for f in filenames:
+        if f.endswith('.pyc'):
+            continue
+        target_file = os.path.join(dirpath, f)
+        plugins.append(target_file)
+
 setup(
     name='opencafe',
-    version='0.2.1',
+    version='0.2.2',
     description='The Common Automation Framework Engine',
     long_description='{0}'.format(open('README.rst').read()),
     author='CafeHub',
     author_email='cloud-cafe@lists.rackspace.com',
     url='http://opencafe.readthedocs.org',
-    install_requires=requires,
-    packages=find_packages(),
-    namespace_packages=['cafe'],
+    install_requires=['six'],
+    packages=find_packages(exclude=('tests*', 'docs')),
+    package_data={'cafe': plugins},
     license=open('LICENSE').read(),
-    zip_safe=False,
     classifiers=(
         'Development Status :: 4 - Beta',
         'Intended Audience :: Developers',
@@ -115,6 +74,4 @@ setup(
          'specter-runner = cafe.drivers.specter.runner:entry_point',
          'cafe-config = cafe.configurator.cli:entry_point']},
     tests_require=['tox'],
-    cmdclass={
-        'install': install,
-        'test': Tox})
+    cmdclass={'test': Tox})

test-requires

@@ -1,4 +1,5 @@
 tox
 mock
 flake8
-nose
+nose
+virtualenv

tests/common/reporting/test_cclogging.py

@@ -1,10 +1,8 @@
 import unittest
 import mock
 from uuid import uuid4
-from requests import Response
 import logging
 from cafe.common.reporting.cclogging import getLogger as CC_getLogger
-from cafe.common.reporting.cclogging import init_root_log_handler as IRLH
 
 mock_dict = dict(
     CAFE_TEST_LOG_PATH='/tmp',

tox.ini

@@ -3,8 +3,7 @@ envlist=pep8,py27,py34
 
 [testenv]
 setenv=VIRTUAL_ENV={envdir}
-deps=-r{toxinidir}/pip-requires
-     -r{toxinidir}/test-requirements.txt
+deps=-r{toxinidir}/test-requires
 
 [testenv:py27]
 commands=nosetests
@@ -17,4 +16,4 @@ commands=flake8
 
 [flake8]
 ignore=F401,E402
-exclude=.git,.idea,docs,.tox,bin,dist,tools,*.egg-info
+exclude=.git,.idea,docs,.tox,bin,dist,tools,*.egg-info