openstack/pbr
Code Issues Proposed changes

Clean up hacking and path issues with d2to1

Browse Source 
We imported in another code base. It takes a little bit of path
adjusting to get things right.

Change-Id: Ifb96652c822d5d243a6bedb77bc34e919be2d3a8
This commit is contained in:
Monty Taylor 2013-07-05 16:39:50 -04:00
parent 295dbe1da3
commit 746b78d637
26 changed files with 555 additions and 1253 deletions

2
.mailmap

@ -1,4 +1,6 @@
# Format is: # Format is:
# <preferred e-mail> <other e-mail 1> # <preferred e-mail> <other e-mail 1>
# <preferred e-mail> <other e-mail 2> # <preferred e-mail> <other e-mail 2>
Davanum Srinivas <dims@linux.vnet.ibm.com> <davanum@gmail.com>
Erik M. Bray <embray@stsci.edu> Erik Bray <embray@stsci.edu>
Zhongyue Luo <zhongyue.nah@intel.com> <lzyeval@gmail.com> Zhongyue Luo <zhongyue.nah@intel.com> <lzyeval@gmail.com>

23
README.rst

@ -13,9 +13,20 @@ it's simple and repeatable. If you want to do things differently, cool! But
you've already got the power of python at your fingertips, so you don't you've already got the power of python at your fingertips, so you don't
really need PBR. really need PBR.
PBR builds on top of `d2to1` to provide for declarative configuration. It PBR builds on top of the work that `d2to1` started to provide for declarative
then filters the `setup.cfg` data through a setup hook to fill in default configuration. `d2to1` is itself an implementation of the ideas behind
values and provide more sensible behaviors. `distutils2`. Although `distutils2` is now abandoned in favor of work towards
PEP 426 and Metadata 2.0, declarative config is still a great idea and
specifically important in trying to distribute setup code as a library
when that library itself will alter how the setup is processed. As Metadata
2.0 and other modern Python packaging PEPs come out, `pbr` aims to support
them as quickly as possible.
`pbr` reads and then filters the `setup.cfg` data through a setup hook to
fill in default values and provide more sensible behaviors, and then feeds
the results in as the arguments to a call to `setup.py` - so the heavy
lifting of handling python packaging needs is still being done by
`setuptools`.
Behaviors Behaviors
========= =========
@ -124,11 +135,11 @@ The minimal setup.py should look something like this::
from setuptools import setup from setuptools import setup
setup( setup(
setup_requires=['d2to1', 'pbr'], setup_requires=['pbr'],
d2to1=True, pbr=True,
) )
Note that it's important to specify `d2to1=True` or else the pbr functionality Note that it's important to specify `pbr=True` or else the pbr functionality
will not be enabled. will not be enabled.
It should also work fine if additional arguments are passed to `setup()`, It should also work fine if additional arguments are passed to `setup()`,

127
pbr/core.py Normal file

@ -0,0 +1,127 @@
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
from distutils import core
from distutils import errors
import os
import sys
import warnings
from setuptools import dist
from pbr.d2to1 import util
core.Distribution = dist._get_unpatched(core.Distribution)
if sys.version_info[0] == 3:
string_type = str
integer_types = int
else:
string_type = basestring
integer_types = (int, long)
def pbr(dist, attr, value):
"""Implements the actual pbr setup() keyword. When used, this should be
the only keyword in your setup() aside from `setup_requires`.
If given as a string, the value of pbr is assumed to be the relative path
to the setup.cfg file to use. Otherwise, if it evaluates to true, it
simply assumes that pbr should be used, and the default 'setup.cfg' is
used.
This works by reading the setup.cfg file, parsing out the supported
metadata and command options, and using them to rebuild the
`DistributionMetadata` object and set the newly added command options.
The reason for doing things this way is that a custom `Distribution` class
will not play nicely with setup_requires; however, this implementation may
not work well with distributions that do use a `Distribution` subclass.
"""
if not value:
return
if isinstance(value, string_type):
path = os.path.abspath(value)
else:
path = os.path.abspath('setup.cfg')
if not os.path.exists(path):
raise errors.DistutilsFileError(
'The setup.cfg file %s does not exist.' % path)
# Converts the setup.cfg file to setup() arguments
try:
attrs = util.cfg_to_args(path)
except Exception:
e = sys.exc_info()[1]
raise errors.DistutilsSetupError(
'Error parsing %s: %s: %s' % (path, e.__class__.__name__, e))
# Repeat some of the Distribution initialization code with the newly
# provided attrs
if attrs:
# Skips 'options' and 'licence' support which are rarely used; may add
# back in later if demanded
for key, val in attrs.items():
if hasattr(dist.metadata, 'set_' + key):
getattr(dist.metadata, 'set_' + key)(val)
elif hasattr(dist.metadata, key):
setattr(dist.metadata, key, val)
elif hasattr(dist, key):
setattr(dist, key, val)
else:
msg = 'Unknown distribution option: %s' % repr(key)
warnings.warn(msg)
# Re-finalize the underlying Distribution
core.Distribution.finalize_options(dist)
# This bit comes out of distribute/setuptools
if isinstance(dist.metadata.version, integer_types + (float,)):
# Some people apparently take "version number" too literally :)
dist.metadata.version = str(dist.metadata.version)
# This bit of hackery is necessary so that the Distribution will ignore
# normally unsupport command options (namely pre-hooks and post-hooks).
# dist.command_options is normally a dict mapping command names to dicts of
# their options. Now it will be a defaultdict that returns IgnoreDicts for
# the each command's options so we can pass through the unsupported options
ignore = ['pre_hook.*', 'post_hook.*']
dist.command_options = util.DefaultGetDict(lambda: util.IgnoreDict(ignore))

4
pbr/d2to1/__init__.py

@ -1,4 +0,0 @@
try:
__version__ = __import__('pkg_resources').get_distribution('d2to1').version
except:
__version__ = ''

82
pbr/d2to1/core.py

@ -1,82 +0,0 @@
import os
import sys
import warnings
from distutils.core import Distribution as _Distribution
from distutils.errors import DistutilsFileError, DistutilsSetupError
from setuptools.dist import _get_unpatched
from .extern import six
from .util import DefaultGetDict, IgnoreDict, cfg_to_args
_Distribution = _get_unpatched(_Distribution)
def d2to1(dist, attr, value):
"""Implements the actual d2to1 setup() keyword. When used, this should be
the only keyword in your setup() aside from `setup_requires`.
If given as a string, the value of d2to1 is assumed to be the relative path
to the setup.cfg file to use. Otherwise, if it evaluates to true, it
simply assumes that d2to1 should be used, and the default 'setup.cfg' is
used.
This works by reading the setup.cfg file, parsing out the supported
metadata and command options, and using them to rebuild the
`DistributionMetadata` object and set the newly added command options.
The reason for doing things this way is that a custom `Distribution` class
will not play nicely with setup_requires; however, this implementation may
not work well with distributions that do use a `Distribution` subclass.
"""
if not value:
return
if isinstance(value, six.string_types):
path = os.path.abspath(value)
else:
path = os.path.abspath('setup.cfg')
if not os.path.exists(path):
raise DistutilsFileError(
'The setup.cfg file %s does not exist.' % path)
# Converts the setup.cfg file to setup() arguments
try:
attrs = cfg_to_args(path)
except:
e = sys.exc_info()[1]
raise DistutilsSetupError(
'Error parsing %s: %s: %s' % (path, e.__class__.__name__, e))
# Repeat some of the Distribution initialization code with the newly
# provided attrs
if attrs:
# Skips 'options' and 'licence' support which are rarely used; may add
# back in later if demanded
for key, val in six.iteritems(attrs):
if hasattr(dist.metadata, 'set_' + key):
getattr(dist.metadata, 'set_' + key)(val)
elif hasattr(dist.metadata, key):
setattr(dist.metadata, key, val)
elif hasattr(dist, key):
setattr(dist, key, val)
else:
msg = 'Unknown distribution option: %s' % repr(key)
warnings.warn(msg)
# Re-finalize the underlying Distribution
_Distribution.finalize_options(dist)
# This bit comes out of distribute/setuptools
if isinstance(dist.metadata.version, six.integer_types + (float,)):
# Some people apparently take "version number" too literally :)
dist.metadata.version = str(dist.metadata.version)
# This bit of hackery is necessary so that the Distribution will ignore
# normally unsupport command options (namely pre-hooks and post-hooks).
# dist.command_options is normally a dict mapping command names to dicts of
# their options. Now it will be a defaultdict that returns IgnoreDicts for
# the each command's options so we can pass through the unsupported options
ignore = ['pre_hook.*', 'post_hook.*']
dist.command_options = DefaultGetDict(lambda: IgnoreDict(ignore))

0
pbr/d2to1/extern/__init__.py vendored

386
pbr/d2to1/extern/six.py vendored

@ -1,386 +0,0 @@
# Copyright (c) 2010-2011 Benjamin Peterson
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
"""Utilities for writing code that runs on Python 2 and 3"""
import operator
import sys
import types
__author__ = "Benjamin Peterson <benjamin@python.org>"
__version__ = "1.2.0"
# True if we are running on Python 3.
PY3 = sys.version_info[0] == 3
if PY3:
string_types = str,
integer_types = int,
class_types = type,
text_type = str
binary_type = bytes
MAXSIZE = sys.maxsize
else:
string_types = basestring,
integer_types = (int, long)
class_types = (type, types.ClassType)
text_type = unicode
binary_type = str
if sys.platform == "java":
# Jython always uses 32 bits.
MAXSIZE = int((1 << 31) - 1)
else:
# It's possible to have sizeof(long) != sizeof(Py_ssize_t).
class X(object):
def __len__(self):
return 1 << 31
try:
len(X())
except OverflowError:
# 32-bit
MAXSIZE = int((1 << 31) - 1)
else:
# 64-bit
MAXSIZE = int((1 << 63) - 1)
del X
def _add_doc(func, doc):
"""Add documentation to a function."""
func.__doc__ = doc
def _import_module(name):
"""Import module, returning the module after the last dot."""
__import__(name)
return sys.modules[name]
class _LazyDescr(object):
def __init__(self, name):
self.name = name
def __get__(self, obj, tp):
result = self._resolve()
setattr(obj, self.name, result)
# This is a bit ugly, but it avoids running this again.
delattr(tp, self.name)
return result
class MovedModule(_LazyDescr):
def __init__(self, name, old, new=None):
super(MovedModule, self).__init__(name)
if PY3:
if new is None:
new = name
self.mod = new
else:
self.mod = old
def _resolve(self):
return _import_module(self.mod)
class MovedAttribute(_LazyDescr):
def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
super(MovedAttribute, self).__init__(name)
if PY3:
if new_mod is None:
new_mod = name
self.mod = new_mod
if new_attr is None:
if old_attr is None:
new_attr = name
else:
new_attr = old_attr
self.attr = new_attr
else:
self.mod = old_mod
if old_attr is None:
old_attr = name
self.attr = old_attr
def _resolve(self):
module = _import_module(self.mod)
return getattr(module, self.attr)
class _MovedItems(types.ModuleType):
"""Lazy loading of moved objects"""
_moved_attributes = [
MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"),
MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
MovedAttribute("map", "itertools", "builtins", "imap", "map"),
MovedAttribute("reload_module", "__builtin__", "imp", "reload"),
MovedAttribute("reduce", "__builtin__", "functools"),
MovedAttribute("StringIO", "StringIO", "io"),
MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
MovedModule("builtins", "__builtin__"),
MovedModule("configparser", "ConfigParser"),
MovedModule("copyreg", "copy_reg"),
MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
MovedModule("http_cookies", "Cookie", "http.cookies"),
MovedModule("html_entities", "htmlentitydefs", "html.entities"),
MovedModule("html_parser", "HTMLParser", "html.parser"),
MovedModule("http_client", "httplib", "http.client"),
MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"),
MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"),
MovedModule("cPickle", "cPickle", "pickle"),
MovedModule("queue", "Queue"),
MovedModule("reprlib", "repr"),
MovedModule("socketserver", "SocketServer"),
MovedModule("tkinter", "Tkinter"),
MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
MovedModule("tkinter_colorchooser", "tkColorChooser",
"tkinter.colorchooser"),
MovedModule("tkinter_commondialog", "tkCommonDialog",
"tkinter.commondialog"),
MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"),
MovedModule("tkinter_font", "tkFont", "tkinter.font"),
MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"),
MovedModule("tkinter_tksimpledialog", "tkSimpleDialog",
"tkinter.simpledialog"),
MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
MovedModule("winreg", "_winreg"),
]
for attr in _moved_attributes:
setattr(_MovedItems, attr.name, attr)
del attr
moves = sys.modules["six.moves"] = _MovedItems("moves")
def add_move(move):
"""Add an item to six.moves."""
setattr(_MovedItems, move.name, move)
def remove_move(name):
"""Remove item from six.moves."""
try:
delattr(_MovedItems, name)
except AttributeError:
try:
del moves.__dict__[name]
except KeyError:
raise AttributeError("no such move, %r" % (name,))
if PY3:
_meth_func = "__func__"
_meth_self = "__self__"
_func_code = "__code__"
_func_defaults = "__defaults__"
_iterkeys = "keys"
_itervalues = "values"
_iteritems = "items"
else:
_meth_func = "im_func"
_meth_self = "im_self"
_func_code = "func_code"
_func_defaults = "func_defaults"
_iterkeys = "iterkeys"
_itervalues = "itervalues"
_iteritems = "iteritems"
try:
advance_iterator = next
except NameError:
def advance_iterator(it):
return it.next()
next = advance_iterator
if PY3:
def get_unbound_function(unbound):
return unbound
Iterator = object
def callable(obj):
return any("__call__" in klass.__dict__ for klass in type(obj).__mro__)
else:
def get_unbound_function(unbound):
return unbound.im_func
class Iterator(object):
def next(self):
return type(self).__next__(self)
callable = callable
_add_doc(get_unbound_function,
"""Get the function out of a possibly unbound function""")
get_method_function = operator.attrgetter(_meth_func)
get_method_self = operator.attrgetter(_meth_self)
get_function_code = operator.attrgetter(_func_code)
get_function_defaults = operator.attrgetter(_func_defaults)
def iterkeys(d):
"""Return an iterator over the keys of a dictionary."""
return iter(getattr(d, _iterkeys)())
def itervalues(d):
"""Return an iterator over the values of a dictionary."""
return iter(getattr(d, _itervalues)())
def iteritems(d):
"""Return an iterator over the (key, value) pairs of a dictionary."""
return iter(getattr(d, _iteritems)())
if PY3:
def b(s):
return s.encode("latin-1")
def u(s):
return s
if sys.version_info[1] <= 1:
def int2byte(i):
return bytes((i,))
else:
# This is about 2x faster than the implementation above on 3.2+
int2byte = operator.methodcaller("to_bytes", 1, "big")
import io
StringIO = io.StringIO
BytesIO = io.BytesIO
else:
def b(s):
return s
def u(s):
return unicode(s, "unicode_escape")
int2byte = chr
import StringIO
StringIO = BytesIO = StringIO.StringIO
_add_doc(b, """Byte literal""")
_add_doc(u, """Text literal""")
if PY3:
import builtins
exec_ = getattr(builtins, "exec")
def reraise(tp, value, tb=None):
if value.__traceback__ is not tb:
raise value.with_traceback(tb)
raise value
print_ = getattr(builtins, "print")
del builtins
else:
def exec_(code, globs=None, locs=None):
"""Execute code in a namespace."""
if globs is None:
frame = sys._getframe(1)
globs = frame.f_globals
if locs is None:
locs = frame.f_locals
del frame
elif locs is None:
locs = globs
exec("""exec code in globs, locs""")
exec_("""def reraise(tp, value, tb=None):
raise tp, value, tb
""")
def print_(*args, **kwargs):
"""The new-style print function."""
fp = kwargs.pop("file", sys.stdout)
if fp is None:
return
def write(data):
if not isinstance(data, basestring):
data = str(data)
fp.write(data)
want_unicode = False
sep = kwargs.pop("sep", None)
if sep is not None:
if isinstance(sep, unicode):
want_unicode = True
elif not isinstance(sep, str):
raise TypeError("sep must be None or a string")
end = kwargs.pop("end", None)
if end is not None:
if isinstance(end, unicode):
want_unicode = True
elif not isinstance(end, str):
raise TypeError("end must be None or a string")
if kwargs:
raise TypeError("invalid keyword arguments to print()")
if not want_unicode:
for arg in args:
if isinstance(arg, unicode):
want_unicode = True
break
if want_unicode:
newline = unicode("\n")
space = unicode(" ")
else:
newline = "\n"
space = " "
if sep is None:
sep = space
if end is None:
end = newline
for i, arg in enumerate(args):
if i:
write(sep)
write(arg)
write(end)
_add_doc(reraise, """Reraise an exception.""")
def with_metaclass(meta, base=object):
"""Create a base class with a metaclass."""
return meta("NewBase", (base,), {})

110
pbr/d2to1/tests/__init__.py

@ -1,81 +1,77 @@
from __future__ import with_statement # Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
import os import os
import shutil import shutil
import subprocess import subprocess
import sys import sys
import tempfile
import pkg_resources import fixtures
import testtools
from .util import rmtree, open_config
D2TO1_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), class D2to1TestCase(testtools.TestCase):
os.pardir, os.pardir)) def setUp(self):
super(D2to1TestCase, self).setUp()
self.temp_dir = self.useFixture(fixtures.TempDir()).path
def fake_d2to1_dist():
# Fake a d2to1 distribution from the d2to1 package that these tests reside
# in and make sure it's active on the path with the appropriate entry
# points installed
class _FakeProvider(pkg_resources.EmptyProvider):
"""A fake metadata provider that does almost nothing except to return
entry point metadata.
"""
def has_metadata(self, name):
return name == 'entry_points.txt'
def get_metadata(self, name):
if name == 'entry_points.txt':
return '[distutils.setup_keywords]\nd2to1 = d2to1.core:d2to1\n'
else:
return ''
sys.path.insert(0, D2TO1_DIR)
if 'd2to1' in sys.modules:
del sys.modules['d2to1']
if 'd2to1' in pkg_resources.working_set.by_key:
del pkg_resources.working_set.by_key['d2to1']
dist = pkg_resources.Distribution(location=D2TO1_DIR, project_name='d2to1',
metadata=_FakeProvider())
pkg_resources.working_set.add(dist)
class D2to1TestCase(object):
def setup(self):
self.temp_dir = tempfile.mkdtemp(prefix='d2to1-test-')
self.package_dir = os.path.join(self.temp_dir, 'testpackage') self.package_dir = os.path.join(self.temp_dir, 'testpackage')
shutil.copytree(os.path.join(os.path.dirname(__file__), 'testpackage'), shutil.copytree(os.path.join(os.path.dirname(__file__), 'testpackage'),
self.package_dir) self.package_dir)
self.oldcwd = os.getcwd() self.addCleanup(os.chdir, os.getcwd())
os.chdir(self.package_dir) os.chdir(self.package_dir)
def teardown(self): def tearDown(self):
os.chdir(self.oldcwd)
# Remove d2to1.testpackage from sys.modules so that it can be freshly # Remove d2to1.testpackage from sys.modules so that it can be freshly
# re-imported by the next test # re-imported by the next test
for k in list(sys.modules): for k in list(sys.modules):
if (k == 'd2to1_testpackage' or if (k == 'd2to1_testpackage' or
k.startswith('d2to1_testpackage.')): k.startswith('d2to1_testpackage.')):
del sys.modules[k] del sys.modules[k]
rmtree(self.temp_dir) super(D2to1TestCase, self).tearDown()
def run_setup(self, *args): def run_setup(self, *args):
cmd = ('-c', return self._run_cmd(sys.executable, ('setup.py',) + args)
'import sys;sys.path.insert(0, %r);'
'from d2to1.tests import fake_d2to1_dist;'
'from d2to1.extern.six import exec_;'
'fake_d2to1_dist();exec_(open("setup.py").read())' % D2TO1_DIR)
return self._run_cmd(sys.executable, cmd + args)
def run_svn(self, *args):
return self._run_cmd('svn', args)
def _run_cmd(self, cmd, args): def _run_cmd(self, cmd, args):
""" """Run a command in the root of the test working copy.
Runs a command, with the given argument list, in the root of the test Runs a command, with the given argument list, in the root of the test
working copy--returns the stdout and stderr streams and the exit code working copy--returns the stdout and stderr streams and the exit code
from the subprocess. from the subprocess.

57
pbr/d2to1/tests/test_commands.py

@ -1,13 +1,58 @@
from . import D2to1TestCase # Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
from testtools import content
from pbr.d2to1 import tests
class TestCommands(D2to1TestCase): class TestCommands(tests.D2to1TestCase):
def test_custom_build_py_command(self): def test_custom_build_py_command(self):
""" """Test custom build_py command.
Test that a custom subclass of the build_py command runs when listed in Test that a custom subclass of the build_py command runs when listed in
the commands [global] option, rather than the normal build command. the commands [global] option, rather than the normal build command.
""" """
stdout, _, return_code = self.run_setup('build_py') stdout, stderr, return_code = self.run_setup('build_py')
assert 'Running custom build_py command.' in stdout self.addDetail('stdout', content.text_content(stdout))
assert return_code == 0 self.addDetail('stderr', content.text_content(stderr))
self.assertIn('Running custom build_py command.', stdout)
self.assertEqual(return_code, 0)

63
pbr/d2to1/tests/test_core.py

@ -1,26 +1,55 @@
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
import glob import glob
import os import os
import tarfile import tarfile
from . import D2to1TestCase from pbr.d2to1 import tests
VERSION = '0.1.dev' class TestCore(tests.D2to1TestCase):
class TestCore(D2to1TestCase):
def test_setup_py_version(self):
"""
Test that the `./setup.py --version` command returns the correct
value without balking.
"""
self.run_setup('egg_info')
stdout, _, _ = self.run_setup('--version')
assert stdout == VERSION
def test_setup_py_keywords(self): def test_setup_py_keywords(self):
""" """setup.py --keywords.
Test that the `./setup.py --keywords` command returns the correct Test that the `./setup.py --keywords` command returns the correct
value without balking. value without balking.
""" """
@ -30,9 +59,7 @@ class TestCore(D2to1TestCase):
assert stdout == 'packaging,distutils,setuptools' assert stdout == 'packaging,distutils,setuptools'
def test_sdist_extra_files(self): def test_sdist_extra_files(self):
""" """Test that the extra files are correctly added."""
Test that the extra files are correctly added.
"""
stdout, _, return_code = self.run_setup('sdist', '--formats=gztar') stdout, _, return_code = self.run_setup('sdist', '--formats=gztar')

61
pbr/d2to1/tests/test_hooks.py

@ -1,16 +1,55 @@
from __future__ import with_statement # Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
import os import os
import textwrap import textwrap
from . import D2to1TestCase from pbr.d2to1 import tests
from .util import open_config from pbr.d2to1.tests import util
class TestHooks(D2to1TestCase): class TestHooks(tests.D2to1TestCase):
def setup(self): def setUp(self):
super(TestHooks, self).setup() super(TestHooks, self).setUp()
with open_config(os.path.join(self.package_dir, 'setup.cfg')) as cfg: with util.open_config(
os.path.join(self.package_dir, 'setup.cfg')) as cfg:
cfg.set('global', 'setup-hooks', cfg.set('global', 'setup-hooks',
'd2to1_testpackage._setup_hooks.test_hook_1\n' 'd2to1_testpackage._setup_hooks.test_hook_1\n'
'd2to1_testpackage._setup_hooks.test_hook_2') 'd2to1_testpackage._setup_hooks.test_hook_2')
@ -20,7 +59,8 @@ class TestHooks(D2to1TestCase):
'd2to1_testpackage._setup_hooks.test_post_hook') 'd2to1_testpackage._setup_hooks.test_post_hook')
def test_global_setup_hooks(self): def test_global_setup_hooks(self):
""" """Test setup_hooks.
Test that setup_hooks listed in the [global] section of setup.cfg are Test that setup_hooks listed in the [global] section of setup.cfg are
executed in order. executed in order.
""" """
@ -30,7 +70,8 @@ class TestHooks(D2to1TestCase):
assert return_code == 0 assert return_code == 0
def test_command_hooks(self): def test_command_hooks(self):
""" """Test command hooks.
Simple test that the appropriate command hooks run at the Simple test that the appropriate command hooks run at the
beginning/end of the appropriate command. beginning/end of the appropriate command.
""" """
@ -45,7 +86,7 @@ class TestHooks(D2to1TestCase):
running build_ext running build_ext
running pre_hook d2to1_testpackage._setup_hooks.test_pre_hook for command build_ext running pre_hook d2to1_testpackage._setup_hooks.test_pre_hook for command build_ext
build_ext pre-hook build_ext pre-hook
""") in stdout """) in stdout # flake8: noqa
assert stdout.endswith('build_ext post-hook') assert stdout.endswith('build_ext post-hook')
assert return_code == 0 assert return_code == 0

46
pbr/d2to1/tests/testpackage/d2to1_testpackage/_setup_hooks.py

@ -1,4 +1,44 @@
from distutils.command.build_py import build_py # Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
from distutils.command import build_py
def test_hook_1(config): def test_hook_1(config):
@ -9,12 +49,12 @@ def test_hook_2(config):
print('test_hook_2') print('test_hook_2')
class test_command(build_py): class test_command(build_py.build_py):
command_name = 'build_py' command_name = 'build_py'
def run(self): def run(self):
print('Running custom build_py command.') print('Running custom build_py command.')
return build_py.run(self) return build_py.build_py.run(self)
def test_pre_hook(cmdobj): def test_pre_hook(cmdobj):

485
pbr/d2to1/tests/testpackage/distribute_setup.py

@ -1,485 +0,0 @@
#!python
"""Bootstrap distribute installation
If you want to use setuptools in your package's setup.py, just include this
file in the same directory with it, and add this to the top of your setup.py::
from distribute_setup import use_setuptools
use_setuptools()
If you want to require a specific version of setuptools, set a download
mirror, or use an alternate download directory, you can do so by supplying
the appropriate options to ``use_setuptools()``.
This file can also be run as a script to install or upgrade setuptools.
"""
import os
import sys
import time
import fnmatch
import tempfile
import tarfile
from distutils import log
try:
from site import USER_SITE
except ImportError:
USER_SITE = None
try:
import subprocess
def _python_cmd(*args):
args = (sys.executable,) + args
return subprocess.call(args) == 0
except ImportError:
# will be used for python 2.3
def _python_cmd(*args):
args = (sys.executable,) + args
# quoting arguments if windows
if sys.platform == 'win32':
def quote(arg):
if ' ' in arg:
return '"%s"' % arg
return arg
args = [quote(arg) for arg in args]
return os.spawnl(os.P_WAIT, sys.executable, *args) == 0
DEFAULT_VERSION = "0.6.19"
DEFAULT_URL = "http://pypi.python.org/packages/source/d/distribute/"
SETUPTOOLS_FAKED_VERSION = "0.6c11"
SETUPTOOLS_PKG_INFO = """\
Metadata-Version: 1.0
Name: setuptools
Version: %s
Summary: xxxx
Home-page: xxx
Author: xxx
Author-email: xxx
License: xxx
Description: xxx
""" % SETUPTOOLS_FAKED_VERSION
def _install(tarball):
# extracting the tarball
tmpdir = tempfile.mkdtemp()
log.warn('Extracting in %s', tmpdir)
old_wd = os.getcwd()
try:
os.chdir(tmpdir)
tar = tarfile.open(tarball)
_extractall(tar)
tar.close()
# going in the directory
subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0])
os.chdir(subdir)
log.warn('Now working in %s', subdir)
# installing
log.warn('Installing Distribute')
if not _python_cmd('setup.py', 'install'):
log.warn('Something went wrong during the installation.')
log.warn('See the error message above.')
finally:
os.chdir(old_wd)
def _build_egg(egg, tarball, to_dir):
# extracting the tarball
tmpdir = tempfile.mkdtemp()
log.warn('Extracting in %s', tmpdir)
old_wd = os.getcwd()
try:
os.chdir(tmpdir)
tar = tarfile.open(tarball)
_extractall(tar)
tar.close()
# going in the directory
subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0])
os.chdir(subdir)
log.warn('Now working in %s', subdir)
# building an egg
log.warn('Building a Distribute egg in %s', to_dir)
_python_cmd('setup.py', '-q', 'bdist_egg', '--dist-dir', to_dir)
finally:
os.chdir(old_wd)
# returning the result
log.warn(egg)
if not os.path.exists(egg):
raise IOError('Could not build the egg.')
def _do_download(version, download_base, to_dir, download_delay):
egg = os.path.join(to_dir, 'distribute-%s-py%d.%d.egg'
% (version, sys.version_info[0], sys.version_info[1]))
if not os.path.exists(egg):
tarball = download_setuptools(version, download_base,
to_dir, download_delay)
_build_egg(egg, tarball, to_dir)
sys.path.insert(0, egg)
import setuptools
setuptools.bootstrap_install_from = egg
def use_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL,
to_dir=os.curdir, download_delay=15, no_fake=True):
# making sure we use the absolute path
to_dir = os.path.abspath(to_dir)
was_imported = 'pkg_resources' in sys.modules or \
'setuptools' in sys.modules
try:
try:
import pkg_resources
if not hasattr(pkg_resources, '_distribute'):
if not no_fake:
_fake_setuptools()
raise ImportError
except ImportError:
return _do_download(version, download_base, to_dir, download_delay)
try:
pkg_resources.require("distribute>="+version)
return
except pkg_resources.VersionConflict:
e = sys.exc_info()[1]
if was_imported:
sys.stderr.write(
"The required version of distribute (>=%s) is not available,\n"
"and can't be installed while this script is running. Please\n"
"install a more recent version first, using\n"
"'easy_install -U distribute'."
"\n\n(Currently using %r)\n" % (version, e.args[0]))
sys.exit(2)
else:
del pkg_resources, sys.modules['pkg_resources'] # reload ok
return _do_download(version, download_base, to_dir,
download_delay)
except pkg_resources.DistributionNotFound:
return _do_download(version, download_base, to_dir,
download_delay)
finally:
if not no_fake:
_create_fake_setuptools_pkg_info(to_dir)
def download_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL,
to_dir=os.curdir, delay=15):
"""Download distribute from a specified location and return its filename
`version` should be a valid distribute version number that is available
as an egg for download under the `download_base` URL (which should end
with a '/'). `to_dir` is the directory where the egg will be downloaded.
`delay` is the number of seconds to pause before an actual download
attempt.
"""
# making sure we use the absolute path
to_dir = os.path.abspath(to_dir)
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
tgz_name = "distribute-%s.tar.gz" % version
url = download_base + tgz_name
saveto = os.path.join(to_dir, tgz_name)
src = dst = None
if not os.path.exists(saveto): # Avoid repeated downloads
try:
log.warn("Downloading %s", url)
src = urlopen(url)
# Read/write all in one block, so we don't create a corrupt file
# if the download is interrupted.
data = src.read()
dst = open(saveto, "wb")
dst.write(data)
finally:
if src:
src.close()
if dst:
dst.close()
return os.path.realpath(saveto)
def _no_sandbox(function):
def __no_sandbox(*args, **kw):
try:
from setuptools.sandbox import DirectorySandbox
if not hasattr(DirectorySandbox, '_old'):
def violation(*args):
pass
DirectorySandbox._old = DirectorySandbox._violation
DirectorySandbox._violation = violation
patched = True
else:
patched = False
except ImportError:
patched = False
try:
return function(*args, **kw)
finally:
if patched:
DirectorySandbox._violation = DirectorySandbox._old
del DirectorySandbox._old
return __no_sandbox
def _patch_file(path, content):
"""Will backup the file then patch it"""
existing_content = open(path).read()
if existing_content == content:
# already patched
log.warn('Already patched.')
return False
log.warn('Patching...')
_rename_path(path)
f = open(path, 'w')
try:
f.write(content)
finally:
f.close()
return True
_patch_file = _no_sandbox(_patch_file)
def _same_content(path, content):
return open(path).read() == content
def _rename_path(path):
new_name = path + '.OLD.%s' % time.time()
log.warn('Renaming %s into %s', path, new_name)
os.rename(path, new_name)
return new_name
def _remove_flat_installation(placeholder):
if not os.path.isdir(placeholder):
log.warn('Unkown installation at %s', placeholder)
return False
found = False
for file in os.listdir(placeholder):
if fnmatch.fnmatch(file, 'setuptools*.egg-info'):
found = True
break
if not found:
log.warn('Could not locate setuptools*.egg-info')
return
log.warn('Removing elements out of the way...')
pkg_info = os.path.join(placeholder, file)
if os.path.isdir(pkg_info):
patched = _patch_egg_dir(pkg_info)
else:
patched = _patch_file(pkg_info, SETUPTOOLS_PKG_INFO)
if not patched:
log.warn('%s already patched.', pkg_info)
return False
# now let's move the files out of the way
for element in ('setuptools', 'pkg_resources.py', 'site.py'):
element = os.path.join(placeholder, element)
if os.path.exists(element):
_rename_path(element)
else:
log.warn('Could not find the %s element of the '
'Setuptools distribution', element)
return True
_remove_flat_installation = _no_sandbox(_remove_flat_installation)
def _after_install(dist):
log.warn('After install bootstrap.')
placeholder = dist.get_command_obj('install').install_purelib
_create_fake_setuptools_pkg_info(placeholder)
def _create_fake_setuptools_pkg_info(placeholder):
if not placeholder or not os.path.exists(placeholder):
log.warn('Could not find the install location')
return
pyver = '%s.%s' % (sys.version_info[0], sys.version_info[1])
setuptools_file = 'setuptools-%s-py%s.egg-info' % \
(SETUPTOOLS_FAKED_VERSION, pyver)
pkg_info = os.path.join(placeholder, setuptools_file)
if os.path.exists(pkg_info):
log.warn('%s already exists', pkg_info)
return
log.warn('Creating %s', pkg_info)
f = open(pkg_info, 'w')
try:
f.write(SETUPTOOLS_PKG_INFO)
finally:
f.close()
pth_file = os.path.join(placeholder, 'setuptools.pth')
log.warn('Creating %s', pth_file)
f = open(pth_file, 'w')
try:
f.write(os.path.join(os.curdir, setuptools_file))
finally:
f.close()
_create_fake_setuptools_pkg_info = _no_sandbox(_create_fake_setuptools_pkg_info)
def _patch_egg_dir(path):
# let's check if it's already patched
pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO')
if os.path.exists(pkg_info):
if _same_content(pkg_info, SETUPTOOLS_PKG_INFO):
log.warn('%s already patched.', pkg_info)
return False
_rename_path(path)
os.mkdir(path)
os.mkdir(os.path.join(path, 'EGG-INFO'))
pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO')
f = open(pkg_info, 'w')
try:
f.write(SETUPTOOLS_PKG_INFO)
finally:
f.close()
return True
_patch_egg_dir = _no_sandbox(_patch_egg_dir)
def _before_install():
log.warn('Before install bootstrap.')
_fake_setuptools()
def _under_prefix(location):
if 'install' not in sys.argv:
return True
args = sys.argv[sys.argv.index('install')+1:]
for index, arg in enumerate(args):
for option in ('--root', '--prefix'):
if arg.startswith('%s=' % option):
top_dir = arg.split('root=')[-1]
return location.startswith(top_dir)
elif arg == option:
if len(args) > index:
top_dir = args[index+1]
return location.startswith(top_dir)
if arg == '--user' and USER_SITE is not None:
return location.startswith(USER_SITE)
return True
def _fake_setuptools():
log.warn('Scanning installed packages')
try:
import pkg_resources
except ImportError:
# we're cool
log.warn('Setuptools or Distribute does not seem to be installed.')
return
ws = pkg_resources.working_set
try:
setuptools_dist = ws.find(pkg_resources.Requirement.parse('setuptools',
replacement=False))
except TypeError:
# old distribute API
setuptools_dist = ws.find(pkg_resources.Requirement.parse('setuptools'))
if setuptools_dist is None:
log.warn('No setuptools distribution found')
return
# detecting if it was already faked
setuptools_location = setuptools_dist.location
log.warn('Setuptools installation detected at %s', setuptools_location)
# if --root or --preix was provided, and if
# setuptools is not located in them, we don't patch it
if not _under_prefix(setuptools_location):
log.warn('Not patching, --root or --prefix is installing Distribute'
' in another location')
return
# let's see if its an egg
if not setuptools_location.endswith('.egg'):
log.warn('Non-egg installation')
res = _remove_flat_installation(setuptools_location)
if not res:
return
else:
log.warn('Egg installation')
pkg_info = os.path.join(setuptools_location, 'EGG-INFO', 'PKG-INFO')
if (os.path.exists(pkg_info) and
_same_content(pkg_info, SETUPTOOLS_PKG_INFO)):
log.warn('Already patched.')
return
log.warn('Patching...')
# let's create a fake egg replacing setuptools one
res = _patch_egg_dir(setuptools_location)
if not res:
return
log.warn('Patched done.')
_relaunch()
def _relaunch():
log.warn('Relaunching...')
# we have to relaunch the process
# pip marker to avoid a relaunch bug
if sys.argv[:3] == ['-c', 'install', '--single-version-externally-managed']:
sys.argv[0] = 'setup.py'
args = [sys.executable] + sys.argv
sys.exit(subprocess.call(args))
def _extractall(self, path=".", members=None):
"""Extract all members from the archive to the current working
directory and set owner, modification time and permissions on
directories afterwards. `path' specifies a different directory
to extract to. `members' is optional and must be a subset of the
list returned by getmembers().
"""
import copy
import operator
from tarfile import ExtractError
directories = []
if members is None:
members = self
for tarinfo in members:
if tarinfo.isdir():
# Extract directories with a safe mode.
directories.append(tarinfo)
tarinfo = copy.copy(tarinfo)
tarinfo.mode = 448 # decimal for oct 0700
self.extract(tarinfo, path)
# Reverse sort directories.
if sys.version_info < (2, 4):
def sorter(dir1, dir2):
return cmp(dir1.name, dir2.name)
directories.sort(sorter)
directories.reverse()
else:
directories.sort(key=operator.attrgetter('name'), reverse=True)
# Set correct owner, mtime and filemode on directories.
for tarinfo in directories:
dirpath = os.path.join(path, tarinfo.name)
try:
self.chown(tarinfo, dirpath)
self.utime(tarinfo, dirpath)
self.chmod(tarinfo, dirpath)
except ExtractError:
e = sys.exc_info()[1]
if self.errorlevel > 1:
raise
else:
self._dbg(1, "tarfile: %s" % e)
def main(argv, version=DEFAULT_VERSION):
"""Install or upgrade setuptools and EasyInstall"""
tarball = download_setuptools()
_install(tarball)
if __name__ == '__main__':
main(sys.argv[1:])

28
pbr/d2to1/tests/testpackage/setup.py

@ -1,12 +1,22 @@
#!/usr/bin/env python #!/usr/bin/env python
try: # Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
from setuptools import setup #
except ImportError: # Licensed under the Apache License, Version 2.0 (the "License");
from distribute_setup import use_setuptools # you may not use this file except in compliance with the License.
use_setuptools() # You may obtain a copy of the License at
from setuptools import setup #
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
setup( import setuptools
setup_requires=['d2to1'],
d2to1=True, setuptools.setup(
setup_requires=['pbr'],
pbr=True,
) )

55
pbr/d2to1/tests/util.py

@ -1,18 +1,57 @@
from __future__ import with_statement # Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
import contextlib import contextlib
import os import os
import shutil import shutil
import stat import stat
try:
from ..extern.six import moves as m import configparser
ConfigParser = m.configparser.ConfigParser except ImportError:
import ConfigParser as configparser
@contextlib.contextmanager @contextlib.contextmanager
def open_config(filename): def open_config(filename):
cfg = ConfigParser() cfg = configparser.ConfigParser()
cfg.read(filename) cfg.read(filename)
yield cfg yield cfg
with open(filename, 'w') as fp: with open(filename, 'w') as fp:
@ -20,9 +59,9 @@ def open_config(filename):
def rmtree(path): def rmtree(path):
""" """shutil.rmtree() with error handler.
shutil.rmtree() with error handler for 'access denied' from trying to
delete read-only files. Handle 'access denied' from trying to delete read-only files.
""" """
def onerror(func, path, exc_info): def onerror(func, path, exc_info):

52
pbr/d2to1/util.py

@ -1,3 +1,43 @@
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
"""The code in this module is mostly copy/pasted out of the distutils2 source """The code in this module is mostly copy/pasted out of the distutils2 source
code, as recommended by Tarek Ziade. As such, it may be subject to some change code, as recommended by Tarek Ziade. As such, it may be subject to some change
as distutils2 development continues, and will have to be kept up to date. as distutils2 development continues, and will have to be kept up to date.
@ -11,10 +51,10 @@ to be an installation dependency for our packages yet--it is still too unstable
# irritating Python bug that can crop up when using ./setup.py test. # irritating Python bug that can crop up when using ./setup.py test.
# See: http://www.eby-sarna.com/pipermail/peak/2010-May/003355.html # See: http://www.eby-sarna.com/pipermail/peak/2010-May/003355.html
try: try:
import multiprocessing import multiprocessing # flake8: noqa
except ImportError: except ImportError:
pass pass
import logging import logging # flake8: noqa
import os import os
import re import re
@ -32,8 +72,10 @@ from setuptools.command.egg_info import manifest_maker
from setuptools.dist import Distribution from setuptools.dist import Distribution
from setuptools.extension import Extension from setuptools.extension import Extension
from .extern.six import moves as m try:
RawConfigParser = m.configparser.RawConfigParser import configparser
except ImportError:
import ConfigParser as configparser
# A simplified RE for this; just checks that the line ends with version # A simplified RE for this; just checks that the line ends with version
@ -155,7 +197,7 @@ def cfg_to_args(path='setup.cfg'):
""" """
# The method source code really starts here. # The method source code really starts here.
parser = RawConfigParser() parser = configparser.RawConfigParser()
if not os.path.exists(path): if not os.path.exists(path):
raise DistutilsFileError("file '%s' does not exist" % raise DistutilsFileError("file '%s' does not exist" %
os.path.abspath(path)) os.path.abspath(path))

159
pbr/d2to1/zestreleaser.py

@ -1,159 +0,0 @@
"""zest.releaser entry points to support projects using distutils2-like
setup.cfg files. The only actual functionality this adds is to update the
version option in a setup.cfg file, if it exists. If setup.cfg does not exist,
or does not contain a version option, then this does nothing.
TODO: d2to1 theoretically supports using a different filename for setup.cfg;
this does not support that. We could hack in support, though I'm not sure how
useful the original functionality is to begin with (and it might be removed) so
we ignore that for now.
TODO: There exists a proposal
(http://mail.python.org/pipermail/distutils-sig/2011-March/017628.html) to add
a 'version-from-file' option (or something of the like) to distutils2; if this
is added then support for it should be included here as well.
"""
import logging
import os
from .extern.six import moves as m
ConfigParser = m.configparser.ConfigParser
logger = logging.getLogger(__name__)
def update_setupcfg_version(filename, version):
"""Opens the given setup.cfg file, locates the version option in the
[metadata] section, updates it to the new version.
"""
setup_cfg = open(filename).readlines()
current_section = None
updated = False
for idx, line in enumerate(setup_cfg):
m = ConfigParser.SECTCRE.match(line)
if m:
if current_section == 'metadata':
# We already parsed the entire metadata section without finding
# a version line, and are now moving into a new section
break
current_section = m.group('header')
continue
if '=' not in line:
continue
opt, val = line.split('=', 1)
opt, val = opt.strip(), val.strip()
if current_section == 'metadata' and opt == 'version':
setup_cfg[idx] = 'version = %s\n' % version
updated = True
break
if updated:
open(filename, 'w').writelines(setup_cfg)
logger.info("Set %s's version to %r" % (os.path.basename(filename),
version))
def prereleaser_middle(data):
filename = os.path.join(data['workingdir'], 'setup.cfg')
if os.path.exists(filename):
update_setupcfg_version(filename, data['new_version'])
def releaser_middle(data):
"""
releaser.middle hook to monkey-patch zest.releaser to support signed
tagging--currently this is the only way to do this. Also monkey-patches to
disable an annoyance where zest.releaser only creates .zip source
distributions. This is supposedly a workaround for a bug in Python 2.4,
but we don't care about Python 2.4.
"""
import os
import sys
from zest.releaser.git import Git
from zest.releaser.release import Releaser
# Copied verbatim from zest.releaser, but with the cmd string modified to
# use the -s option to create a signed tag
def _my_create_tag(self, version):
msg = "Tagging %s" % (version,)
cmd = 'git tag -s %s -m "%s"' % (version, msg)
if os.path.isdir('.git/svn'):
print "\nEXPERIMENTAL support for git-svn tagging!\n"
cur_branch = open('.git/HEAD').read().strip().split('/')[-1]
print "You are on branch %s." % (cur_branch,)
if cur_branch != 'master':
print "Only the master branch is supported for git-svn tagging."
print "Please tag yourself."
print "'git tag' needs to list tag named %s." % (version,)
sys.exit()
cmd = [cmd]
local_head = open('.git/refs/heads/master').read()
trunk = open('.git/refs/remotes/trunk').read()
if local_head != trunk:
print "Your local master diverges from trunk.\n"
# dcommit before local tagging
cmd.insert(0, 'git svn dcommit')
# create tag in svn
cmd.append('git svn tag -m "%s" %s' % (msg, version))
return cmd
# Similarly copied from zer.releaser to support use of 'v' in front
# of the version number
def _my_make_tag(self):
from zest.releaser import utils
from os import system
if self.data['tag_already_exists']:
return
cmds = self.vcs.cmd_create_tag(self.data['version'])
if not isinstance(cmds, list):
cmds = [cmds]
if len(cmds) == 1:
print "Tag needed to proceed, you can use the following command:"
for cmd in cmds:
print cmd
if utils.ask("Run this command"):
print system(cmd)
else:
# all commands are needed in order to proceed normally
print "Please create a tag for %s yourself and rerun." % \
(self.data['version'],)
sys.exit()
if not self.vcs.tag_exists('v' + self.data['version']):
print "\nFailed to create tag %s!" % (self.data['version'],)
sys.exit()
# Normally all this does is to return '--formats=zip', which is currently
# hard-coded as an option to always add to the sdist command; they ought to
# make this actually optional
def _my_sdist_options(self):
return ''
Git.cmd_create_tag = _my_create_tag
Releaser._make_tag = _my_make_tag
Releaser._sdist_options = _my_sdist_options
def postreleaser_before(data):
"""
Fix the irritating .dev0 default appended to new development versions by
zest.releaser to just append ".dev" without the "0".
"""
data['dev_version_template'] = '%(new_version)s.dev'
def postreleaser_middle(data):
filename = os.path.join(data['workingdir'], 'setup.cfg')
if os.path.exists(filename):
update_setupcfg_version(filename, data['dev_version'])

10
pbr/packaging.py

@ -26,7 +26,6 @@ import re
import subprocess import subprocess
import sys import sys
from d2to1.extern import six
from distutils.command import install as du_install from distutils.command import install as du_install
import distutils.errors import distutils.errors
from distutils import log from distutils import log
@ -34,6 +33,11 @@ import pkg_resources
from setuptools.command import install from setuptools.command import install
from setuptools.command import sdist from setuptools.command import sdist
try:
import cStringIO as io
except ImportError:
import io
log.set_verbosity(log.INFO) log.set_verbosity(log.INFO)
TRUE_VALUES = ('true', '1', 'yes') TRUE_VALUES = ('true', '1', 'yes')
REQUIREMENTS_FILES = ('requirements.txt', 'tools/pip-requires') REQUIREMENTS_FILES = ('requirements.txt', 'tools/pip-requires')
@ -111,7 +115,7 @@ def canonicalize_emails(changelog, mapping):
"""Takes in a string and an email alias mapping and replaces all """Takes in a string and an email alias mapping and replaces all
instances of the aliases in the string with their real email. instances of the aliases in the string with their real email.
""" """
for alias, email_address in six.iteritems(mapping): for alias, email_address in mapping.items():
changelog = changelog.replace(alias, email_address) changelog = changelog.replace(alias, email_address)
return changelog return changelog
@ -481,7 +485,7 @@ try:
def _sphinx_run(self): def _sphinx_run(self):
if not self.verbose: if not self.verbose:
status_stream = six.StringIO() status_stream = io.StringIO()
else: else:
status_stream = sys.stdout status_stream = sys.stdout
confoverrides = {} confoverrides = {}

15
pbr/testr_command.py

@ -1,5 +1,18 @@
#
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P. # Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (c) 2013 Testrepository Contributors # Copyright (c) 2013 Testrepository Contributors
# #
# Licensed under either the Apache License, Version 2.0 or the BSD 3-clause # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause

14
pbr/tests/test_setup.py

@ -22,7 +22,13 @@ import os
import sys import sys
import tempfile import tempfile
from d2to1.extern import six try:
import cStringIO as io
BytesIO = io.StringIO
except ImportError:
import io
BytesIO = io.BytesIO
import fixtures import fixtures
import testscenarios import testscenarios
@ -165,8 +171,8 @@ class GitLogsTest(tests.BaseTestCase):
"os.path.exists", "os.path.exists",
lambda path: os.path.abspath(path) in exist_files)) lambda path: os.path.abspath(path) in exist_files))
self.useFixture(fixtures.FakePopen(lambda _: { self.useFixture(fixtures.FakePopen(lambda _: {
"stdout": six.BytesIO("Author: Foo Bar " "stdout": BytesIO("Author: Foo Bar "
"<email@bar.com>\n".encode('utf-8')) "<email@bar.com>\n".encode('utf-8'))
})) }))
def _fake_read_git_mailmap(*args): def _fake_read_git_mailmap(*args):
@ -207,7 +213,7 @@ class GitLogsTest(tests.BaseTestCase):
lambda path: os.path.abspath(path) in exist_files)) lambda path: os.path.abspath(path) in exist_files))
self.useFixture(fixtures.FakePopen(lambda proc_args: { self.useFixture(fixtures.FakePopen(lambda proc_args: {
"stdout": six.BytesIO( "stdout": BytesIO(
self._fake_log_output(proc_args["args"][2], cmd_map)) self._fake_log_output(proc_args["args"][2], cmd_map))
})) }))

2
pbr/tests/test_version.py

@ -28,4 +28,4 @@ class DeferredVersionTestCase(tests.BaseTestCase):
deferred_string = MyVersionInfo("openstack").\ deferred_string = MyVersionInfo("openstack").\
cached_version_string() cached_version_string()
self.assertEquals("5.5.5.5", deferred_string) self.assertEqual("5.5.5.5", deferred_string)

1
requirements.txt

@ -1,2 +1 @@
d2to1>=0.2.10,<0.3
setuptools_git>=0.4 setuptools_git>=0.4

5
setup.cfg

@ -6,6 +6,7 @@ summary = Python Build Reasonableness
description-file = description-file =
README.rst README.rst
home-page = http://pypi.python.org/pypi/pbr home-page = http://pypi.python.org/pypi/pbr
requires-python = >=2.6
classifier = classifier =
Development Status :: 4 - Beta Development Status :: 4 - Beta
Environment :: Console Environment :: Console
@ -26,3 +27,7 @@ setup-hooks =
[pbr] [pbr]
warnerrors = True warnerrors = True
[entry_points]
distutils.setup_keywords =
pbr = pbr.core:pbr

5
setup.py

@ -16,6 +16,7 @@
import setuptools import setuptools
from pbr.d2to1 import util
setuptools.setup( setuptools.setup(
setup_requires=['d2to1>=0.2.10,<0.3'], **util.cfg_to_args())
d2to1=True)

14
tools/integration.sh

@ -41,8 +41,14 @@ mkdir -p $pypidir
jeepybvenv=$tmpdir/jeepyb jeepybvenv=$tmpdir/jeepyb
rm -f ~/.pip/pip.conf ~/.pydistutils.cfg rm -f ~/.pip/pip.conf ~/.pydistutils.cfg
mkdir -p ~/.pip
mkvenv $jeepybvenv distribute pip cat <<EOF > ~/.pip/pip.conf
[global]
log = /home/jenkins/pip.log
EOF
mkvenv $jeepybvenv 'setuptools>=0.7' pip
$jeepybvenv/bin/pip install -U git+https://review.openstack.org/p/openstack-infra/jeepyb.git $jeepybvenv/bin/pip install -U git+https://review.openstack.org/p/openstack-infra/jeepyb.git
cat <<EOF > $tmpdir/mirror.yaml cat <<EOF > $tmpdir/mirror.yaml
@ -77,11 +83,11 @@ cat <<EOF > ~/.pydistutils.cfg
index_url = $pypiurl index_url = $pypiurl
EOF EOF
mkdir -p ~/.pip
cat <<EOF > ~/.pip/pip.conf cat <<EOF > ~/.pip/pip.conf
[global] [global]
index-url = $pypiurl index-url = $pypiurl
extra-index-url = http://pypi.openstack.org/openstack extra-index-url = http://pypi.openstack.org/openstack
log = /home/jenkins/pip.log
EOF EOF
projectdir=$tmpdir/projects projectdir=$tmpdir/projects
@ -98,6 +104,10 @@ for PROJECT in $PROJECTS ; do
# Tempest doesn't really install # Tempest doesn't really install
continue continue
fi fi
if [ $SHORT_PROJECT = 'requirements' ]; then
# requirements doesn't really install
continue
fi
shortprojectdir=$projectdir/$SHORT_PROJECT shortprojectdir=$projectdir/$SHORT_PROJECT
git clone $REPODIR/$SHORT_PROJECT $shortprojectdir git clone $REPODIR/$SHORT_PROJECT $shortprojectdir

2
tox.ini

@ -9,7 +9,7 @@ setenv = VIRTUAL_ENV={envdir}
deps = -r{toxinidir}/requirements.txt deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt -r{toxinidir}/test-requirements.txt
commands = commands =
python setup.py testr --slowest --testr-args='{posargs}' python setup.py testr --testr-args='{posargs}'
[tox:jenkins] [tox:jenkins]
sitepackages = True sitepackages = True