openstack/pbr
Code Issues Proposed changes

Clean up hacking and path issues with d2to1

Browse Source 
We imported in another code base. It takes a little bit of path
adjusting to get things right.

Change-Id: Ifb96652c822d5d243a6bedb77bc34e919be2d3a8
This commit is contained in:
Monty Taylor 2013-07-05 16:39:50 -04:00
parent 295dbe1da3
commit 746b78d637
26 changed files with 555 additions and 1253 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.mailmap
View File

@ -1,4 +1,6 @@
# Format is:
# <preferred e-mail> <other e-mail 1>
# <preferred e-mail> <other e-mail 2>
Davanum Srinivas <dims@linux.vnet.ibm.com> <davanum@gmail.com>
Erik M. Bray <embray@stsci.edu> Erik Bray <embray@stsci.edu>
Zhongyue Luo <zhongyue.nah@intel.com> <lzyeval@gmail.com>

23
README.rst
View File

@ -13,9 +13,20 @@ it's simple and repeatable. If you want to do things differently, cool! But
you've already got the power of python at your fingertips, so you don't
really need PBR.
PBR builds on top of `d2to1` to provide for declarative configuration. It
then filters the `setup.cfg` data through a setup hook to fill in default
values and provide more sensible behaviors.
PBR builds on top of the work that `d2to1` started to provide for declarative
configuration. `d2to1` is itself an implementation of the ideas behind
`distutils2`. Although `distutils2` is now abandoned in favor of work towards
PEP 426 and Metadata 2.0, declarative config is still a great idea and
specifically important in trying to distribute setup code as a library
when that library itself will alter how the setup is processed. As Metadata
2.0 and other modern Python packaging PEPs come out, `pbr` aims to support
them as quickly as possible.
`pbr` reads and then filters the `setup.cfg` data through a setup hook to
fill in default values and provide more sensible behaviors, and then feeds
the results in as the arguments to a call to `setup.py` - so the heavy
lifting of handling python packaging needs is still being done by
`setuptools`.
Behaviors
=========
@ -124,11 +135,11 @@ The minimal setup.py should look something like this::
from setuptools import setup
setup(
setup_requires=['d2to1', 'pbr'],
d2to1=True,
setup_requires=['pbr'],
pbr=True,
)
Note that it's important to specify `d2to1=True` or else the pbr functionality
Note that it's important to specify `pbr=True` or else the pbr functionality
will not be enabled.
It should also work fine if additional arguments are passed to `setup()`,

127
pbr/core.py Normal file
View File

@ -0,0 +1,127 @@
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
from distutils import core
from distutils import errors
import os
import sys
import warnings
from setuptools import dist
from pbr.d2to1 import util
core.Distribution = dist._get_unpatched(core.Distribution)
if sys.version_info[0] == 3:
string_type = str
integer_types = int
else:
string_type = basestring
integer_types = (int, long)
def pbr(dist, attr, value):
"""Implements the actual pbr setup() keyword. When used, this should be
the only keyword in your setup() aside from `setup_requires`.
If given as a string, the value of pbr is assumed to be the relative path
to the setup.cfg file to use. Otherwise, if it evaluates to true, it
simply assumes that pbr should be used, and the default 'setup.cfg' is
used.
This works by reading the setup.cfg file, parsing out the supported
metadata and command options, and using them to rebuild the
`DistributionMetadata` object and set the newly added command options.
The reason for doing things this way is that a custom `Distribution` class
will not play nicely with setup_requires; however, this implementation may
not work well with distributions that do use a `Distribution` subclass.
"""
if not value:
return
if isinstance(value, string_type):
path = os.path.abspath(value)
else:
path = os.path.abspath('setup.cfg')
if not os.path.exists(path):
raise errors.DistutilsFileError(
'The setup.cfg file %s does not exist.' % path)
# Converts the setup.cfg file to setup() arguments
try:
attrs = util.cfg_to_args(path)
except Exception:
e = sys.exc_info()[1]
raise errors.DistutilsSetupError(
'Error parsing %s: %s: %s' % (path, e.__class__.__name__, e))
# Repeat some of the Distribution initialization code with the newly
# provided attrs
if attrs:
# Skips 'options' and 'licence' support which are rarely used; may add
# back in later if demanded
for key, val in attrs.items():
if hasattr(dist.metadata, 'set_' + key):
getattr(dist.metadata, 'set_' + key)(val)
elif hasattr(dist.metadata, key):
setattr(dist.metadata, key, val)
elif hasattr(dist, key):
setattr(dist, key, val)
else:
msg = 'Unknown distribution option: %s' % repr(key)
warnings.warn(msg)
# Re-finalize the underlying Distribution
core.Distribution.finalize_options(dist)
# This bit comes out of distribute/setuptools
if isinstance(dist.metadata.version, integer_types + (float,)):
# Some people apparently take "version number" too literally :)
dist.metadata.version = str(dist.metadata.version)
# This bit of hackery is necessary so that the Distribution will ignore
# normally unsupport command options (namely pre-hooks and post-hooks).
# dist.command_options is normally a dict mapping command names to dicts of
# their options. Now it will be a defaultdict that returns IgnoreDicts for
# the each command's options so we can pass through the unsupported options
ignore = ['pre_hook.*', 'post_hook.*']
dist.command_options = util.DefaultGetDict(lambda: util.IgnoreDict(ignore))

4
pbr/d2to1/__init__.py
View File

@ -1,4 +0,0 @@
try:
__version__ = __import__('pkg_resources').get_distribution('d2to1').version
except:
__version__ = ''

82
pbr/d2to1/core.py
View File

@ -1,82 +0,0 @@
import os
import sys
import warnings
from distutils.core import Distribution as _Distribution
from distutils.errors import DistutilsFileError, DistutilsSetupError
from setuptools.dist import _get_unpatched
from .extern import six
from .util import DefaultGetDict, IgnoreDict, cfg_to_args
_Distribution = _get_unpatched(_Distribution)
def d2to1(dist, attr, value):
"""Implements the actual d2to1 setup() keyword. When used, this should be
the only keyword in your setup() aside from `setup_requires`.
If given as a string, the value of d2to1 is assumed to be the relative path
to the setup.cfg file to use. Otherwise, if it evaluates to true, it
simply assumes that d2to1 should be used, and the default 'setup.cfg' is
used.
This works by reading the setup.cfg file, parsing out the supported
metadata and command options, and using them to rebuild the
`DistributionMetadata` object and set the newly added command options.
The reason for doing things this way is that a custom `Distribution` class
will not play nicely with setup_requires; however, this implementation may
not work well with distributions that do use a `Distribution` subclass.
"""
if not value:
return
if isinstance(value, six.string_types):
path = os.path.abspath(value)
else:
path = os.path.abspath('setup.cfg')
if not os.path.exists(path):
raise DistutilsFileError(
'The setup.cfg file %s does not exist.' % path)
# Converts the setup.cfg file to setup() arguments
try:
attrs = cfg_to_args(path)
except:
e = sys.exc_info()[1]
raise DistutilsSetupError(
'Error parsing %s: %s: %s' % (path, e.__class__.__name__, e))
# Repeat some of the Distribution initialization code with the newly
# provided attrs
if attrs:
# Skips 'options' and 'licence' support which are rarely used; may add
# back in later if demanded
for key, val in six.iteritems(attrs):
if hasattr(dist.metadata, 'set_' + key):
getattr(dist.metadata, 'set_' + key)(val)
elif hasattr(dist.metadata, key):
setattr(dist.metadata, key, val)
elif hasattr(dist, key):
setattr(dist, key, val)
else:
msg = 'Unknown distribution option: %s' % repr(key)
warnings.warn(msg)
# Re-finalize the underlying Distribution
_Distribution.finalize_options(dist)
# This bit comes out of distribute/setuptools
if isinstance(dist.metadata.version, six.integer_types + (float,)):
# Some people apparently take "version number" too literally :)
dist.metadata.version = str(dist.metadata.version)
# This bit of hackery is necessary so that the Distribution will ignore
# normally unsupport command options (namely pre-hooks and post-hooks).
# dist.command_options is normally a dict mapping command names to dicts of
# their options. Now it will be a defaultdict that returns IgnoreDicts for
# the each command's options so we can pass through the unsupported options
ignore = ['pre_hook.*', 'post_hook.*']
dist.command_options = DefaultGetDict(lambda: IgnoreDict(ignore))

0
pbr/d2to1/extern/__init__.py vendored
View File

386
pbr/d2to1/extern/six.py vendored
View File

@ -1,386 +0,0 @@
# Copyright (c) 2010-2011 Benjamin Peterson
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
"""Utilities for writing code that runs on Python 2 and 3"""
import operator
import sys
import types
__author__ = "Benjamin Peterson <benjamin@python.org>"
__version__ = "1.2.0"
# True if we are running on Python 3.
PY3 = sys.version_info[0] == 3
if PY3:
string_types = str,
integer_types = int,
class_types = type,
text_type = str
binary_type = bytes
MAXSIZE = sys.maxsize
else:
string_types = basestring,
integer_types = (int, long)
class_types = (type, types.ClassType)
text_type = unicode
binary_type = str
if sys.platform == "java":
# Jython always uses 32 bits.
MAXSIZE = int((1 << 31) - 1)
else:
# It's possible to have sizeof(long) != sizeof(Py_ssize_t).
class X(object):
def __len__(self):
return 1 << 31
try:
len(X())
except OverflowError:
# 32-bit
MAXSIZE = int((1 << 31) - 1)
else:
# 64-bit
MAXSIZE = int((1 << 63) - 1)
del X
def _add_doc(func, doc):
"""Add documentation to a function."""
func.__doc__ = doc
def _import_module(name):
"""Import module, returning the module after the last dot."""
__import__(name)
return sys.modules[name]
class _LazyDescr(object):
def __init__(self, name):
self.name = name
def __get__(self, obj, tp):
result = self._resolve()
setattr(obj, self.name, result)
# This is a bit ugly, but it avoids running this again.
delattr(tp, self.name)
return result
class MovedModule(_LazyDescr):
def __init__(self, name, old, new=None):
super(MovedModule, self).__init__(name)
if PY3:
if new is None:
new = name
self.mod = new
else:
self.mod = old
def _resolve(self):
return _import_module(self.mod)
class MovedAttribute(_LazyDescr):
def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
super(MovedAttribute, self).__init__(name)
if PY3:
if new_mod is None:
new_mod = name
self.mod = new_mod
if new_attr is None:
if old_attr is None:
new_attr = name
else:
new_attr = old_attr
self.attr = new_attr
else:
self.mod = old_mod
if old_attr is None:
old_attr = name
self.attr = old_attr
def _resolve(self):
module = _import_module(self.mod)
return getattr(module, self.attr)
class _MovedItems(types.ModuleType):
"""Lazy loading of moved objects"""
_moved_attributes = [
MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"),
MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
MovedAttribute("map", "itertools", "builtins", "imap", "map"),
MovedAttribute("reload_module", "__builtin__", "imp", "reload"),
MovedAttribute("reduce", "__builtin__", "functools"),
MovedAttribute("StringIO", "StringIO", "io"),
MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
MovedModule("builtins", "__builtin__"),
MovedModule("configparser", "ConfigParser"),
MovedModule("copyreg", "copy_reg"),
MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
MovedModule("http_cookies", "Cookie", "http.cookies"),
MovedModule("html_entities", "htmlentitydefs", "html.entities"),
MovedModule("html_parser", "HTMLParser", "html.parser"),
MovedModule("http_client", "httplib", "http.client"),
MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"),
MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"),
MovedModule("cPickle", "cPickle", "pickle"),
MovedModule("queue", "Queue"),
MovedModule("reprlib", "repr"),
MovedModule("socketserver", "SocketServer"),
MovedModule("tkinter", "Tkinter"),
MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
MovedModule("tkinter_colorchooser", "tkColorChooser",
"tkinter.colorchooser"),
MovedModule("tkinter_commondialog", "tkCommonDialog",
"tkinter.commondialog"),
MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"),
MovedModule("tkinter_font", "tkFont", "tkinter.font"),
MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"),
MovedModule("tkinter_tksimpledialog", "tkSimpleDialog",
"tkinter.simpledialog"),
MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
MovedModule("winreg", "_winreg"),
]
for attr in _moved_attributes:
setattr(_MovedItems, attr.name, attr)
del attr
moves = sys.modules["six.moves"] = _MovedItems("moves")
def add_move(move):
"""Add an item to six.moves."""
setattr(_MovedItems, move.name, move)
def remove_move(name):
"""Remove item from six.moves."""
try:
delattr(_MovedItems, name)
except AttributeError:
try:
del moves.__dict__[name]
except KeyError:
raise AttributeError("no such move, %r" % (name,))
if PY3:
_meth_func = "__func__"
_meth_self = "__self__"
_func_code = "__code__"
_func_defaults = "__defaults__"
_iterkeys = "keys"
_itervalues = "values"
_iteritems = "items"
else:
_meth_func = "im_func"
_meth_self = "im_self"
_func_code = "func_code"
_func_defaults = "func_defaults"
_iterkeys = "iterkeys"
_itervalues = "itervalues"
_iteritems = "iteritems"
try:
advance_iterator = next
except NameError:
def advance_iterator(it):
return it.next()
next = advance_iterator
if PY3:
def get_unbound_function(unbound):
return unbound
Iterator = object
def callable(obj):
return any("__call__" in klass.__dict__ for klass in type(obj).__mro__)
else:
def get_unbound_function(unbound):
return unbound.im_func
class Iterator(object):
def next(self):
return type(self).__next__(self)
callable = callable
_add_doc(get_unbound_function,
"""Get the function out of a possibly unbound function""")
get_method_function = operator.attrgetter(_meth_func)
get_method_self = operator.attrgetter(_meth_self)
get_function_code = operator.attrgetter(_func_code)
get_function_defaults = operator.attrgetter(_func_defaults)
def iterkeys(d):
"""Return an iterator over the keys of a dictionary."""
return iter(getattr(d, _iterkeys)())
def itervalues(d):
"""Return an iterator over the values of a dictionary."""
return iter(getattr(d, _itervalues)())
def iteritems(d):
"""Return an iterator over the (key, value) pairs of a dictionary."""
return iter(getattr(d, _iteritems)())
if PY3:
def b(s):
return s.encode("latin-1")
def u(s):
return s
if sys.version_info[1] <= 1:
def int2byte(i):
return bytes((i,))
else:
# This is about 2x faster than the implementation above on 3.2+
int2byte = operator.methodcaller("to_bytes", 1, "big")
import io
StringIO = io.StringIO
BytesIO = io.BytesIO
else:
def b(s):
return s
def u(s):
return unicode(s, "unicode_escape")
int2byte = chr
import StringIO
StringIO = BytesIO = StringIO.StringIO
_add_doc(b, """Byte literal""")
_add_doc(u, """Text literal""")
if PY3:
import builtins
exec_ = getattr(builtins, "exec")
def reraise(tp, value, tb=None):
if value.__traceback__ is not tb:
raise value.with_traceback(tb)
raise value
print_ = getattr(builtins, "print")
del builtins
else:
def exec_(code, globs=None, locs=None):
"""Execute code in a namespace."""
if globs is None:
frame = sys._getframe(1)
globs = frame.f_globals
if locs is None:
locs = frame.f_locals
del frame
elif locs is None:
locs = globs
exec("""exec code in globs, locs""")
exec_("""def reraise(tp, value, tb=None):
raise tp, value, tb
""")
def print_(*args, **kwargs):
"""The new-style print function."""
fp = kwargs.pop("file", sys.stdout)
if fp is None:
return
def write(data):
if not isinstance(data, basestring):
data = str(data)
fp.write(data)
want_unicode = False
sep = kwargs.pop("sep", None)
if sep is not None:
if isinstance(sep, unicode):
want_unicode = True
elif not isinstance(sep, str):
raise TypeError("sep must be None or a string")
end = kwargs.pop("end", None)
if end is not None:
if isinstance(end, unicode):
want_unicode = True
elif not isinstance(end, str):
raise TypeError("end must be None or a string")
if kwargs:
raise TypeError("invalid keyword arguments to print()")
if not want_unicode:
for arg in args:
if isinstance(arg, unicode):
want_unicode = True
break
if want_unicode:
newline = unicode("\n")
space = unicode(" ")
else:
newline = "\n"
space = " "
if sep is None:
sep = space
if end is None:
end = newline
for i, arg in enumerate(args):
if i:
write(sep)
write(arg)
write(end)
_add_doc(reraise, """Reraise an exception.""")
def with_metaclass(meta, base=object):
"""Create a base class with a metaclass."""
return meta("NewBase", (base,), {})

110
pbr/d2to1/tests/__init__.py
View File

@ -1,81 +1,77 @@
from __future__ import with_statement
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
import os
import shutil
import subprocess
import sys
import tempfile
import pkg_resources
from .util import rmtree, open_config
import fixtures
import testtools
D2TO1_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__),
os.pardir, os.pardir))
def fake_d2to1_dist():
# Fake a d2to1 distribution from the d2to1 package that these tests reside
# in and make sure it's active on the path with the appropriate entry
# points installed
class _FakeProvider(pkg_resources.EmptyProvider):
"""A fake metadata provider that does almost nothing except to return
entry point metadata.
"""
def has_metadata(self, name):
return name == 'entry_points.txt'
def get_metadata(self, name):
if name == 'entry_points.txt':
return '[distutils.setup_keywords]\nd2to1 = d2to1.core:d2to1\n'
else:
return ''
sys.path.insert(0, D2TO1_DIR)
if 'd2to1' in sys.modules:
del sys.modules['d2to1']
if 'd2to1' in pkg_resources.working_set.by_key:
del pkg_resources.working_set.by_key['d2to1']
dist = pkg_resources.Distribution(location=D2TO1_DIR, project_name='d2to1',
metadata=_FakeProvider())
pkg_resources.working_set.add(dist)
class D2to1TestCase(object):
def setup(self):
self.temp_dir = tempfile.mkdtemp(prefix='d2to1-test-')
class D2to1TestCase(testtools.TestCase):
def setUp(self):
super(D2to1TestCase, self).setUp()
self.temp_dir = self.useFixture(fixtures.TempDir()).path
self.package_dir = os.path.join(self.temp_dir, 'testpackage')
shutil.copytree(os.path.join(os.path.dirname(__file__), 'testpackage'),
self.package_dir)
self.oldcwd = os.getcwd()
self.addCleanup(os.chdir, os.getcwd())
os.chdir(self.package_dir)
def teardown(self):
os.chdir(self.oldcwd)
def tearDown(self):
# Remove d2to1.testpackage from sys.modules so that it can be freshly
# re-imported by the next test
for k in list(sys.modules):
if (k == 'd2to1_testpackage' or
k.startswith('d2to1_testpackage.')):
k.startswith('d2to1_testpackage.')):
del sys.modules[k]
rmtree(self.temp_dir)
super(D2to1TestCase, self).tearDown()
def run_setup(self, *args):
cmd = ('-c',
'import sys;sys.path.insert(0, %r);'
'from d2to1.tests import fake_d2to1_dist;'
'from d2to1.extern.six import exec_;'
'fake_d2to1_dist();exec_(open("setup.py").read())' % D2TO1_DIR)
return self._run_cmd(sys.executable, cmd + args)
def run_svn(self, *args):
return self._run_cmd('svn', args)
return self._run_cmd(sys.executable, ('setup.py',) + args)
def _run_cmd(self, cmd, args):
"""
"""Run a command in the root of the test working copy.
Runs a command, with the given argument list, in the root of the test
working copy--returns the stdout and stderr streams and the exit code
from the subprocess.

57
pbr/d2to1/tests/test_commands.py
View File

@ -1,13 +1,58 @@
from . import D2to1TestCase
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
from testtools import content
from pbr.d2to1 import tests
class TestCommands(D2to1TestCase):
class TestCommands(tests.D2to1TestCase):
def test_custom_build_py_command(self):
"""
"""Test custom build_py command.
Test that a custom subclass of the build_py command runs when listed in
the commands [global] option, rather than the normal build command.
"""
stdout, _, return_code = self.run_setup('build_py')
assert 'Running custom build_py command.' in stdout
assert return_code == 0
stdout, stderr, return_code = self.run_setup('build_py')
self.addDetail('stdout', content.text_content(stdout))
self.addDetail('stderr', content.text_content(stderr))
self.assertIn('Running custom build_py command.', stdout)
self.assertEqual(return_code, 0)

63
pbr/d2to1/tests/test_core.py
View File

@ -1,26 +1,55 @@
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
import glob
import os
import tarfile
from . import D2to1TestCase
from pbr.d2to1 import tests
VERSION = '0.1.dev'
class TestCore(D2to1TestCase):
def test_setup_py_version(self):
"""
Test that the `./setup.py --version` command returns the correct
value without balking.
"""
self.run_setup('egg_info')
stdout, _, _ = self.run_setup('--version')
assert stdout == VERSION
class TestCore(tests.D2to1TestCase):
def test_setup_py_keywords(self):
"""
"""setup.py --keywords.
Test that the `./setup.py --keywords` command returns the correct
value without balking.
"""
@ -30,9 +59,7 @@ class TestCore(D2to1TestCase):
assert stdout == 'packaging,distutils,setuptools'
def test_sdist_extra_files(self):
"""
Test that the extra files are correctly added.
"""
"""Test that the extra files are correctly added."""
stdout, _, return_code = self.run_setup('sdist', '--formats=gztar')

61
pbr/d2to1/tests/test_hooks.py
View File

@ -1,16 +1,55 @@
from __future__ import with_statement
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
import os
import textwrap
from . import D2to1TestCase
from .util import open_config
from pbr.d2to1 import tests
from pbr.d2to1.tests import util
class TestHooks(D2to1TestCase):
def setup(self):
super(TestHooks, self).setup()
with open_config(os.path.join(self.package_dir, 'setup.cfg')) as cfg:
class TestHooks(tests.D2to1TestCase):
def setUp(self):
super(TestHooks, self).setUp()
with util.open_config(
os.path.join(self.package_dir, 'setup.cfg')) as cfg:
cfg.set('global', 'setup-hooks',
'd2to1_testpackage._setup_hooks.test_hook_1\n'
'd2to1_testpackage._setup_hooks.test_hook_2')
@ -20,7 +59,8 @@ class TestHooks(D2to1TestCase):
'd2to1_testpackage._setup_hooks.test_post_hook')
def test_global_setup_hooks(self):
"""
"""Test setup_hooks.
Test that setup_hooks listed in the [global] section of setup.cfg are
executed in order.
"""
@ -30,7 +70,8 @@ class TestHooks(D2to1TestCase):
assert return_code == 0
def test_command_hooks(self):
"""
"""Test command hooks.
Simple test that the appropriate command hooks run at the
beginning/end of the appropriate command.
"""
@ -45,7 +86,7 @@ class TestHooks(D2to1TestCase):
running build_ext
running pre_hook d2to1_testpackage._setup_hooks.test_pre_hook for command build_ext
build_ext pre-hook
""") in stdout
""") in stdout # flake8: noqa
assert stdout.endswith('build_ext post-hook')
assert return_code == 0

46
pbr/d2to1/tests/testpackage/d2to1_testpackage/_setup_hooks.py
View File

@ -1,4 +1,44 @@
from distutils.command.build_py import build_py
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
from distutils.command import build_py
def test_hook_1(config):
@ -9,12 +49,12 @@ def test_hook_2(config):
print('test_hook_2')
class test_command(build_py):
class test_command(build_py.build_py):
command_name = 'build_py'
def run(self):
print('Running custom build_py command.')
return build_py.run(self)
return build_py.build_py.run(self)
def test_pre_hook(cmdobj):

485
pbr/d2to1/tests/testpackage/distribute_setup.py
View File

@ -1,485 +0,0 @@
#!python
"""Bootstrap distribute installation
If you want to use setuptools in your package's setup.py, just include this
file in the same directory with it, and add this to the top of your setup.py::
from distribute_setup import use_setuptools
use_setuptools()
If you want to require a specific version of setuptools, set a download
mirror, or use an alternate download directory, you can do so by supplying
the appropriate options to ``use_setuptools()``.
This file can also be run as a script to install or upgrade setuptools.
"""
import os
import sys
import time
import fnmatch
import tempfile
import tarfile
from distutils import log
try:
from site import USER_SITE
except ImportError:
USER_SITE = None
try:
import subprocess
def _python_cmd(*args):
args = (sys.executable,) + args
return subprocess.call(args) == 0
except ImportError:
# will be used for python 2.3
def _python_cmd(*args):
args = (sys.executable,) + args
# quoting arguments if windows
if sys.platform == 'win32':
def quote(arg):
if ' ' in arg:
return '"%s"' % arg
return arg
args = [quote(arg) for arg in args]
return os.spawnl(os.P_WAIT, sys.executable, *args) == 0
DEFAULT_VERSION = "0.6.19"
DEFAULT_URL = "http://pypi.python.org/packages/source/d/distribute/"
SETUPTOOLS_FAKED_VERSION = "0.6c11"
SETUPTOOLS_PKG_INFO = """\
Metadata-Version: 1.0
Name: setuptools
Version: %s
Summary: xxxx
Home-page: xxx
Author: xxx
Author-email: xxx
License: xxx
Description: xxx
""" % SETUPTOOLS_FAKED_VERSION
def _install(tarball):
# extracting the tarball
tmpdir = tempfile.mkdtemp()
log.warn('Extracting in %s', tmpdir)
old_wd = os.getcwd()
try:
os.chdir(tmpdir)
tar = tarfile.open(tarball)
_extractall(tar)
tar.close()
# going in the directory
subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0])
os.chdir(subdir)
log.warn('Now working in %s', subdir)
# installing
log.warn('Installing Distribute')
if not _python_cmd('setup.py', 'install'):
log.warn('Something went wrong during the installation.')
log.warn('See the error message above.')
finally:
os.chdir(old_wd)
def _build_egg(egg, tarball, to_dir):
# extracting the tarball
tmpdir = tempfile.mkdtemp()
log.warn('Extracting in %s', tmpdir)
old_wd = os.getcwd()
try:
os.chdir(tmpdir)
tar = tarfile.open(tarball)
_extractall(tar)
tar.close()
# going in the directory
subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0])
os.chdir(subdir)
log.warn('Now working in %s', subdir)
# building an egg
log.warn('Building a Distribute egg in %s', to_dir)
_python_cmd('setup.py', '-q', 'bdist_egg', '--dist-dir', to_dir)
finally:
os.chdir(old_wd)
# returning the result
log.warn(egg)
if not os.path.exists(egg):
raise IOError('Could not build the egg.')
def _do_download(version, download_base, to_dir, download_delay):
egg = os.path.join(to_dir, 'distribute-%s-py%d.%d.egg'
% (version, sys.version_info[0], sys.version_info[1]))
if not os.path.exists(egg):
tarball = download_setuptools(version, download_base,
to_dir, download_delay)
_build_egg(egg, tarball, to_dir)
sys.path.insert(0, egg)
import setuptools
setuptools.bootstrap_install_from = egg
def use_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL,
to_dir=os.curdir, download_delay=15, no_fake=True):
# making sure we use the absolute path
to_dir = os.path.abspath(to_dir)
was_imported = 'pkg_resources' in sys.modules or \
'setuptools' in sys.modules
try:
try:
import pkg_resources
if not hasattr(pkg_resources, '_distribute'):
if not no_fake:
_fake_setuptools()
raise ImportError
except ImportError:
return _do_download(version, download_base, to_dir, download_delay)
try:
pkg_resources.require("distribute>="+version)
return
except pkg_resources.VersionConflict:
e = sys.exc_info()[1]
if was_imported:
sys.stderr.write(
"The required version of distribute (>=%s) is not available,\n"
"and can't be installed while this script is running. Please\n"
"install a more recent version first, using\n"
"'easy_install -U distribute'."
"\n\n(Currently using %r)\n" % (version, e.args[0]))
sys.exit(2)
else:
del pkg_resources, sys.modules['pkg_resources'] # reload ok
return _do_download(version, download_base, to_dir,
download_delay)
except pkg_resources.DistributionNotFound:
return _do_download(version, download_base, to_dir,
download_delay)
finally:
if not no_fake:
_create_fake_setuptools_pkg_info(to_dir)
def download_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL,
to_dir=os.curdir, delay=15):
"""Download distribute from a specified location and return its filename
`version` should be a valid distribute version number that is available
as an egg for download under the `download_base` URL (which should end
with a '/'). `to_dir` is the directory where the egg will be downloaded.
`delay` is the number of seconds to pause before an actual download
attempt.
"""
# making sure we use the absolute path
to_dir = os.path.abspath(to_dir)
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
tgz_name = "distribute-%s.tar.gz" % version
url = download_base + tgz_name
saveto = os.path.join(to_dir, tgz_name)
src = dst = None
if not os.path.exists(saveto): # Avoid repeated downloads
try:
log.warn("Downloading %s", url)
src = urlopen(url)
# Read/write all in one block, so we don't create a corrupt file
# if the download is interrupted.
data = src.read()
dst = open(saveto, "wb")
dst.write(data)
finally:
if src:
src.close()
if dst:
dst.close()
return os.path.realpath(saveto)
def _no_sandbox(function):
def __no_sandbox(*args, **kw):
try:
from setuptools.sandbox import DirectorySandbox
if not hasattr(DirectorySandbox, '_old'):
def violation(*args):
pass
DirectorySandbox._old = DirectorySandbox._violation
DirectorySandbox._violation = violation
patched = True
else:
patched = False
except ImportError:
patched = False
try:
return function(*args, **kw)
finally:
if patched:
DirectorySandbox._violation = DirectorySandbox._old
del DirectorySandbox._old
return __no_sandbox
def _patch_file(path, content):
"""Will backup the file then patch it"""
existing_content = open(path).read()
if existing_content == content:
# already patched
log.warn('Already patched.')
return False
log.warn('Patching...')
_rename_path(path)
f = open(path, 'w')
try:
f.write(content)
finally:
f.close()
return True
_patch_file = _no_sandbox(_patch_file)
def _same_content(path, content):
return open(path).read() == content
def _rename_path(path):
new_name = path + '.OLD.%s' % time.time()
log.warn('Renaming %s into %s', path, new_name)
os.rename(path, new_name)
return new_name
def _remove_flat_installation(placeholder):
if not os.path.isdir(placeholder):
log.warn('Unkown installation at %s', placeholder)
return False
found = False
for file in os.listdir(placeholder):
if fnmatch.fnmatch(file, 'setuptools*.egg-info'):
found = True
break
if not found:
log.warn('Could not locate setuptools*.egg-info')
return
log.warn('Removing elements out of the way...')
pkg_info = os.path.join(placeholder, file)
if os.path.isdir(pkg_info):
patched = _patch_egg_dir(pkg_info)
else:
patched = _patch_file(pkg_info, SETUPTOOLS_PKG_INFO)
if not patched:
log.warn('%s already patched.', pkg_info)
return False
# now let's move the files out of the way
for element in ('setuptools', 'pkg_resources.py', 'site.py'):
element = os.path.join(placeholder, element)
if os.path.exists(element):
_rename_path(element)
else:
log.warn('Could not find the %s element of the '
'Setuptools distribution', element)
return True
_remove_flat_installation = _no_sandbox(_remove_flat_installation)
def _after_install(dist):
log.warn('After install bootstrap.')
placeholder = dist.get_command_obj('install').install_purelib
_create_fake_setuptools_pkg_info(placeholder)
def _create_fake_setuptools_pkg_info(placeholder):
if not placeholder or not os.path.exists(placeholder):
log.warn('Could not find the install location')
return
pyver = '%s.%s' % (sys.version_info[0], sys.version_info[1])
setuptools_file = 'setuptools-%s-py%s.egg-info' % \
(SETUPTOOLS_FAKED_VERSION, pyver)
pkg_info = os.path.join(placeholder, setuptools_file)
if os.path.exists(pkg_info):
log.warn('%s already exists', pkg_info)
return
log.warn('Creating %s', pkg_info)
f = open(pkg_info, 'w')
try:
f.write(SETUPTOOLS_PKG_INFO)
finally:
f.close()
pth_file = os.path.join(placeholder, 'setuptools.pth')
log.warn('Creating %s', pth_file)
f = open(pth_file, 'w')
try:
f.write(os.path.join(os.curdir, setuptools_file))
finally:
f.close()
_create_fake_setuptools_pkg_info = _no_sandbox(_create_fake_setuptools_pkg_info)
def _patch_egg_dir(path):
# let's check if it's already patched
pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO')
if os.path.exists(pkg_info):
if _same_content(pkg_info, SETUPTOOLS_PKG_INFO):
log.warn('%s already patched.', pkg_info)
return False
_rename_path(path)
os.mkdir(path)
os.mkdir(os.path.join(path, 'EGG-INFO'))
pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO')
f = open(pkg_info, 'w')
try:
f.write(SETUPTOOLS_PKG_INFO)
finally:
f.close()
return True
_patch_egg_dir = _no_sandbox(_patch_egg_dir)
def _before_install():
log.warn('Before install bootstrap.')
_fake_setuptools()
def _under_prefix(location):
if 'install' not in sys.argv:
return True
args = sys.argv[sys.argv.index('install')+1:]
for index, arg in enumerate(args):
for option in ('--root', '--prefix'):
if arg.startswith('%s=' % option):
top_dir = arg.split('root=')[-1]
return location.startswith(top_dir)
elif arg == option:
if len(args) > index:
top_dir = args[index+1]
return location.startswith(top_dir)
if arg == '--user' and USER_SITE is not None:
return location.startswith(USER_SITE)
return True
def _fake_setuptools():
log.warn('Scanning installed packages')
try:
import pkg_resources
except ImportError:
# we're cool
log.warn('Setuptools or Distribute does not seem to be installed.')
return
ws = pkg_resources.working_set
try:
setuptools_dist = ws.find(pkg_resources.Requirement.parse('setuptools',
replacement=False))
except TypeError:
# old distribute API
setuptools_dist = ws.find(pkg_resources.Requirement.parse('setuptools'))
if setuptools_dist is None:
log.warn('No setuptools distribution found')
return
# detecting if it was already faked
setuptools_location = setuptools_dist.location
log.warn('Setuptools installation detected at %s', setuptools_location)
# if --root or --preix was provided, and if
# setuptools is not located in them, we don't patch it
if not _under_prefix(setuptools_location):
log.warn('Not patching, --root or --prefix is installing Distribute'
' in another location')
return
# let's see if its an egg
if not setuptools_location.endswith('.egg'):
log.warn('Non-egg installation')
res = _remove_flat_installation(setuptools_location)
if not res:
return
else:
log.warn('Egg installation')
pkg_info = os.path.join(setuptools_location, 'EGG-INFO', 'PKG-INFO')
if (os.path.exists(pkg_info) and
_same_content(pkg_info, SETUPTOOLS_PKG_INFO)):
log.warn('Already patched.')
return
log.warn('Patching...')
# let's create a fake egg replacing setuptools one
res = _patch_egg_dir(setuptools_location)
if not res:
return
log.warn('Patched done.')
_relaunch()
def _relaunch():
log.warn('Relaunching...')
# we have to relaunch the process
# pip marker to avoid a relaunch bug
if sys.argv[:3] == ['-c', 'install', '--single-version-externally-managed']:
sys.argv[0] = 'setup.py'
args = [sys.executable] + sys.argv
sys.exit(subprocess.call(args))
def _extractall(self, path=".", members=None):
"""Extract all members from the archive to the current working
directory and set owner, modification time and permissions on
directories afterwards. `path' specifies a different directory
to extract to. `members' is optional and must be a subset of the
list returned by getmembers().
"""
import copy
import operator
from tarfile import ExtractError
directories = []
if members is None:
members = self
for tarinfo in members:
if tarinfo.isdir():
# Extract directories with a safe mode.
directories.append(tarinfo)
tarinfo = copy.copy(tarinfo)
tarinfo.mode = 448 # decimal for oct 0700
self.extract(tarinfo, path)
# Reverse sort directories.
if sys.version_info < (2, 4):
def sorter(dir1, dir2):
return cmp(dir1.name, dir2.name)
directories.sort(sorter)
directories.reverse()
else:
directories.sort(key=operator.attrgetter('name'), reverse=True)
# Set correct owner, mtime and filemode on directories.
for tarinfo in directories:
dirpath = os.path.join(path, tarinfo.name)
try:
self.chown(tarinfo, dirpath)
self.utime(tarinfo, dirpath)
self.chmod(tarinfo, dirpath)
except ExtractError:
e = sys.exc_info()[1]
if self.errorlevel > 1:
raise
else:
self._dbg(1, "tarfile: %s" % e)
def main(argv, version=DEFAULT_VERSION):
"""Install or upgrade setuptools and EasyInstall"""
tarball = download_setuptools()
_install(tarball)
if __name__ == '__main__':
main(sys.argv[1:])

28
pbr/d2to1/tests/testpackage/setup.py
View File

@ -1,12 +1,22 @@
#!/usr/bin/env python
try:
from setuptools import setup
except ImportError:
from distribute_setup import use_setuptools
use_setuptools()
from setuptools import setup
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
setup(
setup_requires=['d2to1'],
d2to1=True,
import setuptools
setuptools.setup(
setup_requires=['pbr'],
pbr=True,
)

55
pbr/d2to1/tests/util.py
View File

@ -1,18 +1,57 @@
from __future__ import with_statement
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
import contextlib
import os
import shutil
import stat
from ..extern.six import moves as m
ConfigParser = m.configparser.ConfigParser
try:
import configparser
except ImportError:
import ConfigParser as configparser
@contextlib.contextmanager
def open_config(filename):
cfg = ConfigParser()
cfg = configparser.ConfigParser()
cfg.read(filename)
yield cfg
with open(filename, 'w') as fp:
@ -20,9 +59,9 @@ def open_config(filename):
def rmtree(path):
"""
shutil.rmtree() with error handler for 'access denied' from trying to
delete read-only files.
"""shutil.rmtree() with error handler.
Handle 'access denied' from trying to delete read-only files.
"""
def onerror(func, path, exc_info):

52
pbr/d2to1/util.py
View File

@ -1,3 +1,43 @@
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (C) 2013 Association of Universities for Research in Astronomy
# (AURA)
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
#
# 3. The name of AURA and its representatives may not be used to
# endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
"""The code in this module is mostly copy/pasted out of the distutils2 source
code, as recommended by Tarek Ziade. As such, it may be subject to some change
as distutils2 development continues, and will have to be kept up to date.
@ -11,10 +51,10 @@ to be an installation dependency for our packages yet--it is still too unstable
# irritating Python bug that can crop up when using ./setup.py test.
# See: http://www.eby-sarna.com/pipermail/peak/2010-May/003355.html
try:
import multiprocessing
import multiprocessing # flake8: noqa
except ImportError:
pass
import logging
import logging # flake8: noqa
import os
import re
@ -32,8 +72,10 @@ from setuptools.command.egg_info import manifest_maker
from setuptools.dist import Distribution
from setuptools.extension import Extension
from .extern.six import moves as m
RawConfigParser = m.configparser.RawConfigParser
try:
import configparser
except ImportError:
import ConfigParser as configparser
# A simplified RE for this; just checks that the line ends with version
@ -155,7 +197,7 @@ def cfg_to_args(path='setup.cfg'):
"""
# The method source code really starts here.
parser = RawConfigParser()
parser = configparser.RawConfigParser()
if not os.path.exists(path):
raise DistutilsFileError("file '%s' does not exist" %
os.path.abspath(path))

159
pbr/d2to1/zestreleaser.py
View File

@ -1,159 +0,0 @@
"""zest.releaser entry points to support projects using distutils2-like
setup.cfg files. The only actual functionality this adds is to update the
version option in a setup.cfg file, if it exists. If setup.cfg does not exist,
or does not contain a version option, then this does nothing.
TODO: d2to1 theoretically supports using a different filename for setup.cfg;
this does not support that. We could hack in support, though I'm not sure how
useful the original functionality is to begin with (and it might be removed) so
we ignore that for now.
TODO: There exists a proposal
(http://mail.python.org/pipermail/distutils-sig/2011-March/017628.html) to add
a 'version-from-file' option (or something of the like) to distutils2; if this
is added then support for it should be included here as well.
"""
import logging
import os
from .extern.six import moves as m
ConfigParser = m.configparser.ConfigParser
logger = logging.getLogger(__name__)
def update_setupcfg_version(filename, version):
"""Opens the given setup.cfg file, locates the version option in the
[metadata] section, updates it to the new version.
"""
setup_cfg = open(filename).readlines()
current_section = None
updated = False
for idx, line in enumerate(setup_cfg):
m = ConfigParser.SECTCRE.match(line)
if m:
if current_section == 'metadata':
# We already parsed the entire metadata section without finding
# a version line, and are now moving into a new section
break
current_section = m.group('header')
continue
if '=' not in line:
continue
opt, val = line.split('=', 1)
opt, val = opt.strip(), val.strip()
if current_section == 'metadata' and opt == 'version':
setup_cfg[idx] = 'version = %s\n' % version
updated = True
break
if updated:
open(filename, 'w').writelines(setup_cfg)
logger.info("Set %s's version to %r" % (os.path.basename(filename),
version))
def prereleaser_middle(data):
filename = os.path.join(data['workingdir'], 'setup.cfg')
if os.path.exists(filename):
update_setupcfg_version(filename, data['new_version'])
def releaser_middle(data):
"""
releaser.middle hook to monkey-patch zest.releaser to support signed
tagging--currently this is the only way to do this. Also monkey-patches to
disable an annoyance where zest.releaser only creates .zip source
distributions. This is supposedly a workaround for a bug in Python 2.4,
but we don't care about Python 2.4.
"""
import os
import sys
from zest.releaser.git import Git
from zest.releaser.release import Releaser
# Copied verbatim from zest.releaser, but with the cmd string modified to
# use the -s option to create a signed tag
def _my_create_tag(self, version):
msg = "Tagging %s" % (version,)
cmd = 'git tag -s %s -m "%s"' % (version, msg)
if os.path.isdir('.git/svn'):
print "\nEXPERIMENTAL support for git-svn tagging!\n"
cur_branch = open('.git/HEAD').read().strip().split('/')[-1]
print "You are on branch %s." % (cur_branch,)
if cur_branch != 'master':
print "Only the master branch is supported for git-svn tagging."
print "Please tag yourself."
print "'git tag' needs to list tag named %s." % (version,)
sys.exit()
cmd = [cmd]
local_head = open('.git/refs/heads/master').read()
trunk = open('.git/refs/remotes/trunk').read()
if local_head != trunk:
print "Your local master diverges from trunk.\n"
# dcommit before local tagging
cmd.insert(0, 'git svn dcommit')
# create tag in svn
cmd.append('git svn tag -m "%s" %s' % (msg, version))
return cmd
# Similarly copied from zer.releaser to support use of 'v' in front
# of the version number
def _my_make_tag(self):
from zest.releaser import utils
from os import system
if self.data['tag_already_exists']:
return
cmds = self.vcs.cmd_create_tag(self.data['version'])
if not isinstance(cmds, list):
cmds = [cmds]
if len(cmds) == 1:
print "Tag needed to proceed, you can use the following command:"
for cmd in cmds:
print cmd
if utils.ask("Run this command"):
print system(cmd)
else:
# all commands are needed in order to proceed normally
print "Please create a tag for %s yourself and rerun." % \
(self.data['version'],)
sys.exit()
if not self.vcs.tag_exists('v' + self.data['version']):
print "\nFailed to create tag %s!" % (self.data['version'],)
sys.exit()
# Normally all this does is to return '--formats=zip', which is currently
# hard-coded as an option to always add to the sdist command; they ought to
# make this actually optional
def _my_sdist_options(self):
return ''
Git.cmd_create_tag = _my_create_tag
Releaser._make_tag = _my_make_tag
Releaser._sdist_options = _my_sdist_options
def postreleaser_before(data):
"""
Fix the irritating .dev0 default appended to new development versions by
zest.releaser to just append ".dev" without the "0".
"""
data['dev_version_template'] = '%(new_version)s.dev'
def postreleaser_middle(data):
filename = os.path.join(data['workingdir'], 'setup.cfg')
if os.path.exists(filename):
update_setupcfg_version(filename, data['dev_version'])

10
pbr/packaging.py
View File

@ -26,7 +26,6 @@ import re
import subprocess
import sys
from d2to1.extern import six
from distutils.command import install as du_install
import distutils.errors
from distutils import log
@ -34,6 +33,11 @@ import pkg_resources
from setuptools.command import install
from setuptools.command import sdist
try:
import cStringIO as io
except ImportError:
import io
log.set_verbosity(log.INFO)
TRUE_VALUES = ('true', '1', 'yes')
REQUIREMENTS_FILES = ('requirements.txt', 'tools/pip-requires')
@ -111,7 +115,7 @@ def canonicalize_emails(changelog, mapping):
"""Takes in a string and an email alias mapping and replaces all
instances of the aliases in the string with their real email.
"""
for alias, email_address in six.iteritems(mapping):
for alias, email_address in mapping.items():
changelog = changelog.replace(alias, email_address)
return changelog
@ -481,7 +485,7 @@ try:
def _sphinx_run(self):
if not self.verbose:
status_stream = six.StringIO()
status_stream = io.StringIO()
else:
status_stream = sys.stdout
confoverrides = {}

15
pbr/testr_command.py
View File

@ -1,5 +1,18 @@
#
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Copyright (c) 2013 Testrepository Contributors
#
# Licensed under either the Apache License, Version 2.0 or the BSD 3-clause

14
pbr/tests/test_setup.py
View File

@ -22,7 +22,13 @@ import os
import sys
import tempfile
from d2to1.extern import six
try:
import cStringIO as io
BytesIO = io.StringIO
except ImportError:
import io
BytesIO = io.BytesIO
import fixtures
import testscenarios
@ -165,8 +171,8 @@ class GitLogsTest(tests.BaseTestCase):
"os.path.exists",
lambda path: os.path.abspath(path) in exist_files))
self.useFixture(fixtures.FakePopen(lambda _: {
"stdout": six.BytesIO("Author: Foo Bar "
"<email@bar.com>\n".encode('utf-8'))
"stdout": BytesIO("Author: Foo Bar "
"<email@bar.com>\n".encode('utf-8'))
}))
def _fake_read_git_mailmap(*args):
@ -207,7 +213,7 @@ class GitLogsTest(tests.BaseTestCase):
lambda path: os.path.abspath(path) in exist_files))
self.useFixture(fixtures.FakePopen(lambda proc_args: {
"stdout": six.BytesIO(
"stdout": BytesIO(
self._fake_log_output(proc_args["args"][2], cmd_map))
}))

2
pbr/tests/test_version.py
View File

@ -28,4 +28,4 @@ class DeferredVersionTestCase(tests.BaseTestCase):
deferred_string = MyVersionInfo("openstack").\
cached_version_string()
self.assertEquals("5.5.5.5", deferred_string)
self.assertEqual("5.5.5.5", deferred_string)

1
requirements.txt
View File

@ -1,2 +1 @@
d2to1>=0.2.10,<0.3
setuptools_git>=0.4

5
setup.cfg
View File

@ -6,6 +6,7 @@ summary = Python Build Reasonableness
description-file =
README.rst
home-page = http://pypi.python.org/pypi/pbr
requires-python = >=2.6
classifier =
Development Status :: 4 - Beta
Environment :: Console
@ -26,3 +27,7 @@ setup-hooks =
[pbr]
warnerrors = True
[entry_points]
distutils.setup_keywords =
pbr = pbr.core:pbr

5
setup.py
View File

@ -16,6 +16,7 @@
import setuptools
from pbr.d2to1 import util
setuptools.setup(
setup_requires=['d2to1>=0.2.10,<0.3'],
d2to1=True)
**util.cfg_to_args())

14
tools/integration.sh
View File

@ -41,8 +41,14 @@ mkdir -p $pypidir
jeepybvenv=$tmpdir/jeepyb
rm -f ~/.pip/pip.conf ~/.pydistutils.cfg
mkdir -p ~/.pip
mkvenv $jeepybvenv distribute pip
cat <<EOF > ~/.pip/pip.conf
[global]
log = /home/jenkins/pip.log
EOF
mkvenv $jeepybvenv 'setuptools>=0.7' pip
$jeepybvenv/bin/pip install -U git+https://review.openstack.org/p/openstack-infra/jeepyb.git
cat <<EOF > $tmpdir/mirror.yaml
@ -77,11 +83,11 @@ cat <<EOF > ~/.pydistutils.cfg
index_url = $pypiurl
EOF
mkdir -p ~/.pip
cat <<EOF > ~/.pip/pip.conf
[global]
index-url = $pypiurl
extra-index-url = http://pypi.openstack.org/openstack
log = /home/jenkins/pip.log
EOF
projectdir=$tmpdir/projects
@ -98,6 +104,10 @@ for PROJECT in $PROJECTS ; do
# Tempest doesn't really install
continue
fi
if [ $SHORT_PROJECT = 'requirements' ]; then
# requirements doesn't really install
continue
fi
shortprojectdir=$projectdir/$SHORT_PROJECT
git clone $REPODIR/$SHORT_PROJECT $shortprojectdir

2
tox.ini
View File

@ -9,7 +9,7 @@ setenv = VIRTUAL_ENV={envdir}
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
commands =
python setup.py testr --slowest --testr-args='{posargs}'
python setup.py testr --testr-args='{posargs}'
[tox:jenkins]
sitepackages = True