openstack/deb-python-eventlet
Code Issues Proposed changes

python3 compat: remove lots of Python 2.5 and earlier dependent code; use print() function syntax

Browse Source
This commit is contained in:
Sergey Shepelev
2013-10-07 16:48:34 +04:00
parent 35f600600c
commit d2bbbd79d3
23 changed files with 439 additions and 568 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.travis.yml
View File

@@ -1,18 +1,12 @@
language: python
python: 2.7
env:
- TOX_ENV=py25selects
- TOX_ENV=py25poll
- TOX_ENV=py26selects
- TOX_ENV=py26poll
- TOX_ENV=py26epolls
- TOX_ENV=py27selects
- TOX_ENV=py27poll
- TOX_ENV=py27epolls
matrix:
allow_failures:
- env: TOX_ENV=py25selects
- env: TOX_ENV=py25poll
install:
- sudo apt-get update -qq
- sudo apt-get install -qq libssl-dev libmysqlclient-dev libpq-dev

27
doc/modules/timeout.rst
View File

@@ -11,7 +11,7 @@
finally:
timeout.cancel()
When *exception* is omitted or ``None``, the :class:`Timeout` instance
When *exception* is omitted or ``None``, the :class:`Timeout` instance
itself is raised:
>>> Timeout(0.1)
@@ -20,16 +20,15 @@
...
Timeout: 0.1 seconds
In Python 2.5 and newer, you can use the ``with`` statement for additional
convenience::
You can use the ``with`` statement for additional convenience::
with Timeout(seconds, exception) as timeout:
pass # ... code block ...
This is equivalent to the try/finally block in the first example.
There is an additional feature when using the ``with`` statement: if
*exception* is ``False``, the timeout is still raised, but the with
This is equivalent to the try/finally block in the first example.
There is an additional feature when using the ``with`` statement: if
*exception* is ``False``, the timeout is still raised, but the with
statement suppresses it, so the code outside the with-block won't see it::
data = None
@@ -39,12 +38,12 @@
... # 5 seconds passed without reading a line
else:
... # a line was read within 5 seconds
As a very special case, if *seconds* is None, the timer is not scheduled,
As a very special case, if *seconds* is None, the timer is not scheduled,
and is only useful if you're planning to raise it directly.
There are two Timeout caveats to be aware of:
* If the code block in the try/finally or with-block never cooperatively yields, the timeout cannot be raised. In Eventlet, this should rarely be a problem, but be aware that you cannot time out CPU-only operations with this class.
* If the code block catches and doesn't re-raise :class:`BaseException` (for example, with ``except:``), then it will catch the Timeout exception, and might not abort as intended.
@@ -58,7 +57,7 @@
except Timeout, t:
if t is not timeout:
raise # not my timeout
.. automethod:: cancel
.. autoattribute:: pending
@@ -76,7 +75,7 @@
:param \*\*kwds: keyword arguments to pass to *func*
:param timeout_value: value to return if timeout occurs (by default raises
:class:`Timeout`)
:rtype: Value returned by *func* if *func* returns before *seconds*, else
*timeout_value* if provided, else raises :class:`Timeout`.
@@ -88,6 +87,6 @@
data = with_timeout(30, urllib2.open, 'http://www.google.com/', timeout_value="")
Here *data* is either the result of the ``get()`` call, or the empty string
if it took too long to return. Any exception raised by the ``get()`` call
Here *data* is either the result of the ``get()`` call, or the empty string
if it took too long to return. Any exception raised by the ``get()`` call
is passed through to the caller.

14
doc/testing.rst
View File

@@ -6,7 +6,7 @@ Eventlet is tested using `Nose <http://somethingaboutorange.com/mrl/projects/nos
.. code-block:: sh
$ python setup.py test
If you want access to all the nose plugins via command line, you can run:
.. code-block:: sh
@@ -23,8 +23,6 @@ That's it! The output from running nose is the same as unittest's output, if th
Many tests are skipped based on environmental factors; for example, it makes no sense to test Twisted-specific functionality when Twisted is not installed. These are printed as S's during execution, and in the summary printed after the tests run it will tell you how many were skipped.
.. note:: If running Python version 2.4, use this command instead: ``python tests/nosewrapper.py``. There are several tests which make use of the `with` statement and therefore will cause nose grief when it tries to import them; nosewrapper.py excludes these tests so they are skipped.
Doctests
--------
@@ -33,7 +31,7 @@ To run the doctests included in many of the eventlet modules, use this command:
.. code-block :: sh
$ nosetests --with-doctest eventlet/*.py
Currently there are 16 doctests.
Standard Library Tests
@@ -46,7 +44,7 @@ There's a convenience module called all.py designed to handle the impedance mism
.. code-block:: sh
$ nosetests tests/stdlib/all.py
That will run all the tests, though the output will be a little weird because it will look like Nose is running about 20 tests, each of which consists of a bunch of sub-tests. Not all test modules are present in all versions of Python, so there will be an occasional printout of "Not importing %s, it doesn't exist in this installation/version of Python".
If you see "Ran 0 tests in 0.001s", it means that your Python installation lacks its own tests. This is usually the case for Linux distributions. One way to get the missing tests is to download a source tarball (of the same version you have installed on your system!) and copy its Lib/test directory into the correct place on your PYTHONPATH.
@@ -75,7 +73,7 @@ If you are writing a test that involves a client connecting to a spawned server,
server_sock = eventlet.listener(('127.0.0.1', 0))
client_sock = eventlet.connect(('localhost', server_sock.getsockname()[1]))
Coverage
--------
@@ -84,7 +82,7 @@ Coverage.py is an awesome tool for evaluating how much code was exercised by uni
.. code-block:: sh
nosetests --with-coverage --cover-package=eventlet
After running the tests to completion, this will emit a huge wodge of module names and line numbers. For some reason, the ``--cover-inclusive`` option breaks everything rather than serving its purpose of limiting the coverage to the local files, so don't use that.
The html option is quite useful because it generates nicely-formatted HTML that are much easier to read than line-number soup. Here's a command that generates the annotation, dumping the html files into a directory called "cover":
@@ -92,5 +90,5 @@ The html option is quite useful because it generates nicely-formatted HTML that
.. code-block:: sh
coverage html -d cover --omit='tempmod,<console>,tests'
(``tempmod`` and ``console`` are omitted because they gets thrown away at the completion of their unit tests and coverage.py isn't smart enough to detect this.)

22
eventlet/api.py
View File

@@ -6,7 +6,7 @@ import linecache
import inspect
import warnings
from eventlet.support import greenlets as greenlet, BaseException
from eventlet.support import greenlets as greenlet
from eventlet import hubs
from eventlet import greenthread
from eventlet import debug
@@ -24,7 +24,7 @@ warnings.warn("eventlet.api is deprecated! Nearly everything in it has moved "
def get_hub(*a, **kw):
warnings.warn("eventlet.api.get_hub has moved to eventlet.hubs.get_hub",
DeprecationWarning, stacklevel=2)
return hubs.get_hub(*a, **kw)
return hubs.get_hub(*a, **kw)
def get_default_hub(*a, **kw):
warnings.warn("eventlet.api.get_default_hub has moved to"
" eventlet.hubs.get_default_hub",
@@ -34,7 +34,7 @@ def use_hub(*a, **kw):
warnings.warn("eventlet.api.use_hub has moved to eventlet.hubs.use_hub",
DeprecationWarning, stacklevel=2)
return hubs.use_hub(*a, **kw)
def switch(coro, result=None, exc=None):
if exc is not None:
@@ -85,7 +85,7 @@ def connect_tcp(address, localaddr=None):
"""
warnings.warn("""eventlet.api.connect_tcp is deprecated. Please use eventlet.connect instead.""",
DeprecationWarning, stacklevel=2)
from eventlet import greenio, util
desc = greenio.GreenSocket(util.tcp_socket())
if localaddr is not None:
@@ -111,13 +111,15 @@ call_after_global = greenthread.call_after_global
class _SilentException(BaseException):
pass
class FakeTimer(object):
def cancel(self):
pass
class timeout(object):
"""Raise an exception in the block after timeout.
Example::
with timeout(10):
@@ -126,14 +128,13 @@ class timeout(object):
Assuming code block is yielding (i.e. gives up control to the hub),
an exception provided in *exc* argument will be raised
(:class:`~eventlet.api.TimeoutError` if *exc* is omitted)::
try:
with timeout(10, MySpecialError, error_arg_1):
urllib2.open('http://example.com')
except MySpecialError, e:
except MySpecialError as e:
print "special error received"
When *exc* is ``None``, code block is interrupted silently.
"""
@@ -160,10 +161,11 @@ class timeout(object):
if typ is _SilentException and value in self.throw_args:
return True
with_timeout = greenthread.with_timeout
exc_after = greenthread.exc_after
exc_after = greenthread.exc_after
sleep = greenthread.sleep
getcurrent = greenlet.getcurrent

61
eventlet/convenience.py
View File

@@ -1,11 +1,12 @@
import sys
from eventlet import greenio
from eventlet import greenthread
from eventlet import greenpool
from eventlet import greenthread
from eventlet.green import socket
from eventlet.support import greenlets as greenlet
def connect(addr, family=socket.AF_INET, bind=None):
"""Convenience function for opening client sockets.
@@ -25,7 +26,7 @@ def listen(addr, family=socket.AF_INET, backlog=50):
"""Convenience function for opening server sockets. This
socket can be used in :func:`~eventlet.serve` or a custom ``accept()`` loop.
Sets SO_REUSEADDR on the socket to save on annoyance.
Sets SO_REUSEADDR on the socket to save on annoyance.
:param addr: Address to listen on. For TCP sockets, this is a (host, port) tuple.
:param family: Socket family, optional. See :mod:`socket` documentation for available families.
@@ -39,10 +40,12 @@ def listen(addr, family=socket.AF_INET, backlog=50):
sock.listen(backlog)
return sock
class StopServe(Exception):
"""Exception class used for quitting :func:`~eventlet.serve` gracefully."""
pass
def _stop_checker(t, server_gt, conn):
try:
try:
@@ -54,29 +57,30 @@ def _stop_checker(t, server_gt, conn):
except Exception:
greenthread.kill(server_gt, *sys.exc_info())
def serve(sock, handle, concurrency=1000):
"""Runs a server on the supplied socket. Calls the function *handle* in a
"""Runs a server on the supplied socket. Calls the function *handle* in a
separate greenthread for every incoming client connection. *handle* takes
two arguments: the client socket object, and the client address::
def myhandle(client_sock, client_addr):
print "client connected", client_addr
eventlet.serve(eventlet.listen(('127.0.0.1', 9999)), myhandle)
Returning from *handle* closes the client socket.
:func:`serve` blocks the calling greenthread; it won't return until
:func:`serve` blocks the calling greenthread; it won't return until
the server completes. If you desire an immediate return,
spawn a new greenthread for :func:`serve`.
Any uncaught exceptions raised in *handle* are raised as exceptions
from :func:`serve`, terminating the server, so be sure to be aware of the
exceptions your application can raise. The return value of *handle* is
ignored.
Raise a :class:`~eventlet.StopServe` exception to gracefully terminate the
server -- that's the only way to get the server() function to return rather
Any uncaught exceptions raised in *handle* are raised as exceptions
from :func:`serve`, terminating the server, so be sure to be aware of the
exceptions your application can raise. The return value of *handle* is
ignored.
Raise a :class:`~eventlet.StopServe` exception to gracefully terminate the
server -- that's the only way to get the server() function to return rather
than raise.
The value in *concurrency* controls the maximum number of
@@ -86,7 +90,7 @@ def serve(sock, handle, concurrency=1000):
"""
pool = greenpool.GreenPool(concurrency)
server_gt = greenthread.getcurrent()
while True:
try:
conn, addr = sock.accept()
@@ -100,10 +104,9 @@ def serve(sock, handle, concurrency=1000):
def wrap_ssl(sock, *a, **kw):
"""Convenience function for converting a regular socket into an
SSL socket. Has the same interface as :func:`ssl.wrap_socket`,
but works on 2.5 or earlier, using PyOpenSSL (though note that it
ignores the *cert_reqs*, *ssl_version*, *ca_certs*,
*do_handshake_on_connect*, and *suppress_ragged_eofs* arguments
when using PyOpenSSL).
but can also use PyOpenSSL. Though, note that it ignores the
`cert_reqs`, `ssl_version`, `ca_certs`, `do_handshake_on_connect`,
and `suppress_ragged_eofs` arguments when using PyOpenSSL.
The preferred idiom is to call wrap_ssl directly on the creation
method, e.g., ``wrap_ssl(connect(addr))`` or
@@ -119,15 +122,18 @@ try:
from eventlet.green import ssl
wrap_ssl_impl = ssl.wrap_socket
except ImportError:
# < 2.6, trying PyOpenSSL
# trying PyOpenSSL
try:
from eventlet.green.OpenSSL import SSL
except ImportError:
def wrap_ssl_impl(*a, **kw):
raise ImportError("To use SSL with Eventlet, you must install PyOpenSSL or use Python 2.6 or later.")
else:
def wrap_ssl_impl(sock, keyfile=None, certfile=None, server_side=False,
cert_reqs=None, ssl_version=None, ca_certs=None,
do_handshake_on_connect=True,
do_handshake_on_connect=True,
suppress_ragged_eofs=True, ciphers=None):
# theoretically the ssl_version could be respected in this
# next line
# theoretically the ssl_version could be respected in this line
context = SSL.Context(SSL.SSLv23_METHOD)
if certfile is not None:
context.use_certificate_file(certfile)
@@ -141,8 +147,3 @@ except ImportError:
else:
connection.set_connect_state()
return connection
except ImportError:
def wrap_ssl_impl(*a, **kw):
raise ImportError("To use SSL with Eventlet, "
"you must install PyOpenSSL or use Python 2.6 or later.")

7
eventlet/debug.py
View File

@@ -161,9 +161,8 @@ def hub_blocking_detection(state=False, resolution=1):
blocking detector (don't use it in production!).
The *resolution* argument governs how long the SIGALARM timeout
waits in seconds. If on Python 2.6 or later, the implementation
uses :func:`signal.setitimer` and can be specified as a
floating-point value. On 2.5 or earlier, 1 second is the minimum.
waits in seconds. The implementation uses :func:`signal.setitimer`
and can be specified as a floating-point value.
The shorter the resolution, the greater the chance of false
positives.
"""
@@ -171,5 +170,5 @@ def hub_blocking_detection(state=False, resolution=1):
assert resolution > 0
hubs.get_hub().debug_blocking = state
hubs.get_hub().debug_blocking_resolution = resolution
if(not state):
if not state:
hubs.get_hub().block_detect_post()

16
eventlet/event.py
View File

@@ -101,7 +101,7 @@ class Event(object):
>>> evt = event.Event()
>>> def wait_on():
... retval = evt.wait()
... print("waited for", retval)
... print("waited for {0}".format(retval))
>>> _ = eventlet.spawn(wait_on)
>>> evt.send('result')
>>> eventlet.sleep(0)
@@ -134,7 +134,7 @@ class Event(object):
>>> def waiter():
... print('about to wait')
... result = evt.wait()
... print('waited for', result)
... print('waited for {0}'.format(result))
>>> _ = eventlet.spawn(waiter)
>>> eventlet.sleep(0)
about to wait
@@ -170,12 +170,12 @@ class Event(object):
def send_exception(self, *args):
"""Same as :meth:`send`, but sends an exception to waiters.
The arguments to send_exception are the same as the arguments
to ``raise``. If a single exception object is passed in, it
will be re-raised when :meth:`wait` is called, generating a
new stacktrace.
new stacktrace.
>>> from eventlet import event
>>> evt = event.Event()
>>> evt.send_exception(RuntimeError())
@@ -185,7 +185,7 @@ class Event(object):
File "eventlet/event.py", line 120, in wait
current.throw(*self._exc)
RuntimeError
If it's important to preserve the entire original stack trace,
you must pass in the entire :func:`sys.exc_info` tuple.
@@ -195,7 +195,7 @@ class Event(object):
... raise RuntimeError()
... except RuntimeError:
... evt.send_exception(*sys.exc_info())
...
...
>>> evt.wait()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
@@ -203,7 +203,7 @@ class Event(object):
current.throw(*self._exc)
File "<stdin>", line 2, in <module>
RuntimeError
Note that doing so stores a traceback object directly on the
Event object, which may cause reference cycles. See the
:func:`sys.exc_info` documentation.

27
eventlet/green/_socket_nodns.py
View File

@@ -4,7 +4,7 @@ __all__ = __socket.__all__
__patched__ = ['fromfd', 'socketpair', 'ssl', 'socket']
from eventlet.patcher import slurp_properties
slurp_properties(__socket, globals(),
slurp_properties(__socket, globals(),
ignore=__patched__, srckeys=dir(__socket))
os = __import__('os')
@@ -87,24 +87,15 @@ class GreenSSLObject(object):
try:
try:
# >= Python 2.6
from eventlet.green import ssl as ssl_module
sslerror = __socket.sslerror
__socket.ssl
def ssl(sock, certificate=None, private_key=None):
warnings.warn("socket.ssl() is deprecated. Use ssl.wrap_socket() instead.",
DeprecationWarning, stacklevel=2)
return ssl_module.sslwrap_simple(sock, private_key, certificate)
except ImportError:
# <= Python 2.5 compatibility
sslerror = __socket.sslerror
__socket.ssl
def ssl(sock, certificate=None, private_key=None):
from eventlet import util
wrapped = util.wrap_ssl(sock, certificate, private_key)
return GreenSSLObject(wrapped)
from eventlet.green import ssl as ssl_module
sslerror = __socket.sslerror
__socket.ssl
except AttributeError:
# if the real socket module doesn't have the ssl method or sslerror
# exception, we can't emulate them
pass
else:
def ssl(sock, certificate=None, private_key=None):
warnings.warn("socket.ssl() is deprecated. Use ssl.wrap_socket() instead.",
DeprecationWarning, stacklevel=2)
return ssl_module.sslwrap_simple(sock, private_key, certificate)

38
eventlet/green/ssl.py
View File

@@ -22,20 +22,20 @@ else:
__patched__ = ['SSLSocket', 'wrap_socket', 'sslwrap_simple']
class GreenSSLSocket(__ssl.SSLSocket):
""" This is a green version of the SSLSocket class from the ssl module added
""" This is a green version of the SSLSocket class from the ssl module added
in 2.6. For documentation on it, please see the Python standard
documentation.
Python nonblocking ssl objects don't give errors when the other end
of the socket is closed (they do notice when the other end is shutdown,
though). Any write/read operations will simply hang if the socket is
closed from the other end. There is no obvious fix for this problem;
it appears to be a limitation of Python's ssl object implementation.
A workaround is to set a reasonable timeout on the socket using
settimeout(), and to close/reopen the connection when a timeout
settimeout(), and to close/reopen the connection when a timeout
occurs at an unexpected juncture in the code.
"""
# we are inheriting from SSLSocket because its constructor calls
# we are inheriting from SSLSocket because its constructor calls
# do_handshake whose behavior we wish to override
def __init__(self, sock, *args, **kw):
if not isinstance(sock, GreenSocket):
@@ -44,7 +44,7 @@ class GreenSSLSocket(__ssl.SSLSocket):
self.act_non_blocking = sock.act_non_blocking
self._timeout = sock.gettimeout()
super(GreenSSLSocket, self).__init__(sock.fd, *args, **kw)
# the superclass initializer trashes the methods so we remove
# the local-object versions of them and let the actual class
# methods shine through
@@ -53,13 +53,13 @@ class GreenSSLSocket(__ssl.SSLSocket):
delattr(self, fn)
except AttributeError:
pass
def settimeout(self, timeout):
self._timeout = timeout
def gettimeout(self):
return self._timeout
def setblocking(self, flag):
if flag:
self.act_non_blocking = False
@@ -141,7 +141,7 @@ class GreenSSLSocket(__ssl.SSLSocket):
if self.act_non_blocking:
raise
if get_errno(e) == errno.EWOULDBLOCK:
trampoline(self, write=True,
trampoline(self, write=True,
timeout=self.gettimeout(), timeout_exc=timeout_exc('timed out'))
if get_errno(e) in SOCKET_CLOSED:
return ''
@@ -164,13 +164,13 @@ class GreenSSLSocket(__ssl.SSLSocket):
if self.act_non_blocking:
raise
if get_errno(e) == errno.EWOULDBLOCK:
trampoline(self, read=True,
trampoline(self, read=True,
timeout=self.gettimeout(), timeout_exc=timeout_exc('timed out'))
if get_errno(e) in SOCKET_CLOSED:
return ''
raise
def recv_into (self, buffer, nbytes=None, flags=0):
if not self.act_non_blocking:
trampoline(self, read=True, timeout=self.gettimeout(), timeout_exc=timeout_exc('timed out'))
@@ -180,7 +180,7 @@ class GreenSSLSocket(__ssl.SSLSocket):
if not self.act_non_blocking:
trampoline(self, read=True, timeout=self.gettimeout(), timeout_exc=timeout_exc('timed out'))
return super(GreenSSLSocket, self).recvfrom(addr, buflen, flags)
def recvfrom_into (self, buffer, nbytes=None, flags=0):
if not self.act_non_blocking:
trampoline(self, read=True, timeout=self.gettimeout(), timeout_exc=timeout_exc('timed out'))
@@ -190,7 +190,7 @@ class GreenSSLSocket(__ssl.SSLSocket):
return GreenSocket(self._call_trampolining(
super(GreenSSLSocket, self).unwrap))
def do_handshake(self):
def do_handshake(self):
"""Perform a TLS/SSL handshake."""
return self._call_trampolining(
super(GreenSSLSocket, self).do_handshake)
@@ -220,7 +220,7 @@ class GreenSSLSocket(__ssl.SSLSocket):
real_connect(self, addr)
except orig_socket.error as exc:
if get_errno(exc) in CONNECT_ERR:
trampoline(self, write=True,
trampoline(self, write=True,
timeout=end-time.time(), timeout_exc=timeout_exc('timed out'))
elif get_errno(exc) in CONNECT_SUCCESS:
return
@@ -228,7 +228,7 @@ class GreenSSLSocket(__ssl.SSLSocket):
raise
if time.time() >= end:
raise timeout_exc('timed out')
def connect(self, addr):
"""Connects to remote ADDR, and then wraps the connection in
@@ -281,7 +281,7 @@ class GreenSSLSocket(__ssl.SSLSocket):
def dup(self):
raise NotImplementedError("Can't dup an ssl object")
SSLSocket = GreenSSLSocket
def wrap_socket(sock, *a, **kw):
@@ -294,8 +294,8 @@ if hasattr(__ssl, 'sslwrap_simple'):
for compability with Python 2.5 and earlier. Will disappear in
Python 3.0."""
ssl_sock = GreenSSLSocket(sock, keyfile=keyfile, certfile=certfile,
server_side=False,
cert_reqs=CERT_NONE,
ssl_version=PROTOCOL_SSLv23,
server_side=False,
cert_reqs=CERT_NONE,
ssl_version=PROTOCOL_SSLv23,
ca_certs=None)
return ssl_sock

30
eventlet/green/subprocess.py
View File

@@ -77,29 +77,19 @@ class Popen(subprocess_orig.Popen):
# don't want to rewrite the original _communicate() method, we
# just want a version that uses eventlet.green.select.select()
# instead of select.select().
_communicate = new.function(subprocess_orig.Popen._communicate.im_func.func_code,
globals())
try:
_communicate = new.function(subprocess_orig.Popen._communicate.im_func.func_code,
globals())
try:
_communicate_with_select = new.function(
subprocess_orig.Popen._communicate_with_select.im_func.func_code,
globals())
_communicate_with_poll = new.function(
subprocess_orig.Popen._communicate_with_poll.im_func.func_code,
globals())
except AttributeError:
pass
_communicate_with_select = new.function(
subprocess_orig.Popen._communicate_with_select.im_func.func_code,
globals())
_communicate_with_poll = new.function(
subprocess_orig.Popen._communicate_with_poll.im_func.func_code,
globals())
except AttributeError:
# 2.4 only has communicate
_communicate = new.function(subprocess_orig.Popen.communicate.im_func.func_code,
globals())
def communicate(self, input=None):
return self._communicate(input)
pass
# Borrow subprocess.call() and check_call(), but patch them so they reference
# OUR Popen class rather than subprocess.Popen.
call = new.function(subprocess_orig.call.func_code, globals())
try:
check_call = new.function(subprocess_orig.check_call.func_code, globals())
except AttributeError:
pass # check_call added in 2.5
check_call = new.function(subprocess_orig.check_call.func_code, globals())

5
eventlet/greenio.py
View File

@@ -486,10 +486,7 @@ class GreenPipe(_fileobject):
return n
def _get_readahead_len(self):
try:
return len(self._rbuf.getvalue()) # StringIO in 2.5
except AttributeError:
return len(self._rbuf) # str in 2.4
return len(self._rbuf.getvalue())
def _clear_readahead_buf(self):
len = self._get_readahead_len()

60
eventlet/pools.py
View File

@@ -1,38 +1,13 @@
from __future__ import print_function
import collections
from contextlib import contextmanager
from eventlet import queue
__all__ = ['Pool', 'TokenPool']
# have to stick this in an exec so it works in 2.4
try:
from contextlib import contextmanager
exec('''
@contextmanager
def item_impl(self):
""" Get an object out of the pool, for use with with statement.
>>> from eventlet import pools
>>> pool = pools.TokenPool(max_size=4)
>>> with pool.item() as obj:
... print("got token")
...
got token
>>> pool.free()
4
"""
obj = self.get()
try:
yield obj
finally:
self.put(obj)
''')
except ImportError:
item_impl = None
class Pool(object):
"""
@@ -69,15 +44,6 @@ class Pool(object):
with mypool.item() as thing:
thing.dostuff()
If stuck on 2.4, the :meth:`get` and :meth:`put` methods are the preferred
nomenclature. Use a ``finally`` to ensure that nothing is leaked::
thing = self.pool.get()
try:
thing.dostuff()
finally:
self.pool.put(thing)
The maximum size of the pool can be modified at runtime via
the :meth:`resize` method.
@@ -121,13 +87,29 @@ class Pool(object):
created = self.create()
except:
self.current_size -= 1
raise
raise
return created
self.current_size -= 1 # did not create
return self.channel.get()
if item_impl is not None:
item = item_impl
@contextmanager
def item(self):
""" Get an object out of the pool, for use with with statement.
>>> from eventlet import pools
>>> pool = pools.TokenPool(max_size=4)
>>> with pool.item() as obj:
... print("got token")
...
got token
>>> pool.free()
4
"""
obj = self.get()
try:
yield obj
finally:
self.put(obj)
def put(self, item):
"""Put an item back into the pool, when done. This may

13
eventlet/support/__init__.py
View File

@@ -22,19 +22,12 @@ def get_errno(exc):
except IndexError:
return None
if sys.version_info[0]<3 and not greenlets.preserves_excinfo:
if sys.version_info[0] < 3 and not greenlets.preserves_excinfo:
from sys import exc_clear as clear_sys_exc_info
else:
def clear_sys_exc_info():
"""No-op In py3k.
"""No-op In py3k.
Exception information is not visible outside of except statements.
sys.exc_clear became obsolete and removed."""
pass
if sys.version_info[0]==2 and sys.version_info[1]<5:
class BaseException: # pylint: disable-msg=W0622
# not subclassing from object() intentionally, because in
# that case "raise Timeout" fails with TypeError.
pass
else:
from __builtin__ import BaseException

7
eventlet/timeout.py
View File

@@ -20,7 +20,7 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.from eventlet.support import greenlets as greenlet
from eventlet.support import greenlets as greenlet, BaseException
from eventlet.support import greenlets as greenlet
from eventlet.hubs import get_hub
__all__ = ['Timeout',
@@ -84,10 +84,7 @@ class Timeout(BaseException):
self.timer = None
def __repr__(self):
try:
classname = self.__class__.__name__
except AttributeError: # Python < 2.5
classname = 'Timeout'
classname = self.__class__.__name__
if self.pending:
pending = ' pending'
else:

71
eventlet/util.py
View File

@@ -1,68 +1,31 @@
import socket
import sys
import warnings
def g_log(*args):
warnings.warn("eventlet.util.g_log is deprecated because "
"we're pretty sure no one uses it. "
"Send mail to eventletdev@lists.secondlife.com "
"if you are actually using it.",
DeprecationWarning, stacklevel=2)
import sys
from eventlet.support import greenlets as greenlet
g_id = id(greenlet.getcurrent())
if g_id is None:
if greenlet.getcurrent().parent is None:
ident = 'greenlet-main'
else:
g_id = id(greenlet.getcurrent())
if g_id < 0:
g_id += 1 + ((sys.maxint + 1) << 1)
ident = '%08X' % (g_id,)
else:
ident = 'greenlet-%d' % (g_id,)
print('[%s] %s' % (ident, ' '.join(map(str, args))), file=sys.stderr)
__original_socket__ = socket.socket
def tcp_socket():
warnings.warn("eventlet.util.tcp_socket is deprecated."
"Please use the standard socket technique for this instead:"
warnings.warn("eventlet.util.tcp_socket is deprecated. "
"Please use the standard socket technique for this instead: "
"sock = socket.socket()",
DeprecationWarning, stacklevel=2)
s = __original_socket__(socket.AF_INET, socket.SOCK_STREAM)
return s
try:
# if ssl is available, use eventlet.green.ssl for our ssl implementation
from eventlet.green import ssl
def wrap_ssl(sock, certificate=None, private_key=None, server_side=False):
return ssl.wrap_socket(sock,
keyfile=private_key, certfile=certificate,
server_side=server_side, cert_reqs=ssl.CERT_NONE,
ssl_version=ssl.PROTOCOL_SSLv23, ca_certs=None,
do_handshake_on_connect=True,
suppress_ragged_eofs=True)
except ImportError:
# if ssl is not available, use PyOpenSSL
def wrap_ssl(sock, certificate=None, private_key=None, server_side=False):
try:
from eventlet.green.OpenSSL import SSL
except ImportError:
raise ImportError("To use SSL with Eventlet, "
"you must install PyOpenSSL or use Python 2.6 or later.")
context = SSL.Context(SSL.SSLv23_METHOD)
if certificate is not None:
context.use_certificate_file(certificate)
if private_key is not None:
context.use_privatekey_file(private_key)
context.set_verify(SSL.VERIFY_NONE, lambda *x: True)
connection = SSL.Connection(context, sock)
if server_side:
connection.set_accept_state()
else:
connection.set_connect_state()
return connection
# if ssl is available, use eventlet.green.ssl for our ssl implementation
from eventlet.green import ssl
def wrap_ssl(sock, certificate=None, private_key=None, server_side=False):
warnings.warn("eventlet.util.wrap_ssl is deprecated. "
"Please use the eventlet.green.ssl.wrap_socket()",
DeprecationWarning, stacklevel=2)
return ssl.wrap_socket(
sock,
keyfile=private_key,
certfile=certificate,
server_side=server_side,
)
def wrap_socket_with_coroutine_socket(use_thread_pool=None):
warnings.warn("eventlet.util.wrap_socket_with_coroutine_socket() is now "
@@ -79,6 +42,7 @@ def wrap_pipes_with_coroutine_pipes():
from eventlet import patcher
patcher.monkey_patch(all=False, os=True)
def wrap_select_with_coroutine_select():
warnings.warn("eventlet.util.wrap_select_with_coroutine_select() is now "
"eventlet.patcher.monkey_patch(all=False, select=True)",
@@ -86,6 +50,7 @@ def wrap_select_with_coroutine_select():
from eventlet import patcher
patcher.monkey_patch(all=False, select=True)
def wrap_threading_local_with_coro_local():
"""
monkey patch ``threading.local`` with something that is greenlet aware.

2
setup.py
View File

@@ -29,8 +29,6 @@ setup(
"Operating System :: MacOS :: MacOS X",
"Operating System :: POSIX",
"Operating System :: Microsoft :: Windows",
"Programming Language :: Python :: 2.4",
"Programming Language :: Python :: 2.5",
"Programming Language :: Python :: 2.6",
"Programming Language :: Python :: 2.7",
"Topic :: Internet",

535
tests/mock.py
View File

@@ -1,271 +1,264 @@
# mock.py
# Test tools for mocking and patching.
# Copyright (C) 2007-2009 Michael Foord
# E-mail: fuzzyman AT voidspace DOT org DOT uk
# mock 0.6.0
# http://www.voidspace.org.uk/python/mock/
# Released subject to the BSD License
# Please see http://www.voidspace.org.uk/python/license.shtml
# Scripts maintained at http://www.voidspace.org.uk/python/index.shtml
# Comments, suggestions and bug reports welcome.
__all__ = (
'Mock',
'patch',
'patch_object',
'sentinel',
'DEFAULT'
)
__version__ = '0.6.0'
class SentinelObject(object):
def __init__(self, name):
self.name = name
def __repr__(self):
return '<SentinelObject "%s">' % self.name
class Sentinel(object):
def __init__(self):
self._sentinels = {}
def __getattr__(self, name):
return self._sentinels.setdefault(name, SentinelObject(name))
sentinel = Sentinel()
DEFAULT = sentinel.DEFAULT
class OldStyleClass:
pass
ClassType = type(OldStyleClass)
def _is_magic(name):
return '__%s__' % name[2:-2] == name
def _copy(value):
if type(value) in (dict, list, tuple, set):
return type(value)(value)
return value
class Mock(object):
def __init__(self, spec=None, side_effect=None, return_value=DEFAULT,
name=None, parent=None, wraps=None):
self._parent = parent
self._name = name
if spec is not None and not isinstance(spec, list):
spec = [member for member in dir(spec) if not _is_magic(member)]
self._methods = spec
self._children = {}
self._return_value = return_value
self.side_effect = side_effect
self._wraps = wraps
self.reset_mock()
def reset_mock(self):
self.called = False
self.call_args = None
self.call_count = 0
self.call_args_list = []
self.method_calls = []
for child in self._children.itervalues():
child.reset_mock()
if isinstance(self._return_value, Mock):
self._return_value.reset_mock()
def __get_return_value(self):
if self._return_value is DEFAULT:
self._return_value = Mock()
return self._return_value
def __set_return_value(self, value):
self._return_value = value
return_value = property(__get_return_value, __set_return_value)
def __call__(self, *args, **kwargs):
self.called = True
self.call_count += 1
self.call_args = (args, kwargs)
self.call_args_list.append((args, kwargs))
parent = self._parent
name = self._name
while parent is not None:
parent.method_calls.append((name, args, kwargs))
if parent._parent is None:
break
name = parent._name + '.' + name
parent = parent._parent
ret_val = DEFAULT
if self.side_effect is not None:
if (isinstance(self.side_effect, Exception) or
isinstance(self.side_effect, (type, ClassType)) and
issubclass(self.side_effect, Exception)):
raise self.side_effect
ret_val = self.side_effect(*args, **kwargs)
if ret_val is DEFAULT:
ret_val = self.return_value
if self._wraps is not None and self._return_value is DEFAULT:
return self._wraps(*args, **kwargs)
if ret_val is DEFAULT:
ret_val = self.return_value
return ret_val
def __getattr__(self, name):
if self._methods is not None:
if name not in self._methods:
raise AttributeError("Mock object has no attribute '%s'" % name)
elif _is_magic(name):
raise AttributeError(name)
if name not in self._children:
wraps = None
if self._wraps is not None:
wraps = getattr(self._wraps, name)
self._children[name] = Mock(parent=self, name=name, wraps=wraps)
return self._children[name]
def assert_called_with(self, *args, **kwargs):
assert self.call_args == (args, kwargs), 'Expected: %s\nCalled with: %s' % ((args, kwargs), self.call_args)
def _dot_lookup(thing, comp, import_path):
try:
return getattr(thing, comp)
except AttributeError:
__import__(import_path)
return getattr(thing, comp)
def _importer(target):
components = target.split('.')
import_path = components.pop(0)
thing = __import__(import_path)
for comp in components:
import_path += ".%s" % comp
thing = _dot_lookup(thing, comp, import_path)
return thing
class _patch(object):
def __init__(self, target, attribute, new, spec, create):
self.target = target
self.attribute = attribute
self.new = new
self.spec = spec
self.create = create
self.has_local = False
def __call__(self, func):
if hasattr(func, 'patchings'):
func.patchings.append(self)
return func
def patched(*args, **keywargs):
# don't use a with here (backwards compatability with 2.5)
extra_args = []
for patching in patched.patchings:
arg = patching.__enter__()
if patching.new is DEFAULT:
extra_args.append(arg)
args += tuple(extra_args)
try:
return func(*args, **keywargs)
finally:
for patching in getattr(patched, 'patchings', []):
patching.__exit__()
patched.patchings = [self]
patched.__name__ = func.__name__
patched.compat_co_firstlineno = getattr(func, "compat_co_firstlineno",
func.func_code.co_firstlineno)
return patched
def get_original(self):
target = self.target
name = self.attribute
create = self.create
original = DEFAULT
if _has_local_attr(target, name):
try:
original = target.__dict__[name]
except AttributeError:
# for instances of classes with slots, they have no __dict__
original = getattr(target, name)
elif not create and not hasattr(target, name):
raise AttributeError("%s does not have the attribute %r" % (target, name))
return original
def __enter__(self):
new, spec, = self.new, self.spec
original = self.get_original()
if new is DEFAULT:
# XXXX what if original is DEFAULT - shouldn't use it as a spec
inherit = False
if spec == True:
# set spec to the object we are replacing
spec = original
if isinstance(spec, (type, ClassType)):
inherit = True
new = Mock(spec=spec)
if inherit:
new.return_value = Mock(spec=spec)
self.temp_original = original
setattr(self.target, self.attribute, new)
return new
def __exit__(self, *_):
if self.temp_original is not DEFAULT:
setattr(self.target, self.attribute, self.temp_original)
else:
delattr(self.target, self.attribute)
del self.temp_original
def patch_object(target, attribute, new=DEFAULT, spec=None, create=False):
return _patch(target, attribute, new, spec, create)
def patch(target, new=DEFAULT, spec=None, create=False):
try:
target, attribute = target.rsplit('.', 1)
except (TypeError, ValueError):
raise TypeError("Need a valid target to patch. You supplied: %r" % (target,))
target = _importer(target)
return _patch(target, attribute, new, spec, create)
def _has_local_attr(obj, name):
try:
return name in vars(obj)
except TypeError:
# objects without a __dict__
return hasattr(obj, name)
# mock.py
# Test tools for mocking and patching.
# Copyright (C) 2007-2009 Michael Foord
# E-mail: fuzzyman AT voidspace DOT org DOT uk
# mock 0.6.0
# http://www.voidspace.org.uk/python/mock/
# Released subject to the BSD License
# Please see http://www.voidspace.org.uk/python/license.shtml
# Scripts maintained at http://www.voidspace.org.uk/python/index.shtml
# Comments, suggestions and bug reports welcome.
__all__ = (
'Mock',
'patch',
'patch_object',
'sentinel',
'DEFAULT'
)
__version__ = '0.6.0'
class SentinelObject(object):
def __init__(self, name):
self.name = name
def __repr__(self):
return '<SentinelObject "%s">' % self.name
class Sentinel(object):
def __init__(self):
self._sentinels = {}
def __getattr__(self, name):
return self._sentinels.setdefault(name, SentinelObject(name))
sentinel = Sentinel()
DEFAULT = sentinel.DEFAULT
class OldStyleClass:
pass
ClassType = type(OldStyleClass)
def _is_magic(name):
return '__%s__' % name[2:-2] == name
def _copy(value):
if type(value) in (dict, list, tuple, set):
return type(value)(value)
return value
class Mock(object):
def __init__(self, spec=None, side_effect=None, return_value=DEFAULT,
name=None, parent=None, wraps=None):
self._parent = parent
self._name = name
if spec is not None and not isinstance(spec, list):
spec = [member for member in dir(spec) if not _is_magic(member)]
self._methods = spec
self._children = {}
self._return_value = return_value
self.side_effect = side_effect
self._wraps = wraps
self.reset_mock()
def reset_mock(self):
self.called = False
self.call_args = None
self.call_count = 0
self.call_args_list = []
self.method_calls = []
for child in self._children.itervalues():
child.reset_mock()
if isinstance(self._return_value, Mock):
self._return_value.reset_mock()
def __get_return_value(self):
if self._return_value is DEFAULT:
self._return_value = Mock()
return self._return_value
def __set_return_value(self, value):
self._return_value = value
return_value = property(__get_return_value, __set_return_value)
def __call__(self, *args, **kwargs):
self.called = True
self.call_count += 1
self.call_args = (args, kwargs)
self.call_args_list.append((args, kwargs))
parent = self._parent
name = self._name
while parent is not None:
parent.method_calls.append((name, args, kwargs))
if parent._parent is None:
break
name = parent._name + '.' + name
parent = parent._parent
ret_val = DEFAULT
if self.side_effect is not None:
if (isinstance(self.side_effect, Exception) or
isinstance(self.side_effect, (type, ClassType)) and
issubclass(self.side_effect, Exception)):
raise self.side_effect
ret_val = self.side_effect(*args, **kwargs)
if ret_val is DEFAULT:
ret_val = self.return_value
if self._wraps is not None and self._return_value is DEFAULT:
return self._wraps(*args, **kwargs)
if ret_val is DEFAULT:
ret_val = self.return_value
return ret_val
def __getattr__(self, name):
if self._methods is not None:
if name not in self._methods:
raise AttributeError("Mock object has no attribute '%s'" % name)
elif _is_magic(name):
raise AttributeError(name)
if name not in self._children:
wraps = None
if self._wraps is not None:
wraps = getattr(self._wraps, name)
self._children[name] = Mock(parent=self, name=name, wraps=wraps)
return self._children[name]
def assert_called_with(self, *args, **kwargs):
assert self.call_args == (args, kwargs), 'Expected: %s\nCalled with: %s' % ((args, kwargs), self.call_args)
def _dot_lookup(thing, comp, import_path):
try:
return getattr(thing, comp)
except AttributeError:
__import__(import_path)
return getattr(thing, comp)
def _importer(target):
components = target.split('.')
import_path = components.pop(0)
thing = __import__(import_path)
for comp in components:
import_path += ".%s" % comp
thing = _dot_lookup(thing, comp, import_path)
return thing
class _patch(object):
def __init__(self, target, attribute, new, spec, create):
self.target = target
self.attribute = attribute
self.new = new
self.spec = spec
self.create = create
self.has_local = False
def __call__(self, func):
if hasattr(func, 'patchings'):
func.patchings.append(self)
return func
def patched(*args, **keywargs):
# don't use a with here (backwards compatability with 2.5)
extra_args = []
for patching in patched.patchings:
arg = patching.__enter__()
if patching.new is DEFAULT:
extra_args.append(arg)
args += tuple(extra_args)
try:
return func(*args, **keywargs)
finally:
for patching in getattr(patched, 'patchings', []):
patching.__exit__()
patched.patchings = [self]
patched.__name__ = func.__name__
patched.compat_co_firstlineno = getattr(func, "compat_co_firstlineno",
func.func_code.co_firstlineno)
return patched
def get_original(self):
target = self.target
name = self.attribute
create = self.create
original = DEFAULT
if _has_local_attr(target, name):
try:
original = target.__dict__[name]
except AttributeError:
# for instances of classes with slots, they have no __dict__
original = getattr(target, name)
elif not create and not hasattr(target, name):
raise AttributeError("%s does not have the attribute %r" % (target, name))
return original
def __enter__(self):
new, spec, = self.new, self.spec
original = self.get_original()
if new is DEFAULT:
# XXXX what if original is DEFAULT - shouldn't use it as a spec
inherit = False
if spec == True:
# set spec to the object we are replacing
spec = original
if isinstance(spec, (type, ClassType)):
inherit = True
new = Mock(spec=spec)
if inherit:
new.return_value = Mock(spec=spec)
self.temp_original = original
setattr(self.target, self.attribute, new)
return new
def __exit__(self, *_):
if self.temp_original is not DEFAULT:
setattr(self.target, self.attribute, self.temp_original)
else:
delattr(self.target, self.attribute)
del self.temp_original
def patch_object(target, attribute, new=DEFAULT, spec=None, create=False):
return _patch(target, attribute, new, spec, create)
def patch(target, new=DEFAULT, spec=None, create=False):
try:
target, attribute = target.rsplit('.', 1)
except (TypeError, ValueError):
raise TypeError("Need a valid target to patch. You supplied: %r" % (target,))
target = _importer(target)
return _patch(target, attribute, new, spec, create)
def _has_local_attr(obj, name):
try:
return name in vars(obj)
except TypeError:
# objects without a __dict__
return hasattr(obj, name)

15
tests/nosewrapper.py
View File

@@ -1,25 +1,20 @@
""" This script simply gets the paths correct for testing eventlet with the
""" This script simply gets the paths correct for testing eventlet with the
hub extension for Nose."""
import nose
from os.path import dirname, realpath, abspath
import sys
parent_dir = dirname(dirname(realpath(abspath(__file__))))
if parent_dir not in sys.path:
sys.path.insert(0, parent_dir)
# hacky hacks: skip test__api_timeout when under 2.4 because otherwise it SyntaxErrors
if sys.version_info < (2,5):
argv = sys.argv + ["--exclude=.*_with_statement.*"]
else:
argv = sys.argv
# hudson does a better job printing the test results if the exit value is 0
zero_status = '--force-zero-status'
if zero_status in argv:
argv.remove(zero_status)
if zero_status in sys.argv:
sys.argv.remove(zero_status)
launch = nose.run
else:
launch = nose.main
launch(argv=argv)
launch(argv=sys.argv)

8
tests/stdlib/test_select.py
View File

@@ -1,12 +1,10 @@
from eventlet import patcher
from eventlet.green import select
patcher.inject('test.test_select',
globals(),
('select', select))
if __name__ == "__main__":
try:
test_main()
except NameError:
pass # 2.5
test_main()

8
tests/stdlib/test_thread.py
View File

@@ -2,9 +2,6 @@ from eventlet import patcher
from eventlet.green import thread
from eventlet.green import time
# necessary to initialize the hub before running on 2.5
from eventlet import hubs
hubs.get_hub()
patcher.inject('test.test_thread', globals())
@@ -15,7 +12,4 @@ except NameError:
pass
if __name__ == "__main__":
try:
test_main()
except NameError:
pass # 2.5
test_main()

5
tests/stdlib/test_urllib2.py
View File

@@ -9,10 +9,7 @@ patcher.inject('test.test_urllib2',
HandlerTests.test_file = patcher.patch_function(HandlerTests.test_file, ('socket', socket))
HandlerTests.test_cookie_redirect = patcher.patch_function(HandlerTests.test_cookie_redirect, ('urllib2', urllib2))
try:
OpenerDirectorTests.test_badly_named_methods = patcher.patch_function(OpenerDirectorTests.test_badly_named_methods, ('urllib2', urllib2))
except AttributeError:
pass # 2.4 doesn't have this test method
OpenerDirectorTests.test_badly_named_methods = patcher.patch_function(OpenerDirectorTests.test_badly_named_methods, ('urllib2', urllib2))
if __name__ == "__main__":
test_main()

14
tests/timeout_test_with_statement.py
View File

@@ -1,19 +1,22 @@
""" Tests with-statement behavior of Timeout class. Don't import when
using Python 2.4. """
"""Tests with-statement behavior of Timeout class."""
from __future__ import with_statement
import sys
import time
import unittest
import weakref
import time
from eventlet import sleep
from eventlet.timeout import Timeout
from tests import LimitedTestCase
DELAY = 0.01
class Error(Exception):
pass
class Test(LimitedTestCase):
def test_cancellation(self):
# Nothing happens if with-block finishes before the timeout expires
@@ -46,7 +49,7 @@ class Test(LimitedTestCase):
assert ex is t, (ex, t)
else:
raise AssertionError('must raise Timeout')
def test_raising_custom_exception(self):
# You can customize the exception raised:
try:
@@ -83,7 +86,6 @@ class Test(LimitedTestCase):
timer.cancel()
sleep(DELAY*2)
def test_silent_block(self):
# To silence the exception before exiting the block, pass
# False as second parameter.

16
tox.ini
View File

@@ -11,7 +11,7 @@ ignore = E261
max-line-length = 101
[tox]
envlist = py25selects,py25poll,py26selects,py26poll,py26epolls,py27selects,py27poll,py27epolls
envlist = py26selects,py26poll,py26epolls,py27selects,py27poll,py27epolls
[testenv]
downloadcache = {toxworkdir}/pip_download_cache
@@ -26,20 +26,6 @@ commands =
eventlet/pool.py eventlet/pools.py eventlet/proc.py \
eventlet/queue.py eventlet/timeout.py
[testenv:py25selects]
basepython = python2.5
setenv = EVENTLET_HUB = selects
deps =
{[testenv]deps}
pyzmq<2.2
[testenv:py25poll]
basepython = python2.5
setenv = EVENTLET_HUB = poll
deps =
{[testenv]deps}
pyzmq<2.2
[testenv:py26selects]
basepython = python2.6
setenv = EVENTLET_HUB = selects