[svn r3] Moved eventlet into trunk.

This commit is contained in:
which.linden
2007-08-23 19:57:41 -04:00
parent 6e4eb16e81
commit 03f38ec9f0
36 changed files with 5376 additions and 0 deletions

38
README Normal file
View File

@@ -0,0 +1,38 @@
= eventlet =
Eventlet is a networking library written in Python. It achieves high
scalability by using non-blocking io while at the same time retaining
high programmer usability by using coroutines to make the non-blocking
io operations appear blocking at the source code level.
== requirements ===
Eventlet runs on Python version 2.3 or greater, with the following dependenceis:
* [http://cheeseshop.python.org/pypi/greenlet
* (if running python versions < 2.4) collections.py from the 2.4 distribution or later
== limitations ==
* Sorely lacking in documentation
* Not enough test coverage -- the goal is 100%, but we are not there yet.
* Eventlet does not currently run on stackless using tasklets, though it is a goal to do so in the future.
* The SSL client does not properly connect to the SSL server, though both client and server interoperate with other SSL implementations (e.g. curl and apache).
== getting started ==
% python
>>> import eventlet.api
>>> help(eventlet.api)
Also, look at the examples in the examples directory.
== eventlet history ==
eventlet began life as Donovan Preston was talking to Bob Ippolito about coroutine-based non-blocking networking frameworks in Python. Most non-blocking frameworks require you to run the "main loop" in order to perform all network operations, but Donovan wondered if a library written using a trampolining style could get away with transparently running the main loop any time i/o was required, stopping the main loop once no more i/o was scheduled. Bob spent a few days during PyCon 2005 writing a proof-of-concept. He named it eventlet, after the coroutine implementation it used, [[greenlet]]. Donovan began using eventlet as a light-weight network library for his spare-time project Pavel, and also began writing some unittests.
* http://svn.red-bean.com/bob/eventlet/trunk/
* http://soundfarmer.com/Pavel/trunk/
When Donovan started at Linden Lab in May of 2006, he added eventlet as an svn external in the indra/lib/python directory, to be a dependency of the yet-to-be-named [[backbone]] project (at the time, it was named restserv). However, including eventlet as an svn external meant that any time the externally hosted project had hosting issues, Linden developers were not able to perform svn updates. Thus, the eventlet source was imported into the linden source tree at the same location, and became a fork.
Bob Ippolito has ceased working on eventlet and has stated his desire for Linden to take it's fork forward to the open source world as "the" eventlet.

24
eventlet/__init__.py Normal file
View File

@@ -0,0 +1,24 @@
"""\
@file __init__.py
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
# This text exists only for the purpose of not making a complete
# mockery of the above copyright header.

305
eventlet/api.py Normal file
View File

@@ -0,0 +1,305 @@
"""\
@file api.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
import socket
import string
import linecache
import inspect
import traceback
try:
import greenlet
except ImportError:
try:
import pylibsupport
pylibsupport.emulate()
greenlet = sys.modules['greenlet']
except ImportError:
import stacklesssupport
stacklesssupport.emulate()
greenlet = sys.modules['greenlet']
from eventlet import greenlib, tls
__all__ = [
'use_hub', 'get_hub', 'sleep', 'spawn', 'kill',
'call_after', 'exc_after', 'trampoline', 'tcp_listener', 'tcp_server',
]
class TimeoutError(Exception):
pass
_threadlocal = tls.local()
def tcp_listener(address):
"""
Listen on the given (ip, port) address with a TCP socket.
Returns a socket object which one should call accept() on to
accept a connection on the newly bound socket.
Generally, the returned socket will be passed to tcp_server,
which accepts connections forever and spawns greenlets for
each incoming connection.
"""
from eventlet import wrappedfd, util
socket = wrappedfd.wrapped_fd(util.tcp_socket())
util.socket_bind_and_listen(socket, address)
return socket
def ssl_listener(address, certificate, private_key):
"""Listen on the given (ip, port) address with a TCP socket that
can do SSL.
Returns a socket object which one should call accept() on to
accept a connection on the newly bound socket.
Generally, the returned socket will be passed to tcp_server,
which accepts connections forever and spawns greenlets for
each incoming connection.
"""
from eventlet import util
socket = util.wrap_ssl(util.tcp_socket(), certificate, private_key)
util.socket_bind_and_listen(socket, address)
socket.is_secure = True
return socket
def connect_tcp(address):
"""
Create a TCP connection to address (host, port) and return the socket.
"""
from eventlet import wrappedfd, util
desc = wrappedfd.wrapped_fd(util.tcp_socket())
desc.connect(address)
return desc
def tcp_server(listensocket, server, *args, **kw):
"""
Given a socket, accept connections forever, spawning greenlets
and executing "server" for each new incoming connection.
When listensocket is closed, the tcp_server greenlet will end.
listensocket:
The socket to accept connections from.
server:
The callable to call when a new connection is made.
*args:
The arguments to pass to the call to server.
**kw:
The keyword arguments to pass to the call to server.
"""
try:
try:
while True:
spawn(server, listensocket.accept(), *args, **kw)
except socket.error, e:
# Broken pipe means it was shutdown
if e[0] != 32:
raise
finally:
listensocket.close()
def trampoline(fd, read=None, write=None, timeout=None):
t = None
hub = get_hub()
self = greenlet.getcurrent()
fileno = getattr(fd, 'fileno', lambda: fd)()
def _do_close(fn):
hub.remove_descriptor(fn)
greenlib.switch(self, exc=socket.error(32, 'Broken pipe'))
def _do_timeout(fn):
hub.remove_descriptor(fn)
greenlib.switch(self, exc=TimeoutError())
def cb(_fileno):
if t is not None:
t.cancel()
hub.remove_descriptor(fileno)
greenlib.switch(self, fd)
if timeout is not None:
t = hub.schedule_call(timeout, _do_timeout)
hub.add_descriptor(fileno, read and cb, write and cb, _do_close)
return hub.switch()
def _spawn_startup(cb, args, kw, cancel=None):
try:
greenlib.switch(greenlet.getcurrent().parent)
cancel = None
finally:
if cancel is not None:
cancel()
return cb(*args, **kw)
def _spawn(g):
g.parent = greenlet.getcurrent()
greenlib.switch(g)
def spawn(cb, *args, **kw):
# killable
t = None
g = greenlib.tracked_greenlet()
t = get_hub().schedule_call(0, _spawn, g)
greenlib.switch(g, (_spawn_startup, cb, args, kw, t.cancel))
return g
kill = greenlib.kill
def call_after(seconds, cb, *args, **kw):
# cancellable
def startup():
g = greenlib.tracked_greenlet()
greenlib.switch(g, (_spawn_startup, cb, args, kw))
greenlib.switch(g)
return get_hub().schedule_call(seconds, startup)
def exc_after(seconds, exc):
return call_after(seconds, switch, getcurrent(), None, exc)
def get_default_hub():
try:
import eventlet.kqueuehub
except ImportError:
pass
else:
return eventlet.kqueuehub
import select
if hasattr(select, 'poll'):
import eventlet.pollhub
return eventlet.pollhub
else:
import eventlet.selecthub
return eventlet.selecthub
def use_hub(mod=None):
if mod is None:
mod = get_default_hub()
if hasattr(_threadlocal, 'hub'):
del _threadlocal.hub
if hasattr(mod, 'Hub'):
_threadlocal.Hub = mod.Hub
else:
_threadlocal.Hub = mod
def get_hub():
try:
hub = _threadlocal.hub
except AttributeError:
try:
_threadlocal.Hub
except AttributeError:
use_hub()
hub = _threadlocal.hub = _threadlocal.Hub()
return hub
def sleep(timeout=0):
hub = get_hub()
hub.schedule_call(timeout, greenlib.switch, greenlet.getcurrent())
hub.switch()
switch = greenlib.switch
getcurrent = greenlet.getcurrent
class Spew(object):
def __init__(self, trace_names=None):
self.trace_names = trace_names
def __call__(self, frame, event, arg):
if event == 'line':
lineno = frame.f_lineno
if '__file__' in frame.f_globals:
filename = frame.f_globals['__file__']
if (filename.endswith('.pyc') or
filename.endswith('.pyo')):
filename = filename[:-1]
name = frame.f_globals['__name__']
line = linecache.getline(filename, lineno)
else:
name = '[unknown]'
try:
src = inspect.getsourcelines(frame)
line = src[lineno]
except IOError:
line = 'Unknown code named [%s]. VM instruction #%d' % (
frame.f_code.co_name, frame.f_lasti)
if self.trace_names is None or name in self.trace_names:
print '%s:%s: %s' % (name, lineno, line.rstrip())
details = '\t'
tokens = line.translate(
string.maketrans(' ,.()', '\0' * 5)).split('\0')
for tok in tokens:
if tok in frame.f_globals:
details += '%s=%r ' % (tok, frame.f_globals[tok])
if tok in frame.f_locals:
details += '%s=%r ' % (tok, frame.f_locals[tok])
if details.strip():
print details
return self
def spew(trace_names=None):
sys.settrace(Spew(trace_names))
def unspew():
sys.settrace(None)
def named(name):
"""Return an object given its name. The name uses a module-like
syntax, eg:
os.path.join
or
mulib.mu.Resource
"""
toimport = name
obj = None
while toimport:
try:
obj = __import__(toimport)
break
except ImportError, err:
# print 'Import error on %s: %s' % (toimport, err) # debugging spam
toimport = '.'.join(toimport.split('.')[:-1])
if obj is None:
raise ImportError('%s could not be imported' % (name, ))
for seg in name.split('.')[1:]:
try:
obj = getattr(obj, seg)
except AttributeError:
dirobj = dir(obj)
dirobj.sort()
raise AttributeError('attribute %r missing from %r (%r) %r' % (
seg, obj, dirobj, name))
return obj

164
eventlet/api_test.py Normal file
View File

@@ -0,0 +1,164 @@
"""\
@file api_test.py
@author Donovan Preston
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
from eventlet import tests
from eventlet import api, wrappedfd, util
import socket
def check_hub():
# Clear through the descriptor queue
api.sleep(0)
api.sleep(0)
assert not api.get_hub().descriptors, repr(api.get_hub().descriptors)
# Stop the runloop
api.get_hub().runloop.abort()
api.sleep(0)
assert not api.get_hub().runloop.running
class TestApi(tests.TestCase):
mode = 'static'
def test_tcp_listener(self):
socket = api.tcp_listener(('0.0.0.0', 0))
assert socket.getsockname()[0] == '0.0.0.0'
socket.close()
check_hub()
def dont_test_connect_tcp(self):
"""This test is broken. Please name it test_connect_tcp and fix
the bug (or the test) so it passes.
"""
def accept_once(listenfd):
try:
conn, addr = listenfd.accept()
conn.write('hello\n')
conn.close()
finally:
listenfd.close()
server = api.tcp_listener(('0.0.0.0', 0))
api.spawn(accept_once, server)
client = api.connect_tcp(('127.0.0.1', server.getsockname()[1]))
assert client.readline() == 'hello\n'
assert client.read() == ''
client.close()
check_hub()
def test_server(self):
server = api.tcp_listener(('0.0.0.0', 0))
bound_port = server.getsockname()[1]
connected = []
def accept_twice((conn, addr)):
print 'connected'
connected.append(True)
conn.close()
if len(connected) == 2:
server.close()
api.call_after(0, api.connect_tcp, ('127.0.0.1', bound_port))
api.call_after(0, api.connect_tcp, ('127.0.0.1', bound_port))
api.tcp_server(server, accept_twice)
assert len(connected) == 2
check_hub()
def dont_test_trampoline_timeout(self):
"""This test is broken. Please change it's name to test_trampoline_timeout,
and fix the bug (or fix the test)
"""
server = api.tcp_listener(('0.0.0.0', 0))
bound_port = server.getsockname()[1]
try:
desc = wrappedfd.wrapped_fd(util.tcp_socket())
api.trampoline(desc, read=True, write=True, timeout=0.1)
except api.TimeoutError:
pass # test passed
else:
assert False, "Didn't timeout"
check_hub()
def test_timeout_cancel(self):
server = api.tcp_listener(('0.0.0.0', 0))
bound_port = server.getsockname()[1]
def client_connected((conn, addr)):
conn.close()
def go():
client = util.tcp_socket()
desc = wrappedfd.wrapped_fd(client)
desc.connect(('127.0.0.1', bound_port))
try:
api.trampoline(desc, read=True, write=True, timeout=0.1)
except api.TimeoutError:
assert False, "Timed out"
server.close()
client.close()
api.call_after(0, go)
api.tcp_server(server, client_connected)
check_hub()
def dont_test_explicit_hub(self):
"""This test is broken. please change it's name to test_explicit_hub
and make it pass (or fix the test)
"""
api.use_hub(Foo)
assert isinstance(api.get_hub(), Foo), api.get_hub()
api.use_hub(api.get_default_hub())
check_hub()
def test_named(self):
named_foo = api.named('api_test.Foo')
self.assertEquals(
named_foo.__name__,
"Foo")
def test_naming_missing_class(self):
self.assertRaises(
ImportError, api.named, 'this_name_should_hopefully_not_exist.Foo')
class Foo(object):
pass
if __name__ == '__main__':
tests.main()

85
eventlet/backdoor.py Normal file
View File

@@ -0,0 +1,85 @@
"""\
@file backdoor.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
from code import InteractiveConsole
from eventlet import greenlib
try:
sys.ps1
except AttributeError:
sys.ps1 = '>>> '
try:
sys.ps2
except AttributeError:
sys.ps2 = '... '
class SocketConsole(greenlib.GreenletContext):
def __init__(self, desc):
# mangle the socket
self.desc = desc
readline = desc.readline
self.old = {}
self.fixups = {
'softspace': 0,
'isatty': lambda: True,
'flush': lambda: None,
'readline': lambda *a: readline(*a).replace('\r\n', '\n'),
}
for key, value in self.fixups.iteritems():
if hasattr(desc, key):
self.old[key] = getattr(desc, key)
setattr(desc, key, value)
def finalize(self):
# restore the state of the socket
for key in self.fixups:
try:
value = self.old[key]
except KeyError:
delattr(self.desc, key)
else:
setattr(self.desc, key, value)
self.fixups.clear()
self.old.clear()
self.desc = None
def swap_in(self):
self.saved = sys.stdin, sys.stderr, sys.stdout
sys.stdin = sys.stdout = sys.stderr = self.desc
def swap_out(self):
sys.stdin, sys.stderr, sys.stdout = self.saved
def backdoor((conn, addr), locals=None):
host, port = addr
print "backdoor to %s:%s" % (host, port)
ctx = SocketConsole(conn)
ctx.register()
try:
console = InteractiveConsole(locals)
console.interact()
finally:
ctx.unregister()

98
eventlet/channel.py Normal file
View File

@@ -0,0 +1,98 @@
"""\
@file channel.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import collections
from eventlet import api, greenlib
import greenlet
__all__ = ['channel']
class channel(object):
"""A channel is a control flow primitive for co-routines. It is a
"thread-like" queue for controlling flow between two (or more) co-routines.
The state model is:
* If one co-routine calls send(), it is unscheduled until another
co-routine calls receive().
* If one co-rounte calls receive(), it is unscheduled until another
co-routine calls send().
* Once a paired send()/receive() have been called, both co-routeines
are rescheduled.
This is similar to: http://stackless.com/wiki/Channels
"""
balance = 0
def _tasklet_loop(self):
deque = self.deque = collections.deque()
hub = api.get_hub()
switch = greenlib.switch
direction, caller, args = switch()
try:
while True:
if direction == -1:
# waiting to receive
if self.balance > 0:
sender, args = deque.popleft()
hub.schedule_call(0, switch, sender)
hub.schedule_call(0, switch, caller, *args)
else:
deque.append(caller)
else:
# waiting to send
if self.balance < 0:
receiver = deque.popleft()
hub.schedule_call(0, switch, receiver, *args)
hub.schedule_call(0, switch, caller)
else:
deque.append((caller, args))
self.balance += direction
direction, caller, args = hub.switch()
finally:
deque.clear()
del self.deque
self.balance = 0
def _send_tasklet(self, *args):
try:
t = self._tasklet
except AttributeError:
t = self._tasklet = greenlib.tracked_greenlet()
greenlib.switch(t, (self._tasklet_loop,))
if args:
return greenlib.switch(t, (1, greenlet.getcurrent(), args))
else:
return greenlib.switch(t, (-1, greenlet.getcurrent(), args))
def receive(self):
return self._send_tasklet()
def send(self, value):
return self._send_tasklet(value)
def send_exception(self, exc):
return self._send_tasklet(None, exc)

167
eventlet/coros.py Normal file
View File

@@ -0,0 +1,167 @@
"""\
@file coros.py
@author Donovan Preston
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import time
import traceback
import greenlet
from eventlet import api
from eventlet import channel
from eventlet import pools
from eventlet import greenlib
class Cancelled(RuntimeError):
pass
NOT_USED = object()
class event(object):
"""An abstraction where an arbitrary number of coroutines
can wait for one event from another.
"""
_result = None
def __init__(self):
self.reset()
def reset(self):
""" Reset this event so it can be used to send again.
Can only be called after send has been called."""
assert self._result is not NOT_USED
self.epoch = time.time()
self._result = NOT_USED
self._waiters = {}
def wait(self):
"""wait until another coroutine calls send.
Returns the value the other coroutine passed to
send. Returns immediately if the event has already
occured.
"""
if self._result is NOT_USED:
self._waiters[greenlet.getcurrent()] = True
return api.get_hub().switch()
if self._exc is not None:
raise self._exc
return self._result
def cancel(self, waiter):
"""Raise an exception into a coroutine which called
wait() an this event instead of returning a value
from wait. Sends the eventlet.coros.Cancelled
exception
waiter: The greenlet (greenlet.getcurrent()) of the
coroutine to cancel
"""
if waiter in self._waiters:
del self._waiters[waiter]
api.get_hub().schedule_call(
0, greenlib.switch, waiter, None, Cancelled())
def send(self, result=None, exc=None):
"""Resume all previous and further
calls to wait() with result.
"""
assert self._result is NOT_USED
self._result = result
self._exc = exc
hub = api.get_hub()
for waiter in self._waiters:
hub.schedule_call(0, greenlib.switch, waiter, self._result)
class CoroutinePool(pools.Pool):
""" Like a thread pool, but with coroutines. """
def _main_loop(self, sender):
while True:
recvd = sender.wait()
sender.reset()
(evt, func, args, kw) = recvd
try:
result = func(*args, **kw)
if evt is not None:
evt.send(result)
except greenlet.GreenletExit:
pass
except Exception, e:
traceback.print_exc()
if evt is not None:
evt.send(exc=e)
self.put(sender)
def create(self):
"""Private implementation of eventlet.pools.Pool
interface. Creates an event and spawns the
_main_loop coroutine, passing the event.
The event is used to send a callable into the
new coroutine, to be executed.
"""
sender = event()
api.spawn(self._main_loop, sender)
return sender
def execute(self, func, *args, **kw):
"""Execute func in one of the coroutines maintained
by the pool, when one is free.
Immediately returns an eventlet.coros.event object which
func's result will be sent to when it is available.
"""
sender = self.get()
receiver = event()
sender.send((receiver, func, args, kw))
return receiver
def execute_async(self, func, *args, **kw):
"""Execute func in one of the coroutines maintained
by the pool, when one is free.
This version does not provide the return value.
"""
sender = self.get()
sender.send((None, func, args, kw))
class pipe(object):
""" Implementation of pipe using events. Not tested! Not used, either."""
def __init__(self):
self._event = event()
self._buffer = ''
def send(self, txt):
self._buffer += txt
evt, self._event = self._event, event()
evt.send()
def recv(self, num=16384):
if not self._buffer:
self._event.wait()
if num >= len(self._buffer):
buf, self._buffer = self._buffer, ''
else:
buf, self._buffer = self._buffer[:num], self._buffer[num:]
return buf

144
eventlet/coros_test.py Normal file
View File

@@ -0,0 +1,144 @@
"""\
@file coros_test.py
@author Donovan Preston, Ryan Williams
Copyright (c) 2000-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
from eventlet import tests
from eventlet import coros, api
class TestEvent(tests.TestCase):
mode = 'static'
def setUp(self):
# raise an exception if we're waiting forever
self._cancel_timeout = api.exc_after(1, RuntimeError())
def tearDown(self):
self._cancel_timeout.cancel()
def test_waiting_for_event(self):
evt = coros.event()
value = 'some stuff'
def send_to_event():
evt.send(value)
api.spawn(send_to_event)
self.assertEqual(evt.wait(), value)
def test_multiple_waiters(self):
evt = coros.event()
value = 'some stuff'
results = []
def wait_on_event(i_am_done):
evt.wait()
results.append(True)
i_am_done.send()
waiters = []
count = 5
for i in range(count):
waiters.append(coros.event())
api.spawn(wait_on_event, waiters[-1])
evt.send()
for w in waiters:
w.wait()
self.assertEqual(len(results), count)
def test_cancel(self):
evt = coros.event()
# close over the current coro so we can cancel it explicitly
current = api.getcurrent()
def cancel_event():
evt.cancel(current)
api.spawn(cancel_event)
self.assertRaises(coros.Cancelled, evt.wait)
def test_reset(self):
evt = coros.event()
# calling reset before send should throw
self.assertRaises(AssertionError, evt.reset)
value = 'some stuff'
def send_to_event():
evt.send(value)
api.spawn(send_to_event)
self.assertEqual(evt.wait(), value)
# now try it again, and we should get the same exact value,
# and we shouldn't be allowed to resend without resetting
value2 = 'second stuff'
self.assertRaises(AssertionError, evt.send, value2)
self.assertEqual(evt.wait(), value)
# reset and everything should be happy
evt.reset()
def send_to_event2():
evt.send(value2)
api.spawn(send_to_event2)
self.assertEqual(evt.wait(), value2)
class TestCoroutinePool(tests.TestCase):
mode = 'static'
def setUp(self):
# raise an exception if we're waiting forever
self._cancel_timeout = api.exc_after(1, RuntimeError())
def tearDown(self):
self._cancel_timeout.cancel()
def test_execute_async(self):
done = coros.event()
def some_work():
done.send()
pool = coros.CoroutinePool(0, 2)
pool.execute_async(some_work)
done.wait()
def test_execute(self):
value = 'return value'
def some_work():
return value
pool = coros.CoroutinePool(0, 2)
worker = pool.execute(some_work)
self.assertEqual(value, worker.wait())
def test_multiple_coros(self):
evt = coros.event()
results = []
def producer():
results.append('prod')
evt.send()
def consumer():
results.append('cons1')
evt.wait()
results.append('cons2')
pool = coros.CoroutinePool(0, 2)
done = pool.execute(consumer)
pool.execute_async(producer)
done.wait()
self.assertEquals(['cons1', 'prod', 'cons2'], results)
if __name__ == '__main__':
tests.main()

323
eventlet/greenlib.py Normal file
View File

@@ -0,0 +1,323 @@
"""\
@file greenlib.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
import itertools
import greenlet
from eventlet import tls
__all__ = [
'switch', 'kill', 'tracked_greenlets',
'greenlet_id', 'greenlet_dict', 'GreenletContext',
'tracked_greenlet',
]
try:
reversed
except NameError:
def reversed(something):
for x in something[::-1]:
yield x
_threadlocal = tls.local()
def tracked_greenlet():
"""
Returns a greenlet that has a greenlet-local dictionary and can be
used with GreenletContext and enumerated with tracked_greenlets
"""
return greenlet.greenlet(greenlet_body)
class GreenletContextManager(object):
"""
Per-thread manager for GreenletContext. Created lazily on registration
"""
def __new__(cls, *args, **kw):
dct = greenlet_dict()
self = dct.get('greenlet_context', None)
if self is not None:
return self
self = super(GreenletContextManager, cls).__new__(cls, *args, **kw)
dct['greenlet_context'] = self
self.contexts = []
return self
def add_context(self, ctx):
fn = getattr(ctx, '_swap_in', None)
if fn is not None:
fn()
self.contexts.append(ctx)
def remove_context(self, ctx):
try:
idx = self.contexts.index(ctx)
except ValueError:
return
else:
del self.contexts[idx]
fn = getattr(ctx, '_swap_out', None)
if fn is not None:
fn()
fn = getattr(ctx, '_finalize', None)
if fn is not None:
fn()
def swap_in(self):
for ctx in self.contexts:
fn = getattr(ctx, '_swap_in', None)
if fn is not None:
fn()
def swap_out(self):
for ctx in reversed(self.contexts):
fn = getattr(ctx, '_swap_out', None)
if fn is not None:
fn()
def finalize(self):
for ctx in reversed(self.contexts):
fn = getattr(ctx, '_swap_out', None)
if fn is not None:
fn()
fn = getattr(ctx, '_finalize', None)
if fn is not None:
fn()
del self.contexts[:]
try:
del greenlet_dict()['greenlet_context']
except KeyError:
pass
class GreenletContext(object):
"""
A context manager to be triggered when a specific tracked greenlet is
swapped in, swapped out, or finalized.
To use, subclass and override the swap_in, swap_out, and/or finalize
methods, for example::
import greenlib
from greenlib import greenlet_id, tracked_greenlet, switch
class NotifyContext(greenlib.GreenletContext):
def swap_in(self):
print "swap_in"
def swap_out(self):
print "swap_out"
def finalize(self):
print "finalize"
def another_greenlet():
print "another_greenlet"
def notify_demo():
print "starting"
NotifyContext().register()
switch(tracked_greenlet(), (another_greenlet,))
print "finishing"
# we could have kept the NotifyContext object
# to unregister it here but finalization of all
# contexts is implicit when the greenlet returns
t = tracked_greenlet()
switch(t, (notify_demo,))
The output should be:
starting
swap_in
swap_out
another_greenlet
swap_in
finishing
swap_out
finalize
"""
_balance = 0
def _swap_in(self):
if self._balance != 0:
raise RuntimeError("balance != 0: %r" % (self._balance,))
self._balance = self._balance + 1
fn = getattr(self, 'swap_in', None)
if fn is not None:
fn()
def _swap_out(self):
if self._balance != 1:
raise RuntimeError("balance != 1: %r" % (self._balance,))
self._balance = self._balance - 1
fn = getattr(self, 'swap_out', None)
if fn is not None:
fn()
def register(self):
GreenletContextManager().add_context(self)
def unregister(self):
GreenletContextManager().remove_context(self)
def _finalize(self):
fn = getattr(self, 'finalize', None)
if fn is not None:
fn()
def kill(g):
"""
Kill the given greenlet if it is alive by sending it a GreenletExit.
Note that of any other exception is raised, it will pass-through!
"""
if not g:
return
kill_exc = greenlet.GreenletExit()
try:
try:
g.parent = greenlet.getcurrent()
except ValueError:
pass
try:
switch(g, exc=kill_exc)
except SwitchingToDeadGreenlet:
pass
except greenlet.GreenletExit, e:
if e is not kill_exc:
raise
def tracked_greenlets():
"""
Return a list of greenlets tracked in this thread. Tracked greenlets
use greenlet_body() to ensure that they have greenlet-local storage.
"""
try:
return _threadlocal.greenlets.keys()
except AttributeError:
return []
def greenlet_id():
"""
Get the id of the current tracked greenlet, returns None if the
greenlet is not tracked.
"""
try:
d = greenlet_dict()
except RuntimeError:
return None
return d['greenlet_id']
def greenlet_dict():
"""
Return the greenlet local storage for this greenlet. Raises RuntimeError
if this greenlet is not tracked.
"""
self = greenlet.getcurrent()
try:
return _threadlocal.greenlets[self]
except (AttributeError, KeyError):
raise RuntimeError("greenlet %r is not tracked" % (self,))
def _greenlet_context(dct=None):
if dct is None:
try:
dct = greenlet_dict()
except RuntimeError:
return None
return dct.get('greenlet_context', None)
def _greenlet_context_call(name, dct=None):
ctx = _greenlet_context(dct)
fn = getattr(ctx, name, None)
if fn is not None:
fn()
def greenlet_body(value, exc):
"""
Track the current greenlet during the execution of the given callback,
normally you would use tracked_greenlet() to get a greenlet that uses this.
Greenlets using this body must be greenlib.switch()'ed to
"""
if exc is not None:
if isinstance(exc, tuple):
raise exc[0], exc[1], exc[2]
raise exc
cb, args = value[0], value[1:]
try:
greenlets = _threadlocal.greenlets
except AttributeError:
greenlets = _threadlocal.greenlets = {}
else:
if greenlet.getcurrent() in greenlets:
raise RuntimeError("greenlet_body can not be called recursively!")
try:
greenlet_id = _threadlocal.next_greenlet_id.next()
except AttributeError:
greenlet_id = 1
_threadlocal.next_greenlet_id = itertools.count(2)
greenlets[greenlet.getcurrent()] = {'greenlet_id': greenlet_id}
try:
return cb(*args)
finally:
_greenlet_context_call('finalize')
greenlets.pop(greenlet.getcurrent(), None)
class SwitchingToDeadGreenlet(RuntimeError):
pass
def switch(other=None, value=None, exc=None):
"""
Switch to another greenlet, passing value or exception
"""
self = greenlet.getcurrent()
if other is None:
other = self.parent
if other is None:
other = self
if not (other or hasattr(other, 'run')):
raise SwitchingToDeadGreenlet("Switching to dead greenlet %r %r %r" % (other, value, exc))
_greenlet_context_call('swap_out')
try:
rval = other.switch(value, exc)
if not rval or not other:
res, exc = rval, None
else:
res, exc = rval
except:
res, exc = None, sys.exc_info()
_greenlet_context_call('swap_in')
if isinstance(exc, tuple):
typ, exc, tb = exc
raise typ, exc, tb
elif exc is not None:
raise exc
return res

334
eventlet/httpc.py Executable file
View File

@@ -0,0 +1,334 @@
"""\
@file httpc.py
@author Donovan Preston
Copyright (c) 2005-2006, Donovan Preston
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import datetime
import httplib
import os.path
import os
import time
import urlparse
from mx.DateTime import Parser
_old_HTTPConnection = httplib.HTTPConnection
_old_HTTPSConnection = httplib.HTTPSConnection
HTTP_TIME_FORMAT = '%a, %d %b %Y %H:%M:%S GMT'
to_http_time = lambda t: time.strftime(HTTP_TIME_FORMAT, time.gmtime(t))
from_http_time = lambda t: int(Parser.DateTimeFromString(t).gmticks())
def host_and_port_from_url(url):
"""@brief Simple function to get host and port from an http url.
@return Returns host, port and port may be None.
"""
host = None
port = None
#print url
parsed_url = urlparse.urlparse(url)
try:
host, port = parsed_url[1].split(':')
except ValueError:
host = parsed_url[1].split(':')
return host, port
def better_putrequest(self, method, url, skip_host=0):
self.method = method
self.path = url
self.old_putrequest(method, url, skip_host)
class HttpClient(httplib.HTTPConnection):
"""A subclass of httplib.HTTPConnection which works around a bug
in the interaction between eventlet sockets and httplib. httplib relies
on gc to close the socket, causing the socket to be closed too early.
This is an awful hack and the bug should be fixed properly ASAP.
"""
def __init__(self, host, port=None, strict=None):
_old_HTTPConnection.__init__(self, host, port, strict)
def close(self):
pass
old_putrequest = httplib.HTTPConnection.putrequest
putrequest = better_putrequest
def wrap_httplib_with_httpc():
httplib.HTTP._connection_class = httplib.HTTPConnection = HttpClient
httplib.HTTPS._connection_class = httplib.HTTPSConnection = HttpsClient
class HttpsClient(httplib.HTTPSConnection):
def close(self):
pass
old_putrequest = httplib.HTTPSConnection.putrequest
putrequest = better_putrequest
class FileScheme(object):
"""Retarded scheme to local file wrapper."""
host = '<file>'
port = '<file>'
reason = '<none>'
def __init__(self, location):
pass
def request(self, method, fullpath, body='', headers=None):
self.status = 200
self.path = fullpath.split('?')[0]
self.method = method = method.lower()
assert method in ('get', 'put', 'delete')
if method == 'delete':
try:
os.remove(self.path)
except OSError:
pass # don't complain if already deleted
elif method == 'put':
try:
f = file(self.path, 'w')
f.write(body)
f.close()
except IOError, e:
self.status = 500
self.raise_connection_error()
elif method == 'get':
if not os.path.exists(self.path):
self.status = 404
self.raise_connection_error(NotFound)
def connect(self):
pass
def getresponse(self):
return self
def getheader(self, header):
if header == 'content-length':
try:
return os.path.getsize(self.path)
except OSError:
return 0
def read(self, howmuch=None):
if self.method == 'get':
try:
fl = file(self.path, 'r')
if howmuch is None:
return fl.read()
else:
return fl.read(howmuch)
except IOError:
self.status = 500
self.raise_connection_error()
return ''
def raise_connection_error(self, klass=None):
if klass is None:
klass=ConnectionError
raise klass(
self.method, self.host, self.port,
self.path, self.status, self.reason, '')
class ConnectionError(Exception):
def __init__(self, method, host, port, path, status, reason, body):
self.method = method
self.host = host
self.port = port
self.path = path
self.status = status
self.reason = reason
self.body = body
Exception.__init__(self)
def __repr__(self):
return "ConnectionError(%r, %r, %r, %r, %r, %r, %r)" % (
self.method, self.host, self.port,
self.path, self.status, self.reason, self.body)
__str__ = __repr__
class UnparseableResponse(ConnectionError):
def __init__(self, content_type, response):
self.content_type = content_type
self.response = response
Exception.__init__(self)
def __repr__(self):
return "UnparseableResponse(%r, %r)" % (
self.content_type, self.response)
__str__ = __repr__
class Accepted(ConnectionError):
pass
class NotFound(ConnectionError):
pass
class Forbidden(ConnectionError):
pass
class InternalServerError(ConnectionError):
pass
class Gone(ConnectionError):
pass
status_to_error_map = {
500: InternalServerError,
410: Gone,
404: NotFound,
403: Forbidden,
202: Accepted,
}
scheme_to_factory_map = {
'http': HttpClient,
'https': HttpsClient,
'file': FileScheme,
}
def make_connection(scheme, location, use_proxy):
if use_proxy:
if "http_proxy" in os.environ:
location = os.environ["http_proxy"]
elif "ALL_PROXY" in os.environ:
location = os.environ["ALL_PROXY"]
else:
location = "localhost:3128" #default to local squid
# run a little heuristic to see if it's an url, and if so parse out the hostpart
if location.startswith('http'):
_scheme, location, path, parameters, query, fragment = urlparse.urlparse(location)
result = scheme_to_factory_map[scheme](location)
result.connect()
return result
def connect(url, use_proxy=False):
scheme, location, path, params, query, id = urlparse.urlparse(url)
return make_connection(scheme, location, use_proxy)
def request(connection, method, url, body='', headers=None, dumper=None, loader=None, use_proxy=False, verbose=False, ok=None):
if ok is None:
ok = (200, 201, 204)
if headers is None:
headers = {}
if not use_proxy:
scheme, location, path, params, query, id = urlparse.urlparse(url)
url = path
if query:
url += "?" + query
else:
scheme, location, path, params, query, id = urlparse.urlparse(url)
headers.update({ "host" : location })
if scheme == 'file':
use_proxy = False
if dumper is not None:
body = dumper(body)
headers['content-length'] = len(body)
connection.request(method, url, body, headers)
response = connection.getresponse()
if (response.status not in ok):
klass = status_to_error_map.get(response.status, ConnectionError)
raise klass(
connection.method, connection.host, connection.port,
connection.path, response.status, response.reason, response.read())
body = response.read()
if loader is None:
return body
try:
body = loader(body)
except Exception, e:
raise UnparseableResponse(loader, body)
if verbose:
return response.status, response.msg, body
return body
def make_suite(dumper, loader, fallback_content_type):
def get(url, headers=None, use_proxy=False, verbose=False, ok=None):
#import pdb; pdb.Pdb().set_trace()
if headers is None:
headers = {}
connection = connect(url)
return request(connection, 'GET', url, '', headers, None, loader, use_proxy, verbose, ok)
def put(url, data, headers=None, content_type=None, verbose=False, ok=None):
if headers is None:
headers = {}
if content_type is not None:
headers['content-type'] = content_type
else:
headers['content-type'] = fallback_content_type
connection = connect(url)
return request(connection, 'PUT', url, data, headers, dumper, loader, verbose=verbose, ok=ok)
def delete(url, verbose=False, ok=None):
return request(connect(url), 'DELETE', url, verbose=verbose, ok=ok)
def post(url, data='', headers=None, content_type=None, verbose=False, ok=None):
connection = connect(url)
if headers is None:
headers = {}
if 'content-type' not in headers:
if content_type is not None:
headers['content-type'] = content_type
else:
headers['content-type'] = fallback_content_type
return request(connect(url), 'POST', url, data, headers, dumper, loader, verbose=verbose, ok=ok)
return get, put, delete, post
get, put, delete, post = make_suite(str, None, 'text/plain')

491
eventlet/httpd.py Normal file
View File

@@ -0,0 +1,491 @@
"""\
@file httpd.py
@author Donovan Preston
Copyright (c) 2005-2006, Donovan Preston
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import cgi
import cStringIO
import errno
import socket
import sys
import time
import urllib
import socket
import traceback
import cStringIO
import BaseHTTPServer
from eventlet import api
from eventlet import coros
USE_ACCESS_LOG = True
CONNECTION_CLOSED = (errno.EPIPE, errno.ECONNRESET)
class Request(object):
_method = None
_path = None
_responsecode = 200
_reason_phrase = None
_request_started = False
_chunked = False
_producer_adapters = {}
depth = 0
def __init__(self, protocol, method, path, headers):
self.request_start_time = time.time()
self.site = protocol.server.site
self.protocol = protocol
self._method = method
if '?' in path:
self._path, self._query = path.split('?', 1)
self._query = self._query.replace('&amp;', '&')
else:
self._path, self._query = path, None
self._incoming_headers = headers
self._outgoing_headers = dict()
def response(self, code, reason_phrase=None, headers=None, body=None):
"""Change the response code. This will not be sent until some
data is written; last call to this method wins. Default is
200 if this is not called.
"""
self._responsecode = code
self._reason_phrase = reason_phrase
self.protocol.set_response_code(self, code, reason_phrase)
if headers is not None:
for key, value in headers:
self.set_header(key, value)
if body is not None:
self.write(body)
def is_okay(self):
return 200 <= self._responsecode <= 299
def full_url(self):
path = self.path()
query = self.query()
if query:
path = path + '?' + query
via = self.get_header('via', '')
if via.strip():
next_part = iter(via.split()).next
received_protocol = next_part()
received_by = next_part()
if received_by.endswith(','):
received_by = received_by[:-1]
else:
comment = ''
while not comment.endswith(','):
try:
comment += next_part()
except StopIteration:
comment += ','
break
comment = comment[:-1]
else:
received_by = self.get_header('host')
return '%s://%s%s' % (self.request_protocol(), received_by, path)
def begin_response(self, length="-"):
"""Begin the response, and return the initial response text
"""
self._request_started = True
request_time = time.time() - self.request_start_time
code = self._responsecode
proto = self.protocol
if USE_ACCESS_LOG:
proto.server.write_access_log_line(
proto.client_address[0],
time.strftime("%d/%b/%Y %H:%M:%S"),
proto.requestline,
code,
length,
request_time)
if self._reason_phrase is not None:
message = self._reason_phrase.split("\n")[0]
elif code in proto.responses:
message = proto.responses[code][0]
else:
message = ''
if proto.request_version == 'HTTP/0.9':
return []
response_lines = proto.generate_status_line()
if not self._outgoing_headers.has_key('connection'):
con = self.get_header('connection')
if con is None and proto.request_version == 'HTTP/1.0':
con = 'close'
if con is not None:
self.set_header('connection', con)
for key, value in self._outgoing_headers.items():
key = '-'.join([x.capitalize() for x in key.split('-')])
response_lines.append("%s: %s" % (key, value))
response_lines.append("")
return response_lines
def write(self, obj):
"""Writes an arbitrary object to the response, using
the sitemap's adapt method to convert it to bytes.
"""
if isinstance(obj, str):
self._write_bytes(obj)
elif isinstance(obj, unicode):
# use utf8 encoding for now, *TODO support charset negotiation
# Content-Type: text/html; charset=utf-8
ctype = self._outgoing_headers.get('content-type', 'text/html')
ctype = ctype + '; charset=utf-8'
self._outgoing_headers['content-type'] = ctype
self._write_bytes(obj.encode('utf8'))
else:
self.site.adapt(obj, self)
def _write_bytes(self, data):
"""Write all the data of the response.
Can be called just once.
"""
if self._request_started:
print "Request has already written a response:"
traceback.print_stack()
return
self._outgoing_headers['content-length'] = len(data)
response_lines = self.begin_response(len(data))
response_lines.append(data)
self.protocol.wfile.write("\r\n".join(response_lines))
if hasattr(self.protocol.wfile, 'flush'):
self.protocol.wfile.flush()
def method(self):
return self._method
def path(self):
return self._path
def path_segments(self):
return [urllib.unquote_plus(x) for x in self._path.split('/')[1:]]
def query(self):
return self._query
def uri(self):
if self._query:
return '%s?%s' % (
self._path, self._query)
return self._path
def get_headers(self):
return self._incoming_headers
def get_header(self, header_name, default=None):
return self.get_headers().get(header_name.lower(), default)
def get_query_pairs(self):
if not hasattr(self, '_split_query'):
if self._query is None:
self._split_query = ()
else:
spl = self._query.split('&')
spl = [x.split('=', 1) for x in spl if x]
self._split_query = []
for query in spl:
if len(query) == 1:
key = query[0]
value = ''
else:
key, value = query
self._split_query.append((urllib.unquote_plus(key), urllib.unquote_plus(value)))
return self._split_query
def get_queries_generator(self, name):
"""Generate all query parameters matching the given name.
"""
for key, value in self.get_query_pairs():
if key == name or not name:
yield value
get_queries = lambda self, name: list(self.get_queries_generator)
def get_query(self, name, default=None):
try:
return self.get_queries_generator(name).next()
except StopIteration:
return default
def get_arg_list(self, name):
return self.get_field_storage().getlist(name)
def get_arg(self, name, default=None):
return self.get_field_storage().getfirst(name, default)
def get_field_storage(self):
if not hasattr(self, '_field_storage'):
if self.method() == 'GET':
data = ''
if self._query:
data = self._query
fl = cStringIO.StringIO(data)
else:
fl = self.protocol.rfile
## Allow our resource to provide the FieldStorage instance for
## customization purposes.
headers = self.get_headers()
environ = dict(
REQUEST_METHOD='POST',
QUERY_STRING=self._query or '')
if (hasattr(self, 'resource') and
hasattr(self.resource, 'getFieldStorage')):
self._field_storage = self.resource.getFieldStorage(
self, fl, headers, environ)
else:
self._field_storage = cgi.FieldStorage(
fl, headers, environ=environ)
return self._field_storage
def set_header(self, key, value):
if key.lower() == 'connection' and value.lower() == 'close':
self.protocol.close_connection = 1
self._outgoing_headers[key.lower()] = value
__setitem__ = set_header
def get_outgoing_header(self, key):
return self._outgoing_headers[key.lower()]
def has_outgoing_header(self, key):
return self._outgoing_headers.has_key(key.lower())
def socket(self):
return self.protocol.socket
def error(self, response=None, body=None, log_traceback=True):
if log_traceback:
traceback.print_exc()
if response is None:
response = 500
if body is None:
typ, val, tb = sys.exc_info()
body = dict(type=str(typ), error=True, reason=str(val))
self.response(response)
if type(body) is str:
self.write(body)
return
try:
produce(body, self)
except Exception, e:
traceback.print_exc()
if not self.response_written():
self.write('Internal Server Error')
def not_found(self):
self.error(404, 'Not Found\n', log_traceback=False)
def read_body(self):
if not hasattr(self, '_cached_parsed_body'):
if not hasattr(self, '_cached_body'):
length = self.get_header('content-length')
if length:
length = int(length)
if length:
self._cached_body = self.protocol.rfile.read(length)
else:
self._cached_body = ''
body = self._cached_body
if hasattr(self.site, 'parsers'):
parser = self.site.parsers.get(
self.get_header('content-type'))
if parser is not None:
body = parser(body)
self._cached_parsed_body = body
return self._cached_parsed_body
def response_written(self):
## TODO change badly named variable
return self._request_started
def request_version(self):
return self.protocol.request_version
def request_protocol(self):
if self.protocol.socket.is_secure:
return "https"
return "http"
def server_address(self):
return self.protocol.server.address
def __repr__(self):
return "<Request %s %s>" % (
getattr(self, '_method'), getattr(self, '_path'))
DEFAULT_TIMEOUT = 300
class Timeout(RuntimeError):
pass
class HttpProtocol(BaseHTTPServer.BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def __init__(self, request, client_address, server):
self.socket = self.request = self.rfile = self.wfile = request
self.client_address = client_address
self.server = server
self._code = 200
self._message = 'OK'
def set_response_code(self, request, code, message):
self._code = code
if message is not None:
self._message = message.split("\n")[0]
elif code in self.responses:
self._message = self.responses[code][0]
else:
self._message = ''
def generate_status_line(self):
return [
"%s %d %s" % (
self.protocol_version, self._code, self._message)]
def handle(self):
self.close_connection = 0
timeout = DEFAULT_TIMEOUT
while not self.close_connection:
if timeout == 0:
break
cancel = api.exc_after(timeout, Timeout)
try:
self.raw_requestline = self.rfile.readline()
except socket.error, e:
if e[0] in CONNECTION_CLOSED:
self.close_connection = True
cancel.cancel()
continue
except Timeout:
self.close_connection = True
continue
cancel.cancel()
if not self.raw_requestline or not self.parse_request():
self.close_connection = True
continue
request = Request(self, self.command, self.path, self.headers)
request.set_header('Server', self.version_string())
request.set_header('Date', self.date_time_string())
try:
timeout = int(request.get_header('keep-alive'))
except (TypeError, ValueError), e:
pass
try:
self.server.site.handle_request(request)
# throw an exception if it failed to write a body
if not request.response_written():
raise NotImplementedError("Handler failed to write response to request: %s" % request)
if not hasattr(self, '_cached_body'):
request.read_body() ## read & discard body
continue
except socket.error, e:
# Broken pipe, connection reset by peer
if e[0] in CONNECTION_CLOSED:
#print "Remote host closed connection before response could be sent"
pass
else:
raise
except Exception, e:
if not request.response_written():
request.response(500)
request.write('Internal Server Error')
self.socket.close()
raise
self.socket.close()
class Server(BaseHTTPServer.HTTPServer):
def __init__(self, socket, address, site, log):
self.socket = socket
self.address = address
self.site = site
if log:
self.log = log
if hasattr(log, 'info'):
log.write = log.info
else:
self.log = self
def write(self, something):
sys.stdout.write('%s\n' % (something, ))
def log_message(self, message):
self.log.write(message)
def log_exception(self, type, value, tb):
print ''.join(traceback.format_exception(type, value, tb))
def write_access_log_line(self, *args):
"""Write a line to the access.log. Arguments:
client_address, date_time, requestline, code, size, request_time
"""
self.log.write(
'%s - - [%s] "%s" %s %s %.6f' % args)
def server(sock, site, log=None, max_size=512):
pool = coros.CoroutinePool(max_size=max_size)
serv = Server(sock, sock.getsockname(), site, log)
try:
print "httpd starting up on", sock.getsockname()
while True:
try:
new_sock, address = sock.accept()
proto = HttpProtocol(new_sock, address, serv)
pool.execute_async(proto.handle)
except KeyboardInterrupt:
api.get_hub().remove_descriptor(sock.fileno())
print "httpd exiting"
break
finally:
try:
sock.close()
except socket.error:
pass

139
eventlet/httpd_test.py Normal file
View File

@@ -0,0 +1,139 @@
"""\
@file httpd_test.py
@author Donovan Preston
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
from eventlet import api
from eventlet import httpd
from eventlet import processes
from eventlet import util
util.wrap_socket_with_coroutine_socket()
from eventlet import tests
class Site(object):
def handle_request(self, req):
req.write('hello world')
def adapt(self, obj, req):
req.write(str(obj))
CONTENT_LENGTH = 'content-length'
"""
HTTP/1.1 200 OK
Date: foo
Content-length: 11
hello world
"""
class ConnectionClosed(Exception):
pass
def read_http(sock):
response_line = sock.readline()
if not response_line:
raise ConnectionClosed
raw_headers = sock.readuntil('\r\n\r\n').strip()
#print "R", response_line, raw_headers
headers = dict()
for x in raw_headers.split('\r\n'):
#print "X", x
key, value = x.split(': ', 1)
headers[key.lower()] = value
if CONTENT_LENGTH in headers:
num = int(headers[CONTENT_LENGTH])
body = sock.read(num)
#print body
class TestHttpd(tests.TestCase):
mode = 'static'
def setUp(self):
self.killer = api.spawn(
httpd.server, api.tcp_listener(('0.0.0.0', 12346)), Site(), max_size=128)
def tearDown(self):
api.kill(self.killer)
def test_001_server(self):
sock = api.connect_tcp(
('127.0.0.1', 12346))
sock.write('GET / HTTP/1.0\r\nHost: localhost\r\n\r\n')
result = sock.read()
sock.close()
## The server responds with the maximum version it supports
self.assert_(result.startswith('HTTP'), result)
self.assert_(result.endswith('hello world'))
def test_002_keepalive(self):
sock = api.connect_tcp(
('127.0.0.1', 12346))
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
read_http(sock)
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
read_http(sock)
sock.close()
def test_003_passing_non_int_to_read(self):
# This should go in test_wrappedfd
sock = api.connect_tcp(
('127.0.0.1', 12346))
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
cancel = api.exc_after(1, RuntimeError)
self.assertRaises(TypeError, sock.read, "This shouldn't work")
cancel.cancel()
sock.close()
def test_004_close_keepalive(self):
sock = api.connect_tcp(
('127.0.0.1', 12346))
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
read_http(sock)
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n')
read_http(sock)
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
self.assertRaises(ConnectionClosed, read_http, sock)
sock.close()
def test_005_run_apachebench(self):
url = 'http://localhost:12346/'
out = processes.Process('/usr/sbin/ab', ['-c','64','-n','1024', '-k', url])
print out.read()
if __name__ == '__main__':
tests.main()

39
eventlet/httpdate.py Normal file
View File

@@ -0,0 +1,39 @@
"""\
@file httpdate.py
@author Donovan Preston
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import time
__all__ = ['format_date_time']
# Weekday and month names for HTTP date/time formatting; always English!
_weekdayname = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
_monthname = [None, # Dummy so we can use 1-based month numbers
"Jan", "Feb", "Mar", "Apr", "May", "Jun",
"Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]
def format_date_time(timestamp):
year, month, day, hh, mm, ss, wd, y, z = time.gmtime(timestamp)
return "%s, %02d %3s %4d %02d:%02d:%02d GMT" % (
_weekdayname[wd], day, _monthname[month], year, hh, mm, ss
)

31
eventlet/jsonhttp.py Normal file
View File

@@ -0,0 +1,31 @@
"""\
@file jsonhttp.py
@author Donovan Preston
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
from eventlet import httpc
import simplejson
get, put, delete, post = httpc.make_suite(
simplejson.dumps, simplejson.loads, 'application/json')

219
eventlet/kqueuehub.py Normal file
View File

@@ -0,0 +1,219 @@
"""\
@file kqueuehub.py
@author Donovan Preston
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
import select
import kqueue
import traceback
from errno import EBADF
from eventlet import greenlib
from eventlet.runloop import RunLoop, Timer
import greenlet
class Hub(object):
def __init__(self):
self.runloop = RunLoop(self.wait)
self.descriptor_queue = {}
self.descriptors = {}
self.greenlet = None
self.kfd = None
def stop(self):
self.process_queue()
self.descriptors, self.descriptor_queue = self.descriptor_queue, {}
os.close(self.kfd)
self.kfd = None
self.runloop.abort()
if self.greenlet is not greenlet.getcurrent():
self.switch()
def schedule_call(self, *args, **kw):
return self.runloop.schedule_call(*args, **kw)
def switch(self):
if not self.greenlet:
self.greenlet = greenlib.tracked_greenlet()
args = ((self.runloop.run,),)
else:
args = ()
try:
greenlet.getcurrent().parent = self.greenlet
except ValueError:
pass
return greenlib.switch(self.greenlet, *args)
def add_descriptor(self, fileno, read=None, write=None, exc=None):
self.descriptor_queue[fileno] = read, write, exc
def remove_descriptor(self, fileno):
self.descriptor_queue[fileno] = None, None, None
def exc_descriptor(self, fileno):
# We must handle two cases here, the descriptor
# may be changing or removing its exc handler
# in the queue, or it may be waiting on the queue.
exc = None
try:
exc = self.descriptor_queue[fileno][2]
except KeyError:
try:
exc = self.descriptors[fileno][2]
except KeyError:
pass
if exc is not None:
try:
exc()
except self.runloop.SYSTEM_EXCEPTIONS:
self.squelch_exception(fileno, sys.exc_info())
def squelch_exception(self, fileno, exc_info):
traceback.print_exception(*exc_info)
print >>sys.stderr, "Removing descriptor: %r" % (fileno,)
try:
self.remove_descriptor(fileno)
except Exception, e:
print >>sys.stderr, "Exception while removing descriptor! %r" % (e,)
def process_queue(self):
if self.kfd is None:
self.kfd = kqueue.kqueue()
d = self.descriptors
E_R = kqueue.EVFILT_READ
E_W = kqueue.EVFILT_WRITE
E = kqueue.Event
E_ADD = kqueue.EV_ADD
E_DEL = kqueue.EV_DELETE
kevent = kqueue.kevent
kfd = self.kfd
for fileno, rwe in self.descriptor_queue.iteritems():
read, write, exc = rwe
if read is None and write is None and exc is None:
try:
read, write, exc = d.pop(fileno)
except KeyError:
pass
else:
l = []
if read is not None:
l.append(E(fileno, E_R, E_DEL))
if write is not None:
l.append(E(fileno, E_W, E_DEL))
if l:
try:
kevent(kfd, l, 0, 0)
except OSError, e:
if e[0] != EBADF:
raise
else:
l = []
try:
oldr, oldw, olde = d[fileno]
except KeyError:
pass
else:
if oldr is not None:
if read is None:
l.append(E(fileno, E_R, E_DEL))
else:
read = None
if oldw is not None:
if write is None:
l.append(E(fileno, E_W, E_DEL))
else:
write = None
if read is not None:
l.append(E(fileno, E_R, E_ADD))
if write is not None:
l.append(E(fileno, E_W, E_ADD))
if l:
try:
kevent(kfd, l, 0, 0)
except OSError, e:
if e[0] != EBADF:
raise
try:
del d[fileno]
except KeyError:
pass
if exc is not None:
try:
exc(fileno)
except SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_exception(fileno, sys.exc_info())
continue
d[fileno] = rwe
self.descriptor_queue.clear()
def wait(self, seconds=None):
self.process_queue()
if seconds is not None:
seconds *= 1000000000.0
dct = self.descriptors
events = kqueue.kevent(self.kfd, [], len(dct), seconds)
SYSTEM_EXCEPTIONS = self.runloop.SYSTEM_EXCEPTIONS
E_R = kqueue.EVFILT_READ
E_W = kqueue.EVFILT_WRITE
E_EOF = kqueue.EV_EOF
for e in events:
fileno = e.ident
event = e.filter
try:
read, write, exc = dct[fileno]
except KeyError:
continue
if read is not None and event == E_R:
try:
read(fileno)
except SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_exception(fileno, sys.exc_info())
elif exc is not None and e.fflags & E_EOF:
try:
exc(fileno)
except SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_exception(fileno, sys.exc_info())
if write is not None and event == E_W:
try:
write(fileno)
except SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_exception(fileno, sys.exc_info())

112
eventlet/logutil.py Normal file
View File

@@ -0,0 +1,112 @@
"""\
@file logutil.py
@author Donovan Preston
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import syslog
import logging
def file_logger(filename):
"""Create a logger. This sucks, the logging module sucks, but
it'll do for now.
"""
handler = logging.FileHandler(filename)
formatter = logging.Formatter()
handler.setFormatter(formatter)
log = logging.getLogger(filename)
log.addHandler(handler)
log.setLevel(logging.DEBUG)
return log, handler
def stream_logger(stream):
"""Create a logger. This sucks."""
handler = logging.StreamHandler(stream)
formatter = logging.Formatter()
handler.setFormatter(formatter)
log = logging.getLogger()
log.addHandler(handler)
log.setLevel(logging.DEBUG)
return log, handler
class LineLogger(object):
towrite = ''
def __init__(self, emit=None):
if emit is not None:
self.emit = emit
def write(self, stuff):
self.towrite += stuff
if '\n' in self.towrite:
self.flush()
def flush(self):
try:
newline = self.towrite.index('\n')
except ValueError:
newline = len(self.towrite)
while True:
self.emit(self.towrite[:newline])
self.towrite = self.towrite[newline+1:]
try:
newline = self.towrite.index('\n')
except ValueError:
break
def close(self):
pass
def emit(self, *args):
pass
class SysLogger(LineLogger):
"""A file-like object which writes to syslog. Can be inserted
as sys.stdin and sys.stderr to have logging output redirected
to syslog.
"""
def __init__(self, priority):
self.priority = priority
def emit(self, line):
syslog.syslog(self.priority, line)
class TeeLogger(LineLogger):
def __init__(self, one, two):
self.one, self.two = one, two
def emit(self, line):
self.one.emit(line)
self.two.emit(line)
class FileLogger(LineLogger):
def __init__(self, file):
self.file = file
def emit(self, line):
self.file.write(line + '\n')
self.file.flush()

189
eventlet/pollhub.py Normal file
View File

@@ -0,0 +1,189 @@
"""\
@file pollhub.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
import select
import socket
import errno
import traceback
from time import sleep
from eventlet import greenlib
from eventlet.runloop import RunLoop, Timer
import greenlet
EXC_MASK = select.POLLERR | select.POLLHUP | select.POLLNVAL
READ_MASK = select.POLLIN
WRITE_MASK = select.POLLOUT
class Hub(object):
def __init__(self):
self.runloop = RunLoop(self.wait)
self.descriptor_queue = {}
self.descriptors = {}
self.greenlet = None
self.poll = select.poll()
def stop(self):
self.process_queue()
self.runloop.abort()
if self.greenlet is not greenlet.getcurrent():
self.switch()
def schedule_call(self, *args, **kw):
return self.runloop.schedule_call(*args, **kw)
def switch(self):
if not self.greenlet:
self.greenlet = greenlib.tracked_greenlet()
args = ((self.runloop.run,),)
else:
args = ()
try:
greenlet.getcurrent().parent = self.greenlet
except ValueError:
pass
return greenlib.switch(self.greenlet, *args)
def add_descriptor(self, fileno, read=None, write=None, exc=None):
if fileno in self.descriptor_queue:
oread, owrite, oexc = self.descriptor_queue[fileno]
read, write, exc = read or oread, write or owrite, exc or oexc
self.descriptor_queue[fileno] = read, write, exc
def remove_descriptor(self, fileno):
self.descriptor_queue[fileno] = None, None, None
def exc_descriptor(self, fileno):
# We must handle two cases here, the descriptor
# may be changing or removing its exc handler
# in the queue, or it may be waiting on the queue.
exc = None
try:
exc = self.descriptor_queue[fileno][2]
except KeyError:
try:
exc = self.descriptors[fileno][2]
except KeyError:
pass
if exc is not None:
try:
exc(fileno)
except self.runloop.SYSTEM_EXCEPTIONS:
self.squelch_exception(fileno, sys.exc_info())
def squelch_exception(self, fileno, exc_info):
traceback.print_exception(*exc_info)
print >>sys.stderr, "Removing descriptor: %r" % (fileno,)
try:
self.remove_descriptor(fileno)
except Exception, e:
print >>sys.stderr, "Exception while removing descriptor! %r" % (e,)
def process_queue(self):
d = self.descriptors
reg = self.poll.register
unreg = self.poll.unregister
rm = READ_MASK
wm = WRITE_MASK
for fileno, rwe in self.descriptor_queue.iteritems():
read, write, exc = rwe
if read is None and write is None and exc is None:
try:
del d[fileno]
except KeyError:
pass
else:
try:
unreg(fileno)
except socket.error:
# print "squelched socket err on unreg", fileno
pass
else:
mask = 0
if read is not None:
mask |= rm
if write is not None:
mask |= wm
oldmask = 0
try:
oldr, oldw, olde = d[fileno]
except KeyError:
pass
else:
if oldr is not None:
oldmask |= rm
if oldw is not None:
oldmask |= wm
if mask != oldmask:
reg(fileno, mask)
d[fileno] = rwe
self.descriptor_queue.clear()
def wait(self, seconds=None):
self.process_queue()
if not self.descriptors:
if seconds:
sleep(seconds)
return
try:
presult = self.poll.poll(seconds * 1000.0)
except select.error, e:
if e.args[0] == errno.EINTR:
return
raise
SYSTEM_EXCEPTIONS = self.runloop.SYSTEM_EXCEPTIONS
dct = self.descriptors
for fileno, event in presult:
try:
read, write, exc = dct[fileno]
except KeyError:
continue
if read is not None and event & READ_MASK:
try:
read(fileno)
except SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_exception(fileno, sys.exc_info())
elif exc is not None and event & EXC_MASK:
try:
exc(fileno)
except SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_exception(fileno, sys.exc_info())
if write is not None and event & WRITE_MASK:
try:
write(fileno)
except SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_exception(fileno, sys.exc_info())

174
eventlet/pools.py Normal file
View File

@@ -0,0 +1,174 @@
"""\
@file pools.py
@author Donovan Preston, Aaron Brashears
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import os
import socket
from eventlet import api
from eventlet import channel
from eventlet import httpc
class FanFailed(RuntimeError):
pass
class SomeFailed(FanFailed):
pass
class AllFailed(FanFailed):
pass
class Pool(object):
"""
When using the pool, if you do a get, you should ALWAYS do a put.
The pattern is:
thing = self.pool.get()
try:
# do stuff
finally:
self.pool.put(thing)
"""
def __init__(self, min_size=0, max_size=4):
self.min_size = min_size
self.max_size = max_size
self.current_size = 0
self.channel = channel.channel()
self.free_items = []
for x in range(min_size):
self.current_size += 1
self.free_items.append(self.create())
def get(self):
"""Return an item from the pool, when one is available
"""
if self.free_items:
return self.free_items.pop(0)
if self.current_size < self.max_size:
self.current_size += 1
return self.create()
return self.channel.receive()
def put(self, item):
"""Put an item back into the pool, when done
"""
if self.channel.balance < 0:
self.channel.send(item)
else:
self.free_items.append(item)
def free(self):
"""Return the number of free items in the pool.
"""
return len(self.free_items) + self.max_size - self.current_size
def waiting(self):
"""Return the number of routines waiting for a pool item.
"""
if self.channel.balance < 0:
return -self.channel.balance
return 0
def create(self):
"""Generate a new pool item
"""
raise NotImplementedError("Implement in subclass")
def fan(self, block, input_list):
chan = channel.channel()
results = []
exceptional_results = 0
for index, input_item in enumerate(input_list):
pool_item = self.get()
## Fan out
api.spawn(
self._invoke, block, pool_item, input_item, index, chan)
## Fan back in
for i in range(len(input_list)):
## Wait for all guys to send to the queue
index, value = chan.receive()
if isinstance(value, Exception):
exceptional_results += 1
results.append((index, value))
results.sort()
results = [value for index, value in results]
if exceptional_results:
if exceptional_results == len(results):
raise AllFailed(results)
raise SomeFailed(results)
return results
def _invoke(self, block, pool_item, input_item, index, chan):
try:
result = block(pool_item, input_item)
except Exception, e:
self.put(pool_item)
chan.send((index, e))
return
self.put(pool_item)
chan.send((index, result))
class Token(object):
pass
class TokenPool(Pool):
"""A pool which gives out tokens, an object indicating that
the person who holds the token has a right to consume some
limited resource.
"""
def create(self):
return Token()
class ConnectionPool(Pool):
"""A Pool which can limit the number of concurrent http operations
being made to a given server.
*NOTE: *TODO:
This does NOT currently keep sockets open. It discards the created
http object when it is put back in the pool. This is because we do
not yet have a combination of http clients and servers which can work
together to do HTTP keepalive sockets without errors.
"""
def __init__(self, proto, netloc, use_proxy, min_size=0, max_size=4):
self.proto = proto
self.netloc = netloc
self.use_proxy = use_proxy
Pool.__init__(self, min_size, max_size)
def create(self):
return httpc.make_connection(self.proto, self.netloc, self.use_proxy)
def put(self, item):
## Discard item, create a new connection for the pool
Pool.put(self, self.create())

179
eventlet/pools_test.py Normal file
View File

@@ -0,0 +1,179 @@
"""\
@file test_pools.py
@author Donovan Preston, Aaron Brashears
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import unittest
from eventlet import api
from eventlet import channel
from eventlet import pools
class IntPool(pools.Pool):
def create(self):
self.current_integer = getattr(self, 'current_integer', 0) + 1
return self.current_integer
class TestIntPool(unittest.TestCase):
mode = 'static'
def setUp(self):
self.pool = IntPool(min_size=0, max_size=4)
def test_integers(self):
# Do not actually use this pattern in your code. The pool will be
# exhausted, and unrestoreable.
# If you do a get, you should ALWAYS do a put, probably like this:
# try:
# thing = self.pool.get()
# # do stuff
# finally:
# self.pool.put(thing)
# with self.pool.some_api_name() as thing:
# # do stuff
self.assertEquals(self.pool.get(), 1)
self.assertEquals(self.pool.get(), 2)
self.assertEquals(self.pool.get(), 3)
self.assertEquals(self.pool.get(), 4)
def test_free(self):
self.assertEquals(self.pool.free(), 4)
gotten = self.pool.get()
self.assertEquals(self.pool.free(), 3)
self.pool.put(gotten)
self.assertEquals(self.pool.free(), 4)
def test_exhaustion(self):
waiter = channel.channel()
def consumer():
gotten = None
try:
gotten = self.pool.get()
finally:
waiter.send(gotten)
api.spawn(consumer)
one, two, three, four = (
self.pool.get(), self.pool.get(), self.pool.get(), self.pool.get())
self.assertEquals(self.pool.free(), 0)
# Let consumer run; nothing will be in the pool, so he will wait
api.sleep(0)
# Wake consumer
self.pool.put(one)
# wait for the consumer
self.assertEquals(waiter.receive(), one)
def test_blocks_on_pool(self):
waiter = channel.channel()
def greedy():
self.pool.get()
self.pool.get()
self.pool.get()
self.pool.get()
# No one should be waiting yet.
self.assertEquals(self.pool.waiting(), 0)
# The call to the next get will unschedule this routine.
self.pool.get()
# So this send should never be called.
waiter.send('Failed!')
killable = api.spawn(greedy)
# no one should be waiting yet.
self.assertEquals(self.pool.waiting(), 0)
## Wait for greedy
api.sleep(0)
## Greedy should be blocking on the last get
self.assertEquals(self.pool.waiting(), 1)
## Send will never be called, so balance should be 0.
self.assertEquals(waiter.balance, 0)
api.kill(killable)
class TestAbstract(unittest.TestCase):
mode = 'static'
def test_abstract(self):
## Going for 100% coverage here
## A Pool cannot be used without overriding create()
pool = pools.Pool()
self.assertRaises(NotImplementedError, pool.get)
class TestIntPool2(unittest.TestCase):
mode = 'static'
def setUp(self):
self.pool = IntPool(min_size=3, max_size=3)
def test_something(self):
self.assertEquals(len(self.pool.free_items), 3)
## Cover the clause in get where we get from the free list instead of creating
## an item on get
gotten = self.pool.get()
self.assertEquals(gotten, 1)
ALWAYS = RuntimeError('I always fail')
SOMETIMES = RuntimeError('I fail half the time')
class TestFan(unittest.TestCase):
mode = 'static'
def setUp(self):
self.pool = IntPool(max_size=2)
def test_with_list(self):
list_of_input = ['agent-one', 'agent-two', 'agent-three']
def my_callable(pool_item, next_thing):
## Do some "blocking" (yielding) thing
api.sleep(0.01)
return next_thing
output = self.pool.fan(my_callable, list_of_input)
self.assertEquals(list_of_input, output)
def test_all_fail(self):
def my_failure(pool_item, next_thing):
raise ALWAYS
self.assertRaises(pools.AllFailed, self.pool.fan, my_failure, range(4))
def test_some_fail(self):
def my_failing_callable(pool_item, next_thing):
if next_thing % 2:
raise SOMETIMES
return next_thing
self.assertRaises(pools.SomeFailed, self.pool.fan, my_failing_callable, range(4))
if __name__ == '__main__':
unittest.main()

138
eventlet/processes.py Normal file
View File

@@ -0,0 +1,138 @@
"""\
@file processes.py
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import os
import popen2
import signal
from eventlet import util, pools
from eventlet import wrappedfd
class DeadProcess(RuntimeError):
pass
class Process(object):
process_number = 0
def __init__(self, command, args, dead_callback=lambda:None):
self.process_number = self.process_number + 1
Process.process_number = self.process_number
self.command = command
self.args = args
self._dead_callback = dead_callback
self.dead = False
self.started = False
self.popen4 = None
## We use popen4 so that read() will read from either stdout or stderr
self.popen4 = popen2.Popen4([self.command] + self.args)
child_stdout_stderr = self.popen4.fromchild
child_stdin = self.popen4.tochild
util.set_nonblocking(child_stdout_stderr)
util.set_nonblocking(child_stdin)
self.child_stdout_stderr = wrappedfd.wrapped_file(child_stdout_stderr)
self.child_stdout_stderr.newlines = '\n' # the default is \r\n, which aren't sent over pipes
self.child_stdin = wrappedfd.wrapped_file(child_stdin)
self.child_stdin.newlines = '\n'
self.sendall = self.child_stdin.write
self.send = self.child_stdin.write
self.recv = self.child_stdout_stderr.read
self.readline = self.child_stdout_stderr.readline
def dead_callback(self):
self.dead = True
if self._dead_callback:
self._dead_callback()
def makefile(self, mode, *arg):
if mode.startswith('r'):
return self.child_stdout_stderr
if mode.startswith('w'):
return self.child_stdin
raise RuntimeError("Unknown mode", mode)
def read(self, amount=None):
result = self.child_stdout_stderr.read(amount)
if result == '':
# This process is dead.
self.dead_callback()
raise DeadProcess
return result
def write(self, stuff):
written = self.child_stdin.send(stuff)
try:
self.child_stdin.flush()
except ValueError, e:
## File was closed
assert str(e) == 'I/O operation on closed file'
if written == 0:
self.dead_callback()
raise DeadProcess
def flush(self):
self.child_stdin.flush()
def close(self):
self.child_stdout_stderr.close()
self.child_stdin.close()
self.dead_callback()
def close_stdin(self):
self.child_stdin.close()
def kill(self, sig=None):
if sig == None:
sig = signal.SIGTERM
os.kill(self.popen4.pid, sig)
def getpid(self):
return self.popen4.pid
class ProcessPool(pools.Pool):
def __init__(self, command, args=None, min_size=0, max_size=4):
"""@param command the command to run
"""
self.command = command
if args is None:
args = []
self.args = args
pools.Pool.__init__(self, min_size, max_size)
def create(self):
"""Generate a process
"""
def dead_callback():
self.current_size -= 1
return Process(self.command, self.args, dead_callback)
def put(self, item):
if not item.dead:
if item.popen4.poll() != -1:
item.dead_callback()
else:
pools.Pool.put(self, item)

134
eventlet/processes_test.py Normal file
View File

@@ -0,0 +1,134 @@
"""\
@file processes_test.py
@author Donovan Preston, Aaron Brashears
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
from eventlet import tests
from eventlet import api
from eventlet import channel
from eventlet import processes
class TestEchoPool(tests.TestCase):
mode = 'static'
def setUp(self):
self.pool = processes.ProcessPool('echo', ["hello"])
def test_echo(self):
result = None
proc = self.pool.get()
try:
result = proc.read()
finally:
self.pool.put(proc)
self.assertEquals(result, 'hello\n')
def test_read_eof(self):
proc = self.pool.get()
try:
proc.read()
self.assertRaises(processes.DeadProcess, proc.read)
finally:
self.pool.put(proc)
class TestCatPool(tests.TestCase):
mode = 'static'
def setUp(self):
self.pool = processes.ProcessPool('cat')
def test_cat(self):
result = None
proc = self.pool.get()
try:
proc.write('goodbye')
proc.close_stdin()
result = proc.read()
finally:
self.pool.put(proc)
self.assertEquals(result, 'goodbye')
def test_write_to_dead(self):
result = None
proc = self.pool.get()
try:
proc.write('goodbye')
proc.close_stdin()
result = proc.read()
self.assertRaises(processes.DeadProcess, proc.write, 'foo')
finally:
self.pool.put(proc)
def test_close(self):
result = None
proc = self.pool.get()
try:
proc.write('hello')
proc.close()
self.assertRaises(processes.DeadProcess, proc.write, 'goodbye')
finally:
self.pool.put(proc)
class TestDyingProcessesLeavePool(tests.TestCase):
mode = 'static'
def setUp(self):
self.pool = processes.ProcessPool('echo', ['hello'], max_size=1)
def test_dead_process_not_inserted_into_pool(self):
proc = self.pool.get()
try:
result = proc.read()
self.assertEquals(result, 'hello\n')
finally:
self.pool.put(proc)
proc2 = self.pool.get()
self.assert_(proc is not proc2)
class TestProcessLivesForever(tests.TestCase):
mode = 'static'
def setUp(self):
self.pool = processes.ProcessPool('yes', max_size=1)
def test_reading_twice_from_same_process(self):
proc = self.pool.get()
try:
result = proc.read(2)
self.assertEquals(result, 'y\n')
finally:
self.pool.put(proc)
proc2 = self.pool.get()
self.assert_(proc is proc2)
try:
result = proc2.read(2)
self.assertEquals(result, 'y\n')
finally:
self.pool.put(proc2)
if __name__ == '__main__':
tests.main()

42
eventlet/pylibsupport.py Normal file
View File

@@ -0,0 +1,42 @@
"""\
@file pylibsupport.py
@author Donovan Preston
Copyright (c) 2005-2006, Donovan Preston
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
from py.magic import greenlet
import sys
import types
def emulate():
module = types.ModuleType('greenlet')
sys.modules['greenlet'] = module
module.greenlet = greenlet
module.getcurrent = greenlet.getcurrent
module.GreenletExit = greenlet.GreenletExit

200
eventlet/runloop.py Normal file
View File

@@ -0,0 +1,200 @@
"""\
@file runloop.py
@author Bob Ippolito
Defines the core eventlet runloop. The runloop keeps track of scheduled
events and observers which watch for specific portions of the runloop to
be executed.
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import time
import bisect
import sys
import traceback
from eventlet.timer import Timer
class RunLoop(object):
SYSTEM_EXCEPTIONS = (KeyboardInterrupt, SystemExit)
def __init__(self, wait=None, clock=None):
if clock is None:
clock = self.default_clock()
self.clock = clock
if wait is None:
wait = self.default_wait
self.wait = wait
self.stopping = False
self.running = False
self.timers = []
self.next_timers = []
self.observers = {}
self.observer_modes = {
'entry': [],
'before_timers': [],
'before_waiting': [],
'after_waiting': [],
'exit': [],
}
def default_wait(self, time):
return None
def default_clock(self):
return time.time
def default_sleep(self):
return 60.0
def sleep_until(self):
t = self.timers
if not t:
return None
return t[0][0]
def run(self):
"""Run the runloop until abort is called.
"""
if self.running:
raise RuntimeError("Already running!")
try:
self.running = True
self.stopping = False
self.fire_observers('entry')
while not self.stopping:
self.prepare_timers()
self.fire_observers('before_timers')
self.fire_timers(self.clock())
self.prepare_timers()
wakeup_when = self.sleep_until()
if wakeup_when is None:
sleep_time = self.default_sleep()
else:
sleep_time = wakeup_when - self.clock()
if sleep_time > 0:
self.fire_observers('before_waiting')
self.wait(sleep_time)
self.fire_observers('after_waiting')
else:
self.wait(0)
else:
del self.timers[:]
del self.next_timers[:]
self.fire_observers('exit')
finally:
self.running = False
self.stopping = False
def abort(self):
"""Stop the runloop. If run is executing, it will exit after completing
the next runloop iteration.
"""
if self.running:
self.stopping = True
def add_observer(self, observer, *modes):
"""Add an event observer to this runloop with the given modes.
Valid modes are:
entry: The runloop is being entered.
before_timers: Before the expired timers for this iteration are executed.
before_waiting: Before waiting for the calculated wait_time
where nothing will happen.
after_waiting: After waiting, immediately before starting the top of the
runloop again.
exit: The runloop is exiting.
If no mode is passed or mode is all, the observer will be fired for every
event type.
"""
if not modes or modes == ('all',):
modes = tuple(self.observer_modes)
self.observers[observer] = modes
for mode in modes:
self.observer_modes[mode].append(observer)
def remove_observer(self, observer):
"""Remove a previously registered observer from all event types.
"""
for mode in self.observers.pop(observer, ()):
self.observer_modes[mode].remove(observer)
def squelch_observer_exception(self, observer, exc_info):
traceback.print_exception(*exc_info)
print >>sys.stderr, "Removing observer: %r" % (observer,)
self.remove_observer(observer)
def fire_observers(self, activity):
for observer in self.observer_modes[activity]:
try:
observer(self, activity)
except self.SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_observer_exception(observer, sys.exc_info())
def squelch_timer_exception(self, timer, exc_info):
traceback.print_exception(*exc_info)
print >>sys.stderr, "Timer raised: %r" % (timer,)
def _add_absolute_timer(self, when, info):
# the 0 placeholder makes it easy to bisect_right using (now, 1)
self.next_timers.append((when, 0, info))
def add_timer(self, timer):
scheduled_time = self.clock() + timer.seconds
self._add_absolute_timer(scheduled_time, timer)
return scheduled_time
def prepare_timers(self):
ins = bisect.insort_right
t = self.timers
for item in self.next_timers:
ins(t, item)
del self.next_timers[:]
def schedule_call(self, seconds, cb, *args, **kw):
"""Schedule a callable to be called after 'seconds' seconds have
elapsed.
seconds: The number of seconds to wait.
cb: The callable to call after the given time.
*args: Arguments to pass to the callable when called.
**kw: Keyword arguments to pass to the callable when called.
"""
t = Timer(seconds, cb, *args, **kw)
self.add_timer(t)
return t
def fire_timers(self, when):
t = self.timers
last = bisect.bisect_right(t, (when, 1))
i = 0
for i in xrange(last):
timer = t[i][2]
try:
timer()
except self.SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_timer_exception(timer, sys.exc_info())
del t[:last]

157
eventlet/runloop_test.py Normal file
View File

@@ -0,0 +1,157 @@
"""\
@file test_runloop.py
@author Donovan Preston
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
import time
import StringIO
import unittest
from eventlet import runloop
class TestRunloop(unittest.TestCase):
mode = 'static'
def test_empty(self):
r = runloop.RunLoop()
r.schedule_call(0, r.abort)
r.run()
assert not r.running
def test_timer(self):
r = runloop.RunLoop()
r.schedule_call(0.125, r.abort)
start_time = time.time()
r.run()
assert time.time() - start_time >= 0.125
assert not r.running
def test_observer(self):
observed = []
r = runloop.RunLoop()
r.add_observer(lambda runloop, activity: observed.append(activity))
r.schedule_call(0, r.abort)
r.run()
assert observed == ['entry', 'before_timers', 'before_waiting', 'after_waiting', 'exit']
assert not r.running
def test_remove_observer(self):
r = runloop.RunLoop()
observed = []
def observe(runloop, mode):
observed.append(mode)
r.remove_observer(observe)
looped = []
def run_loop_twice(runloop, mode):
if looped:
r.abort()
else:
looped.append(True)
r.add_observer(observe, 'before_timers')
r.add_observer(run_loop_twice, 'after_waiting')
r.run()
assert len(observed) == 1
assert not r.running
def test_observer_exception(self):
r = runloop.RunLoop()
observed = []
def observe(runloop, mode):
observed.append(mode)
raise Exception("Squelch me please")
looped = []
def run_loop_twice(runloop, mode):
if looped:
r.abort()
else:
looped.append(True)
saved = sys.stderr
sys.stderr = err = StringIO.StringIO()
r.add_observer(observe, 'before_timers')
r.add_observer(run_loop_twice, 'after_waiting')
r.run()
err.seek(0)
sys.stderr = saved
assert len(observed) == 1
assert err.read()
assert not r.running
def test_timer_exception(self):
r = runloop.RunLoop()
observed = []
def timer():
observed.append(True)
raise Exception("Squelch me please")
looped = []
def run_loop_twice(runloop, mode):
if looped:
r.abort()
else:
looped.append(True)
saved = sys.stderr
sys.stderr = err = StringIO.StringIO()
r.schedule_call(0, timer)
r.add_observer(run_loop_twice, 'after_waiting')
r.run()
err.seek(0)
sys.stderr = saved
assert len(observed) == 1
assert err.read()
assert not r.running
def test_timer_system_exception(self):
r = runloop.RunLoop()
def timer():
raise SystemExit
r.schedule_call(0, timer)
caught = []
try:
r.run()
except SystemExit:
caught.append(True)
assert caught
assert not r.running
if __name__ == '__main__':
unittest.main()

173
eventlet/selecthub.py Normal file
View File

@@ -0,0 +1,173 @@
"""\
@file selecthub.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
import select
import errno
import traceback
import time
from bisect import insort, bisect_left
from eventlet import greenlib
from eventlet.runloop import RunLoop, Timer
import greenlet
class Hub(object):
def __init__(self):
self.runloop = RunLoop(self.wait)
self.readers = {}
self.writers = {}
self.excs = {}
self.descriptors = {}
self.descriptor_queue = {}
self.greenlet = None
def stop(self):
self.process_queue()
self.runloop.abort()
if self.greenlet is not greenlet.getcurrent():
self.switch()
def schedule_call(self, *args, **kw):
return self.runloop.schedule_call(*args, **kw)
def switch(self):
if not self.greenlet:
self.greenlet = greenlib.tracked_greenlet()
args = ((self.runloop.run,),)
else:
args = ()
try:
greenlet.getcurrent().parent = self.greenlet
except ValueError:
pass
return greenlib.switch(self.greenlet, *args)
def add_descriptor(self, fileno, read=None, write=None, exc=None):
self.descriptor_queue[fileno] = read, write, exc
def remove_descriptor(self, fileno):
self.descriptor_queue[fileno] = None, None, None
def exc_descriptor(self, fileno):
# We must handle two cases here, the descriptor
# may be changing or removing its exc handler
# in the queue, or it may be waiting on the queue.
exc = None
try:
exc = self.descriptor_queue[fileno][2]
except KeyError:
try:
exc = self.excs[fileno]
except KeyError:
pass
if exc is not None:
try:
exc()
except self.runloop.SYSTEM_EXCEPTIONS:
self.squelch_exception(fileno, sys.exc_info())
def squelch_exception(self, fileno, exc_info):
traceback.print_exception(*exc_info)
print >>sys.stderr, "Removing descriptor: %r" % (fileno,)
try:
self.remove_descriptor(fileno)
except Exception, e:
print >>sys.stderr, "Exception while removing descriptor! %r" % (e,)
def process_queue(self):
readers = self.readers
writers = self.writers
excs = self.excs
descriptors = self.descriptors
for fileno, rwe in self.descriptor_queue.iteritems():
read, write, exc = rwe
if read is None and write is None and exc is None:
try:
del descriptors[fileno]
except KeyError:
continue
try:
del readers[fileno]
except KeyError:
pass
try:
del writers[fileno]
except KeyError:
pass
try:
del excs[fileno]
except KeyError:
pass
else:
if read is not None:
readers[fileno] = read
else:
try:
del readers[fileno]
except KeyError:
pass
if write is not None:
writers[fileno] = write
else:
try:
del writers[fileno]
except KeyError:
pass
if exc is not None:
excs[fileno] = exc
else:
try:
del excs[fileno]
except KeyError:
pass
descriptors[fileno] = rwe
self.descriptor_queue.clear()
def wait(self, seconds=None):
self.process_queue()
if not self.descriptors:
if seconds:
time.sleep(seconds)
return
readers = self.readers
writers = self.writers
excs = self.excs
try:
r, w, ig = select.select(readers, writers, [], seconds)
except select.error, e:
if e.args[0] == errno.EINTR:
return
raise
SYSTEM_EXCEPTIONS = self.runloop.SYSTEM_EXCEPTIONS
for observed, events in ((readers, r), (writers, w)):
for fileno in events:
try:
observed[fileno](fileno)
except SYSTEM_EXCEPTIONS:
raise
except:
self.squelch_exception(fileno, sys.exc_info())

View File

@@ -0,0 +1,110 @@
"""\
@file __init__.py
@brief Support for using stackless python. Broken and riddled with
print statements at the moment. Please fix it!
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
import types
import stackless
import traceback
caller = None
coro_args = {}
tasklet_to_greenlet = {}
def getcurrent():
return tasklet_to_greenlet[stackless.getcurrent()]
class FirstSwitch(object):
def __init__(self, gr):
self.gr = gr
def __call__(self, *args, **kw):
print "first call", args, kw
gr = self.gr
del gr.switch
run, gr.run = gr.run, None
t = stackless.tasklet(run)
gr.t = t
tasklet_to_greenlet[t] = gr
t.setup(*args, **kw)
result = t.run()
class greenlet(object):
def __init__(self, run=None, parent=None):
self.dead = False
if parent is None:
parent = getcurrent()
self.parent = parent
if run is not None:
self.run = run
self.switch = FirstSwitch(self)
def switch(self, *args):
print "switch", args
global caller
caller = stackless.getcurrent()
coro_args[self] = args
self.t.insert()
stackless.schedule()
if caller is not self.t:
caller.remove()
rval = coro_args[self]
return rval
def run(self):
pass
def __bool__(self):
return self.run is None and not self.dead
class GreenletExit(Exception):
pass
def emulate():
module = types.ModuleType('greenlet')
sys.modules['greenlet'] = module
module.greenlet = greenlet
module.getcurrent = getcurrent
module.GreenletExit = GreenletExit
caller = t = stackless.getcurrent()
tasklet_to_greenlet[t] = None
main_coro = greenlet()
tasklet_to_greenlet[t] = main_coro
main_coro.t = t
del main_coro.switch ## It's already running
coro_args[main_coro] = None

36
eventlet/tests.py Normal file
View File

@@ -0,0 +1,36 @@
"""\
@file tests.py
@author Donovan Preston
@brief Indirection layer for tests in case we want to fix unittest.
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import atexit
import sys
import unittest
TestCase = unittest.TestCase
name = getattr(sys.modules['__main__'], '__name__', None)
main = unittest.main

72
eventlet/timer.py Normal file
View File

@@ -0,0 +1,72 @@
"""\
@file timer.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
from eventlet.api import get_hub
class Timer(object):
__slots__ = ['seconds', 'tpl', 'called', 'cancelled', 'scheduled_time']
def __init__(self, seconds, cb, *args, **kw):
"""Create a timer.
seconds: The minimum number of seconds to wait before calling
cb: The callback to call when the timer has expired
*args: The arguments to pass to cb
**kw: The keyword arguments to pass to cb
This timer will not be run unless it is scheduled in a runloop by
calling timer.schedule() or runloop.add_timer(timer).
"""
self.cancelled = False
self.seconds = seconds
self.tpl = cb, args, kw
self.called = False
def __repr__(self):
secs = getattr(self, 'seconds', None)
cb, args, kw = getattr(self, 'tpl', (None, None, None))
return "Timer(%s, %s, *%s, **%s)" % (
secs, cb, args, kw)
def copy(self):
cb, args, kw = self.tpl
return self.__class__(self.seconds, cb, *args, **kw)
def schedule(self):
"""Schedule this timer to run in the current runloop.
"""
self.called = False
self.scheduled_time = get_hub().runloop.add_timer(self)
return self
def __call__(self):
if not self.called:
self.called = True
cb, args, kw = self.tpl
cb(*args, **kw)
def cancel(self):
"""Prevent this timer from being called. If the timer has already
been called, has no effect.
"""
self.cancelled = True
self.called = True

66
eventlet/timer_test.py Normal file
View File

@@ -0,0 +1,66 @@
"""\
@file test_timer.py
@author Donovan Preston
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import unittest
from eventlet import api, runloop, tests, timer
class TestTimer(tests.TestCase):
mode = 'static'
def test_copy(self):
t = timer.Timer(0, lambda: None)
t2 = t.copy()
assert t.seconds == t2.seconds
assert t.tpl == t2.tpl
assert t.called == t2.called
def test_cancel(self):
r = runloop.RunLoop()
called = []
t = timer.Timer(0, lambda: called.append(True))
t.cancel()
r.add_timer(t)
r.add_observer(lambda r, activity: r.abort(), 'after_waiting')
r.run()
assert not called
assert not r.running
def test_schedule(self):
hub = api.get_hub()
r = hub.runloop
# clean up the runloop, preventing side effects from previous tests
# on this thread
if r.running:
r.abort()
api.sleep(0)
called = []
t = timer.Timer(0, lambda: (called.append(True), hub.runloop.abort()))
t.schedule()
r.default_sleep = lambda: 0.0
r.run()
assert called
assert not r.running
if __name__ == '__main__':
unittest.main()

57
eventlet/tls.py Normal file
View File

@@ -0,0 +1,57 @@
"""\
@file tls.py
@author Donovan Preston
Copyright (c) 2006-2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import threading
import weakref
__all__ = ['local']
class _local(object):
"""
Crappy Python 2.3 compatible implementation of thread-local storage
"""
__slots__ = ('__thread_dict__',)
def __init__(self):
object.__setattr__(self, '__thread_dict__', weakref.WeakKeyDictionary())
def __getattr__(self, attr):
try:
return self.__thread_dict__[threading.currentThread()][attr]
except KeyError:
raise AttributeError(attr)
def __setattr__(self, attr, value):
t = threading.currentThread()
try:
d = self.__thread_dict__[t]
except KeyError:
d = self.__thread_dict__[t] = {}
d[attr] = value
try:
local = threading.local
except AttributeError:
local = _local

134
eventlet/twistedsupport.py Normal file
View File

@@ -0,0 +1,134 @@
"""\
@file twistedsupport.py
@author Donovan Preston
Copyright (c) 2005-2006, Donovan Preston
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
import traceback
from eventlet import api
from eventlet import timer
from twisted.internet import posixbase
from twisted.internet.interfaces import IReactorFDSet
try:
from zope.interface import implements
_working = True
except ImportError:
_working = False
def implements(*args, **kw):
pass
class TwistedTimer(object):
def __init__(self, timer):
self.timer = timer
def cancel(self):
self.timer.cancel()
def getTime(self):
return self.timer.scheduled_time
def delay(self, seconds):
hub = api.get_hub()
new_time = hub.clock() - self.timer_scheduled_time + seconds
self.timer.cancel()
cb, args, kw = self.timer.tpl
self.timer = hub.schedule_call(new_time, cb, *args, **kw)
def reset(self, new_time):
self.timer.cancel()
cb, args, kw = self.timer.tpl
self.timer = api.get_hub().schedule_call(new_time, cb, *args, **kw)
def active(self):
return not self.timer.called
class EventletReactor(posixbase.PosixReactorBase):
implements(IReactorFDSet)
def __init__(self, *args, **kw):
self._readers = {}
self._writers = {}
posixbase.PosixReactorBase.__init__(self, *args, **kw)
def callLater(self, func, *args, **kw):
return TwistedTimer(api.call_after(func, *args, **kw))
def run(self):
self.running = True
self._stopper = api.call_after(sys.maxint / 1000.0, lambda: None)
## schedule a call way in the future, and cancel it in stop?
api.get_hub().runloop.run()
def stop(self):
self._stopper.cancel()
posixbase.PosixReactorBase.stop(self)
api.get_hub().remove_descriptor(self._readers.keys()[0])
api.get_hub().runloop.abort()
def addReader(self, reader):
fileno = reader.fileno()
self._readers[fileno] = reader
api.get_hub().add_descriptor(fileno, read=self._got_read)
def _got_read(self, fileno):
self._readers[fileno].doRead()
def addWriter(self, writer):
fileno = writer.fileno()
self._writers[fileno] = writer
api.get_hub().add_descriptor(fileno, write=self._got_write)
def _got_write(self, fileno):
self._writers[fileno].doWrite()
def removeReader(self, reader):
fileno = reader.fileno()
if fileno in self._readers:
self._readers.pop(fileno)
api.get_hub().remove_descriptor(fileno)
def removeWriter(self, writer):
fileno = writer.fileno()
if fileno in self._writers:
self._writers.pop(fileno)
api.get_hub().remove_descriptor(fileno)
def removeAll(self):
return self._removeAll(self._readers.values(), self._writers.values())
def emulate():
if not _working:
raise RuntimeError, "Can't use twistedsupport because zope.interface is not installed."
reactor = EventletReactor()
from twisted.internet.main import installReactor
installReactor(reactor)
__all__ = ['emulate']

214
eventlet/util.py Normal file
View File

@@ -0,0 +1,214 @@
"""\
@file util.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import os
import fcntl
import socket
import errno
from errno import EWOULDBLOCK, EAGAIN
try:
from OpenSSL import SSL
except ImportError:
class SSL(object):
class WantWriteError(object):
pass
class WantReadError(object):
pass
class ZeroReturnError(object):
pass
class SysCallError(object):
pass
def g_log(*args):
import sys
import greenlet
from eventlet.greenlib import greenlet_id
g_id = greenlet_id()
if g_id is None:
if greenlet.getcurrent().parent is None:
ident = 'greenlet-main'
else:
g_id = id(greenlet.getcurrent())
if g_id < 0:
g_id += 1 + ((sys.maxint + 1) << 1)
ident = '%08X' % (g_id,)
else:
ident = 'greenlet-%d' % (g_id,)
print >>sys.stderr, '[%s] %s' % (ident, ' '.join(map(str, args)))
CONNECT_ERR = (errno.EINPROGRESS, errno.EALREADY, EWOULDBLOCK)
CONNECT_SUCCESS = (0, errno.EISCONN)
def socket_connect(descriptor, address):
err = descriptor.connect_ex(address)
if err in CONNECT_ERR:
return None
if err not in CONNECT_SUCCESS:
raise socket.error(err, errno.errorcode[err])
return descriptor
__original_socket__ = socket.socket
def tcp_socket():
s = __original_socket__(socket.AF_INET, socket.SOCK_STREAM)
set_nonblocking(s)
return s
__original_ssl__ = socket.ssl
def wrap_ssl(sock, certificate=None, private_key=None):
from OpenSSL import SSL
from eventlet import wrappedfd, util
context = SSL.Context(SSL.SSLv23_METHOD)
print certificate, private_key
if certificate is not None:
context.use_certificate_file(certificate)
if private_key is not None:
context.use_privatekey_file(private_key)
context.set_verify(SSL.VERIFY_NONE, lambda *x: True)
## TODO only do this on client sockets? how?
connection = SSL.Connection(context, sock)
connection.set_connect_state()
return wrappedfd.wrapped_fd(connection)
def wrap_socket_with_coroutine_socket():
def new_socket(*args, **kw):
from eventlet import wrappedfd
s = __original_socket__(*args, **kw)
set_nonblocking(s)
return wrappedfd.wrapped_fd(s)
socket.socket = new_socket
socket.ssl = wrap_ssl
def socket_bind_and_listen(descriptor, addr=('', 0), backlog=50):
set_reuse_addr(descriptor)
descriptor.bind(addr)
descriptor.listen(backlog)
return descriptor
def socket_accept(descriptor):
try:
return descriptor.accept()
except socket.error, e:
if e[0] == EWOULDBLOCK:
return None
raise
def socket_send(descriptor, data):
try:
return descriptor.send(data)
except socket.error, e:
if e[0] == EWOULDBLOCK:
return 0
raise
except SSL.WantWriteError:
return 0
except SSL.WantReadError:
return 0
# winsock sometimes throws ENOTCONN
SOCKET_CLOSED = (errno.ECONNRESET, errno.ENOTCONN, errno.ESHUTDOWN)
def socket_recv(descriptor, buflen):
try:
return descriptor.recv(buflen)
except socket.error, e:
if e[0] == EWOULDBLOCK:
return None
if e[0] in SOCKET_CLOSED:
return ''
raise
except SSL.WantReadError:
return None
except SSL.ZeroReturnError:
return ''
except SSL.SysCallError, e:
if e[0] == -1 or e[0] > 0:
raise socket.error(errno.ECONNRESET, errno.errorcode[errno.ECONNRESET])
raise
def file_recv(fd, buflen):
try:
return fd.read(buflen)
except IOError, e:
if e[0] == EAGAIN:
return None
return ''
except socket.error, e:
if e[0] == errno.EPIPE:
return ''
raise
def file_send(fd, data):
try:
fd.write(data)
fd.flush()
return len(data)
except IOError, e:
if e[0] == EAGAIN:
return 0
except ValueError, e:
written = 0
except socket.error, e:
if e[0] == errno.EPIPE:
written = 0
def set_reuse_addr(descriptor):
try:
descriptor.setsockopt(
socket.SOL_SOCKET,
socket.SO_REUSEADDR,
descriptor.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR) | 1,
)
except socket.error:
pass
def set_nonblocking(descriptor):
if hasattr(descriptor, 'setblocking'):
# socket
descriptor.setblocking(0)
else:
# fd
if hasattr(descriptor, 'fileno'):
fd = descriptor.fileno()
else:
fd = descriptor
flags = fcntl.fcntl(fd, fcntl.F_GETFL)
fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
return descriptor

262
eventlet/wrappedfd.py Normal file
View File

@@ -0,0 +1,262 @@
"""\
@file wrappedfd.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
from eventlet.api import trampoline, get_hub
from eventlet import util
BUFFER_SIZE = 4096
import socket, errno
def higher_order_recv(recv_func):
def recv(self, buflen):
buf = self.recvbuffer
if buf:
chunk, self.recvbuffer = buf[:buflen], buf[buflen:]
return chunk
fd = self.fd
bytes = recv_func(fd, buflen)
while bytes is None:
try:
trampoline(fd, read=True)
except socket.error, e:
if e[0] == errno.EPIPE:
bytes = ''
else:
raise
else:
bytes = recv_func(fd, buflen)
self.recvcount += len(bytes)
return bytes
return recv
def higher_order_send(send_func):
def send(self, data):
count = send_func(self.fd, data)
if not count:
return 0
self.sendcount += count
return count
return send
class wrapped_fd(object):
newlines = '\r\n'
mode = 'wb+'
is_secure = False
def __init__(self, fd):
self._closed = False
self.fd = fd
self._fileno = fd.fileno()
self.recvbuffer = ''
self.recvcount = 0
self.sendcount = 0
def getpeername(self, *args, **kw):
fn = self.getpeername = self.fd.getpeername
return fn(*args, **kw)
def getsockname(self, *args, **kw):
fn = self.getsockname = self.fd.getsockname
return fn(*args, **kw)
def listen(self, *args, **kw):
fn = self.listen = self.fd.listen
return fn(*args, **kw)
def bind(self, *args, **kw):
fn = self.bind = self.fd.bind
return fn(*args, **kw)
def getsockopt(self, *args, **kw):
fn = self.getsockopt = self.fd.getsockopt
return fn(*args, **kw)
def setsockopt(self, *args, **kw):
fn = self.setsockopt = self.fd.setsockopt
return fn(*args, **kw)
def connect_ex(self, *args, **kw):
fn = self.connect_ex = self.fd.connect_ex
return fn(*args, **kw)
def fileno(self, *args, **kw):
fn = self.fileno = self.fd.fileno
return fn(*args, **kw)
def setblocking(self, *args, **kw):
fn = self.setblocking = self.fd.setblocking
return fn(*args, **kw)
def close(self, *args, **kw):
if self._closed:
return
self._closed = True
fn = self.close = self.fd.close
try:
res = fn(*args, **kw)
finally:
# This will raise socket.error(32, 'Broken pipe') if there's
# a caller waiting on trampoline (e.g. server on .accept())
get_hub().exc_descriptor(self._fileno)
return res
def accept(self):
fd = self.fd
while True:
res = util.socket_accept(fd)
if res is not None:
client, addr = res
util.set_nonblocking(client)
return type(self)(client), addr
trampoline(fd, read=True, write=True)
def connect(self, address):
fd = self.fd
connect = util.socket_connect
while not connect(fd, address):
trampoline(fd, read=True, write=True)
recv = higher_order_recv(util.socket_recv)
def recvfrom(self, *args):
trampoline(self.fd, read=True)
return self.fd.recvfrom(*args)
send = higher_order_send(util.socket_send)
def sendto(self, *args):
trampoline(self.fd, write=True)
return self.fd.sendto(*args)
def sendall(self, data):
fd = self.fd
tail = self.send(data)
while tail < len(data):
trampoline(self.fd, write=True)
tail += self.send(data[tail:])
def write(self, data):
return self.sendall(data)
def readuntil(self, terminator, size=None):
buf, self.recvbuffer = self.recvbuffer, ''
checked = 0
if size is None:
while True:
found = buf.find(terminator, checked)
if found != -1:
found += len(terminator)
chunk, self.recvbuffer = buf[:found], buf[found:]
return chunk
checked = len(buf)
d = self.recv(BUFFER_SIZE)
if not d:
break
buf += d
return buf
while len(buf) < size:
found = buf.find(terminator, checked)
if found != -1:
found += len(terminator)
chunk, self.recvbuffer = buf[:found], buf[found:]
return chunk
checked = len(buf)
d = self.recv(BUFFER_SIZE)
if not d:
break
buf += d
chunk, self.recvbuffer = buf[:size], buf[size:]
def readline(self, size=None):
return self.readuntil(self.newlines, size=size)
def __iter__(self):
return self.xreadlines()
def readlines(self, size=None):
return list(self.xreadlines(size=size))
def xreadlines(self, size=None):
if size is None:
while True:
line = self.readline()
if not line:
break
yield line
else:
while size > 0:
line = self.readline(size)
if not line:
break
yield line
size -= len(line)
def writelines(self, lines):
for line in lines:
self.write(line)
def read(self, size=None):
if size is not None and not isinstance(size, int):
raise TypeError
buf, self.recvbuffer = self.recvbuffer, ''
lst = [buf]
if size is None:
while True:
d = self.recv(BUFFER_SIZE)
if not d:
break
lst.append(d)
else:
buflen = len(buf)
while buflen < size:
d = self.recv(BUFFER_SIZE)
if not d:
break
buflen += len(d)
lst.append(d)
else:
d = lst[-1]
overbite = buflen - size
if overbite:
lst[-1], self.recvbuffer = d[:-overbite], d[-overbite:]
else:
lst[-1], self.recvbuffer = d, ''
return ''.join(lst)
def makefile(self, *args, **kw):
return type(self)(self.fd)
class wrapped_file(wrapped_fd):
recv = higher_order_recv(util.file_recv)
send = higher_order_send(util.file_send)
def flush(self):
self.fd.flush()

219
eventlet/wsgi.py Normal file
View File

@@ -0,0 +1,219 @@
"""\
@file wsgi.py
@author Bob Ippolito
Copyright (c) 2005-2006, Bob Ippolito
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
import sys
import time
import urllib
import socket
import cStringIO
import SocketServer
import BaseHTTPServer
from eventlet import api
from eventlet.httpdate import format_date_time
class HttpProtocol(BaseHTTPServer.BaseHTTPRequestHandler):
def log_message(self, format, *args):
self.server.log_message("%s - - [%s] %s" % (
self.address_string(),
self.log_date_time_string(),
format % args))
def handle_one_request(self):
self.raw_requestline = self.rfile.readline()
if not self.raw_requestline:
self.close_connection = 1
return
if not self.parse_request():
return
self.environ = self.get_environ()
try:
self.handle_one_response()
except socket.error, e:
# Broken pipe, connection reset by peer
if e[0] in (32, 54):
pass
else:
raise
def handle_one_response(self):
headers_set = []
headers_sent = []
# set of lowercase header names that were sent
header_dict = {}
wfile = self.wfile
num_blocks = None
def write(data, _write=wfile.write):
if not headers_set:
raise AssertionError("write() before start_response()")
elif not headers_sent:
status, response_headers = headers_set
headers_sent.append(1)
for k, v in response_headers:
header_dict[k.lower()] = k
_write('HTTP/1.0 %s\r\n' % status)
# send Date header?
if 'date' not in header_dict:
_write('Date: %s\r\n' % (format_date_time(time.time()),))
if 'content-length' not in header_dict and num_blocks == 1:
_write('Content-Length: %s\r\n' % (len(data),))
for header in response_headers:
_write('%s: %s\r\n' % header)
_write('\r\n')
_write(data)
def start_request(status, response_headers, exc_info=None):
if exc_info:
try:
if headers_sent:
# Re-raise original exception if headers sent
raise exc_info[0], exc_info[1], exc_info[2]
finally:
# Avoid dangling circular ref
exc_info = None
elif headers_set:
raise AssertionError("Headers already set!")
headers_set[:] = [status, response_headers]
return write
result = self.server.app(self.environ, start_request)
try:
num_blocks = len(result)
except (TypeError, AttributeError, NotImplementedError):
pass
try:
for data in result:
if data:
write(data)
if not headers_sent:
write('')
finally:
if hasattr(result, 'close'):
result.close()
def get_environ(self):
env = self.server.get_environ()
env['REQUEST_METHOD'] = self.command
env['SCRIPT_NAME'] = ''
if '?' in self.path:
path, query = self.path.split('?', 1)
else:
path, query = self.path, ''
env['PATH_INFO'] = urllib.unquote(path)
env['QUERY_STRING'] = query
if self.headers.typeheader is None:
env['CONTENT_TYPE'] = self.headers.type
else:
env['CONTENT_TYPE'] = self.headers.typeheader
length = self.headers.getheader('content-length')
if length:
env['CONTENT_LENGTH'] = length
env['SERVER_PROTOCOL'] = 'HTTP/1.0'
host, port = self.request.getsockname()
env['SERVER_NAME'] = host
env['SERVER_PORT'] = str(port)
env['REMOTE_ADDR'] = self.client_address[0]
env['GATEWAY_INTERFACE'] = 'CGI/1.1'
for h in self.headers.headers:
k, v = h.split(':', 1)
k = k.replace('-', '_').upper()
v = v.strip()
if k in env:
continue
envk = 'HTTP_' + k
if envk in env:
env[envk] += ',' + v
else:
env[envk] = v
return env
def finish(self):
# Override SocketServer.StreamRequestHandler.finish because
# we only need to call close on the socket, not the makefile'd things
self.request.close()
class Server(BaseHTTPServer.HTTPServer):
def __init__(self, socket, address, app, log, environ=None):
self.socket = socket
self.address = address
if log:
self.log = log
log.write = log.info
else:
self.log = sys.stderr
self.app = app
self.environ = environ
def get_environ(self):
socket = self.socket
d = {
'wsgi.input': socket,
'wsgi.errors': sys.stderr,
'wsgi.version': (1, 0),
'wsgi.multithread': True,
'wsgi.multiprocess': False,
'wsgi.run_once': False,
'wsgi.url_scheme': 'http',
}
if self.environ is not None:
d.update(self.environ)
return d
def process_request(self, (socket, address)):
proto = HttpProtocol(socket, address, self)
def log_message(self, message):
self.log.write(message + '\n')
def server(socket, site, log=None, environ=None):
serv = Server(socket, socket.getsockname(), site, log, environ=None)
try:
print "wsgi starting up on", socket.getsockname()
while True:
try:
api.spawn(serv.process_request, socket.accept())
except KeyboardInterrupt:
api.get_hub().remove_descriptor(socket.fileno())
print "wsgi exiting"
break
finally:
socket.close()

52
examples/echoserver.py Normal file
View File

@@ -0,0 +1,52 @@
"""\
@file echoserver.py
Simple server that listens on port 6000 and echos back every input to
the client. To try out the server, start it up by running this file.
Connect to it with:
telnet localhost 6000
You terminate your connection by terminating telnet (typically Ctrl-]
and then 'quit')
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
from eventlet import api
def handle_socket(client):
print "client connected"
while True:
# pass through every non-eof line
x = client.readline()
if not x: break
client.write(x)
print "echoed", x
print "client disconnected"
# server socket listening on port 6000
server = api.tcp_listener(('0.0.0.0', 6000))
while True:
new_sock, address = server.accept()
# handle every new connection with a new coroutine
api.spawn(handle_socket, new_sock)
server.close()

55
examples/webcrawler.py Normal file
View File

@@ -0,0 +1,55 @@
"""\
@file webcrawler.py
This is a simple web "crawler" that fetches a bunch of urls using a coroutine pool. It fetches as
many urls at time as coroutines in the pool.
Copyright (c) 2007, Linden Research, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
urls = ["http://www.google.com/intl/en_ALL/images/logo.gif",
"http://wiki.secondlife.com/w/images/secondlife.jpg",
"http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"]
import time
from eventlet import coros, httpc, util
# replace socket with a cooperative coroutine socket because httpc
# uses httplib, which uses socket. Removing this serializes the http
# requests, because the standard socket is blocking.
util.wrap_socket_with_coroutine_socket()
def fetch(url):
# we could do something interesting with the result, but this is
# example code, so we'll just report that we did it
print "%s fetching %s" % (time.asctime(), url)
httpc.get(url)
print "%s fetched %s" % (time.asctime(), url)
pool = coros.CoroutinePool(max_size=4)
waiters = []
for url in urls:
waiters.append(pool.execute(fetch, url))
# wait for all the coroutines to come back before exiting the process
for waiter in waiters:
waiter.wait()