Make eventlet.tpool's thread count configurable in object server

If you're running servers_per_port > 0 and threads_per_disk = 0 (as it
should be with servers_per_port on), each object-server process will
have 20 IO threads waiting around to service eventlet.tpool
calls. This is far too many; with servers_per_port, there's no real
benefit to having so many IO threads.

This commit makes it so that, when servers_per_port > 0, each object
server defaults to having one main thread and one IO thread.

Also, eventlet's tpool size is now configurable via the object-server
config file. If a tpool size is set, that's what we'll use regardless
of servers_per_port. This allows operators with an excess of threads
to remove some regardless of servers_per_port.

Change-Id: I8f8914b7e70f2510393eb7c5e6be9708631ac027
Closes-Bug: 1554233

etc/object-server.conf-sample

@@ -124,6 +124,29 @@ use = egg:swift#object
 #
 # auto_create_account_prefix = .
 #
+
+# The number of threads in eventlet's thread pool. Most IO will occur
+# in the object server's main thread, but certain "heavy" IO
+# operations will occur in separate IO threads, managed by eventlet.
+#
+# The default value is auto, whose actual value is dependant on the
+# servers_per_port value:
+#
+#  - When servers_per_port is zero, the default value of
+#    eventlet_tpool_num_threads is empty, which uses eventlet's default
+#    (currently 20 threads).
+#
+#  - When servers_per_port is nonzero, the default value of
+#    eventlet_tpool_num_threads is 1.
+#
+# But you may override this value to any integer value.
+#
+# Note that this value is threads per object-server process, so to
+# compute the total number of IO threads on a node, you must multiply
+# this by the number of object-server processes on the node.
+#
+# eventlet_tpool_num_threads = auto
+
 # Configure parameter for creating specific server
 # To handle all verbs, including replication verbs, do not specify
 # "replication_server" (this is the default). To only handle replication,