Local write affinity for object PUT requests.

The proxy can now be configured to prefer local object servers for PUT
requests, where "local" is governed by the "write_affinity". The
"write_affinity_node_count" setting controls how many local object
servers to try before giving up and going on to remote ones.

I chose to simply re-order the object servers instead of filtering out
nonlocal ones so that, if all of the local ones are down, clients can
still get successful responses (just slower).

The goal is to trade availability for throughput. By writing to local
object servers across fast LAN links, clients get better throughput
than if the object servers were far away over slow WAN links. The
downside, of course, is that data availability (not durability) may
suffer when drives fail.

The default configuration has no write affinity in it, so the default
behavior is unchanged.

Added some words about these settings to the admin guide.

DocImpact

Change-Id: I09a0bd00524544ff627a3bccdcdc48f40720a86e

swift/proxy/server.py

@@ -35,7 +35,7 @@ from eventlet import Timeout
 from swift.common.ring import Ring
 from swift.common.utils import cache_from_env, get_logger, \
     get_remote_client, split_path, config_true_value, generate_trans_id, \
-    affinity_key_function
+    affinity_key_function, affinity_locality_predicate
 from swift.common.constraints import check_utf8
 from swift.proxy.controllers import AccountController, ObjectController, \
     ContainerController
@@ -133,6 +133,25 @@ class Application(object):
             # make the message a little more useful
             raise ValueError("Invalid read_affinity value: %r (%s)" %
                              (read_affinity, err.message))
+        try:
+            write_affinity = conf.get('write_affinity', '')
+            self.write_affinity_is_local_fn \
+                = affinity_locality_predicate(write_affinity)
+        except ValueError as err:
+            # make the message a little more useful
+            raise ValueError("Invalid write_affinity value: %r (%s)" %
+                             (write_affinity, err.message))
+        value = conf.get('write_affinity_node_count',
+                         '2 * replicas').lower().split()
+        if len(value) == 1:
+            value = int(value[0])
+            self.write_affinity_node_count = lambda r: value
+        elif len(value) == 3 and value[1] == '*' and value[2] == 'replicas':
+            value = int(value[0])
+            self.write_affinity_node_count = lambda r: value * r.replica_count
+        else:
+            raise ValueError(
+                'Invalid write_affinity_node_count value: %r' % ''.join(value))
 
     def get_controller(self, path):
         """