This change fixes two endpoint-related races:
* We now shut down the aws driver delete thread rather than setting
it to daemon. This provides more predictable behavior in tests.
Some tests started launcher endpoints out-of-band (ie, they didn't
really use the launcher, but it still started and stopped outside
of the main action of the test). In order to ensure that every
endpoint that we start threads for is also correctly stopped,
standard stop/start methods are added to endpoints and locks are
used to ensure that the start/stop sequence is predictable.
This also lets us use endpoints in the scheduler/web without
starting background threads for them.
* It was possible to create multiple endpoints for a given region
if they were created around the same time since we did not use
a lock around the critical section of the cache. To correct that,
surrond the cache lookup with a lock and move the behavior to a
mixin class for reusability. (A cachetools decorator around
getEndpoint was considered but the provider object in the method
signature makes that difficult).
Change-Id: I5e8ca06e76fced1bb342250a953ecda576168874
James E. Blair
a14971778a