We need to tune the default rabbitmq tcp listen backlog. Currently it defaults
to 128, but here's what happens:
Say we have 1500 total rabbitmq client connections spread across a 3 node
cluster, evenly distributed so each node has 500 clients.
Then, we stop rabbitmq on one of the nodes.
Now those 500 client connections all immediately fail over to the other two
node. Assume roughly even split, and each gets 250 connections simultaneously.
Since the tcp listen backlog is only 128, a large number of the failover
connections cannot connect and get ECONNREFUSED because the kernel just drops
Eventually things retry and the backlog clears, but it just makes things noisy
in the logs and makes failover take a little bit longer.
Upstream docs discuss here:
Suggested-By: John Eckersberg <email@example.com>
(cherry picked from commit