We've seen some issues related to the fact that on CNI DEL we wait for
pod annotation to appear in CNI registry:
1. Overloaded kuryr-daemon HTTP server, because many stale DEL requests
from CRI will saturate the number of free connections, causing CNI to
answer requests extremely slowly.
2. Some CRI's will not delete network namespace before getting a
successful CNI DEL. If we'll assign the same IP (or vlan) to a next
pod, we might end up with an IP or vlan conflict and NetlinkError.
This commit makes sure we only wait for VIFs up to 5 seconds on DEL
requests, enough time for watcher to populate registry on restart. Also
some code got moved around to make implementating the above simpler and
some logs got clarified to make debugging of such issues easier later
on.
Change-Id: I9221b6bc9166597837a4b53382862aa6c6f3e94c
Closes-Bug: 1882083
Related-Bug: 1854928
bd46075acd