Avoid parallel discover_hosts (nova-related race condition)

In a rare event both kolla-ansible and nova-scheduler try to do
the mapping at the same time and one of them fails.
Since kolla-ansible runs host discovery on each deployment,
there is no need to change the default of no periodic host discovery.

I added some notes for future. They are not critical.
I made the decision explicit in the comments.
I changed the task name to satisfy recommendations.
I removed the variable because it is not used (to avoid future doubts).

Closes-Bug: #1832987

Change-Id: I3128472f028a2dbd7ace02abc179a9629ad74ceb
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
This commit is contained in:
Radosław Piliszek 2019-06-16 20:37:35 +02:00
parent cb0cb441c2
commit ce680bcfe2
2 changed files with 7 additions and 3 deletions

View File

@ -23,15 +23,16 @@
- nova_compute_services is success
- nova_compute_services.stdout | from_json | length != 0
- name: Discovering nova hosts
# TODO(yoctozepto): no need to do --by-service if ironic not used
- name: Discover nova hosts
become: true
command: >
docker exec nova_api nova-manage cell_v2 discover_hosts --by-service
register: discover_hosts
changed_when: False
run_once: True
delegate_to: "{{ groups['nova-api'][0] }}"
# NOTE(yoctozepto): SIGHUP is probably unnecessary
- name: Refresh cell cache in nova scheduler
become: true
command: docker kill --signal HUP nova_scheduler

View File

@ -254,7 +254,10 @@ secure_proxy_ssl_header = HTTP_X_FORWARDED_PROTO
[scheduler]
max_attempts = 10
discover_hosts_in_cells_interval = 60
# NOTE(yoctozepto): kolla-ansible handles cell mapping by itself on each deploy
# periodic run must be disabled to avoid random failures (where both try to map)
# -1 is default and means periodic discovery is disabled
discover_hosts_in_cells_interval = -1
{% if enable_nova_fake | bool %}
default_filters = RetryFilter,AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter