From c30879be2e1acdcefe8819b6c69ec294320cb415 Mon Sep 17 00:00:00 2001 From: Shuquan Huang Date: Sun, 25 Oct 2015 10:57:44 +0800 Subject: [PATCH] Add troubleshooting docs for "no valid host found" In bare metal case this error happens much more often, we are adding a new troubleshooting document and add it as part of this new document. Change-Id: I155b5c7160fbefe61c6a7bcdc5aa51452c8b8ab7 Closes-bug: #1503735 --- doc/source/deploy/troubleshooting.rst | 72 +++++++++++++++++++++++++++ doc/source/index.rst | 1 + 2 files changed, 73 insertions(+) create mode 100644 doc/source/deploy/troubleshooting.rst diff --git a/doc/source/deploy/troubleshooting.rst b/doc/source/deploy/troubleshooting.rst new file mode 100644 index 0000000000..f548920c1e --- /dev/null +++ b/doc/source/deploy/troubleshooting.rst @@ -0,0 +1,72 @@ +.. _troubleshooting: + +====================== +Troubleshooting Ironic +====================== + +Nova returns "No valid host was found" Error +============================================ + +Sometimes Nova Conductor log file "nova-conductor.log" or a message returned +from Nova API contains the following error:: + + NoValidHost: No valid host was found. There are not enough hosts available. + +"No valid host was found" means that the Nova Scheduler could not find a bare +metal node suitable for booting the new instance. + +This in turn usually means some mismatch between resources that Nova expects +to find and resources that Ironic advertised to Nova. + +A few things should be checked in this case: + +#. Inspection should have succeeded for you before, or you should have + entered the required Ironic node properties manually. For each node with + available state in ``ironic node-list --provision-state available`` use + :: + + ironic node-show + + and make sure that ``properties`` JSON field has valid values for keys + ``cpus``, ``cpu_arch``, ``memory_mb`` and ``local_gb``. + +#. The Nova flavor that you are using does not match any properties of the + available Ironic nodes. Use + :: + + nova flavor-show + + to compare. If you're using exact match filters in Nova Scheduler, please + make sure the flavor and the node properties match exactly. Regarding + the extra specs in flavor, you should make sure they map to + ``node.properties['capabilities']``. + +#. Make sure that enough nodes are in ``available`` state according to + ``ironic node-list --provision-state available``. + +#. Make sure nodes you're going to deploy to are not in maintenance mode. + Again, use ``ironic node-list`` to check. A node automatically going to + maintenance mode usually means wrong power credentials for this node. Check + them and then remove maintenance mode:: + + ironic node-set-maintenance off + +#. After making changes to nodes in Ironic, it takes time for those changes + to propagate from Ironic to Nova. + Check that + :: + + nova hypervisor-stats + + correctly shows total amount of resources in your system. You can also + check ``nova hypervisor-list`` to see the status of individual Ironic + nodes as reported to Nova. And you can correlate the Nova "hypervisor + hostname" to the Ironic node UUID. + +#. If none of the above helped, check Ironic conductor log carefully to see + if there are any conductor-related errors which are the root cause for + "No valid host was found". If there are any "Error in deploy of node + : [Errno 28] ..." error messages in Ironic conductor + log, it means the conductor run into a special error during deployment. + So you can check the log carefully to fix or work around and then try + again. diff --git a/doc/source/index.rst b/doc/source/index.rst index 8ad6c3aca3..412cdc9290 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -41,6 +41,7 @@ Overview Configuration Reference (Kilo) deploy/drivers deploy/cleaning + deploy/troubleshooting Commands --------