Benjamin Schanzel 15b8bf7466 Metastatic: Handle lost backing nodes
This change makes the metastatic driver handle cases where it has
backing nodes in its in-memory records that do not exist anymore.

This can happen when there are two equal metastatic providers running in
parallel [1], where one of them removes a backing node (e.g. when it
reached its max-grace-time) but the other one not knowing about this
because each has their own in-memory records.

Until now, this leads to unrecoverable errors, because the metastatic
driver keeps trying to load node data from zk for a backing node that is
not existing. Instead, when we encounter such a node, delete it from the
backing node records and try again (creating a new backing node).

[1] This usually happens on rolling restarts, where one launcher
instance is kept running until a new one is ready to take over the work.

Change-Id: I9a92b1fbe23e4627eca0354080165ab06f19f6d2
2026-03-10 15:25:46 +01:00
2025-12-04 07:27:14 -08:00
2018-04-26 11:52:17 -05:00
2021-02-19 18:42:33 +00:00
2019-04-19 19:25:31 +00:00
2018-04-26 11:52:17 -05:00
2024-08-26 14:29:33 -07:00
2024-04-17 11:31:43 +02:00
2018-01-19 00:30:22 +00:00
2023-01-10 10:30:14 -08:00
2026-01-15 13:38:24 +01:00
2025-06-13 10:46:02 -07:00
2015-09-14 16:19:13 -04:00
2026-02-23 11:04:40 +01:00
2023-01-10 10:30:14 -08:00

Nodepool

Nodepool is a system for managing test node resources. It supports launching single-use test nodes from cloud providers as well as managing access to pre-defined pre-existing nodes. Nodepool is part of a suite of tools that form a comprehensive test system, including Zuul.

The latest documentation for Nodepool is published at: https://zuul-ci.org/docs/nodepool/

The latest documentation for Zuul is published at: https://zuul-ci.org/docs/zuul/

Getting Help

There are two Zuul-related mailing lists:

zuul-announce

A low-traffic announcement-only list to which every Zuul operator or power-user should subscribe.

zuul-discuss

General discussion about Zuul, including questions about how to use it, and future development.

You will also find Zuul developers in the #zuul channel on Freenode IRC.

Contributing

To browse the latest code, see: https://opendev.org/zuul/nodepool To clone the latest code, use git clone https://opendev.org/zuul/nodepool

Bugs are handled at: https://storyboard.openstack.org/#!/project/zuul/nodepool

Code reviews are handled by gerrit at https://review.opendev.org

After creating a Gerrit account, use git review to submit patches. Example:

# Do your commits
$ git review
# Enter your username if prompted

Join #zuul on Freenode to discuss development or usage.

License

Nodepool is free software, licensed under the Apache License, version 2.0.

Python Version Support

Nodepool requires Python 3. It does not support Python 2.

Description
Manage a pool of nodes for a distributed test infrastructure
Readme 36 MiB
Languages
Python 98.3%
Shell 1.1%
Dockerfile 0.3%
Jinja 0.3%