657ed505aa
The kubernetes project made an interesting and not universally loved decision[1] to fix an issue where kubectl port-forward could get stuck in an infinite loop on some errors by causing it to exit on any error, even benign ones such as a connection refused due to the target service not yet listening. This means that, in Zuul, if a playbook ever runs a command task in a job on a pod before the zuul_console task, the port-forward will crash and never recover (and so will not be available for subsequent tasks). "Don't do that" is one solution to this, but isn't particularly robust, and there could be other temporary issues that might similarly cause a disconnect. Further, it's conceivable that if another job starts a port-foward that happens to land on the same port, the first job, if it's still running, might end up streaming the logs from the second job. To avoid all of this, this change does the following: 1) It runs the port-forward in a shell with an infinite while loop. This essentially lets us outsource process monitoring to a shell subprocess rather than needing to dedicate an internal thread to restarting the port-forward on failure. 2) In order to kill the port-forward (which is now protected from signals by the shell), we start it in a process group and kill the whole group (as we do for Ansible). 3) Because the port used for each "host" (ie, pod) must be known before we start running Ansible, we can no longer let kubectl pick the ephemeral local port. Instead, we will bind a socket ourselves to reserve an ephemeral port number, and then provide that port number to kubectl. As long as we aren't listening on the socket, kubectl is able to create its own socket and bind it. 4) Because of #3, we take extra care to close the socket when finished. A test was written in order to verify the problem and solution described above, but running it in the gate is currently impractical since we have no real pod running. It is likely that we can add such a system, but that is left to future work. In the interim though, this change also adds all of the infrastructure that would be needed to run such a test, including new test-only environment variables to indicate how to connect to a pod (this mirrors the way we specify how to connect to a vm). [1] https://github.com/kubernetes/kubernetes/pull/103526 Change-Id: Iaa5ab61d6dabd08ad9f9e04d78b4d2110ce24c13
Zuul
Zuul is a project gating system.
The latest documentation for Zuul v3 is published at: https://zuul-ci.org/docs/zuul/
If you are looking for the Edge routing service named Zuul that is related to Netflix, it can be found here: https://github.com/Netflix/zuul
If you are looking for the Javascript testing tool named Zuul, it can be found here: https://github.com/defunctzombie/zuul
Getting Help
There are two Zuul-related mailing lists:
- zuul-announce
-
A low-traffic announcement-only list to which every Zuul operator or power-user should subscribe.
- zuul-discuss
-
General discussion about Zuul, including questions about how to use it, and future development.
You will also find Zuul developers on Matrix <https://matrix.to/#/#zuul:opendev.org>.
Contributing
To browse the latest code, see: https://opendev.org/zuul/zuul To clone the latest code, use git clone https://opendev.org/zuul/zuul
Bugs are handled at: https://storyboard.openstack.org/#!/project/zuul/zuul
Suspected security vulnerabilities are most appreciated if first reported privately following any of the supported mechanisms described at https://zuul-ci.org/docs/zuul/user/vulnerabilities.html
Code reviews are handled by gerrit at https://review.opendev.org
After creating a Gerrit account, use git review to submit patches. Example:
# Do your commits
$ git review
# Enter your username if promptedJoin us on Matrix to discuss development or usage.
License
Zuul is free software. Most of Zuul is licensed under the Apache License, version 2.0. Some parts of Zuul are licensed under the General Public License, version 3.0. Please see the license headers at the tops of individual source files.
Python Version Support
Zuul requires Python 3. It does not support Python 2.
Since Zuul uses Ansible to drive CI jobs, Zuul can run tests anywhere Ansible can, including Python 2 environments.