Via 31cfdbd407
oslo.rootwrap gained support eventlet when using subprocess. By moving
to oslo_rootwrap.subprocess we make sure that with python3 the
subprocess calls use eventlet.green.subprocess if eventlet is used.
This worked on python2 because (from above commit):
"""
On Python 2, it "works" to use directly subprocess: subprocess.Popen
calls os.pipe() and os.fdopen(fd) which are both monkey-patched. On
Python 3, it doesn't work because subprocess uses os.pipe() and
io.open(fd), and the io module is *not* monkey-patched at all.
"""
By applying this change what happens is that the heartbeat thread is
able to be scheduled every 15seconds by default. Without this patch
what we have been observing with python3 is that while running ansible
mistral would constantly log error messages like the following:
2019-05-02 19:14:36.702 8 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: amqp.exceptions.ConnectionForced: Too many heartbeats missed
With this change we could not reproduce this issue during a deployment
and no missed heartbeat messages were observed during the deploy.
Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>
Co-Authored-By: Hervé Beraud <hberaud@redhat.com>
Closes-Bug: #1826281
Change-Id: Id22b1465d6d2424d90781983b970aba4545feb8a
We have jsonschema capped at a fairly old version. Other than some
specific releases, it looks like keeping it below 3.0 was added in
I943fd68b9fab3bce1764305a5058df5339470757 without really any explanation
why.
In order to update to a 3.x release we need to:
1. Remove the cap from global-requirements.txt (see Depends-On), leaving
upper-constraints.txt at a 2.x release
2. Remove the cap from all consumers (this change)
3. Release a new version of consumers that are published to pypi
4. Update upper-constraints.txt with those new releases
5. Update jsonschema in upper-constraints.txt to a 3.X release
(See: https://review.openstack.org/649789)
6. Test consumers with the change from 5.
7. [Optional] fix issues in consumers that arise from 6.
8. Merge the change from 5.
Change-Id: Ic782c3085fb85634d2ae2b7d5538ae2948d2f9b8
Co-Authored-by: Sean McGinnis <sean.mcginnis@gmail.com>
Depends-On: https://review.openstack.org/649669
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.
Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.
Add openstack-tox-lower-constraints job to the zuul configuration.
See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.
Also adjusted a warning check in the utils.config test due to python3
Also updated requests to 2.18.0 so that the python uploader can use
requests as a context manager which was not added until v2.18.0. See
4847f5b8cd
Change-Id: I78bffdf69113988192afc466d4ce04b3b97e796a
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Co-Authored-By: Alex Schultz <aschultz@redhat.com>
The mistral executor uses eventlet patching, but gitpython's subprocess
module is not being properly patched. Patching it manually fix the issues
Change-Id: I49d041b48d9939844bcc3a571819887772caf491
Closes-bug: #1818757
The docker based image uploader can no longer be used because the
docker service has been removed from the undercloud.
This change removes the DockerImageUploader class, relieving us of the
burden of maintaining unused code.
Change-Id: I7be5915a0aee561250c16487c1fee7afd28b9036
Blueprint: podman-support
* Replace image detection with the call from metalsmith
* Validate image parameters early to detect trivial errors
* Add support for specifying subnets in NICs
* Use more specific exceptions when checking for existing instances
* Drop compatibility code for nodes in < 0.9.0
Change-Id: I719082c0845b517172c309838e37a5e38a04c04c
Implements: blueprint nova-less-deploy
This change adds a workflow to provision bare metal machines using
Ironic API without Nova (through the metalsmith library for convenience).
A simple workflow to unprovision nodes is included as well.
This is the first step towards supporting deployment without Nova.
Change-Id: I7c7aeb83691865d37ebed4d6cad7524339fdb799
Implements: blueprint nova-less-deploy
This is needed to land I2c5b511df50f29c63aa613899c2bebb506360bf4,
see the reasoning on that patch.
This reverts commit 6fa7a0974a.
Partial-Bug: #1771627
Change-Id: Ia86842b0b1f42512f25390d6bdb695e0f8133c6d
This gets a TLS certificate for the overcloud when necessary:
* If no incoming cert/key is provided and we don't expect the
overcloud's certmonger instances to request the certificates,
we request one to the undercloud's certmonger local CA.
* If a certificate was provided, we verify if it's user-provided
or if it was autogenerated.
- If it was user-provided we pass through that certificate
- If it was autogenerated, we request or resubmit the request
if it's needed.
* We also accept the EnableTLS flag, which the deployer can
explicitly turn off if they decide not to use TLS.
Depends-On: Ic70dd323b33596eaa3fc18bdc69a7c011ccd7fa1
Change-Id: I3d3cad0eb1396e7bee146794b29badad302efdf3
Using process based concurrency doesn't work with mistral's threading
model. Switching to concurrent.futures allows the threading executor
to be used instead.
Change-Id: I2c7a1a7cb8d8e195e041b5b185c1ceba3a6d0e86
This change replaces the custom image pull/push retry loop with a
tenacity annotation.
The docker client uses requests, so this fixes an issue where requests
can raise a vendored urllib3 exception if the registry doesn't respond
at all. This will now be caught by tenacity and treated as a retry
condition.
Change-Id: Ifa08f21ccbc1bf04732462d28449546cf146c53a
Closes-Bug: #1749663
We were using deprecated arguments to create the client, which have
been removed in I9df8296ddc1b41cde22ff8b0f26d73c4408741ce.
Also make explicit the keystoneauth1 requirement - the library is
already used in other parts of the code but wasn't explicitly
referenced.
Change-Id: I2314020a6414f90d964f7be474668658913e61b9
Closes-Bug: #1747057
At some point on 2018-02-02, all tripleo-common tox python jobs started
failing with:
2018-02-02 15:05:50.952672 | ubuntu-xenial | ImportError: No module
named keystoneclient
We did't have python-keystoneclient in requirements.txt, though we
probably should. We must have been implicitly getting it from somewhere
else previously and something must have changed to where it's now
explicitly required.
Change-Id: I272292f2f411d1195e5986f0f6a24b379fbf2b6e
Closes-Bug: #1747160
Newer versions of gitdb are conflicting with the dependencies of newer
versions of GitPython, so they can not be properly coinstalled
This dep was introduced in If069ebb33ee4cb7f81308c68849ba9f969752d25
but not actually used back then. drop it.
Change-Id: Idc30b635dcba04aec47a3537cb9cc490eb255da9
In order to allow sending messages to zaqar queue we need to
provide Zaqar client in TripleOAction.
Change-Id: I2378e3626a415fd0cdf085735a4ed890e6aa61f4
New workflow discover_nodes works by trying to log into given IP addresses
with given set of credentials.
New workflow discover_and_enroll_nodes uses results of the previous workflow
to create new nodes and optionally move them to manageable.
Implements blueprint node-discovery-by-range
Change-Id: I158f0b8f5251d9d94e7e57b3fe24362316d26599
The pypi package 'docker-py' [1] has been renamed to 'docker' [2].
It is better to move to the new 'docker' package because the old
package will be deprecated and all the new features will go into
the new package only.
Package 'docker' has been added to requirements [3]. The old
package 'docker-py' is still allowed to be in the global requirements
during the transition period but it should be removed after all or
most of the projects finsih the migration.
[1] https://pypi.python.org/pypi/docker-py
[2] https://pypi.python.org/pypi/docker
[3] https://review.openstack.org/#/c/423715/
Change-Id: Id440e7362a65ddd56cfb77f3a8b502750c283a21
The keystone utils for getting project endpoints is the last reason that
we import directly from Mistral. This change copies over the relevant
code that we need.
Removing the dep also discovered two dependancies that we had but didn't
explicitly state previously.
Change-Id: If4cb4ac4ca75ea0f165196b4a0a08ea3d3a16e25
mistral_lib base TripleoAction has a context in the run method
signature. These changes remove the mistral.context and make use
of the one provide by the default executor
Change-Id: Ib1a5aa8d5735b05f5308dc943ac088b5eeeec253
This reuses the existing password generation mistral action to generate an ssh
keypair to be used for nova cold migration
Paramiko is used to generate the ssh key, based on the existing approach in the
nova keypair api.
Also update validation ssh key generation to reuse the same method.
Change-Id: I9e7a1862911312ad942233ac8fc828f4e1be1dcf
This patch refactors the plan_management workbook to reduce
redundancy in creating deployment plans and adds the ability
to use templates stored in an external git repo when creating
or updating a deployment plan. The workbook has tasks added
to the workflows that cleanup temporary files created during
the git clone operations.
A tag or branch can be specified in the url with an '@'.
Example: https://github.com/openstack/project.git@stable/newton
Implements: blueprint git-deployment-plan
Change-Id: If069ebb33ee4cb7f81308c68849ba9f969752d25
This commit allows image uploader config file with missing
'push_destination' attributes. In which case, the host's control plane
(br-ctlplane) IP address with port 8787 is used or localhost if there
is no such interface.
Change-Id: Idfe18f25ddee944ef96ab14f8edeb056656a06bc
Michele Baldessari