It appears this setting inadvertently had an extra word "ignore"
included in the key name, and later versions of the git module in
Ansible (5) throw an error in that case. Remove it so that this
role will work in all versions.
This has been previously tested in a base-test cycle via
test-mirror-workspace-git-repos.
Change-Id: I597aa8d331e53955e826f09aa51a579e12c94f85
If the checkout failed the mirror-workspace-git-repos must fail the
job. Otherwise this can lead to a broken gate because a different
state than the state that should be tested can be tested silently.
Change-Id: I98e948776ec4b749bc1475ee3a17e37007591118
The mirroring of git workspaces to k8s pod nodes is done by piping a git
push command through `kubectl exec`. Without specifying a k8s context
for kubectl, the `current-context` from the kubeconfig is used. This is
problematic in multi-node scenarios where different nodes reside in
different namespaces and thus, are reachable via separate contexts.
Ansible cannot handle the context switch automatically as it would via
the inventory and its kubectl connection plugin because we are running
the git push from the executor/localhost.
This change makes sure the correct context of the corresponding node is
passed to `kubectl exec` explicitly, thus pushing the git repos to the
correct k8s pod.
Change-Id: Ic0b11b5860bb0062b3a1c068f4f963fb76018715
The `mirror-workspace-git-repos` uses Git to mirror repos from the
executor to build nodes. This comes in handy when the node already
holds a copy of the repo(s), as Git can incrementally update them.
Currently, `mirror-workspace-git-repos` assumes the build node is
directly reachable via ssh. This is (usually) not the case when
running against a Kubernetes Pod build node, thus we can not
efficiently update repos on such nodes using Git.
This change allows pushing Git repos to Kubernetes Pods by bridging
the Git communication over `kubectl` and thereby allowing for the
same behaviour as for standard, non-container, ssh-reachable build
nodes.
(cf. https://git-scm.com/docs/git-remote-ext)
Change-Id: I71919d3bc93e48d5bb24234265fca9c84ce23b87
In case of reused workspaces on static nodes remote
non-bare repos needs to be cleaned up to make sure
that any untracked files from any earlier jobs are removed.
Change-Id: I00dda298f499b241a224c19657ebbb8a9015eb4d
With the arrival of ansible-lint 4, Jinja2 variable expansions must
include spaces before and after the variable name inside the
brackets.
Adjust the new violations accordingly and remove the rule
206 exclusion.
Change-Id: Ib3ff7b0233a5d5cf99772f9c2adc81861cf34ffa
Especially when using Github branch and push workflows repos can
easily have hundreds of branches. The role mirror-workspace-git-repos
which pushes them onto the node prints all of them in every
build. This can get quite messy. Thus we need to optionally be able to
silence the git push operation. This is done on purpose by the --quiet
parameter instead of using no_log because this way errors are still
logged normally and normal operation produces no log output for this
task.
Change-Id: Id77cc00972df9139e4544dd79954d91f4ab6b60b
This expilicitly notes the HEAD of the checked out tree in the logs.
Currently you'll see something like
... HEAD is now at 9dfb46a18 [wip] install ansible as editable during devel jobs
... Already on 'master'
... ok: Item: Runtime: 0:00:00.079626
for each project as this loop happens. However there's no explicit
mention of the project name and you have to reverse engineer what
menioned change relates to what project.
Where this would be useful is when you have an external project
dependency and suddenly your testing stops working due to a change in
it. You can quickly compare the last good run with the failing HEAD
mentioned here to help bisect the problem.
Change-Id: I84d9a8a460e99d3bc558b3a7584d787b22f2ed33
In the case of using a custom SSH port to connect to the ansible host,
the task of pushing the repository failed, as it assumed the SSH port to
be the default 22.
Now the command is aware of the custom SSH port, as it is specified in
the host facts.
Note that we have to escape the hostname if it is an ipv6 address
(because : is a special character in ipv6 addresses). Thankfully ansible
provides a filter to do that for us:
https://docs.ansible.com/ansible/2.5/user_guide/playbooks_filters_ipaddr.html#wrapping-ipv6-addresses-in-brackets
Change-Id: I76969850fc3a57078370fe38efdebac146720797
This reverts commit 34cfe44ca5c63c354339aa502052d9054a36bf60.
The :port suffix doesn't work if the hostname is an ipv6 ip address as
the hostname needs to be wrapped in []s in that case. Revert this so
that we can fix this in a followup.
Change-Id: I2dfee56d40a4cdd096946a2f67c37117fff74cce
In the case of using a custom SSH port to connect to the ansible host,
the task of pushing the repository failed, as it assumed the SSH port to
be the default 22.
Now the command is aware of the custom SSH port, as it is specified in
the host facts.
Change-Id: I3418016ad5a57e3024a8a553340a7e69b7d1c2fe
We need to set the remote HEAD of the repos after we mirror them. We
have been doing that by looking at git status on the local repos to find
what their head has been set to. However, we have 'checkout' in
zuul.projects now, so we can really just use that.
Change-Id: I251da11b7767793278c3f08dcf64aa008e759669
Depends-On: https://review.openstack.org/540948
Sometimes the task to determine the local HEAD of projects we've
checked out is failing in a way that we can't parse. Add a step before
the one that fails to show the full git status output to help with
debugging.
Change-Id: I095489249593442e1e9a3b346aed1ecf4135825f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Zuul has transitioned zuul.projects to a dict that mirrors
zuul._projects. Convert users back to zuul.projects so we can remove
the underscore version in zuul.
Change-Id: I1408a15d4da274f3fc911f03d5572f4818bc4ebb
Depends-On: I3c011f72933e98ccbf8badf0e9197c8659766c51
This is a revert of commit f28cc33bc3bc7f8a30d868964ec7712c26cef8f9
which was a revert of the original attempt at moving to zuul._projects
(df987735730e712ea3bc57f99d3c96cb561c43a9).
The issue was that the order of projects in "Determine local HEAD" was
not the same as "Update remote repository state" and we ended up
checking out the wrong thing.
To avoid this, build the projects and their current heads on the
exectutor into a separate dict, and as we iterate the projects look up
the correct values in that.
Change-Id: I0d9a66435714efe42fb653aafc6cb71b08ca6b8a
This reverts commit df987735730e712ea3bc57f99d3c96cb561c43a9.
The order in which we iterate over items in the "determine local HEAD"
task is not always the order in which we iterate over mirror-workspace-
git-repos.
Change-Id: I181d6a2cfa2c3104730f20362bbdc10fa582339f
Zuul exports the projects list as a dictionary in _projects in the
depends-on. The idea is that we switch these jobs to that, and do
some testing. When ready, we switch "projects" to be the dictionary
view, and revert "_project" calls here to "project".
Change-Id: Id9a7c137ca5bed25d81087201091157c8401576a
Depends-On: I4476b9d4915d107e29b91229287865bff0ada305
Fix a few problems noticed when using this in project-config:
* Ignore apt-get warning in configure-mirrors
* All tasks should be named in upload-logs, emit-job-header
* use git_config in mirror-workspace-git-repos
* Ignore git warnings in mirror-workspace-git-repos
Change-Id: Ibcdc8d5a42fb081b5faa0cc89460e4cf88d369ba
There are no playbooks that use the roles in this repo in a way that
would lint them, however running linters in project-config found
numerous issues in these roles. Correct the roles.
Change-Id: I017532c8aed8fd90474052423c7ba2510cf84631
Signed-off-by: Jesse Keating <omgjlk@us.ibm.com>
When we're pushing to a non-bare repo, git has a sad if we push to the
branch that's checked out. But that's ok - we're going to update the
checked out state anyway as a follow up. Tell git it's ok.
Then, after we've pushed, we need to reset after having pushed branch HEAD
to a non bare repo. Clean up after ourselves by removing the config
setting we added to allow the push in the first place.
Change-Id: Id84e9af9d33519e515b8e6d330b375a8e17c14f5