openstack
/
diskimage-builder
Code Issues Proposed changes

source-repositories: don't use --git-dir 

A recent(-ish) change in git [1] has exposed a bug in caching that
appears in one very specific circumstance -- updating the
openstack/openstack super-repo [2].

This repo gets a submodule update every time something is pushed.  By
using "--git-dir" while the cwd is one-level above the actual repo we
are confusing [1] which is not finding the submodule directories
correctly and giving us an error:

 Could not access submodule 'foo'

for every submodule that has updated between now and the last time we
updated the cache. [3]

The git manual does warn about this

 If you just want to run git as if it was started in <path> then use
 git -C <path>.

Indeed, that is what we want to do in this path.  Modify the calls to
use -C.

[1] 505a276596
[2] https://opendev.org/openstack/openstack/
[3] The result for opendev production is that image builds fail every
    time an openstack/* project is checked in; we then race to retry
    the build before another commit lands and updates the submodules
    again.

Change-Id: Iadb23454e29d8869e11407e1592007b0f0963e17
This commit is contained in:
Ian Wienand 2021-11-16 19:16:03 +11:00
parent 85e20d9852
commit ddb06b6657
1 changed files with 5 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
diskimage_builder/elements/source-repositories/extra-data.d/98-source-repositories
View File

@ -125,13 +125,13 @@ function get_repos_for_element(){
fi
if [ "$REPOREF" != "*" ] ; then
HAS_REF=$(git --git-dir=$CACHE_PATH/.git name-rev $REPOREF 2>/dev/null || true)
HAS_REF=$(git -C ${CACHE_PATH} name-rev $REPOREF 2>/dev/null || true)
fi
if [ -z "$DIB_OFFLINE" -o -z "${HAS_REF:-}" ] ; then
echo "Updating cache of $REPOLOCATION in $CACHE_PATH with ref $REPOREF"
# Copy named refs (which might be outside the usual heads
# pattern) - e.g. gerrit
if [ "$REPOREF" == "*" ] || ! git --git-dir=$CACHE_PATH/.git fetch -q --prune --update-head-ok $REPOLOCATION \
if [ "$REPOREF" == "*" ] || ! git -C ${CACHE_PATH} fetch -q --prune --update-head-ok $REPOLOCATION \
+${REPOREF}:${REPOREF} ; then
# Copy all heads from the remote repository - this permits
# using a SHA1 object reference so long as the object
@ -139,12 +139,12 @@ function get_repos_for_element(){
# not permit arbitrary sha fetching from remote servers.
# This is a separate fetch to the prior one as the prior
# one will fail when REPOREF is a SHA1.
git --git-dir=$CACHE_PATH/.git fetch -q --prune --update-head-ok $REPOLOCATION \
git -C ${CACHE_PATH} fetch -q --prune --update-head-ok $REPOLOCATION \
+refs/heads/*:refs/heads/* +refs/tags/*:refs/tags/*
fi
# Ensure that we have a reference to the revision.
if [ "$REPOREF" != "*" ] ; then
if ! git --git-dir=$CACHE_PATH/.git rev-parse -q --verify $REPOREF^{commit} > /dev/null; then
if ! git -C ${CACHE_PATH} rev-parse -q --verify $REPOREF^{commit} > /dev/null; then
echo "Failed to find reference to $REPOREF"
exit 1
fi
@ -154,7 +154,7 @@ function get_repos_for_element(){
echo "Cloning from $REPONAME cache and applying ref $REPOREF"
# If the local dir is already used, see if the pertinent details differ
if [[ -d $REPO_DEST ]]; then
DESIRED="$(sudo git --git-dir=$REPO_DEST/.git config remote.origin.url)"
DESIRED="$(sudo git -C ${REPO_DEST} config remote.origin.url)"
if [[ "$CACHE_PATH" != "$DESIRED" ]]; then
echo "REPOLOCATIONS don't match ("$CACHE_PATH" != "$DESIRED")" >&2
exit 1