3d5a035197
We implemented retries of console log grabbing to workaround occasional jenkins 502 errors, but this is running under set -e so curl returning a non zero error message resulted in short circuiting the retries and failed the jobs. Fix this with a simple || true appendage to the curl command so that any failed curls can error properly without affecting the error checking in the script. We will then retry until we eventually give up. Change-Id: I0a784aa45bff7de5b411372ccecc569ec148ecc5
39 lines
1.4 KiB
Bash
Executable File
39 lines
1.4 KiB
Bash
Executable File
#!/bin/bash -xe
|
|
|
|
RETRY_LIMIT=20
|
|
|
|
# Keep fetching until this uuid appears in the logs before uploading
|
|
END_UUID=$(cat /proc/sys/kernel/random/uuid)
|
|
|
|
echo "Grabbing consoleLog ($END_UUID)"
|
|
|
|
# Since we are appending to fetched logs, remove any possibly old runs
|
|
rm -f /tmp/console.html
|
|
|
|
# Grab the HTML version of the log (includes timestamps)
|
|
TRIES=0
|
|
console_log_path='logText/progressiveHtml'
|
|
while ! grep -q "$END_UUID" /tmp/console.html; do
|
|
TRIES=$((TRIES+1))
|
|
if [ $TRIES -gt $RETRY_LIMIT ]; then
|
|
break
|
|
fi
|
|
sleep 3
|
|
# -X POST because Jenkins doesn't do partial gets properly when
|
|
# job is running.
|
|
# --data start=X instructs Jenkins to mimic a partial get using
|
|
# POST. We determine how much data we need based on
|
|
# how much we already have.
|
|
# --fail will cause curl to not output data if the request
|
|
# fails. This allows us to retry when we have Jenkins proxy
|
|
# errors without polluting the output document.
|
|
# --insecure because our Jenkins masters use self signed SSL certs.
|
|
curl -X POST --data "start=$(stat -c %s /tmp/console.html || echo 0)" --fail --insecure $BUILD_URL$console_log_path >> /tmp/console.html || true
|
|
done
|
|
|
|
# We need to add <pre> tags around the output for log-osanalyze to not escape
|
|
# the content
|
|
|
|
sed -i '1s/^/<pre>\n/' /tmp/console.html
|
|
echo "</pre>" >> /tmp/console.html
|