Sean Mooney c2a5090c2f allow get_extra_files to retry on common errors ...

This change add 4 new varibles to devstack to allow controlling
the behvaior fo downloadign files via get_extra_files

EXTRA_FILES_RETRY=${EXTRA_FILES_RETRY:-3}
EXTRA_FILES_RETRY_ERRORS=${EXTRA_FILES_RETRY_ERRORS:-"500,503"}
EXTRA_FILES_DOWNLOAD_TIMEOUT=${EXTRA_FILES_DOWNLOAD_TIMEOUT:-2}
EXTRA_FILES_RETRY_TIMEOUT=${EXTRA_FILES_RETRY_TIMEOUT:-10}

get_extra_files checks if a file is precached and downlaods
it if not present

The new fucntionaltiy modifes the wget invocation to retry
up to 3 times if there are dns name issues, connection refused
or the server return a 500 or 503

This change will not retry on 403 or 429 error codes
which are often used to indicate rate limits.

The download timeout remains at 2 seconds but
can now be set if required. A 10 second over all retry
wait timeout is added matching wgets default.

finally the use of sed to strip the file:// prefix
if present is replace with usign bash parmater expansion
echo "${file_url#file://}"

Assisted-By: gemini pro 3
Change-Id: Iedbb8974dce34a3d1c89f2a2561df76e696afae6
Signed-off-by: Sean Mooney <work@seanmooney.info>

2025-11-24 19:15:51 +00:00

32 KiB

Raw Blame History

View Raw