dl_tarball.sh: Support Broadcom tar file download

This commit adds support for downloading Broadcom's NetXtreme-E
driver/library tar archive file. The tar file consists of multiple
nested archives and source RPM files. We are only interested in the
libbnxt_re Infiniband verbs library SRPM and the bnxt_en/bnxt_re kernel
driver source code archive:
- libbnxt_re-220.0.5.0-rhel7u9.src.rpm
- netxtreme-bnxt_en-1.10.2-220.0.13.0.tar.gz

If the archive has already been downloaded, the sha256sum of the archive
is verified. (The checksum is stored in the utility/"util" field in the
listing file.) If not already downloaded, then the archive is downloaded
and the sha256sum of the archive is checked. Finally, the desired files
are extracted from the main tar archive.

(Checksum verification is added as a package-specific behaviour, because
this feature does not exist in the build system, and we would like to be
aware in case the software package is modified.)

Testing:
- dl_tarball.sh correctly removes a pre-existing tar archive with an
  incorrect sha256sum and re-downloads the tar archive.
- If the tar archive does not already exist, then the archive is
  correctly downloaded and the sha256sum is correctly checked.
- If the download_file function or the check_sha256sum function fails,
  then the shell script correctly reports an error message and
  continues. (This was verified with shell script instrumentation.)
- If the tar archive extraction fails or if the expected/desired files
  cannot be found, an error is correctly reported by the shell script,
  and the tar archive is removed to allow follow-up attempts to try
  again. (Also verified with instrumentation.)

Story: 2009915
Task: 44761

Change-Id: Id021a33e7f26643139d6ef0dda5c7146cfb7f172
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>

centos-mirror-tools/dl_tarball.sh

@@ -507,6 +507,53 @@ for line in $(cat $tarball_file); do
 
             popd >/dev/null # pushd "$directory_name"
             rm -rf "$directory_name"
+        elif [ "${tarball_name}" = "bcm_220.0.83.0.tar.gz" ]; then
+
+            # "${util}" is the expected sha256sum of the downloaded tar archive.
+            #
+            # Check if the file is already downloaded and if its sha256sum is
+            # correct.
+            if [ -f "${tarball_name}" ] && \
+                    ! check_sha256sum "${tarball_name}" "${util}"; then
+                # Incorrect checksum. Maybe the previous download attempt
+                # failed? Remove the file and attempt to re-download.
+                rm -f "${tarball_name}"
+            fi
+
+            if ! [ -f "${tarball_name}" ]; then
+                download_file --quiet "${tarball_url}" "${tarball_name}"
+                if [ $? -ne 0 ]; then
+                    echo "Warning: failed to download '${tarball_url}'"
+                    error_count=$((error_count + 1))
+                    popd > /dev/null   # pushd $output_tarball
+                    continue
+                fi
+
+                if ! check_sha256sum "${tarball_name}" "${util}"; then
+                    echo "Warning: incorrect sha256sum for '${tarball_url}'"
+                    error_count=$((error_count + 1))
+                    popd > /dev/null   # pushd $output_tarball
+                    continue
+                fi
+            fi
+
+            rm -rf "${directory_name}"
+
+            if ! tar -xf "${tarball_name}" || \
+                    ! cp "${directory_name}/Linux/Linux_Driver/netxtreme-bnxt_en-1.10.2-220.0.13.0.tar.gz" . || \
+                    ! cp "${directory_name}/Linux/KMP-RoCE-Lib/KMP/Redhat/rhel7.9/libbnxt_re-220.0.5.0-rhel7u9.src.rpm" . ; then
+                # Extraction failed. Remove the tar archive to allow another
+                # attempt.
+                rm -f "${tarball_name}"
+                echo "Warning: Could not extract '${tarball_name}' or could not find expected files."
+                error_count=$((error_count + 1))
+            fi
+
+            rm -rf "${directory_name}"
+
+            # We do not delete the original tar archive we just extracted from,
+            # so that it will not need to be downloaded again.
+            #   rm -f "${tarball_name}"
         fi
         popd > /dev/null # pushd $output_tarball
         continue

centos-mirror-tools/utils.sh

@@ -279,3 +279,10 @@ get_from() {
     from=$(echo $base | rev | cut -d'_' -f1-1 | rev)
     echo $from
 }
+
+check_sha256sum() {
+    local file="${1}"
+    local sha256sum="${2}"
+
+    sha256sum "${file}" | cut -d' ' -f1 | grep -q -F -x "${sha256sum}"
+}