Improve generate-cgcs-centos-repo.sh efficiency

The generate-cgcs-centos-repo.sh utility sets up symlinks
in the cgcs-centos-repo directory for each entry in the
defined LST files, pointing to files from a mirror. In the
processing of the LST files, a complete "find" of the
specified mirror is done for each file, looking for a match.
Given there are more than 2000 such entries, this causes
the tool to take a long time to do its job.

This update enhances the tool to do a single find upfront,
caching the result in a temporary file. During processing of
the LST files, then, the tool can grep this file to find the
corresponding file, rather than repeating the find.

In addition, this update fixes an issue with error reporting,
where a flag was used inside the processing loop to indicate
missing files. However, because the loop runs in a subshell,
this status flag is no longer set when checked outside the loop.
This is replaced with a check of the missing file report.

Change-Id: I0eb41f9da3d5afd010ade9308474e537c409f540
Closes-Bug: 1831488
Signed-off-by: Don Penney <don.penney@windriver.com>
This commit is contained in:
Don Penney 2019-06-03 14:45:28 -04:00
parent 78258f737f
commit cac209d677

View File

@ -59,17 +59,18 @@ for t in "Binary" "Source" ; do
fi
done
unsuccessful_file=0
mirror_content=$(mktemp -t centos-repo-XXXXXX)
find ${mirror_dir} -type f > ${mirror_content}
for lst_file in ${rpm_lst_files} ; do
grep -v "^#" ${lst_file_dir}/${lst_file} | while IFS="#" read rpmname extrafields; do
if [ -z "${rpmname}" ]; then
continue
fi
mirror_file=$(find ${mirror_dir} -name ${rpmname})
mirror_file=$(grep "/${rpmname}$" ${mirror_content})
if [ -z "${mirror_file}" ]; then
echo "Error -- could not find requested ${rpmname} in ${mirror_dir}"
echo ${rpmname} >> ${missing_rpms_file}
unsuccessful_file=1
continue
fi
@ -87,11 +88,11 @@ for lst_file in ${rpm_lst_files} ; do
ln -sf "${mirror_dir}/$ff" "${dest_dir}/${sub_dir}"
if [ $? -ne 0 ]; then
echo "Failed ${mirror_file}: ln -sf \"${mirror_dir}/$ff\" \"${dest_dir}/${sub_dir}\""
unsuccessful_file=1
fi
done
done
rm -f ${mirror_content}
if [ ! -f "$mock_cfg_file" ]; then
echo "Cannot find mock.cfg.proto file!"
@ -130,6 +131,10 @@ cat ${lst_file_dir}/${other_lst_file} | grep -v "#" | while IFS=":" read targett
done
echo "Done creating repo directory"
if [ ${unsuccessful_file} -ne 0 ]; then
declare -i missing_rpms_file_count=$(wc -l ${missing_rpms_file} 2>/dev/null | awk '{print $1}')
if [ ${missing_rpms_file_count} -gt 0 ]; then
echo "WARNING: Some targets could not be found. Your repo may be incomplete."
echo "Missing targets:"
cat ${missing_rpms_file}
exit 1
fi