StarlingX open source release updates
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
This commit is contained in:
Dean Troyer
parent
2cfd128c20
commit
cfe45dadae
4
.gitignore
vendored
Normal file
4
.gitignore
vendored
Normal file
@ -0,0 +1,4 @@
|
||||
*.swp
|
||||
/cgcs-centos-repo
|
||||
/cgcs-tis-repo
|
||||
/cgcs-3rd-party-repo
|
||||
202
LICENSE
Normal file
202
LICENSE
Normal file
@ -0,0 +1,202 @@
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
3
addons/.gitignore
vendored
Normal file
3
addons/.gitignore
vendored
Normal file
@ -0,0 +1,3 @@
|
||||
/wr-avs
|
||||
/wr-cgcs
|
||||
|
||||
19
build-data/unbuilt_rpm_patterns
Normal file
19
build-data/unbuilt_rpm_patterns
Normal file
@ -0,0 +1,19 @@
|
||||
[-]locale[-]
|
||||
[-]doc[-]
|
||||
[-]dbg[-]
|
||||
vswitch-staticdev
|
||||
guest-scale-agent-staticdev
|
||||
vim-spell
|
||||
openssh-server-sysvinit
|
||||
openstack-neutron-linuxbridge
|
||||
^libcacard-
|
||||
^kernel-bootwrapper
|
||||
^kernel-doc-
|
||||
^kernel-abi-whitelists
|
||||
^kernel-debug-
|
||||
^kernel-kdump
|
||||
^kernel-rt-bootwrapper
|
||||
^kernel-rt-doc-
|
||||
^kernel-rt-abi-whitelists
|
||||
^kernel-rt-debug-
|
||||
^kernel-rt-kdump
|
||||
78
build-tools/Cached_Data.txt
Normal file
78
build-tools/Cached_Data.txt
Normal file
@ -0,0 +1,78 @@
|
||||
Data on an source rpm:
|
||||
|
||||
location:
|
||||
${MY_WORKSPACE}/${BUILD_TYPE}/rpmbuild/SPECS/${SRPM_FILE_NAME}/
|
||||
|
||||
files:
|
||||
*.spec # spec file found in the source rpm
|
||||
|
||||
subdirectories:
|
||||
NAMES/ # Directory contains an emtpy file, where the file name
|
||||
# is the name of the source rpm.
|
||||
|
||||
SERVICES/ # Directory contains zero or more emtpy files, where the
|
||||
# file name is the name of the service provided by one
|
||||
# or more of the rpms.
|
||||
|
||||
BUILDS/ # Directory contains emtpy files, where the file name is
|
||||
# the name of a binary rpm built from the source rpm.
|
||||
|
||||
BUILDS_VR/ # Directory contains emtpy files, where the file name is
|
||||
# the name-verion-release of a binary rpm built from the
|
||||
# source rpm.
|
||||
|
||||
location:
|
||||
${MY_WORKSPACE}/${BUILD_TYPE}/rpmbuild/SOURCES/${SRPM_FILE_NAME}/
|
||||
|
||||
files:
|
||||
BIG # if it exists, it contains one line, the numeric value
|
||||
# extracted from build_srpms.data if the line
|
||||
# BUILD_IS_BIG=### if present.
|
||||
# This is the estimated filesystem size (GB) required to
|
||||
# host a mock build of the package.
|
||||
# Note: not all parallel build environments are the same
|
||||
# size. The smallest build environmnet is 3 GB and this
|
||||
# is sufficient for most packages. Don't bother adding a
|
||||
# BUILD_IS_BIG=### directive unless 3 gb is proven to be
|
||||
# insufficient.
|
||||
|
||||
SLOW # if it exists, it contains one line, the numeric value i
|
||||
# extracted from build_srpms.data if the line
|
||||
# BUILD_IS_SLOW=### if present.
|
||||
# This is the estimated build time (minutes) required to
|
||||
# host perform a mock build of the package.
|
||||
# Note: Currently we only use this value as a boolean.
|
||||
# Non-zero and we try to start the build of this package
|
||||
# earlier rather than later. Build times >= 3 minutes are
|
||||
# worth anotating. Else don't bother adding a
|
||||
# BUILD_IS_SLOW=### directive
|
||||
e.g.
|
||||
|
||||
cd $MY_WORKSPACE/std/rpmbuild/SPECS/openstack-cinder-9.1.1-0.tis.40.src.rpm
|
||||
find .
|
||||
./BUILDS
|
||||
./BUILDS/openstack-cinder
|
||||
./BUILDS/python-cinder
|
||||
./BUILDS/python-cinder-tests
|
||||
./NAMES
|
||||
./NAMES/openstack-cinder
|
||||
./SERVICES
|
||||
./SERVICES/cinder
|
||||
./BUILDS_VR
|
||||
./BUILDS_VR/openstack-cinder-9.1.1-0.tis.40
|
||||
./BUILDS_VR/python-cinder-9.1.1-0.tis.40
|
||||
./BUILDS_VR/python-cinder-tests-9.1.1-0.tis.40
|
||||
./openstack-cinder.spec
|
||||
|
||||
|
||||
e.g.
|
||||
cd $MY_WORKSPACE/std/rpmbuild/SOURCES/kernel-3.10.0-514.16.1.el7.29.tis.src.rpm
|
||||
find .
|
||||
./BIG
|
||||
./SLOW
|
||||
|
||||
cat ./BIG
|
||||
8
|
||||
|
||||
cat ./SLOW
|
||||
12
|
||||
70
build-tools/audit-pkgs
Executable file
70
build-tools/audit-pkgs
Executable file
@ -0,0 +1,70 @@
|
||||
#!/bin/bash
|
||||
|
||||
rpm_compare () {
|
||||
local r="$1"
|
||||
local r2="$2"
|
||||
local line
|
||||
local f=$(basename $r)
|
||||
local f2=$(basename $r2)
|
||||
|
||||
rpm -q --dump --nosignature -p $r | awk ' { print $1 "\n" $1 " " $5 " " $6 " " $7 " " $8 " " $9 " " $10 " " $11 } ' > /tmp/dump.new
|
||||
rpm -q --dump --nosignature -p $r2 | awk ' { print $1 "\n" $1 " " $5 " " $6 " " $7 " " $8 " " $9 " " $10 " " $11 } ' > /tmp/dump.old
|
||||
first_line=1
|
||||
diff -y -W 200 --suppress-common-lines /tmp/dump.new /tmp/dump.old | grep '|' |
|
||||
while read -r line; do
|
||||
left=$(echo "$line" | awk -F '|' '{ print $1 }')
|
||||
right=$(echo "$line" | awk -F '|' '{ print $2 }')
|
||||
left_f=$(echo "$left" | awk '{ print $1 }')
|
||||
right_f=$(echo "$right" | awk '{ print $1 }')
|
||||
if [ "$left_f" != "$right_f" ];then
|
||||
continue
|
||||
fi
|
||||
if [ $first_line -eq 1 ]; then
|
||||
echo ""
|
||||
echo "$f vs $f2"
|
||||
first_line=0
|
||||
fi
|
||||
echo "$line"
|
||||
done
|
||||
}
|
||||
|
||||
echo ""
|
||||
echo "======================================================"
|
||||
echo "Auditing built packages vs unpatched upstream packages"
|
||||
echo "======================================================"
|
||||
for r in $(find $MY_WORKSPACE/*/rpmbuild/RPMS -name '*.rpm' | grep -v '.src.rpm' | grep -v debuginfo); do
|
||||
f=$(basename $r)
|
||||
f2=$(echo $f | sed 's#[.]tis[.][0-9]*[.]#.#' | sed 's#[.]tis[.]#.#')
|
||||
r2=$(find $MY_REPO/cgcs-centos-repo/Binary/ -name $f2)
|
||||
if [ "$r2" == "" ]; then
|
||||
# Probably one of our own
|
||||
# echo "Couldn't find '$f2'"
|
||||
continue
|
||||
fi
|
||||
rpm_compare "$r" "$r2"
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "============================"
|
||||
echo "Auditing built for conflicts"
|
||||
echo "============================"
|
||||
grep 'conflicts with file from package' -r --binary-files=without-match $MY_WORKSPACE/*/results/ |
|
||||
while read -r line; do
|
||||
w=$(echo "$line" | awk '{ print $8 }')".rpm"
|
||||
w2=$(echo "$line" | awk '{ print $14 }')".rpm"
|
||||
echo "$w $w2"
|
||||
done | sort --unique | sed 's#bash-completion-1:#bash-completion-#' |
|
||||
while read -r line2; do
|
||||
f=$(echo "$line2" | awk '{ print $1 }')
|
||||
f2=$(echo "$line2" | awk '{ print $2 }')
|
||||
r=$(find $MY_REPO/cgcs-centos-repo/Binary/ $MY_WORKSPACE/*/rpmbuild/RPMS -name $f)
|
||||
r2=$(find $MY_REPO/cgcs-centos-repo/Binary/ $MY_WORKSPACE/*/rpmbuild/RPMS -name $f2)
|
||||
# echo ""
|
||||
# echo "$f vs $f2"
|
||||
# echo "$r vs $r2"
|
||||
if [ "$r" != "" ] && [ "$r2" != "" ]; then
|
||||
rpm_compare "$r" "$r2"
|
||||
fi
|
||||
done
|
||||
|
||||
|
||||
86
build-tools/branching/branch_and_tag.sh
Executable file
86
build-tools/branching/branch_and_tag.sh
Executable file
@ -0,0 +1,86 @@
|
||||
#!/bin/bash
|
||||
|
||||
# The purpose of this script is to create branches and tags that follow a convention
|
||||
# If the desired branch already exists, it is skipped.
|
||||
# If the desired tag already exists, it is skipped.
|
||||
|
||||
OLD_TAG=vCGCS_DEV_0018
|
||||
NEW_TAG=vCGCS_DEV_0019
|
||||
|
||||
OLD_BRANCH=CGCS_DEV_0018
|
||||
NEW_BRANCH=CGCS_DEV_0019
|
||||
|
||||
if [ -z "$MY_REPO" ]; then
|
||||
echo "MY_REPO is unset"
|
||||
exit 1
|
||||
else
|
||||
echo "MY_REPO is set to '$MY_REPO'"
|
||||
fi
|
||||
|
||||
if [ -d "$MY_REPO" ]; then
|
||||
cd $MY_REPO
|
||||
echo "checking out and pulling old branch"
|
||||
wrgit checkout $OLD_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: wrgit checkout $OLD_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
wrgit pull
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: wrgit pull"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "Could not change to diectory '$MY_REPO'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Finding subgits"
|
||||
SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
|
||||
|
||||
# Go through all subgits and create the NEW_BRANCH if it does not already exist
|
||||
# Go through all subgits and create the NEW_TAG if it does not already exist
|
||||
for subgit in $SUBGITS; do
|
||||
echo ""
|
||||
echo ""
|
||||
pushd $subgit > /dev/null
|
||||
git fetch
|
||||
git fetch --tags
|
||||
# check if destination branch already exists
|
||||
echo "$subgit"
|
||||
branch_check=`git branch -a --list $NEW_BRANCH`
|
||||
if [ -z "$branch_check" ]
|
||||
then
|
||||
echo "Creating $NEW_BRANCH"
|
||||
git checkout $OLD_BRANCH
|
||||
git checkout -b $NEW_BRANCH
|
||||
git push origin $NEW_BRANCH:$NEW_BRANCH
|
||||
else
|
||||
echo "$NEW_BRANCH already exists"
|
||||
fi
|
||||
tag_check=`git tag -l $NEW_TAG`
|
||||
if [ -z "$tag_check" ]
|
||||
then
|
||||
echo "Creating $NEW_TAG"
|
||||
# create tag
|
||||
git checkout $NEW_BRANCH
|
||||
git pull origin
|
||||
git tag $NEW_TAG
|
||||
git push origin $NEW_TAG
|
||||
else
|
||||
echo "$NEW_TAG already exists"
|
||||
fi
|
||||
|
||||
popd > /dev/null
|
||||
done
|
||||
|
||||
echo "All done. branches and tags are pushed"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
56
build-tools/branching/create_branches_and_tags.sh
Executable file
56
build-tools/branching/create_branches_and_tags.sh
Executable file
@ -0,0 +1,56 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ x"$1" = x ] ; then
|
||||
echo "ERROR: You must specify a name to create branches and tags"
|
||||
exit 1
|
||||
fi
|
||||
branch=$1
|
||||
tag="v$branch"
|
||||
|
||||
|
||||
|
||||
echo "Finding subgits"
|
||||
SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
|
||||
|
||||
# Go through all subgits and create the branch and tag if they does not already exist
|
||||
for subgit in $SUBGITS; do
|
||||
echo ""
|
||||
echo ""
|
||||
pushd $subgit > /dev/null
|
||||
|
||||
# check if destination branch already exists
|
||||
echo "$subgit"
|
||||
branch_check=`git branch -a --list $branch`
|
||||
if [ -z "$branch_check" ]
|
||||
then
|
||||
echo "Creating branch $branch"
|
||||
git checkout -b $branch
|
||||
if [ $? != 0 ] ; then
|
||||
echo "ERROR: Could not exec: git checkout -b $branch"
|
||||
popd > /dev/null
|
||||
exit 1
|
||||
fi
|
||||
# git push origin $branch:$branch
|
||||
else
|
||||
echo "Branch $branch already exists"
|
||||
git checkout $branch
|
||||
fi
|
||||
|
||||
tag_check=`git tag -l $tag`
|
||||
if [ -z "$tag_check" ]
|
||||
then
|
||||
echo "Creating tag $tag"
|
||||
git tag $tag
|
||||
if [ $? != 0 ] ; then
|
||||
echo "ERROR: Could not exec: git tag $tag"
|
||||
popd > /dev/null
|
||||
exit 1
|
||||
fi
|
||||
# git push origin $tag
|
||||
else
|
||||
echo "Tag $tag already exists"
|
||||
fi
|
||||
|
||||
popd > /dev/null
|
||||
done
|
||||
|
||||
36
build-tools/branching/create_tags.sh
Executable file
36
build-tools/branching/create_tags.sh
Executable file
@ -0,0 +1,36 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ x"$1" = x ] ; then
|
||||
echo "ERROR: You must specify a name to create tags"
|
||||
exit 1
|
||||
fi
|
||||
tag=$1
|
||||
|
||||
|
||||
echo "Finding subgits"
|
||||
SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
|
||||
|
||||
# Go through all subgits and create the tag if it does not already exist
|
||||
for subgit in $SUBGITS; do
|
||||
echo ""
|
||||
echo ""
|
||||
pushd $subgit > /dev/null
|
||||
|
||||
tag_check=`git tag -l $tag`
|
||||
if [ -z "$tag_check" ]
|
||||
then
|
||||
echo "Creating tag $tag"
|
||||
git tag $tag
|
||||
if [ $? != 0 ] ; then
|
||||
echo "ERROR: Could not exec: git tag $tag"
|
||||
popd > /dev/null
|
||||
exit 1
|
||||
fi
|
||||
# git push origin $tag
|
||||
else
|
||||
echo "Tag $tag already exists"
|
||||
fi
|
||||
|
||||
popd > /dev/null
|
||||
done
|
||||
|
||||
42
build-tools/branching/push_branches_tags.sh
Executable file
42
build-tools/branching/push_branches_tags.sh
Executable file
@ -0,0 +1,42 @@
|
||||
|
||||
#!/bin/bash
|
||||
|
||||
if [ x"$1" = x ] ; then
|
||||
echo "ERROR: You must specify a name to create branches and tags"
|
||||
exit 1
|
||||
fi
|
||||
branch=$1
|
||||
tag="v$branch"
|
||||
|
||||
|
||||
|
||||
echo "Finding subgits"
|
||||
SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
|
||||
|
||||
# Go through all subgits and create the branch and tag if they does not already exist
|
||||
for subgit in $SUBGITS; do
|
||||
echo ""
|
||||
echo ""
|
||||
pushd $subgit > /dev/null
|
||||
|
||||
# check if destination branch already exists
|
||||
echo "$subgit"
|
||||
echo "Pushing branch $branch"
|
||||
git push origin $branch:$branch
|
||||
if [ $? != 0 ] ; then
|
||||
echo "ERROR: Could not exec: git push origin $branch:$branch"
|
||||
popd > /dev/null
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Pushing tag $tag"
|
||||
git push origin $tag
|
||||
if [ $? != 0 ] ; then
|
||||
echo "ERROR: Could not exec: git push origin $tag"
|
||||
popd > /dev/null
|
||||
exit 1
|
||||
fi
|
||||
|
||||
popd > /dev/null
|
||||
done
|
||||
|
||||
29
build-tools/branching/push_tags.sh
Executable file
29
build-tools/branching/push_tags.sh
Executable file
@ -0,0 +1,29 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ x"$1" = x ] ; then
|
||||
echo "ERROR: You must specify a name to push tags"
|
||||
exit 1
|
||||
fi
|
||||
tag=$1
|
||||
|
||||
|
||||
echo "Finding subgits"
|
||||
SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
|
||||
|
||||
# Go through all subgits and create the tag if it does not already exist
|
||||
for subgit in $SUBGITS; do
|
||||
echo ""
|
||||
echo ""
|
||||
pushd $subgit > /dev/null
|
||||
|
||||
echo "Creating tag $tag"
|
||||
git push origin $tag
|
||||
if [ $? != 0 ] ; then
|
||||
echo "ERROR: Could not exec: git push origin $tag"
|
||||
popd > /dev/null
|
||||
exit 1
|
||||
fi
|
||||
|
||||
popd > /dev/null
|
||||
done
|
||||
|
||||
338
build-tools/build-guest
Executable file
338
build-tools/build-guest
Executable file
@ -0,0 +1,338 @@
|
||||
#!/bin/env bash
|
||||
|
||||
#
|
||||
# Build the tis-centos-image.img or tis-centos-image-rt.img file
|
||||
#
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
|
||||
# NOTE: TMP_DIR must end in '/'
|
||||
# NOTE: /tmp/ is now tmpfs like. Can't be trusted across multiple mock commands
|
||||
# TMP_DIR=/tmp/
|
||||
TMP_DIR=/
|
||||
|
||||
# Use RPMs from the std build only, for now
|
||||
export BUILD_TYPE=std
|
||||
export MY_BUILD_DIR_TOP=$MY_BUILD_DIR
|
||||
|
||||
function init_vars {
|
||||
# Output path (current dir unless MY_WORKSPACE defined)
|
||||
OUTPUT_DIR="$PWD/export"
|
||||
if [ ! -z "$MY_WORKSPACE" ] && [ -d "$MY_WORKSPACE" ] ; then
|
||||
OUTPUT_DIR="$MY_WORKSPACE/export"
|
||||
CGCS_REPO_DIR="$MY_WORKSPACE/rpmbuild/RPMS"
|
||||
fi
|
||||
|
||||
if [ -n "$MY_GUEST_DIR" ]; then
|
||||
GUEST_DIR=$MY_GUEST_DIR
|
||||
else
|
||||
GUEST_DIR=$MY_WORKSPACE/guest
|
||||
fi
|
||||
|
||||
MOCK=/usr/bin/mock
|
||||
if [ $VERBOSE -eq 0 ]; then
|
||||
MOCK="$MOCK -q"
|
||||
fi
|
||||
|
||||
# Path to guest configuration
|
||||
GUEST_BUILD_DIR=$DIR/build_guest
|
||||
GUEST_BUILD_CMD=$GUEST_BUILD_DIR/build-guest-image.py
|
||||
if [ $VERBOSE -eq 1 ]; then
|
||||
GUEST_BUILD_CMD="$GUEST_BUILD_CMD -x"
|
||||
fi
|
||||
|
||||
if [ $BUILD_MODE == 'std' ]; then
|
||||
OUTPUT_FILE=$OUTPUT_DIR/tis-centos-guest.img
|
||||
elif [ $BUILD_MODE == 'rt' ]; then
|
||||
OUTPUT_FILE=$OUTPUT_DIR/tis-centos-guest-rt.img
|
||||
else
|
||||
printf " Error -- unknown BUILD_MODE '$BUILD_MODE'\n";
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
function check_vars {
|
||||
# Where to store data
|
||||
printf "Finding cgcs-root\n"
|
||||
printf " Checking \$MY_REPO (value \"$MY_REPO\")\n"
|
||||
|
||||
if [ ! -z "$MY_REPO" ] && [ -d "$MY_REPO" ] ; then
|
||||
INTERNAL_REPO_ROOT=$MY_REPO
|
||||
printf " Found!\n"
|
||||
fi
|
||||
|
||||
if [ -z "$INTERNAL_REPO_ROOT" ] ; then
|
||||
printf " No joy -- checking \$MY_REPO_ROOT_DIR (value \"$MY_REPO_ROOT_DIR\")\n"
|
||||
if [ ! -z "$MY_REPO_ROOT_DIR" ] && [ -d "$MY_REPO_ROOT_DIR/cgcs-root" ] ; then
|
||||
INTERNAL_REPO_ROOT=$MY_REPO_ROOT_DIR/cgcs-root
|
||||
printf " Found!\n"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$INTERNAL_REPO_ROOT" ] ; then
|
||||
printf " No joy -- checking for \$MY_WORKSPACE/cgcs-root\n"
|
||||
if [ -d "$MY_WORKSPACE/cgcs-root" ] ; then
|
||||
INTERNAL_REPO_ROOT=$MY_WORKSPACE/cgcs-root
|
||||
printf " Found!\n"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$INTERNAL_REPO_ROOT" ] ; then
|
||||
printf " Error -- could not locate cgcs-root repo.\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "x$MY_BUILD_CFG" == "x" ];then
|
||||
printf " Error -- reqiure MY_BUILD_CFG to be defined.\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
RELEASE_INFO=$INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/middleware/recipes-common/build-info/release-info.inc
|
||||
export PLATFORM_RELEASE=$(source $RELEASE_INFO && echo $PLATFORM_RELEASE)
|
||||
}
|
||||
|
||||
|
||||
function create_rootfs {
|
||||
printf "\nCreating guest file system\n"
|
||||
|
||||
mkdir -p $GUEST_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Could not create $GUEST_DIR\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Place build-time environment variables in mock configuration
|
||||
GUEST_ENV="${MY_BUILD_ENVIRONMENT}-guest"
|
||||
GUEST_CFG=$GUEST_DIR/$MY_BUILD_ENVIRONMENT_FILE
|
||||
|
||||
MY_BUILD_ENVIRONMENT=$GUEST_ENV ${DIR}/modify-build-cfg $GUEST_CFG
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Could not update $GUEST_CFG\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Setup mock directories for the guest
|
||||
if [ -d /localdisk/loadbuild/mock ]; then
|
||||
LNK=/localdisk/loadbuild/mock/$GUEST_ENV
|
||||
if [ ! -L $LNK ]; then
|
||||
ln -s $GUEST_DIR $LNK
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -d /localdisk/loadbuild/mock-cache ]; then
|
||||
mkdir -p $GUEST_DIR/cache
|
||||
LNK=/localdisk/loadbuild/mock-cache/$GUEST_ENV
|
||||
if [ ! -L $LNK ]; then
|
||||
ln -s $GUEST_DIR/cache $LNK
|
||||
fi
|
||||
fi
|
||||
|
||||
# Setup mock chroot environment
|
||||
$MOCK -r $GUEST_CFG --clean && $MOCK -r $GUEST_CFG --init
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Failed to setup guest mock chroot\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Install the RPMs to the root filesystem
|
||||
|
||||
# Note that the "rt" build needs access to both local-std and local-rt repos
|
||||
local EXTRA_REPOS=""
|
||||
|
||||
if [ $BUILD_MODE == 'std' ]; then
|
||||
INC_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/rpm-install-list.txt)
|
||||
TIS_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/image.inc)
|
||||
elif [ $BUILD_MODE == 'rt' ]; then
|
||||
INC_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/rpm-install-list-rt.txt)
|
||||
TIS_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/image-rt.inc)
|
||||
EXTRA_REPOS="--enablerepo local-rt"
|
||||
else
|
||||
printf " Error -- unknown BUILD_MODE '$BUILD_MODE'\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
$MOCK -r $GUEST_CFG --install ${INC_RPM_LIST} ${TIS_RPM_LIST} ${EXTRA_REPOS}
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Failed to install RPM packages\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Remove RPMs that are not required in image (pruned package list)
|
||||
# NOTE: these are automatically installed from the mock init not
|
||||
# through dependencies.
|
||||
EXC_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/rpm-remove-list.txt)
|
||||
|
||||
$MOCK -r $GUEST_CFG --remove ${EXC_RPM_LIST}
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Failed to remove RPM packages\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
printf " Done\n"
|
||||
}
|
||||
|
||||
|
||||
function update_rootfs {
|
||||
printf "\nCustomizing guest file system\n"
|
||||
|
||||
# Copy over skeleton configuration files
|
||||
for GUEST_ROOTFS in $GUEST_BUILD_DIR/rootfs $GUEST_BUILD_DIR/rootfs-$BUILD_MODE;
|
||||
do
|
||||
for f in $(cd $GUEST_ROOTFS && find . -type f | cut -c3-);
|
||||
do
|
||||
echo "$MOCK -r $GUEST_CFG --copyin $GUEST_ROOTFS/$f $f"
|
||||
$MOCK -r $GUEST_CFG --copyin $GUEST_ROOTFS/$f $f
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Failed to copyin file $f\n";
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
done
|
||||
|
||||
# Run the root file system setup script inside the chroot
|
||||
ROOTFS_SETUP=rootfs-setup.sh
|
||||
$MOCK -r $GUEST_CFG --copyin $GUEST_BUILD_DIR/$ROOTFS_SETUP $TMP_DIR && \
|
||||
if [ $BUILD_MODE == 'rt' ]; then
|
||||
ROOTFS_SETUP_CMD="$TMP_DIR$ROOTFS_SETUP --rt"
|
||||
elif [ $BUILD_MODE == 'std' ]; then
|
||||
ROOTFS_SETUP_CMD="$TMP_DIR$ROOTFS_SETUP --std"
|
||||
else
|
||||
ROOTFS_SETUP_CMD="$TMP_DIR$ROOTFS_SETUP"
|
||||
fi
|
||||
$MOCK -r $GUEST_CFG --chroot "$ROOTFS_SETUP_CMD"
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Failed to run guest $ROOTFS_SETUP\n";
|
||||
exit 1
|
||||
fi
|
||||
$MOCK -r $GUEST_CFG --chroot "rm -f $TMP_DIR$ROOTFS_SETUP"
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Failed to delete $ROOTFS_SETUP from guest\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
printf " Done\n"
|
||||
}
|
||||
|
||||
|
||||
function build_image {
|
||||
# Build the image
|
||||
printf "\nBuilding guest image $OUTPUT_FILE\n"
|
||||
|
||||
mkdir -p $OUTPUT_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Could not create $OUTPUT_DIR\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Build guest rootfs archive
|
||||
ROOTFS_SPACE=$((500*1024*1024))
|
||||
ROOTFS_TAR=rootfs.tar
|
||||
ROOTFS_EXCLUDE=rootfs-exclude.txt
|
||||
|
||||
$MOCK -r $GUEST_CFG --copyin $GUEST_BUILD_DIR/$ROOTFS_EXCLUDE $TMP_DIR
|
||||
$MOCK -r $GUEST_CFG --chroot -- tar -cf $TMP_DIR$ROOTFS_TAR -X $TMP_DIR$ROOTFS_EXCLUDE --numeric-owner /
|
||||
$MOCK -r $GUEST_CFG --copyout $TMP_DIR$ROOTFS_TAR $GUEST_DIR
|
||||
$MOCK -r $GUEST_CFG --chroot -- rm -f $TMP_DIR$ROOTFS_TAR
|
||||
|
||||
$GUEST_BUILD_CMD -i $GUEST_DIR/$ROOTFS_TAR -o $OUTPUT_FILE -s $ROOTFS_SPACE
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Failed to build guest image\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
printf " Done\n"
|
||||
}
|
||||
|
||||
|
||||
function clean_guest {
|
||||
printf "\nCleaning the guest $GUEST_DIR\n"
|
||||
|
||||
if [ ! -e $GUEST_DIR ]; then
|
||||
printf " Done...nothing to do\n";
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Place build-time environment variables in mock configuration
|
||||
GUEST_ENV="${MY_BUILD_ENVIRONMENT}-guest"
|
||||
GUEST_CFG=$GUEST_DIR/$MY_BUILD_ENVIRONMENT_FILE
|
||||
|
||||
if [ ! -e $GUEST_CFG ]; then
|
||||
MY_BUILD_ENVIRONMENT=$GUEST_ENV ${DIR}/modify-build-cfg $GUEST_CFG
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Could not update $GUEST_CFG\n";
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
$MOCK -r $GUEST_CFG --clean
|
||||
$MOCK -r $GUEST_CFG --scrub=cache
|
||||
|
||||
rm -rf $GUEST_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
printf " Error -- Failed to remove guest $GUEST_DIR\n";
|
||||
exit 1
|
||||
fi
|
||||
|
||||
printf " Done\n"
|
||||
}
|
||||
|
||||
#############################################
|
||||
# Main code
|
||||
#############################################
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " build-guest [--rt | --std] [--verbose]"
|
||||
echo " build-guest [--help]"
|
||||
echo " build-guest [--clean]"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# Default argument values
|
||||
HELP=0
|
||||
CLEAN=0
|
||||
VERBOSE=0
|
||||
BUILD_MODE='std'
|
||||
|
||||
# read the options
|
||||
TEMP=`getopt -o h --long clean --long rt --long std --long verbose -n 'test.sh' -- "$@"`
|
||||
eval set -- "$TEMP"
|
||||
|
||||
# extract options and their arguments into variables.
|
||||
while true ; do
|
||||
case "$1" in
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
--clean) CLEAN=1 ; shift ;;
|
||||
--verbose) VERBOSE=1 ; shift ;;
|
||||
--rt) BUILD_MODE='rt' ; shift ;;
|
||||
--std) BUILD_MODE='std' ; shift ;;
|
||||
--) shift ; break ;;
|
||||
*) echo "Internal error!" ; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
(
|
||||
printf "\n*****************************\n"
|
||||
printf "Create Titanium Cloud/CentOS Guest Image\n"
|
||||
printf "*****************************\n\n"
|
||||
|
||||
init_vars
|
||||
check_vars
|
||||
|
||||
if [ $CLEAN -eq 1 ]; then
|
||||
clean_guest
|
||||
exit 0
|
||||
fi
|
||||
|
||||
create_rootfs
|
||||
update_rootfs
|
||||
build_image
|
||||
|
||||
) 2>&1 | stdbuf -o0 awk '{ print strftime("%H:%M:%S"), $0; fflush(); }' ; exit ${PIPESTATUS[0]}
|
||||
94
build-tools/build-img
Executable file
94
build-tools/build-img
Executable file
@ -0,0 +1,94 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Build an IMG file capable of being booted in a virtual environment
|
||||
# The default settings are vda device which the Cumulus environment expects
|
||||
# and controller mode
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " build-img [--cpe] [--dest <filename>] [--part [1 | 2]]"
|
||||
echo " --dest <tis.img>"
|
||||
echo " --cpe Boots in CPE mode. Default is controller mode."
|
||||
echo ""
|
||||
}
|
||||
|
||||
DEST_ISO=bootimage_auto.iso
|
||||
DEST_IMG=tis.img
|
||||
AUTO_MODE=controller
|
||||
HELP=0
|
||||
PART=0
|
||||
|
||||
# read the options
|
||||
TEMP=`getopt -o hp:d: --long help,cpe,part:,dest: -n 'test.sh' -- "$@"`
|
||||
eval set -- "$TEMP"
|
||||
|
||||
# extract options and their arguments into variables.
|
||||
while true ; do
|
||||
case "$1" in
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
--cpe) AUTO_MODE=cpe; shift ;;
|
||||
-d | --dest) DEST_IMG="$2"; shift; shift ;;
|
||||
-p | --part) PART="$2"; shift; shift ;;
|
||||
--) shift ; break ;;
|
||||
*) echo "Internal error!" ; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo PART=$PART
|
||||
|
||||
# Step 1: Build an ISO that autoboots
|
||||
|
||||
# Cumulus default device is vda
|
||||
if [ $PART -ne 2 ]; then
|
||||
build-iso --file bootimage_auto.iso --auto $AUTO_MODE --device vda --cumulus
|
||||
fi
|
||||
|
||||
# Step 2: Convert the ISO to IMG
|
||||
if [ $PART -ne 1 ]; then
|
||||
if [ ! -e "/dev/loop-control" -o ! -e "/dev/kvm" ]; then
|
||||
CMD="cd $MY_WORKSPACE/export; \
|
||||
$MY_REPO/addons/wr-cgcs/layers/cgcs/extras.ND/scripts/install_iso_to_disk_image.sh bootimage_auto.iso $DEST_IMG"
|
||||
|
||||
if [ "$HOSTNAME" == "yow-cgts3-centos7" ]; then
|
||||
echo "Attempting to run kvm_iso_to_img on yow-cgts3-lx"
|
||||
ssh -o StrictHostKeyChecking=no yow-cgts3-lx "$CMD"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to run update-efiboot-image on yow-cgts3-lx"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$HOSTNAME" == "yow-cgts2-centos7" ]; then
|
||||
echo "Attempting to run kvm_iso_to_img on yow-cgts2-lx"
|
||||
ssh -o StrictHostKeyChecking=no yow-cgts2-lx "$CMD"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to run update-efiboot-image on yow-cgts2-lx"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -f "$MY_WORKSPACE/export/$DEST_IMG" ]; then
|
||||
printf "\n"
|
||||
printf "****************************************************************** \n"
|
||||
printf "No kvm and/or loop device on this machine. To complete the build \n"
|
||||
printf "please copy '$MY_WORKSPACE/export/bootimage_auto.iso' to a machine \n"
|
||||
printf "that supports kvm and loop devices and run ... \n"
|
||||
printf " $MY_REPO/addons/wr-cgcs/layers/cgcs/extras.ND/scripts/install_iso_to_disk_image.sh bootimage_auto.iso $DEST_IMG\n"
|
||||
printf "****************************************************************** \n"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -f "$MY_WORKSPACE/export/$DEST_IMG" ]; then
|
||||
(
|
||||
cd $MY_WORKSPACE/export
|
||||
$MY_REPO/addons/wr-cgcs/layers/cgcs/extras.ND/scripts/install_iso_to_disk_image.sh bootimage_auto.iso $DEST_IMG
|
||||
exit $?
|
||||
)
|
||||
fi
|
||||
fi
|
||||
|
||||
702
build-tools/build-iso
Executable file
702
build-tools/build-iso
Executable file
@ -0,0 +1,702 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Build the export/bootimage.iso file
|
||||
#
|
||||
# This script uses environment variables to determine the source of
|
||||
# packages, and bundles the packages into a bootable .iso
|
||||
#
|
||||
# It starts by building a basic "vanilla CentOS" ISO, and then adds our
|
||||
# packages to it.
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " build-iso [--auto <mode>] [--file <filename>] [--device <device>] [--skip-sign]"
|
||||
echo " --file <bootimage.iso> destination ISO file"
|
||||
echo " --auto <controller|cpe> Modify kickstart to auto-install controller or cpe mode"
|
||||
echo " --device <sda> Use a different boot/rootds device (default is sda)"
|
||||
echo " --skip-sign do not add file signature to RPMs"
|
||||
echo ""
|
||||
}
|
||||
|
||||
MY_YUM_CONF=""
|
||||
|
||||
NPROCS=$(nproc)
|
||||
|
||||
export MOCK=/usr/bin/mock
|
||||
|
||||
CREATEREPO=$(which createrepo_c)
|
||||
if [ $? -ne 0 ]; then
|
||||
CREATEREPO="createrepo"
|
||||
fi
|
||||
|
||||
# TEMPORARY: Check for isohybrid now to give a warning about installing pkg
|
||||
if [ ! -f /usr/bin/isohybrid ]; then
|
||||
echo "Missing required utility: /usr/bin/isohybrid"
|
||||
echo "Installation of syslinux is required:"
|
||||
echo " sudo yum install -y syslinux"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
function install_pkg_list {
|
||||
local PKGLIST=$1
|
||||
if [ "x$PKGLIST" == "x" ]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
OLD_PWD=$PWD
|
||||
|
||||
echo "Installing packages listed in $PKGLIST and dependancies"
|
||||
\rm -f $OUTPUT_DIR/dist/report_deps.txt
|
||||
$CREATEREPO $CGCS_REPO_DIR
|
||||
$CREATEREPO $CGCS_RT_REPO_DIR
|
||||
|
||||
\cp -v $MY_YUM_CONF $OUTPUT_DIR
|
||||
|
||||
\cd $OUTPUT_DIST_DIR/isolinux/Packages
|
||||
$INTERNAL_REPO_ROOT/build-tools/build_iso/cgts_deps.sh --deps=$PKGLIST
|
||||
|
||||
if [ $? -ne 0 ]
|
||||
then
|
||||
echo "Could not install dependencies"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# clean up
|
||||
echo "Removing local-std yum repo $CGCS_REPO_DIR/repodata"
|
||||
echo "Removing local-rt yum repo $CGCS_RT_REPO_DIR/repodata"
|
||||
|
||||
\cd $OLD_PWD
|
||||
}
|
||||
|
||||
# Generate the report of where all packages come from
|
||||
function make_report {
|
||||
local PKGLISTFILES=$@
|
||||
if [ "x$PKGLISTFILES" == "x" ]; then
|
||||
return 1
|
||||
fi
|
||||
echo "MAKING $REPORT_FILE"
|
||||
echo "-----------------" >> $REPORT_FILE
|
||||
|
||||
echo "ISO REPORT" > $REPORT_FILE
|
||||
date >> $REPORT_FILE
|
||||
echo "-----------------" >> $REPORT_FILE
|
||||
|
||||
echo " " >> $REPORT_FILE
|
||||
echo "-----------------" >> $REPORT_FILE
|
||||
echo "EXPLICIT INCLUDES" >> $REPORT_FILE
|
||||
echo "-----------------" >> $REPORT_FILE
|
||||
for PKGLIST in $PKGLISTFILES; do
|
||||
while read PKG; do
|
||||
PKG=`echo $PKG | sed "s/#.*//"`;
|
||||
if [ "${PKG}x" != "x" ]; then
|
||||
echo $PKG >> $REPORT_FILE
|
||||
fi
|
||||
done < $PKGLIST
|
||||
done
|
||||
|
||||
echo " " >> $REPORT_FILE
|
||||
echo "-----------------" >> $REPORT_FILE
|
||||
echo " PACKAGES " >> $REPORT_FILE
|
||||
echo "-----------------" >> $REPORT_FILE
|
||||
cat $BUILT_REPORT | sort | uniq >> $REPORT_FILE
|
||||
|
||||
echo " " >> $REPORT_FILE
|
||||
echo "-----------------" >> $REPORT_FILE
|
||||
echo " WARNINGS " >> $REPORT_FILE
|
||||
echo "-----------------" >> $REPORT_FILE
|
||||
|
||||
# Note that the warnings file may have multiple lines for the same
|
||||
# missing dependency. A sort | uniq solves this so we don't duplicate
|
||||
# warnings
|
||||
cat $WARNINGS_REPORT | sort | uniq >> $REPORT_FILE
|
||||
|
||||
echo "ISO REPORT: $REPORT_FILE"
|
||||
}
|
||||
|
||||
function init_vars {
|
||||
#####################################
|
||||
# Input definitions
|
||||
|
||||
# Where all CentOS packages live
|
||||
# Where essential CentOS (minimal install) packages live
|
||||
INTERNAL_REPO_ROOT=
|
||||
# Where our own packages live
|
||||
CGCS_REPO_DIR=$MY_WORKSPACE/std/rpmbuild/RPMS
|
||||
CGCS_RT_REPO_DIR=$MY_WORKSPACE/rt/rpmbuild/RPMS
|
||||
|
||||
MY_YUM_CONF=$(create-yum-conf)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: create-yum-conf failed"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
DISTRO_REPO_DIR=$(for d in $(grep baseurl $MY_YUM_CONF | grep file: | awk -F : '{print $2}' | sed 's:///:/:g'); do if [ -d $d/images ]; then echo $d ;fi; done)
|
||||
|
||||
#####################################
|
||||
# Output definitons
|
||||
|
||||
# where to put stuff (curent dir unless MY_WORKSPACE defined)
|
||||
OUTPUT_DIR="$PWD/export"
|
||||
if [ ! -z "$MY_WORKSPACE" ] && [ -d "$MY_WORKSPACE" ] ; then
|
||||
OUTPUT_DIR="$MY_WORKSPACE/export"
|
||||
CGCS_REPO_DIR="$MY_WORKSPACE/std/rpmbuild/RPMS"
|
||||
CGCS_RT_REPO_DIR="$MY_WORKSPACE/rt/rpmbuild/RPMS"
|
||||
fi
|
||||
|
||||
# Directory in which to populate files to be distributed
|
||||
if [ $CUMULUS -eq 0 ]; then
|
||||
OUTPUT_DIST_DIR=$OUTPUT_DIR/dist
|
||||
else
|
||||
OUTPUT_DIST_DIR=$OUTPUT_DIR/dist-cumulus
|
||||
fi
|
||||
|
||||
# Package disc image
|
||||
OUTPUT_FILE=$OUTPUT_DIR/$DEST_FILE
|
||||
|
||||
# Generate an error if the output file is below this threshold
|
||||
MINIMUM_EXPECTED_SIZE=500000000
|
||||
|
||||
|
||||
# report variables
|
||||
REPORT_FILE=$OUTPUT_DIR/report.txt
|
||||
BUILT_REPORT=$OUTPUT_DIR/local.txt
|
||||
CLOUD_REPORT=$OUTPUT_DIR/cloud.txt
|
||||
CLOUD_COMMON_REPORT=$OUTPUT_DIR/cloudcommon.txt
|
||||
CENTOS_REPORT=$OUTPUT_DIR/centos.txt
|
||||
EPEL_REPORT=$OUTPUT_DIR/epel.txt
|
||||
WARNINGS_REPORT=$OUTPUT_DIR/warnings.txt
|
||||
|
||||
\rm -f $REPORT_FILE
|
||||
\rm -f $BUILT_REPORT
|
||||
\rm -f $CLOUD_REPORT
|
||||
\rm -f $CLOUD_COMMON_REPORT
|
||||
\rm -f $CENTOS_REPORT
|
||||
\rm -f $WARNINGS_REPORT
|
||||
}
|
||||
|
||||
# check input variables
|
||||
function check_vars {
|
||||
# Where to store data
|
||||
printf "Finding cgcs-root\n"
|
||||
printf " Checking \$MY_REPO (value \"$MY_REPO\")\n"
|
||||
|
||||
if [ ! -z "$MY_REPO" ] && [ -d "$MY_REPO" ] ; then
|
||||
INTERNAL_REPO_ROOT=$MY_REPO
|
||||
printf " Found!\n"
|
||||
fi
|
||||
|
||||
if [ -z "$INTERNAL_REPO_ROOT" ] ; then
|
||||
printf " No joy -- checking \$MY_REPO_ROOT_DIR (value \"$MY_REPO_ROOT_DIR\")\n"
|
||||
if [ ! -z "$MY_REPO_ROOT_DIR" ] && [ -d "$MY_REPO_ROOT_DIR/cgcs-root" ] ; then
|
||||
INTERNAL_REPO_ROOT=$MY_REPO_ROOT_DIR/cgcs-root
|
||||
printf " Found!\n"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$INTERNAL_REPO_ROOT" ] ; then
|
||||
printf " No joy -- checking for \$MY_WORKSPACE/cgcs-root\n"
|
||||
if [ -d "$MY_WORKSPACE/cgcs-root" ] ; then
|
||||
INTERNAL_REPO_ROOT=$MY_WORKSPACE/cgcs-root
|
||||
printf " Found!\n"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$INTERNAL_REPO_ROOT" ] ; then
|
||||
printf " Error -- could not locate cgcs-root repo.\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
printf "\nChecking that we can access $DISTRO_REPO_DIR\n"
|
||||
if [ ! -d "$DISTRO_REPO_DIR" ] ; then
|
||||
printf " Error -- could not access $DISTRO_REPO_DIR\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -e "$DISTRO_REPO_DIR/repodata" ] ; then
|
||||
printf " Error -- $DISTRO_REPO_DIR is there, but does not seem sane\n"
|
||||
fi
|
||||
|
||||
printf "\nOkay, input looks fine...\n\n"
|
||||
printf "Creating output directory $OUTPUT_DIST_DIR\n"
|
||||
if [ $CLEAN_FLAG -eq 1 ]; then
|
||||
echo " Cleaning..."
|
||||
if [ -e $OUTPUT_DIST_DIR ] ; then
|
||||
chmod -R a+w $OUTPUT_DIST_DIR
|
||||
\rm -rf $OUTPUT_DIST_DIR
|
||||
fi
|
||||
if [ -e $OUTPUT_DIST_DIR ] ; then
|
||||
printf "Error: could not remove old $OUTPUT_DIST_DIR\n"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
\mkdir -p $OUTPUT_DIST_DIR
|
||||
if [ ! -d $OUTPUT_DIST_DIR ] ; then
|
||||
printf "Error: could not create $OUTPUT_DIST_DIR\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
RELEASE_INFO=$INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/middleware/recipes-common/build-info/release-info.inc
|
||||
export PLATFORM_RELEASE=$(source $RELEASE_INFO && echo $PLATFORM_RELEASE)
|
||||
|
||||
echo " Done"
|
||||
echo ""
|
||||
}
|
||||
|
||||
function init_output_dir {
|
||||
echo "Creating base output directory in $OUTPUT_DIST_DIR"
|
||||
\mkdir -p $OUTPUT_DIST_DIR/isolinux/images
|
||||
\mkdir -p $OUTPUT_DIST_DIR/isolinux/ks
|
||||
\mkdir -p $OUTPUT_DIST_DIR/isolinux/LiveOS
|
||||
\mkdir -p $OUTPUT_DIST_DIR/isolinux/Packages
|
||||
\mkdir -p $OUTPUT_DIST_DIR/utils
|
||||
|
||||
\mkdir -p $OUTPUT_DIST_DIR/isolinux/EFI
|
||||
# This directory will contains files required for the PXE network installer
|
||||
\mkdir -p $OUTPUT_DIST_DIR/isolinux/pxeboot
|
||||
|
||||
echo " Copying base files"
|
||||
|
||||
# Generate .discinfo file
|
||||
date +%s.%N > $OUTPUT_DIST_DIR/isolinux/.discinfo
|
||||
echo $PLATFORM_RELEASE >> $OUTPUT_DIST_DIR/isolinux/.discinfo
|
||||
echo "x86_64" >> $OUTPUT_DIST_DIR/isolinux/.discinfo
|
||||
|
||||
\cp -L -ru $DISTRO_REPO_DIR/isolinux/* $OUTPUT_DIST_DIR/isolinux/
|
||||
\cp -L -ru $DISTRO_REPO_DIR/images/pxeboot $OUTPUT_DIST_DIR/isolinux/images/
|
||||
|
||||
echo " Installing startup files"
|
||||
|
||||
\cp -L $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/centos.syslinux.cfg $OUTPUT_DIST_DIR/isolinux/syslinux.cfg
|
||||
\cp -L $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/centos.syslinux.cfg $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
|
||||
sed -i 's/wr_usb_boot/oe_iso_boot/' $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
|
||||
|
||||
# Modify the isolinux.cfg to auto install if requested
|
||||
# Option 0 is Controller(serial). Option 2 is CPE serial.
|
||||
if [ "$AUTO_INSTALL" == "controller" ] ; then
|
||||
echo "Modifying ISO to auto-install controller load"
|
||||
perl -p -i -e 's/timeout 0/timeout 1\ndefault 0/' $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
|
||||
elif [ "$AUTO_INSTALL" == "cpe" ] ; then
|
||||
echo "Modifying ISO to auto-install CPE (combined load)"
|
||||
perl -p -i -e 's/timeout 0/timeout 1\ndefault 2/' $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
|
||||
fi
|
||||
|
||||
# Modify the device if requested
|
||||
if [ ! -z "$DEVICE" ] ; then
|
||||
echo "Modifying ISO to use device $DEVICE"
|
||||
perl -p -i -e "s/device=sda/device=${DEVICE}/g" $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
|
||||
fi
|
||||
|
||||
|
||||
|
||||
# Copy UEFI files
|
||||
\cp -L -ru $DISTRO_REPO_DIR/EFI/* $OUTPUT_DIST_DIR/isolinux/EFI/
|
||||
\cp -L $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/grub.cfg $OUTPUT_DIST_DIR/isolinux/EFI/BOOT/grub.cfg
|
||||
\cp -L $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/pxeboot_grub.cfg $OUTPUT_DIST_DIR/isolinux/pxeboot/pxeboot_grub.cfg
|
||||
|
||||
# Update the efiboot.img (See https://wiki.archlinux.org/index.php/Remastering_the_Install_ISO)
|
||||
# We need to mount the image file, replace the grub.cfg file with the Titanium Cloud one, and unmount.
|
||||
# Script update-efiboot-image will do this. If there is not loop device on the build machine
|
||||
# then this script must be executed manually prior.
|
||||
|
||||
if [ ! -e "/dev/loop-control" -a ! -f "$OUTPUT_DIR/efiboot.img" ]; then
|
||||
CMD="export PROJECT=$PROJECT; \
|
||||
export SRC_BUILD_ENVIRONMENT=$SRC_BUILD_ENVIRONMENT; \
|
||||
export MY_BUILD_ENVIRONMENT=$MY_BUILD_ENVIRONMENT; \
|
||||
export MY_BUILD_ENVIRONMENT_FILE=$MY_BUILD_ENVIRONMENT_FILE; \
|
||||
export MY_BUILD_DIR=$MY_BUILD_DIR; \
|
||||
export MY_WORKSPACE=$MY_WORKSPACE; \
|
||||
export MY_REPO=$MY_REPO; \
|
||||
export MY_BUILD_CFG=$MY_BUILD_CFG; \
|
||||
export MY_MOCK_ROOT=$MY_MOCK_ROOT; \
|
||||
export PATH=$MY_REPO/build-tools:\$PATH; \
|
||||
update-efiboot-image"
|
||||
|
||||
if [ "$HOSTNAME" == "yow-cgts3-centos7" ]; then
|
||||
echo "Attempting to run update-efiboot-image on yow-cgts3-lx"
|
||||
ssh -o StrictHostKeyChecking=no yow-cgts3-lx "$CMD"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to run update-efiboot-image on yow-cgts3-lx"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$HOSTNAME" == "yow-cgts2-centos7" ]; then
|
||||
echo "Attempting to run update-efiboot-image on yow-cgts2-lx"
|
||||
ssh -o StrictHostKeyChecking=no yow-cgts2-lx "$CMD"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to run update-efiboot-image on yow-cgts2-lx"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -e "/dev/loop-control" -a ! -f "$OUTPUT_DIR/efiboot.img" ]; then
|
||||
printf "\n**************************************************************************************************** \n"
|
||||
printf "No loop device on this machine. Please ensure $OUTPUT_DIR/efiboot.img \n"
|
||||
printf "exist prior to executing build-iso by. It can be created by running \n"
|
||||
printf " $INTERNAL_REPO_ROOT/build-tools/update-efiboot-image \n"
|
||||
printf "on a machine that does support a loop device. Please ensure all standard \n"
|
||||
printf "build environment variables are defined (e.g. MY_REPO, MY_WORKSPACE, etc.). \n"
|
||||
printf " \n"
|
||||
printf "e.g. If building on yow-cgts3-centos7, you'll want to run the script on \n"
|
||||
printf " yow-cgts3-lx which shares the same file system, but supports loop devices \n"
|
||||
printf "****************************************************************************************************** \n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -f "$OUTPUT_DIR/efiboot.img" ]; then
|
||||
|
||||
# The script update-efiboot-image was run outside the build-iso script, do nothing.
|
||||
printf " The image file $OUTPUT_DIR/efiboot.img already exist\n"
|
||||
else
|
||||
printf " The image file $OUTPUT_DIR/efiboot.img does not exist \n"
|
||||
if [ ! -f "$INTERNAL_REPO_ROOT/build-tools/update-efiboot-image" ]; then
|
||||
printf "*** Error: script update-efiboot-image does not exist *** \n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Run the script
|
||||
$INTERNAL_REPO_ROOT/build-tools/update-efiboot-image
|
||||
RET=$?
|
||||
if [ $RET != 0 ]; then
|
||||
printf "*** Error: update-efiboot-image script returned failure $RET *** \n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
fi
|
||||
|
||||
\cp -L $OUTPUT_DIR/efiboot.img $OUTPUT_DIST_DIR/isolinux/images/
|
||||
\rm -f $OUTPUT_DIR/efiboot.img
|
||||
|
||||
# Copy and set up pxeboot setup files
|
||||
\cp $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/pxeboot_setup.sh $OUTPUT_DIST_DIR/isolinux/pxeboot_setup.sh
|
||||
\cp $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/pxeboot.cfg $OUTPUT_DIST_DIR/isolinux/pxeboot/pxeboot.cfg
|
||||
chmod +x $OUTPUT_DIST_DIR/isolinux/pxeboot_setup.sh
|
||||
|
||||
\rm -f $OUTPUT_DIST_DIR/comps.xml
|
||||
\cp -L $INTERNAL_REPO_ROOT/build-tools/build_iso/comps.xml.gz $OUTPUT_DIST_DIR/
|
||||
gunzip $OUTPUT_DIST_DIR/comps.xml.gz
|
||||
|
||||
TMP_DIR=$MY_WORKSPACE/tmp
|
||||
\mkdir -p $TMP_DIR
|
||||
TMPDIR=$TMP_DIR yum clean all -c $MY_YUM_CONF
|
||||
\rm -rf $TMP_DIR/yum-$USER-*
|
||||
echo " Done"
|
||||
echo ""
|
||||
}
|
||||
|
||||
function final_touches {
|
||||
OLD_PWD=$PWD
|
||||
|
||||
# Update the comps.xml
|
||||
if [ ! -f $OUTPUT_DIST_DIR/comps.xml.bak ]; then
|
||||
\cp $OUTPUT_DIST_DIR/comps.xml $OUTPUT_DIST_DIR/comps.xml.bak
|
||||
fi
|
||||
|
||||
python $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/platform_comps.py \
|
||||
--pkgdir $OUTPUT_DIST_DIR/isolinux/Packages \
|
||||
--groups $OUTPUT_DIST_DIR/comps.xml
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to update comps.xml"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# create the repo
|
||||
\cd $OUTPUT_DIST_DIR/isolinux
|
||||
$CREATEREPO -q -g ../comps.xml .
|
||||
|
||||
# build the ISO
|
||||
printf "Building image $OUTPUT_FILE\n"
|
||||
\cd $OUTPUT_DIST_DIR
|
||||
chmod 664 isolinux/isolinux.bin
|
||||
mkisofs -o $OUTPUT_FILE \
|
||||
-R -D -A 'oe_iso_boot' -V 'oe_iso_boot' \
|
||||
-quiet \
|
||||
-b isolinux.bin -c boot.cat -no-emul-boot \
|
||||
-boot-load-size 4 -boot-info-table \
|
||||
-eltorito-alt-boot \
|
||||
-e images/efiboot.img \
|
||||
-no-emul-boot \
|
||||
isolinux/
|
||||
|
||||
isohybrid --uefi $OUTPUT_FILE
|
||||
implantisomd5 $OUTPUT_FILE
|
||||
|
||||
\cd $OLD_PWD
|
||||
}
|
||||
|
||||
function extract_pkg_from_local_repo {
|
||||
local repodir=$1
|
||||
local pkgname=$2
|
||||
|
||||
local pkgfile=$(repoquery --repofrompath local,${repodir} --location -q ${pkgname})
|
||||
if [ -z "${pkgfile}" ]; then
|
||||
echo "Could not find package $pkgname in $repodir"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
rpm2cpio ${pkgfile/file://} | cpio -idmv
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to extract files from ${pkgfile/file://}"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
function extract_installer_files {
|
||||
# Changes to copied files here must also be reflected in patch-iso
|
||||
|
||||
CGCSDIR=$INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs
|
||||
PKGDIR=$OUTPUT_DIST_DIR/isolinux/Packages
|
||||
|
||||
(
|
||||
\cd $OUTPUT_DIR
|
||||
\rm -rf kickstarts extra_cfgs kickstart.work
|
||||
\mkdir kickstarts extra_cfgs kickstart.work
|
||||
|
||||
echo "Retrieving kickstarts..."
|
||||
|
||||
\cd kickstart.work
|
||||
|
||||
extract_pkg_from_local_repo ${CGCS_REPO_DIR} platform-kickstarts
|
||||
extract_pkg_from_local_repo ${CGCS_REPO_DIR} platform-kickstarts-pxeboot
|
||||
extract_pkg_from_local_repo ${CGCS_REPO_DIR} platform-kickstarts-extracfgs
|
||||
|
||||
\cp --preserve=all www/pages/feed/rel-*/*.cfg pxeboot/*.cfg ../kickstarts/ &&
|
||||
\cp --preserve=all extra_cfgs/*.cfg ../extra_cfgs/
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to copy extracted kickstarts"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
\cd ..
|
||||
|
||||
# Copy kickstarts to ISO
|
||||
\cp --preserve=all kickstarts/controller_ks.cfg $OUTPUT_DIST_DIR/isolinux/ks.cfg
|
||||
# Modify the kickstart to shutdown instead of reboot if doing an auto install
|
||||
if [ ! -z "$AUTO_INSTALL" ] ; then
|
||||
sed -i 's/^reboot --eject/shutdown/' $OUTPUT_DIST_DIR/isolinux/ks.cfg
|
||||
fi
|
||||
|
||||
\mv kickstarts/pxeboot* $OUTPUT_DIST_DIR/isolinux/pxeboot/
|
||||
\cp --preserve=all kickstarts/* $OUTPUT_DIST_DIR/isolinux
|
||||
|
||||
# Update OAM interface for cumulus auto install
|
||||
if [ $CUMULUS -eq 1 ]; then
|
||||
# Cumulus wants tty1
|
||||
perl -p -i -e 's/console=tty0/console=tty1/' $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
|
||||
|
||||
# CUMULUS setup scripts specify ens3 for OAM
|
||||
OAM_IFNAME=ens3
|
||||
|
||||
cat <<EOM >> $OUTPUT_DIST_DIR/isolinux/ks.cfg
|
||||
%post
|
||||
#For cumulus tis on tis automated install
|
||||
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-${OAM_IFNAME}
|
||||
IPADDR=10.10.10.3
|
||||
NETMASK=255.255.255.0
|
||||
BOOTPROTO=static
|
||||
ONBOOT=yes
|
||||
DEVICE=${OAM_IFNAME}
|
||||
MTU=1500
|
||||
GATEWAY=10.10.10.1
|
||||
EOF
|
||||
%end
|
||||
EOM
|
||||
fi
|
||||
|
||||
# For PXE boot network installer
|
||||
|
||||
echo ${OUTPUT_DIST_DIR}/isolinux/Packages
|
||||
|
||||
local WORKDIR=pxe-network-installer.content
|
||||
local ORIG_PWD=$PWD
|
||||
|
||||
\rm -rf $WORKDIR
|
||||
\mkdir $WORKDIR
|
||||
\cd $WORKDIR
|
||||
|
||||
extract_pkg_from_local_repo ${CGCS_REPO_DIR} pxe-network-installer
|
||||
extract_pkg_from_local_repo ${CGCS_REPO_DIR} grub2-efi-pxeboot
|
||||
|
||||
\mkdir -p $OUTPUT_DIST_DIR/isolinux/pxeboot/EFI/centos/x86_64-efi
|
||||
|
||||
\cp --preserve=all pxeboot/pxelinux.0 pxeboot/menu.c32 pxeboot/chain.c32 $OUTPUT_DIST_DIR/isolinux/pxeboot &&
|
||||
\cp --preserve=all pxeboot/EFI/centos/x86_64-efi/* $OUTPUT_DIST_DIR/isolinux/pxeboot/EFI/centos/x86_64-efi/ &&
|
||||
\cp --preserve=all pxeboot/EFI/grubx64.efi $OUTPUT_DIST_DIR/isolinux/pxeboot/EFI/
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Error: Could not copy all files from installer"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
\cp --preserve=all www/pages/feed/rel-*/LiveOS/squashfs.img $OUTPUT_DIST_DIR/isolinux/LiveOS
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Error: Could not copy squashfs from LiveOS"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
# Replace vmlinuz and initrd.img with our own pre-built ones
|
||||
\rm -f \
|
||||
$OUTPUT_DIST_DIR/isolinux/vmlinuz \
|
||||
$OUTPUT_DIST_DIR/isolinux/images/pxeboot/vmlinuz \
|
||||
$OUTPUT_DIST_DIR/isolinux/initrd.img \
|
||||
$OUTPUT_DIST_DIR/isolinux/images/pxeboot/initrd.img
|
||||
\cp --preserve=all pxeboot/rel-*/installer-bzImage_1.0 \
|
||||
$OUTPUT_DIST_DIR/isolinux/vmlinuz &&
|
||||
\cp --preserve=all pxeboot/rel-*/installer-bzImage_1.0 \
|
||||
$OUTPUT_DIST_DIR/isolinux/images/pxeboot/vmlinuz &&
|
||||
\cp --preserve=all pxeboot/rel-*/installer-intel-x86-64-initrd_1.0 \
|
||||
$OUTPUT_DIST_DIR/isolinux/initrd.img &&
|
||||
\cp --preserve=all pxeboot/rel-*/installer-intel-x86-64-initrd_1.0 \
|
||||
$OUTPUT_DIST_DIR/isolinux/images/pxeboot/initrd.img
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Error: Failed to copy installer images"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
\cd $ORIG_PWD
|
||||
\rm -rf $WORKDIR
|
||||
)
|
||||
if [ $? -ne 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
function setup_upgrades_files {
|
||||
# Changes to copied files here must also be reflected in patch-iso
|
||||
|
||||
# Copy the upgrade files
|
||||
UPGRADES_DIR="$OUTPUT_DIST_DIR/isolinux/upgrades"
|
||||
\rm -rf $UPGRADES_DIR
|
||||
\mkdir -p $UPGRADES_DIR
|
||||
\cp $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/upgrades/* $UPGRADES_DIR
|
||||
sed -i "s/xxxSW_VERSIONxxx/${PLATFORM_RELEASE}/g" $UPGRADES_DIR/metadata.xml
|
||||
chmod +x $UPGRADES_DIR/*.sh
|
||||
# Write the version out (used in upgrade scripts - this is the same as SW_VERSION)
|
||||
echo "VERSION=$PLATFORM_RELEASE" > $UPGRADES_DIR/version
|
||||
}
|
||||
|
||||
function sign_iso {
|
||||
# Sign the .iso with the developer private key
|
||||
# Sigining with the formal key is only to be done for customer release
|
||||
# builds
|
||||
local isofilename=$(basename $OUTPUT_DIR/$DEST_FILE)
|
||||
local isofilenoext="${isofilename%.*}"
|
||||
openssl dgst -sha256 -sign ${MY_REPO}/build-tools/signing/dev-private-key.pem -binary -out $OUTPUT_DIR/$isofilenoext.sig $OUTPUT_DIR/$DEST_FILE
|
||||
}
|
||||
|
||||
#############################################
|
||||
# Main code
|
||||
#############################################
|
||||
|
||||
# Check args
|
||||
HELP=0
|
||||
CLEAN_FLAG=1 # TODO -- doesn't yet work without --clean
|
||||
DEST_FILE=bootimage.iso
|
||||
AUTO_FLAG=0
|
||||
AUTO_INSTALL=""
|
||||
CUMULUS=0
|
||||
SIGN_RPM_FILES=1
|
||||
DEVICE=""
|
||||
|
||||
# read the options
|
||||
TEMP=`getopt -o hf:a:d: --long help,file:,auto:,device:,cumulus,clean,skip-sign -n 'test.sh' -- "$@"`
|
||||
eval set -- "$TEMP"
|
||||
|
||||
# extract options and their arguments into variables.
|
||||
while true ; do
|
||||
case "$1" in
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
--clean) CLEAN_FLAG=1 ; shift ;;
|
||||
--skip-sign) SIGN_RPM_FILES=0 ; shift ;;
|
||||
--cumulus) CUMULUS=1 ; shift ;;
|
||||
-f | --file) DEST_FILE="$2"; shift; shift ;;
|
||||
-d | --device) DEVICE="$2"; shift; shift ;;
|
||||
-a | --auto) AUTO_FLAG=1; AUTO_INSTALL="$2"; shift; shift ;;
|
||||
--) shift ; break ;;
|
||||
*) echo "Internal error!" ; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $AUTO_FLAG -eq 1 ]; then
|
||||
if [[ "$AUTO_INSTALL" != "controller" && "$AUTO_INSTALL" != "cpe" ]] ; then
|
||||
echo "Unsupported --auto value: $AUTO_INSTALL"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
|
||||
(
|
||||
printf "\n*************************\n"
|
||||
printf "Create Titanium Cloud/CentOS Boot CD\n"
|
||||
printf "*************************\n\n"
|
||||
|
||||
# Init variables
|
||||
init_vars
|
||||
check_vars
|
||||
PKGLIST_MINIMAL=$INTERNAL_REPO_ROOT/build-tools/build_iso/minimal_rpm_list.txt
|
||||
PKGLIST_TI=$INTERNAL_REPO_ROOT/build-tools/build_iso/image.inc
|
||||
PKGLIST_DEV=$INTERNAL_REPO_ROOT/build-tools/build_iso/image-dev.inc
|
||||
|
||||
# Create skeleton build dir
|
||||
init_output_dir
|
||||
|
||||
# Create the vanilla DVD
|
||||
echo "Copying vanilla CentOS RPMs"
|
||||
install_pkg_list $PKGLIST_MINIMAL
|
||||
|
||||
# Find all CGCS packages
|
||||
# SAL exit 0
|
||||
echo "Installing Titanium Cloud packages"
|
||||
install_pkg_list $PKGLIST_TI
|
||||
if [ $? -eq 2 ]; then
|
||||
exit 1
|
||||
fi
|
||||
if [ "x${RELEASE_BUILD}" == "x" ]; then
|
||||
echo "Installing Titanium Cloud developer packages"
|
||||
install_pkg_list ${PKGLIST_DEV}
|
||||
if [ $? -eq 2 ]; then
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Extract installer files
|
||||
extract_installer_files
|
||||
|
||||
# Upgrades files
|
||||
setup_upgrades_files
|
||||
|
||||
# add file signatures to all rpms
|
||||
if [ $SIGN_RPM_FILES -ne 0 ]; then
|
||||
sign-rpms -d $OUTPUT_DIST_DIR/isolinux/Packages
|
||||
if [ $? -ne 0 ] ; then
|
||||
echo "failed to add file signatures to RPMs"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Finalize and build ISO
|
||||
final_touches
|
||||
|
||||
# Sign the ISO
|
||||
sign_iso
|
||||
|
||||
make_report $PKGLIST_MINIMAL $PKGLIST_TI
|
||||
|
||||
# Check sanity
|
||||
FILESIZE=$(wc -c <"$OUTPUT_FILE")
|
||||
if [ $FILESIZE -ge $MINIMUM_EXPECTED_SIZE ]; then
|
||||
printf "Done."
|
||||
printf "Output file: $OUTPUT_FILE\n\n"
|
||||
else
|
||||
printf "Output file $OUTPUT_FILE smaller than expected -- probable error\n\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
) 2>&1 | stdbuf -o0 awk '{ print strftime("%H:%M:%S"), $0; fflush(); }' ; exit ${PIPESTATUS[0]}
|
||||
63
build-tools/build-pkg-srpm
Normal file
63
build-tools/build-pkg-srpm
Normal file
@ -0,0 +1,63 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Available environment
|
||||
# SRC_BASE = absolute path to cgcs-root
|
||||
# AVS_BASE = absolute path to AVS source
|
||||
# CGCS_BASE = absolute path to CGCS source
|
||||
# RPM_BUILD_BASE = Directory where the package .distro directory can be found
|
||||
# SRPM_OUT = Directory into which SRC RPMS are copied in preparation for mock build
|
||||
# RPM_DIR = Directory into which binary RPMs are delivered by mock
|
||||
|
||||
SRC_DIR="/sources"
|
||||
VERSION=$(grep '^Version:' PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
|
||||
TAR_NAME=$(grep '^Name:' PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
|
||||
CUR_DIR=`pwd`
|
||||
BUILD_DIR=".distro/centos7/rpmbuild"
|
||||
|
||||
mkdir -p $BUILD_DIR/SRPMS
|
||||
|
||||
TAR="$TAR_NAME-$VERSION.tar.gz"
|
||||
TAR_PATH="$BUILD_DIR/SOURCES/$TAR"
|
||||
|
||||
TAR_NEEDED=0
|
||||
if [ -f $TAR_PATH ]; then
|
||||
n=`find . -cnewer $TAR_PATH -and ! -path './.git*' \
|
||||
-and ! -path './build/*' \
|
||||
-and ! -path './.pc/*' \
|
||||
-and ! -path './patches/*' \
|
||||
-and ! -path './.distro/*' \
|
||||
-and ! -path './pbr-*.egg/*' \
|
||||
| wc -l`
|
||||
if [ $n -gt 0 ]; then
|
||||
TAR_NEEDED=1
|
||||
fi
|
||||
else
|
||||
TAR_NEEDED=1
|
||||
fi
|
||||
|
||||
if [ $TAR_NEEDED -gt 0 ]; then
|
||||
tar czvf $TAR_PATH .$SRC_DIR --exclude '.git*' --exclude 'build' --exclude='.pc' --exclude='patches' --exclude='.distro' --exclude='pbr-*.egg' --transform "s,^\.$SRC_DIR,$TAR_NAME-$VERSION,"
|
||||
fi
|
||||
|
||||
for SPEC in `ls $BUILD_DIR/SPECS`; do
|
||||
SPEC_PATH="$BUILD_DIR/SPECS/$SPEC"
|
||||
RELEASE=$(grep '^Release:' $SPEC_PATH | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
|
||||
NAME=`echo $SPEC | sed 's/.spec$//'`
|
||||
SRPM="$NAME-$VERSION-$RELEASE.src.rpm"
|
||||
SRPM_PATH="$BUILD_DIR/SRPMS/$SRPM"
|
||||
|
||||
BUILD_NEEDED=0
|
||||
if [ -f $SRPM_PATH ]; then
|
||||
n=`find . -cnewer $SRPM_PATH | wc -l`
|
||||
if [ $n -gt 0 ]; then
|
||||
BUILD_NEEDED=1
|
||||
fi
|
||||
else
|
||||
BUILD_NEEDED=1
|
||||
fi
|
||||
|
||||
if [ $BUILD_NEEDED -gt 0 ]; then
|
||||
rpmbuild -bs $SPEC_PATH --define="%_topdir $CUR_DIR/$BUILD_DIR" --define="_tis_dist .tis"
|
||||
fi
|
||||
done
|
||||
|
||||
34
build-tools/build-pkgs
Executable file
34
build-tools/build-pkgs
Executable file
@ -0,0 +1,34 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This program is a wrapper around build-pkgs-parallel and build-pkgs-serial
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " Create source and binary rpms:"
|
||||
echo " build-pkgs [--serial] [args]"
|
||||
}
|
||||
|
||||
SERIAL_FLAG=0
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$1" in
|
||||
--serial) SERIAL_FLAG=1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
which mock_tmpfs_umount >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
SERIAL_FLAG=1
|
||||
fi
|
||||
|
||||
if [ $SERIAL_FLAG -eq 1 ]; then
|
||||
echo "build-pkgs-serial $@"
|
||||
build-pkgs-serial "$@"
|
||||
else
|
||||
echo "build-pkgs-parallel $@"
|
||||
build-pkgs-parallel "$@"
|
||||
fi
|
||||
|
||||
370
build-tools/build-pkgs-parallel
Executable file
370
build-tools/build-pkgs-parallel
Executable file
@ -0,0 +1,370 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This program is a wrapper around build-srpms-parallel and build-rpms-parallel
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " Create source and Binary rpms:"
|
||||
echo " Build optimizations (--no-descendants, --no-required, --no-build-info,"
|
||||
echo " --no-autoclean) are not recommended for the first build after a clone/pull,"
|
||||
echo " nor the final build prior to creating an iso or patch, but can be used"
|
||||
echo " for intermediate builds. i.e. while debugging compilation failures."
|
||||
echo " build-pkgs-parallel [--no-descendants] [--no-required] [--no-build-info] [--no-autoclean] [--careful] [--formal] [ list of package names ]"
|
||||
echo ""
|
||||
echo " Delete source rpms, and the directories associated with it's creation:"
|
||||
echo " Note: does not clean an edit environment"
|
||||
echo " build-pkgs-parallel --clean [ list of package names ]"
|
||||
echo ""
|
||||
echo " Extract an src.rpm into a pair of git trees to aid in editing it's contents,"
|
||||
echo " one for source code and one for metadata such as the spec file."
|
||||
echo " If --no-meta-patch is specified, then WRS patches are omitted."
|
||||
echo " build-pkgs-parallel --edit [--no-meta-patch] [ list of package names ]"
|
||||
echo ""
|
||||
echo " Delete an edit environment"
|
||||
echo " build-pkgs-parallel --edit --clean [ list of package names ]"
|
||||
echo ""
|
||||
echo " This help page"
|
||||
echo " build-pkgs-parallel [--help]"
|
||||
echo ""
|
||||
}
|
||||
|
||||
|
||||
HELP=0
|
||||
CLEAN_FLAG=0
|
||||
EDIT_FLAG=0
|
||||
STD_BUILD=1
|
||||
RT_BUILD=1
|
||||
INSTALLER_BUILD=0
|
||||
|
||||
# read the options
|
||||
TEMP=$(getopt -o h --long rt,std,installer,edit,no-meta-patch,no-descendants,no-required,no-build-info,no-autoclean,formal,careful,help,clean -n 'build-pkgs-parallel' -- "$@")
|
||||
if [ $? -ne 0 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
eval set -- "$TEMP"
|
||||
|
||||
# extract options and their arguments into variables.
|
||||
EXTRA_ARGS_COMMON=""
|
||||
EXTRA_ARGS_SRPM=""
|
||||
EXTRA_ARGS_RPM=""
|
||||
while true ; do
|
||||
case "$1" in
|
||||
--no-descendants) EXTRA_ARGS_COMMON+=" --no-descendants" ; shift ;;
|
||||
--formal) EXTRA_ARGS_COMMON+=" --formal" ; shift ;;
|
||||
--careful) EXTRA_ARGS_RPM+=" --careful" ; shift ;;
|
||||
--no-required) EXTRA_ARGS_RPM+=" --no-required" ; shift ;;
|
||||
--no-build-info) EXTRA_ARGS_COMMON+=" --no-build-info" ; shift ;;
|
||||
--no-autoclean) EXTRA_ARGS_RPM+=" --no-autoclean" ; shift ;;
|
||||
--no-meta-patch) EXTRA_ARGS_SRPM+=" --no-meta-patch" ; shift ;;
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
--clean) CLEAN_FLAG=1 ; shift ;;
|
||||
--edit) EDIT_FLAG=1 ; EXTRA_ARGS_SRPM+=" --edit"; shift ;;
|
||||
--rt) STD_BUILD=0 ; shift ;;
|
||||
--std) RT_BUILD=0 ; shift ;;
|
||||
--installer) INSTALLER_BUILD=1 ; STD_BUILD=0 ; RT_BUILD=0 ; shift ;;
|
||||
--) shift ; break ;;
|
||||
*) usage; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
function my_exit() {
|
||||
build-rpms-parallel --std --tmpfs-clean
|
||||
build-rpms-parallel --rt --tmpfs-clean
|
||||
}
|
||||
|
||||
function my_sigint() {
|
||||
echo "build-pkgs-parallel sigint"
|
||||
pkill -SIGABRT -P $BASHPID &> /dev/null
|
||||
echo "build-pkgs-parallel waiting"
|
||||
wait
|
||||
echo "build-pkgs-parallel wait complete"
|
||||
|
||||
}
|
||||
|
||||
function my_sighup() {
|
||||
echo "build-pkgs-parallel sighup"
|
||||
pkill -SIGABRT -P $BASHPID &> /dev/null
|
||||
echo "build-pkgs-parallel waiting"
|
||||
wait
|
||||
echo "build-pkgs-parallel wait complete"
|
||||
}
|
||||
|
||||
function my_sigabrt() {
|
||||
echo "build-pkgs-parallel sigabrt"
|
||||
pkill -SIGABRT -P $BASHPID &> /dev/null
|
||||
echo "build-pkgs-parallel waiting"
|
||||
wait
|
||||
echo "build-pkgs-parallel wait complete"
|
||||
}
|
||||
|
||||
function my_sigterm() {
|
||||
echo "build-pkgs-parallel sigterm"
|
||||
pkill -SIGABRT -P $BASHPID &> /dev/null
|
||||
echo "build-pkgs-parallel waiting"
|
||||
wait
|
||||
echo "build-pkgs-parallel wait complete"
|
||||
}
|
||||
|
||||
trap my_exit EXIT
|
||||
trap my_sigint INT
|
||||
trap my_sighup HUP
|
||||
trap my_sigabrt ABRT
|
||||
trap my_sigterm TERM
|
||||
|
||||
TARGETS=" $@ "
|
||||
TARGETS_STD=" "
|
||||
TARGETS_RT=" "
|
||||
TARGETS_INSTALLER=" "
|
||||
TARGETS_MISC=" "
|
||||
|
||||
find_targets () {
|
||||
local centos_pkg_dirs=$1
|
||||
local d=""
|
||||
local d2=""
|
||||
local g=""
|
||||
local x=""
|
||||
local name=""
|
||||
local path=""
|
||||
local RESULT=" "
|
||||
local FOUND=0
|
||||
|
||||
for g in $(find $MY_REPO -type d -name .git); do
|
||||
d=$(dirname $g)
|
||||
if [ -f $d/$centos_pkg_dirs ]; then
|
||||
for d2 in $(grep -v '^#' $d/$centos_pkg_dirs); do
|
||||
name=""
|
||||
if [ -f $d/$d2/centos/srpm_path ]; then
|
||||
path=$(cat $d/$d2/centos/srpm_path | head -n 1 | sed "s#^mirror:CentOS/tis-r3-CentOS/mitaka#$MY_REPO/cgcs-centos-repo#" | sed "s#^mirror:#$MY_REPO/cgcs-centos-repo/#" | sed "s#^repo:#$MY_REPO/#" | sed "s#^3rd_party:#$MY_REPO/cgcs-3rd-party-repo/#" | sed "s#^Source/#$MY_REPO/cgcs-centos-repo/Source/#")
|
||||
name=$(rpm -q --qf='%{NAME}' --nosignature -p $path)
|
||||
else
|
||||
path=$(find $d/$d2/centos/ -name '*.spec' | head -n 1)
|
||||
if [[ ( -z "$path" ) && ( -f $d/$d2/centos/spec_path ) ]]; then
|
||||
path=$(find $MY_REPO/$(cat $d/$d2/centos/spec_path) -maxdepth 1 -name '*.spec' | head -n 1)
|
||||
fi
|
||||
if [ "$path" != "" ]; then
|
||||
name=$(spec_find_tag Name "$path" 2>> /dev/null)
|
||||
fi
|
||||
fi
|
||||
if [ "$name" != "" ]; then
|
||||
if [ "$BUILD_TYPE" == "rt" ]; then
|
||||
FOUND=0
|
||||
for x in $TARGETS; do
|
||||
if [ "${x: -3}" == "-rt" ]; then
|
||||
if [ "${name}" == "$x" ] || [ "${name}-rt" == "${x}" ]; then
|
||||
RESULT+="$x "
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
if [ $FOUND -eq 0 ]; then
|
||||
for x in $TARGETS; do
|
||||
if [ "${name}" == "${x}-rt" ]; then
|
||||
RESULT+="$x-rt "
|
||||
FOUND=1
|
||||
break
|
||||
else
|
||||
if [ "${name}" == "$x" ] || [ "${name}-rt" == "${x}" ]; then
|
||||
RESULT+="$x "
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
else
|
||||
for x in $TARGETS; do
|
||||
if [ "${name}" == "$x" ]; then
|
||||
RESULT+="$x "
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
done
|
||||
|
||||
echo "$RESULT"
|
||||
return 0
|
||||
}
|
||||
|
||||
echo "CLEAN_FLAG=$CLEAN_FLAG"
|
||||
echo "EDIT_FLAG=$EDIT_FLAG"
|
||||
|
||||
if [ "x$TARGETS" != "x " ]; then
|
||||
source $MY_REPO/build-tools/spec-utils
|
||||
TARGETS_STD="$(find_targets centos_pkg_dirs)"
|
||||
|
||||
BUILD_TYPE_SAVE="$BUILD_TYPE"
|
||||
BUILD_TYPE="rt"
|
||||
TARGETS_RT="$(find_targets centos_pkg_dirs_rt)"
|
||||
BUILD_TYPE="$BUILD_TYPE_SAVE"
|
||||
|
||||
echo "TARGETS_STD=$TARGETS_STD"
|
||||
echo "TARGETS_RT=$TARGETS_RT"
|
||||
|
||||
for x in $TARGETS; do
|
||||
if [[ $TARGETS_STD == *" $x "* ]]
|
||||
then
|
||||
echo "found $x" >> /dev/null;
|
||||
else
|
||||
if [[ $TARGETS_RT == *" $x "* ]]
|
||||
then
|
||||
echo "found $x" >> /dev/null;
|
||||
else
|
||||
TARGETS_MISC+="$x "
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
echo "EXTRA_ARGS_COMMON='$EXTRA_ARGS_COMMON'"
|
||||
echo "EXTRA_ARGS_SRPM='$EXTRA_ARGS_SRPM'"
|
||||
echo "EXTRA_ARGS_RPM='$EXTRA_ARGS_RPM'"
|
||||
echo "TARGETS='$TARGETS'"
|
||||
echo "TARGETS_STD='$TARGETS_STD'"
|
||||
echo "TARGETS_RT='$TARGETS_RT'"
|
||||
echo "TARGETS_MISC='$TARGETS_MISC'"
|
||||
|
||||
if [ $CLEAN_FLAG -eq 1 ]; then
|
||||
if [ $STD_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
if [ $EDIT_FLAG -ne 1 ]; then
|
||||
echo "$DIR/build-rpms-parallel --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_STD $TARGETS_MISC"
|
||||
$DIR/build-rpms-parallel --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_STD $TARGETS_MISC || exit 1
|
||||
fi
|
||||
fi
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
echo "$DIR/build-srpms-parallel --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_STD $TARGETS_MISC"
|
||||
$DIR/build-srpms-parallel --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_STD $TARGETS_MISC || exit 1
|
||||
fi
|
||||
fi
|
||||
if [ $RT_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
if [ $EDIT_FLAG -ne 1 ]; then
|
||||
echo "$DIR/build-rpms-parallel --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_RT $TARGETS_MISC"
|
||||
$DIR/build-rpms-parallel --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_RT $TARGETS_MISC || exit 1
|
||||
fi
|
||||
fi
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
echo "$DIR/build-srpms-parallel --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_RT $TARGETS_MISC"
|
||||
$DIR/build-srpms-parallel --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_RT $TARGETS_MISC || exit 1
|
||||
fi
|
||||
fi
|
||||
exit $?
|
||||
fi
|
||||
|
||||
function launch_build()
|
||||
{
|
||||
local build_type=$1
|
||||
shift
|
||||
|
||||
local logfile=$MY_WORKSPACE/build-$build_type.log
|
||||
local rc
|
||||
local targets
|
||||
|
||||
if [ "$build_type" == "std" ]; then
|
||||
targets="$TARGETS_STD $TARGETS_MISC"
|
||||
else
|
||||
if [ "$build_type" == "rt" ]; then
|
||||
targets="$TARGETS_RT $TARGETS_MISC"
|
||||
else
|
||||
if [ "$build_type" == "installer" ]; then
|
||||
targets="$TARGETS_INSTALLER $TARGETS_MISC"
|
||||
else
|
||||
targets="$TARGETS"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Launching $build_type build, logging to $logfile"
|
||||
\rm $logfile
|
||||
|
||||
echo -e "\n######## $(date): Launching build-srpms-parallel --$build_type $EXTRA_ARGS $@\n" | tee --append $logfile
|
||||
# No clean flag, call build-srpms-parallel followed by build-rpms-parallel
|
||||
echo "$DIR/build-srpms-parallel --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets" | tee --append $logfile
|
||||
$DIR/build-srpms-parallel --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets 2>&1 | tee --append $logfile
|
||||
rc=${PIPESTATUS[0]}
|
||||
if [ $rc -eq 0 ]; then
|
||||
echo -e "\n######## $(date): build-srpm-parallel --$build_type was successful" | tee --append $logfile
|
||||
else
|
||||
echo -e "\n######## $(date): build-srpm-parallel --$build_type failed with rc=$rc" | tee --append $logfile
|
||||
echo -e "\n$(date): build-srpm-parallel --$build_type failed with rc=$rc"
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
if [ $EDIT_FLAG -ne 1 ]; then
|
||||
echo -e "\n######## $(date): Launching build-rpms-parallel --$build_type $EXTRA_ARGS $@\n" | tee --append $logfile
|
||||
echo "$DIR/build-rpms-parallel --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $targets" | tee --append $logfile
|
||||
$DIR/build-rpms-parallel --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $targets 2>&1 | tee --append $logfile
|
||||
rc=${PIPESTATUS[0]}
|
||||
if [ $rc -eq 0 ]; then
|
||||
echo -e "\n######## $(date): build-rpm-parallel --$build_type was successful" | tee --append $logfile
|
||||
else
|
||||
echo -e "\n######## $(date): build-rpm-parallel --$build_type failed with rc=$rc" | tee --append $logfile
|
||||
echo -e "\n$(date): build-rpm-parallel --$build_type failed with rc=$rc"
|
||||
exit $rc
|
||||
fi
|
||||
fi
|
||||
|
||||
echo -e "\n$(date): $build_type complete\n"
|
||||
#exit $rc
|
||||
}
|
||||
|
||||
function progbar()
|
||||
{
|
||||
while :; do
|
||||
for s in / - \\ \|; do
|
||||
printf "\r$s"
|
||||
sleep .5
|
||||
done
|
||||
done
|
||||
}
|
||||
|
||||
# Create $MY_WORKSPACE if it doesn't exist already
|
||||
mkdir -p $MY_WORKSPACE
|
||||
|
||||
if [ $STD_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
launch_build std
|
||||
else
|
||||
echo "Skipping 'std' build, no valid targets in list: $TARGETS"
|
||||
fi
|
||||
else
|
||||
echo "Skipping 'std' build"
|
||||
fi
|
||||
if [ $RT_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
launch_build rt
|
||||
else
|
||||
echo "Skipping 'rt' build, no valid targets in list: $TARGETS"
|
||||
fi
|
||||
else
|
||||
echo "Skipping 'rt' build"
|
||||
fi
|
||||
if [ $INSTALLER_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_INSTALLER" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
launch_build installer
|
||||
else
|
||||
echo "Skipping 'installer' build, no valid targets in list: $TARGETS"
|
||||
fi
|
||||
else
|
||||
echo "Skipping 'installer' build"
|
||||
fi
|
||||
|
||||
|
||||
echo "All builds were successful"
|
||||
|
||||
exit 0
|
||||
|
||||
379
build-tools/build-pkgs-serial
Executable file
379
build-tools/build-pkgs-serial
Executable file
@ -0,0 +1,379 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This program is a wrapper around build-srpms and build-rpms
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " Create source and Binary rpms:"
|
||||
echo " Build optimizations (--no-descendants, --no-required, --no-build-info,"
|
||||
echo " --no-autoclean) are not recommended for the first build after a clone/pull,"
|
||||
echo " nor the final build prior to creating an iso or patch, but can be used"
|
||||
echo " for intermediate builds. i.e. while debugging compilation failures."
|
||||
echo " build-pkgs [--no-descendants] [--no-required] [--no-build-info] [--no-autoclean] [--careful] [--formal] [ list of package names ]"
|
||||
echo ""
|
||||
echo " Delete source rpms, and the directories associated with it's creation:"
|
||||
echo " Note: does not clean an edit environment"
|
||||
echo " build-pkgs --clean [ list of package names ]"
|
||||
echo ""
|
||||
echo " Extract an src.rpm into a pair of git trees to aid in editing it's contents,"
|
||||
echo " one for source code and one for metadata such as the spec file."
|
||||
echo " If --no-meta-patch is specified, then WRS patches are omitted."
|
||||
echo " build-pkgs --edit [--no-meta-patch] [ list of package names ]"
|
||||
echo ""
|
||||
echo " Delete an edit environment"
|
||||
echo " build-pkgs --edit --clean [ list of package names ]"
|
||||
echo ""
|
||||
echo " This help page"
|
||||
echo " build-pkgs [--help]"
|
||||
echo ""
|
||||
}
|
||||
|
||||
|
||||
HELP=0
|
||||
CLEAN_FLAG=0
|
||||
EDIT_FLAG=0
|
||||
STD_BUILD=1
|
||||
RT_BUILD=1
|
||||
INSTALLER_BUILD=0
|
||||
|
||||
# read the options
|
||||
TEMP=`getopt -o h --long serial,rt,std,edit,no-meta-patch,no-descendants,no-required,no-build-info,no-autoclean,formal,careful,help,clean -n 'build-pkgs' -- "$@"`
|
||||
if [ $? -ne 0 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
eval set -- "$TEMP"
|
||||
|
||||
# extract options and their arguments into variables.
|
||||
EXTRA_ARGS_COMMON=""
|
||||
EXTRA_ARGS_SRPM=""
|
||||
EXTRA_ARGS_RPM=""
|
||||
while true ; do
|
||||
case "$1" in
|
||||
--no-descendants) EXTRA_ARGS_COMMON+=" --no-descendants" ; shift ;;
|
||||
--formal) EXTRA_ARGS_COMMON+=" --formal" ; shift ;;
|
||||
--careful) EXTRA_ARGS_RPM+=" --careful" ; shift ;;
|
||||
--no-required) EXTRA_ARGS_RPM+=" --no-required" ; shift ;;
|
||||
--no-build-info) EXTRA_ARGS_COMMON+=" --no-build-info" ; shift ;;
|
||||
--no-autoclean) EXTRA_ARGS_RPM+=" --no-autoclean" ; shift ;;
|
||||
--no-meta-patch) EXTRA_ARGS_SRPM+=" --no-meta-patch" ; shift ;;
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
--clean) CLEAN_FLAG=1 ; shift ;;
|
||||
--edit) EDIT_FLAG=1 ; EXTRA_ARGS_SRPM+=" --edit"; shift ;;
|
||||
--rt) STD_BUILD=0 ; shift ;;
|
||||
--std) RT_BUILD=0 ; shift ;;
|
||||
--installer) INSTALLER_BUILD=1 ; STD_BUILD=0 ; RT_BUILD=0 ; shift ;;
|
||||
--serial) shift ;;
|
||||
--) shift ; break ;;
|
||||
*) usage; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
TARGETS=" $@ "
|
||||
TARGETS_STD=" "
|
||||
TARGETS_RT=" "
|
||||
TARGETS_INSTALLER=" "
|
||||
TARGETS_MISC=" "
|
||||
|
||||
find_targets () {
|
||||
local centos_pkg_dirs=$1
|
||||
local d=""
|
||||
local d2=""
|
||||
local g=""
|
||||
local x=""
|
||||
local name=""
|
||||
local path=""
|
||||
local RESULT=" "
|
||||
local FOUND=0
|
||||
|
||||
for g in $(find $MY_REPO -type d -name .git); do
|
||||
d=$(dirname $g)
|
||||
if [ -f $d/$centos_pkg_dirs ]; then
|
||||
for d2 in $(grep -v '^#' $d/$centos_pkg_dirs); do
|
||||
name=""
|
||||
if [ -f $d/$d2/centos/srpm_path ]; then
|
||||
path=$(cat $d/$d2/centos/srpm_path | head -n 1 | sed "s#^mirror:CentOS/tis-r3-CentOS/mitaka#$MY_REPO/cgcs-centos-repo#" | sed "s#^mirror:#$MY_REPO/cgcs-centos-repo/#" | sed "s#^repo:#$MY_REPO/#" | sed "s#^3rd_party:#$MY_REPO/cgcs-3rd-party-repo/#" | sed "s#^Source/#$MY_REPO/cgcs-centos-repo/Source/#")
|
||||
name=$(rpm -q --qf='%{NAME}' --nosignature -p $path)
|
||||
else
|
||||
path=$(find $d/$d2/centos/ -name '*.spec' | head -n 1)
|
||||
if [[ ( -z "$path" ) && ( -f $d/$d2/centos/spec_path ) ]]; then
|
||||
path=$(find $MY_REPO/$(cat $d/$d2/centos/spec_path) -maxdepth 1 -name '*.spec' | head -n 1)
|
||||
fi
|
||||
if [ "$path" != "" ]; then
|
||||
name=$(spec_find_tag Name "$path" 2>> /dev/null)
|
||||
fi
|
||||
fi
|
||||
if [ "$name" != "" ]; then
|
||||
if [ "$BUILD_TYPE" == "rt" ]; then
|
||||
FOUND=0
|
||||
for x in $TARGETS; do
|
||||
if [ "${x: -3}" == "-rt" ]; then
|
||||
if [ "${name}" == "$x" ] || [ "${name}-rt" == "${x}" ]; then
|
||||
RESULT+="$x "
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
if [ $FOUND -eq 0 ]; then
|
||||
for x in $TARGETS; do
|
||||
if [ "${name}" == "${x}-rt" ]; then
|
||||
RESULT+="$x-rt "
|
||||
FOUND=1
|
||||
break
|
||||
else
|
||||
if [ "${name}" == "$x" ] || [ "${name}-rt" == "${x}" ]; then
|
||||
RESULT+="$x "
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
else
|
||||
for x in $TARGETS; do
|
||||
if [ "${name}" == "$x" ]; then
|
||||
RESULT+="$x "
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
done
|
||||
|
||||
echo "$RESULT"
|
||||
return 0
|
||||
}
|
||||
|
||||
echo "CLEAN_FLAG=$CLEAN_FLAG"
|
||||
echo "EDIT_FLAG=$EDIT_FLAG"
|
||||
|
||||
if [ "x$TARGETS" != "x " ]; then
|
||||
source $MY_REPO/build-tools/spec-utils
|
||||
TARGETS_STD="$(find_targets centos_pkg_dirs)"
|
||||
|
||||
BUILD_TYPE_SAVE="$BUILD_TYPE"
|
||||
BUILD_TYPE="rt"
|
||||
TARGETS_RT="$(find_targets centos_pkg_dirs_rt)"
|
||||
BUILD_TYPE="installer"
|
||||
TARGETS_INSTALLER="$(find_targets centos_pkg_dirs_installer)"
|
||||
BUILD_TYPE="$BUILD_TYPE_SAVE"
|
||||
|
||||
echo "TARGETS_STD=$TARGETS_STD"
|
||||
echo "TARGETS_RT=$TARGETS_RT"
|
||||
echo "TARGETS_INSTALLER=$TARGETS_INSTALLER"
|
||||
|
||||
for x in $TARGETS; do
|
||||
if [[ $TARGETS_STD == *" $x "* ]]
|
||||
then
|
||||
echo "found $x" >> /dev/null;
|
||||
else
|
||||
if [[ $TARGETS_RT == *" $x "* ]]
|
||||
then
|
||||
echo "found $x" >> /dev/null;
|
||||
else
|
||||
if [[ $TARGETS_INSTALLER == *" $x "* ]]
|
||||
then
|
||||
echo "found $x" >> /dev/null;
|
||||
INSTALLER_BUILD=1
|
||||
else
|
||||
TARGETS_MISC+="$x "
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
echo "EXTRA_ARGS_COMMON='$EXTRA_ARGS_COMMON'"
|
||||
echo "EXTRA_ARGS_SRPM='$EXTRA_ARGS_SRPM'"
|
||||
echo "EXTRA_ARGS_RPM='$EXTRA_ARGS_RPM'"
|
||||
echo "TARGETS='$TARGETS'"
|
||||
echo "TARGETS_STD='$TARGETS_STD'"
|
||||
echo "TARGETS_RT='$TARGETS_RT'"
|
||||
echo "TARGETS_INSTALLER='$TARGETS_INSTALLER'"
|
||||
echo "TARGETS_MISC='$TARGETS_MISC'"
|
||||
|
||||
if [ $CLEAN_FLAG -eq 1 ]; then
|
||||
if [ $STD_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
if [ $EDIT_FLAG -ne 1 ]; then
|
||||
echo "$DIR/build-rpms-serial --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_STD $TARGETS_MISC"
|
||||
$DIR/build-rpms-serial --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_STD $TARGETS_MISC || exit 1
|
||||
fi
|
||||
fi
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
echo "$DIR/build-srpms-serial --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_STD $TARGETS_MISC"
|
||||
$DIR/build-srpms-serial --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_STD $TARGETS_MISC || exit 1
|
||||
fi
|
||||
fi
|
||||
if [ $RT_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x" ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
if [ $EDIT_FLAG -ne 1 ]; then
|
||||
echo "$DIR/build-rpms-serial --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_RT $TARGETS_MISC"
|
||||
$DIR/build-rpms-serial --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_RT $TARGETS_MISC || exit 1
|
||||
fi
|
||||
fi
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
echo "$DIR/build-srpms-serial --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_RT $TARGETS_MISC"
|
||||
$DIR/build-srpms-serial --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_RT $TARGETS_MISC || exit 1
|
||||
fi
|
||||
fi
|
||||
if [ $INSTALLER_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x" ] || [ "$TARGETS_INSTALLER" != " " ]; then
|
||||
if [ $EDIT_FLAG -ne 1 ]; then
|
||||
echo "$DIR/build-rpms-serial --installer --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_INSTALLER"
|
||||
$DIR/build-rpms-serial --installer --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_INSTALLER || exit 1
|
||||
fi
|
||||
fi
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_INSTALLER" != " " ]; then
|
||||
echo "$DIR/build-srpms-serial --installer --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_INSTALLER"
|
||||
$DIR/build-srpms-serial --installer --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_INSTALLER || exit 1
|
||||
fi
|
||||
fi
|
||||
exit $?
|
||||
fi
|
||||
|
||||
function launch_build()
|
||||
{
|
||||
local build_type=$1
|
||||
shift
|
||||
|
||||
local logfile=$MY_WORKSPACE/build-$build_type.log
|
||||
local rc
|
||||
local targets
|
||||
|
||||
if [ "$build_type" == "std" ]; then
|
||||
targets="$TARGETS_STD $TARGETS_MISC"
|
||||
else
|
||||
if [ "$build_type" == "rt" ]; then
|
||||
targets="$TARGETS_RT $TARGETS_MISC"
|
||||
else
|
||||
if [ "$build_type" == "installer" ]; then
|
||||
targets="$TARGETS_INSTALLER"
|
||||
else
|
||||
targets="$TARGETS"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Launching $build_type build, logging to $logfile"
|
||||
|
||||
echo -e "\n######## $(date): Launching build-srpms-serial --$build_type $EXTRA_ARGS $@\n" | tee --append $logfile
|
||||
# No clean flag, call build-srpms-serial followed by build-rpms-serial
|
||||
echo "$DIR/build-srpms-serial --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets" | tee --append $logfile
|
||||
$DIR/build-srpms-serial --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets 2>&1 | tee --append $logfile
|
||||
rc=${PIPESTATUS[0]}
|
||||
if [ $rc -eq 0 ]; then
|
||||
echo -e "\n######## $(date): build-srpm-serial --$build_type was successful" | tee --append $logfile
|
||||
else
|
||||
echo -e "\n######## $(date): build-srpm-serial --$build_type failed with rc=$rc" | tee --append $logfile
|
||||
echo -e "\n$(date): build-srpm-serial --$build_type failed with rc=$rc"
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
if [ $EDIT_FLAG -ne 1 ]; then
|
||||
echo -e "\n######## $(date): Launching build-rpms-serial --$build_type $EXTRA_ARGS $@\n" | tee --append $logfile
|
||||
echo "$DIR/build-rpms-serial --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets" | tee --append $logfile
|
||||
$DIR/build-rpms-serial --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $targets 2>&1 | tee --append $logfile
|
||||
rc=${PIPESTATUS[0]}
|
||||
if [ $rc -eq 0 ]; then
|
||||
echo -e "\n######## $(date): build-rpm-serial --$build_type was successful" | tee --append $logfile
|
||||
else
|
||||
echo -e "\n######## $(date): build-rpm-serial --$build_type failed with rc=$rc" | tee --append $logfile
|
||||
echo -e "\n$(date): build-rpm-serial --$build_type failed with rc=$rc"
|
||||
exit $rc
|
||||
fi
|
||||
fi
|
||||
|
||||
echo -e "\n$(date): $build_type complete\n"
|
||||
#exit $rc
|
||||
}
|
||||
|
||||
function progbar()
|
||||
{
|
||||
while :; do
|
||||
for s in / - \\ \|; do
|
||||
printf "\r$s"
|
||||
sleep .5
|
||||
done
|
||||
done
|
||||
}
|
||||
|
||||
# Create $MY_WORKSPACE if it doesn't exist already
|
||||
mkdir -p $MY_WORKSPACE
|
||||
|
||||
if [ $STD_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
launch_build std
|
||||
else
|
||||
echo "Skipping 'std' build, no valid targets in list: $TARGETS"
|
||||
fi
|
||||
else
|
||||
echo "Skipping 'std' build"
|
||||
fi
|
||||
if [ $RT_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
|
||||
launch_build rt
|
||||
else
|
||||
echo "Skipping 'rt' build, no valid targets in list: $TARGETS"
|
||||
fi
|
||||
else
|
||||
echo "Skipping 'rt' build"
|
||||
fi
|
||||
if [ $INSTALLER_BUILD -eq 1 ]; then
|
||||
if [ "x$TARGETS" == "x " ] || [ "$TARGETS_INSTALLER" != " " ]; then
|
||||
launch_build installer
|
||||
else
|
||||
echo "Skipping 'installer' build, no valid targets in list: $TARGETS"
|
||||
fi
|
||||
# else
|
||||
# echo "Skipping 'installer' build"
|
||||
fi
|
||||
|
||||
|
||||
#progbar &
|
||||
|
||||
#function killemall() {
|
||||
# for pid in $(jobs -p)
|
||||
# do
|
||||
# pgid=$(echo -n $(ps -o pgid= $pid))
|
||||
# if [ -n "$pgid" ]; then
|
||||
# echo "Killing child progress group: kill -TERM -$pgid"
|
||||
# kill -TERM -$pgid
|
||||
# fi
|
||||
# done
|
||||
# echo "Bad mojo when you ctrl-C, so don't"
|
||||
#}
|
||||
|
||||
#trap killemall INT
|
||||
|
||||
## Wait for builds to complete
|
||||
#wait %1
|
||||
#STD_BUILD_RC=$?
|
||||
|
||||
#wait %2
|
||||
#RT_BUILD_RC=$?
|
||||
|
||||
# Kill the progress bar
|
||||
#kill %3
|
||||
|
||||
#if [ $STD_BUILD_RC -ne 0 -o $RT_BUILD_RC -ne 0 ]; then
|
||||
# echo "One or more builds failed"
|
||||
# exit 1
|
||||
#fi
|
||||
|
||||
echo "All builds were successful"
|
||||
|
||||
exit 0
|
||||
|
||||
1
build-tools/build-pkgs4
Symbolic link
1
build-tools/build-pkgs4
Symbolic link
@ -0,0 +1 @@
|
||||
build-pkgs-parallel
|
||||
34
build-tools/build-rpms
Executable file
34
build-tools/build-rpms
Executable file
@ -0,0 +1,34 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This program is a wrapper around build-rpms-parallel and build-rpms-serial
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " Create binary rpms:"
|
||||
echo " build-rpms [--serial] [args]"
|
||||
}
|
||||
|
||||
SERIAL_FLAG=0
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$1" in
|
||||
--serial) SERIAL_FLAG=1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
which mock_tmpfs_umount >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
SERIAL_FLAG=1
|
||||
fi
|
||||
|
||||
if [ $SERIAL_FLAG -eq 1 ]; then
|
||||
echo "build-rpms-serial $@"
|
||||
build-rpms-serial "$@"
|
||||
else
|
||||
echo "build-rpms-parallel $@"
|
||||
build-rpms-parallel "$@"
|
||||
fi
|
||||
|
||||
2343
build-tools/build-rpms-parallel
Executable file
2343
build-tools/build-rpms-parallel
Executable file
File diff suppressed because it is too large
Load Diff
1328
build-tools/build-rpms-serial
Executable file
1328
build-tools/build-rpms-serial
Executable file
File diff suppressed because it is too large
Load Diff
1
build-tools/build-rpms4
Symbolic link
1
build-tools/build-rpms4
Symbolic link
@ -0,0 +1 @@
|
||||
build-rpms-parallel
|
||||
70
build-tools/build-sdk
Executable file
70
build-tools/build-sdk
Executable file
@ -0,0 +1,70 @@
|
||||
#!/bin/bash
|
||||
|
||||
# We find SDK files in two ways -- one is that any package with an
|
||||
# "-cgts-sdk" component in the filename is deemed to have SDK content
|
||||
# (this works great for packages that produce installable content, as well
|
||||
# as SDK content which is placed in a separate -cgts-sdk package)
|
||||
#
|
||||
# The second way is explicitly name packages that contain SDK content. This
|
||||
# works well for packages which only contain SDK content. The other reason
|
||||
# that calling out packages explcitly may be of use is that some packages
|
||||
# (like restapi-doc) may not be built automatically because they get caught
|
||||
# in the blacklist filter (i.e. we normally don't build restapi-doc because
|
||||
# it contains "doc").
|
||||
|
||||
EXPLICIT_PACKAGES=( \
|
||||
wrs-branding \
|
||||
wrs-heat-templates \
|
||||
install-log-server \
|
||||
restapi-doc \
|
||||
remote-clients \
|
||||
)
|
||||
|
||||
extract_to_sdk () {
|
||||
pushd $SDKTMP
|
||||
rpm2cpio $1 | cpio -id
|
||||
popd
|
||||
}
|
||||
|
||||
SDKDIR=$MY_WORKSPACE/export/cgts-sdk
|
||||
SDKTMP=$MY_WORKSPACE/export/cgts-sdk_tmp
|
||||
SDKTMPFILE=$MY_WORKSPACE/export/sdk_tmp_file
|
||||
|
||||
# These patterns match packages for which we find SDK tarballs
|
||||
PATTERNS=(wrs-branding*.x86_64.rpm \
|
||||
wrs-heat-templates*.x86_64.rpm \
|
||||
*cgts-sdk*.x86_64.rpm \
|
||||
install-log-server*.x86_64.rpm \
|
||||
restapi-doc*.x86_64.rpm \
|
||||
remote-clients*.x86_64.rpm )
|
||||
|
||||
rm -f $SDKTMPFILE
|
||||
|
||||
for pkg in "${EXPLICIT_PACKAGES[@]}"; do
|
||||
ls $MY_WORKSPACE/std/rpmbuild/RPMS/$pkg*.x86_64.rpm > /dev/null
|
||||
if [ $? -eq 2 ]; then # no page found...
|
||||
build-rpms --std $pkg --no-descendants
|
||||
fi
|
||||
done
|
||||
|
||||
for pat in "${PATTERNS[@]}"; do
|
||||
find $MY_WORKSPACE/std/rpmbuild/RPMS/$pat >> $SDKTMPFILE
|
||||
done
|
||||
|
||||
rm -rf $SDKDIR $SDKTMP
|
||||
mkdir -p $SDKDIR
|
||||
mkdir -p $SDKTMP
|
||||
|
||||
# extract the files
|
||||
while read rpm; do
|
||||
extract_to_sdk $rpm
|
||||
done < $SDKTMPFILE
|
||||
|
||||
find $SDKTMP -name "*.tgz" -exec cp -v {} $SDKDIR \;
|
||||
|
||||
rm -f $SDKTMPFILE
|
||||
rm -rf $SDKTMP
|
||||
|
||||
echo ""
|
||||
echo "Done. SDK in $SDKDIR"
|
||||
echo ""
|
||||
34
build-tools/build-srpms
Executable file
34
build-tools/build-srpms
Executable file
@ -0,0 +1,34 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This program is a wrapper around build-srpms-parallel and build-srpms-serial
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " Create source rpms:"
|
||||
echo " build-srpms [--serial] [args]"
|
||||
}
|
||||
|
||||
SERIAL_FLAG=0
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$1" in
|
||||
--serial) SERIAL_FLAG=1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
which mock_tmpfs_umount >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
SERIAL_FLAG=1
|
||||
fi
|
||||
|
||||
if [ $SERIAL_FLAG -eq 1 ]; then
|
||||
echo "build-srpms-serial $@"
|
||||
build-srpms-serial "$@"
|
||||
else
|
||||
echo "build-srpms-parallel $@"
|
||||
build-srpms-parallel "$@"
|
||||
fi
|
||||
|
||||
1438
build-tools/build-srpms-parallel
Executable file
1438
build-tools/build-srpms-parallel
Executable file
File diff suppressed because it is too large
Load Diff
1161
build-tools/build-srpms-serial
Executable file
1161
build-tools/build-srpms-serial
Executable file
File diff suppressed because it is too large
Load Diff
1
build-tools/build-srpms4
Symbolic link
1
build-tools/build-srpms4
Symbolic link
@ -0,0 +1 @@
|
||||
build-srpms-parallel
|
||||
123
build-tools/build_guest/build-guest-image.py
Executable file
123
build-tools/build_guest/build-guest-image.py
Executable file
@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
#
|
||||
# Build a bootable guest image from the supplied rootfs archive
|
||||
#
|
||||
|
||||
import getopt
|
||||
import guestfs
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
MBR_FILE='/usr/share/syslinux/mbr.bin'
|
||||
MBR_SIZE=440
|
||||
|
||||
def build_image(inputfile, outputfile, extrasize, trace):
|
||||
g = guestfs.GuestFS(python_return_dict=True)
|
||||
|
||||
# Set the trace flag so that we can see each libguestfs call.
|
||||
if trace:
|
||||
g.set_trace(1)
|
||||
|
||||
# Create a raw-format sparse disk image with padding of size
|
||||
inputsize = os.path.getsize(inputfile)
|
||||
g.disk_create(outputfile, "raw", inputsize + extrasize)
|
||||
|
||||
# Attach the new disk image to libguestfs.
|
||||
g.add_drive_opts(outputfile, format="raw", readonly=0)
|
||||
|
||||
# Run the libguestfs back-end.
|
||||
g.launch()
|
||||
|
||||
# Get the list of devices. Because we only added one drive
|
||||
# above, we expect that this list should contain a single
|
||||
# element.
|
||||
devices = g.list_devices()
|
||||
assert(len(devices) == 1)
|
||||
|
||||
# Partition the disk as one single MBR partition.
|
||||
g.part_disk(devices[0], "mbr")
|
||||
|
||||
# Get the list of partitions. We expect a single element, which
|
||||
# is the partition we have just created.
|
||||
partitions = g.list_partitions()
|
||||
assert(len(partitions) == 1)
|
||||
|
||||
# Create a filesystem on the partition.
|
||||
# NOTE: extlinux does not support 64-bit file systems
|
||||
g.mkfs("ext4", partitions[0], features="^64bit")
|
||||
|
||||
# Now mount the filesystem so that we can add files.
|
||||
g.mount(partitions[0], "/")
|
||||
|
||||
# Upload file system files and directories.
|
||||
g.tar_in(inputfile, "/")
|
||||
|
||||
# Install the boot loader
|
||||
g.extlinux("/boot")
|
||||
|
||||
# Unmount the file systems.
|
||||
g.umount_all();
|
||||
|
||||
# Write the master boot record.
|
||||
with open(MBR_FILE, mode='rb') as mbr:
|
||||
mbr_data = mbr.read()
|
||||
assert(len(mbr_data) == MBR_SIZE)
|
||||
g.pwrite_device(devices[0], mbr_data, 0)
|
||||
|
||||
# Mark the device as bootable.
|
||||
g.part_set_bootable(devices[0], 1, 1)
|
||||
|
||||
# Label the boot disk for root identification
|
||||
g.set_label(partitions[0], "wrs_guest")
|
||||
|
||||
# Shutdown and close guest image
|
||||
g.shutdown()
|
||||
g.close()
|
||||
|
||||
|
||||
def exit_usage(result=0):
|
||||
print('USAGE: -i <input-file> -o <output-file> [-s <extra-bytes>]')
|
||||
sys.exit(result)
|
||||
|
||||
|
||||
def main(argv):
|
||||
inputfile = None
|
||||
outputfile = None
|
||||
extrasize = None
|
||||
trace = False
|
||||
|
||||
try:
|
||||
opts, args = getopt.getopt(argv,"hxi:o:s:",
|
||||
["input=", "output=", "size="])
|
||||
except getopt.GetoptError:
|
||||
exit_usage(2)
|
||||
for opt, arg in opts:
|
||||
if opt == '-h':
|
||||
exit_usage()
|
||||
if opt == '-x':
|
||||
trace = True
|
||||
elif opt in ("-i", "--input"):
|
||||
inputfile = arg
|
||||
elif opt in ("-o", "--output"):
|
||||
outputfile = arg
|
||||
elif opt in ("-s", "--size"):
|
||||
extrasize = int(arg)
|
||||
|
||||
if not inputfile:
|
||||
print(stderr, "ERROR: missing input file")
|
||||
exit_usage(-1)
|
||||
|
||||
if not outputfile:
|
||||
print(stderr, "ERROR: missing output file")
|
||||
exit_usage(-1)
|
||||
|
||||
if not extrasize:
|
||||
extrasize = 0
|
||||
|
||||
build_image(inputfile, outputfile, extrasize, trace)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv[1:])
|
||||
39
build-tools/build_guest/image-rt.inc
Normal file
39
build-tools/build_guest/image-rt.inc
Normal file
@ -0,0 +1,39 @@
|
||||
# List of CGTS packages to be included/installed in guest image
|
||||
# If these have dependencies, they will be pulled in automatically
|
||||
#
|
||||
build-info
|
||||
|
||||
# custom cloud-init configuration
|
||||
|
||||
# guest extensions
|
||||
guest-scale-agent
|
||||
guest-host-comm
|
||||
guest-client
|
||||
|
||||
# Latest version of the i40e drivers from Intel.
|
||||
kmod-i40e-rt
|
||||
kmod-i40evf-rt
|
||||
|
||||
# Latest version of the ixgbe drivers from Intel.
|
||||
kmod-ixgbe-rt
|
||||
kmod-ixgbevf-rt
|
||||
|
||||
# Configure connectx pro devices to ethernet mode.
|
||||
# This is needed for PT devices
|
||||
# mlx4-config
|
||||
|
||||
# Intel QuickAssist
|
||||
# qat16-guest
|
||||
# qat16-guest-app
|
||||
qat17-rt
|
||||
|
||||
# TPM2 libs to enable vTPM on Guest
|
||||
tss2
|
||||
tpm2-tools
|
||||
kmod-tpm-rt
|
||||
|
||||
# ima plugin for RPM
|
||||
rpm-plugin-systemd-inhibit
|
||||
|
||||
# Add cfn-push-stats for heat demos
|
||||
heat-cfntools
|
||||
44
build-tools/build_guest/image.inc
Normal file
44
build-tools/build_guest/image.inc
Normal file
@ -0,0 +1,44 @@
|
||||
# List of CGTS packages to be included/installed in guest image
|
||||
# If these have dependencies, they will be pulled in automatically
|
||||
#
|
||||
build-info
|
||||
|
||||
# custom cloud-init configuration
|
||||
|
||||
# guest extensions
|
||||
guest-scale-agent
|
||||
guest-host-comm
|
||||
guest-client
|
||||
|
||||
# Latest version of the i40e drivers from Intel.
|
||||
kmod-i40e
|
||||
kmod-i40evf
|
||||
|
||||
# Latest version of the ixgbe drivers from Intel.
|
||||
kmod-ixgbe
|
||||
kmod-ixgbevf
|
||||
|
||||
# Intel QuickAssist
|
||||
# qat16-guest
|
||||
# qat16-guest-app
|
||||
qat17
|
||||
|
||||
# TPM2 libs to enable vTPM on Guest
|
||||
tss2
|
||||
tpm2-tools
|
||||
kmod-tpm
|
||||
|
||||
# This will help us have our automation debug TC failures when pings to VMs fail.
|
||||
qemu-guest-agent
|
||||
|
||||
# ima plugin for rpm
|
||||
rpm-plugin-systemd-inhibit
|
||||
|
||||
# Add debugging tools
|
||||
perf
|
||||
zip
|
||||
unzip
|
||||
traceroute
|
||||
|
||||
# Add cfn-push-stats for heat demos
|
||||
heat-cfntools
|
||||
13
build-tools/build_guest/rootfs-exclude.txt
Normal file
13
build-tools/build_guest/rootfs-exclude.txt
Normal file
@ -0,0 +1,13 @@
|
||||
# exclude special filesystems
|
||||
/builddir
|
||||
/dev/*
|
||||
/proc/*
|
||||
/tmp/*
|
||||
/sys/*
|
||||
/root/rootfs.tar
|
||||
|
||||
# exclude local repo yum configuration
|
||||
/etc/yum/yum.conf
|
||||
|
||||
# omit platform hooks to check install uuid
|
||||
/etc/dhcp/dhclient-enter-hooks
|
||||
7
build-tools/build_guest/rootfs-rt/boot/extlinux.conf
Normal file
7
build-tools/build_guest/rootfs-rt/boot/extlinux.conf
Normal file
@ -0,0 +1,7 @@
|
||||
SERIAL 0 115200
|
||||
|
||||
DEFAULT linux
|
||||
LABEL linux
|
||||
KERNEL vmlinuz
|
||||
INITRD initramfs.img
|
||||
APPEND rw root=LABEL=wrs_guest clocksource=pit console=tty0 console=ttyS0 biosdevname=0 net.ifnames=0 no_timer_check audit=0 cgroup_disable=memory isolcpus=1-3 irqaffinity=0 nmi_watchdog=0 softlockup_panic=0 intel_idle.max_cstate=0 processor.max_cstate=1 idle=poll
|
||||
90
build-tools/build_guest/rootfs-setup.sh
Executable file
90
build-tools/build_guest/rootfs-setup.sh
Executable file
@ -0,0 +1,90 @@
|
||||
#!/bin/bash
|
||||
|
||||
BUILD_MODE=''
|
||||
if [ "$1" == "--rt" ]; then
|
||||
BUILD_MODE="rt"
|
||||
fi
|
||||
if [ "$1" == "--std" ]; then
|
||||
BUILD_MODE="std"
|
||||
fi
|
||||
|
||||
# Setup boot directory for syslinux configuration (/boot/extlinux.conf)
|
||||
ln -s $(ls /boot/vmlinuz-*.x86_64 | head -1) /boot/vmlinuz
|
||||
ln -s $(ls /boot/initramfs-*.x86_64.img | head -1) /boot/initramfs.img
|
||||
|
||||
# Setup root and wrsroot users
|
||||
usermod -p $(openssl passwd -1 root) root
|
||||
useradd -p $(openssl passwd -1 wrsroot) wrsroot
|
||||
|
||||
# Enable SUDO access for wrsroot
|
||||
echo "wrsroot ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
|
||||
|
||||
# Enable remote root login to permit automated tools to run privileged commands
|
||||
sed -i 's%^#\(PermitRootLogin \)%\1%' /etc/ssh/sshd_config
|
||||
sed -i 's#^\(PermitRootLogin \).*#\1yes#' /etc/ssh/sshd_config
|
||||
|
||||
# Enable password login to permit automated tools to run commands
|
||||
sed -i 's%^#\(PasswordAuthentication \)%\1%' /etc/ssh/sshd_config
|
||||
sed -i 's#^\(PasswordAuthentication \).*#\1yes#' /etc/ssh/sshd_config
|
||||
|
||||
# Disable PAM authentication
|
||||
sed -i 's#^\(UsePAM \).*#\1no#' /etc/ssh/sshd_config
|
||||
|
||||
# Prevent cloud_init for reverting our changes
|
||||
sed -i 's#^\(ssh_pwauth:\).*#\1 1#' /etc/cloud/cloud.cfg
|
||||
sed -i 's#^\(disable_root:\).*#\1 0#' /etc/cloud/cloud.cfg
|
||||
|
||||
# Setup SSHD to mark packets for QoS processing in the host (this seems to
|
||||
# be broken in our version of SSHd so equivalent iptables rules are being
|
||||
# added to compensate.
|
||||
echo "IPQoS cs7" >> /etc/ssh/sshd_config
|
||||
|
||||
# Disable reverse path filtering to permit traffic testing from
|
||||
# foreign routes.
|
||||
sed -i 's#^\(net.ipv4.conf.*.rp_filter=\).*#\10#' /etc/sysctl.conf
|
||||
|
||||
# Change /etc/rc.local to touch a file to indicate that the init has
|
||||
# completed. This is required by the AVS vbenchmark tool so that it knows
|
||||
# that the VM is ready to run. This was added because VM instances take a
|
||||
# long time (2-3 minutes) to resize their filesystem when run on a system with
|
||||
# HDD instead of SSD.
|
||||
chmod +x /etc/rc.d/rc.local
|
||||
echo "touch /var/run/.init-complete" >> /etc/rc.local
|
||||
|
||||
if [ "$BUILD_MODE" == "rt" ]; then
|
||||
# Adjust system tuning knobs during init when using rt kernel (CGTS-7047)
|
||||
echo "echo 1 > /sys/devices/virtual/workqueue/cpumask" >> /etc/rc.local
|
||||
echo "echo 1 > /sys/bus/workqueue/devices/writeback/cpumask" >> /etc/rc.local
|
||||
echo "echo -1 > /proc/sys/kernel/sched_rt_runtime_us" >> /etc/rc.local
|
||||
echo "echo 0 > /proc/sys/kernel/timer_migration" >> /etc/rc.local
|
||||
echo "echo 10 > /proc/sys/vm/stat_interval" >> /etc/rc.local
|
||||
fi
|
||||
|
||||
# Disable audit service by default
|
||||
# With this enabled, it causes system delays when running at maximum
|
||||
# capacity that impacts the traffic processing enough to cause unclean
|
||||
# traffic runs when doing benchmark tests.
|
||||
systemctl disable auditd
|
||||
|
||||
if [ "$BUILD_MODE" == "rt" ]; then
|
||||
# Additional services to disable on rt guest (CGTS-7047)
|
||||
systemctl disable polkit.service
|
||||
systemctl disable tuned.service
|
||||
fi
|
||||
|
||||
# Clean the yum cache. We don't want to maintain it on the guest file system.
|
||||
yum clean all
|
||||
|
||||
# update /etc/rsyslog.conf to have OmitLocalLogging off
|
||||
sed -i 's#OmitLocalLogging on#OmitLocalLogging off#g' /etc/rsyslog.conf
|
||||
|
||||
# select correct kernel and initrd
|
||||
if [ "$BUILD_MODE" == "rt" ]; then
|
||||
PATTERN=$(rpm -q --qf '%{VERSION}-%{RELEASE}' kernel-rt)
|
||||
else
|
||||
PATTERN=$(rpm -q --qf '%{VERSION}-%{RELEASE}' kernel)
|
||||
fi
|
||||
cd /boot
|
||||
rm -f vmlinuz initramfs.img
|
||||
ln -s $(ls -1 vmlinuz-$PATTERN*) vmlinuz
|
||||
ln -s $(ls -1 initramfs-$PATTERN*img) initramfs.img
|
||||
7
build-tools/build_guest/rootfs-std/boot/extlinux.conf
Normal file
7
build-tools/build_guest/rootfs-std/boot/extlinux.conf
Normal file
@ -0,0 +1,7 @@
|
||||
SERIAL 0 115200
|
||||
|
||||
DEFAULT linux
|
||||
LABEL linux
|
||||
KERNEL vmlinuz
|
||||
INITRD initramfs.img
|
||||
APPEND rw root=LABEL=wrs_guest clocksource=pit console=tty0 console=ttyS0 biosdevname=0 net.ifnames=0 no_timer_check
|
||||
@ -0,0 +1,18 @@
|
||||
# Override the datasource list to use only those that are expected (and needed)
|
||||
# to work in our lab environment.
|
||||
#
|
||||
datasource_list:
|
||||
- NoCloud
|
||||
- ConfigDrive
|
||||
- Ec2
|
||||
- None
|
||||
|
||||
# Adjust the Ec2 max_wait to be 30 seconds instead of the default 120 seconds,
|
||||
# and set the list of URLs to be the only one that we expect to work in our lab
|
||||
# environment so that we avoid DNS lookup failures for alternate choices.
|
||||
#
|
||||
datasource:
|
||||
Ec2:
|
||||
timeout: 10
|
||||
max_wait: 30
|
||||
metadata_urls: ['http://169.254.169.254']
|
||||
21
build-tools/build_guest/rootfs/etc/dhcp/dhclient.conf
Normal file
21
build-tools/build_guest/rootfs/etc/dhcp/dhclient.conf
Normal file
@ -0,0 +1,21 @@
|
||||
## Use a CID based on the hardware address for both IPv4 and IPv6. This mostly
|
||||
## useful for IPv6 to ensure that the client is not using a random DUID for the
|
||||
## CID on each reboot.
|
||||
send dhcp6.client-id = concat(00:03:00, hardware);
|
||||
send dhcp-client-identifier = concat(00:03:00, hardware);
|
||||
|
||||
## Defaults for all interfaces
|
||||
request interface-mtu, subnet-mask, broadcast-address, time-offset,
|
||||
classless-static-routes;
|
||||
|
||||
interface "eth0" {
|
||||
## Override for eth0 to add requests for attributes that we only care to
|
||||
## configure for our primary network interface
|
||||
request interface-mtu, subnet-mask, broadcast-address, time-offset,
|
||||
domain-name, domain-name-servers, host-name,
|
||||
classless-static-routes, routers;
|
||||
}
|
||||
|
||||
timeout 15;
|
||||
|
||||
retry 5;
|
||||
12
build-tools/build_guest/rootfs/etc/iptables.rules
Normal file
12
build-tools/build_guest/rootfs/etc/iptables.rules
Normal file
@ -0,0 +1,12 @@
|
||||
*mangle
|
||||
:PREROUTING ACCEPT [0:0]
|
||||
:INPUT ACCEPT [0:0]
|
||||
:FORWARD ACCEPT [0:0]
|
||||
:OUTPUT ACCEPT [0:0]
|
||||
:POSTROUTING ACCEPT [0:0]
|
||||
-A OUTPUT -o eth0 -p tcp --sport 22 -j DSCP --set-dscp-class CS7
|
||||
-A OUTPUT -o eth0 -p tcp --dport 22 -j DSCP --set-dscp-class CS7
|
||||
-A OUTPUT -o eth0 -p udp --sport 67:68 -j DSCP --set-dscp-class CS7
|
||||
-A OUTPUT -o eth0 -p udp --dport 67:68 -j DSCP --set-dscp-class CS7
|
||||
-A OUTPUT -o eth0 -d 169.254.169.254 -j DSCP --set-dscp-class CS7
|
||||
COMMIT
|
||||
@ -0,0 +1 @@
|
||||
blacklist floppy
|
||||
@ -0,0 +1 @@
|
||||
options wrs_avp kthread_cpulist=0-7 kthread_policy=0
|
||||
@ -0,0 +1 @@
|
||||
wrs_avp
|
||||
@ -0,0 +1,8 @@
|
||||
DEVICE=eth0
|
||||
BOOTPROTO=dhcp
|
||||
ONBOOT=yes
|
||||
TYPE=Ethernet
|
||||
USERCTL=yes
|
||||
PEERDNS=yes
|
||||
IPV6INIT=no
|
||||
PERSISTENT_DHCLIENT=1
|
||||
@ -0,0 +1,4 @@
|
||||
# Renames interfaces to be sequential ethX interface names regardless of interface type
|
||||
# This is required to avoid a kernel host patch that starts number at 1000 and to
|
||||
# override slot specific naming for non-kernel interfaces.
|
||||
ACTION=="add", SUBSYSTEM=="net", DRIVERS=="?*", ATTR{type}=="1", KERNEL=="eth?*" PROGRAM=="/usr/lib/udev/renumber_device", NAME="$result"
|
||||
12
build-tools/build_guest/rootfs/usr/lib/udev/renumber_device
Executable file
12
build-tools/build_guest/rootfs/usr/lib/udev/renumber_device
Executable file
@ -0,0 +1,12 @@
|
||||
#!/bin/bash
|
||||
# Renames interfaces to be sequential ethX interface names regardless of interface type
|
||||
# This is required to avoid a kernel host patch that starts number at 1000 and to
|
||||
# override slot specific naming for non-kernel interfaces.
|
||||
|
||||
# The ifindex for the first interface that is not 'lo' will be 2.
|
||||
# Therefore adjust the numbering to start at 0 for eth0..ethN naming
|
||||
|
||||
INDEX=$(($IFINDEX-2))
|
||||
echo "eth$INDEX"
|
||||
|
||||
exit 0
|
||||
305
build-tools/build_guest/rpm-install-list-rt.txt
Normal file
305
build-tools/build_guest/rpm-install-list-rt.txt
Normal file
@ -0,0 +1,305 @@
|
||||
# list of standard packages to include in the guest image
|
||||
acl
|
||||
acpid
|
||||
audit
|
||||
audit-libs
|
||||
audit-libs-python
|
||||
authconfig
|
||||
basesystem
|
||||
bash
|
||||
bind-libs-lite
|
||||
bind-license
|
||||
binutils
|
||||
bonnie++
|
||||
bridge-utils
|
||||
btrfs-progs
|
||||
bzip2-libs
|
||||
ca-certificates
|
||||
centos-logos
|
||||
centos-release
|
||||
checkpolicy
|
||||
chkconfig
|
||||
chrony
|
||||
cloud-init
|
||||
cloud-utils-growpart
|
||||
coreutils
|
||||
cpio
|
||||
cracklib
|
||||
cracklib-dicts
|
||||
cronie
|
||||
cronie-anacron
|
||||
crontabs
|
||||
cryptsetup-libs
|
||||
curl
|
||||
cyrus-sasl-lib
|
||||
dbus
|
||||
dbus-glib
|
||||
dbus-libs
|
||||
dbus-python
|
||||
device-mapper
|
||||
device-mapper-libs
|
||||
dhclient
|
||||
dhcp-common
|
||||
dhcp-libs
|
||||
diffutils
|
||||
dmidecode
|
||||
dnsmasq
|
||||
dracut
|
||||
dracut-config-generic
|
||||
dracut-config-rescue
|
||||
dracut-network
|
||||
e2fsprogs
|
||||
e2fsprogs-libs
|
||||
elfutils-libelf
|
||||
elfutils-libs
|
||||
ethtool
|
||||
expat
|
||||
file
|
||||
file-libs
|
||||
filesystem
|
||||
findutils
|
||||
fipscheck
|
||||
fipscheck-lib
|
||||
freetype
|
||||
gawk
|
||||
gdbm
|
||||
gettext
|
||||
gettext-libs
|
||||
glib2
|
||||
glibc
|
||||
glibc-common
|
||||
glib-networking
|
||||
gmp
|
||||
gnupg2
|
||||
gnutls
|
||||
gobject-introspection
|
||||
gpgme
|
||||
grep
|
||||
groff-base
|
||||
grub2
|
||||
grub2-tools
|
||||
grubby
|
||||
gsettings-desktop-schemas
|
||||
gssproxy
|
||||
gzip
|
||||
hardlink
|
||||
hostname
|
||||
info
|
||||
initscripts
|
||||
iperf3
|
||||
iproute
|
||||
iptables
|
||||
iputils
|
||||
irqbalance
|
||||
iwl7265-firmware
|
||||
jansson
|
||||
jbigkit-libs
|
||||
json-c
|
||||
kbd
|
||||
kbd-legacy
|
||||
kbd-misc
|
||||
kernel-rt
|
||||
kernel-rt-tools
|
||||
kernel-rt-tools-libs
|
||||
kexec-tools
|
||||
keyutils
|
||||
keyutils-libs
|
||||
kmod
|
||||
kmod-libs
|
||||
kpartx
|
||||
krb5-libs
|
||||
less
|
||||
libacl
|
||||
libassuan
|
||||
libattr
|
||||
libbasicobjects
|
||||
libblkid
|
||||
libcap
|
||||
libcap-ng
|
||||
libcgroup
|
||||
libcollection
|
||||
libcom_err
|
||||
libcroco
|
||||
libcurl
|
||||
libdaemon
|
||||
libdb
|
||||
libdb-utils
|
||||
libedit
|
||||
libestr
|
||||
libevent
|
||||
libffi
|
||||
libgcc
|
||||
libgcrypt
|
||||
libgomp
|
||||
libgpg-error
|
||||
libgudev1
|
||||
libidn
|
||||
libini_config
|
||||
libjpeg-turbo
|
||||
libmnl
|
||||
libmodman
|
||||
libmount
|
||||
libndp
|
||||
libnetfilter_conntrack
|
||||
libnfnetlink
|
||||
libnfsidmap
|
||||
libnl3
|
||||
libnl3-cli
|
||||
libpath_utils
|
||||
libpcap
|
||||
libpipeline
|
||||
libproxy
|
||||
libpwquality
|
||||
libref_array
|
||||
libselinux
|
||||
libselinux-python
|
||||
libselinux-utils
|
||||
libsemanage
|
||||
libsemanage-python
|
||||
libsepol
|
||||
libsoup
|
||||
libss
|
||||
libssh2
|
||||
libstdc++
|
||||
libsysfs
|
||||
libtalloc
|
||||
libtasn1
|
||||
libteam
|
||||
libtevent
|
||||
libtiff
|
||||
libtirpc
|
||||
libunistring
|
||||
libuser
|
||||
libutempter
|
||||
libuuid
|
||||
libverto
|
||||
libverto-tevent
|
||||
libwebp
|
||||
libxml2
|
||||
libyaml
|
||||
logrotate
|
||||
lua
|
||||
lzo
|
||||
make
|
||||
man-db
|
||||
mariadb-libs
|
||||
microcode_ctl
|
||||
mozjs17
|
||||
ncurses
|
||||
ncurses-base
|
||||
ncurses-libs
|
||||
nettle
|
||||
net-tools
|
||||
newt
|
||||
newt-python
|
||||
nfs-utils
|
||||
nspr
|
||||
nss
|
||||
nss-softokn
|
||||
nss-softokn-freebl
|
||||
nss-sysinit
|
||||
nss-tools
|
||||
nss-util
|
||||
numactl-libs
|
||||
openssh
|
||||
openssh-clients
|
||||
openssh-server
|
||||
openssl
|
||||
openssl-libs
|
||||
os-prober
|
||||
p11-kit
|
||||
p11-kit-trust
|
||||
pam
|
||||
parted
|
||||
passwd
|
||||
pciutils
|
||||
pciutils-libs
|
||||
pcre
|
||||
pinentry
|
||||
pkgconfig
|
||||
policycoreutils
|
||||
policycoreutils-python
|
||||
polkit
|
||||
polkit-pkla-compat
|
||||
popt
|
||||
postfix
|
||||
ppp
|
||||
procps-ng
|
||||
pth
|
||||
pygobject3-base
|
||||
pygpgme
|
||||
pyliblzma
|
||||
python
|
||||
python-backports
|
||||
python-backports-ssl_match_hostname
|
||||
python-chardet
|
||||
python-cheetah
|
||||
python-configobj
|
||||
python-decorator
|
||||
python-iniparse
|
||||
python-IPy
|
||||
python-jsonpatch
|
||||
python-jsonpointer
|
||||
python-kitchen
|
||||
python-libs
|
||||
python-markdown
|
||||
python-perf
|
||||
python-pillow
|
||||
python-prettytable
|
||||
python-pycurl
|
||||
python-pygments
|
||||
python-pyudev
|
||||
python-requests
|
||||
python2-six
|
||||
python-urlgrabber
|
||||
python-urllib3
|
||||
pyxattr
|
||||
PyYAML
|
||||
qrencode-libs
|
||||
quota
|
||||
quota-nls
|
||||
rdma
|
||||
readline
|
||||
rootfiles
|
||||
rpcbind
|
||||
rpm
|
||||
rpm-build-libs
|
||||
rpm-libs
|
||||
rpm-python
|
||||
rsync
|
||||
rsyslog
|
||||
sed
|
||||
rt-setup
|
||||
rtctl
|
||||
shadow-utils
|
||||
shared-mime-info
|
||||
slang
|
||||
snappy
|
||||
sqlite
|
||||
sudo
|
||||
systemd
|
||||
systemd-libs
|
||||
systemd-sysv
|
||||
sysvinit-tools
|
||||
tar
|
||||
tcpdump
|
||||
tcp_wrappers
|
||||
tcp_wrappers-libs
|
||||
teamd
|
||||
trousers
|
||||
tuned
|
||||
tzdata
|
||||
ustr
|
||||
util-linux
|
||||
vim-minimal
|
||||
virt-what
|
||||
wget
|
||||
which
|
||||
wpa_supplicant
|
||||
xz
|
||||
xz-libs
|
||||
yum
|
||||
yum-metadata-parser
|
||||
yum-plugin-fastestmirror
|
||||
yum-utils
|
||||
zlib
|
||||
303
build-tools/build_guest/rpm-install-list.txt
Normal file
303
build-tools/build_guest/rpm-install-list.txt
Normal file
@ -0,0 +1,303 @@
|
||||
# list of standard packages to include in the guest image
|
||||
acl
|
||||
acpid
|
||||
audit
|
||||
audit-libs
|
||||
audit-libs-python
|
||||
authconfig
|
||||
basesystem
|
||||
bash
|
||||
bind-libs-lite
|
||||
bind-license
|
||||
binutils
|
||||
bonnie++
|
||||
bridge-utils
|
||||
btrfs-progs
|
||||
bzip2-libs
|
||||
ca-certificates
|
||||
centos-logos
|
||||
centos-release
|
||||
checkpolicy
|
||||
chkconfig
|
||||
chrony
|
||||
cloud-init
|
||||
cloud-utils-growpart
|
||||
coreutils
|
||||
cpio
|
||||
cracklib
|
||||
cracklib-dicts
|
||||
cronie
|
||||
cronie-anacron
|
||||
crontabs
|
||||
cryptsetup-libs
|
||||
curl
|
||||
cyrus-sasl-lib
|
||||
dbus
|
||||
dbus-glib
|
||||
dbus-libs
|
||||
dbus-python
|
||||
device-mapper
|
||||
device-mapper-libs
|
||||
dhclient
|
||||
dhcp-common
|
||||
dhcp-libs
|
||||
diffutils
|
||||
dmidecode
|
||||
dnsmasq
|
||||
dracut
|
||||
dracut-config-generic
|
||||
dracut-config-rescue
|
||||
dracut-network
|
||||
e2fsprogs
|
||||
e2fsprogs-libs
|
||||
elfutils-libelf
|
||||
elfutils-libs
|
||||
ethtool
|
||||
expat
|
||||
file
|
||||
file-libs
|
||||
filesystem
|
||||
findutils
|
||||
fipscheck
|
||||
fipscheck-lib
|
||||
freetype
|
||||
gawk
|
||||
gdbm
|
||||
gettext
|
||||
gettext-libs
|
||||
glib2
|
||||
glibc
|
||||
glibc-common
|
||||
glib-networking
|
||||
gmp
|
||||
gnupg2
|
||||
gnutls
|
||||
gobject-introspection
|
||||
gpgme
|
||||
grep
|
||||
groff-base
|
||||
grub2
|
||||
grub2-tools
|
||||
grubby
|
||||
gsettings-desktop-schemas
|
||||
gssproxy
|
||||
gzip
|
||||
hardlink
|
||||
hostname
|
||||
info
|
||||
initscripts
|
||||
iperf3
|
||||
iproute
|
||||
iptables
|
||||
iputils
|
||||
irqbalance
|
||||
iwl7265-firmware
|
||||
jansson
|
||||
jbigkit-libs
|
||||
json-c
|
||||
kbd
|
||||
kbd-legacy
|
||||
kbd-misc
|
||||
kernel
|
||||
kernel-tools
|
||||
kernel-tools-libs
|
||||
kexec-tools
|
||||
keyutils
|
||||
keyutils-libs
|
||||
kmod
|
||||
kmod-libs
|
||||
kpartx
|
||||
krb5-libs
|
||||
less
|
||||
libacl
|
||||
libassuan
|
||||
libattr
|
||||
libbasicobjects
|
||||
libblkid
|
||||
libcap
|
||||
libcap-ng
|
||||
libcgroup
|
||||
libcollection
|
||||
libcom_err
|
||||
libcroco
|
||||
libcurl
|
||||
libdaemon
|
||||
libdb
|
||||
libdb-utils
|
||||
libedit
|
||||
libestr
|
||||
libevent
|
||||
libffi
|
||||
libgcc
|
||||
libgcrypt
|
||||
libgomp
|
||||
libgpg-error
|
||||
libgudev1
|
||||
libidn
|
||||
libini_config
|
||||
libjpeg-turbo
|
||||
libmnl
|
||||
libmodman
|
||||
libmount
|
||||
libndp
|
||||
libnetfilter_conntrack
|
||||
libnfnetlink
|
||||
libnfsidmap
|
||||
libnl3
|
||||
libnl3-cli
|
||||
libpath_utils
|
||||
libpcap
|
||||
libpipeline
|
||||
libproxy
|
||||
libpwquality
|
||||
libref_array
|
||||
libselinux
|
||||
libselinux-python
|
||||
libselinux-utils
|
||||
libsemanage
|
||||
libsemanage-python
|
||||
libsepol
|
||||
libsoup
|
||||
libss
|
||||
libssh2
|
||||
libstdc++
|
||||
libsysfs
|
||||
libtalloc
|
||||
libtasn1
|
||||
libteam
|
||||
libtevent
|
||||
libtiff
|
||||
libtirpc
|
||||
libunistring
|
||||
libuser
|
||||
libutempter
|
||||
libuuid
|
||||
libverto
|
||||
libverto-tevent
|
||||
libwebp
|
||||
libxml2
|
||||
libyaml
|
||||
logrotate
|
||||
lua
|
||||
lzo
|
||||
make
|
||||
man-db
|
||||
mariadb-libs
|
||||
microcode_ctl
|
||||
mozjs17
|
||||
ncurses
|
||||
ncurses-base
|
||||
ncurses-libs
|
||||
nettle
|
||||
net-tools
|
||||
newt
|
||||
newt-python
|
||||
nfs-utils
|
||||
nspr
|
||||
nss
|
||||
nss-softokn
|
||||
nss-softokn-freebl
|
||||
nss-sysinit
|
||||
nss-tools
|
||||
nss-util
|
||||
numactl-libs
|
||||
openssh
|
||||
openssh-clients
|
||||
openssh-server
|
||||
openssl
|
||||
openssl-libs
|
||||
os-prober
|
||||
p11-kit
|
||||
p11-kit-trust
|
||||
pam
|
||||
parted
|
||||
passwd
|
||||
pciutils
|
||||
pciutils-libs
|
||||
pcre
|
||||
pinentry
|
||||
pkgconfig
|
||||
policycoreutils
|
||||
policycoreutils-python
|
||||
polkit
|
||||
polkit-pkla-compat
|
||||
popt
|
||||
postfix
|
||||
ppp
|
||||
procps-ng
|
||||
pth
|
||||
pygobject3-base
|
||||
pygpgme
|
||||
pyliblzma
|
||||
python
|
||||
python-backports
|
||||
python-backports-ssl_match_hostname
|
||||
python-chardet
|
||||
python-cheetah
|
||||
python-configobj
|
||||
python-decorator
|
||||
python-iniparse
|
||||
python-IPy
|
||||
python-jsonpatch
|
||||
python-jsonpointer
|
||||
python-kitchen
|
||||
python-libs
|
||||
python-markdown
|
||||
python-perf
|
||||
python-pillow
|
||||
python-prettytable
|
||||
python-pycurl
|
||||
python-pygments
|
||||
python-pyudev
|
||||
python-requests
|
||||
python2-six
|
||||
python-urlgrabber
|
||||
python-urllib3
|
||||
pyxattr
|
||||
PyYAML
|
||||
qrencode-libs
|
||||
quota
|
||||
quota-nls
|
||||
rdma
|
||||
readline
|
||||
rootfiles
|
||||
rpcbind
|
||||
rpm
|
||||
rpm-build-libs
|
||||
rpm-libs
|
||||
rpm-python
|
||||
rsync
|
||||
rsyslog
|
||||
sed
|
||||
setup
|
||||
shadow-utils
|
||||
shared-mime-info
|
||||
slang
|
||||
snappy
|
||||
sqlite
|
||||
sudo
|
||||
systemd
|
||||
systemd-libs
|
||||
systemd-sysv
|
||||
sysvinit-tools
|
||||
tar
|
||||
tcpdump
|
||||
tcp_wrappers
|
||||
tcp_wrappers-libs
|
||||
teamd
|
||||
trousers
|
||||
tzdata
|
||||
ustr
|
||||
util-linux
|
||||
vim-enhanced
|
||||
virt-what
|
||||
wget
|
||||
which
|
||||
wpa_supplicant
|
||||
xz
|
||||
xz-libs
|
||||
yum
|
||||
yum-metadata-parser
|
||||
yum-plugin-fastestmirror
|
||||
yum-utils
|
||||
zlib
|
||||
7
build-tools/build_guest/rpm-remove-list.txt
Normal file
7
build-tools/build_guest/rpm-remove-list.txt
Normal file
@ -0,0 +1,7 @@
|
||||
# list of packages to be excluded from guest image
|
||||
cpp
|
||||
gcc
|
||||
gcc-c++
|
||||
gdb
|
||||
linux-firmware
|
||||
rpm-build
|
||||
40
build-tools/build_iso/anaconda-ks.cfg
Normal file
40
build-tools/build_iso/anaconda-ks.cfg
Normal file
@ -0,0 +1,40 @@
|
||||
#version=DEVEL
|
||||
# System authorization information
|
||||
auth --enableshadow --passalgo=sha512
|
||||
# Use CDROM installation media
|
||||
cdrom
|
||||
# Use graphical install
|
||||
graphical
|
||||
# Run the Setup Agent on first boot
|
||||
firstboot --enable
|
||||
ignoredisk --only-use=sda
|
||||
# Keyboard layouts
|
||||
keyboard --vckeymap=us --xlayouts='us'
|
||||
# System language
|
||||
lang en_US.UTF-8
|
||||
|
||||
# Network information
|
||||
network --bootproto=dhcp --device=enp0s3 --onboot=off --ipv6=auto
|
||||
network --bootproto=static --device=enp0s8 --ip=10.10.10.10 --netmask=255.255.255.0 --ipv6=auto --activate
|
||||
network --device=lo --hostname=localhost.localdomain
|
||||
|
||||
#Root password
|
||||
rootpw --lock
|
||||
# System timezone
|
||||
timezone America/New_York --isUtc
|
||||
user --groups=wheel --name=wrsroot --password=$6$c3gaCcJlh.rp//Yx$/mIjNNoUDS1qZldBL29YSJdsA9ttPA/nXN1CPsIcCmionXC22APT3IoRSd9j5dPiZoviDdQf7YxLsOYdieOQr/ --iscrypted --gecos="wrsroot"
|
||||
# System bootloader configuration
|
||||
bootloader --location=mbr --boot-drive=sda
|
||||
autopart --type=lvm
|
||||
# Partition clearing information
|
||||
clearpart --all --initlabel --drives=sda
|
||||
|
||||
%packages
|
||||
@^minimal
|
||||
@core
|
||||
|
||||
%end
|
||||
|
||||
%addon com_redhat_kdump --disable --reserve-mb='auto'
|
||||
|
||||
%end
|
||||
296
build-tools/build_iso/cgts_deps.sh
Executable file
296
build-tools/build_iso/cgts_deps.sh
Executable file
@ -0,0 +1,296 @@
|
||||
#!/bin/env bash
|
||||
|
||||
# Here's the score, kids. There are a few different places from which we can
|
||||
# get packages. In priority order, they are:
|
||||
#
|
||||
# The CGTS packages we've built ourselves
|
||||
# The CGTS packages that Jenkins has built (coming soon to a script near you)
|
||||
# The CentOS packages in various repos
|
||||
# - Base OS
|
||||
# - OpenStack Repos
|
||||
# EPEL (Extra Packages for Enterprise Linux)
|
||||
#
|
||||
# This script can function in two ways:
|
||||
# If you specify a filename, it assumes the file is a list of packages you
|
||||
# want to install, or dependencies you want to meet. It installs whatever
|
||||
# is in the list into current directory. Failure to find a dependency
|
||||
# results in a return code of 1
|
||||
#
|
||||
# If no file is specified, we generate a file ($DEPLISTFILE) of dependencies
|
||||
# based on current directory
|
||||
#
|
||||
# We then continuously loop through generating new dependencies and installing
|
||||
# them until either all dependencies are met, or we cannot install anymore
|
||||
#
|
||||
# We also log where dependencies were installed from into
|
||||
# export/dist/report_deps.txt
|
||||
#
|
||||
|
||||
# This function generates a simple file of dependencies we're trying to resolve
|
||||
function generate_dep_list {
|
||||
TMP_RPM_DB=$(mktemp -d $(pwd)/tmp_rpm_db_XXXXXX)
|
||||
mkdir -p $TMP_RPM_DB
|
||||
rpm --initdb --dbpath $TMP_RPM_DB
|
||||
rpm --dbpath $TMP_RPM_DB --test -Uvh --replacefiles '*.rpm' >> $DEPDETAILLISTFILE 2>&1
|
||||
rpm --dbpath $TMP_RPM_DB --test -Uvh --replacefiles '*.rpm' 2>&1 \
|
||||
| grep -v "error:" \
|
||||
| grep -v "warning:" \
|
||||
| grep -v "Preparing..." \
|
||||
| sed "s/ is needed by.*$//" | sed "s/ >=.*$//" | sort -u > $DEPLISTFILE
|
||||
\rm -rf $TMP_RPM_DB
|
||||
}
|
||||
|
||||
# Takes a list of requirements (either explcit package name, or capabilities
|
||||
# to provide) and install packages to meet those dependancies
|
||||
#
|
||||
# We take the list of requirements and first try to look them up based on
|
||||
# package name. If we can't find a package with the name of the requirement,
|
||||
# we use --whatprovides to complete the lookup.
|
||||
#
|
||||
# The reason for this initial name-based attempt is that a couple of funky
|
||||
# packages (notably -devel packages) have "Provides:" capabilities which
|
||||
# conflict with named packages. So if explictly say we want "xyz" then we'll
|
||||
# install the "xyz" package, rather than "something-devel" which has "xyz"
|
||||
# capabilities.
|
||||
function install_deps {
|
||||
local DEP_LIST=""
|
||||
local DEP_LIST_FILE="$1"
|
||||
|
||||
# Temporary files are used in a few different ways
|
||||
# Here we essenitally create variable aliases to make it easier to read
|
||||
# the script
|
||||
local UNSORTED_PACKAGES=$TMPFILE
|
||||
local SORTED_PACKAGES=$TMPFILE1
|
||||
local UNRESOLVED_PACKAGES=$TMPFILE2
|
||||
|
||||
rm -f $UNSORTED_PACKAGES
|
||||
|
||||
while read DEP
|
||||
do
|
||||
DEP_LIST="${DEP_LIST} ${DEP}"
|
||||
done < $DEP_LIST_FILE
|
||||
|
||||
echo "Debug: List of deps to resolve: ${DEP_LIST}"
|
||||
|
||||
if [ -z "${DEP_LIST}" ]
|
||||
then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# go through each repo and convert deps to packages based on package name
|
||||
for REPOID in `grep '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
|
||||
echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch ${DEP_LIST} --qf='%{name}'"
|
||||
TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --qf='%{name}' ${DEP_LIST} | sed "s/kernel-debug/kernel/g" >> $UNSORTED_PACKAGES
|
||||
\rm -rf $TMP_DIR/yum-$USER-*
|
||||
done
|
||||
sort $UNSORTED_PACKAGES -u > $SORTED_PACKAGES
|
||||
|
||||
# figure out any dependancies which could not be resolved based on
|
||||
# package name. We use --whatpovides to deal with this
|
||||
#
|
||||
# First, we build a new DEP_LIST based on what was NOT found in
|
||||
# search-by-name attempt
|
||||
sort $DEP_LIST_FILE -u > $TMPFILE
|
||||
comm -2 -3 $TMPFILE $SORTED_PACKAGES > $UNRESOLVED_PACKAGES
|
||||
|
||||
# If there are any requirements not resolved, look up the packages with
|
||||
# --whatprovides
|
||||
if [ -s $UNRESOLVED_PACKAGES ]; then
|
||||
DEP_LIST=""
|
||||
\cp $SORTED_PACKAGES $UNSORTED_PACKAGES
|
||||
while read DEP
|
||||
do
|
||||
DEP_LIST="${DEP_LIST} ${DEP}"
|
||||
done < $UNRESOLVED_PACKAGES
|
||||
|
||||
DEP_LIST=$(echo "$DEP_LIST" | sed 's/^ //g')
|
||||
if [ "$DEP_LIST" != "" ]; then
|
||||
|
||||
for REPOID in `grep '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
|
||||
echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --whatprovides ${DEP_LIST} --qf='%{name}'"
|
||||
TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --qf='%{name}' --whatprovides ${DEP_LIST} | sed "s/kernel-debug/kernel/g" >> $UNSORTED_PACKAGES
|
||||
\rm -rf $TMP_DIR/yum-$USER-*
|
||||
done
|
||||
fi
|
||||
|
||||
sort -u $UNSORTED_PACKAGES > $SORTED_PACKAGES
|
||||
fi
|
||||
|
||||
# clean up
|
||||
\rm -f $UNSORTED_PACKAGES $UNRESOLVED_PACKAGES
|
||||
|
||||
# We now have, in SORTED_PACKAGES, a list of all packages that we need to install
|
||||
# to meet our dependancies
|
||||
DEP_LIST=" "
|
||||
while read DEP
|
||||
do
|
||||
DEP_LIST="${DEP_LIST}${DEP} "
|
||||
done < $SORTED_PACKAGES
|
||||
rm $SORTED_PACKAGES
|
||||
|
||||
# go through each repo and install packages
|
||||
local TARGETS=${DEP_LIST}
|
||||
echo "Debug: Resolved list of deps to install: ${TARGETS}"
|
||||
local UNRESOLVED
|
||||
for REPOID in `grep '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
|
||||
UNRESOLVED="$TARGETS"
|
||||
|
||||
if [[ ! -z "${TARGETS// }" ]]; then
|
||||
REPO_PATH=$(cat $YUM | sed -n "/^\[$REPOID\]\$/,\$p" | grep '^baseurl=' | head -n 1 | awk -F 'file://' '{print $2}' | sed 's:/$::')
|
||||
>&2 echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf='%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}'"
|
||||
TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf="%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}" | sort -r -V > $TMPFILE
|
||||
\rm -rf $TMP_DIR/yum-$USER-*
|
||||
|
||||
while read STR
|
||||
do
|
||||
>&2 echo "STR=$STR"
|
||||
if [ "x$STR" == "x" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
PKG=`echo $STR | cut -d " " -f 1`
|
||||
PKG_FILE=`echo $STR | cut -d " " -f 2`
|
||||
PKG_REL_PATH=`echo $STR | cut -d " " -f 3`
|
||||
PKG_PATH="${REPO_PATH}/${PKG_REL_PATH}"
|
||||
|
||||
>&2 echo "Installing PKG=$PKG PKG_FILE=$PKG_FILE PKG_REL_PATH=$PKG_REL_PATH PKG_PATH=$PKG_PATH from repo $REPOID"
|
||||
cp $PKG_PATH .
|
||||
if [ $? -ne 0 ]
|
||||
then
|
||||
>&2 echo " Here's what I have to work with..."
|
||||
>&2 echo " TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $PKG --qf=\"%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}\""
|
||||
>&2 echo " PKG=$PKG PKG_FILE=$PKG_FILE REPO_PATH=$REPO_PATH PKG_REL_PATH=$PKG_REL_PATH PKG_PATH=$PKG_PATH"
|
||||
fi
|
||||
|
||||
echo $UNRESOLVED | grep $PKG >> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "$PKG found in $REPOID as $PKG" >> $BUILT_REPORT
|
||||
echo "$PKG_PATH" >> $BUILT_REPORT
|
||||
UNRESOLVED=$(echo "$UNRESOLVED" | sed "s# $PKG # #g")
|
||||
else
|
||||
echo "$PKG satisfies unknown target in $REPOID" >> $BUILT_REPORT
|
||||
echo " but it doesn't match targets, $UNRESOLVED" >> $BUILT_REPORT
|
||||
echo " path $PKG_PATH" >> $BUILT_REPORT
|
||||
FOUND_UNKNOWN=1
|
||||
fi
|
||||
done < $TMPFILE #<<< "$(TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf=\"%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}\" | sort -r -V)"
|
||||
\rm -rf $TMP_DIR/yum-$USER-*
|
||||
TARGETS="$UNRESOLVED"
|
||||
fi
|
||||
done
|
||||
>&2 echo "Debug: Packages still unresolved: $UNRESOLVED"
|
||||
echo "Debug: Packages still unresolved: $UNRESOLVED" >> $WARNINGS_REPORT
|
||||
echo "Debug: Packages still unresolved: $UNRESOLVED" >> $BUILT_REPORT
|
||||
>&2 echo ""
|
||||
}
|
||||
|
||||
function check_all_explicit_deps_installed
|
||||
{
|
||||
|
||||
PKGS_TO_CHECK=" "
|
||||
while read PKG_TO_ADD
|
||||
do
|
||||
PKGS_TO_CHECK="$PKGS_TO_CHECK ${PKG_TO_ADD}"
|
||||
done < $DEPLISTFILE
|
||||
rpm -qp $MY_WORKSPACE/export/dist/isolinux/Packages/*.rpm --qf="%{name}\n" --nosignature > $TMPFILE
|
||||
|
||||
while read INSTALLED_PACKAGE
|
||||
do
|
||||
echo $PKGS_TO_CHECK | grep -q "${INSTALLED_PACKAGE}"
|
||||
if [ $? -eq 0 ]; then
|
||||
PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/^${INSTALLED_PACKAGE} //"`
|
||||
PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/ ${INSTALLED_PACKAGE} / /"`
|
||||
PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/ ${INSTALLED_PACKAGE}\$//"`
|
||||
PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/^${INSTALLED_PACKAGE}\$//"`
|
||||
fi
|
||||
done < $TMPFILE
|
||||
|
||||
if [ -z "$PKGS_TO_CHECK" ]
|
||||
then
|
||||
>&2 echo "All explicitly specified packages resolved!"
|
||||
else
|
||||
>&2 echo "Could not resolve packages: $PKGS_TO_CHECK"
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
ATTEMPTED=0
|
||||
DISCOVERED=0
|
||||
OUTPUT_DIR=$MY_WORKSPACE/export
|
||||
TMP_DIR=$MY_WORKSPACE/tmp
|
||||
YUM=$OUTPUT_DIR/yum.conf
|
||||
DEPLISTFILE=$OUTPUT_DIR/deps.txt
|
||||
DEPDETAILLISTFILE=$OUTPUT_DIR/deps_detail.txt
|
||||
|
||||
BUILT_REPORT=$OUTPUT_DIR/local.txt
|
||||
WARNINGS_REPORT=$OUTPUT_DIR/warnings.txt
|
||||
LAST_TEST=$OUTPUT_DIR/last_test.txt
|
||||
TMPFILE=$OUTPUT_DIR/cgts_deps_tmp.txt
|
||||
TMPFILE1=$OUTPUT_DIR/cgts_deps_tmp1.txt
|
||||
TMPFILE2=$OUTPUT_DIR/cgts_deps_tmp2.txt
|
||||
|
||||
touch "$BUILT_REPORT"
|
||||
touch "$WARNINGS_REPORT"
|
||||
|
||||
for i in "$@"
|
||||
do
|
||||
case $i in
|
||||
-d=*|--deps=*)
|
||||
DEPS="${i#*=}"
|
||||
shift # past argument=value
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
mkdir -p $TMP_DIR
|
||||
|
||||
rm -f "$DEPDETAILLISTFILE"
|
||||
# FIRST PASS we are being given a list of REQUIRED dependencies
|
||||
if [ "${DEPS}x" != "x" ]; then
|
||||
cat $DEPS | grep -v "^#" | sed '/^\s*$/d' > $DEPLISTFILE
|
||||
install_deps $DEPLISTFILE
|
||||
if [ $? -ne 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# check that we resolved them all
|
||||
check_all_explicit_deps_installed
|
||||
if [ $? -ne 0 ]; then
|
||||
>&2 echo "Error -- could not install all explicitly listed packages"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ALL_RESOLVED=0
|
||||
|
||||
while [ $ALL_RESOLVED -eq 0 ]; do
|
||||
cp $DEPLISTFILE $DEPLISTFILE.old
|
||||
generate_dep_list
|
||||
if [ ! -s $DEPLISTFILE ]; then
|
||||
# no more dependencies!
|
||||
ALL_RESOLVED=1
|
||||
else
|
||||
DIFFLINES=`diff $DEPLISTFILE.old $DEPLISTFILE | wc -l`
|
||||
if [ $DIFFLINES -eq 0 ]; then
|
||||
>&2 echo "Warning: Infinite loop detected in dependency resolution. See $DEPLISTFILE for details -- exiting"
|
||||
>&2 echo "These RPMS had problems (likely version conflicts)"
|
||||
>&2 cat $DEPLISTFILE
|
||||
|
||||
echo "Warning: Infinite loop detected in dependency resolution See $DEPLISTFILE for details -- exiting" >> $WARNINGS_REPORT
|
||||
echo "These RPMS had problems (likely version conflicts)" >> $WARNINGS_REPORT
|
||||
cat $DEPLISTFILE >> $WARNINGS_REPORT
|
||||
|
||||
date > $LAST_TEST
|
||||
|
||||
rm -f $DEPLISTFILE.old
|
||||
exit 1 # nothing fixed
|
||||
fi
|
||||
install_deps $DEPLISTFILE
|
||||
if [ $? -ne 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
exit 0
|
||||
BIN
build-tools/build_iso/comps.xml.gz
Normal file
BIN
build-tools/build_iso/comps.xml.gz
Normal file
Binary file not shown.
122
build-tools/build_iso/gather_packages.pl
Executable file
122
build-tools/build_iso/gather_packages.pl
Executable file
@ -0,0 +1,122 @@
|
||||
#!/usr/bin/perl
|
||||
|
||||
# Copy/pasted from http://www.smorgasbork.com/content/gather_packages.txt
|
||||
# As referenced by http://www.smorgasbork.com/2012/01/04/building-a-custom-centos-7-kickstart-disc-part-2/
|
||||
|
||||
use XML::Simple;
|
||||
|
||||
my ($comps_file, $rpm_src_path, $rpm_dst_path, $arch, @extra_groups_and_packages) = @ARGV;
|
||||
|
||||
if (!-e $comps_file)
|
||||
{
|
||||
print_usage ("Can't find '$comps_file'");
|
||||
}
|
||||
if (!-e $rpm_src_path)
|
||||
{
|
||||
print_usage ("RPM source path '$rpm_src_path' does not exist");
|
||||
}
|
||||
if (!-e $rpm_dst_path)
|
||||
{
|
||||
print_usage ("RPM destination path '$rpm_dst_path' does not exist");
|
||||
}
|
||||
if (!$arch)
|
||||
{
|
||||
print_usage ("Architecture not specified");
|
||||
}
|
||||
|
||||
#### we always gather core and base; note that for CentOS 7, we also need
|
||||
#### to include the grub2 package, or installation will fail
|
||||
@desired_groups = ('core', 'base', 'grub2');
|
||||
foreach (@extra_groups_and_packages)
|
||||
{
|
||||
push (@desired_groups, $_);
|
||||
}
|
||||
|
||||
$regex = '^(' . join ('|', @desired_groups) . ')$';
|
||||
|
||||
print "reading $comps_file...\n";
|
||||
print "getting RPMs from $rpm_src_path...\n";
|
||||
|
||||
$xml = new XML::Simple;
|
||||
$comps = $xml->XMLin($comps_file);
|
||||
|
||||
$cmd = "rm $rpm_dst_path/*";
|
||||
print "$cmd\n";
|
||||
`$cmd`;
|
||||
|
||||
%copied_groups = {};
|
||||
%copied_packages = {};
|
||||
|
||||
foreach $group (@{$comps->{group}})
|
||||
{
|
||||
$id = $group->{id};
|
||||
if ($id !~ m#$regex#)
|
||||
{
|
||||
next;
|
||||
}
|
||||
|
||||
print "#### group \@$id\n";
|
||||
$packagelist = $group->{packagelist};
|
||||
foreach $pr (@{$packagelist->{packagereq}})
|
||||
{
|
||||
if ($pr->{type} eq 'optional' || $pr->{type} eq 'conditional')
|
||||
{
|
||||
next;
|
||||
}
|
||||
|
||||
$cmd = "cp $rpm_src_path/" . $pr->{content} . "-[0-9]*.$arch.rpm"
|
||||
. " $rpm_src_path/" . $pr->{content} . "-[0-9]*.noarch.rpm $rpm_dst_path";
|
||||
print "$cmd\n";
|
||||
`$cmd 2>&1`;
|
||||
|
||||
$copied_packages{$pr->{content}} = 1;
|
||||
}
|
||||
|
||||
$copied_groups{$group} = 1;
|
||||
}
|
||||
|
||||
#### assume that any strings that weren't matched in the comps file's group list
|
||||
#### are actually packages
|
||||
|
||||
foreach $group (@desired_groups)
|
||||
{
|
||||
if ($copied_groups{$group})
|
||||
{
|
||||
next;
|
||||
}
|
||||
|
||||
$cmd = "cp $rpm_src_path/" . $group . "-[0-9]*.$arch.rpm"
|
||||
. " $rpm_src_path/" . $group . "-[0-9]*.noarch.rpm $rpm_dst_path";
|
||||
print "$cmd\n";
|
||||
`$cmd 2>&1`;
|
||||
}
|
||||
|
||||
sub print_usage
|
||||
{
|
||||
my ($msg) = @_;
|
||||
|
||||
($msg) && print "$msg\n\n";
|
||||
|
||||
print <<__TEXT__;
|
||||
|
||||
parse_comps.pl comps_file rpm_src_path arch [xtra_grps_and_pkgs]
|
||||
|
||||
comps_file the full path to the comps.xml file (as provided
|
||||
in the original distro
|
||||
|
||||
rpm_src_path the full path to the directory of all RPMs from
|
||||
the distro
|
||||
|
||||
rpm_dst_path the full path to the directory where you want
|
||||
to save the RPMs for your kickstart
|
||||
|
||||
arch the target system architecture (e.g. x86_64)
|
||||
|
||||
xtra_grps_and_pkgs a list of extra groups and packages, separated by spaces
|
||||
|
||||
|
||||
__TEXT__
|
||||
|
||||
exit;
|
||||
}
|
||||
|
||||
4
build-tools/build_iso/image-dev.inc
Normal file
4
build-tools/build_iso/image-dev.inc
Normal file
@ -0,0 +1,4 @@
|
||||
# The following packages will not be included in the customer ISO
|
||||
#
|
||||
# They are exceptional packages only to be included in developer builds
|
||||
enable-dev-patch
|
||||
364
build-tools/build_iso/image.inc
Normal file
364
build-tools/build_iso/image.inc
Normal file
@ -0,0 +1,364 @@
|
||||
# List of CGTS packages to be included/installed in ISO
|
||||
# If these have dependencies, they will be pulled in automatically
|
||||
#
|
||||
acpid
|
||||
audit
|
||||
build-info
|
||||
bash
|
||||
ceph-manager
|
||||
cgcs-patch-agent
|
||||
cgcs-patch-controller
|
||||
cgts-mtce-compute
|
||||
cgts-mtce-control
|
||||
cgts-mtce-storage
|
||||
computeconfig
|
||||
computeconfig-standalone
|
||||
computeconfig-subfunction
|
||||
compute-huge
|
||||
dnsmasq
|
||||
dnsmasq-utils
|
||||
dpkg
|
||||
filesystem-scripts
|
||||
fm-api
|
||||
gdb
|
||||
grub2
|
||||
grub2-efi
|
||||
grub2-efi-modules
|
||||
grub2-tools
|
||||
haproxy
|
||||
io-monitor
|
||||
iperf3
|
||||
isomd5sum
|
||||
iscsi-initiator-utils
|
||||
iscsi-initiator-utils-iscsiuio
|
||||
kmod-drbd
|
||||
libevent
|
||||
libtrap-handler
|
||||
libvirt-docs
|
||||
libvirt-python
|
||||
logrotate
|
||||
lshell
|
||||
net-snmp-libs
|
||||
net-snmp-python
|
||||
net-snmp-utils
|
||||
nfv
|
||||
nfv-common
|
||||
nfv-plugins
|
||||
nfv-tools
|
||||
nfv-vim
|
||||
nfv-client
|
||||
nova-utils
|
||||
ntpdate
|
||||
openldap
|
||||
openldap-clients
|
||||
openldap-servers
|
||||
openssh
|
||||
openssh-clients
|
||||
openssh-server
|
||||
openstack-ras
|
||||
pam
|
||||
parted
|
||||
platform-util
|
||||
platform-util-noncontroller
|
||||
python-cephclient
|
||||
python-keyring
|
||||
qemu-kvm-tools-ev
|
||||
rsync
|
||||
sm-client
|
||||
sm-tools
|
||||
snmp-ext
|
||||
storageconfig
|
||||
sysinv
|
||||
sysinv-agent
|
||||
tsconfig
|
||||
tboot
|
||||
rabbitmq-server
|
||||
cgts-client
|
||||
python-django-horizon
|
||||
libvirt
|
||||
libvirt-client
|
||||
libvirt-daemon
|
||||
libvirt-daemon-config-network
|
||||
libvirt-daemon-config-nwfilter
|
||||
libvirt-daemon-driver-lxc
|
||||
libvirt-daemon-driver-network
|
||||
libvirt-daemon-driver-nodedev
|
||||
libvirt-daemon-driver-nwfilter
|
||||
libvirt-daemon-driver-qemu
|
||||
libvirt-daemon-driver-secret
|
||||
libvirt-daemon-driver-storage
|
||||
nova-api-proxy
|
||||
configutilities
|
||||
controllerconfig
|
||||
fm-common
|
||||
fm-mgr
|
||||
logmgmt
|
||||
cgts-mtce-common
|
||||
cgts-mtce-common-pmon
|
||||
cgts-mtce-common-rmon
|
||||
cgts-mtce-common-hostw
|
||||
cgts-mtce-common-hwmon
|
||||
cgts-mtce-common-guestAgent
|
||||
cgts-mtce-common-guestServer
|
||||
cgcs-patch
|
||||
patch-alarm
|
||||
# patch-boot-args is not ported to centos yet
|
||||
wrs-ssl
|
||||
tss2
|
||||
tpm2-tools
|
||||
tpm2-openssl-engine
|
||||
libtpms
|
||||
swtpm
|
||||
swtpm-tools
|
||||
swtpm-cuse
|
||||
monitor-tools
|
||||
python2-aodhclient
|
||||
openstack-aodh-api
|
||||
openstack-aodh-evaluator
|
||||
openstack-aodh-expirer
|
||||
openstack-aodh-listener
|
||||
openstack-aodh-notifier
|
||||
python-ceilometer
|
||||
#openstack-ceilometer-alarm provided in openstack-aodh-compat package
|
||||
openstack-aodh-compat
|
||||
openstack-ceilometer-api
|
||||
openstack-ceilometer-collector
|
||||
openstack-ceilometer-common
|
||||
openstack-ceilometer-ipmi
|
||||
openstack-ceilometer-notification
|
||||
openstack-ceilometer-polling
|
||||
openstack-cinder
|
||||
openstack-glance
|
||||
openstack-heat-api
|
||||
openstack-heat-api-cfn
|
||||
openstack-heat-api-cloudwatch
|
||||
openstack-heat-common
|
||||
openstack-heat-engine
|
||||
openstack-ironic-api
|
||||
openstack-ironic-common
|
||||
openstack-ironic-conductor
|
||||
python2-ironicclient
|
||||
# doc generation turned off in Mitaka by default
|
||||
#python-django-horizon-doc
|
||||
openstack-keystone
|
||||
openstack-murano-api
|
||||
openstack-murano-cf-api
|
||||
openstack-murano-common
|
||||
openstack-murano-doc
|
||||
openstack-murano-engine
|
||||
openstack-murano-ui
|
||||
openstack-murano-ui-doc
|
||||
python2-muranoclient
|
||||
python-muranoclient-doc
|
||||
openstack-magnum-api
|
||||
openstack-magnum-common
|
||||
openstack-magnum-conductor
|
||||
openstack-magnum-doc
|
||||
openstack-magnum-ui
|
||||
python2-magnumclient
|
||||
python-magnum
|
||||
python-magnumclient-doc
|
||||
python-magnumclient-tests
|
||||
python-magnum-tests
|
||||
openstack-neutron
|
||||
openstack-neutron-common
|
||||
openstack-neutron-sriov-nic-agent
|
||||
openstack-neutron-ml2
|
||||
openstack-neutron-openvswitch
|
||||
python-neutron
|
||||
openstack-nova-api
|
||||
openstack-nova-cells
|
||||
openstack-nova-cert
|
||||
openstack-nova-common
|
||||
openstack-nova-compute
|
||||
openstack-nova-conductor
|
||||
openstack-nova-console
|
||||
openstack-nova-network
|
||||
openstack-nova-novncproxy
|
||||
# openstack-nova-objectstore removed in Mitaka
|
||||
#openstack-nova-objectstore
|
||||
openstack-nova-scheduler
|
||||
openstack-nova-serialproxy
|
||||
openstack-nova-spicehtml5proxy
|
||||
openstack-nova-placement-api
|
||||
python-nova
|
||||
python2-novaclient
|
||||
python2-openstackclient
|
||||
python2-oslo-log
|
||||
python2-django-openstack-auth
|
||||
python2-six
|
||||
#base-files is not yet ported to CENTOS
|
||||
cgcs-users
|
||||
namespace-utils
|
||||
nss-pam-ldapd
|
||||
# thin-provisioning-tools is not yet ported to CENTOS
|
||||
# cluster-resource-agents is not yet ported to CENTOS
|
||||
dhcp
|
||||
nfs-utils
|
||||
initscripts
|
||||
# note -- the "systemd" package provides udev
|
||||
systemd
|
||||
# udev-extraconf is not yet ported to CENTOS
|
||||
# cdrkit is only available as a src.rpm
|
||||
python-d2to1
|
||||
# eucatools is not yet ported to CENTOS
|
||||
facter
|
||||
hiera
|
||||
# libaugesas-ruby is not yet ported to CENTOS
|
||||
nfscheck
|
||||
python2-pecan
|
||||
# pseudo not available
|
||||
# ps-byte-compile is not yet ported to CENTOS
|
||||
python
|
||||
python-configobj
|
||||
python-pep8
|
||||
# python-pyflakes is not yet ported to CENTOS
|
||||
python2-rsa
|
||||
# python-xmlrpclib is not yet ported to CENTOS
|
||||
ruby-shadow
|
||||
swig
|
||||
syslinux
|
||||
iotop
|
||||
iptables
|
||||
iptables-services
|
||||
iptables-utils
|
||||
ldapscripts
|
||||
lighttpd
|
||||
lighttpd-fastcgi
|
||||
lighttpd-mod_geoip
|
||||
lighttpd-mod_mysql_vhost
|
||||
novnc
|
||||
ntp
|
||||
ntp-perl
|
||||
procps-ng
|
||||
python-daemon
|
||||
python2-gunicorn
|
||||
python-pyudev
|
||||
python-smartpm
|
||||
# shadow is not yet ported to CENTOS
|
||||
shadow-utils
|
||||
syslog-ng
|
||||
syslog-ng-libdbi
|
||||
drbd
|
||||
drbd-bash-completion
|
||||
# drbd-debuginfo
|
||||
drbd-heartbeat
|
||||
# drbd-kernel-debuginfo
|
||||
drbd-pacemaker
|
||||
drbd-udev
|
||||
drbd-utils
|
||||
# drbd-tools is not yet ported to CENTOS
|
||||
# nfv-docs should not be included
|
||||
# nfv-tests should not be included
|
||||
curl
|
||||
lvm2
|
||||
# tgt is ported, but not yet added here
|
||||
sm
|
||||
sm-common
|
||||
sm-common-libs
|
||||
sm-db
|
||||
sm-api
|
||||
sm-eru
|
||||
time
|
||||
puppet
|
||||
puppet-manifests
|
||||
openstack-dashboard
|
||||
# openstack-dashboard-theme TODO: fix build-iso
|
||||
dhclient
|
||||
postgresql
|
||||
postgresql-server
|
||||
postgresql-contrib
|
||||
python-psycopg2
|
||||
setup
|
||||
targetcli
|
||||
sudo
|
||||
pxe-network-installer
|
||||
strace
|
||||
resource-agents
|
||||
lldpd
|
||||
wget
|
||||
bind-utils
|
||||
selinux-policy
|
||||
pm-utils
|
||||
centos-release
|
||||
tcpdump
|
||||
config-gate
|
||||
config-gate-compute
|
||||
sysstat
|
||||
smartmontools
|
||||
collector
|
||||
io-scheduler
|
||||
fm-doc
|
||||
vm-topology
|
||||
python2-wsme
|
||||
ceph
|
||||
ceph-common
|
||||
ceph-fuse
|
||||
ceph-radosgw
|
||||
libcephfs1
|
||||
python-ceph-compat
|
||||
python-cephfs
|
||||
socat
|
||||
irqbalance
|
||||
kmod-e1000e
|
||||
kmod-i40e
|
||||
kmod-ixgbe
|
||||
kmod-ixgbevf
|
||||
kmod-tpm
|
||||
kmod-integrity
|
||||
vim-enhanced
|
||||
wrs-heat-templates
|
||||
# heat-contrib-nova-flavor is now included with openstack-heat
|
||||
tis-extensions
|
||||
tis-extensions-controller
|
||||
qat17
|
||||
mlx4-config
|
||||
host-guest-comm
|
||||
guest-scale-helper
|
||||
python-networking-odl
|
||||
qemu-kvm-ev
|
||||
qemu-img-ev
|
||||
# for realtime kernel
|
||||
kernel-rt
|
||||
kernel-rt-kvm
|
||||
kmod-e1000e-rt
|
||||
kmod-i40e-rt
|
||||
kmod-ixgbe-rt
|
||||
kmod-tpm-rt
|
||||
kmod-integrity-rt
|
||||
rtctl
|
||||
rt-setup
|
||||
qat17-rt
|
||||
kernel-rt-tools
|
||||
# For low-latency compute
|
||||
kmod-drbd-rt
|
||||
snmp-audittrail
|
||||
OVMF
|
||||
# neutron bgp
|
||||
python2-networking-bgpvpn
|
||||
python-networking-bgpvpn-dashboard
|
||||
python-networking-bgpvpn-heat
|
||||
python2-neutron-dynamic-routing
|
||||
python2-ryu
|
||||
python-ryu-common
|
||||
openvswitch
|
||||
python2-networking-sfc
|
||||
openstack-panko-api
|
||||
openstack-panko-common
|
||||
python-panko
|
||||
python2-pankoclient
|
||||
# distributed cloud
|
||||
distributedcloud-dcmanager
|
||||
distributedcloud-client-dcmanagerclient
|
||||
distributedcloud-dcorch
|
||||
# ima plugin for RPM
|
||||
rpm-plugin-systemd-inhibit
|
||||
platform-kickstarts
|
||||
python-lefthandclient
|
||||
python-3parclient
|
||||
|
||||
# Add debugging tools
|
||||
perf
|
||||
zip
|
||||
unzip
|
||||
traceroute
|
||||
125
build-tools/build_iso/isolinux.cfg
Normal file
125
build-tools/build_iso/isolinux.cfg
Normal file
@ -0,0 +1,125 @@
|
||||
default vesamenu.c32
|
||||
timeout 600
|
||||
|
||||
display boot.msg
|
||||
|
||||
# Clear the screen when exiting the menu, instead of leaving the menu displayed.
|
||||
# For vesamenu, this means the graphical background is still displayed without
|
||||
# the menu itself for as long as the screen remains in graphics mode.
|
||||
menu clear
|
||||
menu background splash.png
|
||||
menu title CentOS 7
|
||||
menu vshift 8
|
||||
menu rows 18
|
||||
menu margin 8
|
||||
#menu hidden
|
||||
menu helpmsgrow 15
|
||||
menu tabmsgrow 13
|
||||
|
||||
# Border Area
|
||||
menu color border * #00000000 #00000000 none
|
||||
|
||||
# Selected item
|
||||
menu color sel 0 #ffffffff #00000000 none
|
||||
|
||||
# Title bar
|
||||
menu color title 0 #ff7ba3d0 #00000000 none
|
||||
|
||||
# Press [Tab] message
|
||||
menu color tabmsg 0 #ff3a6496 #00000000 none
|
||||
|
||||
# Unselected menu item
|
||||
menu color unsel 0 #84b8ffff #00000000 none
|
||||
|
||||
# Selected hotkey
|
||||
menu color hotsel 0 #84b8ffff #00000000 none
|
||||
|
||||
# Unselected hotkey
|
||||
menu color hotkey 0 #ffffffff #00000000 none
|
||||
|
||||
# Help text
|
||||
menu color help 0 #ffffffff #00000000 none
|
||||
|
||||
# A scrollbar of some type? Not sure.
|
||||
menu color scrollbar 0 #ffffffff #ff355594 none
|
||||
|
||||
# Timeout msg
|
||||
menu color timeout 0 #ffffffff #00000000 none
|
||||
menu color timeout_msg 0 #ffffffff #00000000 none
|
||||
|
||||
# Command prompt text
|
||||
menu color cmdmark 0 #84b8ffff #00000000 none
|
||||
menu color cmdline 0 #ffffffff #00000000 none
|
||||
|
||||
# Do not display the actual menu unless the user presses a key. All that is displayed is a timeout message.
|
||||
|
||||
menu tabmsg Press Tab for full configuration options on menu items.
|
||||
|
||||
menu separator # insert an empty line
|
||||
menu separator # insert an empty line
|
||||
|
||||
label tis
|
||||
menu label ^Install Titanium Cloud
|
||||
menu default
|
||||
kernel vmlinuz
|
||||
append initrd=initrd.img inst.ks=cdrom:/dev/cdrom:/ks/ks.cfg
|
||||
|
||||
label linux
|
||||
menu label ^Install CentOS 7
|
||||
kernel vmlinuz
|
||||
append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 quiet
|
||||
|
||||
label check
|
||||
menu label Test this ^media & install CentOS 7
|
||||
kernel vmlinuz
|
||||
append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rd.live.check quiet
|
||||
|
||||
menu separator # insert an empty line
|
||||
|
||||
# utilities submenu
|
||||
menu begin ^Troubleshooting
|
||||
menu title Troubleshooting
|
||||
|
||||
label vesa
|
||||
menu indent count 5
|
||||
menu label Install CentOS 7 in ^basic graphics mode
|
||||
text help
|
||||
Try this option out if you're having trouble installing
|
||||
CentOS 7.
|
||||
endtext
|
||||
kernel vmlinuz
|
||||
append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 xdriver=vesa nomodeset quiet
|
||||
|
||||
label rescue
|
||||
menu indent count 5
|
||||
menu label ^Rescue a CentOS system
|
||||
text help
|
||||
If the system will not boot, this lets you access files
|
||||
and edit config files to try to get it booting again.
|
||||
endtext
|
||||
kernel vmlinuz
|
||||
append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rescue quiet
|
||||
|
||||
label memtest
|
||||
menu label Run a ^memory test
|
||||
text help
|
||||
If your system is having issues, a problem with your
|
||||
system's memory may be the cause. Use this utility to
|
||||
see if the memory is working correctly.
|
||||
endtext
|
||||
kernel memtest
|
||||
|
||||
menu separator # insert an empty line
|
||||
|
||||
label local
|
||||
menu label Boot from ^local drive
|
||||
localboot 0xffff
|
||||
|
||||
menu separator # insert an empty line
|
||||
menu separator # insert an empty line
|
||||
|
||||
label returntomain
|
||||
menu label Return to ^main menu
|
||||
menu exit
|
||||
|
||||
menu end
|
||||
36
build-tools/build_iso/ks.cfg
Normal file
36
build-tools/build_iso/ks.cfg
Normal file
@ -0,0 +1,36 @@
|
||||
install
|
||||
text
|
||||
lang en_US.UTF-8
|
||||
keyboard us
|
||||
reboot --eject
|
||||
firstboot --enable
|
||||
auth --enableshadow --passalgo=sha512
|
||||
|
||||
# Network information
|
||||
network --bootproto=dhcp --device=enp0s3 --onboot=on --ipv6=auto --activate
|
||||
network --bootproto=static --device=enp0s8 --ip=10.10.10.12 --netmask=255.255.255.0 --ipv6=auto --activate
|
||||
network --device=lo --hostname=localhost.localdomain
|
||||
|
||||
rootpw --lock
|
||||
timezone America/New_York --isUtc
|
||||
user --groups=wheel --name=wrsroot --password=$6$c3gaCcJlh.rp//Yx$/mIjNNoUDS1qZldBL29YSJdsA9ttPA/nXN1CPsIcCmionXC22APT3IoRSd9j5dPiZoviDdQf7YxLsOYdieOQr/ --iscrypted --gecos="wrsroot"
|
||||
|
||||
# System bootloader configuration
|
||||
#bootloader --location=mbr --boot-drive=sda
|
||||
|
||||
autopart --type=lvm
|
||||
# Partition clearing information
|
||||
clearpart --all --initlabel --drives=sda
|
||||
|
||||
cdrom
|
||||
#repo --name=base --baseurl=http://mirror.cogentco.com/pub/linux/centos/7/os/x86_64/
|
||||
#url --url="http://mirror.cogentco.com/pub/linux/centos/7/os/x86_64/"
|
||||
|
||||
%packages --nobase --ignoremissing
|
||||
@^minimal
|
||||
@core
|
||||
kexec-tools
|
||||
net-tools
|
||||
# CGTS packages
|
||||
# end CGTS packages
|
||||
%end
|
||||
258
build-tools/build_iso/minimal_rpm_list.txt
Normal file
258
build-tools/build_iso/minimal_rpm_list.txt
Normal file
@ -0,0 +1,258 @@
|
||||
acl
|
||||
alsa-lib
|
||||
audit
|
||||
audit-libs
|
||||
authconfig
|
||||
basesystem
|
||||
bind-libs-lite
|
||||
bind-license
|
||||
binutils
|
||||
biosdevname
|
||||
btrfs-progs
|
||||
bzip2-libs
|
||||
ca-certificates
|
||||
centos-logos
|
||||
chkconfig
|
||||
coreutils
|
||||
cpio
|
||||
cracklib
|
||||
cracklib-dicts
|
||||
cronie
|
||||
cronie-anacron
|
||||
crontabs
|
||||
cryptsetup
|
||||
cryptsetup-libs
|
||||
curl
|
||||
cyrus-sasl-lib
|
||||
dbus
|
||||
dbus-glib
|
||||
dbus-libs
|
||||
dbus-python
|
||||
device-mapper
|
||||
device-mapper-event
|
||||
device-mapper-event-libs
|
||||
device-mapper-libs
|
||||
device-mapper-multipath
|
||||
device-mapper-multipath-libs
|
||||
device-mapper-persistent-data
|
||||
diffutils
|
||||
dmidecode
|
||||
dosfstools
|
||||
dracut
|
||||
dracut-config-rescue
|
||||
dracut-network
|
||||
e2fsprogs
|
||||
e2fsprogs-libs
|
||||
ebtables
|
||||
efibootmgr
|
||||
efivar-libs
|
||||
elfutils-libelf
|
||||
elfutils-libs
|
||||
ethtool
|
||||
expat
|
||||
file
|
||||
file-libs
|
||||
filesystem
|
||||
findutils
|
||||
fipscheck
|
||||
fipscheck-lib
|
||||
firewalld
|
||||
freetype
|
||||
gawk
|
||||
gdbm
|
||||
gettext
|
||||
gettext-libs
|
||||
glib2
|
||||
glibc
|
||||
glibc-common
|
||||
glib-networking
|
||||
gmp
|
||||
gnupg2
|
||||
gnutls
|
||||
gobject-introspection
|
||||
gpgme
|
||||
grep
|
||||
groff-base
|
||||
grub2
|
||||
grub2-efi
|
||||
grub2-tools
|
||||
grubby
|
||||
gsettings-desktop-schemas
|
||||
gzip
|
||||
hardlink
|
||||
hostname
|
||||
hwdata
|
||||
info
|
||||
iproute
|
||||
iprutils
|
||||
iputils
|
||||
jansson
|
||||
json-c
|
||||
kbd
|
||||
kbd-legacy
|
||||
kbd-misc
|
||||
kernel-tools
|
||||
kernel-tools-libs
|
||||
kexec-tools
|
||||
keyutils-libs
|
||||
kmod
|
||||
kmod-libs
|
||||
kpartx
|
||||
krb5-libs
|
||||
less
|
||||
libacl
|
||||
libaio
|
||||
libassuan
|
||||
libattr
|
||||
libblkid
|
||||
libcap
|
||||
libcap-ng
|
||||
libcom_err
|
||||
libconfig
|
||||
libcroco
|
||||
libcurl
|
||||
libdaemon
|
||||
libdb
|
||||
libdb-utils
|
||||
libdrm
|
||||
libedit
|
||||
libestr
|
||||
libffi
|
||||
libgcc
|
||||
libgcrypt
|
||||
libgomp
|
||||
libgpg-error
|
||||
libgudev1
|
||||
libidn
|
||||
libmnl
|
||||
libmodman
|
||||
libmount
|
||||
libndp
|
||||
libnetfilter_conntrack
|
||||
libnfnetlink
|
||||
libnl
|
||||
libnl3
|
||||
libnl3-cli
|
||||
libpcap
|
||||
libpciaccess
|
||||
libpipeline
|
||||
libproxy
|
||||
libpwquality
|
||||
libreport-filesystem
|
||||
libselinux
|
||||
libselinux-python
|
||||
libselinux-utils
|
||||
libsemanage
|
||||
libsepol
|
||||
libss
|
||||
libssh2
|
||||
libstdc++
|
||||
libsysfs
|
||||
libtasn1
|
||||
libteam
|
||||
libunistring
|
||||
libuser
|
||||
libutempter
|
||||
libuuid
|
||||
libverto
|
||||
libxml2
|
||||
libxslt
|
||||
linux-firmware
|
||||
lldpad
|
||||
lsscsi
|
||||
lua
|
||||
lvm2
|
||||
lvm2-libs
|
||||
lzo
|
||||
make
|
||||
man-db
|
||||
mariadb-libs
|
||||
mdadm
|
||||
microcode_ctl
|
||||
mokutil
|
||||
mozjs17
|
||||
ncurses
|
||||
ncurses-base
|
||||
ncurses-libs
|
||||
nettle
|
||||
newt
|
||||
newt-python
|
||||
nspr
|
||||
nss
|
||||
nss-softokn
|
||||
nss-softokn-freebl
|
||||
nss-sysinit
|
||||
nss-tools
|
||||
nss-util
|
||||
numactl-libs
|
||||
openscap
|
||||
openscap-scanner
|
||||
openssl
|
||||
openssl-libs
|
||||
os-prober
|
||||
p11-kit
|
||||
p11-kit-trust
|
||||
passwd
|
||||
pciutils-libs
|
||||
pcre
|
||||
pinentry
|
||||
pkgconfig
|
||||
policycoreutils
|
||||
popt
|
||||
procps-ng
|
||||
pth
|
||||
pygobject3-base
|
||||
pygpgme
|
||||
pyliblzma
|
||||
python
|
||||
python-backports
|
||||
python-backports-ssl_match_hostname
|
||||
python-configobj
|
||||
python-decorator
|
||||
python-iniparse
|
||||
python-libs
|
||||
python-perf
|
||||
python-pycurl
|
||||
python-pyudev
|
||||
python2-setuptools
|
||||
python-slip
|
||||
python-slip-dbus
|
||||
python-urlgrabber
|
||||
pyxattr
|
||||
qrencode-libs
|
||||
rdma
|
||||
readline
|
||||
rootfiles
|
||||
rpm
|
||||
rpm-build-libs
|
||||
rpm-libs
|
||||
rpm-python
|
||||
sed
|
||||
shared-mime-info
|
||||
shim
|
||||
shim-unsigned
|
||||
slang
|
||||
snappy
|
||||
sqlite
|
||||
systemd
|
||||
systemd-libs
|
||||
systemd-sysv
|
||||
sysvinit-tools
|
||||
tar
|
||||
tcp_wrappers-libs
|
||||
teamd
|
||||
time
|
||||
trousers
|
||||
tzdata
|
||||
ustr
|
||||
util-linux
|
||||
virt-what
|
||||
which
|
||||
xfsprogs
|
||||
xml-common
|
||||
xz
|
||||
xz-libs
|
||||
zlib
|
||||
lksctp-tools
|
||||
boost-thread
|
||||
boost-system
|
||||
2
build-tools/build_iso/openstack_kilo.txt
Normal file
2
build-tools/build_iso/openstack_kilo.txt
Normal file
@ -0,0 +1,2 @@
|
||||
# Files copied in from /import/mirrors/CentOS/7.2.1511/cloud/x86_64/openstack-kilo
|
||||
|
||||
112
build-tools/build_minimal_iso/README
Normal file
112
build-tools/build_minimal_iso/README
Normal file
@ -0,0 +1,112 @@
|
||||
This document describes how to generate a DVD image (.iso) which installs
|
||||
a minimal CentOS installation where the entirety of the installed system is
|
||||
build from the provided source.
|
||||
|
||||
There are three parts to this document:
|
||||
How to build binary RPMs from source RPMS
|
||||
How to build the install disk from the binary RPMS
|
||||
How to install the minimal system
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
How to build the binary RPMs from the source RPMS
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
(note - building the binary RPMs is expected to take a long time, ~ 15 hours
|
||||
on a typical system)
|
||||
|
||||
The source RPMs in the "srcs" subdirectory are compiled in an environment
|
||||
called "mock" which builds each package in a chroot jail to ensure the output
|
||||
is not influenced by the build system. Mock is controlled by a config file.
|
||||
The example srcs/build.cfg is provided as a starting point, however it does
|
||||
to be adjusted for your build environment. In particular, the paths and repo
|
||||
locations need to be configured for your system. It is highly recommended that
|
||||
a local mirror of the CentOS repos be used for speed. The example config file
|
||||
is configured to use an localhost http mirror of the CentOS repos.
|
||||
|
||||
To build the binary RPMs from the source RPMs change to the "srcs" subdirectory
|
||||
and execute the "build.sh" script.
|
||||
|
||||
# cd srcs
|
||||
# ./build.sh
|
||||
|
||||
This will use build.cfg and mock to compile every source RPM listed in list.txt.
|
||||
The output binary RPMs will be in srcs/results. There will also be success.txt
|
||||
and fail.txt files which list any RPMs that failed to build. Debugging why RPMs
|
||||
fail to build is beyond the scope of this document, however be aware that RPMs
|
||||
often fail in the "check" phase of the build (i.e. the package compiled fine
|
||||
but tests failed). Notably, the python package may fail due to a "test_nis"
|
||||
failure, and the "attr" and "e2fsprogs" packages may or may not fail depending
|
||||
on the host file system used for compilation. These failures may or may not be
|
||||
false positives (for example, the mock environment does not have NIS configured
|
||||
which is why python's test_nis reports a failure -- the code is actually fine,
|
||||
we just can't run the test in the mock environment).
|
||||
|
||||
To disable the check phase, add the line
|
||||
|
||||
config_opts['rpmbuild_opts'] = '--nocheck'
|
||||
|
||||
to build.cfg. You can then run build.sh again with list.txt containing
|
||||
packages which failed:
|
||||
|
||||
# cp list.txt list.txt.orig
|
||||
# cp fail.txt list.txt
|
||||
# ./build.sh
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
How to build the install disk from the binary RPMS
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
Once srcs/results is populated with binary RPMs, an installation disk can be
|
||||
built. Edit the yum.conf file and place an (arbitrary) path for yum log and
|
||||
cache locations, and make sure that the repository path points to srcs/results.
|
||||
Run the build_centos.sh script to build the installation DVD:
|
||||
|
||||
# ./build_centos.sh
|
||||
|
||||
Scroll up the output to the top of the "Spawning worker" messages. You should
|
||||
observe a line indicating that there are no remaining unresolved dependencies:
|
||||
|
||||
...
|
||||
Installing PKG=dhcp-common PKG_FILE=dhcp-common-4.2.5-42.el7.centos.tis.1.x86_64.rpm PKG_REL_PATH=dhcp-common-4.2.5-42.el7.centos.tis.1.x86_64.rpm PKG_PATH=/localdisk/loadbuild/jmckenna/centos/srcs/results/dhcp-common-4.2.5-42.el7.centos.tis.1.x86_64.rpm from repo local-std
|
||||
dhcp-common
|
||||
Debug: Packages still unresolved:
|
||||
|
||||
Spawning worker 0 with 4 pkgs
|
||||
Spawning worker 1 with 4 pkgs
|
||||
Spawning worker 2 with 4 pkgs
|
||||
...
|
||||
|
||||
This is your confirmation that all required pacakges were found and installed
|
||||
on the ISO. You should also now see a new file called "centosIso.iso":
|
||||
|
||||
# ls -l centosIso.iso
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
How to install the minimal system
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
The centosIso.iso file can be burned to a DVD or booted in a virtual
|
||||
environment. It is configured to self-install on boot. After installation,
|
||||
a user with sudo access must be created manually. The system can then be
|
||||
booted.
|
||||
|
||||
Power the system on with the DVD inserted. A system install will take place
|
||||
(takes approximately 2 minutes). The system will then report an error and
|
||||
ask you if you wish to report a bug, debug, or quit. Hit control-alt-F2 to
|
||||
switch to a terminal window. Enter the following commands to change to the
|
||||
installed system root, and create a (wrsroot) with sudo access:
|
||||
|
||||
cd /mnt/sysimage
|
||||
chroot .
|
||||
groupadd -r wrs
|
||||
groupadd -f -g 345 wrs_protected
|
||||
useradd -m -g wrs -G root,wrs_protected,wheel -d /home/wrsroot -p cBglipPpsKwBQ -s /bin/sh wrsroot
|
||||
exit
|
||||
|
||||
Change back to the main window with control-alt-F1.
|
||||
Hit 3 <enter> (the "Quit" option). The system will reboot (make sure you eject
|
||||
the DVD or use your BIOS to boot from hard disk rather than DVD; the installer
|
||||
will re-run if the DVD boots again).
|
||||
|
||||
You can log into the system as user "wrsroot" with password "wrsroot".
|
||||
|
||||
5
build-tools/build_minimal_iso/README.2
Normal file
5
build-tools/build_minimal_iso/README.2
Normal file
@ -0,0 +1,5 @@
|
||||
The files in this directory are to be used as described at
|
||||
http://twiki.wrs.com/PBUeng/DeliveryExtras#Minimal_CentOS_install
|
||||
|
||||
They include the scripts (and customer README) for building a minimual
|
||||
CentOS ISO from our modified sources.
|
||||
108
build-tools/build_minimal_iso/build.cfg
Normal file
108
build-tools/build_minimal_iso/build.cfg
Normal file
@ -0,0 +1,108 @@
|
||||
config_opts['root'] = 'jmckenna-centos/mock'
|
||||
config_opts['target_arch'] = 'x86_64'
|
||||
config_opts['legal_host_arches'] = ('x86_64',)
|
||||
config_opts['chroot_setup_cmd'] = 'install @buildsys-build'
|
||||
config_opts['dist'] = 'el7' # only useful for --resultdir variable subst
|
||||
config_opts['releasever'] = '7'
|
||||
|
||||
config_opts['yum.conf'] = """
|
||||
[main]
|
||||
keepcache=1
|
||||
debuglevel=2
|
||||
reposdir=/dev/null
|
||||
logfile=/var/log/yum.log
|
||||
retries=20
|
||||
obsoletes=1
|
||||
gpgcheck=0
|
||||
assumeyes=1
|
||||
syslog_ident=mock
|
||||
syslog_device=
|
||||
|
||||
# repos
|
||||
[my-build]
|
||||
name=my-build
|
||||
baseurl=http://127.0.0.1:8088/localdisk/loadbuild/centos/src/results
|
||||
enabled=1
|
||||
skip_if_unavailable=1
|
||||
metadata_expire=0
|
||||
|
||||
[base]
|
||||
name=CentOS-$releasever - Base
|
||||
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
|
||||
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
|
||||
baseurl=http://127.0.0.1:8088/CentOS/7.2.1511/os/$basearch/
|
||||
gpgcheck=1
|
||||
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
|
||||
|
||||
#released updates
|
||||
[updates]
|
||||
name=CentOS-$releasever - Updates
|
||||
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra
|
||||
#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
|
||||
baseurl=http://127.0.0.1:8088/CentOS/7.2.1511/updates/$basearch/
|
||||
gpgcheck=1
|
||||
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
|
||||
|
||||
#additional packages that may be useful
|
||||
[extras]
|
||||
name=CentOS-$releasever - Extras
|
||||
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra
|
||||
#baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
|
||||
baseurl=http://127.0.0.1:8088/CentOS/7.2.1511/extras/$basearch/
|
||||
gpgcheck=1
|
||||
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
|
||||
|
||||
#additional packages that extend functionality of existing packages
|
||||
[centosplus]
|
||||
name=CentOS-$releasever - Plus
|
||||
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus&infra=$infra
|
||||
#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/
|
||||
baseurl=http://127.0.0.1:8088/CentOS/7.2.1511/centosplus/$basearch/
|
||||
gpgcheck=1
|
||||
enabled=1
|
||||
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
|
||||
|
||||
[epel]
|
||||
name=Extra Packages for Enterprise Linux 7 - $basearch
|
||||
baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch
|
||||
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch
|
||||
failovermethod=priority
|
||||
enabled=1
|
||||
gpgcheck=1
|
||||
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
|
||||
|
||||
[epel-debuginfo]
|
||||
name=Extra Packages for Enterprise Linux 7 - $basearch - Debug
|
||||
baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch/debug
|
||||
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-7&arch=$basearch
|
||||
failovermethod=priority
|
||||
enabled=0
|
||||
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
|
||||
gpgcheck=1
|
||||
|
||||
[epel-source]
|
||||
name=Extra Packages for Enterprise Linux 7 - $basearch - Source
|
||||
baseurl=http://download.fedoraproject.org/pub/epel/7/SRPMS
|
||||
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-source-7&arch=$basearch
|
||||
failovermethod=priority
|
||||
enabled=1
|
||||
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
|
||||
gpgcheck=1
|
||||
|
||||
|
||||
"""
|
||||
config_opts['environment']['BUILD_BY'] = 'jmckenna'
|
||||
config_opts['environment']['BUILD_DATE'] = '2016-10-31 14:27:28 -0400'
|
||||
config_opts['environment']['REPO'] = '/localdisk/designer/jmckenna/dev0019/cgcs-root'
|
||||
config_opts['environment']['WRS_GIT_BRANCH'] = 'CGCS_DEV_0019'
|
||||
config_opts['environment']['CGCS_GIT_BRANCH'] = 'CGCS_DEV_0019'
|
||||
config_opts['macros']['%_no_cgcs_license_check'] = '1'
|
||||
config_opts['macros']['%_tis_build_type'] = 'std'
|
||||
config_opts['chroot_setup_cmd'] = 'install @buildsys-build pigz lbzip2 yum shadow-utils rpm-build lbzip2 gcc glibc-headers make gcc-c++ java-devel'
|
||||
config_opts['macros']['%__gzip'] = '/usr/bin/pigz'
|
||||
config_opts['macros']['%__bzip2'] = '/usr/bin/lbzip2'
|
||||
config_opts['macros']['%_patch_confdir'] = '%{_sysconfdir}/patching'
|
||||
config_opts['macros']['%_patch_scripts'] = '%{_patch_confdir}/patch-scripts'
|
||||
config_opts['macros']['%_runtime_patch_scripts'] = '/run/patching/patch-scripts'
|
||||
config_opts['macros']['%_tis_dist'] = '.tis'
|
||||
#config_opts['rpmbuild_opts'] = '--nocheck'
|
||||
45
build-tools/build_minimal_iso/build.sh
Executable file
45
build-tools/build_minimal_iso/build.sh
Executable file
@ -0,0 +1,45 @@
|
||||
#!/bin/sh
|
||||
|
||||
CREATEREPO=$(which createrepo_c)
|
||||
if [ $? -ne 0 ]; then
|
||||
CREATEREPO="createrepo"
|
||||
fi
|
||||
|
||||
# If a file listed in list.txt is missing, this function attempts to find the
|
||||
# RPM and copy it to the local directory. This should not be required normally
|
||||
# and is only used when collecting the source RPMs initially.
|
||||
function findSrc {
|
||||
local lookingFor=$1
|
||||
find $MY_REPO/cgcs-centos-repo/Source -name $lookingFor | xargs -I '{}' cp '{}' .
|
||||
find $MY_REPO/cgcs-tis-repo/Source -name $lookingFor | xargs -I '{}' cp '{}' .
|
||||
find $MY_WORKSPACE/std/rpmbuild/SRPMS -name $lookingFor | xargs -I '{}' cp '{}' .
|
||||
}
|
||||
|
||||
rm -f success.txt
|
||||
rm -f fail.txt
|
||||
rm -f missing.txt
|
||||
mkdir -p results
|
||||
infile=list.txt
|
||||
|
||||
while read p; do
|
||||
|
||||
if [ ! -f "$p" ]; then
|
||||
findSrc $p
|
||||
if [ ! -f "$p" ]; then
|
||||
echo "couldn't find" >> missing.txt
|
||||
echo "couldn't find $p" >> missing.txt
|
||||
continue
|
||||
fi
|
||||
echo "found $p"
|
||||
fi
|
||||
|
||||
mock -r build.cfg $p --resultdir=results --no-clean
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "$p" >> success.txt
|
||||
cd results
|
||||
$CREATEREPO .
|
||||
cd ..
|
||||
else
|
||||
echo "$p" >> fail.txt
|
||||
fi
|
||||
done < $infile
|
||||
62
build-tools/build_minimal_iso/build_centos.sh
Executable file
62
build-tools/build_minimal_iso/build_centos.sh
Executable file
@ -0,0 +1,62 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Build a basic CentOS system
|
||||
|
||||
CREATEREPO=$(which createrepo_c)
|
||||
if [ $? -ne 0 ]; then
|
||||
CREATEREPO="createrepo"
|
||||
fi
|
||||
|
||||
function final_touches {
|
||||
# create the repo
|
||||
cd ${ROOTDIR}/${DEST}/isolinux
|
||||
$CREATEREPO -g ../comps.xml .
|
||||
|
||||
# build the ISO
|
||||
printf "Building image $OUTPUT_FILE\n"
|
||||
cd ${ROOTDIR}/${DEST}
|
||||
chmod 664 isolinux/isolinux.bin
|
||||
mkisofs -o $OUTPUT_FILE \
|
||||
-R -D -A 'oe_iso_boot' -V 'oe_iso_boot' \
|
||||
-b isolinux.bin -c boot.cat -no-emul-boot \
|
||||
-boot-load-size 4 -boot-info-table \
|
||||
-eltorito-alt-boot \
|
||||
-e images/efiboot.img \
|
||||
-no-emul-boot \
|
||||
isolinux/
|
||||
|
||||
isohybrid --uefi $OUTPUT_FILE
|
||||
implantisomd5 $OUTPUT_FILE
|
||||
|
||||
cd $ROOTDIR
|
||||
}
|
||||
|
||||
function setup_disk {
|
||||
tar xJf emptyInstaller.tar.xz
|
||||
mkdir ${DEST}/isolinux/Packages
|
||||
}
|
||||
|
||||
function install_packages {
|
||||
cd ${DEST}/isolinux/Packages
|
||||
ROOT=${ROOTDIR} ../../../cgts_deps.sh --deps=../../../${MINIMAL}
|
||||
cd ${ROOTDIR}
|
||||
}
|
||||
|
||||
|
||||
ROOTDIR=$PWD
|
||||
INSTALLER_SRC=basicDisk
|
||||
DEST=newDisk
|
||||
PKGS_DIR=all_rpms
|
||||
MINIMAL=minimal_rpm_list.txt
|
||||
OUTPUT_FILE=${ROOTDIR}/centosIso.iso
|
||||
|
||||
# Make a basic install disk (no packages, at this point)
|
||||
rm -rf ${DEST}
|
||||
setup_disk
|
||||
|
||||
# install the packages (initially from minimal list, then resolve deps)
|
||||
install_packages
|
||||
|
||||
# build the .iso
|
||||
final_touches
|
||||
|
||||
222
build-tools/build_minimal_iso/cgts_deps.sh
Executable file
222
build-tools/build_minimal_iso/cgts_deps.sh
Executable file
@ -0,0 +1,222 @@
|
||||
#!/bin/env bash
|
||||
function generate_dep_list {
|
||||
TMP_RPM_DB=$(mktemp -d $(pwd)/tmp_rpm_db_XXXXXX)
|
||||
mkdir -p $TMP_RPM_DB
|
||||
rpm --initdb --dbpath $TMP_RPM_DB
|
||||
rpm --dbpath $TMP_RPM_DB --test -Uvh --replacefiles '*.rpm' >> $DEPDETAILLISTFILE 2>&1
|
||||
rpm --dbpath $TMP_RPM_DB --test -Uvh --replacefiles '*.rpm' 2>&1 \
|
||||
| grep -v "error:" \
|
||||
| grep -v "warning:" \
|
||||
| grep -v "Preparing..." \
|
||||
| sed "s/ is needed by.*$//" | sed "s/ >=.*$//" | sort -u > $DEPLISTFILE
|
||||
rm -rf $TMP_RPM_DB
|
||||
}
|
||||
|
||||
function install_deps {
|
||||
local DEP_LIST=""
|
||||
local DEP_LIST_FILE="$1"
|
||||
|
||||
rm -f $TMPFILE
|
||||
|
||||
while read DEP
|
||||
do
|
||||
DEP_LIST="${DEP_LIST} ${DEP}"
|
||||
done < $DEP_LIST_FILE
|
||||
|
||||
echo "Debug: List of deps to resolve: ${DEP_LIST}"
|
||||
|
||||
if [ -z "${DEP_LIST}" ]
|
||||
then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# go through each repo and convert deps to packages
|
||||
|
||||
for REPOID in `grep '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
|
||||
echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --whatprovides ${DEP_LIST} --qf='%{name}'"
|
||||
TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --qf='%{name}' --whatprovides ${DEP_LIST} | sed "s/kernel-debug/kernel/g" >> $TMPFILE
|
||||
\rm -rf $TMP_DIR/yum-$USER-*
|
||||
done
|
||||
sort $TMPFILE -u > $TMPFILE1
|
||||
rm $TMPFILE
|
||||
|
||||
DEP_LIST=""
|
||||
while read DEP
|
||||
do
|
||||
DEP_LIST="${DEP_LIST} ${DEP}"
|
||||
done < $TMPFILE1
|
||||
rm $TMPFILE1
|
||||
|
||||
# next go through each repo and install packages
|
||||
local TARGETS=${DEP_LIST}
|
||||
echo "Debug: Resolved list of deps to install: ${TARGETS}"
|
||||
local UNRESOLVED
|
||||
for REPOID in `grep '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
|
||||
UNRESOLVED=" $TARGETS "
|
||||
|
||||
if [[ ! -z "${TARGETS// }" ]]; then
|
||||
REPO_PATH=$(cat $YUM | sed -n "/^\[$REPOID\]\$/,\$p" | grep '^baseurl=' | head -n 1 | awk -F 'file://' '{print $2}' | sed 's:/$::')
|
||||
>&2 echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf='%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}'"
|
||||
TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf="%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}" | sort -r -V >> $TMPFILE
|
||||
\rm -rf $TMP_DIR/yum-$USER-*
|
||||
|
||||
while read STR
|
||||
do
|
||||
>&2 echo "STR=$STR"
|
||||
if [ "x$STR" == "x" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
PKG=`echo $STR | cut -d " " -f 1`
|
||||
PKG_FILE=`echo $STR | cut -d " " -f 2`
|
||||
PKG_REL_PATH=`echo $STR | cut -d " " -f 3`
|
||||
PKG_PATH="${REPO_PATH}/${PKG_REL_PATH}"
|
||||
|
||||
>&2 echo "Installing PKG=$PKG PKG_FILE=$PKG_FILE PKG_REL_PATH=$PKG_REL_PATH PKG_PATH=$PKG_PATH from repo $REPOID"
|
||||
cp $PKG_PATH .
|
||||
if [ $? -ne 0 ]
|
||||
then
|
||||
>&2 echo " Here's what I have to work with..."
|
||||
>&2 echo " TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $PKG --qf=\"%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}\""
|
||||
>&2 echo " PKG=$PKG PKG_FILE=$PKG_FILE REPO_PATH=$REPO_PATH PKG_REL_PATH=$PKG_REL_PATH PKG_PATH=$PKG_PATH"
|
||||
fi
|
||||
|
||||
echo $UNRESOLVED | grep $PKG
|
||||
echo $UNRESOLVED | grep $PKG >> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "$PKG found in $REPOID as $PKG" >> $BUILT_REPORT
|
||||
echo "$PKG_PATH" >> $BUILT_REPORT
|
||||
UNRESOLVED=$(echo "$UNRESOLVED" | sed "s# $PKG # #g")
|
||||
else
|
||||
echo "$PKG satisfies unknown target in $REPOID" >> $BUILT_REPORT
|
||||
echo " but it doesn't match targets, $UNRESOLVED" >> $BUILT_REPORT
|
||||
echo " path $PKG_PATH" >> $BUILT_REPORT
|
||||
FOUND_UNKNOWN=1
|
||||
fi
|
||||
done < $TMPFILE #<<< "$(TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf=\"%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}\" | sort -r -V)"
|
||||
\rm -rf $TMP_DIR/yum-$USER-*
|
||||
TARGETS="$UNRESOLVED"
|
||||
fi
|
||||
done
|
||||
>&2 echo "Debug: Packages still unresolved: $UNRESOLVED"
|
||||
echo "Debug: Packages still unresolved: $UNRESOLVED" >> $WARNINGS_REPORT
|
||||
echo "Debug: Packages still unresolved: $UNRESOLVED" >> $BUILT_REPORT
|
||||
>&2 echo ""
|
||||
}
|
||||
|
||||
function check_all_explicit_deps_installed
|
||||
{
|
||||
|
||||
PKGS_TO_CHECK=" "
|
||||
while read PKG_TO_ADD
|
||||
do
|
||||
PKGS_TO_CHECK="$PKGS_TO_CHECK ${PKG_TO_ADD}"
|
||||
done < $DEPLISTFILE
|
||||
rpm -qp ${INSTALLDIR}/*.rpm --qf="%{name}\n" > $TMPFILE
|
||||
|
||||
echo "checking... $PKGS_TO_CHECK vs ${INSTALLED_PACKAGE}"
|
||||
|
||||
while read INSTALLED_PACKAGE
|
||||
do
|
||||
echo $PKGS_TO_CHECK | grep -q "${INSTALLED_PACKAGE}"
|
||||
if [ $? -eq 0 ]; then
|
||||
PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/^${INSTALLED_PACKAGE} //"`
|
||||
PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/ ${INSTALLED_PACKAGE} / /"`
|
||||
PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/ ${INSTALLED_PACKAGE}\$//"`
|
||||
PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/^${INSTALLED_PACKAGE}\$//"`
|
||||
fi
|
||||
done < $TMPFILE
|
||||
|
||||
if [ -z "$PKGS_TO_CHECK" ]
|
||||
then
|
||||
>&2 echo "All explicitly specified packages resolved!"
|
||||
else
|
||||
>&2 echo "Could not resolve packages: $PKGS_TO_CHECK"
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
if [ "x${ROOT}" == "x" ]; then
|
||||
ROOT=/localdisk/loadbuild/centos
|
||||
fi
|
||||
|
||||
ATTEMPTED=0
|
||||
DISCOVERED=0
|
||||
OUTPUT_DIR=${ROOT}/newDisk
|
||||
YUM=${ROOT}/yum.conf
|
||||
TMP_DIR=${ROOT}/tmp
|
||||
DEPLISTFILE=${ROOT}/deps.txt
|
||||
DEPDETAILLISTFILE=${ROOT}/deps_detail.txt
|
||||
INSTALLDIR=${ROOT}/newDisk/isolinux/Packages
|
||||
|
||||
BUILT_REPORT=${ROOT}/local.txt
|
||||
WARNINGS_REPORT=${ROOT}/warnings.txt
|
||||
LAST_TEST=${ROOT}/last_test.txt
|
||||
TMPFILE=${ROOT}/cgts_deps_tmp.txt
|
||||
TMPFILE1=${ROOT}/cgts_deps_tmp1.txt
|
||||
|
||||
touch "$BUILT_REPORT"
|
||||
touch "$WARNINGS_REPORT"
|
||||
|
||||
for i in "$@"
|
||||
do
|
||||
case $i in
|
||||
-d=*|--deps=*)
|
||||
DEPS="${i#*=}"
|
||||
shift # past argument=value
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
mkdir -p $TMP_DIR
|
||||
|
||||
rm -f "$DEPDETAILLISTFILE"
|
||||
# FIRST PASS we are being given a list of REQUIRED dependencies
|
||||
if [ "${DEPS}x" != "x" ]; then
|
||||
cat $DEPS | grep -v "^#" > $DEPLISTFILE
|
||||
install_deps $DEPLISTFILE
|
||||
if [ $? -ne 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# check that we resolved them all
|
||||
check_all_explicit_deps_installed
|
||||
if [ $? -ne 0 ]; then
|
||||
>&2 echo "Error -- could not install all explicitly listed packages"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ALL_RESOLVED=0
|
||||
|
||||
while [ $ALL_RESOLVED -eq 0 ]; do
|
||||
cp $DEPLISTFILE $DEPLISTFILE.old
|
||||
generate_dep_list
|
||||
if [ ! -s $DEPLISTFILE ]; then
|
||||
# no more dependencies!
|
||||
ALL_RESOLVED=1
|
||||
else
|
||||
DIFFLINES=`diff $DEPLISTFILE.old $DEPLISTFILE | wc -l`
|
||||
if [ $DIFFLINES -eq 0 ]; then
|
||||
>&2 echo "Warning: Infinite loop detected in dependency resolution. See $DEPLISTFILE for details -- exiting"
|
||||
>&2 echo "These RPMS had problems (likely version conflicts)"
|
||||
>&2 cat $DEPLISTFILE
|
||||
|
||||
echo "Warning: Infinite loop detected in dependency resolution See $DEPLISTFILE for details -- exiting" >> $WARNINGS_REPORT
|
||||
echo "These RPMS had problems (likely version conflicts)" >> $WARNINGS_REPORT
|
||||
cat $DEPLISTFILE >> $WARNINGS_REPORT
|
||||
|
||||
date > $LAST_TEST
|
||||
|
||||
rm -f $DEPLISTFILE.old
|
||||
exit 1 # nothing fixed
|
||||
fi
|
||||
install_deps $DEPLISTFILE
|
||||
if [ $? -ne 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
exit 0
|
||||
22
build-tools/build_minimal_iso/yum.conf
Normal file
22
build-tools/build_minimal_iso/yum.conf
Normal file
@ -0,0 +1,22 @@
|
||||
|
||||
[main]
|
||||
cachedir=/localdisk/loadbuild/jmckenna/centos/yum/cache
|
||||
keepcache=1
|
||||
debuglevel=2
|
||||
reposdir=/dev/null
|
||||
logfile=/localdisk/loadbuild/jmckenna/centos/yum/yum.log
|
||||
retries=20
|
||||
obsoletes=1
|
||||
gpgcheck=0
|
||||
assumeyes=1
|
||||
syslog_ident=mock
|
||||
syslog_device=
|
||||
|
||||
# repos
|
||||
[local-std]
|
||||
name=local-std
|
||||
baseurl=file:///localdisk/loadbuild/jmckenna/centos/srcs/results
|
||||
enabled=1
|
||||
skip_if_unavailable=1
|
||||
metadata_expire=0
|
||||
|
||||
BIN
build-tools/certificates/TiBoot.crt
Normal file
BIN
build-tools/certificates/TiBoot.crt
Normal file
Binary file not shown.
67
build-tools/classify
Normal file
67
build-tools/classify
Normal file
@ -0,0 +1,67 @@
|
||||
classify () {
|
||||
local pkg_dir="$1"
|
||||
|
||||
if [ -f $pkg_dir/centos/srpm_path ]; then
|
||||
# echo "srpm + patch: $(basename $(cat $pkg_dir/centos/srpm_path | head -n 1))"
|
||||
echo "srpm + patches"
|
||||
elif [ -f $pkg_dir/centos/*.spec ]; then
|
||||
if [ -f $pkg_dir/centos/build_srpm ]; then
|
||||
# echo "spec + custom_script: $pkg_dir"
|
||||
echo "spec + custom_script"
|
||||
elif [ -f $pkg_dir/centos/build_srpm.data ]; then
|
||||
local ALLOW_EMPTY_RPM=""
|
||||
local COPY_LIST=""
|
||||
local SRC_DIR=""
|
||||
local PKG_BASE="$pkg_dir"
|
||||
source $pkg_dir/centos/build_srpm.data
|
||||
|
||||
if [ "" != "$SRC_DIR" ] ; then
|
||||
# echo "spec + src_dir: $pkg_dir/$SRC_DIR"
|
||||
echo "spec + src_dir"
|
||||
elif [ "" != "$COPY_LIST" ] ; then
|
||||
local TARBALL=""
|
||||
for f in $COPY_LIST; do
|
||||
case $f in
|
||||
*.tar.gz) TARBALL=$f ;;
|
||||
*.tgz) TARBALL=$f ;;
|
||||
*.tar.bz2) TARBALL=$f ;;
|
||||
*.tar.xz) TARBALL=$f ;;
|
||||
*.tar) TARBALL=$f ;;
|
||||
esac
|
||||
done
|
||||
if [ "" != "$TARBALL" ]; then
|
||||
# echo "spec + tarball: $pkg_dir/$TARBALL"
|
||||
echo "spec + tarball"
|
||||
else
|
||||
# echo "spec + files: $pkg_dir"
|
||||
echo "spec + files"
|
||||
fi
|
||||
elif [ "$ALLOW_EMPTY_RPM" == "true" ] ; then
|
||||
# echo "spec + empty: $pkg_dir"
|
||||
echo "spec + empty"
|
||||
else
|
||||
# echo "spec + build_srpm.data + unknown: $pkg_dir"
|
||||
# cat $pkg_dir/centos/build_srpm.data
|
||||
echo "spec + build_srpm.data + unknown"
|
||||
fi
|
||||
else
|
||||
# echo "spec + unknown: $pkg_dir"
|
||||
echo "spec + unknown"
|
||||
fi
|
||||
else
|
||||
# echo "unknown: $pkg_dir"
|
||||
echo "unknown"
|
||||
fi
|
||||
}
|
||||
|
||||
# for g in $(find $MY_REPO -type d -name .git); do
|
||||
# d=$(dirname $g)
|
||||
# c="$d/centos_pkg_dirs"
|
||||
# if [ -f $c ]; then
|
||||
# for d2 in $(cat $c); do
|
||||
# pkg_dir="$d/$d2"
|
||||
# echo "$(classify $pkg_dir): $pkg_dir"
|
||||
# done
|
||||
# fi
|
||||
# done
|
||||
|
||||
54
build-tools/create-cgcs-centos-repo
Executable file
54
build-tools/create-cgcs-centos-repo
Executable file
@ -0,0 +1,54 @@
|
||||
#!/bin/bash
|
||||
|
||||
DEST_DIR="$MY_REPO/cgcs-centos-repo"
|
||||
ORIGIN="$DEST_DIR/origin"
|
||||
|
||||
CREATEREPO=$(which createrepo_c)
|
||||
if [ $? -ne 0 ]; then
|
||||
CREATEREPO="createrepo"
|
||||
fi
|
||||
|
||||
if [ ! -f $ORIGIN ]; then
|
||||
echo "ERROR: file not found '$ORIGIN'"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
ORIGIN_DIR=$(cat $MY_REPO/cgcs-centos-repo/origin | grep -v '^#' | head -n 1)
|
||||
|
||||
if [ ! -d $ORIGIN_DIR ]; then
|
||||
echo "ERROR: directory not found '$ORIGIN_DIR'"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
echo "ORIGIN_DIR=$ORIGIN_DIR"
|
||||
for d in $(find $ORIGIN_DIR -type d | tail -n +2); do
|
||||
RELATIVE_DIR=$(echo $d | sed "s#^$ORIGIN_DIR/##")
|
||||
if [ -d $DEST_DIR/$RELATIVE_DIR ]; then
|
||||
rm -rf $DEST_DIR/$RELATIVE_DIR/*
|
||||
fi
|
||||
mkdir -p $DEST_DIR/$RELATIVE_DIR
|
||||
done
|
||||
|
||||
for d in $(find $ORIGIN_DIR -type d | tail -n +2); do
|
||||
for f in $(find $d -maxdepth 1 -type f); do
|
||||
RELATIVE_FILE=$(echo $f | sed "s#^$ORIGIN_DIR/##")
|
||||
if [ -e "$DEST_DIR/$RELATIVE_FILE" ]; then
|
||||
rm -f "$DEST_DIR/$RELATIVE_FILE"
|
||||
fi
|
||||
|
||||
ln -s $f "$DEST_DIR/$RELATIVE_FILE"
|
||||
done
|
||||
done
|
||||
|
||||
for d in `find -L $DEST_DIR -type d -name repodata`; do
|
||||
(cd $d/..
|
||||
cp -L -f repodata/*comps*xml comps.xml
|
||||
rm -rf repodata
|
||||
if [ -f comps.xml ]; then
|
||||
$CREATEREPO -g comps.xml --workers $(cat /usr/bin/nproc) $(pwd)
|
||||
else
|
||||
$CREATEREPO --workers $(cat /usr/bin/nproc) $(pwd)
|
||||
fi
|
||||
)
|
||||
done
|
||||
|
||||
55
build-tools/create-cgcs-tis-repo
Executable file
55
build-tools/create-cgcs-tis-repo
Executable file
@ -0,0 +1,55 @@
|
||||
#!/bin/bash
|
||||
|
||||
DEST_DIR="$MY_REPO/cgcs-tis-repo"
|
||||
ORIGIN="$DEST_DIR/origin"
|
||||
|
||||
CREATEREPO=$(which createrepo_c)
|
||||
if [ $? -ne 0 ]; then
|
||||
CREATEREPO="createrepo"
|
||||
fi
|
||||
|
||||
|
||||
if [ ! -f $ORIGIN ]; then
|
||||
echo "ERROR: file not found '$ORIGIN'"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
ORIGIN_DIR=$(cat $MY_REPO/cgcs-tis-repo/origin | grep -v '^#' | head -n 1)
|
||||
|
||||
if [ ! -d $ORIGIN_DIR ]; then
|
||||
echo "ERROR: directory not found '$ORIGIN_DIR'"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
echo "ORIGIN_DIR=$ORIGIN_DIR"
|
||||
for d in $(find $ORIGIN_DIR -type d | tail -n +2); do
|
||||
RELATIVE_DIR=$(echo $d | sed "s#^$ORIGIN_DIR/##")
|
||||
if [ -d $DEST_DIR/$RELATIVE_DIR ]; then
|
||||
rm -rf $DEST_DIR/$RELATIVE_DIR/*
|
||||
fi
|
||||
echo "mkdir -p $RELATIVE_DIR"
|
||||
mkdir -p $DEST_DIR/$RELATIVE_DIR
|
||||
done
|
||||
|
||||
for d in $(find $ORIGIN_DIR -type d | tail -n +2); do
|
||||
for f in $(find $d -maxdepth 1 -type f); do
|
||||
RELATIVE_FILE=$(echo $f | sed "s#^$ORIGIN_DIR/##")
|
||||
if [ -e "$DEST_DIR/$RELATIVE_FILE" ]; then
|
||||
rm -f "$DEST_DIR/$RELATIVE_FILE"
|
||||
fi
|
||||
|
||||
ln -s $f "$DEST_DIR/$RELATIVE_FILE"
|
||||
done
|
||||
done
|
||||
|
||||
for d in `find -L $DEST_DIR -type d -name repodata`; do
|
||||
(cd $d/..
|
||||
rm -rf repodata
|
||||
if [ -f comps.xml ]; then
|
||||
$CREATEREPO -g comps.xml `pwd`
|
||||
else
|
||||
$CREATEREPO `pwd`
|
||||
fi
|
||||
)
|
||||
done
|
||||
|
||||
59
build-tools/create-yum-conf
Executable file
59
build-tools/create-yum-conf
Executable file
@ -0,0 +1,59 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ "$MY_WORKSPACE" == "" ]; then
|
||||
echo "ERROR: MY_WORKSPACE not defined"
|
||||
exit 1;
|
||||
fi
|
||||
|
||||
if [ "$MY_REPO" == "" ]; then
|
||||
echo "ERROR: MY_REPO not defined"
|
||||
exit 1;
|
||||
fi
|
||||
|
||||
if [ "$MY_BUILD_ENVIRONMENT" == "" ]; then
|
||||
echo "ERROR: MY_BUILD_ENVIRONMENT not defined"
|
||||
exit 1;
|
||||
fi
|
||||
|
||||
if [ "$MY_BUILD_DIR" == "" ]; then
|
||||
echo "ERROR: MY_BUILD_DIR not defined"
|
||||
exit 1;
|
||||
fi
|
||||
|
||||
MY_YUM_CONF="$MY_WORKSPACE/yum.conf"
|
||||
MOCK_CFG_PROTO="$MY_REPO/cgcs-centos-repo/mock.cfg.proto"
|
||||
YUM_DIR="$MY_WORKSPACE/yum"
|
||||
YUM_CACHE="$YUM_DIR/cache"
|
||||
|
||||
if [ -f "$MOCK_CFG_PROTO" ]; then
|
||||
if [ -f "$MY_YUM_CONF" ]; then
|
||||
N=$(find $MOCK_CFG_PROTO $MY_REPO/build-tools/create-yum-conf -cnewer $MY_YUM_CONF | wc -l)
|
||||
if [ $N -gt 0 ]; then
|
||||
# New inputs, remove to force regeneration of yum.conf
|
||||
\rm -f "$MY_YUM_CONF"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -f "$MY_YUM_CONF" ]; then
|
||||
if [ -f "$MOCK_CFG_PROTO" ]; then
|
||||
mock_cfg_to_yum_conf.py "$MOCK_CFG_PROTO" > "$MY_YUM_CONF"
|
||||
sed -i "s%\[main\]%&\ncachedir=$YUM_CACHE%" "$MY_YUM_CONF"
|
||||
sed -i "s%logfile=.*%logfile=$YUM_DIR/yum.log%" "$MY_YUM_CONF"
|
||||
sed -i "s%LOCAL_BASE%file://%g" "$MY_YUM_CONF"
|
||||
sed -i "s%MIRROR_BASE%file:///import/mirrors%g" "$MY_YUM_CONF"
|
||||
sed -i "s%BUILD_ENV%$MY_BUILD_ENVIRONMENT%g" "$MY_YUM_CONF"
|
||||
sed -i "s%/MY_BUILD_DIR%$MY_BUILD_DIR%g" "$MY_YUM_CONF"
|
||||
sed -i "s%/MY_REPO_DIR%$MY_REPO%g" "$MY_YUM_CONF"
|
||||
else
|
||||
echo "ERROR: Could not find yum.conf or MOCK_CFG_PROTO"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -d "$YUM_CACHE" ]; then
|
||||
mkdir -p "$YUM_CACHE"
|
||||
fi
|
||||
|
||||
echo "$MY_YUM_CONF"
|
||||
exit 0
|
||||
674
build-tools/create_dependancy_cache.py
Executable file
674
build-tools/create_dependancy_cache.py
Executable file
@ -0,0 +1,674 @@
|
||||
#!/usr/bin/python
|
||||
|
||||
import xml.etree.ElementTree as ET
|
||||
import fnmatch
|
||||
import os
|
||||
import gzip
|
||||
import getopt
|
||||
import sys
|
||||
import string
|
||||
|
||||
|
||||
ns = { 'root': 'http://linux.duke.edu/metadata/common',
|
||||
'filelists': 'http://linux.duke.edu/metadata/filelists',
|
||||
'rpm': 'http://linux.duke.edu/metadata/rpm' }
|
||||
|
||||
build_types=['std', 'rt']
|
||||
rpm_types=['RPM', 'SRPM']
|
||||
default_arch = 'x86_64'
|
||||
default_arch_list = [ 'x86_64', 'noarch' ]
|
||||
default_arch_by_type = {'RPM': [ 'x86_64', 'noarch' ],
|
||||
'SRPM': [ 'src' ]
|
||||
}
|
||||
|
||||
repodata_dir="/export/jenkins/mirrors"
|
||||
if not os.path.isdir(repodata_dir):
|
||||
repodata_dir="/import/mirrors"
|
||||
if not os.path.isdir(repodata_dir):
|
||||
print("ERROR: directory not found %s" % repodata_dir)
|
||||
sys.exit(1)
|
||||
|
||||
publish_cache_dir="%s/cgcs-tis-repo/dependancy-cache" % os.environ['MY_REPO']
|
||||
centos_repo_dir="%s/cgcs-centos-repo" % os.environ['MY_REPO']
|
||||
third_party_repo_dir="%s/cgcs-3rd-party-repo" % os.environ['MY_REPO']
|
||||
tis_repo_dir="%s/cgcs-tis-repo" % os.environ['MY_REPO']
|
||||
workspace_repo_dirs={}
|
||||
for rt in rpm_types:
|
||||
workspace_repo_dirs[rt]={}
|
||||
for bt in build_types:
|
||||
workspace_repo_dirs[rt][bt]="%s/%s/rpmbuild/%sS" % (os.environ['MY_WORKSPACE'], bt, rt)
|
||||
|
||||
if not os.path.isdir(os.environ['MY_REPO']):
|
||||
print("ERROR: directory not found MY_REPO=%s" % os.environ['MY_REPO'])
|
||||
sys.exit(1)
|
||||
|
||||
if not os.path.isdir(centos_repo_dir):
|
||||
print("ERROR: directory not found %s" % centos_repo_dir)
|
||||
sys.exit(1)
|
||||
|
||||
if not os.path.isdir(third_party_repo_dir):
|
||||
print("ERROR: directory not found %s" % third_party_repo_dir)
|
||||
sys.exit(1)
|
||||
|
||||
if not os.path.isdir(tis_repo_dir):
|
||||
print("ERROR: directory not found %s" % tis_repo_dir)
|
||||
sys.exit(1)
|
||||
|
||||
# bin_rpm_mirror_roots = ["%s/fedora/epel/7" % repodata_dir,
|
||||
# "%s/CentOS/7.2.1511" % repodata_dir,
|
||||
# "%s/CentOS/tis-r3/" % repodata_dir ]
|
||||
|
||||
# src_rpm_mirror_roots = ["%s/fedora/dl.fedoraproject.org/pub/epel/7/SRPMS" % repodata_dir,
|
||||
# "%s/CentOS/vault.centos.org/7.2.1511" % repodata_dir,
|
||||
# "%s/CentOS/tis-r3/Source" % repodata_dir ]
|
||||
|
||||
bin_rpm_mirror_roots = ["%s/Binary" % centos_repo_dir,
|
||||
"%s/Binary" % third_party_repo_dir ]
|
||||
|
||||
src_rpm_mirror_roots = ["%s/Source" % centos_repo_dir,
|
||||
"%s/Source" % third_party_repo_dir ]
|
||||
|
||||
for bt in build_types:
|
||||
bin_rpm_mirror_roots.append(workspace_repo_dirs['RPM'][bt])
|
||||
src_rpm_mirror_roots.append(workspace_repo_dirs['SRPM'][bt])
|
||||
|
||||
short_options=''
|
||||
long_options=[ 'cache_dir=' ]
|
||||
|
||||
options, remainder = getopt.getopt(sys.argv[1:], short_options, long_options)
|
||||
|
||||
for opt, arg in options:
|
||||
if opt in ('--cache_dir'):
|
||||
publish_cache_dir = arg
|
||||
|
||||
if not os.path.isdir(publish_cache_dir):
|
||||
print("ERROR: directory not found %s" % publish_cache_dir)
|
||||
sys.exit(1)
|
||||
|
||||
# The Main data structure
|
||||
pkg_data={}
|
||||
|
||||
for rpm_type in rpm_types:
|
||||
pkg_data[rpm_type]={}
|
||||
|
||||
# map provided_name -> pkg_name
|
||||
pkg_data[rpm_type]['providers']={}
|
||||
|
||||
# map pkg_name -> required_names ... could be a pkg, capability or file
|
||||
pkg_data[rpm_type]['requires']={}
|
||||
|
||||
# map file_name -> pkg_name
|
||||
pkg_data[rpm_type]['file_owners']={}
|
||||
|
||||
# map pkg_name -> file_name
|
||||
pkg_data[rpm_type]['files']={}
|
||||
|
||||
# map pkg_name -> required_pkg_names ... only pkg names, and only direct requirement
|
||||
pkg_data[rpm_type]['pkg_direct_requires']={}
|
||||
|
||||
# map pkg_name -> required_pkg_names ... only pkg names, but this is the transitive list of all requirements
|
||||
pkg_data[rpm_type]['pkg_transitive_requires']={}
|
||||
|
||||
# map pkg_name -> descendant_pkgs ... only packages the directly require this package
|
||||
pkg_data[rpm_type]['pkg_direct_descendants']={}
|
||||
|
||||
# map pkg_name -> descendant_pkgs ... packages that have a transitive requiremant on this package
|
||||
pkg_data[rpm_type]['pkg_transitive_descendants']={}
|
||||
|
||||
# Map package name to a source rpm file name
|
||||
pkg_data[rpm_type]['sourcerpm']={}
|
||||
pkg_data[rpm_type]['binrpm']={}
|
||||
|
||||
# Map file name to package name
|
||||
pkg_data[rpm_type]['fn_to_name']={}
|
||||
|
||||
pkg_data['SRPM']['pkg_direct_requires_rpm']={}
|
||||
pkg_data['SRPM']['pkg_transitive_requires_rpm']={}
|
||||
|
||||
|
||||
# Return a list of file paths, starting in 'dir', matching 'pattern'
|
||||
# dir= directory to search under
|
||||
# pattern= search for file or directory matching pattern, wildcards allowed
|
||||
# recursive_depth= how many levels of directory before giving up
|
||||
def file_search(dir, pattern, recursive_depth=0):
|
||||
match_list = []
|
||||
new_depth = recursive_depth - 1
|
||||
# print "file_search(%s,%s,%s)" % (dir, pattern, recursive_depth)
|
||||
for file in os.listdir(dir):
|
||||
path = "%s/%s" % (dir, file)
|
||||
if fnmatch.fnmatch(file, pattern):
|
||||
print path
|
||||
match_list.append(path)
|
||||
elif (recursive_depth > 0) and os.path.isdir(path):
|
||||
sub_list = []
|
||||
sub_list = file_search(path, pattern, recursive_depth=new_depth)
|
||||
match_list.extend(sub_list)
|
||||
return match_list
|
||||
|
||||
# Return the list of .../repodate/*primary.xml.gz files
|
||||
# rpm_type= 'RPM' or 'SRPM'
|
||||
# arch= e.g. x86_64, only relevant of rpm_type=='RPM'
|
||||
def get_repo_primary_data_list(rpm_type='RPM', arch_list=default_arch_list):
|
||||
rpm_repo_roots = []
|
||||
rpm_repodata_roots = []
|
||||
repodata_list = []
|
||||
|
||||
if rpm_type == 'RPM':
|
||||
for d in bin_rpm_mirror_roots:
|
||||
sub_list = file_search(d, 'repodata', 25)
|
||||
rpm_repodata_roots.extend(sub_list)
|
||||
elif rpm_type == 'SRPM':
|
||||
for d in src_rpm_mirror_roots:
|
||||
sub_list = file_search(d, 'repodata', 5)
|
||||
rpm_repodata_roots.extend(sub_list)
|
||||
else:
|
||||
print "invalid rpm_type '%s', valid types are %s" % (rpm_type, str(rpm_types))
|
||||
return repodata_list
|
||||
|
||||
for d in rpm_repodata_roots:
|
||||
sub_list = file_search(d, '*primary.xml.gz', 2)
|
||||
repodata_list.extend(sub_list)
|
||||
|
||||
return repodata_list
|
||||
|
||||
|
||||
# Return the list of .../repodate/*filelists.xml.gz files
|
||||
# rpm_type= 'RPM' or 'SRPM'
|
||||
# arch= e.g. x86_64, only relevant of rpm_type=='RPM'
|
||||
def get_repo_filelists_data_list(rpm_type='RPM', arch_list=default_arch_list):
|
||||
rpm_repo_roots = []
|
||||
rpm_repodata_roots = []
|
||||
repodata_list = []
|
||||
|
||||
if rpm_type == 'RPM':
|
||||
for d in bin_rpm_mirror_roots:
|
||||
sub_list = file_search(d, 'repodata', 25)
|
||||
rpm_repodata_roots.extend(sub_list)
|
||||
elif rpm_type == 'SRPM':
|
||||
for d in src_rpm_mirror_roots:
|
||||
sub_list = file_search(d, 'repodata', 5)
|
||||
rpm_repodata_roots.extend(sub_list)
|
||||
else:
|
||||
print "invalid rpm_type '%s', valid types are %s" % (rpm_type, str(rpm_types))
|
||||
return repodata_list
|
||||
|
||||
for d in rpm_repodata_roots:
|
||||
sub_list = file_search(d, '*filelists.xml.gz', 2)
|
||||
repodata_list.extend(sub_list)
|
||||
|
||||
return repodata_list
|
||||
|
||||
|
||||
|
||||
# Process a list of repodata files (*filelists.xml.gz) and extract package data.
|
||||
# Data is saved to the global 'pkg_data'.
|
||||
def read_data_from_repodata_filelists_list(repodata_list, rpm_type='RPM', arch=default_arch):
|
||||
for repodata_path in repodata_list:
|
||||
read_data_from_filelists_xml_gz(repodata_path, rpm_type=rpm_type, arch=arch)
|
||||
|
||||
# Process a single repodata file (*filelists.xml.gz) and extract package data.
|
||||
# Data is saved to the global 'pkg_data'.
|
||||
def read_data_from_filelists_xml_gz(repodata_path, rpm_type='RPM', arch=default_arch):
|
||||
# print "repodata_path=%s" % repodata_path
|
||||
infile = gzip.open(repodata_path)
|
||||
root = ET.parse(infile).getroot()
|
||||
for pkg in root.findall('filelists:package', ns):
|
||||
name=pkg.get('name')
|
||||
pkg_arch=pkg.get('arch')
|
||||
|
||||
version=""
|
||||
release=""
|
||||
|
||||
if arch is not None:
|
||||
if pkg_arch is None:
|
||||
continue
|
||||
if pkg_arch != arch:
|
||||
continue
|
||||
|
||||
v=pkg.find('filelists:version', ns)
|
||||
if v is not None:
|
||||
version=v.get('ver')
|
||||
release=v.get('rel')
|
||||
else:
|
||||
print "%s: %s.%s has no 'filelists:version'" % (repodata_path, name, pkg_arch)
|
||||
|
||||
# print "%s %s %s %s " % (name, pkg_arch, version, release)
|
||||
|
||||
for f in pkg.findall('filelists:file', ns):
|
||||
fn=f.text
|
||||
# print " fn=%s -> plg=%s" % (fn, name)
|
||||
if not name in pkg_data[rpm_type]['files']:
|
||||
pkg_data[rpm_type]['files'][name]=[]
|
||||
pkg_data[rpm_type]['files'][name].append(fn)
|
||||
if not fn in pkg_data[rpm_type]['file_owners']:
|
||||
pkg_data[rpm_type]['file_owners'][fn]=[]
|
||||
pkg_data[rpm_type]['file_owners'][fn]=name
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# Process a list of repodata files (*primary.xml.gz) and extract package data.
|
||||
# Data is saved to the global 'pkg_data'.
|
||||
def read_data_from_repodata_primary_list(repodata_list, rpm_type='RPM', arch=default_arch):
|
||||
for repodata_path in repodata_list:
|
||||
read_data_from_primary_xml_gz(repodata_path, rpm_type=rpm_type, arch=arch)
|
||||
|
||||
# Process a single repodata file (*primary.xml.gz) and extract package data.
|
||||
# Data is saved to the global 'pkg_data'.
|
||||
def read_data_from_primary_xml_gz(repodata_path, rpm_type='RPM', arch=default_arch):
|
||||
# print "repodata_path=%s" % repodata_path
|
||||
infile = gzip.open(repodata_path)
|
||||
root = ET.parse(infile).getroot()
|
||||
for pkg in root.findall('root:package', ns):
|
||||
name=pkg.find('root:name', ns).text
|
||||
pkg_arch=pkg.find('root:arch', ns).text
|
||||
version=""
|
||||
release=""
|
||||
license=""
|
||||
sourcerpm=""
|
||||
|
||||
if arch is not None:
|
||||
if pkg_arch is None:
|
||||
continue
|
||||
if pkg_arch != arch:
|
||||
continue
|
||||
|
||||
pkg_data[rpm_type]['providers'][name]=name
|
||||
pkg_data[rpm_type]['files'][name]=[]
|
||||
pkg_data[rpm_type]['requires'][name] = []
|
||||
pkg_data[rpm_type]['requires'][name].append(name)
|
||||
|
||||
url=pkg.find('root:url', ns).text
|
||||
v=pkg.find('root:version', ns)
|
||||
if v is not None:
|
||||
version=v.get('ver')
|
||||
release=v.get('rel')
|
||||
else:
|
||||
print "%s: %s.%s has no 'root:version'" % (repodata_path, name, pkg_arch)
|
||||
|
||||
fn="%s-%s-%s.%s.rpm" % (name, version, release, arch)
|
||||
pkg_data[rpm_type]['fn_to_name'][fn]=name
|
||||
|
||||
# SAL print "%s %s %s %s " % (name, pkg_arch, version, release)
|
||||
print "%s %s %s %s " % (name, pkg_arch, version, release)
|
||||
f=pkg.find('root:format', ns)
|
||||
if f is not None:
|
||||
license=f.find('rpm:license', ns).text
|
||||
sourcerpm=f.find('rpm:sourcerpm', ns).text
|
||||
if sourcerpm != "":
|
||||
pkg_data[rpm_type]['sourcerpm'][name] = sourcerpm
|
||||
# SAL print "--- requires ---"
|
||||
print "--- requires ---"
|
||||
r=f.find('rpm:requires', ns)
|
||||
if r is not None:
|
||||
for rr in r.findall('rpm:entry', ns):
|
||||
required_name=rr.get('name')
|
||||
# SAL print " %s" % required_name
|
||||
print " %s" % required_name
|
||||
pkg_data[rpm_type]['requires'][name].append(required_name)
|
||||
else:
|
||||
print "%s: %s.%s has no 'rpm:requires'" % (repodata_path, name, pkg_arch)
|
||||
# print "--- provides ---"
|
||||
p=f.find('rpm:provides', ns)
|
||||
if p is not None:
|
||||
for pp in p.findall('rpm:entry', ns):
|
||||
provided_name=pp.get('name')
|
||||
# print " %s" % provided_name
|
||||
if name == "kernel-rt" and provided_name in pkg_data[rpm_type]['providers'] and pkg_data[rpm_type]['providers'][provided_name] == "kernel":
|
||||
continue
|
||||
if name.startswith('kernel-rt'):
|
||||
alt_name=string.replace(name, 'kernel-rt', 'kernel')
|
||||
if provided_name in pkg_data[rpm_type]['providers'] and pkg_data[rpm_type]['providers'][provided_name] == alt_name:
|
||||
continue
|
||||
pkg_data[rpm_type]['providers'][provided_name]=name
|
||||
else:
|
||||
print "%s: %s.%s has no 'rpm:provides'" % (repodata_path, name, pkg_arch)
|
||||
# print "--- files ---"
|
||||
for fn in f.findall('root:file', ns):
|
||||
file_name=fn.text
|
||||
# print " %s" % file_name
|
||||
pkg_data[rpm_type]['files'][name].append(file_name)
|
||||
if name == "kernel-rt" and file_name in pkg_data[rpm_type]['file_owners'] and pkg_data[rpm_type]['file_owners'][file_name] == "kernel":
|
||||
continue
|
||||
if name.startswith('kernel-rt'):
|
||||
alt_name=string.replace(name, 'kernel-rt', 'kernel')
|
||||
if provided_name in pkg_data[rpm_type]['file_owners'] and pkg_data[rpm_type]['file_owners'][file_name] == alt_name:
|
||||
continue
|
||||
pkg_data[rpm_type]['file_owners'][file_name]=name
|
||||
else:
|
||||
print "%s: %s.%s has no 'root:format'" % (repodata_path, name, pkg_arch)
|
||||
# print "%s %s %s %s %s" % (name, pkg_arch, version, release, license)
|
||||
infile.close
|
||||
|
||||
def calulate_all_direct_requires_and_descendants(rpm_type='RPM'):
|
||||
# print "calulate_all_direct_requires_and_descendants rpm_type=%s" % rpm_type
|
||||
for name in pkg_data[rpm_type]['requires']:
|
||||
calulate_pkg_direct_requires_and_descendants(name, rpm_type=rpm_type)
|
||||
|
||||
def calulate_pkg_direct_requires_and_descendants(name, rpm_type='RPM'):
|
||||
print "SAL: %s needs:" % name
|
||||
if not rpm_type in pkg_data:
|
||||
print "Error: unknown rpm_type '%s'" % rpm_type
|
||||
return
|
||||
|
||||
if not name in pkg_data[rpm_type]['requires']:
|
||||
print "Note: No requires data for '%s'" % name
|
||||
return
|
||||
|
||||
for req in pkg_data[rpm_type]['requires'][name]:
|
||||
pro = '???'
|
||||
if rpm_type == 'RPM':
|
||||
if req in pkg_data[rpm_type]['providers']:
|
||||
pro = pkg_data[rpm_type]['providers'][req]
|
||||
elif req in pkg_data[rpm_type]['file_owners']:
|
||||
pro = pkg_data[rpm_type]['file_owners'][req]
|
||||
else:
|
||||
pro = '???'
|
||||
print "package %s has unresolved requirement '%s'" % (name, req)
|
||||
else:
|
||||
# i.e. rpm_type == 'SRPM'
|
||||
rpm_pro = '???'
|
||||
if req in pkg_data['RPM']['providers']:
|
||||
rpm_pro = pkg_data['RPM']['providers'][req]
|
||||
elif req in pkg_data['RPM']['file_owners']:
|
||||
rpm_pro = pkg_data['RPM']['file_owners'][req]
|
||||
else:
|
||||
rpm_pro = '???'
|
||||
print "package %s has unresolved requirement '%s'" % (name, req)
|
||||
|
||||
if rpm_pro is not None and rpm_pro != '???':
|
||||
if not name in pkg_data[rpm_type]['pkg_direct_requires_rpm']:
|
||||
pkg_data[rpm_type]['pkg_direct_requires_rpm'][name] = []
|
||||
if not rpm_pro in pkg_data[rpm_type]['pkg_direct_requires_rpm'][name]:
|
||||
pkg_data[rpm_type]['pkg_direct_requires_rpm'][name].append(rpm_pro)
|
||||
|
||||
if rpm_pro in pkg_data['RPM']['sourcerpm']:
|
||||
fn = pkg_data['RPM']['sourcerpm'][rpm_pro]
|
||||
if fn in pkg_data['SRPM']['fn_to_name']:
|
||||
pro = pkg_data['SRPM']['fn_to_name'][fn]
|
||||
else:
|
||||
pro = '???'
|
||||
print "package %s requires srpm file name %s" % (name,fn)
|
||||
else:
|
||||
pro = '???'
|
||||
print "package %s requires rpm %s, but that rpm has no known srpm" % (name,rpm_pro)
|
||||
|
||||
if pro is not None and pro != '???':
|
||||
if not name in pkg_data[rpm_type]['pkg_direct_requires']:
|
||||
pkg_data[rpm_type]['pkg_direct_requires'][name] = []
|
||||
if not pro in pkg_data[rpm_type]['pkg_direct_requires'][name]:
|
||||
pkg_data[rpm_type]['pkg_direct_requires'][name].append(pro)
|
||||
if not pro in pkg_data[rpm_type]['pkg_direct_descendants']:
|
||||
pkg_data[rpm_type]['pkg_direct_descendants'][pro] = []
|
||||
if not name in pkg_data[rpm_type]['pkg_direct_descendants'][pro]:
|
||||
pkg_data[rpm_type]['pkg_direct_descendants'][pro].append(name)
|
||||
|
||||
print "SAL: %s -> %s" % (req, pro)
|
||||
|
||||
|
||||
|
||||
def calulate_all_transitive_requires(rpm_type='RPM'):
|
||||
for name in pkg_data[rpm_type]['pkg_direct_requires']:
|
||||
calulate_pkg_transitive_requires(name, rpm_type=rpm_type)
|
||||
|
||||
def calulate_pkg_transitive_requires(name, rpm_type='RPM'):
|
||||
if not rpm_type in pkg_data:
|
||||
print "Error: unknown rpm_type '%s'" % rpm_type
|
||||
return
|
||||
|
||||
if not name in pkg_data[rpm_type]['pkg_direct_requires']:
|
||||
print "Note: No direct_requires data for '%s'" % name
|
||||
return
|
||||
|
||||
pkg_data[rpm_type]['pkg_transitive_requires'][name]=[]
|
||||
if rpm_type != 'RPM':
|
||||
pkg_data[rpm_type]['pkg_transitive_requires_rpm'][name]=[]
|
||||
unresolved = []
|
||||
unresolved.append(name)
|
||||
|
||||
while unresolved:
|
||||
n = unresolved.pop(0)
|
||||
# print "%s: remove %s" % (name, n)
|
||||
if rpm_type == 'RPM':
|
||||
direct_requires='pkg_direct_requires'
|
||||
transitive_requires='pkg_transitive_requires'
|
||||
else:
|
||||
direct_requires='pkg_direct_requires_rpm'
|
||||
transitive_requires='pkg_transitive_requires_rpm'
|
||||
if n in pkg_data[rpm_type][direct_requires]:
|
||||
for r in pkg_data[rpm_type][direct_requires][n]:
|
||||
if r != name:
|
||||
if not r in pkg_data[rpm_type][transitive_requires][name]:
|
||||
pkg_data[rpm_type][transitive_requires][name].append(r)
|
||||
if r in pkg_data['RPM']['pkg_transitive_requires']:
|
||||
for r2 in pkg_data['RPM']['pkg_transitive_requires'][r]:
|
||||
if r2 != name:
|
||||
if not r2 in pkg_data[rpm_type][transitive_requires][name]:
|
||||
pkg_data[rpm_type][transitive_requires][name].append(r2)
|
||||
else:
|
||||
if rpm_type == 'RPM':
|
||||
unresolved.append(r)
|
||||
else:
|
||||
print "WARNING: calulate_pkg_transitive_requires: can't append rpm to SRPM list, name=%s, r=%s" % (name, r)
|
||||
# print "%s: add %s" % (name, r)
|
||||
if rpm_type != 'RPM':
|
||||
for r in pkg_data[rpm_type]['pkg_transitive_requires_rpm'][name]:
|
||||
if r in pkg_data['RPM']['sourcerpm']:
|
||||
fn = pkg_data['RPM']['sourcerpm'][r]
|
||||
if fn in pkg_data['SRPM']['fn_to_name']:
|
||||
s = pkg_data['SRPM']['fn_to_name'][fn]
|
||||
pkg_data[rpm_type]['pkg_transitive_requires'][name].append(s)
|
||||
else:
|
||||
print "package %s requires srpm file name %s, but srpm name is not known" % (name, fn)
|
||||
else:
|
||||
print "package %s requires rpm %s, but that rpm has no known srpm" % (name, r)
|
||||
|
||||
def calulate_all_transitive_descendants(rpm_type='RPM'):
|
||||
for name in pkg_data[rpm_type]['pkg_direct_descendants']:
|
||||
calulate_pkg_transitive_descendants(name, rpm_type=rpm_type)
|
||||
|
||||
def calulate_pkg_transitive_descendants(name, rpm_type='RPM'):
|
||||
if not rpm_type in pkg_data:
|
||||
print "Error: unknown rpm_type '%s'" % rpm_type
|
||||
return
|
||||
|
||||
if not name in pkg_data[rpm_type]['pkg_direct_descendants']:
|
||||
print "Note: No direct_requires data for '%s'" % name
|
||||
return
|
||||
|
||||
pkg_data[rpm_type]['pkg_transitive_descendants'][name]=[]
|
||||
unresolved = []
|
||||
unresolved.append(name)
|
||||
|
||||
while unresolved:
|
||||
n = unresolved.pop(0)
|
||||
# print "%s: remove %s" % (name, n)
|
||||
if n in pkg_data[rpm_type]['pkg_direct_descendants']:
|
||||
for r in pkg_data[rpm_type]['pkg_direct_descendants'][n]:
|
||||
if r != name:
|
||||
if not r in pkg_data[rpm_type]['pkg_transitive_descendants'][name]:
|
||||
pkg_data[rpm_type]['pkg_transitive_descendants'][name].append(r)
|
||||
if r in pkg_data[rpm_type]['pkg_transitive_descendants']:
|
||||
for n2 in pkg_data[rpm_type]['pkg_transitive_descendants'][r]:
|
||||
if n2 != name:
|
||||
if not n2 in pkg_data[rpm_type]['pkg_transitive_descendants'][name]:
|
||||
pkg_data[rpm_type]['pkg_transitive_descendants'][name].append(n2)
|
||||
else:
|
||||
unresolved.append(r)
|
||||
# print "%s: add %s" % (name, r)
|
||||
|
||||
def create_dest_rpm_data():
|
||||
for name in sorted(pkg_data['RPM']['sourcerpm']):
|
||||
fn=pkg_data['RPM']['sourcerpm'][name]
|
||||
if fn in pkg_data['SRPM']['fn_to_name']:
|
||||
sname = pkg_data['SRPM']['fn_to_name'][fn]
|
||||
if not sname in pkg_data['SRPM']['binrpm']:
|
||||
pkg_data['SRPM']['binrpm'][sname]=[]
|
||||
pkg_data['SRPM']['binrpm'][sname].append(name)
|
||||
|
||||
def create_cache(cache_dir):
|
||||
for rpm_type in rpm_types:
|
||||
print ""
|
||||
print "==== %s ====" % rpm_type
|
||||
print ""
|
||||
rpm_repodata_primary_list = get_repo_primary_data_list(rpm_type=rpm_type, arch_list=default_arch_by_type[rpm_type])
|
||||
for arch in default_arch_by_type[rpm_type]:
|
||||
read_data_from_repodata_primary_list(rpm_repodata_primary_list, rpm_type=rpm_type, arch=arch)
|
||||
rpm_repodata_filelists_list = get_repo_filelists_data_list(rpm_type=rpm_type, arch_list=default_arch_by_type[rpm_type])
|
||||
for arch in default_arch_by_type[rpm_type]:
|
||||
read_data_from_repodata_filelists_list(rpm_repodata_filelists_list, rpm_type=rpm_type, arch=arch)
|
||||
calulate_all_direct_requires_and_descendants(rpm_type=rpm_type)
|
||||
calulate_all_transitive_requires(rpm_type=rpm_type)
|
||||
calulate_all_transitive_descendants(rpm_type=rpm_type)
|
||||
|
||||
cache_name="%s/%s-direct-requires" % (cache_dir, rpm_type)
|
||||
f=open(cache_name, "w")
|
||||
for name in sorted(pkg_data[rpm_type]['pkg_direct_requires']):
|
||||
print "%s needs %s" % (name, pkg_data[rpm_type]['pkg_direct_requires'][name])
|
||||
f.write("%s;" % name)
|
||||
first=True
|
||||
for req in sorted(pkg_data[rpm_type]['pkg_direct_requires'][name]):
|
||||
if first:
|
||||
first=False
|
||||
f.write("%s" % req)
|
||||
else:
|
||||
f.write(",%s" % req)
|
||||
f.write("\n")
|
||||
f.close()
|
||||
|
||||
cache_name="%s/%s-direct-descendants" % (cache_dir, rpm_type)
|
||||
f=open(cache_name, "w")
|
||||
for name in sorted(pkg_data[rpm_type]['pkg_direct_descendants']):
|
||||
print "%s informs %s" % (name, pkg_data[rpm_type]['pkg_direct_descendants'][name])
|
||||
f.write("%s;" % name)
|
||||
first=True
|
||||
for req in sorted(pkg_data[rpm_type]['pkg_direct_descendants'][name]):
|
||||
if first:
|
||||
first=False
|
||||
f.write("%s" % req)
|
||||
else:
|
||||
f.write(",%s" % req)
|
||||
f.write("\n")
|
||||
f.close()
|
||||
|
||||
cache_name="%s/%s-transitive-requires" % (cache_dir, rpm_type)
|
||||
f=open(cache_name, "w")
|
||||
for name in sorted(pkg_data[rpm_type]['pkg_transitive_requires']):
|
||||
f.write("%s;" % name)
|
||||
first=True
|
||||
for req in sorted(pkg_data[rpm_type]['pkg_transitive_requires'][name]):
|
||||
if first:
|
||||
first=False
|
||||
f.write("%s" % req)
|
||||
else:
|
||||
f.write(",%s" % req)
|
||||
f.write("\n")
|
||||
f.close()
|
||||
|
||||
cache_name="%s/%s-transitive-descendants" % (cache_dir, rpm_type)
|
||||
f=open(cache_name, "w")
|
||||
for name in sorted(pkg_data[rpm_type]['pkg_transitive_descendants']):
|
||||
f.write("%s;" % name)
|
||||
first=True
|
||||
for req in sorted(pkg_data[rpm_type]['pkg_transitive_descendants'][name]):
|
||||
if first:
|
||||
first=False
|
||||
f.write("%s" % req)
|
||||
else:
|
||||
f.write(",%s" % req)
|
||||
f.write("\n")
|
||||
f.close()
|
||||
|
||||
if rpm_type != 'RPM':
|
||||
cache_name="%s/%s-direct-requires-rpm" % (cache_dir, rpm_type)
|
||||
f=open(cache_name, "w")
|
||||
for name in sorted(pkg_data[rpm_type]['pkg_direct_requires_rpm']):
|
||||
print "%s needs rpm %s" % (name, pkg_data[rpm_type]['pkg_direct_requires_rpm'][name])
|
||||
f.write("%s;" % name)
|
||||
first=True
|
||||
for req in sorted(pkg_data[rpm_type]['pkg_direct_requires_rpm'][name]):
|
||||
if first:
|
||||
first=False
|
||||
f.write("%s" % req)
|
||||
else:
|
||||
f.write(",%s" % req)
|
||||
f.write("\n")
|
||||
f.close()
|
||||
|
||||
cache_name="%s/%s-transitive-requires-rpm" % (cache_dir, rpm_type)
|
||||
f=open(cache_name, "w")
|
||||
for name in sorted(pkg_data[rpm_type]['pkg_transitive_requires_rpm']):
|
||||
f.write("%s;" % name)
|
||||
first=True
|
||||
for req in sorted(pkg_data[rpm_type]['pkg_transitive_requires_rpm'][name]):
|
||||
if first:
|
||||
first=False
|
||||
f.write("%s" % req)
|
||||
else:
|
||||
f.write(",%s" % req)
|
||||
f.write("\n")
|
||||
f.close()
|
||||
|
||||
cache_name="%s/rpm-to-srpm" % cache_dir
|
||||
f=open(cache_name, "w")
|
||||
for name in sorted(pkg_data['RPM']['sourcerpm']):
|
||||
f.write("%s;" % name)
|
||||
fn=pkg_data['RPM']['sourcerpm'][name]
|
||||
if fn in pkg_data['SRPM']['fn_to_name']:
|
||||
sname = pkg_data['SRPM']['fn_to_name'][fn]
|
||||
f.write("%s" % sname)
|
||||
f.write("\n")
|
||||
f.close()
|
||||
|
||||
create_dest_rpm_data()
|
||||
cache_name="%s/srpm-to-rpm" % cache_dir
|
||||
f=open(cache_name, "w")
|
||||
for name in sorted(pkg_data['SRPM']['binrpm']):
|
||||
f.write("%s;" % name)
|
||||
first=True
|
||||
for bname in sorted(pkg_data['SRPM']['binrpm'][name]):
|
||||
if first:
|
||||
first=False
|
||||
f.write("%s" % bname)
|
||||
else:
|
||||
f.write(",%s" % bname)
|
||||
f.write("\n")
|
||||
f.close()
|
||||
|
||||
|
||||
|
||||
def test():
|
||||
for rpm_type in rpm_types:
|
||||
print ""
|
||||
print "==== %s ====" % rpm_type
|
||||
print ""
|
||||
rpm_repodata_primary_list = get_repo_primary_data_list(rpm_type=rpm_type, arch_list=default_arch_by_type[rpm_type])
|
||||
for arch in default_arch_by_type[rpm_type]:
|
||||
read_data_from_repodata_primary_list(rpm_repodata_primary_list, rpm_type=rpm_type, arch=arch)
|
||||
rpm_repodata_filelists_list = get_repo_filelists_data_list(rpm_type=rpm_type, arch_list=default_arch_by_type[rpm_type])
|
||||
for arch in default_arch_by_type[rpm_type]:
|
||||
read_data_from_repodata_filelists_list(rpm_repodata_filelists_list, rpm_type=rpm_type, arch=arch)
|
||||
calulate_all_direct_requires_and_descendants(rpm_type=rpm_type)
|
||||
calulate_all_transitive_requires(rpm_type=rpm_type)
|
||||
calulate_all_transitive_descendants(rpm_type=rpm_type)
|
||||
|
||||
for name in pkg_data[rpm_type]['pkg_direct_requires']:
|
||||
print "%s needs %s" % (name, pkg_data[rpm_type]['pkg_direct_requires'][name])
|
||||
|
||||
for name in pkg_data[rpm_type]['pkg_direct_descendants']:
|
||||
print "%s informs %s" % (name, pkg_data[rpm_type]['pkg_direct_descendants'][name])
|
||||
|
||||
for name in pkg_data[rpm_type]['pkg_transitive_requires']:
|
||||
print "%s needs %s" % (name, pkg_data[rpm_type]['pkg_transitive_requires'][name])
|
||||
print ""
|
||||
|
||||
for name in pkg_data[rpm_type]['pkg_transitive_descendants']:
|
||||
print "%s informs %s" % (name, pkg_data[rpm_type]['pkg_transitive_descendants'][name])
|
||||
print ""
|
||||
|
||||
|
||||
if os.path.isdir(publish_cache_dir):
|
||||
create_cache(publish_cache_dir)
|
||||
else:
|
||||
print "ERROR: Directory not found '%s" % publish_cache_dir
|
||||
264
build-tools/default_build_srpm
Executable file
264
build-tools/default_build_srpm
Executable file
@ -0,0 +1,264 @@
|
||||
#!/bin/bash
|
||||
# set -x
|
||||
|
||||
source "$SRC_BASE/build-tools/spec-utils"
|
||||
source "$SRC_BASE/build-tools/srpm-utils"
|
||||
|
||||
CUR_DIR=`pwd`
|
||||
BUILD_DIR="$RPMBUILD_BASE"
|
||||
|
||||
if [ "x$DATA" == "x" ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): Environment variable 'DATA' not defined."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
srpm_source_build_data $DATA
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): Failed to source build data from $DATA"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "x$VERSION" == "x" ]; then
|
||||
for SPEC in `find $SPECS_BASE -name '*.spec' | sort -V`; do
|
||||
SPEC_PATH="$SPEC"
|
||||
|
||||
VERSION_DERIVED=`spec_evaluate '%{version}' "$SPEC_PATH" 2>> /dev/null`
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): '%{version}' not found in '$PKG_BASE/$SPEC_PATH'"
|
||||
VERSION_DERIVED=""
|
||||
fi
|
||||
|
||||
if [ "x$VERSION_DERIVED" != "x" ]; then
|
||||
if [ "x$VERSION" == "x" ]; then
|
||||
VERSION=$VERSION_DERIVED
|
||||
else
|
||||
if [ "x$SRC_DIR" != "x" ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): multiple spec files found, can't set VERSION automatically"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "x$VERSION" == "x" ]; then
|
||||
if [ -f $SRC_DIR/PKG-INFO ]; then
|
||||
VERSION=$(grep '^Version:' $SRC_DIR/PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "x$VERSION" != "x" ]; then
|
||||
echo "Derived VERSION=$VERSION"
|
||||
else
|
||||
echo "ERROR: default_build_srpm (${LINENO}): Failed to derive a good VERSION from SPEC file, and none provided."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "x$TAR_NAME" == "x" ]; then
|
||||
for SPEC in `find $SPECS_BASE -name '*.spec' | sort -V`; do
|
||||
SPEC_PATH="$SPEC"
|
||||
|
||||
SERVICE=`spec_find_global service "$SPEC_PATH" 2>> /dev/null`
|
||||
if [ $? -eq 0 ]; then
|
||||
if [ "x$TAR_NAME" == "x" ]; then
|
||||
TAR_NAME=$SERVICE
|
||||
else
|
||||
if [ "x$SRC_DIR" != "x" ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): multiple spec files found, can't set TAR_NAME automatically"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
else
|
||||
NAME=`spec_find_tag Name "$SPEC_PATH" 2>> /dev/null`
|
||||
if [ $? -eq 0 ]; then
|
||||
if [ "x$TAR_NAME" == "x" ]; then
|
||||
TAR_NAME=$NAME
|
||||
else
|
||||
if [ "x$SRC_DIR" != "x" ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): multiple spec files found, can't set TAR_NAME automatically"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "WARNING: default_build_srpm (${LINENO}): 'Name' not found in '$SPEC_PATH'"
|
||||
NAME=""
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "x$TAR_NAME" == "x" ]; then
|
||||
if [ -f $SRC_DIR/PKG-INFO ]; then
|
||||
TAR_NAME=$(grep '^Name:' $SRC_DIR/PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "x$TAR_NAME" != "x" ]; then
|
||||
echo "Derived TAR_NAME=$TAR_NAME"
|
||||
else
|
||||
echo "ERROR: default_build_srpm (${LINENO}): Failed to derive a good TAR_NAME from SPEC file, and none provided."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "x$TAR" == "x" ]; then
|
||||
TAR="$TAR_NAME-$VERSION.tar.gz"
|
||||
fi
|
||||
|
||||
SOURCE_PATH="$BUILD_DIR/SOURCES"
|
||||
TAR_PATH="$SOURCE_PATH/$TAR"
|
||||
STAGING=""
|
||||
|
||||
if [ "x$COPY_LIST_TO_TAR" != "x" ] || [ "x$EXCLUDE_LIST_FROM_TAR" != "x" ]; then
|
||||
STAGING="$BUILD_DIR/staging"
|
||||
mkdir -p $STAGING
|
||||
fi
|
||||
|
||||
mkdir -p "$BUILD_DIR/SRPMS"
|
||||
mkdir -p "$SOURCE_PATH"
|
||||
|
||||
if [ "x$SRC_DIR" == "x" -a "x$COPY_LIST" == "x" -a "$ALLOW_EMPTY_RPM" != "true" ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): '$PWD/$DATA' failed to provide at least one of 'SRC_DIR' or 'COPY_LIST'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "x$SRC_DIR" != "x" ]; then
|
||||
if [ ! -d "$SRC_DIR" ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): directory not found: '$SRC_DIR'"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "x$COPY_LIST" != "x" ]; then
|
||||
echo "COPY_LIST: $COPY_LIST"
|
||||
for p in $COPY_LIST; do
|
||||
# echo "COPY_LIST: $p"
|
||||
\cp -L -u -r -v $p $SOURCE_PATH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): COPY_LIST: file not found: '$p'"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
if [ "x$STAGING" != "x" ]; then
|
||||
\cp -L -u -r -v $SRC_DIR $STAGING
|
||||
echo "COPY_LIST_TO_TAR: $COPY_LIST_TO_TAR"
|
||||
for p in $COPY_LIST_TO_TAR; do
|
||||
# echo "COPY_LIST_TO_TAR: $p"
|
||||
\cp -L -u -r -v $p $STAGING/$SRC_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): COPY_LIST_TO_TAR: file not found: '$p'"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
echo "EXCLUDE_LIST_FROM_TAR: $EXCLUDE_LIST_FROM_TAR"
|
||||
for p in $EXCLUDE_LIST_FROM_TAR; do
|
||||
# echo "EXCLUDE_LIST_FROM_TAR: $p"
|
||||
echo "rm -rf $STAGING/$SRC_DIR/$p"
|
||||
\rm -rf $STAGING/$SRC_DIR/$p
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): EXCLUDE_LIST_FROM_TAR: could not remove file: '$p'"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
fi
|
||||
|
||||
TRANSFORM=`echo "$SRC_DIR" | sed 's/^\./\\./' | sed 's:^/::'`
|
||||
|
||||
if [ "x$STAGING" != "x" ]; then
|
||||
pushd $STAGING
|
||||
fi
|
||||
|
||||
TAR_NEEDED=0
|
||||
if [ "x$SRC_DIR" != "x" ]; then
|
||||
echo "SRC_DIR=$SRC_DIR"
|
||||
if [ -f $TAR_PATH ]; then
|
||||
n=`find . -cnewer $TAR_PATH -and ! -path './.git*' \
|
||||
-and ! -path './build/*' \
|
||||
-and ! -path './.pc/*' \
|
||||
-and ! -path './patches/*' \
|
||||
-and ! -path "./$DISTRO/*" \
|
||||
-and ! -path './pbr-*.egg/*' \
|
||||
| wc -l`
|
||||
if [ $n -gt 0 ]; then
|
||||
TAR_NEEDED=1
|
||||
fi
|
||||
else
|
||||
TAR_NEEDED=1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ $TAR_NEEDED -gt 0 ]; then
|
||||
echo "Creating tar file: $TAR_PATH ..."
|
||||
tar czf $TAR_PATH $SRC_DIR --exclude '.git*' --exclude 'build' --exclude='.pc' --exclude='patches' --exclude="$SRC_DIR/$DISTRO" --exclude='pbr-*.egg' --transform "s,^$TRANSFORM,$TAR_NAME-$VERSION,"
|
||||
if [ $? -ne 0 ]; then
|
||||
if [ "x$STAGING" != "x" ]; then
|
||||
popd
|
||||
fi
|
||||
|
||||
echo "ERROR: default_build_srpm (${LINENO}): failed to create tar file, cmd: tar czf $TAR_PATH $SRC_DIR --exclude '.git*' --exclude 'build' --exclude='.pc' --exclude='patches' --exclude="$SRC_DIR/$DISTRO" --exclude='pbr-*.egg' --transform \"s,^$TRANSFORM,$TAR_NAME-$VERSION,\""
|
||||
exit 1
|
||||
fi
|
||||
echo "Created tar file: $TAR_PATH"
|
||||
else
|
||||
echo "Tar file not needed."
|
||||
fi
|
||||
|
||||
if [ "x$STAGING" != "x" ]; then
|
||||
popd
|
||||
fi
|
||||
|
||||
if [ ! -d $BUILD_DIR/SPECS ]; then
|
||||
echo "Spec directory '$BUILD_DIR/SPECS' does not exist"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ $(ls -1 $BUILD_DIR/SPECS/*.spec | wc -l) -eq 0 ]; then
|
||||
echo "No spec files found in spec directory '$BUILD_DIR/SPECS'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
for SPEC in `ls -1 $BUILD_DIR/SPECS`; do
|
||||
SPEC_PATH="$BUILD_DIR/SPECS/$SPEC"
|
||||
RELEASE=`spec_find_tag Release "$SPEC_PATH" 2>> /dev/null`
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): 'Release' not found in '$SPEC_PATH'"
|
||||
fi
|
||||
NAME=`spec_find_tag Name "$SPEC_PATH" 2>> /dev/null`
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: default_build_srpm (${LINENO}): 'Name' not found in '$SPEC_PATH'"
|
||||
fi
|
||||
SRPM="$NAME-$VERSION-$RELEASE.src.rpm"
|
||||
SRPM_PATH="$BUILD_DIR/SRPMS/$SRPM"
|
||||
|
||||
spec_validate_tis_release $SPEC_PATH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "TIS Validation of $SPEC_PATH failed"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BUILD_NEEDED=0
|
||||
if [ -f $SRPM_PATH ]; then
|
||||
n=`find . -cnewer $SRPM_PATH | wc -l`
|
||||
if [ $n -gt 0 ]; then
|
||||
BUILD_NEEDED=1
|
||||
fi
|
||||
else
|
||||
BUILD_NEEDED=1
|
||||
fi
|
||||
|
||||
if [ $BUILD_NEEDED -gt 0 ]; then
|
||||
echo "SPEC file: $SPEC_PATH"
|
||||
echo "SRPM build directory: $BUILD_DIR"
|
||||
echo "TIS_PATCH_VER: $TIS_PATCH_VER"
|
||||
|
||||
sed -i -e "1 i%define _tis_build_type $BUILD_TYPE" $SPEC_PATH
|
||||
sed -i -e "1 i%define tis_patch_ver $TIS_PATCH_VER" $SPEC_PATH
|
||||
rpmbuild -bs $SPEC_PATH --define="%_topdir $BUILD_DIR" --undefine=dist --define="_tis_dist .tis"
|
||||
else
|
||||
echo "SRPM build not needed"
|
||||
fi
|
||||
done
|
||||
|
||||
|
||||
59
build-tools/find_klm
Executable file
59
build-tools/find_klm
Executable file
@ -0,0 +1,59 @@
|
||||
#!/bin/bash
|
||||
|
||||
for r in $(find $MY_WORKSPACE/*/rpmbuild/RPMS -name '*.rpm'); do
|
||||
f=$(basename $r)
|
||||
find $MY_WORKSPACE/export/dist/isolinux/Packages | grep $f >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
continue
|
||||
fi
|
||||
n=$(rpm -q --qf='%{NAME}\n' -p $r)
|
||||
d=$(dirname $r)
|
||||
# echo "f=$f"
|
||||
for f in $(rpm -q -p -l $r | grep '[.]ko$' | head -n 1); do
|
||||
FOUND=0
|
||||
s=$(rpm -q --info -p $r | grep 'Source RPM :' | awk -F: '{print $2}' | tr -d '[[:space:]]')
|
||||
NAME=$(rpm -q --qf='%{NAME}\n' -p $d/$s)
|
||||
# echo "NAME=$NAME"
|
||||
for s2 in $(find $MY_WORKSPACE/*/rpmbuild/SRPMS -name "$NAME-[0-9]*.src.rpm"); do
|
||||
NAME2=$(rpm -q --qf='%{NAME}\n' -p $s2)
|
||||
# echo "NAME2=$NAME2"
|
||||
if [ "${NAME}" == "${NAME2}" ]; then
|
||||
echo $NAME | grep '[-]rt' >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
echo $NAME
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
# SIMPLE_NAME=$(echo $NAME | sed 's#-kmod##' | sed 's#-kernel##' | sed 's#^kernel$#linux#' | sed 's#^kernel-rt$#linux-rt#')
|
||||
SIMPLE_NAME=$(echo $NAME | sed 's#^kernel$#linux#' | sed 's#^kernel-rt$#linux-rt#')
|
||||
# echo "SIMPLE_NAME=$SIMPLE_NAME"
|
||||
grep "[/]$SIMPLE_NAME$" $(for g in $(find $MY_REPO -type d -name .git); do d=$(dirname $g); find $d -name 'centos_pkg_dirs*'; done) >> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
echo $NAME
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
SIMPLE_NAME=$(echo $NAME | sed 's#-rt$##' )
|
||||
# echo "SIMPLE_NAME=$SIMPLE_NAME"
|
||||
grep "[/]$SIMPLE_NAME$" $(for g in $(find $MY_REPO -type d -name .git); do d=$(dirname $g); find $d -name 'centos_pkg_dirs*'; done) >> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
echo $SIMPLE_NAME
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
SIMPLE_NAME2=$(echo $SIMPLE_NAME | sed 's#-kmod##' )
|
||||
# echo "SIMPLE_NAME2=$SIMPLE_NAME2"
|
||||
grep "[/-]$SIMPLE_NAME2$" $(for g in $(find $MY_REPO -type d -name .git); do d=$(dirname $g); find $d -name 'centos_pkg_dirs*'; done) >> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
echo $SIMPLE_NAME
|
||||
FOUND=1
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
if [ $FOUND -eq 1 ]; then
|
||||
break
|
||||
fi
|
||||
done
|
||||
# done
|
||||
done | sort --unique
|
||||
54
build-tools/find_patched_srpms_needing_upgrade
Executable file
54
build-tools/find_patched_srpms_needing_upgrade
Executable file
@ -0,0 +1,54 @@
|
||||
#!/bin/bash
|
||||
|
||||
for f in `find $MY_REPO -name srpm_path`; do
|
||||
orig_line=`cat $f`
|
||||
first=`echo $orig_line | awk -F : '{print $1}'`
|
||||
orig_path="/import/mirrors/$orig_line"
|
||||
if [ "$first" == "mirror" ]; then
|
||||
orig_path="/import/mirrors/"$(echo $orig_line | awk -F : '{print $2}');
|
||||
fi
|
||||
if [ "$first" == "repo" ]; then
|
||||
orig_path="$MY_REPO/"$(echo $orig_line | awk -F : '{print $2}')
|
||||
continue
|
||||
fi
|
||||
|
||||
if [ ! -f $orig_path ]; then
|
||||
echo "ERROR: bad srpm path: '$orig_path' derived from '$f'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
orig_dir=$(dirname $orig_path)
|
||||
repodata_dir=$orig_dir/repodata
|
||||
if [ ! -d $repodata_dir ]; then
|
||||
repodata_dir=$orig_dir/../repodata
|
||||
if [ ! -d $repodata_dir ]; then
|
||||
repodata_dir=$orig_dir/../../repodata
|
||||
if [ ! -d $repodata_dir ]; then
|
||||
echo "ERROR: couldn't find repodata for '$orig_path'"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# echo "'$orig_path' -> '$repodata_dir'"
|
||||
name=$(rpm -q --queryformat '%{NAME}\n' -p $orig_path 2>> /dev/null)
|
||||
version=$(rpm -q --queryformat '%{VERSION}\n' -p $orig_path 2>> /dev/null)
|
||||
release=$(rpm -q --queryformat '%{RELEASE}\n' -p $orig_path 2>> /dev/null)
|
||||
orig_name=$(basename $orig_path)
|
||||
best_name="$orig_name"
|
||||
for n in `find $orig_dir -name $name-*`; do
|
||||
if [ "$n" != "$orig_path" ]; then
|
||||
new_name=$(rpm -q --queryformat '%{NAME}\n' -p $n)
|
||||
if [ "$name" == "$new_name" ]; then
|
||||
rpmdev-vercmp $(basename $n) $best_name >> /dev/null
|
||||
if [ $? -eq 11 ]; then
|
||||
best_name=$(basename $n)
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
if [ "$best_name" != "$orig_name" ]; then
|
||||
echo "$f: $orig_name ==> $best_name"
|
||||
fi
|
||||
done
|
||||
|
||||
517
build-tools/ip_report.py
Executable file
517
build-tools/ip_report.py
Executable file
@ -0,0 +1,517 @@
|
||||
#!/usr/bin/python
|
||||
|
||||
import csv
|
||||
import os
|
||||
import rpm
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import getopt
|
||||
|
||||
|
||||
class BinPackage(object):
|
||||
def __init__(self, path, ts):
|
||||
fdno = os.open(path, os.O_RDONLY)
|
||||
hdr = ts.hdrFromFdno(path)
|
||||
os.close(fdno)
|
||||
|
||||
self.source = hdr[rpm.RPMTAG_SOURCERPM]
|
||||
self.desc = hdr[rpm.RPMTAG_DESCRIPTION].replace('\n', ' ')
|
||||
self.dirname = os.path.dirname(path)
|
||||
self.filename = os.path.basename(path)
|
||||
self.path = path
|
||||
self.kernel_module = False
|
||||
self.name = hdr[rpm.RPMTAG_NAME]
|
||||
|
||||
# Does the package contain kernel modules?
|
||||
for filename in hdr[rpm.RPMTAG_BASENAMES]:
|
||||
assert isinstance(filename, basestring)
|
||||
if filename.endswith('.ko'):
|
||||
self.kernel_module = True
|
||||
break
|
||||
|
||||
|
||||
class SrcPackage(object):
|
||||
def __init__(self, path=None):
|
||||
self.bin_pkg = None
|
||||
self.original_src = None
|
||||
self.sha = 'SHA'
|
||||
if path is None:
|
||||
self.filename = None
|
||||
self.path = None
|
||||
else:
|
||||
self.filename = os.path.basename(path)
|
||||
self.path = path
|
||||
ts = rpm.TransactionSet()
|
||||
ts.setVSFlags(rpm._RPMVSF_NODIGESTS | rpm._RPMVSF_NOSIGNATURES)
|
||||
fdno = os.open(self.path, os.O_RDONLY)
|
||||
hdr = ts.hdrFromFdno(self.path)
|
||||
os.close(fdno)
|
||||
self.desc = hdr[rpm.RPMTAG_DESCRIPTION].replace('\n', ' ')
|
||||
self.version = hdr[rpm.RPMTAG_VERSION] + '-' + hdr[rpm.RPMTAG_RELEASE]
|
||||
self.licences = hdr[rpm.RPMTAG_LICENSE]
|
||||
self.name = hdr[rpm.RPMTAG_NAME]
|
||||
self.url = hdr[rpm.RPMTAG_URL]
|
||||
|
||||
self.modified = None
|
||||
self.kernel_module = False
|
||||
self.disclosed_by = 'Jason McKenna'
|
||||
self.shipped_as = 'Binary'
|
||||
self.origin = 'Unknown'
|
||||
self.notes = ''
|
||||
self.wrs = False
|
||||
|
||||
def __lt__(self, other):
|
||||
me = self.name.lower()
|
||||
them = other.name.lower()
|
||||
if me == them:
|
||||
return self.name < other.name
|
||||
else:
|
||||
return me < them
|
||||
|
||||
|
||||
class IPReport(object):
|
||||
__KNOWN_PATHS = [
|
||||
# CentOS 7.4
|
||||
['/import/mirrors/CentOS/7.4.1708/os/Source/SPackages',
|
||||
'http://vault.centos.org/7.4.1708/os/Source/SPackages'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.4.1708/updates/Source/SPackages',
|
||||
'http://vault.centos.org/7.4.1708/updates/Source/SPackages'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.4.1708/cloud/Source/openstack-newton/common',
|
||||
'http://vault.centos.org/7.4.1708/cloud/Source/openstack-newton/common'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.4.1708/cloud/Source/openstack-newton',
|
||||
'http://vault.centos.org/7.4.1708/cloud/Source/openstack-newton'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.4.1708/cloud/Source/openstack-mitaka/common',
|
||||
'http://vault.centos.org/7.4.1708/cloud/Source/openstack-mitaka/common'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.4.1708/cloud/Source/openstack-mitaka',
|
||||
'http://vault.centos.org/7.4.1708/cloud/Source/openstack-mitaka'],
|
||||
['/import/mirrors/CentOS/7.4.1708/extras/Source/SPackages',
|
||||
'http://vault.centos.org/7.4.1708/extras/Source/SPackages'],
|
||||
# CentOS 7.3
|
||||
['/import/mirrors/CentOS/7.3.1611/os/Source/SPackages',
|
||||
'http://vault.centos.org/7.3.1611/os/Source/SPackages'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.3.1611/updates/Source/SPackages',
|
||||
'http://vault.centos.org/7.3.1611/updates/Source/SPackages'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.3.1611/cloud/Source/openstack-newton/common',
|
||||
'http://vault.centos.org/7.3.1611/cloud/Source/openstack-newton/common'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.3.1611/cloud/Source/openstack-newton',
|
||||
'http://vault.centos.org/7.3.1611/cloud/Source/openstack-newton'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.3.1611/cloud/Source/openstack-mitaka/common',
|
||||
'http://vault.centos.org/7.3.1611/cloud/Source/openstack-mitaka/common'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.3.1611/cloud/Source/openstack-mitaka',
|
||||
'http://vault.centos.org/7.3.1611/cloud/Source/openstack-mitaka'],
|
||||
['/import/mirrors/CentOS/7.3.1611/extras/Source/SPackages',
|
||||
'http://vault.centos.org/7.3.1611/extras/Source/SPackages'],
|
||||
# CentOS 7.2
|
||||
['/import/mirrors/CentOS/7.2.1511/os/Source/SPackages', 'http://vault.centos.org/7.2.1511/os/Source/SPackages'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.2.1511/updates/Source/SPackages',
|
||||
'http://vault.centos.org/7.2.1511/updates/Source/SPackages'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.2.1511/cloud/Source/openstack-mitaka/common',
|
||||
'http://vault.centos.org/7.2.1511/cloud/Source/openstack-mitaka/common'],
|
||||
['/import/mirrors/CentOS/vault.centos.org/7.2.1511/cloud/Source/openstack-mitaka',
|
||||
'http://vault.centos.org/7.2.1511/cloud/Source/openstack-mitaka'],
|
||||
['/import/mirrors/CentOS/7.2.1511/extras/Source/SPackages',
|
||||
'http://vault.centos.org/7.2.1511/extras/Source/SPackages'],
|
||||
['/import/mirrors/CentOS/tis-r4-CentOS/newton/Source', 'Unknown'],
|
||||
['/import/mirrors/CentOS/tis-r4-CentOS/tis-r4-3rd-Party', 'Unknown']
|
||||
|
||||
]
|
||||
|
||||
def __init__(self, workspace=None, repo=None):
|
||||
self.workspace = None
|
||||
self.repo = None
|
||||
self.shipped_binaries = list()
|
||||
self.built_binaries = list()
|
||||
self.check_env()
|
||||
if workspace is not None:
|
||||
self.workspace = workspace
|
||||
if repo is not None:
|
||||
self.repo = repo
|
||||
|
||||
# Generate a list of binaries that we shipped
|
||||
for filename in os.listdir(self.workspace + '/export/dist/isolinux/Packages'):
|
||||
if filename.endswith('rpm'):
|
||||
self.shipped_binaries.append(filename)
|
||||
|
||||
# Generate a list of binaries that we built ourselves
|
||||
for build in ['rt', 'std']:
|
||||
for filename in os.listdir(self.workspace + '/' + build + '/rpmbuild/RPMS/'):
|
||||
if filename.endswith('rpm'):
|
||||
self.built_binaries.append(filename)
|
||||
|
||||
print ('Looking up packages for which we have source...')
|
||||
self.original_src_pkgs = dict()
|
||||
self.build_original_src_pkgs()
|
||||
print ('Looking up packages we built...')
|
||||
self.built_src_pkgs = dict()
|
||||
self.build_built_src_pkgs()
|
||||
print ('Looking up packages we built...')
|
||||
self.hardcoded_lookup_dict = dict()
|
||||
self.build_hardcoded_lookup_dict()
|
||||
|
||||
def build_hardcoded_lookup_dict(self):
|
||||
with open(self.repo + '/build-tools/source_lookup.txt', 'r') as lookup_file:
|
||||
for line in lookup_file:
|
||||
line = line.rstrip()
|
||||
words = line.split()
|
||||
if (words is not None) and (len(words) >= 2):
|
||||
self.hardcoded_lookup_dict[words[1]] = (words[0], False)
|
||||
|
||||
with open(self.repo + '/build-tools/wrs_orig.txt', 'r') as lookup_file:
|
||||
for line in lookup_file:
|
||||
line = line.rstrip()
|
||||
words = line.split()
|
||||
if (words is not None) and (len(words) >= 1):
|
||||
self.hardcoded_lookup_dict[words[0]] = ('No download', True)
|
||||
|
||||
@staticmethod
|
||||
def path_to_origin(filepath):
|
||||
for path in IPReport.__KNOWN_PATHS:
|
||||
if filepath.startswith(path[0]) and (not path[1].lower().startswith('unknown')):
|
||||
return path[1] + '/' + os.path.basename(filepath)
|
||||
return 'Unknown'
|
||||
|
||||
def hardcoded_lookup(self, package_name):
|
||||
if package_name in self.hardcoded_lookup_dict.keys():
|
||||
return self.hardcoded_lookup_dict[package_name]
|
||||
return None, False
|
||||
|
||||
def check_env(self):
|
||||
if 'MY_WORKSPACE' in os.environ:
|
||||
self.workspace = os.environ['MY_WORKSPACE']
|
||||
else:
|
||||
print 'Could not find $MY_WORKSPACE'
|
||||
raise IOError('Could not fine $MY_WORKSPACE')
|
||||
|
||||
if 'MY_REPO' in os.environ:
|
||||
self.repo = os.environ['MY_REPO']
|
||||
else:
|
||||
print 'Could not find $MY_REPO'
|
||||
raise IOError('Could not fine $MY_REPO')
|
||||
|
||||
def do_bin_pkgs(self):
|
||||
print ('Gathering binary package information')
|
||||
self.read_bin_pkgs()
|
||||
|
||||
def read_bin_pkgs(self):
|
||||
self.bin_pkgs = list()
|
||||
ts = rpm.TransactionSet()
|
||||
ts.setVSFlags(rpm._RPMVSF_NODIGESTS | rpm._RPMVSF_NOSIGNATURES)
|
||||
for filename in self.shipped_binaries:
|
||||
if filename.endswith('rpm'):
|
||||
bin_pkg = BinPackage(self.workspace + '/export/dist/isolinux/Packages/' + filename, ts)
|
||||
self.bin_pkgs.append(bin_pkg)
|
||||
|
||||
def do_src_report(self, copy_packages=False, do_wrs=True, delta_file=None, output_path=None, strip_unchanged=False):
|
||||
self.bin_to_src()
|
||||
self.src_pkgs.sort()
|
||||
|
||||
if delta_file is not None:
|
||||
self.delta(delta_file)
|
||||
|
||||
if output_path is None:
|
||||
output_path = self.workspace + '/export/ip_report'
|
||||
|
||||
# Create output dir (if required)
|
||||
if not os.path.exists(output_path):
|
||||
os.makedirs(output_path)
|
||||
|
||||
# Create paths for RPMs (if required)
|
||||
if copy_packages:
|
||||
if not os.path.exists(output_path + '/non_wrs'):
|
||||
shutil.rmtree(output_path + '/non_wrs', True)
|
||||
os.makedirs(output_path + '/non_wrs')
|
||||
if do_wrs:
|
||||
shutil.rmtree(output_path + '/wrs', True)
|
||||
os.makedirs(output_path + '/wrs')
|
||||
|
||||
with open(output_path + '/srcreport.csv', 'wb') as src_report_file:
|
||||
src_report_writer = csv.writer(src_report_file)
|
||||
|
||||
# Write header row
|
||||
src_report_writer.writerow(
|
||||
['Package File', 'File Name', 'Package Name', 'Version', 'SHA1', 'Disclosed By',
|
||||
'Description', 'Part Of (Runtime, Host, Both)', 'Modified (Yes, No)', 'Hardware Interfacing (Yes, No)',
|
||||
'License(s) Found', 'Package Download URL', 'Kernel module', 'Notes'])
|
||||
|
||||
for src_pkg in self.src_pkgs:
|
||||
if src_pkg.modified:
|
||||
modified_string = 'Yes'
|
||||
else:
|
||||
modified_string = 'No'
|
||||
if src_pkg.kernel_module:
|
||||
kmod_string = 'Yes'
|
||||
else:
|
||||
kmod_string = 'No'
|
||||
|
||||
# Copy the pacakge and get the SHA
|
||||
if copy_packages:
|
||||
if src_pkg.wrs is False:
|
||||
shutil.copyfile(src_pkg.path, output_path + '/non_wrs/' + src_pkg.filename)
|
||||
shasumout = subprocess.check_output(
|
||||
['shasum', output_path + '/non_wrs/' + src_pkg.filename]).split()[0]
|
||||
src_pkg.sha = shasumout
|
||||
if strip_unchanged and (src_pkg.notes.lower().startswith('unchanged')):
|
||||
os.remove(output_path + '/non_wrs/' + src_pkg.filename)
|
||||
else:
|
||||
if do_wrs:
|
||||
shutil.copyfile(src_pkg.path, output_path + '/wrs/' + src_pkg.filename)
|
||||
shasumout = subprocess.check_output(
|
||||
['shasum', output_path + '/wrs/' + src_pkg.filename]).split()[0]
|
||||
src_pkg.sha = shasumout
|
||||
if strip_unchanged and (src_pkg.notes.lower().startswith('unchanged')):
|
||||
os.remove(output_path + '/wrs/' + src_pkg.filename)
|
||||
|
||||
if do_wrs or (src_pkg.wrs is False):
|
||||
src_report_writer.writerow(
|
||||
[src_pkg.filename, src_pkg.name, src_pkg.version, src_pkg.sha, src_pkg.disclosed_by,
|
||||
src_pkg.desc, 'Runtime', src_pkg.shipped_as, modified_string, 'No', src_pkg.licences,
|
||||
src_pkg.origin, kmod_string, src_pkg.notes])
|
||||
if 'unknown' in src_pkg.origin.lower():
|
||||
print (
|
||||
'Warning: Could not determine origin of ' + src_pkg.name + '. Please investigate/populate manually')
|
||||
|
||||
def bin_to_src(self):
|
||||
self.src_pkgs = list()
|
||||
src_pkg_names = list()
|
||||
for bin_pkg in self.bin_pkgs:
|
||||
if src_pkg_names.__contains__(bin_pkg.source):
|
||||
if bin_pkg.kernel_module:
|
||||
for src_pkg in self.src_pkgs:
|
||||
if src_pkg.filename == bin_pkg.source:
|
||||
src_pkg.kernel_module = True
|
||||
break
|
||||
|
||||
continue
|
||||
|
||||
# if we reach here, then the source package is not yet in our db.
|
||||
# we first search for the source package in the built-rpms
|
||||
if 'shim-signed' in bin_pkg.source:
|
||||
for tmp in self.built_src_pkgs:
|
||||
if 'shim-signed' in tmp:
|
||||
print ('shim-signed hack -- ' + bin_pkg.source + ' to ' + tmp)
|
||||
bin_pkg.source = tmp
|
||||
break
|
||||
if 'shim-unsigned' in bin_pkg.source:
|
||||
for tmp in self.built_src_pkgs:
|
||||
if 'shim-0' in tmp:
|
||||
print ('shim-unsigned hack -- ' + bin_pkg.source + ' to ' + tmp)
|
||||
bin_pkg.source = tmp
|
||||
break
|
||||
if 'grub2-efi-pxeboot' in bin_pkg.source:
|
||||
for tmp in self.built_src_pkgs:
|
||||
if 'grub2-2' in tmp:
|
||||
print ('grub2-efi-pxeboot hack -- ' + bin_pkg.source + ' to ' + tmp)
|
||||
bin_pkg.source = tmp
|
||||
break
|
||||
|
||||
if bin_pkg.source in self.built_src_pkgs:
|
||||
src_pkg = self.built_src_pkgs[bin_pkg.source]
|
||||
src_pkg.modified = True
|
||||
|
||||
# First guess, we see if there's an original source with the source package name
|
||||
# (this is 99% of the cases)
|
||||
src_pkg_orig_name = src_pkg.name
|
||||
if src_pkg_orig_name in self.original_src_pkgs:
|
||||
src_pkg.original_src = self.original_src_pkgs[src_pkg_orig_name]
|
||||
src_pkg.origin = src_pkg.original_src.origin
|
||||
|
||||
else:
|
||||
src_pkg_path = self.locate_in_mirror(bin_pkg.source)
|
||||
if not os.path.isabs(src_pkg_path):
|
||||
continue
|
||||
src_pkg = SrcPackage(src_pkg_path)
|
||||
src_pkg.origin = IPReport.path_to_origin(src_pkg_path)
|
||||
src_pkg.modified = False
|
||||
|
||||
if bin_pkg.kernel_module:
|
||||
src_pkg.kernel_module = True
|
||||
|
||||
src_pkg_names.append(bin_pkg.source)
|
||||
self.src_pkgs.append(src_pkg)
|
||||
|
||||
if src_pkg.origin.lower() == 'unknown':
|
||||
if 'windriver' in src_pkg.licences.lower():
|
||||
src_pkg.origin = 'No download'
|
||||
else:
|
||||
if src_pkg.url is not None:
|
||||
src_pkg.origin = src_pkg.url
|
||||
|
||||
if 'unknown' in src_pkg.origin.lower():
|
||||
(orig, is_wrs) = self.hardcoded_lookup(src_pkg.name)
|
||||
if orig is not None:
|
||||
src_pkg.origin = orig
|
||||
src_pkg.wrs = is_wrs
|
||||
|
||||
if (src_pkg.origin.lower() == 'no download') and ('windriver' in src_pkg.licences.lower()):
|
||||
src_pkg.wrs = True
|
||||
|
||||
def locate_in_mirror(self, filename):
|
||||
""" takes an RPM filename and finds the full path of the file """
|
||||
|
||||
fullpath = None
|
||||
|
||||
filename = filename.replace('mirror:', self.repo + '/cgcs-centos-repo/')
|
||||
filename = filename.replace('repo:', self.repo + '/')
|
||||
filename = filename.replace('3rd_party:', self.repo + '/cgcs-3rd-party-repo/')
|
||||
|
||||
# At this point, filename could be a complete path (incl symlink), or just a filename
|
||||
best_guess = filename
|
||||
filename = os.path.basename(filename)
|
||||
|
||||
for path in IPReport.__KNOWN_PATHS:
|
||||
if os.path.exists(path[0] + '/' + filename):
|
||||
fullpath = path[0] + '/' + filename
|
||||
break
|
||||
|
||||
if fullpath is not None:
|
||||
return fullpath
|
||||
else:
|
||||
return best_guess
|
||||
|
||||
def build_original_src_pkgs(self):
|
||||
for root, dirs, files in os.walk(self.repo):
|
||||
for name in files:
|
||||
if name == 'srpm_path':
|
||||
with open(os.path.join(root, 'srpm_path'), 'r') as srpm_path_file:
|
||||
original_srpm_file = srpm_path_file.readline().rstrip()
|
||||
original_src_pkg_path = self.locate_in_mirror(original_srpm_file)
|
||||
original_src_pkg = SrcPackage(original_src_pkg_path)
|
||||
original_src_pkg.origin = IPReport.path_to_origin(original_src_pkg_path)
|
||||
self.original_src_pkgs[original_src_pkg.name] = original_src_pkg
|
||||
|
||||
def build_built_src_pkgs(self):
|
||||
""" Create a dict of any source package that we built ourselves """
|
||||
for build in ['std', 'rt']:
|
||||
for root, dirs, files in os.walk(self.workspace + '/' + build + '/rpmbuild/SRPMS'):
|
||||
for name in files:
|
||||
if name.endswith('.src.rpm'):
|
||||
built_src_pkg = SrcPackage(os.path.join(root, name))
|
||||
self.built_src_pkgs[built_src_pkg.filename] = built_src_pkg
|
||||
|
||||
def delta(self, orig_report):
|
||||
if orig_report is None:
|
||||
return
|
||||
delta_src_pkgs = self.read_last_report(orig_report)
|
||||
|
||||
for pkg in self.src_pkgs:
|
||||
if pkg.name in delta_src_pkgs:
|
||||
old_pkg = delta_src_pkgs[pkg.name]
|
||||
if old_pkg.version == pkg.version:
|
||||
pkg.notes = 'Unchanged'
|
||||
else:
|
||||
pkg.notes = 'New version'
|
||||
else:
|
||||
pkg.notes = 'New package'
|
||||
|
||||
def read_last_report(self, orig_report):
|
||||
orig_pkg_dict = dict()
|
||||
with open(orig_report, 'rb') as orig_report_file:
|
||||
orig_report_reader = csv.reader(orig_report_file)
|
||||
doneHeader = False
|
||||
for row in orig_report_reader:
|
||||
if (not doneHeader) and ('package file name' in row[0].lower()):
|
||||
doneHeader = True
|
||||
continue
|
||||
doneHeader = True
|
||||
orig_pkg = SrcPackage()
|
||||
orig_pkg.filename = row[0]
|
||||
orig_pkg.name = row[1]
|
||||
orig_pkg.version = row[2]
|
||||
# sha = row[3]
|
||||
orig_pkg.disclosed_by = row[4]
|
||||
orig_pkg.desc = row[5]
|
||||
# runtime = row[6]
|
||||
orig_pkg.shipped_as = row[7]
|
||||
if row[8].lower is 'yes':
|
||||
orig_pkg.modified = True
|
||||
else:
|
||||
orig_pkg.modifed = False
|
||||
# hardware interfacing = row[9]
|
||||
orig_pkg.licences = row[10]
|
||||
orig_pkg.origin = row[11]
|
||||
if row[12].lower is 'yes':
|
||||
orig_pkg.kernel_module = True
|
||||
else:
|
||||
orig_pkg.kernel_module = False
|
||||
orig_pkg_dict[orig_pkg.name] = orig_pkg
|
||||
|
||||
return orig_pkg_dict
|
||||
|
||||
|
||||
def main(argv):
|
||||
# handle command line arguments
|
||||
# -h/--help -- help
|
||||
# -n/--no-copy -- do not copy files (saves time)
|
||||
# -d/--delta= -- compare with an ealier report
|
||||
# -o/--output= -- output report/binaries to specified path
|
||||
# -w/--workspace= -- use specified workspace instead of $WORKSPACE
|
||||
# -r/--repo= -- use sepeciied repo instead of $MY_REPO
|
||||
# -s -- strip (remove) unchanged packages from copy out directory
|
||||
|
||||
try:
|
||||
opts, args = getopt.getopt(argv, "hnd:o:w:r:s",
|
||||
["delta=", "help", "no-copy", "workspace=", "repo=", "output=", "--strip"])
|
||||
except getopt.GetoptError:
|
||||
# todo - output help
|
||||
sys.exit(2)
|
||||
delta_file = None
|
||||
do_copy = True
|
||||
workspace = None
|
||||
repo = None
|
||||
output_path = None
|
||||
strip_unchanged = False
|
||||
|
||||
for opt, arg in opts:
|
||||
if opt in ('-h', '--help'):
|
||||
print 'usage:'
|
||||
print ' ip_report.py [options]'
|
||||
print ' Creates and IP report in $MY_WORKSPACE/export/ip_report '
|
||||
print ' Source RPMs (both Wind River and non WR) are placed in subdirs within that path'
|
||||
print ''
|
||||
print 'Options:'
|
||||
print ' -h/--help - this help'
|
||||
print ' -d <file>/--delta=<file> - create "notes" field, comparing report with a previous report'
|
||||
print ' -n/--no-copy - do not copy files into subdirs (this is faster, but means you'
|
||||
print ' don\'t get SHA sums for files)'
|
||||
print ' -w <path>/--workspace=<path> - use the specified path as workspace, instead of $MY_WORKSPACE'
|
||||
print ' -r <path>/--repo=<path> - use the specified path as repo, instead of $MY_REPO'
|
||||
print ' -o <path>/--output=<path> - output to specified path (instead of $MY_WORKSPACE/export/ip_report)'
|
||||
print ' -s/--strip - strip (remove) unchanged files if copied'
|
||||
exit()
|
||||
elif opt in ('-d', '--delta'):
|
||||
delta_file = os.path.normpath(arg)
|
||||
delta_file = os.path.expanduser(delta_file)
|
||||
if not os.path.exists(delta_file):
|
||||
print 'Cannot locate ' + delta_file
|
||||
exit(1)
|
||||
elif opt in ('-w', '--workspace'):
|
||||
workspace = os.path.normpath(arg)
|
||||
workspace = os.path.expanduser(workspace)
|
||||
elif opt in ('-r', '--repo'):
|
||||
repo = os.path.normpath(arg)
|
||||
repo = os.path.expanduser(repo)
|
||||
elif opt in ('-o', '--output'):
|
||||
output_path = os.path.normpath(arg)
|
||||
output_path = os.path.expanduser(output_path)
|
||||
elif opt in ('-n', '--no-copy'):
|
||||
do_copy = False
|
||||
elif opt in ('-s', '--strip-unchanged'):
|
||||
strip_unchanged = True
|
||||
|
||||
print ('Doing IP report')
|
||||
if delta_file is not None:
|
||||
print 'Delta from ' + delta_file
|
||||
else:
|
||||
print 'No delta specified'
|
||||
ip_report = IPReport(workspace=workspace, repo=repo)
|
||||
|
||||
ip_report.do_bin_pkgs()
|
||||
ip_report.do_src_report(copy_packages=do_copy,
|
||||
delta_file=delta_file,
|
||||
output_path=output_path,
|
||||
strip_unchanged=strip_unchanged)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv[1:])
|
||||
244
build-tools/make-installer-images.sh
Executable file
244
build-tools/make-installer-images.sh
Executable file
@ -0,0 +1,244 @@
|
||||
#!/bin/bash -e
|
||||
## this script is called by "update-pxe-network-installer" and run in "sudo"
|
||||
## created by Yong Hu (yong.hu@intel.com), 05/24/2018
|
||||
|
||||
function clean_rootfs() {
|
||||
rootfs_dir=$1
|
||||
echo "--> remove old files in original rootfs"
|
||||
conf="$(ls ${rootfs_dir}/etc/ld.so.conf.d/kernel-*.conf)"
|
||||
echo "conf basename = $(basename $conf)"
|
||||
old_version="tbd"
|
||||
if [ -f $conf ];then
|
||||
old_version="$(echo $(basename $conf) | rev | cut -d'.' -f2- | rev | cut -d'-' -f2-)"
|
||||
fi
|
||||
echo "old version is $old_version"
|
||||
# remove old files in original initrd.img
|
||||
# do this in chroot to avoid accidentialy wrong operations on host root
|
||||
chroot $rootfs_dir /bin/bash -x <<EOF
|
||||
rm -rf ./boot/ ./etc/modules-load.d/
|
||||
if [ -n $old_version ] && [ -f ./etc/ld.so.conf.d/kernel-${old_version}.conf ]; then
|
||||
rm -rf ./etc/ld.so.conf.d/kernel-${old_version}.conf
|
||||
rm -rf ./lib/modules/${old_version}
|
||||
fi
|
||||
if [ -d ./usr/lib64/python2.7/site-packages/pyanaconda/ ];then
|
||||
rm -rf usr/lib64/python2.7/site-packages/pyanaconda/
|
||||
fi
|
||||
if [ -d ./usr/lib64/python2.7/site-packages/rpm/ ];then
|
||||
rm -rf usr/lib64/python2.7/site-packages/rpm/
|
||||
fi
|
||||
#find old .pyo files and delete them
|
||||
all_pyo="`find ./usr/lib64/python2.7/site-packages/pyanaconda/ usr/lib64/python2.7/site-packages/rpm/ -name *.pyo`"
|
||||
if [ -n $all ]; then
|
||||
for pyo in $all_pyo;do
|
||||
rm -f $pyo
|
||||
done
|
||||
fi
|
||||
exit
|
||||
EOF
|
||||
#back to previous folder
|
||||
}
|
||||
|
||||
|
||||
echo "This script makes new initrd.img, vmlinuz and squashfs.img."
|
||||
echo "NOTE: it has to be executed with *root*!"
|
||||
|
||||
if [ $# -lt 2 ];then
|
||||
echo "$0 <work_dir> <kernel_mode>"
|
||||
echo "kernel_mode: std or rt"
|
||||
exit -1;
|
||||
fi
|
||||
|
||||
work_dir=$1
|
||||
mode=$2
|
||||
output_dir=$work_dir/output
|
||||
if [ ! -d $output_dir ];then mkdir -p $output_dir; fi
|
||||
|
||||
if [ "$mode" != "std" ] && [ "$mode" != "rt" ]; then
|
||||
echo "ERROR: wrong kernel mode, must be std or rt"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
timestamp=$(date +%F_%H%M)
|
||||
|
||||
echo "---------------- start to make new initrd.img and vmlinuz -------------"
|
||||
ORIG_INITRD=$work_dir/orig/initrd.img
|
||||
if [ ! -f $ORIG_INITRD ];then
|
||||
echo "ERROR: $ORIG_INITRD does NOT exist!"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
kernel_rpms_dir=$work_dir/kernel-rpms
|
||||
if [ ! -d $kernel_rpms_dir ];then
|
||||
echo "ERROR: $kernel_rpms_dir does NOT exist!"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
initrd_root=$work_dir/initrd.work
|
||||
if [ -d $initrd_root ];then
|
||||
rm -rf $initrd_root
|
||||
fi
|
||||
mkdir -p $initrd_root
|
||||
|
||||
cd $initrd_root
|
||||
# uncompress initrd.img
|
||||
echo "--> uncompress original initrd.img"
|
||||
/usr/bin/xzcat $ORIG_INITRD | cpio -i
|
||||
|
||||
echo "--> clean up $initrd_root"
|
||||
clean_rootfs $initrd_root
|
||||
|
||||
echo "--> extract files from new kernel and its modular rpms to initrd root"
|
||||
for kf in $kernel_rpms_dir/$mode/*.rpm ; do rpm2cpio $kf | cpio -idu; done
|
||||
|
||||
# by now new kernel and its modules exist!
|
||||
# find new kernel in /boot/
|
||||
echo "--> get new kernel image: vmlinuz"
|
||||
new_kernel="$(ls ./boot/vmlinuz-*)"
|
||||
echo $new_kernel
|
||||
if [ -f $new_kernel ];then
|
||||
#copy out the new kernel
|
||||
if [ $mode == "std" ];then
|
||||
if [ -f $output_dir/new-vmlinuz ]; then
|
||||
mv -f $output_dir/new-vmlinuz $output_dir/vmlinuz-bakcup-$timestamp
|
||||
fi
|
||||
cp -f $new_kernel $output_dir/new-vmlinuz
|
||||
else
|
||||
if [ -f $output_dir/new-vmlinuz-rt ]; then
|
||||
mv -f $output_dir/new-vmlinuz-rt $output_dir/vmlinuz-rt-bakcup-$timestamp
|
||||
fi
|
||||
cp -f $new_kernel $output_dir/new-vmlinuz-rt
|
||||
fi
|
||||
kernel_name=$(basename $new_kernel)
|
||||
new_ver=$(echo $kernel_name | cut -d'-' -f2-)
|
||||
echo $new_ver
|
||||
else
|
||||
echo "ERROR: new kernel is NOT found!"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
echo "-->check module dependencies in new initrd.img in chroot context"
|
||||
chroot $initrd_root /bin/bash -x <<EOF
|
||||
/usr/sbin/depmod -aeF "/boot/System.map-$new_ver" "$new_ver"
|
||||
if [ $? == 0 ]; then echo "module dependencies are satisfied!" ; fi
|
||||
## Remove the bisodevname package!
|
||||
rm -f ./usr/lib/udev/rules.d/71-biosdevname.rules ./usr/sbin/biosdevname
|
||||
exit
|
||||
EOF
|
||||
|
||||
echo "--> Rebuild the initrd"
|
||||
if [ -f $output_dir/new-initrd.img ]; then
|
||||
mv -f $output_dir/new-initrd.img $output_dir/initrd.img-bakcup-$timestamp
|
||||
fi
|
||||
find . | cpio -o -H newc | xz --check=crc32 --x86 --lzma2=dict=512KiB > $output_dir/new-initrd.img
|
||||
if [ $? != 0 ];then
|
||||
echo "ERROR: failed to create new initrd.img"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
cd $work_dir
|
||||
|
||||
if [ -f $output_dir/new-initrd.img ];then
|
||||
ls -l $output_dir/new-initrd.img
|
||||
else
|
||||
echo "ERROR: new-initrd.img is not generated!"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
if [ -f $output_dir/new-vmlinuz ];then
|
||||
ls -l $output_dir/new-vmlinuz
|
||||
else
|
||||
echo "ERROR: new-vmlinuz is not generated!"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
echo "---------------- start to make new squashfs.img -------------"
|
||||
ORIG_SQUASHFS=$work_dir/orig/squashfs.img
|
||||
if [ ! -f $ORIG_SQUASHFS ];then
|
||||
echo "ERROR: $ORIG_SQUASHFS does NOT exist!"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
rootfs_rpms_dir=$work_dir/rootfs-rpms
|
||||
if [ ! -d $rootfs_rpms_dir ];then
|
||||
echo "ERROR: $rootfs_rpms_dir does NOT exist!"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
# make squashfs.mnt and ready and umounted
|
||||
if [ ! -d $work_dir/squashfs.mnt ];then
|
||||
mkdir -p $work_dir/squashfs.mnt
|
||||
else
|
||||
# in case it was mounted previously
|
||||
mnt_path=$(mount | grep "squashfs.mnt" | cut -d' ' -f3-3)
|
||||
if [ x"$mnt_path" != "x" ] && [ "$(basename $mnt_path)" == "squashfs.mnt" ];then
|
||||
umount $work_dir/squashfs.mnt
|
||||
fi
|
||||
fi
|
||||
|
||||
# make squashfs.work ready and umounted
|
||||
squashfs_root="$work_dir/squashfs.work"
|
||||
# Now mount the rootfs.img file:
|
||||
if [ ! -d $squashfs_root ];then
|
||||
mkdir -p $squashfs_root
|
||||
else
|
||||
# in case it was mounted previously
|
||||
mnt_path=$(mount | grep "$(basename $squashfs_root)" | cut -d' ' -f3-3)
|
||||
if [ x"$mnt_path" != "x" ] && [ "$(basename $mnt_path)" == "$(basename $squashfs_root)" ];then
|
||||
umount $squashfs_root
|
||||
fi
|
||||
fi
|
||||
|
||||
echo $ORIG_SQUASHFS
|
||||
mount -o loop -t squashfs $ORIG_SQUASHFS $work_dir/squashfs.mnt
|
||||
|
||||
if [ ! -d ./LiveOS ];then mkdir -p ./LiveOS ; fi
|
||||
|
||||
echo "--> copy rootfs.img from original squashfs.img to LiveOS folder"
|
||||
cp -f ./squashfs.mnt/LiveOS/rootfs.img ./LiveOS/.
|
||||
|
||||
echo "--> done to copy rootfs.img, umount squashfs.mnt"
|
||||
umount ./squashfs.mnt
|
||||
|
||||
echo "--> mount rootfs.img into $squashfs_root"
|
||||
mount -o loop LiveOS/rootfs.img $squashfs_root
|
||||
|
||||
echo "--> clean up ./squashfs-rootfs from original squashfs.img in chroot context"
|
||||
clean_rootfs $squashfs_root
|
||||
|
||||
cd $squashfs_root
|
||||
echo "--> extract files from rootfs-rpms to squashfs root"
|
||||
for ff in $rootfs_rpms_dir/*.rpm ; do rpm2cpio $ff | cpio -idu; done
|
||||
|
||||
echo "--> extract files from kernel and its modular rpms to squashfs root"
|
||||
for kf in $kernel_rpms_dir/$mode/*.rpm ; do rpm2cpio $kf | cpio -idu; done
|
||||
|
||||
echo "-->check module dependencies in new squashfs.img in chroot context"
|
||||
#we are using the same new kernel-xxx.rpm, so the $new_ver is the same
|
||||
chroot $squashfs_root /bin/bash -x <<EOF
|
||||
/usr/sbin/depmod -aeF "/boot/System.map-$new_ver" "$new_ver"
|
||||
if [ $? == 0 ]; then echo "module dependencies are satisfied!" ; fi
|
||||
## Remove the bisodevname package!
|
||||
rm -f ./usr/lib/udev/rules.d/71-biosdevname.rules ./usr/sbin/biosdevname
|
||||
exit
|
||||
EOF
|
||||
|
||||
# come back to the original work dir
|
||||
cd $work_dir
|
||||
|
||||
echo "--> unmount $squashfs_root"
|
||||
umount $squashfs_root
|
||||
#rename the old version
|
||||
if [ -f $output_dir/new-squashfs.img ]; then
|
||||
mv -f $output_dir/new-squashfs.img $output_dir/squashfs.img-backup-$timestamp
|
||||
fi
|
||||
|
||||
echo "--> make the new squashfs image"
|
||||
mksquashfs LiveOS $output_dir/new-squashfs.img -keep-as-directory -comp xz -b 1M
|
||||
if [ $? == 0 ];then
|
||||
ls -l $output_dir/new-squashfs.img
|
||||
else
|
||||
echo "ERROR: failed to make a new squashfs.img"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
echo "--> done successfully!"
|
||||
229
build-tools/mirror_rebase/copy_external_mirror_to_tis_mirror
Normal file
229
build-tools/mirror_rebase/copy_external_mirror_to_tis_mirror
Normal file
@ -0,0 +1,229 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ "$USER" != "jenkins" ]; then
|
||||
echo "ERROR: only jenkins should run this"
|
||||
return 1
|
||||
fi
|
||||
|
||||
CENTOS_BASE_VERSION=7.2.1511
|
||||
EPEL_BASE_VERSION=7
|
||||
BIN_ARCH=x86_64
|
||||
|
||||
BIN_DIRS="/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/updates/x86_64
|
||||
/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/cloud/x86_64/openstack-mitaka
|
||||
/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/os/x86_64
|
||||
/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/extras/x86_64
|
||||
/export/jenkins/mirrors/fedora/epel/$EPEL_BASE_VERSION/x86_64
|
||||
"
|
||||
SRC_DIRS="/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/updates/Source
|
||||
/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/cloud/Source/openstack-mitaka
|
||||
/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/os/Source
|
||||
/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/extras/Source
|
||||
/export/jenkins/mirrors/fedora/dl.fedoraproject.org/pub/epel/$EPEL_BASE_VERSION/SRPMS
|
||||
"
|
||||
TESTING_BIN_DIRS="/export/jenkins/mirrors/fedora/epel/testing/7/x86_64
|
||||
"
|
||||
TESTING_SRC_DIRS="/export/jenkins/mirrors/dl.fedoraproject.org/pub/epel/testing/7/SRPMS
|
||||
"
|
||||
|
||||
BIN_ARCH=x86_64
|
||||
|
||||
NEW_DIR="/export/jenkins/mirrors/CentOS/tis-r3-CentOS/mitaka"
|
||||
BACKUP_DIR="/export/jenkins/mirrors/CentOS/tis-r3-CentOS/mitaka.old"
|
||||
|
||||
CREATEREPO=$(which createrepo_c)
|
||||
if [ $? -ne 0 ]; then
|
||||
CREATEREPO="createrepo"
|
||||
fi
|
||||
|
||||
if [ -d $BACKUP_DIR ]; then
|
||||
rm -rf $BACKUP_DIR
|
||||
fi
|
||||
cp -r $NEW_DIR $BACKUP_DIR
|
||||
|
||||
BIN_DEST="$NEW_DIR/Binary"
|
||||
SRC_DEST="$NEW_DIR/Source"
|
||||
LOG_DEST="$NEW_DIR/Data"
|
||||
DATA_DEST="$NEW_DIR/Data"
|
||||
|
||||
mkdir -p $BIN_DEST $SRC_DEST $LOG_DEST
|
||||
|
||||
WHITE_LIST_FILE="$DATA_DIR/white_list.txt"
|
||||
BLACK_LIST_FILE="$DATA_DIR/black_list.txt"
|
||||
OBSOLETE_LOG="$LOG_DEST/obsolete.log"
|
||||
NO_SOURCERPM_LOG="$LOG_DEST/no_sourcerpm.log"
|
||||
SOURCERPM_NOT_FOUND_LOG="$LOG_DEST/sourcerpm_not_found.log"
|
||||
BIN_COPY_LOG="$LOG_DEST/bin_copy.log"
|
||||
SRC_COPY_LOG="$LOG_DEST/src_copy.log"
|
||||
NO_INSTALLED_COPY="$LOG_DEST/installed_copy.log"
|
||||
BLACKLISTED_LOG="$LOG_DEST/blacklisted.log"
|
||||
WHITELISTED_LOG="$LOG_DEST/whitelisted.log"
|
||||
|
||||
rm -f $OBSOLETE_LOG $NO_SOURCERPM_LOG $SOURCERPM_NOT_FOUND_LOG $BIN_COPY_LOG $SRC_COPY_LOG $NO_INSTALLED_COPY $BLACKLISTED_LOG $WHITELISTED_LOG
|
||||
|
||||
LAST_INSTALLED_NAME=""
|
||||
LAST_NAME=""
|
||||
NAME=""
|
||||
|
||||
# The following is similar to a find for *.rpm files,
|
||||
# but we transform the path into <name_and_version>#<filename>#<directory_name> .
|
||||
# Then do a reverse 'version' sort, so that the newest version
|
||||
# of the rpm appears first, no matter what directory it originates from.
|
||||
for dat in $((for bd in `echo $BIN_DIRS`; do
|
||||
for br in $(find $bd/ -name '*.rpm' ); do
|
||||
d=$(dirname $br)
|
||||
b=$(basename $br)
|
||||
s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$s#$b#$d"
|
||||
done
|
||||
done) | sort -r -V)
|
||||
do
|
||||
b=$(echo "$dat" | awk -F '#' '{ print $2 }')
|
||||
d=$(echo "$dat" | awk -F '#' '{ print $3 }')
|
||||
br="$d/$b"
|
||||
echo $br
|
||||
|
||||
MATCH=$(grep "^$b" $BLACK_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
echo "$bs" >> $BLACKLISTED_LOG
|
||||
continue
|
||||
fi
|
||||
|
||||
SOURCERPM=$(rpm -q --queryformat='%{SOURCERPM}' -p $br --nosignature)
|
||||
if [ "x$SOURCERPM" != "x" ]; then
|
||||
MATCH=$(grep "^$SOURCERPM" $BLACK_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
echo "$bs" >> $BLACKLISTED_LOG
|
||||
echo "$SOURCERPM" >> $BLACKLISTED_LOG
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
|
||||
ARCH=$(rpm -q --queryformat='%{ARCH}' -p $br --nosignature)
|
||||
LAST_NAME="$NAME"
|
||||
NAME=$(rpm -q --queryformat='%{NAME}' -p $br --nosignature)
|
||||
if [ "$NAME" != "$LAST_NAME" ] && [ "$LAST_NAME" != "$LAST_INSTALLED_NAME" ]; then
|
||||
echo "$LAST_NAME" >> $NO_INSTALLED_COPY
|
||||
fi
|
||||
if [ "$ARCH" == "$BIN_ARCH" ] || [ "$ARCH" == "noarch" ]; then
|
||||
if [ "$NAME" != "$LAST_INSTALLED_NAME" ]; then
|
||||
if [ "x$SOURCERPM" != "x" ]; then
|
||||
bs=$(find $SRC_DIRS -name $SOURCERPM | head -n 1)
|
||||
if [ "x$bs" != "x" ]; then
|
||||
mkdir -p $BIN_DEST/$ARCH
|
||||
if [ ! -f $BIN_DEST/$ARCH/$b ]; then
|
||||
cp -v $br $BIN_DEST/$ARCH/
|
||||
echo "$br" >> $BIN_COPY_LOG
|
||||
fi
|
||||
if [ ! -f $SRC_DEST/$SOURCERPM ]; then
|
||||
cp -v $bs $SRC_DEST/
|
||||
echo "$bs" >> $SRC_COPY_LOG
|
||||
fi
|
||||
LAST_INSTALLED_NAME=$NAME
|
||||
else
|
||||
echo "$SOURCERPM not found"
|
||||
echo "$br" >> $SOURCERPM_NOT_FOUND_LOG
|
||||
fi
|
||||
else
|
||||
echo "no SOURCERPM for $br"
|
||||
echo "$br" >> $NO_SOURCERPM_LOG
|
||||
fi
|
||||
else
|
||||
echo "$br is obsolete"
|
||||
echo "$br" >> $OBSOLETE_LOG
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
for dat in $((for bd in $(echo $BIN_DIRS; echo $TESTING_BIN_DIRS); do
|
||||
for br in $(find $bd/ -name '*.rpm'); do
|
||||
d=$(dirname $br)
|
||||
b=$(basename $br)
|
||||
s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$s#$b#$d"
|
||||
done
|
||||
done) | sort -r -V)
|
||||
do
|
||||
b=$(echo "$dat" | awk -F '#' '{ print $2 }')
|
||||
d=$(echo "$dat" | awk -F '#' '{ print $3 }')
|
||||
br="$d/$b"
|
||||
echo $br
|
||||
|
||||
MATCH=$(grep "^$b" $WHITE_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
echo "$bs" >> $WHITELISTED_LOG
|
||||
else
|
||||
continue
|
||||
fi
|
||||
|
||||
SOURCERPM=$(rpm -q --queryformat='%{SOURCERPM}' -p $br --nosignature)
|
||||
if [ "x$SOURCERPM" != "x" ]; then
|
||||
grep "^$SOURCERPM" $WHITE_LIST_FILE >> /dev/null || true
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "$bs" >> $WHITELISTED_LOG
|
||||
echo "$SOURCERPM" >> $WHITELISTED_LOG
|
||||
else
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
|
||||
ARCH=$(rpm -q --queryformat='%{ARCH}' -p $br --nosignature)
|
||||
LAST_NAME="$NAME"
|
||||
NAME=$(rpm -q --queryformat='%{NAME}' -p $br --nosignature)
|
||||
if [ "$NAME" != "$LAST_NAME" ] && [ "$LAST_NAME" != "$LAST_INSTALLED_NAME" ]; then
|
||||
echo "$LAST_NAME" >> $NO_INSTALLED_COPY
|
||||
fi
|
||||
if [ "$ARCH" == "$BIN_ARCH" ] || [ "$ARCH" == "noarch" ]; then
|
||||
if [ "$NAME" != "$LAST_INSTALLED_NAME" ]; then
|
||||
if [ "x$SOURCERPM" != "x" ]; then
|
||||
bs=$(find $SRC_DIRS $TESTING_SRC_DIRS -name $SOURCERPM | head -n 1)
|
||||
if [ "x$bs" != "x" ]; then
|
||||
mkdir -p $BIN_DEST/$ARCH
|
||||
if [ ! -f $BIN_DEST/$ARCH/$b ]; then
|
||||
cp -v $br $BIN_DEST/$ARCH/
|
||||
echo "$br" >> $BIN_COPY_LOG
|
||||
fi
|
||||
if [ ! -f $SRC_DEST/$SOURCERPM ]; then
|
||||
cp -v $bs $SRC_DEST/
|
||||
echo "$bs" >> $SRC_COPY_LOG
|
||||
fi
|
||||
LAST_INSTALLED_NAME=$NAME
|
||||
else
|
||||
echo "$SOURCERPM not found"
|
||||
echo "$br" >> $SOURCERPM_NOT_FOUND_LOG
|
||||
fi
|
||||
else
|
||||
echo "no SOURCERPM for $br"
|
||||
echo "$br" >> $NO_SOURCERPM_LOG
|
||||
fi
|
||||
else
|
||||
echo "$br is obsolete"
|
||||
echo "$br" >> $OBSOLETE_LOG
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
|
||||
repodata_update () {
|
||||
DIR=${1}
|
||||
(
|
||||
cd $DIR
|
||||
for d in `find -L . -type d -name repodata`; do
|
||||
(cd $d/..
|
||||
for c in $(find repodata -name '*comps*xml'); do
|
||||
mv -f $c comps.xml
|
||||
done
|
||||
rm -rf repodata
|
||||
if [ -f comps.xml ]; then
|
||||
$CREATEREPO -g comps.xml --workers $(cat /usr/bin/nproc) $(pwd)
|
||||
else
|
||||
$CREATEREPO --workers $(cat /usr/bin/nproc) $(pwd)
|
||||
fi
|
||||
)
|
||||
done
|
||||
)
|
||||
return 0
|
||||
}
|
||||
|
||||
repodata_update $SRC_DEST
|
||||
repodata_update $BIN_DEST
|
||||
216
build-tools/mirror_rebase/copy_external_mirror_to_tis_mirror.old
Executable file
216
build-tools/mirror_rebase/copy_external_mirror_to_tis_mirror.old
Executable file
@ -0,0 +1,216 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ "$USER" != "jenkins" ]; then
|
||||
echo "ERROR: only jenkins should run this"
|
||||
return 1
|
||||
fi
|
||||
|
||||
CENTOS_BASE_VERSION=7.2.1511
|
||||
EPEL_BASE_VERSION=7
|
||||
BIN_ARCH=x86_64
|
||||
|
||||
BIN_DIRS="/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/updates/x86_64
|
||||
/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/cloud/x86_64/openstack-mitaka
|
||||
/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/os/x86_64
|
||||
/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/extras/x86_64
|
||||
/export/jenkins/mirrors/fedora/epel/$EPEL_BASE_VERSION/x86_64
|
||||
"
|
||||
SRC_DIRS="/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/updates/Source
|
||||
/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/cloud/Source/openstack-mitaka
|
||||
/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/os/Source
|
||||
/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/extras/Source
|
||||
/export/jenkins/mirrors/fedora/dl.fedoraproject.org/pub/epel/$EPEL_BASE_VERSION/SRPMS
|
||||
"
|
||||
TESTING_BIN_DIRS="/export/jenkins/mirrors/fedora/epel/testing/7/x86_64
|
||||
"
|
||||
TESTING_SRC_DIRS="/export/jenkins/mirrors/dl.fedoraproject.org/pub/epel/testing/7/SRPMS
|
||||
"
|
||||
|
||||
ORIG_DIR="/export/jenkins/mirrors/CentOS/tis-r3-CentOS/mitaka"
|
||||
NEW_DIR="/export/jenkins/mirrors/CentOS/tis-r3-CentOS/mitaka-2"
|
||||
cp -r $ORIG_DIR $NEW_DIR
|
||||
|
||||
BIN_DEST="$NEW_DIR/Binary"
|
||||
SRC_DEST="$NEW_DIR/Source"
|
||||
LOG_DEST="$NEW_DIR/Data"
|
||||
DATA_DEST="$NEW_DIR/Data"
|
||||
|
||||
rm -rf $LOG_DEST
|
||||
mkdir -p $BIN_DEST $SRC_DEST $LOG_DEST
|
||||
|
||||
WHITE_LIST_FILE="$DATA_DIR/white_list.txt"
|
||||
BLACK_LIST_FILE="$DATA_DIR/black_list.txt"
|
||||
OBSOLETE_LOG="$LOG_DEST/obsolete.log"
|
||||
NO_SOURCERPM_LOG="$LOG_DEST/no_sourcerpm.log"
|
||||
SOURCERPM_NOT_FOUND_LOG="$LOG_DEST/sourcerpm_not_found.log"
|
||||
BIN_COPY_LOG="$LOG_DEST/bin_copy.log"
|
||||
SRC_COPY_LOG="$LOG_DEST/src_copy.log"
|
||||
NO_INSTALLED_COPY="$LOG_DEST/installed_copy.log"
|
||||
BLACKLISTED_LOG="$LOG_DEST/blacklisted.log"
|
||||
WHITELISTED_LOG="$LOG_DEST/whitelisted.log"
|
||||
|
||||
rm -f $OBSOLETE_LOG $NO_SOURCERPM_LOG $SOURCERPM_NOT_FOUND_LOG $BIN_COPY_LOG $SRC_COPY_LOG $NO_INSTALLED_COPY $BLACKLISTED_LOG $WHITELISTED_LOG
|
||||
|
||||
LAST_INSTALLED_NAME=""
|
||||
LAST_NAME=""
|
||||
NAME=""
|
||||
|
||||
# The following is similar to a find for *.rpm files,
|
||||
# but we transform the path into <name_and_version>#<filename>#<directory_name> .
|
||||
# Then do a reverse 'version' sort, so that the newest version
|
||||
# of the rpm appears first, no matter what directory it originates from.
|
||||
for dat in $((for bd in `echo $BIN_DIRS`; do
|
||||
for br in $(find $bd/ -name '*.rpm' ); do
|
||||
d=$(dirname $br)
|
||||
b=$(basename $br)
|
||||
s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$s#$b#$d"
|
||||
done
|
||||
done) | sort -r -V)
|
||||
do
|
||||
b=$(echo "$dat" | awk -F '#' '{ print $2 }')
|
||||
d=$(echo "$dat" | awk -F '#' '{ print $3 }')
|
||||
br="$d/$b"
|
||||
echo $br
|
||||
|
||||
MATCH=$(grep "^$b" $BLACK_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
echo "$bs" >> $BLACKLISTED_LOG
|
||||
continue
|
||||
fi
|
||||
|
||||
SOURCERPM=$(rpm -q --queryformat='%{SOURCERPM}' -p $br --nosignature)
|
||||
if [ "x$SOURCERPM" != "x" ]; then
|
||||
MATCH=$(grep "^$SOURCERPM" $BLACK_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
echo "$bs" >> $BLACKLISTED_LOG
|
||||
echo "$SOURCERPM" >> $BLACKLISTED_LOG
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
|
||||
ARCH=$(rpm -q --queryformat='%{ARCH}' -p $br --nosignature)
|
||||
LAST_NAME="$NAME"
|
||||
NAME=$(rpm -q --queryformat='%{NAME}' -p $br --nosignature)
|
||||
if [ "$NAME" != "$LAST_NAME" ] && [ "$LAST_NAME" != "$LAST_INSTALLED_NAME" ]; then
|
||||
echo "$LAST_NAME" >> $NO_INSTALLED_COPY
|
||||
fi
|
||||
if [ "$ARCH" == "$BIN_ARCH" ] || [ "$ARCH" == "noarch" ]; then
|
||||
if [ "$NAME" != "$LAST_INSTALLED_NAME" ]; then
|
||||
if [ "x$SOURCERPM" != "x" ]; then
|
||||
bs=$(find $SRC_DIRS -name $SOURCERPM | head -n 1)
|
||||
if [ "x$bs" != "x" ]; then
|
||||
mkdir -p $BIN_DEST/$ARCH
|
||||
if [ ! -f $BIN_DEST/$ARCH/$b ]; then
|
||||
cp -v $br $BIN_DEST/$ARCH/
|
||||
echo "$br" >> $BIN_COPY_LOG
|
||||
fi
|
||||
if [ ! -f $SRC_DEST/$SOURCERPM ]; then
|
||||
cp -v $bs $SRC_DEST/
|
||||
echo "$bs" >> $SRC_COPY_LOG
|
||||
fi
|
||||
LAST_INSTALLED_NAME=$NAME
|
||||
else
|
||||
echo "$SOURCERPM not found"
|
||||
echo "$br" >> $SOURCERPM_NOT_FOUND_LOG
|
||||
fi
|
||||
else
|
||||
echo "no SOURCERPM for $br"
|
||||
echo "$br" >> $NO_SOURCERPM_LOG
|
||||
fi
|
||||
else
|
||||
echo "$br is obsolete"
|
||||
echo "$br" >> $OBSOLETE_LOG
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
for dat in $((for bd in $(echo $BIN_DIRS; echo $TESTING_BIN_DIRS); do
|
||||
for br in $(find $bd/ -name '*.rpm'); do
|
||||
d=$(dirname $br)
|
||||
b=$(basename $br)
|
||||
s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$s#$b#$d"
|
||||
done
|
||||
done) | sort -r -V)
|
||||
do
|
||||
b=$(echo "$dat" | awk -F '#' '{ print $2 }')
|
||||
d=$(echo "$dat" | awk -F '#' '{ print $3 }')
|
||||
br="$d/$b"
|
||||
echo $br
|
||||
|
||||
MATCH=$(grep "^$b" $WHITE_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
echo "$bs" >> $WHITELISTED_LOG
|
||||
else
|
||||
continue
|
||||
fi
|
||||
|
||||
SOURCERPM=$(rpm -q --queryformat='%{SOURCERPM}' -p $br --nosignature)
|
||||
if [ "x$SOURCERPM" != "x" ]; then
|
||||
grep "^$SOURCERPM" $WHITE_LIST_FILE >> /dev/null || true
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "$bs" >> $WHITELISTED_LOG
|
||||
echo "$SOURCERPM" >> $WHITELISTED_LOG
|
||||
else
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
|
||||
ARCH=$(rpm -q --queryformat='%{ARCH}' -p $br --nosignature)
|
||||
LAST_NAME="$NAME"
|
||||
NAME=$(rpm -q --queryformat='%{NAME}' -p $br --nosignature)
|
||||
if [ "$NAME" != "$LAST_NAME" ] && [ "$LAST_NAME" != "$LAST_INSTALLED_NAME" ]; then
|
||||
echo "$LAST_NAME" >> $NO_INSTALLED_COPY
|
||||
fi
|
||||
if [ "$ARCH" == "$BIN_ARCH" ] || [ "$ARCH" == "noarch" ]; then
|
||||
if [ "$NAME" != "$LAST_INSTALLED_NAME" ]; then
|
||||
if [ "x$SOURCERPM" != "x" ]; then
|
||||
bs=$(find $SRC_DIRS $TESTING_SRC_DIRS -name $SOURCERPM | head -n 1)
|
||||
if [ "x$bs" != "x" ]; then
|
||||
mkdir -p $BIN_DEST/$ARCH
|
||||
if [ ! -f $BIN_DEST/$ARCH/$b ]; then
|
||||
cp -v $br $BIN_DEST/$ARCH/
|
||||
echo "$br" >> $BIN_COPY_LOG
|
||||
fi
|
||||
if [ ! -f $SRC_DEST/$SOURCERPM ]; then
|
||||
cp -v $bs $SRC_DEST/
|
||||
echo "$bs" >> $SRC_COPY_LOG
|
||||
fi
|
||||
LAST_INSTALLED_NAME=$NAME
|
||||
else
|
||||
echo "$SOURCERPM not found"
|
||||
echo "$br" >> $SOURCERPM_NOT_FOUND_LOG
|
||||
fi
|
||||
else
|
||||
echo "no SOURCERPM for $br"
|
||||
echo "$br" >> $NO_SOURCERPM_LOG
|
||||
fi
|
||||
else
|
||||
echo "$br is obsolete"
|
||||
echo "$br" >> $OBSOLETE_LOG
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
|
||||
TMP_DIR=$(mktemp -d /tmp/copy_external_mirror_to_tis_mirror_XXXXXX)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to create temporary directory"
|
||||
return 1
|
||||
fi
|
||||
|
||||
(cd $ORIG_DIR; find . | sort -V > $TMP_DIR/pre)
|
||||
(cd $NEW_DIR; find . | sort -V > $TMP_DIR/post)
|
||||
echo "Listing deletions"
|
||||
diff $TMP_DIR/pre $TMP_DIR/post | grep -v '^< ./Data/' | grep '^<'
|
||||
if [ $? -eq 0 ]; then
|
||||
echo
|
||||
echo "Cowardly refusing to alter $ORIG_DIR due to deletions: please see $NEW_DIR"
|
||||
return 1
|
||||
fi
|
||||
|
||||
mv -f $ORIG_DIR $ORIG_DIR.old
|
||||
mv -f $NEW_DIR $ORIG_DIR
|
||||
rm -rf $TMP_DIR
|
||||
return 0
|
||||
225
build-tools/mirror_rebase/link_cgcs_centos_repo
Executable file
225
build-tools/mirror_rebase/link_cgcs_centos_repo
Executable file
@ -0,0 +1,225 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
#
|
||||
# This step updates Binary links, and adds Source links
|
||||
#
|
||||
|
||||
MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
BIN_ROOT=$MIRROR_ROOT/Binary
|
||||
SRC_ROOT=$MIRROR_ROOT/Source
|
||||
BLACK_LIST_FILE=$MIRROR_ROOT/Data/black_list.txt
|
||||
WHITE_LIST_FILE=$MIRROR_ROOT/Data/white_list.txt
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
echo 'ERROR: failed to cd to $MY_REPO/cgcs-centos-repo'
|
||||
return 1
|
||||
fi
|
||||
|
||||
names=' '
|
||||
snames=' '
|
||||
|
||||
do_work () {
|
||||
dat=${1}
|
||||
|
||||
b=$(echo "$dat" | awk -F '#' '{ print $2 }')
|
||||
d=$(echo "$dat" | awk -F '#' '{ print $3 }')
|
||||
r="$d/$b"
|
||||
|
||||
DEBUG_INFO=0
|
||||
MATCH=$(echo $b | grep '[-]debuginfo-')
|
||||
if [ "$MATCH" != "" ]; then
|
||||
DEBUG_INFO=1
|
||||
fi
|
||||
|
||||
MATCH=$(grep "^$b" $BLACK_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
echo "NOTE: '$b' is black listed"
|
||||
continue
|
||||
fi
|
||||
|
||||
if [ $DEBUG_INFO -eq 1 ]; then
|
||||
sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM : ' | sed 's#^Source RPM : ##')
|
||||
if [ "x$sb" == "x" ]; then
|
||||
echo "ERROR: no source rpm listed for '$b'"
|
||||
continue
|
||||
fi
|
||||
s=$(find Source -name "$sb")
|
||||
if [ "x$s" == "x" ]; then
|
||||
echo "NOTE: no source rpm '$sb' found for '$b'"
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
|
||||
name=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r)
|
||||
|
||||
MATCH=$(grep "^$b" $WHITE_LIST_FILE || true)
|
||||
if [ "$MATCH" == "" ]; then
|
||||
# Not white listed, check for python2 alternative
|
||||
# python-rpm-macros-3-6.1 is a notable case white_list case...
|
||||
# We need BOTH python-rpm-macros-3-6.1 and python2-rpm-macros-3-6.1
|
||||
# so substituting python-rpm-macros-3-6.1 with python2-rpm-macros-3-6.1 is an error
|
||||
|
||||
altname=$(echo $name | sed 's#^python-#python2-#')
|
||||
if [ "$altname" != "$name" ]; then
|
||||
# look for python2 alternative
|
||||
sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM : ' | sed 's#^Source RPM : ##')
|
||||
|
||||
if [ "x$sb" != "x" ]; then
|
||||
MATCH=$(grep "^$sb" $BLACK_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
echo "NOTE: '$sb' is black listed, so '$b' is ignored"
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
|
||||
for dat2 in $(for br in $(find $BIN_ROOT -name "$altname-*.rpm" | grep -v '.src.rpm$' | grep -v '[-]debuginfo-'); do
|
||||
ddd=$(dirname $br)
|
||||
bbb=$(basename $br)
|
||||
sss=$(echo $bbb | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$sss#$bbb#$ddd"
|
||||
done | sort -r -V)
|
||||
do
|
||||
b2=$(echo "$dat2" | awk -F '#' '{ print $2 }')
|
||||
d2=$(echo "$dat2" | awk -F '#' '{ print $3 }')
|
||||
r2="$d2/$b2"
|
||||
name2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r2)
|
||||
if [ "$name2" != "$altname" ]; then
|
||||
continue
|
||||
fi
|
||||
sb2=$(rpm -q --info --nosignature -p $r2 | grep '^Source RPM : ' | sed 's#^Source RPM : ##')
|
||||
# if [ "$sb" == "$sb2" ]; then
|
||||
# continue
|
||||
# fi
|
||||
|
||||
sbs=$(echo $sb | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
sbs2=$(echo $sb2 | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
newer=$((echo $sbs; echo $sbs2) | sort -r -V | head -n 1)
|
||||
if [ "$sbs" != "$sbs2" ]; then
|
||||
if [ "$newer" == "$sbs2" ]; then
|
||||
# swap alternate for original
|
||||
for link in $(find Binary -name $b); do
|
||||
echo "SUGGEST: rm $link"
|
||||
git rm -f $link
|
||||
done
|
||||
|
||||
r=$r2
|
||||
name=$name2
|
||||
b=$b2
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "$names" | grep " $name " >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM : ' | sed 's#^Source RPM : ##')
|
||||
if [ "x$sb" == "x" ]; then
|
||||
echo "ERROR: no source rpm listed for '$b'"
|
||||
continue
|
||||
fi
|
||||
s=$(find $SRC_ROOT -name "$sb")
|
||||
if [ "x$s" == "x" ]; then
|
||||
echo "ERROR: no source rpm '$sb' found for '$b'"
|
||||
continue
|
||||
fi
|
||||
|
||||
if [ "x$sb" != "x" ]; then
|
||||
MATCH=$(grep "^$sb" $BLACK_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
echo "NOTE: '$sb' is black listed, so '$b' is ignored"
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
|
||||
sname=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $s)
|
||||
|
||||
lb=$(find Binary -name "$b")
|
||||
if [ "x$lb" == "x" ]; then
|
||||
echo "MISSING: '$b'"
|
||||
link=$(echo $r | sed "s#^$MIRROR_ROOT/##")
|
||||
echo "SUGGEST: ln -s $r $link"
|
||||
ln -s $r $link
|
||||
git add $link
|
||||
else
|
||||
echo "OK: '$b'"
|
||||
fi
|
||||
|
||||
for r2 in $(find Binary -name "$name-*.rpm"); do
|
||||
b2=$(basename $r2)
|
||||
if [ "$b" != "$b2" ]; then
|
||||
name2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r2)
|
||||
if [ "$name" == "$name2" ]; then
|
||||
MATCH=$(grep "^$b2" $WHITE_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
link=$(echo $r2 | sed "s#^$MIRROR_ROOT/##")
|
||||
echo "SUGGEST: rm $link"
|
||||
git rm -f $link
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [ $DEBUG_INFO -eq 0 ]; then
|
||||
# Not a debuginfo therefore we can pull in new src.rpm
|
||||
names="${names}${name} "
|
||||
lsb=$(find Source -name "$sb")
|
||||
if [ "x$lsb" == "x" ]; then
|
||||
echo "MISSING: '$sb'"
|
||||
link=$(echo $s | sed "s#^$MIRROR_ROOT/##")
|
||||
echo "SUGGEST: ln -s $s $link"
|
||||
ln -s $s $link
|
||||
git add $link
|
||||
else
|
||||
echo "OK: '$sb'"
|
||||
fi
|
||||
|
||||
echo "$names" | grep " $name " >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
for s2 in $(find Source -name "$sname-*.rpm"); do
|
||||
sb2=$(basename $s2)
|
||||
if [ "$sb" != "$sb2" ]; then
|
||||
sname2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $s2)
|
||||
if [ "$sname" == "$sname2" ]; then
|
||||
MATCH=$(grep "^$sb2" $WHITE_LIST_FILE || true)
|
||||
if [ "$MATCH" != "" ]; then
|
||||
link=$(echo $s2 | sed "s#^$MIRROR_ROOT/##")
|
||||
echo "SUGGEST: rm $link"
|
||||
git rm -f $link
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
snames="${snames}${sname} "
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
for dat in $(for br in $(find $BIN_ROOT -name '*.rpm' | grep -v '.src.rpm$' | grep -v '[-]debuginfo-'); do
|
||||
d=$(dirname $br)
|
||||
b=$(basename $br)
|
||||
s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$s#$b#$d"
|
||||
done | sort -r -V)
|
||||
do
|
||||
do_work ${dat}
|
||||
done
|
||||
|
||||
for dat in $(for br in $(find $BIN_ROOT -name '*.rpm' | grep -v '.src.rpm$' | grep '[-]debuginfo-'); do
|
||||
d=$(dirname $br)
|
||||
b=$(basename $br)
|
||||
s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$s#$b#$d"
|
||||
done | sort -r -V)
|
||||
do
|
||||
do_work ${dat}
|
||||
done
|
||||
82
build-tools/mirror_rebase/link_cgcs_centos_repo_2
Executable file
82
build-tools/mirror_rebase/link_cgcs_centos_repo_2
Executable file
@ -0,0 +1,82 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
#
|
||||
# This step removes obsolete or broken Source links
|
||||
#
|
||||
|
||||
MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
BIN_ROOT=$MIRROR_ROOT/Binary
|
||||
SRC_ROOT=$MIRROR_ROOT/Source
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
echo 'ERROR: failed to cd to $MY_REPO/cgcs-centos-repo'
|
||||
return 1
|
||||
fi
|
||||
|
||||
|
||||
# Clean broken and obsolete srpm links
|
||||
snames=" "
|
||||
for dat in $(for br in $(find Source -name '*.src.rpm'); do
|
||||
d=$(dirname $br)
|
||||
b=$(basename $br)
|
||||
s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$s#$b#$d"
|
||||
done | sort -r -V)
|
||||
do
|
||||
sb=$(echo "$dat" | awk -F '#' '{ print $2 }')
|
||||
d=$(echo "$dat" | awk -F '#' '{ print $3 }')
|
||||
s="$d/$sb"
|
||||
|
||||
if [ ! -f $s ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
link=$(readlink $s)
|
||||
if [ ! -f $link ]; then
|
||||
echo "ERROR: '$sb' link to non-existant file '$link'"
|
||||
echo "SUGGEST: rm $s"
|
||||
git rm -f $s
|
||||
continue
|
||||
fi
|
||||
|
||||
echo $link | grep "$MIRROR_ROOT" >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: '$sb' links to unexpected file '$link'"
|
||||
echo "SUGGEST: rm $s"
|
||||
git rm -f $s
|
||||
continue
|
||||
fi
|
||||
|
||||
sname=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $s)
|
||||
echo "$snames" | grep " $sname " >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
if [ "x$sname" != "x" ]; then
|
||||
for s2 in $(find Source -name "$sname-*.src.rpm"); do
|
||||
sb2=$(basename $s2)
|
||||
if [ "$sb" != "$sb2" ]; then
|
||||
sname2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $s2)
|
||||
if [ "$sname" == "$sname2" ]; then
|
||||
link=$(echo $s2 | sed "s#^$MIRROR_ROOT/##")
|
||||
echo "SUGGEST: rm $link, due to $sb"
|
||||
git rm -f $link
|
||||
|
||||
for r3 in $(find Binary -name "$sname-*.rpm"); do
|
||||
sb3=$(rpm -q --info --nosignature -p $r3 | grep '^Source RPM : ' | sed 's#^Source RPM : ##')
|
||||
if [ "$sb3" == "$sb2" ]; then
|
||||
echo "SUGGEST: rm $r3, due to $sb2"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
snames="${snames}${sname} "
|
||||
fi
|
||||
done
|
||||
|
||||
40
build-tools/mirror_rebase/link_cgcs_centos_repo_3
Executable file
40
build-tools/mirror_rebase/link_cgcs_centos_repo_3
Executable file
@ -0,0 +1,40 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
#
|
||||
# This step removes broken Binary links
|
||||
#
|
||||
|
||||
MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
echo 'ERROR: failed to cd to $MY_REPO/cgcs-centos-repo'
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Clean broken rpm links
|
||||
for r in $(find Binary -name '*.rpm' | grep -v '.src.rpm$' | sort -r -V); do
|
||||
b=$(basename $r)
|
||||
|
||||
link=$(readlink $r)
|
||||
if [ ! -f $link ]; then
|
||||
echo "ERROR: '$b' link to non-existant file '$link'"
|
||||
echo "SUGGEST: rm $r"
|
||||
git rm -f $r
|
||||
continue
|
||||
fi
|
||||
|
||||
echo $link | grep "$MIRROR_ROOT" >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: '$b' links to unexpected file '$link'"
|
||||
echo "SUGGEST: rm $r"
|
||||
git rm -f $r
|
||||
continue
|
||||
fi
|
||||
done
|
||||
|
||||
176
build-tools/mirror_rebase/link_cgcs_centos_repo_4
Executable file
176
build-tools/mirror_rebase/link_cgcs_centos_repo_4
Executable file
@ -0,0 +1,176 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
#
|
||||
# This step removes obsolete Binary links
|
||||
#
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
BIN_ROOT=$MIRROR_ROOT/Binary
|
||||
SRC_ROOT=$MIRROR_ROOT/Source
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
echo 'ERROR: failed to cd to $MY_REPO/cgcs-centos-repo'
|
||||
return 1
|
||||
fi
|
||||
|
||||
for dat in $(for br in $(find Binary -name '*.rpm' | grep -v '.src.rpm$' ); do
|
||||
d=$(dirname $br)
|
||||
b=$(basename $br)
|
||||
s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$s#$b#$d"
|
||||
done | sort -r -V)
|
||||
do
|
||||
b=$(echo "$dat" | awk -F '#' '{ print $2 }')
|
||||
d=$(echo "$dat" | awk -F '#' '{ print $3 }')
|
||||
r="$d/$b"
|
||||
name=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r)
|
||||
|
||||
link=$(readlink $r)
|
||||
sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM : ' | sed 's#^Source RPM : ##')
|
||||
if [ "x$sb" == "x" ]; then
|
||||
echo "ERROR: no source rpm listed for '$b'"
|
||||
continue
|
||||
fi
|
||||
s=$(find Source -name "$sb")
|
||||
if [ "x$s" == "x" ]; then
|
||||
DELETED=0
|
||||
altname=$(echo $name | sed 's#^python-#python2-#')
|
||||
if [ "$altname" != "$name" ]; then
|
||||
# look for python2 alternative
|
||||
|
||||
for dat2 in $(for br in $(find $BIN_ROOT -name "$altname-*.rpm" | grep -v '.src.rpm$' | grep -v '[-]debuginfo-'); do
|
||||
ddd=$(dirname $br)
|
||||
bbb=$(basename $br)
|
||||
sss=$(echo $bbb | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
echo "$sss#$bbb#$ddd"
|
||||
done | sort -r -V)
|
||||
do
|
||||
b2=$(echo "$dat2" | awk -F '#' '{ print $2 }')
|
||||
d2=$(echo "$dat2" | awk -F '#' '{ print $3 }')
|
||||
r2="$d2/$b2"
|
||||
|
||||
name2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r2)
|
||||
if [ "$name2" != "$altname" ]; then
|
||||
continue
|
||||
fi
|
||||
sb2=$(rpm -q --info --nosignature -p $r2 | grep '^Source RPM : ' | sed 's#^Source RPM : ##')
|
||||
if [ "$sb" == "$sb2" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
sbs=$(echo $sb | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
sbs2=$(echo $sb2 | sed -e 's#.centos.#.#' -e 's#.el7.#.#' -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
|
||||
newer=$((echo $sbs; echo $sbs2) | sort -r -V | head -n 1)
|
||||
if [ "$sbs" != "$sbs2" ]; then
|
||||
if [ "$newer" == "$sbs2" ]; then
|
||||
# swap alternate for original
|
||||
echo "SUGGEST: rm $r"
|
||||
git rm -f $r
|
||||
|
||||
DELETED=1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
if [ $DELETED -eq 0 ]; then
|
||||
echo "ERROR: no source rpm '$sb' found for '$b'"
|
||||
echo "SUGGEST: rm $r"
|
||||
git rm -f $r
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
#
|
||||
# The following would delete all binary rpms that Titanium Cloud would otherwise compile.
|
||||
# However for bootstrapping the build there are some packages that we must have...
|
||||
# e.g. bash, kernel-headers, ....
|
||||
# So we will need something smarter than to just delete everything.
|
||||
#
|
||||
# TMP_DIR=$(mktemp -d /tmp/link_cgcs_centos_repo_XXXXXX)
|
||||
#
|
||||
# BUILT_SRPMS_FILE=$TMP_DIR/built_srpms_file
|
||||
#
|
||||
# for r in $(for c in $(find $MY_REPO -type d -name centos); do
|
||||
# for sp in $(find $c -name srpm_path); do
|
||||
# echo "$sp: $(cat $sp)"
|
||||
# done
|
||||
# done | grep 'mirror:' | awk -F ' ' '{ print $2 }') ; do
|
||||
# b=$(basename $r)
|
||||
# s=$(find $MY_REPO/cgcs-centos-repo/Source/ -name $b)
|
||||
# n=$(rpm -q --qf '%{NAME}' --nosignature -p $s)
|
||||
# echo "$n:$b" >> $BUILT_SRPMS_FILE
|
||||
# done
|
||||
#
|
||||
# cd $MY_REPO/cgcs-centos-repo
|
||||
#
|
||||
# for r in $(find Binary -name '*.rpm'); do
|
||||
# b=$(basename $r)
|
||||
# sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM :' | sed 's#^Source RPM : ##')
|
||||
# if [ "x$sb" != "x" ]; then
|
||||
# s=$(find Source/ -name $sb)
|
||||
# if [ "x$s" != "x" ]; then
|
||||
# n=$(rpm -q --qf '%{NAME}' --nosignature -p $s)
|
||||
# grep "^$n:" $BUILT_SRPMS_FILE
|
||||
# if [ $? -eq 0 ]; then
|
||||
# git rm -f $r
|
||||
# fi
|
||||
# fi
|
||||
# fi
|
||||
# done
|
||||
#
|
||||
# \rm $BUILT_SRPMS_FILE
|
||||
# rmdir $TMP_DIR
|
||||
|
||||
TMP_DIR=$(mktemp -d /tmp/link_cgcs_centos_repo_XXXXXX)
|
||||
EXCLUDE_LIST=$TMP_DIR/exclude_list
|
||||
|
||||
# List od packages we compile from scratch, not from centos srpm, goes to $EXCLUDE_LIST
|
||||
for g in $(find $MY_REPO -type d -name .git); do
|
||||
d=$(dirname $g)
|
||||
for cpd in $(find $d -maxdepth 1 -name 'centos_pkg_dir*'); do
|
||||
(
|
||||
cd $d
|
||||
for pd in $(cat $cpd); do
|
||||
(
|
||||
cd $pd/centos
|
||||
if [ -f srpm_path ]; then
|
||||
continue
|
||||
fi
|
||||
for spec in $(find . -name '*.spec'); do
|
||||
n=$(spec_find_tag 'name' $spec '/tmp/' '0' 2> /dev/null)
|
||||
echo "$spec: $n"
|
||||
echo $n >> $EXCLUDE_LIST
|
||||
done
|
||||
)
|
||||
done
|
||||
)
|
||||
done
|
||||
done
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
for r in $(find Binary -name '*.rpm'); do
|
||||
s=$(rpm -q --info --nosignature -p $r | grep '^Source RPM' | sed 's#^Source RPM ..##')
|
||||
n=$(rpm -q --nosignature --qf '%{NAME}' -p $(find Source -name $s) )
|
||||
grep "^$n$" $EXCLUDE_LIST
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "rm $r"
|
||||
git rm $r
|
||||
fi
|
||||
done
|
||||
|
||||
\rm -r $TMP_DIR
|
||||
|
||||
echo "REMINDER: inspect changes in $MY_REPO/cgcs-centos-repo and commit with ..."
|
||||
echo ' TIMESTAMP=$(date +"%Y-%m-%d")'
|
||||
echo ' git commit -m "JENKINS: repo update $TIMESTAMP"'
|
||||
|
||||
|
||||
94
build-tools/mirror_rebase/link_cgcs_centos_repo_5
Executable file
94
build-tools/mirror_rebase/link_cgcs_centos_repo_5
Executable file
@ -0,0 +1,94 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
#
|
||||
# Search for tis patched centos src.rpm's that have been upversioned
|
||||
#
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
# OLD_MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/mitaka
|
||||
OLD_MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
# MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
OLD_THIRD_MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/tis-r4-3rd-Party
|
||||
# THIRD_MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/tis-r4-3rd-Party
|
||||
# BIN_ROOT=$MIRROR_ROOT/Binary
|
||||
# SRC_ROOT=$MIRROR_ROOT/Source
|
||||
UPVERSION_LOG=$MY_WORKSPACE/upversion.log
|
||||
|
||||
REPO_DOWNLOADS_ROOT="$MY_REPO"
|
||||
NEW_MIRROR_ROOT="$MY_REPO/cgcs-centos-repo"
|
||||
THIRD_PARTY_ROOT="$MY_REPO/cgcs-3rd-party-repo"
|
||||
|
||||
|
||||
if [ -f $UPVERSION_LOG ]; then
|
||||
rm -f $UPVERSION_LOG
|
||||
fi
|
||||
|
||||
cd $MY_REPO
|
||||
|
||||
for g in $(find $MY_REPO -type d -name .git); do
|
||||
d=$(dirname $g)
|
||||
for pf in $(find $d -maxdepth 1 -name 'centos_pkg_dirs*'); do
|
||||
if [ -f $pf ]; then
|
||||
for p in $(cat $pf); do
|
||||
pkg_dir="$d/$p"
|
||||
sf="$pkg_dir/centos/srpm_path"
|
||||
if [ -f $sf ]; then
|
||||
for s in $(grep '^[^#]' $sf); do
|
||||
ORIG_SRPM_PATH=""
|
||||
# absolute path source rpms
|
||||
echo "$s" | grep "^/" >/dev/null && ORIG_SRPM_PATH=$s
|
||||
|
||||
if [ "${ORIG_SRPM_PATH}x" == "x" ]; then
|
||||
# handle repo: definitions
|
||||
echo "$s" | grep "^repo:" >/dev/null && ORIG_SRPM_PATH=$(echo $s | sed "s%^repo:%$REPO_DOWNLOADS_ROOT/%")
|
||||
fi
|
||||
|
||||
if [ "${ORIG_SRPM_PATH}x" == "x" ]; then
|
||||
# handle 3rd_party: definitions
|
||||
echo "$s" | grep "^3rd_party:" >/dev/null && ORIG_SRPM_PATH=$(echo $s | sed "s%^3rd_party:%$THIRD_PARTY_ROOT/%")
|
||||
fi
|
||||
|
||||
if [ "${ORIG_SRPM_PATH}x" == "x" ]; then
|
||||
# handle mirror: definitions
|
||||
# SAL TEMPORARY echo "$s" | grep "^mirror:" >/dev/null && ORIG_SRPM_PATH=`echo $s | sed "s%^mirror:%$MIRROR_ROOT/%"`
|
||||
echo "$s" | grep "^mirror:" >/dev/null && ORIG_SRPM_PATH=$(echo $s | sed "s%^mirror:%$NEW_MIRROR_ROOT/%" | sed "s#CentOS/tis-r4-CentOS/kilo/##" | sed "s#CentOS/tis-r4-CentOS/mitaka/##" | sed "s#CentOS/tis-r4-CentOS/newton/##")
|
||||
fi
|
||||
|
||||
if [ "${ORIG_SRPM_PATH}x" == "x" ]; then
|
||||
# we haven't found a valid prefix yet, so assume it's a legacy
|
||||
# file (mirror: interpretation)
|
||||
ORIG_SRPM_PATH="$NEW_MIRROR_ROOT/$s"
|
||||
fi
|
||||
|
||||
if [ ! -f $ORIG_SRPM_PATH ]; then
|
||||
b=$(basename "$ORIG_SRPM_PATH")
|
||||
old_srpm=$(find $OLD_MIRROR_ROOT $OLD_THIRD_MIRROR_ROOT -name $b | head -n 1)
|
||||
old_name=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $old_srpm)
|
||||
if [ "$old_name" == "" ]; then
|
||||
echo "FAILED to find name for '$b', ORIG_SRPM_PATH='$ORIG_SRPM_PATH'"
|
||||
exit 1
|
||||
fi
|
||||
NEW_SRPM_PATH=""
|
||||
for new_srpm in $(find $NEW_MIRROR_ROOT/Source $THIRD_PARTY_ROOT/Source -name "$old_name-[0-9]*.src.rpm"); do
|
||||
new_name=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $new_srpm)
|
||||
if [ "$new_name" == "$old_name" ]; then
|
||||
NEW_SRPM_PATH=$new_srpm
|
||||
break
|
||||
fi
|
||||
done
|
||||
nb=$(basename $NEW_SRPM_PATH)
|
||||
echo "FIX: '$sf' : '$b' -> '$nb'"
|
||||
echo "$old_name#$sf#$s#$b#$nb" >> $UPVERSION_LOG
|
||||
fi
|
||||
|
||||
done
|
||||
fi
|
||||
done
|
||||
fi
|
||||
done
|
||||
done
|
||||
|
||||
91
build-tools/mirror_rebase/link_cgcs_centos_repo_6
Executable file
91
build-tools/mirror_rebase/link_cgcs_centos_repo_6
Executable file
@ -0,0 +1,91 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
#
|
||||
# start an edit session for packages to be upgraded - pre upgrade version
|
||||
#
|
||||
|
||||
UPVERSION_LOG=$MY_WORKSPACE/upversion.log
|
||||
if [ "x$WORKING_BRANCH" == "x" ]; then
|
||||
WORKING_BRANCH=CGCS_DEV_0029_rebase_7_4
|
||||
fi
|
||||
|
||||
if [ ! -f $UPVERSION_LOG ]; then
|
||||
echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
|
||||
fi
|
||||
|
||||
|
||||
# One step back to see the old symlinks
|
||||
cd $MY_REPO/cgcs-3rd-party-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$(pwd)'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout HEAD^
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$(pwd)'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout HEAD^
|
||||
|
||||
#
|
||||
#
|
||||
#
|
||||
FAILED=""
|
||||
for dat in $(cat $UPVERSION_LOG); do
|
||||
name=$(echo $dat | awk -F '#' '{print $1}')
|
||||
srpm_path=$(echo $dat | awk -F '#' '{print $2}')
|
||||
old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
|
||||
new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
|
||||
|
||||
echo "$name $old_src_rpm $new_src_rpm"
|
||||
|
||||
if [ "$name" == "kernel" ]; then
|
||||
build-pkgs --std --edit --clean $name
|
||||
elif [ "$name" == "kernel-rt" ]; then
|
||||
build-pkgs --rt --edit --clean $name
|
||||
else
|
||||
build-pkgs --edit --clean $name
|
||||
fi
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed cmd 'build-pkgs --edit --clean $name'"
|
||||
FAILED="$name $FAILED"
|
||||
break
|
||||
fi
|
||||
echo "$? <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
|
||||
if [ "$name" == "kernel" ]; then
|
||||
build-pkgs --std --edit $name
|
||||
elif [ "$name" == "kernel-rt" ]; then
|
||||
build-pkgs --rt --edit $name
|
||||
else
|
||||
build-pkgs --edit $name
|
||||
fi
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed cmd 'build-pkgs --edit $name'"
|
||||
FAILED="$name $FAILED"
|
||||
break
|
||||
fi
|
||||
echo "$? <=<=<=<=<=<=<=<=<=<=<=<=<=<=<=<="
|
||||
done
|
||||
|
||||
cd $MY_REPO/cgcs-3rd-party-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
|
||||
if [ "$FAILED" != "" ]; then
|
||||
echo "Failed build-pkgs --edit for ... $FAILED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
84
build-tools/mirror_rebase/link_cgcs_centos_repo_7
Executable file
84
build-tools/mirror_rebase/link_cgcs_centos_repo_7
Executable file
@ -0,0 +1,84 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
#
|
||||
# Update srpm_path for packages to be upgraded
|
||||
#
|
||||
|
||||
UPVERSION_LOG=$MY_WORKSPACE/upversion.log
|
||||
if [ "x$ORIGIN_BRANCH" == "x" ]; then
|
||||
ORIGIN_BRANCH=CGCS_DEV_0029
|
||||
fi
|
||||
if [ "x$WORKING_BRANCH" == "x" ]; then
|
||||
WORKING_BRANCH=CGCS_DEV_0029_rebase_7_4
|
||||
fi
|
||||
|
||||
if [ ! -f $UPVERSION_LOG ]; then
|
||||
echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
|
||||
fi
|
||||
|
||||
|
||||
# One step back to see the old symlinks
|
||||
cd $MY_REPO
|
||||
|
||||
FAILED=""
|
||||
for dat in $(cat $UPVERSION_LOG); do
|
||||
name=$(echo $dat | awk -F '#' '{print $1}')
|
||||
srpm_path=$(echo $dat | awk -F '#' '{print $2}')
|
||||
old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
|
||||
new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
|
||||
|
||||
(
|
||||
cd $(dirname $srpm_path)
|
||||
CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
|
||||
if [ "$CURRENT_BRANCH" != "$WORKING_BRANCH" ]; then
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
git checkout $ORIGIN_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$ORIGIN_BRANCH' in directory '$(pwd)'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout -b $WORKING_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to 'git checkout -b $WORKING_BRANCH' from '$(pwd)'"
|
||||
exit 1
|
||||
else
|
||||
echo "created branch '$WORKING_BRANCH' at '$(pwd)'"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
sed -i "s#$old_src_rpm#$new_src_rpm#" $srpm_path
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: sed failed '$old_src_rpm' -> '$new_src_rpm'"
|
||||
exit 1
|
||||
else
|
||||
echo "updated $srpm_path: '$old_src_rpm' -> '$new_src_rpm'"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
)
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed while working on package '$name' at '$srpm_path'"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
for d in $(for dat in $(cat $UPVERSION_LOG); do srpm_path=$(echo $dat | awk -F '#' '{print $2}'); ( cd $(dirname $srpm_path); git rev-parse --show-toplevel ); done | sort --unique); do
|
||||
(
|
||||
cd $d
|
||||
echo "cd $d"
|
||||
for f in $(git status --porcelain | grep 'srpm_path$' | awk '{print $2}'); do
|
||||
echo "git add $f";
|
||||
done
|
||||
echo "git commit -m 'srpm_path updates for centos rebase'"
|
||||
)
|
||||
done
|
||||
echo ""
|
||||
65
build-tools/mirror_rebase/link_cgcs_centos_repo_8
Executable file
65
build-tools/mirror_rebase/link_cgcs_centos_repo_8
Executable file
@ -0,0 +1,65 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
#
|
||||
# Start an edit session for packages to be upgraded - post upgrade version
|
||||
#
|
||||
|
||||
UPVERSION_LOG=$MY_WORKSPACE/upversion.log
|
||||
if [ "x$WORKING_BRANCH" == "x" ]; then
|
||||
WORKING_BRANCH=CGCS_DEV_0029_rebase_7_4
|
||||
fi
|
||||
|
||||
if [ ! -f $UPVERSION_LOG ]; then
|
||||
echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
|
||||
fi
|
||||
|
||||
|
||||
# Restore new symlinks
|
||||
cd $MY_REPO/cgcs-3rd-party-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-3rd-party-repo'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
FAILED=""
|
||||
for dat in $(cat $UPVERSION_LOG); do
|
||||
name=$(echo $dat | awk -F '#' '{print $1}')
|
||||
srpm_path=$(echo $dat | awk -F '#' '{print $2}')
|
||||
old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
|
||||
new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
|
||||
|
||||
echo "$name $old_src_rpm $new_src_rpm"
|
||||
|
||||
if [ "$name" == "kernel" ]; then
|
||||
build-pkgs --std --edit $name --no-meta-patch
|
||||
elif [ "$name" == "kernel-rt" ]; then
|
||||
build-pkgs --rt --edit $name --no-meta-patch
|
||||
else
|
||||
build-pkgs --edit $name --no-meta-patch
|
||||
fi
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed cmd 'build-pkgs --edit $name'"
|
||||
FAILED="$name $FAILED"
|
||||
break
|
||||
fi
|
||||
echo "$? <=<=<=<=<=<=<=<=<=<=<=<=<=<=<=<="
|
||||
done
|
||||
|
||||
if [ "$FAILED" != "" ]; then
|
||||
echo "Failed build-pkgs --edit for ... $FAILED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
346
build-tools/mirror_rebase/link_cgcs_centos_repo_9
Executable file
346
build-tools/mirror_rebase/link_cgcs_centos_repo_9
Executable file
@ -0,0 +1,346 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
|
||||
#
|
||||
# Search for tis patched centos src.rpm's that have been upversioned
|
||||
#
|
||||
|
||||
UPVERSION_LOG=$MY_WORKSPACE/upversion.log
|
||||
if [ "x$WORKING_BRANCH" == "x" ]; then
|
||||
WORKING_BRANCH=CGCS_DEV_0029_rebase_7_4
|
||||
fi
|
||||
|
||||
if [ ! -f $UPVERSION_LOG ]; then
|
||||
echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
|
||||
fi
|
||||
|
||||
if [ "$DISPLAY" == "" ]; then
|
||||
echo "ERROR: X-Windows 'DISPLAY' variable not set. This script needs to open pop-up windows."
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# restore new symlinks
|
||||
cd $MY_REPO/cgcs-3rd-party-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-3rd-party-repo'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
FAILED=""
|
||||
build_types="std rt"
|
||||
for dat in $(cat $UPVERSION_LOG); do
|
||||
(
|
||||
name=$(echo $dat | awk -F '#' '{print $1}')
|
||||
srpm_path=$(echo $dat | awk -F '#' '{print $2}')
|
||||
old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
|
||||
new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
|
||||
|
||||
PKG_DIR=$(dirname $(dirname $srpm_path))
|
||||
OLD_BRANCH=$(echo $old_src_rpm | sed 's#[.]src[.]rpm$##')
|
||||
NEW_BRANCH=$(echo $new_src_rpm | sed 's#[.]src[.]rpm$##')
|
||||
|
||||
WORK_META_DIR=""
|
||||
for dd in $build_types; do
|
||||
WORK_META_DIR=$MY_WORKSPACE/$dd/srpm_work/$name/rpmbuild
|
||||
echo "WORK_META_DIR=$WORK_META_DIR"
|
||||
if [ -d $WORK_META_DIR ]; then
|
||||
break;
|
||||
else
|
||||
WORK_META_DIR=""
|
||||
fi
|
||||
done
|
||||
if [ "$WORK_META_DIR" == "" ]; then
|
||||
echo "ERROR: failed to find srpm_work directory for '$name'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# WORK_SRC_DIR=$(dirname $(find $MY_WORKSPACE/srpm_work/$name/gits/ -type d -name .git))
|
||||
NEW_WORK_SRC_DIR=""
|
||||
OLD_WORK_SRC_DIR=""
|
||||
for dd in $build_types; do
|
||||
for g in $(find $MY_WORKSPACE/$dd/srpm_work/$name/gits/ -type d -name .git); do
|
||||
d=$(dirname $g)
|
||||
if [ -d $d ]; then
|
||||
cd $d;
|
||||
git tag | grep pre_wrs_ >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
continue
|
||||
fi
|
||||
git checkout $OLD_BRANCH 2>> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
OLD_WORK_SRC_DIR=$d
|
||||
fi
|
||||
git checkout $NEW_BRANCH 2>> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
NEW_WORK_SRC_DIR=$d
|
||||
fi
|
||||
fi
|
||||
done
|
||||
done
|
||||
if [ "$WORK_META_DIR" == "" ]; then
|
||||
echo "ERROR: failed to find srpm_work directory for '$name'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "$name $old_src_rpm $new_src_rpm"
|
||||
echo "PKG_DIR=$PKG_DIR"
|
||||
echo "OLD_BRANCH=$OLD_BRANCH"
|
||||
echo "NEW_BRANCH=$NEW_BRANCH"
|
||||
echo "WORK_META_DIR=$WORK_META_DIR"
|
||||
echo "OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
|
||||
echo "NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
|
||||
echo ""
|
||||
|
||||
(
|
||||
cd $WORK_META_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to cd to WORK_META_DIR=$WORK_META_DIR"
|
||||
exit 1
|
||||
fi
|
||||
echo "--- old meta git log (oldest to newest) ---"
|
||||
git checkout $OLD_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
(git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit ; echo "") | sed '/^$/d' | tac
|
||||
PATCH_COMMIT_LIST=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
|
||||
echo "--- new meta git log (oldest to newest) ---"
|
||||
git checkout $NEW_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
(git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit ; echo "") | sed '/^$/d' | tac
|
||||
REFERENCE_COMMIT=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | head -n 1 | awk '{ print $2 }')
|
||||
echo ""
|
||||
|
||||
for COMMIT in ${PATCH_COMMIT_LIST}; do
|
||||
echo "git cherry-pick $COMMIT"
|
||||
git --no-pager show "$COMMIT"
|
||||
git cherry-pick "$COMMIT"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files"
|
||||
git status --porcelain | grep '^UU ' | awk '{ print $2}'
|
||||
echo "pwd=$(pwd)"
|
||||
# gitk &
|
||||
echo "git mergetool --no-prompt"
|
||||
git mergetool --no-prompt
|
||||
# for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
|
||||
# xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
|
||||
# if [ $? -ne 0 ]; then
|
||||
# echo "ERROR: problem launching editor on "
|
||||
# exit 1
|
||||
# fi
|
||||
# done
|
||||
echo "git cherry-pick --continue"
|
||||
git cherry-pick --continue
|
||||
fi
|
||||
done
|
||||
|
||||
PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
|
||||
exit 1
|
||||
fi
|
||||
for PATCH_FILE in ${PATCH_LIST}; do
|
||||
PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
|
||||
echo "$PATCH_FILE -> $PATCH_TARGET"
|
||||
N=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*" | wc -l)
|
||||
if [ $N -eq 1 ]; then
|
||||
PATCH_DEST=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*")
|
||||
echo "cp -f $PATCH_FILE $PATCH_DEST"
|
||||
\cp -f $PATCH_FILE $PATCH_DEST
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: copy failed $WORK_META_DIR/$PATCH_FILE -> $PATCH_DEST"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "ERROR: Don't know what destination file name to use for patch '$WORK_META_DIR/$PATCH_FILE' derived from commit $COMMIT, and to be copied to '$PKG_DIR/centos/meta_patches'"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo ""
|
||||
)
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
FAILED=$name
|
||||
break
|
||||
fi
|
||||
|
||||
(
|
||||
echo "--- old git log (oldest to newest) ---"
|
||||
cd $OLD_WORK_SRC_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to cd to OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout $OLD_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH in directory '$OLD_WORK_SRC_DIR'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
(git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit; echo "") | sed '/^$/d' | tac
|
||||
PATCH_COMMIT_LIST=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit; echo "") | sed '/^$/d' | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
|
||||
|
||||
echo "--- new git log (oldest to newest) ---"
|
||||
cd $NEW_WORK_SRC_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to cd to NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout $NEW_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH in directory '$NEW_WORK_SRC_DIR'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
(git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit; echo "") | sed '/^$/d' | tac
|
||||
REFERENCE_COMMIT=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | head -n 1 | awk '{ print $2 }')
|
||||
echo ""
|
||||
|
||||
if [ "$OLD_WORK_SRC_DIR" == "$NEW_WORK_SRC_DIR" ]; then
|
||||
for COMMIT in ${PATCH_COMMIT_LIST}; do
|
||||
echo "git cherry-pick $COMMIT"
|
||||
git --no-pager show "$COMMIT"
|
||||
git cherry-pick "$COMMIT"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files"
|
||||
git status --porcelain | grep '^UU ' | awk '{ print $2}'
|
||||
echo "pwd=$(pwd)"
|
||||
# gitk &
|
||||
echo "git mergetool --no-prompt"
|
||||
git mergetool --no-prompt
|
||||
# for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
|
||||
# xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
|
||||
# if [ $? -ne 0 ]; then
|
||||
# echo "ERROR: problem launching editor on "
|
||||
# exit 1
|
||||
# fi
|
||||
# done
|
||||
echo "git cherry-pick --continue"
|
||||
git cherry-pick --continue
|
||||
fi
|
||||
done
|
||||
else
|
||||
cd $OLD_WORK_SRC_DIR
|
||||
PATCH_LIST=$(git format-patch -n pre_wrs_$OLD_BRANCH)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=pre_wrs_$OLD_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
cd $NEW_WORK_SRC_DIR
|
||||
for PATCH_FILE in ${PATCH_LIST}; do
|
||||
echo "=== Apply $PATCH_FILE ==="
|
||||
cat $OLD_WORK_SRC_DIR/$PATCH_FILE
|
||||
cat $OLD_WORK_SRC_DIR/$PATCH_FILE | patch -p1
|
||||
if [ $? -ne 0 ]; then
|
||||
for REJECT in $(find . -name '*.rej'); do
|
||||
FILE_NAME=$(echo $REJECT | sed 's#.rej$##')
|
||||
cd $OLD_WORK_SRC_DIR
|
||||
gitk $FILE_NAME &
|
||||
cd $NEW_WORK_SRC_DIR
|
||||
if [ -f $FILE_NAME ] && [ -f $FILE_NAME.orig ]; then
|
||||
\cp -f $FILE_NAME.orig $FILE_NAME
|
||||
xterm -e "vi $FILE_NAME $REJECT"
|
||||
rm -f $REJECT
|
||||
rm -f $FILE_NAME.orig
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
git add --all
|
||||
MSG=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
|
||||
git commit -m "WRS: $MSG"
|
||||
done
|
||||
|
||||
fi
|
||||
|
||||
PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
|
||||
exit 1
|
||||
fi
|
||||
for PATCH_FILE in ${PATCH_LIST}; do
|
||||
PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch[0-9]*-//' | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch//' | sed 's/.patch$//')
|
||||
echo "$PATCH_FILE -> $PATCH_TARGET"
|
||||
PKG_PATCH_DIR="$PKG_DIR/centos/patches"
|
||||
N=0
|
||||
if [ -d "$PKG_PATCH_DIR" ]; then
|
||||
N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
|
||||
fi
|
||||
if [ $N -ne 1 ]; then
|
||||
PKG_PATCH_DIR="$PKG_DIR"
|
||||
if [ -d "$PKG_PATCH_DIR" ]; then
|
||||
N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
|
||||
fi
|
||||
fi
|
||||
echo "N=$N"
|
||||
echo "PKG_PATCH_DIR=$PKG_PATCH_DIR"
|
||||
|
||||
if [ $N -eq 1 ]; then
|
||||
PATCH_DEST=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]')
|
||||
echo "meld $PATCH_FILE -> $PATCH_DEST"
|
||||
meld $PATCH_FILE $PATCH_DEST
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: meld failed $WORK_SRC_DIR/$PATCH_FILE -> $PATCH_DEST"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "ERROR: Don't know what destination file name to use for patch '$OLD_WORK_SRC_DIR/$PATCH_FILE', and to be copied to '$PKG_PATCH_DIR'"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo ""
|
||||
)
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
FAILED=$name
|
||||
break
|
||||
fi
|
||||
|
||||
)
|
||||
|
||||
|
||||
done
|
||||
|
||||
if [ "$FAILED" != "" ]; then
|
||||
echo "Failed for ... $FAILED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
for d in $(for dat in $(cat $UPVERSION_LOG); do srpm_path=$(echo $dat | awk -F '#' '{print $2}'); ( cd $(dirname $srpm_path); git rev-parse --show-toplevel ); done | sort --unique); do
|
||||
(
|
||||
cd $d
|
||||
echo "cd $d"
|
||||
for f in $(git status --porcelain | awk '{print $2}'); do
|
||||
echo "git add $f";
|
||||
done
|
||||
if [ "$PATCH_ID" == "" ]; then
|
||||
echo "git commit -m 'rebased patches'"
|
||||
else
|
||||
echo "git commit -m 'rebased patches'"
|
||||
fi
|
||||
)
|
||||
done
|
||||
echo ""
|
||||
|
||||
|
||||
330
build-tools/mirror_rebase/tarball_upgrade
Executable file
330
build-tools/mirror_rebase/tarball_upgrade
Executable file
@ -0,0 +1,330 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Part of the monthly mirror update
|
||||
#
|
||||
# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r4-CentOS/newton
|
||||
#
|
||||
# Search for tis patched centos src.rpm's that have been upversioned
|
||||
#
|
||||
|
||||
UPVERSION_LOG=$MY_WORKSPACE/upversion.log
|
||||
if [ "x$WORKING_BRANCH" == "x" ]; then
|
||||
WORKING_BRANCH=CGCS_DEV_0026_may_rebase
|
||||
fi
|
||||
|
||||
if [ ! -f $UPVERSION_LOG ]; then
|
||||
echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
|
||||
fi
|
||||
|
||||
if [ "$DISPLAY" == "" ]; then
|
||||
echo "ERROR: X-Windows 'DISPLAY' variable not set. This script needs to open pop-up windows."
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# restore new symlinks
|
||||
cd $MY_REPO/cgcs-3rd-party-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-3rd-party-repo'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
FAILED=""
|
||||
build_types="std rt"
|
||||
for dat in $(cat $UPVERSION_LOG); do
|
||||
(
|
||||
name=$(echo $dat | awk -F '#' '{print $1}')
|
||||
srpm_path=$(echo $dat | awk -F '#' '{print $2}')
|
||||
old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
|
||||
new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
|
||||
|
||||
PKG_DIR=$(dirname $(dirname $srpm_path))
|
||||
OLD_BRANCH=$(echo $old_src_rpm | sed 's#[.]src[.]rpm$##')
|
||||
NEW_BRANCH=$(echo $new_src_rpm | sed 's#[.]src[.]rpm$##')
|
||||
|
||||
WORK_META_DIR=""
|
||||
for dd in $build_types; do
|
||||
WORK_META_DIR=$MY_WORKSPACE/$dd/srpm_work/$name/rpmbuild
|
||||
echo "WORK_META_DIR=$WORK_META_DIR"
|
||||
if [ -d $WORK_META_DIR ]; then
|
||||
break;
|
||||
else
|
||||
WORK_META_DIR=""
|
||||
fi
|
||||
done
|
||||
if [ "$WORK_META_DIR" == "" ]; then
|
||||
echo "ERROR: failed to find srpm_work directory for '$name'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# WORK_SRC_DIR=$(dirname $(find $MY_WORKSPACE/srpm_work/$name/gits/ -type d -name .git))
|
||||
NEW_WORK_SRC_DIR=""
|
||||
OLD_WORK_SRC_DIR=""
|
||||
for dd in $build_types; do
|
||||
for g in $(find $MY_WORKSPACE/$dd/srpm_work/$name/gits/ -type d -name .git); do
|
||||
d=$(dirname $g)
|
||||
if [ -d $d ]; then
|
||||
cd $d;
|
||||
git tag | grep pre_wrs_ >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
continue
|
||||
fi
|
||||
git checkout $OLD_BRANCH 2>> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
OLD_WORK_SRC_DIR=$d
|
||||
fi
|
||||
git checkout $NEW_BRANCH 2>> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
NEW_WORK_SRC_DIR=$d
|
||||
fi
|
||||
fi
|
||||
done
|
||||
done
|
||||
if [ "$WORK_META_DIR" == "" ]; then
|
||||
echo "ERROR: failed to find srpm_work directory for '$name'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "$name $old_src_rpm $new_src_rpm"
|
||||
echo "PKG_DIR=$PKG_DIR"
|
||||
echo "OLD_BRANCH=$OLD_BRANCH"
|
||||
echo "NEW_BRANCH=$NEW_BRANCH"
|
||||
echo "WORK_META_DIR=$WORK_META_DIR"
|
||||
echo "OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
|
||||
echo "NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
|
||||
echo ""
|
||||
|
||||
(
|
||||
cd $WORK_META_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to cd to WORK_META_DIR=$WORK_META_DIR"
|
||||
exit 1
|
||||
fi
|
||||
echo "--- old meta git log (oldest to newest) ---"
|
||||
git checkout $OLD_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
(git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit ; echo "") | sed '/^$/d' | tac
|
||||
PATCH_COMMIT_LIST=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
|
||||
echo "--- new meta git log (oldest to newest) ---"
|
||||
git checkout $NEW_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
(git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit ; echo "") | sed '/^$/d' | tac
|
||||
REFERENCE_COMMIT=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | head -n 1 | awk '{ print $2 }')
|
||||
echo ""
|
||||
|
||||
for COMMIT in ${PATCH_COMMIT_LIST}; do
|
||||
echo "git cherry-pick $COMMIT"
|
||||
git --no-pager show "$COMMIT"
|
||||
git cherry-pick "$COMMIT"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files"
|
||||
git status --porcelain | grep '^UU ' | awk '{ print $2}'
|
||||
echo "pwd=$(pwd)"
|
||||
# gitk &
|
||||
echo "git mergetool --no-prompt"
|
||||
git mergetool --no-prompt
|
||||
# for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
|
||||
# xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
|
||||
# if [ $? -ne 0 ]; then
|
||||
# echo "ERROR: problem launching editor on "
|
||||
# exit 1
|
||||
# fi
|
||||
# done
|
||||
echo "git cherry-pick --continue"
|
||||
git cherry-pick --continue
|
||||
fi
|
||||
done
|
||||
|
||||
PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
|
||||
exit 1
|
||||
fi
|
||||
for PATCH_FILE in ${PATCH_LIST}; do
|
||||
PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
|
||||
echo "$PATCH_FILE -> $PATCH_TARGET"
|
||||
N=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*" | wc -l)
|
||||
if [ $N -eq 1 ]; then
|
||||
PATCH_DEST=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*")
|
||||
echo "cp -f $PATCH_FILE $PATCH_DEST"
|
||||
\cp -f $PATCH_FILE $PATCH_DEST
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: copy failed $WORK_META_DIR/$PATCH_FILE -> $PATCH_DEST"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "ERROR: Don't know what destination file name to use for patch '$WORK_META_DIR/$PATCH_FILE' derived from commit $COMMIT, and to be copied to '$PKG_DIR/centos/meta_patches'"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo ""
|
||||
)
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
FAILED=$name
|
||||
break
|
||||
fi
|
||||
|
||||
(
|
||||
echo "--- old git log (oldest to newest) ---"
|
||||
cd $OLD_WORK_SRC_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to cd to OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout $OLD_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH in directory '$OLD_WORK_SRC_DIR'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
(git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit; echo "") | sed '/^$/d' | tac
|
||||
PATCH_COMMIT_LIST=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit; echo "") | sed '/^$/d' | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
|
||||
|
||||
echo "--- new git log (oldest to newest) ---"
|
||||
cd $NEW_WORK_SRC_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to cd to NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout $NEW_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH in directory '$NEW_WORK_SRC_DIR'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
(git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit; echo "") | sed '/^$/d' | tac
|
||||
REFERENCE_COMMIT=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | head -n 1 | awk '{ print $2 }')
|
||||
echo ""
|
||||
|
||||
if [ "$OLD_WORK_SRC_DIR" == "$NEW_WORK_SRC_DIR" ]; then
|
||||
for COMMIT in ${PATCH_COMMIT_LIST}; do
|
||||
echo "git cherry-pick $COMMIT"
|
||||
git --no-pager show "$COMMIT"
|
||||
git cherry-pick "$COMMIT"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files"
|
||||
git status --porcelain | grep '^UU ' | awk '{ print $2}'
|
||||
echo "pwd=$(pwd)"
|
||||
# gitk &
|
||||
echo "git mergetool --no-prompt"
|
||||
git mergetool --no-prompt
|
||||
# for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
|
||||
# xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
|
||||
# if [ $? -ne 0 ]; then
|
||||
# echo "ERROR: problem launching editor on "
|
||||
# exit 1
|
||||
# fi
|
||||
# done
|
||||
echo "git cherry-pick --continue"
|
||||
git cherry-pick --continue
|
||||
fi
|
||||
done
|
||||
else
|
||||
cd $OLD_WORK_SRC_DIR
|
||||
PATCH_LIST=$(git format-patch -n pre_wrs_$OLD_BRANCH)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=pre_wrs_$OLD_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
cd $NEW_WORK_SRC_DIR
|
||||
for PATCH_FILE in ${PATCH_LIST}; do
|
||||
echo "=== Apply $PATCH_FILE ==="
|
||||
cat $OLD_WORK_SRC_DIR/$PATCH_FILE
|
||||
cat $OLD_WORK_SRC_DIR/$PATCH_FILE | patch -p1
|
||||
if [ $? -ne 0 ]; then
|
||||
for REJECT in $(find . -name '*.rej'); do
|
||||
FILE_NAME=$(echo $REJECT | sed 's#.rej$##')
|
||||
cd $OLD_WORK_SRC_DIR
|
||||
gitk $FILE_NAME &
|
||||
cd $NEW_WORK_SRC_DIR
|
||||
if [ -f $FILE_NAME ] && [ -f $FILE_NAME.orig ]; then
|
||||
\cp -f $FILE_NAME.orig $FILE_NAME
|
||||
xterm -e "vi $FILE_NAME $REJECT"
|
||||
rm -f $REJECT
|
||||
rm -f $FILE_NAME.orig
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
git add --all
|
||||
MSG=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
|
||||
git commit -m "WRS: $MSG"
|
||||
done
|
||||
|
||||
fi
|
||||
|
||||
PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
|
||||
exit 1
|
||||
fi
|
||||
for PATCH_FILE in ${PATCH_LIST}; do
|
||||
PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch[0-9]*-//' | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch//' | sed 's/.patch$//')
|
||||
echo "$PATCH_FILE -> $PATCH_TARGET"
|
||||
PKG_PATCH_DIR="$PKG_DIR/centos/patches"
|
||||
N=0
|
||||
if [ -d "$PKG_PATCH_DIR" ]; then
|
||||
N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
|
||||
fi
|
||||
if [ $N -ne 1 ]; then
|
||||
PKG_PATCH_DIR="$PKG_DIR"
|
||||
if [ -d "$PKG_PATCH_DIR" ]; then
|
||||
N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
|
||||
fi
|
||||
fi
|
||||
echo "N=$N"
|
||||
echo "PKG_PATCH_DIR=$PKG_PATCH_DIR"
|
||||
|
||||
if [ $N -eq 1 ]; then
|
||||
PATCH_DEST=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]')
|
||||
echo "meld $PATCH_FILE -> $PATCH_DEST"
|
||||
meld $PATCH_FILE $PATCH_DEST
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: meld failed $WORK_SRC_DIR/$PATCH_FILE -> $PATCH_DEST"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "ERROR: Don't know what destination file name to use for patch '$OLD_WORK_SRC_DIR/$PATCH_FILE', and to be copied to '$PKG_PATCH_DIR'"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo ""
|
||||
)
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
FAILED=$name
|
||||
break
|
||||
fi
|
||||
|
||||
)
|
||||
|
||||
|
||||
done
|
||||
|
||||
if [ "$FAILED" != "" ]; then
|
||||
echo "Failed for ... $FAILED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
|
||||
31
build-tools/mk/_sign_pkgs.mk
Normal file
31
build-tools/mk/_sign_pkgs.mk
Normal file
@ -0,0 +1,31 @@
|
||||
|
||||
#
|
||||
# this makefile is used by the build-iso process to add file signature to all rpms
|
||||
#
|
||||
# it requires a private key, passed as the variable KEY
|
||||
|
||||
PKGS_LIST := $(wildcard *.rpm)
|
||||
|
||||
# we need to skip the signature of some packages that
|
||||
# might be installed in file systems that do not support extended attributes
|
||||
# in the case of shim- and grub2-efi-, the UEFI configuration installs them in a VFAT file system
|
||||
PKGS_TO_SKIP := $(wildcard grub2-efi-[0-9]*.x86_64.rpm shim-[0-9]*.x86_64.rpm)
|
||||
|
||||
PKGS_TO_SIGN = $(filter-out $(PKGS_TO_SKIP),$(PKGS_LIST))
|
||||
|
||||
define _pkg_sign_tmpl
|
||||
|
||||
_sign_$1 :
|
||||
@ rpmsign --signfiles --fskpath=$(KEY) $1
|
||||
@ chown mockbuild $1
|
||||
@ chgrp users $1
|
||||
|
||||
sign : _sign_$1
|
||||
|
||||
endef
|
||||
|
||||
sign :
|
||||
@echo signed all packages
|
||||
|
||||
$(foreach file,$(PKGS_TO_SIGN),$(eval $(call _pkg_sign_tmpl,$(file))))
|
||||
|
||||
9
build-tools/mock_cfg_to_yum_conf.py
Executable file
9
build-tools/mock_cfg_to_yum_conf.py
Executable file
@ -0,0 +1,9 @@
|
||||
#!/usr/bin/python
|
||||
|
||||
import sys
|
||||
|
||||
FN=sys.argv[1]
|
||||
variables={}
|
||||
variables['config_opts']={}
|
||||
execfile( FN, variables )
|
||||
print variables['config_opts']['yum.conf']
|
||||
1207
build-tools/mockchain-parallel
Executable file
1207
build-tools/mockchain-parallel
Executable file
File diff suppressed because it is too large
Load Diff
130
build-tools/modify-build-cfg
Executable file
130
build-tools/modify-build-cfg
Executable file
@ -0,0 +1,130 @@
|
||||
#!/bin/sh
|
||||
|
||||
# This script modifies a mock configuration file (typically $MY_BUILD_CFG)
|
||||
# to add build time environment variables to the mock environment (things
|
||||
# like what branch we're building on, etc).
|
||||
#
|
||||
# For reasons of security, the host environment variables cannot normally be
|
||||
# passed through to the mock environment, so this scripts sets the variables
|
||||
# to literal values.
|
||||
#
|
||||
# usage: modify-build-cfg [file.cfg]
|
||||
#
|
||||
|
||||
MOCK_CFG_PROTO="$MY_REPO/cgcs-centos-repo/mock.cfg.proto"
|
||||
if [ ! -f "$MOCK_CFG_PROTO" ]; then
|
||||
echo "ERROR: Couldn't find mock config prototype at '$MOCK_CFG_PROTO'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "${1}x" == "x" ]; then
|
||||
FILE=$MY_BUILD_CFG
|
||||
else
|
||||
FILE=$1
|
||||
fi
|
||||
|
||||
if [ -f $MOCK_CFG_PROTO ]; then
|
||||
if [ -f $FILE ]; then
|
||||
NEWER=$(find "$MOCK_CFG_PROTO" -newer "$FILE")
|
||||
if [ "x$NEWER" != "x" ]; then
|
||||
\rm -f -v "$FILE"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -f $FILE ]; then
|
||||
if [ -z $MY_BUILD_ENVIRONMENT ] || [ -z $MY_BUILD_DIR ] || [ -z $MY_REPO ]; then
|
||||
echo "Can't create $FILE without MY_BUILD_ENVIRONMENT, MY_BUILD_DIR and MY_REPO environment variables"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Recreating $FILE"
|
||||
\cp -f -v "$MOCK_CFG_PROTO" "$FILE"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Couldn't find config file '$FILE', nor construct it from '$MOCK_CFG_PROTO'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sed -i "s%LOCAL_BASE%http://127.0.0.1:8088%g" "$FILE"
|
||||
sed -i "s%MIRROR_BASE%http://127.0.0.1:8088%g" "$FILE"
|
||||
sed -i "s%BUILD_ENV%$MY_BUILD_ENVIRONMENT%g" "$FILE"
|
||||
sed -i "s%/MY_BUILD_DIR%$MY_BUILD_DIR_TOP%g" "$FILE"
|
||||
sed -i "s%/MY_REPO_DIR%$MY_REPO%g" "$FILE"
|
||||
|
||||
# Disable all local-* repos for the build-types other than the current one
|
||||
for bt in std rt; do
|
||||
if [ "$bt" != "$BUILD_TYPE" ]; then
|
||||
# Use the range of lines starting with pattern [local-$bt] until the next line starting with []
|
||||
sed -i "/^\[local-$bt\]/,/^\[/ s/enabled=1/enabled=0/" $FILE
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
|
||||
# Add environment variables to mock config if they don't exist
|
||||
grep -q "config_opts\['environment'\]\['BUILD_BY'\]" $FILE || \
|
||||
echo "config_opts['environment']['BUILD_BY']" >> $FILE
|
||||
|
||||
grep -q "config_opts\['environment'\]\['BUILD_DATE'\]" $FILE || \
|
||||
echo "config_opts['environment']['BUILD_DATE']" >> $FILE
|
||||
|
||||
grep -q "config_opts\['environment'\]\['REPO'\]" $FILE || \
|
||||
echo "config_opts['environment']['REPO']" >> $FILE
|
||||
|
||||
grep -q "config_opts\['environment'\]\['WRS_GIT_BRANCH'\]" $FILE || \
|
||||
echo "config_opts['environment']['WRS_GIT_BRANCH']" >> $FILE
|
||||
|
||||
grep -q "config_opts\['environment'\]\['CGCS_GIT_BRANCH'\]" $FILE || \
|
||||
echo "config_opts['environment']['CGCS_GIT_BRANCH']" >> $FILE
|
||||
|
||||
if [ -z $FORMAL_BUILD ]; then
|
||||
grep -q "config_opts\['macros'\]\['%_no_cgcs_license_check'\] = '1'" $FILE || \
|
||||
echo "config_opts['macros']['%_no_cgcs_license_check'] = '1'" >> $FILE
|
||||
else
|
||||
sed -i "/config_opts\['macros'\]\['%_no_cgcs_license_check'\] = '1'/d" $FILE
|
||||
fi
|
||||
|
||||
grep -q "config_opts\['macros'\]\['%_tis_build_type'\] = '$BUILD_TYPE'" $FILE || \
|
||||
echo "config_opts['macros']['%_tis_build_type'] = '$BUILD_TYPE'" >> $FILE
|
||||
|
||||
if [ -f /usr/lib64/nosync/nosync.so ]; then
|
||||
grep -q "config_opts\['nosync'\] = True" $FILE || \
|
||||
echo "config_opts['nosync'] = True" >> $FILE
|
||||
fi
|
||||
|
||||
grep -q "config_opts\['chroot_setup_cmd'\] = 'install @buildsys-build pigz lbzip2 yum'" $FILE || \
|
||||
echo "config_opts['chroot_setup_cmd'] = 'install @buildsys-build pigz lbzip2 yum'" >> $FILE
|
||||
|
||||
#
|
||||
# Read macros from tis.macros to add to the build config file,
|
||||
# for use in RPM spec files
|
||||
#
|
||||
RPM_MACROS=$MY_REPO/build-tools/tis.macros
|
||||
sed 's/#.*//' $RPM_MACROS | grep '=' | while IFS='=' read name value; do
|
||||
# Check if the entry already exists. If so, go to next line
|
||||
grep -q "^config_opts\['macros'\]\['${name}'\] = '${value}'$" $FILE && continue
|
||||
|
||||
# Update or add the entry
|
||||
grep -q "^config_opts\['macros'\]\['${name}'\]" $FILE
|
||||
if [ $? -eq 0 ]; then
|
||||
sed -i -r "s#^(config_opts\['macros'\]\['${name}'\]).*#\1 = '${value}'#" $FILE
|
||||
else
|
||||
echo "config_opts['macros']['${name}'] = '${value}'" >> $FILE
|
||||
fi
|
||||
done
|
||||
|
||||
# okay, now we have lines for each env var. Generate the correct values
|
||||
|
||||
BUILD_DATE=`date "+%F %T %z"`
|
||||
CGCS_GIT_BRANCH=`cd $MY_REPO/addons/wr-cgcs/layers/cgcs/; git rev-parse --abbrev-ref HEAD`
|
||||
WRS_GIT_BRANCH=`cd $MY_REPO; git rev-parse --abbrev-ref HEAD`
|
||||
REPO=$MY_REPO
|
||||
|
||||
# Finally, our good friend sed will place the values in the mock config file
|
||||
sed -i \
|
||||
-e "s#config_opts\['environment'\]\['BUILD_BY'\].*#config_opts\['environment'\]\['BUILD_BY'\] = '$USER'#" \
|
||||
-e "s#config_opts\['environment'\]\['BUILD_DATE'\].*#config_opts\['environment'\]\['BUILD_DATE'\] = '$BUILD_DATE'#" \
|
||||
-e "s#config_opts\['environment'\]\['REPO'\].*#config_opts\['environment'\]\['REPO'\] = '$REPO'#" \
|
||||
-e "s#config_opts\['environment'\]\['WRS_GIT_BRANCH'\].*#config_opts\['environment'\]\['WRS_GIT_BRANCH'\] = '$WRS_GIT_BRANCH'#" \
|
||||
-e "s#config_opts\['environment'\]\['CGCS_GIT_BRANCH'\].*#config_opts\['environment'\]\['CGCS_GIT_BRANCH'\] = '$CGCS_GIT_BRANCH'#" \
|
||||
$FILE
|
||||
320
build-tools/patch-iso
Executable file
320
build-tools/patch-iso
Executable file
@ -0,0 +1,320 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Utility for adding patches to an unpatched ISO
|
||||
#
|
||||
|
||||
if [ -z "${MY_REPO}" ]; then
|
||||
echo "Required environment variable MY_REPO is not set"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
SETUP_PATCH_REPO=${MY_REPO}/addons/wr-cgcs/layers/cgcs/extras.ND/scripts/setup_patch_repo.sh
|
||||
if [ ! -x ${SETUP_PATCH_REPO} ]; then
|
||||
echo "Cannot find or execute ${SETUP_PATCH_REPO}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
REPO_UPGRADES_DIR=${MY_REPO}/addons/wr-cgcs/layers/cgcs/common-bsp/files/upgrades
|
||||
RELEASE_INFO=${MY_REPO}/addons/wr-cgcs/layers/cgcs/middleware/recipes-common/build-info/release-info.inc
|
||||
PLATFORM_RELEASE=$(source $RELEASE_INFO && echo $PLATFORM_RELEASE)
|
||||
|
||||
function usage() {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " $(basename $0) -i <input bootimage.iso> -o <output bootimage.iso> [ -u ] <patch> ..."
|
||||
echo " -i <file>: Specify input ISO file"
|
||||
echo " -o <file>: Specify output ISO file"
|
||||
echo " -u : Update with upgrades files from ${REPO_UPGRADES_DIR}"
|
||||
echo ""
|
||||
}
|
||||
|
||||
function extract_pkg_from_patch_repo() {
|
||||
local repodir=${BUILDDIR}/patches
|
||||
local pkgname=$1
|
||||
|
||||
local pkgfile=$(repoquery --repofrompath local,${repodir} --location -q ${pkgname})
|
||||
if [ -z "${pkgfile}" ]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
rpm2cpio ${pkgfile/file://} | cpio -idmv
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to extract $pkgname files from ${pkgfile/file://}"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
declare INPUT_ISO=
|
||||
declare OUTPUT_ISO=
|
||||
declare ORIG_PWD=$PWD
|
||||
declare DO_UPGRADES=1
|
||||
|
||||
while getopts "i:o:u" opt; do
|
||||
case $opt in
|
||||
i)
|
||||
INPUT_ISO=$OPTARG
|
||||
;;
|
||||
o)
|
||||
OUTPUT_ISO=$OPTARG
|
||||
;;
|
||||
u)
|
||||
DO_UPGRADES=0
|
||||
;;
|
||||
*)
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ -z "$INPUT_ISO" -o -z "$OUTPUT_ISO" ]; then
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -f ${INPUT_ISO} ]; then
|
||||
echo "Input file does not exist: ${INPUT_ISO}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -f ${OUTPUT_ISO} ]; then
|
||||
echo "Output file already exists: ${OUTPUT_ISO}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
shift $((OPTIND-1))
|
||||
|
||||
if [ $# -le 0 ]; then
|
||||
usage
|
||||
exit
|
||||
fi
|
||||
|
||||
for pf in $@; do
|
||||
if [ ! -f $pf ]; then
|
||||
echo "Patch file $pf does not exist"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ ! $pf =~ \.patch$ ]]; then
|
||||
echo "Specified file $pf does not have .patch extension"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
declare MNTDIR=
|
||||
declare BUILDDIR=
|
||||
declare WORKDIR=
|
||||
|
||||
function cleanup() {
|
||||
if [ -n "$MNTDIR" -a -d "$MNTDIR" ]; then
|
||||
guestunmount $MNTDIR
|
||||
\rmdir $MNTDIR
|
||||
fi
|
||||
|
||||
if [ -n "$BUILDDIR" -a -d "$BUILDDIR" ]; then
|
||||
\rm -rf $BUILDDIR
|
||||
fi
|
||||
|
||||
if [ -n "$WORKDIR" -a -d "$WORKDIR" ]; then
|
||||
\rm -rf $WORKDIR
|
||||
fi
|
||||
}
|
||||
|
||||
trap cleanup EXIT
|
||||
|
||||
MNTDIR=$(mktemp -d -p $PWD patchiso_mnt_XXXXXX)
|
||||
if [ -z "${MNTDIR}" -o ! -d ${MNTDIR} ]; then
|
||||
echo "Failed to create mntdir. Aborting..."
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
BUILDDIR=$(mktemp -d -p $PWD patchiso_build_XXXXXX)
|
||||
if [ -z "${BUILDDIR}" -o ! -d ${BUILDDIR} ]; then
|
||||
echo "Failed to create builddir. Aborting..."
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
# Mount the ISO
|
||||
guestmount -a ${INPUT_ISO} -m /dev/sda1 --ro ${MNTDIR}
|
||||
rc=$?
|
||||
if [ $rc -ne 0 ]; then
|
||||
echo "Call to guestmount failed with rc=$rc. Aborting..."
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
rsync -a ${MNTDIR}/ ${BUILDDIR}/
|
||||
rc=$?
|
||||
if [ $rc -ne 0 ]; then
|
||||
echo "Call to rsync ISO content. Aborting..."
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
guestunmount ${MNTDIR}
|
||||
\rmdir ${MNTDIR}
|
||||
|
||||
# Setup the patch repo
|
||||
${SETUP_PATCH_REPO} -o ${BUILDDIR}/patches $@
|
||||
rc=$?
|
||||
if [ $rc -ne 0 ]; then
|
||||
echo "Call to $(basename ${SETUP_PATCH_REPO}) failed with rc=$rc. Aborting..."
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
# Look for components that need modification
|
||||
#extract_pkg_from_patch_repo
|
||||
WORKDIR=$(mktemp -d -p $PWD patchiso_work_XXXXXX)
|
||||
if [ -z "${WORKDIR}" -o ! -d ${WORKDIR} ]; then
|
||||
echo "Failed to create workdir. Aborting..."
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
\cd ${WORKDIR}
|
||||
\mkdir extract
|
||||
\cd extract
|
||||
|
||||
# Changes to copied files here must also be reflected in build-iso
|
||||
|
||||
extract_pkg_from_patch_repo platform-kickstarts
|
||||
if [ $? -eq 0 ]; then
|
||||
# Replace files
|
||||
\rm -f ${BUILDDIR}/*ks.cfg &&
|
||||
\cp --preserve=all www/pages/feed/rel-*/*.cfg ${BUILDDIR}/ &&
|
||||
\cp --preserve=all ${BUILDDIR}/controller_ks.cfg ${BUILDDIR}/ks.cfg
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to copy extracted kickstarts"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
\cd ${WORKDIR}
|
||||
\rm -rf extract
|
||||
|
||||
\mkdir extract
|
||||
\cd extract
|
||||
extract_pkg_from_patch_repo platform-kickstarts-pxeboot
|
||||
if [ $? -eq 0 ]; then
|
||||
# Replace files
|
||||
\rm -f ${BUILDDIR}/pxeboot/pxeboot_controller.cfg \
|
||||
${BUILDDIR}/pxeboot/pxeboot_smallsystem.cfg \
|
||||
${BUILDDIR}/pxeboot/pxeboot_smallsystem_lowlatency.cfg &&
|
||||
\cp --preserve=all pxeboot/* ${BUILDDIR}/pxeboot/
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to copy extracted pxeboot kickstarts"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
\cd ${WORKDIR}
|
||||
\rm -rf extract
|
||||
|
||||
\mkdir extract
|
||||
\cd extract
|
||||
extract_pkg_from_patch_repo pxe-network-installer
|
||||
if [ $? -eq 0 ]; then
|
||||
# Replace files
|
||||
\rm -f ${BUILDDIR}/pxeboot/pxelinux.0 \
|
||||
${BUILDDIR}/pxeboot/menu.c32 \
|
||||
${BUILDDIR}/pxeboot/chain.c32 &&
|
||||
\cp --preserve=all pxeboot/pxelinux.0 pxeboot/menu.c32 pxeboot/chain.c32 ${BUILDDIR}/pxeboot/
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Error: Could not copy all files from installer"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
for f in pxeboot/EFI/centos/x86_64-efi/*; do
|
||||
\rm -f ${BUILDDIR}/${f}
|
||||
done
|
||||
\cp --preserve=all pxeboot/EFI/centos/x86_64-efi/* ${BUILDDIR}/pxeboot/EFI/centos/x86_64-efi/
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Error: Could not copy all files from installer"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
\rm -f ${BUILDDIR}/LiveOS/squashfs.img &&
|
||||
\cp --preserve=all www/pages/feed/rel-*/LiveOS/squashfs.img ${BUILDDIR}/LiveOS/
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Error: Could not copy squashfs from LiveOS"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Replace vmlinuz and initrd.img with our own pre-built ones
|
||||
\rm -f \
|
||||
${BUILDDIR}/vmlinuz \
|
||||
${BUILDDIR}/images/pxeboot/vmlinuz \
|
||||
${BUILDDIR}/initrd.img \
|
||||
${BUILDDIR}/images/pxeboot/initrd.img &&
|
||||
\cp --preserve=all pxeboot/rel-*/installer-bzImage_1.0 \
|
||||
${BUILDDIR}/vmlinuz &&
|
||||
\cp --preserve=all pxeboot/rel-*/installer-bzImage_1.0 \
|
||||
${BUILDDIR}/images/pxeboot/vmlinuz &&
|
||||
\cp --preserve=all pxeboot/rel-*/installer-intel-x86-64-initrd_1.0 \
|
||||
${BUILDDIR}/initrd.img &&
|
||||
\cp --preserve=all pxeboot/rel-*/installer-intel-x86-64-initrd_1.0 \
|
||||
${BUILDDIR}/images/pxeboot/initrd.img
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Error: Failed to copy installer images"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
\cd ${WORKDIR}
|
||||
\rm -rf extract
|
||||
|
||||
\mkdir extract
|
||||
\cd extract
|
||||
extract_pkg_from_patch_repo grub2-efi-pxeboot
|
||||
if [ $? -eq 0 ]; then
|
||||
# Replace files
|
||||
\rm -f ${BUILDDIR}/pxeboot/EFI/grubx64.efi &&
|
||||
\cp --preserve=all pxeboot/EFI/grubx64.efi ${BUILDDIR}/pxeboot/EFI/
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Error: Failed to copy grub2-efi-pxeboot files"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
\cd ${WORKDIR}
|
||||
\rm -rf extract
|
||||
|
||||
\cd ${ORIG_PWD}
|
||||
|
||||
if [ ${DO_UPGRADES} -eq 0 ]; then
|
||||
# Changes to copied files here must also be reflected in build-iso
|
||||
|
||||
echo "Updating upgrade support files"
|
||||
ISO_UPGRADES_DIR="${BUILDDIR}/upgrades"
|
||||
\rm -rf ${ISO_UPGRADES_DIR}
|
||||
\mkdir ${ISO_UPGRADES_DIR}
|
||||
\cp ${REPO_UPGRADES_DIR}/* ${ISO_UPGRADES_DIR}
|
||||
sed -i "s/xxxSW_VERSIONxxx/${PLATFORM_RELEASE}/g" ${ISO_UPGRADES_DIR}/metadata.xml
|
||||
chmod +x ${ISO_UPGRADES_DIR}/*.sh
|
||||
# Write the version out (used in upgrade scripts - this is the same as SW_VERSION)
|
||||
echo "VERSION=$PLATFORM_RELEASE" > ${ISO_UPGRADES_DIR}/version
|
||||
fi
|
||||
|
||||
# Rebuild the ISO
|
||||
mkisofs -o ${OUTPUT_ISO} \
|
||||
-R -D -A 'oe_iso_boot' -V 'oe_iso_boot' \
|
||||
-quiet \
|
||||
-b isolinux.bin -c boot.cat -no-emul-boot \
|
||||
-boot-load-size 4 -boot-info-table \
|
||||
-eltorito-alt-boot \
|
||||
-e images/efiboot.img \
|
||||
-no-emul-boot \
|
||||
${BUILDDIR}
|
||||
|
||||
isohybrid --uefi ${OUTPUT_ISO}
|
||||
implantisomd5 ${OUTPUT_ISO}
|
||||
|
||||
# Sign the .iso with the developer private key
|
||||
# Signing with the formal key is only to be done for customer release
|
||||
# and is a manual step afterwards, as with the GA ISO
|
||||
openssl dgst -sha256 \
|
||||
-sign ${MY_REPO}/build-tools/signing/dev-private-key.pem \
|
||||
-binary \
|
||||
-out ${OUTPUT_ISO/%.iso/.sig} \
|
||||
${OUTPUT_ISO}
|
||||
rc=$?
|
||||
if [ $rc -ne 0 ]; then
|
||||
echo "Call to $(basename ${SETUP_PATCH_REPO}) failed with rc=$rc. Aborting..."
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
echo "Patched ISO: ${OUTPUT_ISO}"
|
||||
|
||||
130
build-tools/patch_rebase_1
Executable file
130
build-tools/patch_rebase_1
Executable file
@ -0,0 +1,130 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Start an edit session for packages to be upgraded - pre upgrade version
|
||||
#
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Step 1: Start an edit session for packages to be upgraded - pre upgrade version"
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " patch_rebase_1 [--origin_branch <branch>] [--working_branch <branch>] [--upversion_data <file>]"
|
||||
echo ""
|
||||
echo "Assumes cgcs-centos-repo already has a working_branch commit that sets the new symlinks."
|
||||
echo ""
|
||||
echo "The upversion_data file has data on all the src.rpm being updated in the format:"
|
||||
echo " export UPVERSION_DATA=$MY_WORKSPACE/upversion.log"
|
||||
echo " PKG=lighttpd"
|
||||
echo " OLD_SRC_RPM=lighttpd-1.4.41-1.el7.src.rpm"
|
||||
echo " NEW_SRC_RPM=lighttpd-1.4.41-2.el7.src.rpm"
|
||||
echo " SRPM_PATH=$MY_REPO/addons/wr-cgcs/layers/cgcs/recipes-extended/lighttpd/centos/srpm_path"
|
||||
echo " echo \"\$PKG#\$SRPM_PATH##\$OLD_SRC_RPM#\$NEW_SRC_RPM\" > UPVERSION_DATA"
|
||||
echo ""
|
||||
}
|
||||
|
||||
TEMP=`getopt -o h --long origin_branch:,working_branch:,upversion_data:,help -n 'test.sh' -- "$@"`
|
||||
eval set -- "$TEMP"
|
||||
|
||||
ORIGIN_BRANCH=""
|
||||
WORKING_BRANCH=""
|
||||
UPVERSION_LOG=""
|
||||
HELP=0
|
||||
|
||||
while true ; do
|
||||
case "$1" in
|
||||
--origin_branch) shift ; ORIGIN_BRANCH="$1" ; shift ;;
|
||||
--working_branch) shift ; WORKING_BRANCH="$1" ; shift ;;
|
||||
--upversion_data) shift ; UPVERSION_LOG="$1" ; shift ;;
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
--) shift ; break ;;
|
||||
*) usage; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$UPVERSION_LOG" == "" ]; then
|
||||
UPVERSION_LOG=$UPVERSION_DATA
|
||||
fi
|
||||
|
||||
if [ "$UPVERSION_LOG" == "" ]; then
|
||||
echo "ERROR: please specify location of upversion data"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -f "$UPVERSION_LOG" ]; then
|
||||
echo "File not found: '$UPVERSION_LOG'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
|
||||
ORIGIN_BRANCH=$PATCH_SOURCE_BRANCH
|
||||
WORKING_BRANCH=$MY_PATCH_BRANCH
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
|
||||
ORIGIN_BRANCH=$SOURCE_BRANCH
|
||||
WORKING_BRANCH=$MY_BRANCH
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ]; then
|
||||
echo "ERROR: please specify a origin branch"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$WORKING_BRANCH" == "" ]; then
|
||||
echo "ERROR: please specify a working branch"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# One step back to see the old symlinks
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$(pwd)'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout HEAD^
|
||||
|
||||
FAILED=""
|
||||
for dat in $(cat $UPVERSION_LOG); do
|
||||
name=$(echo $dat | awk -F '#' '{print $1}')
|
||||
srpm_path=$(echo $dat | awk -F '#' '{print $2}')
|
||||
old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
|
||||
new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
|
||||
|
||||
echo "$name $old_src_rpm $new_src_rpm"
|
||||
|
||||
build-pkgs --edit --clean $name
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed cmd 'build-pkgs --edit --clean $name'"
|
||||
FAILED="$name $FAILED"
|
||||
break
|
||||
fi
|
||||
echo "$? <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
|
||||
build-pkgs --edit $name
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed cmd 'build-pkgs --edit $name'"
|
||||
FAILED="$name $FAILED"
|
||||
break
|
||||
fi
|
||||
echo "$? <=<=<=<=<=<=<=<=<=<=<=<=<=<=<=<="
|
||||
done
|
||||
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
|
||||
if [ "$FAILED" != "" ]; then
|
||||
echo "Failed build-pkgs --edit for ... $FAILED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
148
build-tools/patch_rebase_2
Executable file
148
build-tools/patch_rebase_2
Executable file
@ -0,0 +1,148 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Update srpm_path for packages to be upgraded
|
||||
#
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Step 2: Update srpm_path for packages to be upgraded"
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " patch_rebase_2 [--origin_branch <branch>] [--working_branch <branch>] [--upversion_data <file>]"
|
||||
echo ""
|
||||
echo "Assumes cgcs-centos-repo already has a working_branch commit that sets the new symlinks."
|
||||
echo ""
|
||||
echo "The upversion_data file has data on all the src.rpm being updated in the format:"
|
||||
echo " export UPVERSION_DATA=$MY_WORKSPACE/upversion.log"
|
||||
echo " PKG=lighttpd"
|
||||
echo " OLD_SRC_RPM=lighttpd-1.4.41-1.el7.src.rpm"
|
||||
echo " NEW_SRC_RPM=lighttpd-1.4.41-2.el7.src.rpm"
|
||||
echo " SRPM_PATH=$MY_REPO/addons/wr-cgcs/layers/cgcs/recipes-extended/lighttpd/centos/srpm_path"
|
||||
echo " echo \"\$PKG#\$SRPM_PATH##\$OLD_SRC_RPM#\$NEW_SRC_RPM\" > UPVERSION_DATA"
|
||||
echo ""
|
||||
}
|
||||
|
||||
|
||||
TEMP=`getopt -o h --long origin_branch:,working_branch:,upversion_data:,help -n 'test.sh' -- "$@"`
|
||||
eval set -- "$TEMP"
|
||||
|
||||
ORIGIN_BRANCH=""
|
||||
WORKING_BRANCH=""
|
||||
UPVERSION_LOG=""
|
||||
HELP=0
|
||||
|
||||
while true ; do
|
||||
case "$1" in
|
||||
--origin_branch) shift ; ORIGIN_BRANCH="$1" ; shift ;;
|
||||
--working_branch) shift ; WORKING_BRANCH="$1" ; shift ;;
|
||||
--upversion_data) shift ; UPVERSION_LOG="$1" ; shift ;;
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
--) shift ; break ;;
|
||||
*) usage; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$UPVERSION_LOG" == "" ]; then
|
||||
UPVERSION_LOG=$UPVERSION_DATA
|
||||
fi
|
||||
|
||||
if [ "$UPVERSION_LOG" == "" ]; then
|
||||
echo "ERROR: please specify location of upversion data"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -f "$UPVERSION_LOG" ]; then
|
||||
echo "File not found: '$UPVERSION_LOG'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
|
||||
ORIGIN_BRANCH=$PATCH_SOURCE_BRANCH
|
||||
WORKING_BRANCH=$MY_PATCH_BRANCH
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
|
||||
ORIGIN_BRANCH=$SOURCE_BRANCH
|
||||
WORKING_BRANCH=$MY_BRANCH
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ]; then
|
||||
echo "ERROR: please specify a origin branch"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$WORKING_BRANCH" == "" ]; then
|
||||
echo "ERROR: please specify a working branch"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# One step back to see the old symlinks
|
||||
cd $MY_REPO
|
||||
|
||||
FAILED=""
|
||||
for dat in $(cat $UPVERSION_LOG); do
|
||||
name=$(echo $dat | awk -F '#' '{print $1}')
|
||||
srpm_path=$(echo $dat | awk -F '#' '{print $2}')
|
||||
old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
|
||||
new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
|
||||
|
||||
(
|
||||
cd $(dirname $srpm_path)
|
||||
CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
|
||||
if [ "$CURRENT_BRANCH" != "$WORKING_BRANCH" ]; then
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
git checkout $ORIGIN_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$ORIGIN_BRANCH' in directory '$(pwd)'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout -b $WORKING_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to 'git checkout -b $WORKING_BRANCH' from '$(pwd)'"
|
||||
exit 1
|
||||
else
|
||||
echo "created branch '$WORKING_BRANCH' at '$(pwd)'"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
sed -i "s#$old_src_rpm#$new_src_rpm#" $srpm_path
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: sed failed '$old_src_rpm' -> '$new_src_rpm'"
|
||||
exit 1
|
||||
else
|
||||
echo "updated $srpm_path: '$old_src_rpm' -> '$new_src_rpm'"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
)
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed while working on package '$name' at '$srpm_path'"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
for d in $(for dat in $(cat $UPVERSION_LOG); do srpm_path=$(echo $dat | awk -F '#' '{print $2}'); ( cd $(dirname $srpm_path); git rev-parse --show-toplevel ); done | sort --unique); do
|
||||
(
|
||||
cd $d
|
||||
echo "cd $d"
|
||||
for f in $(git status --porcelain | grep 'srpm_path$' | awk '{print $2}'); do
|
||||
echo "git add $f";
|
||||
done
|
||||
echo "git commit -m 'srpm_path updates for patch $PATCH_ID'"
|
||||
)
|
||||
done
|
||||
echo ""
|
||||
119
build-tools/patch_rebase_3
Executable file
119
build-tools/patch_rebase_3
Executable file
@ -0,0 +1,119 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Start an edit session for packages to be upgraded - post upgrade version
|
||||
#
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Step 3: Start an edit session for packages to be upgraded - post upgrade version"
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " patch_rebase_3 [--origin_branch <branch>] [--working_branch <branch>] [--upversion_data <file>]"
|
||||
echo ""
|
||||
echo "Assumes cgcs-centos-repo already has a working_branch commit that sets the new symlinks."
|
||||
echo ""
|
||||
echo "The upversion_data file has data on all the src.rpm being updated in the format:"
|
||||
echo " export UPVERSION_DATA=$MY_WORKSPACE/upversion.log"
|
||||
echo " PKG=lighttpd"
|
||||
echo " OLD_SRC_RPM=lighttpd-1.4.41-1.el7.src.rpm"
|
||||
echo " NEW_SRC_RPM=lighttpd-1.4.41-2.el7.src.rpm"
|
||||
echo " SRPM_PATH=$MY_REPO/addons/wr-cgcs/layers/cgcs/recipes-extended/lighttpd/centos/srpm_path"
|
||||
echo " echo \"\$PKG#\$SRPM_PATH##\$OLD_SRC_RPM#\$NEW_SRC_RPM\" > UPVERSION_DATA"
|
||||
echo ""
|
||||
}
|
||||
|
||||
|
||||
TEMP=`getopt -o h --long origin_branch:,working_branch:,upversion_data:,help -n 'test.sh' -- "$@"`
|
||||
eval set -- "$TEMP"
|
||||
|
||||
ORIGIN_BRANCH=""
|
||||
WORKING_BRANCH=""
|
||||
UPVERSION_LOG=""
|
||||
HELP=0
|
||||
|
||||
while true ; do
|
||||
case "$1" in
|
||||
--origin_branch) shift ; ORIGIN_BRANCH="$1" ; shift ;;
|
||||
--working_branch) shift ; WORKING_BRANCH="$1" ; shift ;;
|
||||
--upversion_data) shift ; UPVERSION_LOG="$1" ; shift ;;
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
--) shift ; break ;;
|
||||
*) usage; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$UPVERSION_LOG" == "" ]; then
|
||||
UPVERSION_LOG=$UPVERSION_DATA
|
||||
fi
|
||||
|
||||
if [ "$UPVERSION_LOG" == "" ]; then
|
||||
echo "ERROR: please specify location of upversion data"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -f "$UPVERSION_LOG" ]; then
|
||||
echo "File not found: '$UPVERSION_LOG'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
|
||||
ORIGIN_BRANCH=$PATCH_SOURCE_BRANCH
|
||||
WORKING_BRANCH=$MY_PATCH_BRANCH
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
|
||||
ORIGIN_BRANCH=$SOURCE_BRANCH
|
||||
WORKING_BRANCH=$MY_BRANCH
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ]; then
|
||||
echo "ERROR: please specify a origin branch"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$WORKING_BRANCH" == "" ]; then
|
||||
echo "ERROR: please specify a working branch"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# One step back to see the old symlinks
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
FAILED=""
|
||||
for dat in $(cat $UPVERSION_LOG); do
|
||||
name=$(echo $dat | awk -F '#' '{print $1}')
|
||||
srpm_path=$(echo $dat | awk -F '#' '{print $2}')
|
||||
old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
|
||||
new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
|
||||
|
||||
echo "$name $old_src_rpm $new_src_rpm"
|
||||
|
||||
build-pkgs --edit $name --no-meta-patch
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed cmd 'build-pkgs --edit $name'"
|
||||
FAILED="$name $FAILED"
|
||||
break
|
||||
fi
|
||||
echo "$? <=<=<=<=<=<=<=<=<=<=<=<=<=<=<=<="
|
||||
done
|
||||
|
||||
if [ "$FAILED" != "" ]; then
|
||||
echo "Failed build-pkgs --edit for ... $FAILED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
403
build-tools/patch_rebase_4
Executable file
403
build-tools/patch_rebase_4
Executable file
@ -0,0 +1,403 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Migrate Titanium Cloud patches to the new package version
|
||||
#
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Step 4: Migrate Titanium Cloud patches to the new package version"
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " patch_rebase_4 [--origin_branch <branch>] [--working_branch <branch>] [--upversion_data <file>]"
|
||||
echo ""
|
||||
echo "Assumes cgcs-centos-repo already has a working_branch commit that sets the new symlinks."
|
||||
echo ""
|
||||
echo "The upversion_data file has data on all the src.rpm being updated in the format:"
|
||||
echo " export UPVERSION_DATA=$MY_WORKSPACE/upversion.log"
|
||||
echo " PKG=lighttpd"
|
||||
echo " OLD_SRC_RPM=lighttpd-1.4.41-1.el7.src.rpm"
|
||||
echo " NEW_SRC_RPM=lighttpd-1.4.41-2.el7.src.rpm"
|
||||
echo " SRPM_PATH=$MY_REPO/addons/wr-cgcs/layers/cgcs/recipes-extended/lighttpd/centos/srpm_path"
|
||||
echo " echo \"\$PKG#\$SRPM_PATH##\$OLD_SRC_RPM#\$NEW_SRC_RPM\" > UPVERSION_DATA"
|
||||
echo ""
|
||||
}
|
||||
|
||||
|
||||
TEMP=`getopt -o h --long origin_branch:,working_branch:,upversion_data:,help -n 'test.sh' -- "$@"`
|
||||
eval set -- "$TEMP"
|
||||
|
||||
ORIGIN_BRANCH=""
|
||||
WORKING_BRANCH=""
|
||||
UPVERSION_LOG=""
|
||||
HELP=0
|
||||
|
||||
while true ; do
|
||||
case "$1" in
|
||||
--origin_branch) shift ; ORIGIN_BRANCH="$1" ; shift ;;
|
||||
--working_branch) shift ; WORKING_BRANCH="$1" ; shift ;;
|
||||
--upversion_data) shift ; UPVERSION_LOG="$1" ; shift ;;
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
--) shift ; break ;;
|
||||
*) usage; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$UPVERSION_LOG" == "" ]; then
|
||||
UPVERSION_LOG=$UPVERSION_DATA
|
||||
fi
|
||||
|
||||
if [ "$UPVERSION_LOG" == "" ]; then
|
||||
echo "ERROR: please specify location of upversion data"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -f "$UPVERSION_LOG" ]; then
|
||||
echo "File not found: '$UPVERSION_LOG'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
|
||||
ORIGIN_BRANCH=$PATCH_SOURCE_BRANCH
|
||||
WORKING_BRANCH=$MY_PATCH_BRANCH
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
|
||||
ORIGIN_BRANCH=$SOURCE_BRANCH
|
||||
WORKING_BRANCH=$MY_BRANCH
|
||||
fi
|
||||
|
||||
if [ "$ORIGIN_BRANCH" == "" ]; then
|
||||
echo "ERROR: please specify a origin branch"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$WORKING_BRANCH" == "" ]; then
|
||||
echo "ERROR: please specify a working branch"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$DISPLAY" == "" ]; then
|
||||
echo "ERROR: X-Windows 'DISPLAY' variable not set. This script needs to open pop-up windows."
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# One step back to see the old symlinks
|
||||
cd $MY_REPO/cgcs-centos-repo
|
||||
git checkout $WORKING_BRANCH
|
||||
if [ $? != 0 ]; then
|
||||
echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
FAILED=""
|
||||
build_types="std rt"
|
||||
for dat in $(cat $UPVERSION_LOG); do
|
||||
(
|
||||
name=$(echo $dat | awk -F '#' '{print $1}')
|
||||
srpm_path=$(echo $dat | awk -F '#' '{print $2}')
|
||||
old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
|
||||
new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
|
||||
|
||||
PKG_DIR=$(dirname $(dirname $srpm_path))
|
||||
OLD_BRANCH=$(echo $old_src_rpm | sed 's#[.]src[.]rpm$##')
|
||||
NEW_BRANCH=$(echo $new_src_rpm | sed 's#[.]src[.]rpm$##')
|
||||
|
||||
WORK_META_DIR=""
|
||||
for dd in $build_types; do
|
||||
WORK_META_DIR=$MY_WORKSPACE/$dd/srpm_work/$name/rpmbuild
|
||||
echo "WORK_META_DIR=$WORK_META_DIR"
|
||||
if [ -d $WORK_META_DIR ]; then
|
||||
break;
|
||||
else
|
||||
WORK_META_DIR=""
|
||||
fi
|
||||
done
|
||||
if [ "$WORK_META_DIR" == "" ]; then
|
||||
echo "ERROR: failed to find srpm_work directory for '$name'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# WORK_SRC_DIR=$(dirname $(find $MY_WORKSPACE/srpm_work/$name/gits/ -type d -name .git))
|
||||
NEW_WORK_SRC_DIR=""
|
||||
OLD_WORK_SRC_DIR=""
|
||||
for dd in $build_types; do
|
||||
for g in $(find $MY_WORKSPACE/$dd/srpm_work/$name/gits/ -type d -name .git); do
|
||||
d=$(dirname $g)
|
||||
if [ -d $d ]; then
|
||||
cd $d;
|
||||
git tag | grep pre_wrs_ >> /dev/null
|
||||
if [ $? -ne 0 ]; then
|
||||
continue
|
||||
fi
|
||||
git checkout $OLD_BRANCH 2>> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
OLD_WORK_SRC_DIR=$d
|
||||
fi
|
||||
git checkout $NEW_BRANCH 2>> /dev/null
|
||||
if [ $? -eq 0 ]; then
|
||||
NEW_WORK_SRC_DIR=$d
|
||||
fi
|
||||
fi
|
||||
done
|
||||
done
|
||||
if [ "$WORK_META_DIR" == "" ]; then
|
||||
echo "ERROR: failed to find srpm_work directory for '$name'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "$name $old_src_rpm $new_src_rpm"
|
||||
echo "PKG_DIR=$PKG_DIR"
|
||||
echo "OLD_BRANCH=$OLD_BRANCH"
|
||||
echo "NEW_BRANCH=$NEW_BRANCH"
|
||||
echo "WORK_META_DIR=$WORK_META_DIR"
|
||||
echo "OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
|
||||
echo "NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
|
||||
echo ""
|
||||
|
||||
(
|
||||
cd $WORK_META_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to cd to WORK_META_DIR=$WORK_META_DIR"
|
||||
exit 1
|
||||
fi
|
||||
echo "--- old meta git log (oldest to newest) ---"
|
||||
git checkout $OLD_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit | tac
|
||||
PATCH_COMMIT_LIST=$(git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
|
||||
echo "--- new meta git log (oldest to newest) ---"
|
||||
git checkout $NEW_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit | tac
|
||||
REFERENCE_COMMIT=$(git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit | head -n 1 | awk '{ print $2 }')
|
||||
echo ""
|
||||
|
||||
for COMMIT in ${PATCH_COMMIT_LIST}; do
|
||||
echo "git cherry-pick $COMMIT"
|
||||
git cherry-pick "$COMMIT"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files"
|
||||
git status --porcelain | grep '^UU ' | awk '{ print $2}'
|
||||
echo "pwd=$(pwd)"
|
||||
# gitk &
|
||||
echo "git mergetool --no-prompt"
|
||||
git mergetool --no-prompt
|
||||
# for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
|
||||
# xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
|
||||
# if [ $? -ne 0 ]; then
|
||||
# echo "ERROR: problem launching editor on "
|
||||
# exit 1
|
||||
# fi
|
||||
# done
|
||||
echo "git cherry-pick --continue"
|
||||
git cherry-pick --continue
|
||||
fi
|
||||
done
|
||||
|
||||
PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
|
||||
exit 1
|
||||
fi
|
||||
for PATCH_FILE in ${PATCH_LIST}; do
|
||||
PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
|
||||
echo "$PATCH_FILE -> $PATCH_TARGET"
|
||||
N=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*" | wc -l)
|
||||
if [ $N -eq 1 ]; then
|
||||
PATCH_DEST=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*")
|
||||
echo "cp -f $PATCH_FILE $PATCH_DEST"
|
||||
\cp -f $PATCH_FILE $PATCH_DEST
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: copy failed $WORK_META_DIR/$PATCH_FILE -> $PATCH_DEST"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "ERROR: Don't know what destination file name to use for patch '$WORK_META_DIR/$PATCH_FILE' derived from commit $COMMIT, and to be copied to '$PKG_DIR/centos/meta_patches'"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo ""
|
||||
)
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
FAILED=$name
|
||||
break
|
||||
fi
|
||||
|
||||
(
|
||||
echo "--- old git log (oldest to newest) ---"
|
||||
cd $OLD_WORK_SRC_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to cd to OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout $OLD_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH in directory '$OLD_WORK_SRC_DIR'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit | tac
|
||||
PATCH_COMMIT_LIST=$(git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
|
||||
|
||||
echo "--- new git log (oldest to newest) ---"
|
||||
cd $NEW_WORK_SRC_DIR
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to cd to NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout $NEW_BRANCH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH in directory '$NEW_WORK_SRC_DIR'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit | tac
|
||||
REFERENCE_COMMIT=$(git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit | head -n 1 | awk '{ print $2 }')
|
||||
echo ""
|
||||
|
||||
if [ "$OLD_WORK_SRC_DIR" == "$NEW_WORK_SRC_DIR" ]; then
|
||||
for COMMIT in ${PATCH_COMMIT_LIST}; do
|
||||
echo "git cherry-pick $COMMIT"
|
||||
git cherry-pick "$COMMIT"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files"
|
||||
git status --porcelain | grep '^UU ' | awk '{ print $2}'
|
||||
echo "pwd=$(pwd)"
|
||||
# gitk &
|
||||
echo "git mergetool --no-prompt"
|
||||
git mergetool --no-prompt
|
||||
# for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
|
||||
# xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
|
||||
# if [ $? -ne 0 ]; then
|
||||
# echo "ERROR: problem launching editor on "
|
||||
# exit 1
|
||||
# fi
|
||||
# done
|
||||
echo "git cherry-pick --continue"
|
||||
git cherry-pick --continue
|
||||
fi
|
||||
done
|
||||
else
|
||||
cd $OLD_WORK_SRC_DIR
|
||||
PATCH_LIST=$(git format-patch -n pre_wrs_$OLD_BRANCH)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=pre_wrs_$OLD_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
cd $NEW_WORK_SRC_DIR
|
||||
for PATCH_FILE in ${PATCH_LIST}; do
|
||||
cat $OLD_WORK_SRC_DIR/$PATCH_FILE | patch -p1
|
||||
if [ $? -ne 0 ]; then
|
||||
for REJECT in $(find . -name '*.rej'); do
|
||||
FILE_NAME=$(echo $REJECT | sed 's#.rej$##')
|
||||
cd $OLD_WORK_SRC_DIR
|
||||
gitk $FILE_NAME &
|
||||
cd $NEW_WORK_SRC_DIR
|
||||
if [ -f $FILE_NAME ] && [ -f $FILE_NAME.orig ]; then
|
||||
\cp -f $FILE_NAME.orig $FILE_NAME
|
||||
xterm -e "vi $FILE_NAME $REJECT"
|
||||
rm -f $REJECT
|
||||
rm -f $FILE_NAME.orig
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
git add --all
|
||||
MSG=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
|
||||
git commit -m "WRS: $MSG"
|
||||
done
|
||||
|
||||
fi
|
||||
|
||||
PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
|
||||
exit 1
|
||||
fi
|
||||
for PATCH_FILE in ${PATCH_LIST}; do
|
||||
PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch[0-9]*-//' | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch//' | sed 's/.patch$//')
|
||||
echo "$PATCH_FILE -> $PATCH_TARGET"
|
||||
PKG_PATCH_DIR="$PKG_DIR/centos/patches"
|
||||
N=0
|
||||
if [ -d "$PKG_PATCH_DIR" ]; then
|
||||
N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
|
||||
fi
|
||||
if [ $N -ne 1 ]; then
|
||||
PKG_PATCH_DIR="$PKG_DIR"
|
||||
if [ -d "$PKG_PATCH_DIR" ]; then
|
||||
N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
|
||||
fi
|
||||
fi
|
||||
echo "N=$N"
|
||||
echo "PKG_PATCH_DIR=$PKG_PATCH_DIR"
|
||||
|
||||
if [ $N -eq 1 ]; then
|
||||
PATCH_DEST=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]')
|
||||
echo "meld $PATCH_FILE -> $PATCH_DEST"
|
||||
meld $PATCH_FILE $PATCH_DEST
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "ERROR: meld failed $WORK_SRC_DIR/$PATCH_FILE -> $PATCH_DEST"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "ERROR: Don't know what destination file name to use for patch '$OLD_WORK_SRC_DIR/$PATCH_FILE', and to be copied to '$PKG_PATCH_DIR'"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo ""
|
||||
)
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
FAILED=$name
|
||||
break
|
||||
fi
|
||||
|
||||
)
|
||||
|
||||
|
||||
done
|
||||
|
||||
if [ "$FAILED" != "" ]; then
|
||||
echo "Failed for ... $FAILED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
for d in $(for dat in $(cat $UPVERSION_LOG); do srpm_path=$(echo $dat | awk -F '#' '{print $2}'); ( cd $(dirname $srpm_path); git rev-parse --show-toplevel ); done | sort --unique); do
|
||||
(
|
||||
cd $d
|
||||
echo "cd $d"
|
||||
for f in $(git status --porcelain | awk '{print $2}'); do
|
||||
echo "git add $f";
|
||||
done
|
||||
if [ "$PATCH_ID" == "" ]; then
|
||||
echo "git commit -m 'rebased patches'"
|
||||
else
|
||||
echo "git commit -m 'rebased patches for patch $PATCH_ID'"
|
||||
fi
|
||||
)
|
||||
done
|
||||
echo ""
|
||||
|
||||
|
||||
506
build-tools/sign-build
Executable file
506
build-tools/sign-build
Executable file
@ -0,0 +1,506 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This script calls into an external signing server to perform signing of some
|
||||
# packages in the system. The old packages (which are typically generated by
|
||||
# the build system and signed by placeholder keys) are overwritten by the new
|
||||
# packages.
|
||||
#
|
||||
# Three types of packages are signed:
|
||||
# kernels (both std and lowlatency, aka "rt", kernels)
|
||||
# grub
|
||||
# shim
|
||||
#
|
||||
# Kernels and grub are generated by producing (under the normal build system)
|
||||
# two packages -- a package containing the unsigned binaries, and a package
|
||||
# containing binaries signed with temporary keys. All the "accessories" (files,
|
||||
# scripts, etc) are included in the package containing the signed-with-temp-keys
|
||||
# files. The signing server will take both packages, sign the unsigned
|
||||
# binaries, and replace the files in the signed package with the newly signed
|
||||
# ones.
|
||||
#
|
||||
# Typical flow/artifacts
|
||||
# kernel.src.rpm -> produces kernel.rpm and kernel-unsigned.rpm
|
||||
# kernel.rpm -> initially contains binaries signed with a temporary key
|
||||
# -> contains all files used by the kernel
|
||||
# -> can be installed and used in a system (it just won't
|
||||
# secure boot since the key is just a temp key)
|
||||
# kernel-unsigned.rpm -> contains just unsigned kernel binaries
|
||||
#
|
||||
# The signing server will take both packages, sign the binaries in
|
||||
# kernel-unsigned.rpm with our private key, and replace the binaries in
|
||||
# kernel.rpm with the new binaries. The kernel.rpm can then be replaced by the
|
||||
# version generated by the signing server.
|
||||
#
|
||||
# Shim is a bit of a different beast.
|
||||
#
|
||||
# There are two source packages - shim and shim-signed. Frustratingly, "shim"
|
||||
# source produces a "shim-unsigned" binary output. "shim-signed" produces a
|
||||
# "shim" binary output.
|
||||
#
|
||||
# The "shim-signed" source RPM doesn't contain source code -- it just contains
|
||||
# instructions to take the "shim-unsigned" binaries, sign them, and package the
|
||||
# output. We've modified the shim-signed RPM to (rather than sign with a temp
|
||||
# key) use "presigned" binaries from shim-unsigned if the files exist. (It will
|
||||
# still use a temp key of no presigned files are found, which is how the build
|
||||
# system normally runs).
|
||||
#
|
||||
# The signing server will unpack the shim-unsigned package, sign the binaries
|
||||
# (as "presigned") and repack the package.
|
||||
#
|
||||
# A rebuild of shim-signed by the build server is then required.
|
||||
#
|
||||
# Thanks for bearing with me in the convoluted discussion, above.
|
||||
|
||||
|
||||
# Script flow:
|
||||
# - call signing server to sign kernels (if they exist and are new, as with
|
||||
# other RPMs)
|
||||
# - replace old kernel packages with newly signed ones
|
||||
# - call signing server to sign grub (and replace old version with the newly
|
||||
# signed one)
|
||||
# - call signing server to sign shim-unsigned (replace old version)
|
||||
# - rebuild shim-signed
|
||||
# - update our repos to advertize all newly replaced packages
|
||||
|
||||
# check_if_pkg_needs_signing <path/to/filename.rpm>
|
||||
#
|
||||
# Checks to see if a given package needs to be signed. We maintain a list of
|
||||
# MD5 sums for RPMs we have signed. Thus, we can easily see if we've already
|
||||
# signed a package.
|
||||
#
|
||||
# Returns 1 if the package does need signing, or 0 if package does not
|
||||
#
|
||||
# This function expects the package specified to exist.
|
||||
function check_if_pkg_needs_signing
|
||||
{
|
||||
local PKG_TO_CHECK=$1
|
||||
|
||||
if [ ! -e ${SIGNED_PKG_DB} ]; then
|
||||
# We haven't signed anything before, so this package needs signing
|
||||
return 1
|
||||
fi
|
||||
|
||||
local SIGNED_PKG_MD5=`grep ${PKG_TO_CHECK} ${SIGNED_PKG_DB} | cut -d ' ' -f 1`
|
||||
if [ "x${SIGNED_PKG_MD5}" == "x" ]; then
|
||||
# We have no record of having signed the package -- needs signing
|
||||
return 1
|
||||
fi
|
||||
|
||||
local CURRENT_MD5=`md5sum ${PKG_TO_CHECK} | cut -d ' ' -f 1`
|
||||
if [ "${CURRENT_MD5}" != "${SIGNED_PKG_MD5}" ]; then
|
||||
# The package has been regenerated since we last signed it -- needs
|
||||
# signing again
|
||||
return 1
|
||||
fi
|
||||
|
||||
# The package md5 sum matches the md5sum of the package when it was last
|
||||
# signed.
|
||||
return 0
|
||||
}
|
||||
|
||||
# update_signed_pkg_list <path/to/filename.rpm>
|
||||
#
|
||||
# Updated our list of signed packages with the md5 sum of a recently signed
|
||||
# package.
|
||||
#
|
||||
# This function expects the package to exist.
|
||||
function update_signed_pkg_list
|
||||
{
|
||||
local PKG_TO_ADD=$1
|
||||
|
||||
if [ ! -e ${SIGNED_PKG_DB} ]; then
|
||||
touch ${SIGNED_PKG_DB}
|
||||
fi
|
||||
|
||||
# remove current entry for package
|
||||
local TMPFILE=`mktemp`
|
||||
grep -v $(basename ${PKG_TO_ADD}) ${SIGNED_PKG_DB} > ${TMPFILE}
|
||||
mv ${TMPFILE} ${SIGNED_PKG_DB}
|
||||
|
||||
# add MD5 for package to the package list
|
||||
md5sum ${PKG_TO_ADD} >> ${SIGNED_PKG_DB}
|
||||
}
|
||||
|
||||
|
||||
# update_repo <std|rt>
|
||||
#
|
||||
# Updates either the standard or rt repo with latest packages
|
||||
# Checks that you specified a repo, and that the path exists.
|
||||
#
|
||||
# There are actually now two places we need to update -- the
|
||||
# rpmbuild/RPMS/ path, as well as the results/.../ path
|
||||
function update_repo
|
||||
{
|
||||
local BUILD_TYPE=$1
|
||||
local EXTRA_PARAMS=""
|
||||
local RETCODE=0
|
||||
local repopath=""
|
||||
|
||||
if [ "x$BUILD_TYPE" == "x" ]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
if [ "x$MY_BUILD_ENVIRONMENT_TOP" == "x" ]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
for repopath in "$MY_WORKSPACE/$BUILD_TYPE/rpmbuild/RPMS" "$MY_WORKSPACE/$BUILD_TYPE/results/${MY_BUILD_ENVIRONMENT_TOP}-$BUILD_TYPE"; do
|
||||
if [ ! -d "$repopath" ]; then
|
||||
echo "Error - cannot find path $repopath"
|
||||
return 1
|
||||
fi
|
||||
|
||||
cd $repopath
|
||||
if [ -f comps.xml ]; then
|
||||
EXTRA_PARAMS="-g comps.xml"
|
||||
fi
|
||||
createrepo --update $EXTRA_PARAMS . > /dev/null
|
||||
RETCODE=$?
|
||||
cd - > /dev/null
|
||||
if [ 0$RETCODE -ne 0 ]; then
|
||||
return $RETCODE
|
||||
fi
|
||||
done
|
||||
|
||||
return $RETCODE
|
||||
}
|
||||
|
||||
# sign_shims - find and sign any shim package that we need to
|
||||
#
|
||||
function sign_shims
|
||||
{
|
||||
SHIM=`find $MY_WORKSPACE/std/rpmbuild/RPMS -name "shim-unsigned-*.$ARCH.rpm" | grep -v debuginfo`
|
||||
if [ "x${SHIM}" == "x" ]; then
|
||||
echo "Warning -- cannot find shim package to sign"
|
||||
return 0
|
||||
fi
|
||||
sign shim $SHIM
|
||||
|
||||
return $?
|
||||
}
|
||||
|
||||
# sign_grubs - find and sign any grub package that we need to.
|
||||
# Grub (and kernel) are initially signed with temporary keys, so
|
||||
# we need to upload both the complete package, as well as the
|
||||
# unsigned binaries
|
||||
#
|
||||
function sign_grubs
|
||||
{
|
||||
GRUB=`find $MY_WORKSPACE/std/rpmbuild/RPMS -name "grub2-efi-[1-9]*.$ARCH.rpm"`
|
||||
UNSIGNED_GRUB=`find $MY_WORKSPACE/std/rpmbuild/RPMS -name "grub2-efi-unsigned*.$ARCH.rpm"`
|
||||
if [ "x${GRUB}" == "x" ]; then
|
||||
echo "Warning -- cannot find GRUB package to sign"
|
||||
return 0
|
||||
fi
|
||||
if [ "x${UNSIGNED_GRUB}" == "x" ]; then
|
||||
echo "Warning -- cannot find unsigned GRUB package to sign"
|
||||
return 0
|
||||
fi
|
||||
|
||||
sign grub2 $GRUB $UNSIGNED_GRUB
|
||||
return $?
|
||||
}
|
||||
|
||||
# sign_kernels - find and sign any kernel package that we need to.
|
||||
#
|
||||
function sign_kernels
|
||||
{
|
||||
sign_kernel "std" ""
|
||||
sign_kernel "rt" "-rt"
|
||||
}
|
||||
|
||||
# sign_kernel - find and sign kernel package if we need to.
|
||||
# Kernels (and grub) are initially signed with temporary keys, so
|
||||
# we need to upload both the complete package, as well as the
|
||||
# unsigned binaries
|
||||
function sign_kernel
|
||||
{
|
||||
local KERNEL_PATH=$1
|
||||
local KERNEL_EXTRA=$2
|
||||
KERNEL=`find $MY_WORKSPACE/${KERNEL_PATH}/rpmbuild/RPMS -name "kernel${KERNEL_EXTRA}-[1-9]*.$ARCH.rpm"`
|
||||
UNSIGNED_KERNEL=`find $MY_WORKSPACE/${KERNEL_PATH}/rpmbuild/RPMS -name "kernel${KERNEL_EXTRA}-unsigned-[1-9]*.$ARCH.rpm"`
|
||||
if [ "x${KERNEL}" == "x" ]; then
|
||||
echo "Warning -- cannot find kernel package to sign in ${KERNEL_PATH}"
|
||||
return 0
|
||||
fi
|
||||
if [ "x${UNSIGNED_KERNEL}" == "x" ]; then
|
||||
echo "Warning -- cannot find unsigned kernel package to sign in ${KERNEL_PATH}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
sign kernel $KERNEL $UNSIGNED_KERNEL
|
||||
if [ $? -ne 0 ]; then
|
||||
return $?
|
||||
fi
|
||||
}
|
||||
|
||||
# rebuild_pkgs - rebuild any packages that need to be updated from the newly
|
||||
# signed binaries
|
||||
#
|
||||
function rebuild_pkgs
|
||||
{
|
||||
local LOGFILE="$MY_WORKSPACE/export/signed-rebuild.log"
|
||||
local PKGS_TO_REBUILD=${REBUILD_LIST}
|
||||
|
||||
if [ "x${PKGS_TO_REBUILD}" == "x" ]; then
|
||||
# No rebuilds required, return cleanly
|
||||
return 0
|
||||
fi
|
||||
|
||||
# If we reach this point, then we have one or more packages to be rebuilt
|
||||
|
||||
# first, update the repo so it is aware of the "latest" binaries
|
||||
update_repo std
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could not update signed packages -- could not update repo"
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "Performing rebuild of packages: $PKGS_TO_REBUILD"
|
||||
FORMAL_BUILD=0 build-pkgs --no-descendants --no-build-info --no-required --careful $PKGS_TO_REBUILD > $LOGFILE 2>&1
|
||||
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could not rebuild packages: $PKGS_TO_REBUILD -- see $LOGFILE for details"
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "Done"
|
||||
return 0
|
||||
}
|
||||
|
||||
# sign <type_of_pkg> <pkg> [pkg_containing_unsigned_bins]
|
||||
#
|
||||
# This routine uploads a package to the signing server, instructs the signing
|
||||
# signing server to do its' magic, and downloads the updated (signed) package
|
||||
# from the signing server.
|
||||
#
|
||||
# Accessing the signing server -- the signing server cannot just be logged
|
||||
# into by anyone. A small number of users (Jason McKenna, Scott Little, Greg
|
||||
# Waines, etc) have permission to log in as themselves. In addition, there is
|
||||
# a user "signing" who is unique to the server. The "jenkins" user on our
|
||||
# build servers has permission to login/upload files as "signing" due to Jenkins'
|
||||
# private SSH key being added to the signing user's list of keys. This means
|
||||
# that Jenkins can upload and run commands on the server as "signing".
|
||||
#
|
||||
# In addition to uploading files as signing, the signing user has permissions to
|
||||
# run a single command (/opt/signing/sign.sh) as a sudo root user. The signing
|
||||
# user does not have access to modify the script or to run any other commands as
|
||||
# root. The sign.sh script will take inputs (the files that jenkins has
|
||||
# uploaded), verify the contents, sign the images against private keys, and
|
||||
# output a new .rpm contianing the signed version of the files. Assuming all
|
||||
# is successful, the filename of the signed output file is returned, and the
|
||||
# jenkins user can then use that filename to download the file (the "signing"
|
||||
# user does not have access to remove or modify the file once it's created).
|
||||
#
|
||||
# All operations done on the signing server are logged in muliple places, and
|
||||
# the output RPM artifacts are timestamped to ensure that they are not
|
||||
# overwritten by subsequent calls to sign.sh.
|
||||
#
|
||||
# kernel and grub package types require you to specify/upload the unsigned
|
||||
# packages as well as the normal binary
|
||||
function sign
|
||||
{
|
||||
local TYPE=$1
|
||||
local FILE=$2
|
||||
local UNSIGNED=$3
|
||||
local UNSIGNED_OPTION=""
|
||||
local TMPFILE=`mktemp /tmp/sign.XXXXXXXX`
|
||||
|
||||
# Don't sign if we've already signed it
|
||||
check_if_pkg_needs_signing ${FILE}
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "Not signing ${FILE} as we previously signed it"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "Signing $FILE"
|
||||
|
||||
# upload the original package
|
||||
scp -q $FILE $SIGNING_USER@$SIGNING_SERVER:$UPLOAD_PATH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to upload file $FILE"
|
||||
\rm -f $TMPFILE
|
||||
return 1
|
||||
fi
|
||||
|
||||
# upload the unsigned package (if specified)
|
||||
if [ "x$UNSIGNED" != "x" ]; then
|
||||
scp -q $UNSIGNED $SIGNING_USER@$SIGNING_SERVER:$UPLOAD_PATH
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to upload file $UNSIGNED"
|
||||
\rm -f $TMPFILE
|
||||
return 1
|
||||
fi
|
||||
UNSIGNED=$(basename $UNSIGNED)
|
||||
UNSIGNED_OPTION="-u $UPLOAD_PATH/$UNSIGNED"
|
||||
fi
|
||||
|
||||
# Call the magic script on the signing server. Note that the user
|
||||
# ($SIGNING_USER) has sudo permissions but only to invoke this one script.
|
||||
# The signing user cannot make other sudo calls.
|
||||
#
|
||||
# We place output in $TMPFILE to extract the output file name later
|
||||
#
|
||||
ssh $SIGNING_USER@$SIGNING_SERVER sudo $SIGNING_SCRIPT -i $UPLOAD_PATH/$(basename $FILE) $UNSIGNED_OPTION -t $TYPE > $TMPFILE 2>&1
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Signing of $FILE failed"
|
||||
\rm -f $TMPFILE
|
||||
return 1
|
||||
fi
|
||||
|
||||
# The signing server script will output the name by which the newly signed
|
||||
# RPM can be found. This will be a unique filename (based on the unique
|
||||
# upload directory generated by the "-r" option above).
|
||||
#
|
||||
# The reason for this is so that we can archive all output files
|
||||
# and examine them later without them being overwriten. File paths are
|
||||
# typically of the form
|
||||
#
|
||||
# /export/signed_images/XXXXXXX_grub2-efi-2.02-0.44.el7.centos.tis.3.x86_64.rpm
|
||||
#
|
||||
# Extract the output name, and copy the RPM back into our system
|
||||
# (Note that we overwrite our original version of the RPM)
|
||||
#
|
||||
# Note that some packages (like grub) may produce multiple output RPMs (i.e.
|
||||
# multiple lines list output files.
|
||||
OUTPUT=`grep "Output written:" $TMPFILE | sed "s/Output written: //"`
|
||||
|
||||
# Check that we got something
|
||||
if [ "x$OUTPUT" == "x" ]; then
|
||||
echo "Could not determine output file -- check logs on signing server for errors"
|
||||
\cp $TMPFILE $MY_WORKSPACE/export/signing.log
|
||||
\rm -f $TMPFILE
|
||||
return 1
|
||||
fi
|
||||
|
||||
# The signing script can return multiple output files, if appropriate for
|
||||
# the input RPM source type. Copy each output RPM to our repo
|
||||
# Note that after we download the file we extract the base package name
|
||||
# from the RPM to find the name of the file that it *should* be named
|
||||
#
|
||||
# example:
|
||||
# we'd download "Zrqyeuzw_kernel-3.10.0-514.2.2.el7.20.tis.x86_64.rpm"
|
||||
# we'd figure out that the RPM name should be "kernel"
|
||||
# we look for "kernel" in the RPM filename, and rename
|
||||
# "Zrqyeuzw_kernel-3.10.0-514.2.2.el7.20.tis.x86_64.rpm" to
|
||||
# "kernel-3.10.0-514.2.2.el7.20.tis.x86_64.rpm"
|
||||
while read OUTPUT_FILE; do
|
||||
|
||||
# Download the file from the signing server
|
||||
local DOWNLOAD_FILENAME=$(basename $OUTPUT_FILE)
|
||||
scp -q $SIGNING_USER@$SIGNING_SERVER:$OUTPUT_FILE $(dirname $FILE)
|
||||
if [ $? -ne 0 ]; then
|
||||
\rm -f $TMPFILE
|
||||
echo "Copying file from signing server failed"
|
||||
return 1
|
||||
fi
|
||||
echo "Successfully retrieved $OUTPUT_FILE"
|
||||
|
||||
# figure out what the file should be named (strip away leading chars)
|
||||
local RPM_NAME=`rpm -qp $(dirname $FILE)/$DOWNLOAD_FILENAME --qf="%{name}"`
|
||||
local CORRECT_OUTPUT_FILE_NAME=`echo $DOWNLOAD_FILENAME | sed "s/^.*$RPM_NAME/$RPM_NAME/"`
|
||||
|
||||
# rename the file
|
||||
\mv -f $(dirname $FILE)/$DOWNLOAD_FILENAME $(dirname $FILE)/$CORRECT_OUTPUT_FILE_NAME
|
||||
|
||||
# replace the version of the file in results
|
||||
#
|
||||
# Potential hiccup in future -- this code currenty replaces any output file in EITHER
|
||||
# std or rt results which matches the filename we just downloaded from the signing.
|
||||
# server. This means there could be an issue where we sign something-ver-rel.arch.rpm
|
||||
# but we expect different versions of that RPM in std and in rt. Currently, we do not
|
||||
# have any RPMs which have that problem (all produced RPMs in rt have the "-rt" suffix
|
||||
# let along any "signed" rpms) but it's something of which to be aware.
|
||||
#
|
||||
# Also, note that we do not expect multiple RPMs in each repo to have the same filename.
|
||||
# We use "head -n 1" to handle that, but again it shouldn't happen.
|
||||
#
|
||||
for buildtype in std rt; do
|
||||
x=`find $MY_WORKSPACE/$buildtype/results/${MY_BUILD_ENVIRONMENT_TOP}-$buildtype -name $CORRECT_OUTPUT_FILE_NAME | head -n 1`
|
||||
if [ ! -z "$x" ]; then
|
||||
cp $(dirname $FILE)/$CORRECT_OUTPUT_FILE_NAME $x
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Have signed file $(dirname $FILE)/$CORRECT_OUTPUT_FILE_NAME"
|
||||
done <<< "$OUTPUT"
|
||||
|
||||
\rm -f $TMPFILE
|
||||
|
||||
# If we just signed a shim package, flag that shim needs to be rebuilt
|
||||
if [ "${TYPE}" == "shim" ]; then
|
||||
REBUILD_LIST="${REBUILD_LIST} shim-signed"
|
||||
fi
|
||||
|
||||
echo "Done"
|
||||
update_signed_pkg_list ${FILE}
|
||||
|
||||
return 0
|
||||
}
|
||||
|
||||
# Main script
|
||||
|
||||
if [ "x$MY_WORKSPACE" == "x" ]; then
|
||||
echo "Environment not set up -- abort"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ARCH="x86_64"
|
||||
SIGNING_SERVER=yow-tiks01
|
||||
SIGNING_USER=signing
|
||||
SIGNING_SCRIPT=/opt/signing/sign.sh
|
||||
UPLOAD_PATH=`ssh $SIGNING_USER@$SIGNING_SERVER sudo $SIGNING_SCRIPT -r`
|
||||
SIGNED_PKG_DB=${MY_WORKSPACE}/signed_pkg_list.txt
|
||||
REBUILD_LIST=""
|
||||
MY_BUILD_ENVIRONMENT_TOP=${MY_BUILD_ENVIRONMENT_TOP:-$MY_BUILD_ENVIRONMENT}
|
||||
|
||||
# Check that we were able to request a unique path for uploads
|
||||
echo $UPLOAD_PATH | grep -q "^Upload:"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to get upload path -- abort"
|
||||
exit 1
|
||||
fi
|
||||
UPLOAD_PATH=`echo $UPLOAD_PATH | sed "s%^Upload: %%"`
|
||||
|
||||
sign_kernels
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to sign kernels -- abort"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sign_shims
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to sign shims -- abort"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sign_grubs
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to sign grubs -- abort"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
update_repo std
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to update std repo -- abort"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
rebuild_pkgs
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to update builds with signed dependancies -- abort"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
update_repo std
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to update std repo -- abort"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
update_repo rt
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Failed to update rt repo -- abort"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
258
build-tools/sign-rpms
Executable file
258
build-tools/sign-rpms
Executable file
@ -0,0 +1,258 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Add file signature to RPMs
|
||||
#
|
||||
# This script will add file signature to rpms in a given directory.
|
||||
# The directory containing the RPMs must be passed as a parameter. There is no default location.
|
||||
#
|
||||
#
|
||||
|
||||
usage () {
|
||||
echo ""
|
||||
echo "Usage: "
|
||||
echo " sign-rpms -d|--pkg-dir <directory>"
|
||||
echo " -d --pkg-dir <directory> directory contain the RPMs to sign"
|
||||
echo " -h|--help this message"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# number of processors. The process will use all available processors by default.
|
||||
NPROCS=$(nproc)
|
||||
|
||||
export MOCK=/usr/bin/mock
|
||||
|
||||
# check input variables
|
||||
function check_vars {
|
||||
# need access to repo, which should normally be defined as MY_REPO in the env
|
||||
|
||||
if [ ! -z "$MY_REPO" ] && [ -d "$MY_REPO" ] ; then
|
||||
INTERNAL_REPO_ROOT=$MY_REPO
|
||||
fi
|
||||
|
||||
if [ -z "$INTERNAL_REPO_ROOT" ] ; then
|
||||
printf " unable to use \$MY_REPO (value \"$MY_REPO\")\n"
|
||||
printf " -- checking \$MY_REPO_ROOT_DIR (value \"$MY_REPO_ROOT_DIR\")\n"
|
||||
if [ ! -z "$MY_REPO_ROOT_DIR" ] && [ -d "$MY_REPO_ROOT_DIR/cgcs-root" ] ; then
|
||||
INTERNAL_REPO_ROOT=$MY_REPO_ROOT_DIR/cgcs-root
|
||||
printf " Found!\n"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$INTERNAL_REPO_ROOT" ] ; then
|
||||
printf " No joy -- checking for \$MY_WORKSPACE/cgcs-root\n"
|
||||
if [ -d "$MY_WORKSPACE/cgcs-root" ] ; then
|
||||
INTERNAL_REPO_ROOT=$MY_WORKSPACE/cgcs-root
|
||||
printf " Found!\n"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$INTERNAL_REPO_ROOT" ] ; then
|
||||
printf " Error -- could not locate cgcs-root repo.\n"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "$MY_BUILD_ENVIRONMENT" ] ; then
|
||||
printf " Error -- missing environment variable MY_BUILD_ENVIRONMENT"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "$MY_BUILD_DIR" ] ; then
|
||||
printf " Error -- missing environment variable MY_BUILD_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
}
|
||||
|
||||
#
|
||||
# this function will add IMA file signatures to all rpms in the Packages directory
|
||||
#
|
||||
# the process will copy the signing key and a makefile in the mock env under /tmp
|
||||
# it will also mount the Packages directory under /mnt/Packages
|
||||
# then mock will be invoked to sign the packages
|
||||
#
|
||||
# This process is using mock because the build servers do not have the same rpm / rpmsign version
|
||||
#
|
||||
|
||||
|
||||
function sign_packages {
|
||||
OLD_PWD=$PWD
|
||||
|
||||
_MOCK_PKG_DIR=/mnt/Packages
|
||||
_IMA_PRIV_KEY=ima_signing_key.priv
|
||||
_KEY_DIR=$MY_REPO/build-tools/signing
|
||||
_MOCK_KEY_DIR=/mnt/keys
|
||||
_SIGN_MAKEFILE=_sign_pkgs.mk
|
||||
_MK_DIR=$MY_REPO/build-tools/mk
|
||||
_MOCK_MK_DIR=/mnt/mk
|
||||
|
||||
# mock confgiuration file
|
||||
_MOCK_CFG=$MY_BUILD_DIR/${MY_BUILD_ENVIRONMENT}-sign.cfg
|
||||
|
||||
# recreate configuration file
|
||||
rm $_MOCK_CFG
|
||||
export BUILD_TYPE=std
|
||||
export MY_BUILD_DIR_TOP=$MY_BUILD_DIR
|
||||
modify-build-cfg $_MOCK_CFG
|
||||
# and customize
|
||||
echo "config_opts['chroot_setup_cmd'] = 'install shadow-utils make rpm-sign'" >> $_MOCK_CFG
|
||||
echo "config_opts['root'] = 'mock-sign'" >> $_MOCK_CFG
|
||||
echo "config_opts['basedir'] = '${MY_WORKSPACE}'" >> $_MOCK_CFG
|
||||
echo "config_opts['cache_topdir'] = '${MY_WORKSPACE}/mock-cache'" >> $_MOCK_CFG
|
||||
|
||||
echo "Signing packages in $_PKG_DIR with $NPROCS threads"
|
||||
echo "using development key $_KEY_DIR/$_IMA_PRIV_KEY"
|
||||
|
||||
printf "Initializing mock environment\n"
|
||||
|
||||
# invoke make in mock to sign packages.
|
||||
# this call will also create and initialize the mock env
|
||||
eval $MOCK -q -r $_MOCK_CFG \'--plugin-option=bind_mount:dirs=[\(\"$_PKG_DIR\", \"$_MOCK_PKG_DIR\"\),\(\"$_MK_DIR\",\"$_MOCK_MK_DIR\"\),\(\"$_KEY_DIR\",\"$_MOCK_KEY_DIR\"\)]\' --shell \"cd $_MOCK_PKG_DIR\; make -j $NPROCS -f $_MOCK_MK_DIR/$_SIGN_MAKEFILE KEY=$_MOCK_KEY_DIR/$_IMA_PRIV_KEY\"
|
||||
|
||||
retval=$?
|
||||
|
||||
printf "Cleaning mock environment\n"
|
||||
$MOCK -q -r $_MOCK_CFG --scrub=all
|
||||
|
||||
if [ $retval -ne 0 ] ; then
|
||||
echo "failed to add file signatures to RPMs in mock environment."
|
||||
return $retval
|
||||
fi
|
||||
|
||||
cd $OLD_PWD
|
||||
|
||||
}
|
||||
|
||||
function _copy_and_sign {
|
||||
|
||||
# upload rpms to server
|
||||
scp $_PKG_DIR/*.rpm $SIGNING_USER@$SIGNING_SERVER:$_UPLOAD_DIR
|
||||
retval=$?
|
||||
if [ $retval -ne 0 ] ; then
|
||||
echo "ERROR: failed to copy RPM files to signing server."
|
||||
return $retval
|
||||
fi
|
||||
|
||||
# get server to sign packages.
|
||||
ssh $SIGNING_USER@$SIGNING_SERVER -- sudo $SIGNING_SERVER_SCRIPT -s -d $sub
|
||||
retval=$?
|
||||
if [ $retval -ne 0 ] ; then
|
||||
echo "ERROR: failed to sign RPM files."
|
||||
return $retval
|
||||
fi
|
||||
|
||||
# download results back. This overwrites the original files.
|
||||
scp $SIGNING_USER@$SIGNING_SERVER:$_UPLOAD_DIR/*.rpm $_PKG_DIR
|
||||
retval=$?
|
||||
if [ $retval -ne 0 ] ; then
|
||||
echo "ERROR: failed to copy signed RPM files back from signing server."
|
||||
return $retval
|
||||
fi
|
||||
|
||||
return $retval
|
||||
|
||||
}
|
||||
|
||||
|
||||
function sign_packages_on_server {
|
||||
|
||||
retval=0
|
||||
|
||||
# obtain temporary diretory to upload RPMs on signing server
|
||||
_UPLOAD_DIR=`ssh $SIGNING_USER@$SIGNING_SERVER -- sudo $SIGNING_SERVER_SCRIPT -r`
|
||||
|
||||
retval=$?
|
||||
if [ $retval -ne 0 ] ; then
|
||||
echo "failed to obtain upload directory from signing server."
|
||||
return $retval
|
||||
fi
|
||||
|
||||
# extract base chroot dir and rpm dir within chroot
|
||||
read base com sub <<< $_UPLOAD_DIR
|
||||
|
||||
# this is the upload temp dir, outside of chroot env
|
||||
_UPLOAD_DIR=$base$sub
|
||||
|
||||
_copy_and_sign
|
||||
retval=$?
|
||||
|
||||
# cleanup
|
||||
ssh $SIGNING_USER@$SIGNING_SERVER rm $_UPLOAD_DIR/*.rpm
|
||||
if [ $? -ne 0 ] ; then
|
||||
echo "Warning : failed to remove rpms from temporary upload directory."
|
||||
fi
|
||||
ssh $SIGNING_USER@$SIGNING_SERVER rmdir $_UPLOAD_DIR
|
||||
if [ $? -ne 0 ] ; then
|
||||
echo "Warning : failed to remove temporary upload directory."
|
||||
fi
|
||||
|
||||
return $retval
|
||||
}
|
||||
|
||||
|
||||
|
||||
#############################################
|
||||
# Main code
|
||||
#############################################
|
||||
|
||||
# Check args
|
||||
HELP=0
|
||||
SIGNING_SERVER=yow-tiks01
|
||||
SIGNING_USER=signing
|
||||
SIGNING_SERVER_SCRIPT=/opt/signing/sign_rpms_18.03.sh
|
||||
|
||||
# return value
|
||||
retval=0
|
||||
|
||||
# read the options
|
||||
TEMP=`getopt -o hd: --long help,pkg-dir: -n 'test.sh' -- "$@"`
|
||||
if [ $? -ne 0 ] ; then
|
||||
echo "Invalid parameters - exiting"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
eval set -- "$TEMP"
|
||||
|
||||
# extract options and their arguments into variables.
|
||||
while true ; do
|
||||
case "$1" in
|
||||
-h|--help) HELP=1 ; shift ;;
|
||||
-d|--pkg-dir) _PKG_DIR="$2"; shift; shift ;;
|
||||
--) shift ; break ;;
|
||||
*) echo "Internal error : unexpected parameter $2" ; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $HELP -eq 1 ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# package directory must be defined
|
||||
if [ -z "$_PKG_DIR" ]; then
|
||||
echo "Need package directory. Use -d/--pkg-dir option"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ... and must exist
|
||||
if [ ! -d "$_PKG_DIR" ]; then
|
||||
echo "Package directory $_PKG_DIR does not exist"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Init variables
|
||||
check_vars
|
||||
|
||||
echo signing $_PKG_DIR
|
||||
|
||||
# sign all rpms
|
||||
if [ "$USER" == "jenkins" ]; then
|
||||
sign_packages_on_server
|
||||
retval=$?
|
||||
else
|
||||
sign_packages
|
||||
retval=$?
|
||||
fi
|
||||
|
||||
exit $retval
|
||||
|
||||
62
build-tools/sign_iso_formal.sh
Executable file
62
build-tools/sign_iso_formal.sh
Executable file
@ -0,0 +1,62 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This script makes a request to the signing server to sign a .iso with the
|
||||
# formal key. It will only work for users authorized to access the signing
|
||||
# server. The detached signature is placed in the same path as the .iso as
|
||||
# the file bootimage.sig
|
||||
#
|
||||
# Script written to be quite simple
|
||||
|
||||
if [ "x$1" == "x" ]; then
|
||||
echo "You must specify an ISO file to sign"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ISO_FILE_PATH=$1
|
||||
ISO_FILE_NAME=$(basename ${ISO_FILE_PATH})
|
||||
ISO_FILE_ROOT=$(dirname ${ISO_FILE_PATH})
|
||||
ISO_FILE_NOEXT="${ISO_FILE_NAME%.*}"
|
||||
SIGNING_SERVER="signing@yow-tiks01"
|
||||
GET_UPLOAD_PATH="sudo /opt/signing/sign.sh -r"
|
||||
REQUEST_SIGN="sudo /opt/signing/sign_iso.sh"
|
||||
SIGNATURE_FILE="$ISO_FILE_NOEXT.sig"
|
||||
|
||||
# Make a request for an upload path
|
||||
# Output is a path where we can upload stuff, of the form
|
||||
# "Upload: /tmp/sign_upload.5jR11pS0"
|
||||
UPLOAD_PATH=`ssh ${SIGNING_SERVER} ${GET_UPLOAD_PATH}`
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could not get upload path. Do you have permissions on the signing server?"
|
||||
exit 1
|
||||
fi
|
||||
UPLOAD_PATH=`echo ${UPLOAD_PATH} | cut -d ' ' -f 2`
|
||||
|
||||
echo "Uploading file"
|
||||
scp -q ${ISO_FILE_PATH} ${SIGNING_SERVER}:${UPLOAD_PATH}
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could not upload ISO"
|
||||
exit 1
|
||||
fi
|
||||
echo "File uploaded to signing server -- signing"
|
||||
|
||||
# Make the signing request.
|
||||
# Output is path of detached signature
|
||||
RESULT=`ssh ${SIGNING_SERVER} ${REQUEST_SIGN} ${UPLOAD_PATH}/${ISO_FILE_NAME}`
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could not perform signing -- output $RESULT"
|
||||
ssh ${SIGNING_SERVER} rm -f ${UPLOAD_PATH}/${ISO_FILE_NAME}
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Signing complete. Downloading detached signature"
|
||||
scp -q ${SIGNING_SERVER}:${RESULT} ${ISO_FILE_ROOT}/${SIGNATURE_FILE}
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could not download newly signed file"
|
||||
ssh ${SIGNING_SERVER} rm -f ${UPLOAD_PATH}/${ISO_FILE_NAME}
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Clean up (ISOs are big)
|
||||
ssh ${SIGNING_SERVER} rm -f ${UPLOAD_PATH}/${ISO_FILE_NAME}
|
||||
|
||||
echo "${ISO_FILE_ROOT}/${SIGNATURE_FILE} detached signature"
|
||||
51
build-tools/sign_patch_formal.sh
Executable file
51
build-tools/sign_patch_formal.sh
Executable file
@ -0,0 +1,51 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This script makes a request to the signing server to sign a .patch with the
|
||||
# formal key. It will only work for users authorized to access the signing
|
||||
# server.
|
||||
#
|
||||
# Script written to be quite simple
|
||||
|
||||
if [ "x$1" == "x" ]; then
|
||||
echo "You must specify a patch file to sign"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PATCH_FILE_PATH=$1
|
||||
PATCH_FILE_NAME=$(basename ${PATCH_FILE_PATH})
|
||||
SIGNING_SERVER="signing@yow-tiks01"
|
||||
GET_UPLOAD_PATH="sudo /opt/signing/sign.sh -r"
|
||||
REQUEST_SIGN="sudo /opt/signing/sign_patch.sh"
|
||||
|
||||
# Make a request for an upload path
|
||||
# Output is a path where we can upload stuff, of the form
|
||||
# "Upload: /tmp/sign_upload.5jR11pS0"
|
||||
UPLOAD_PATH=`ssh ${SIGNING_SERVER} ${GET_UPLOAD_PATH}`
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could not get upload path. Do you have permissions on the signing server?"
|
||||
exit 1
|
||||
fi
|
||||
UPLOAD_PATH=`echo ${UPLOAD_PATH} | cut -d ' ' -f 2`
|
||||
|
||||
scp -q ${PATCH_FILE_PATH} ${SIGNING_SERVER}:${UPLOAD_PATH}
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could upload patch"
|
||||
exit 1
|
||||
fi
|
||||
echo "File uploaded to signing server"
|
||||
|
||||
# Make the signing request.
|
||||
# Output is path of newly signed file
|
||||
RESULT=`ssh ${SIGNING_SERVER} ${REQUEST_SIGN} ${UPLOAD_PATH}/${PATCH_FILE_NAME}`
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could not perform signing -- output $RESULT"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Signing complete. Downloading"
|
||||
scp -q ${SIGNING_SERVER}:${RESULT} ${PATCH_FILE_PATH}
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Could not download newly signed file"
|
||||
exit 1
|
||||
fi
|
||||
echo "${PATCH_FILE_PATH} now signed with formal key"
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user