StarlingX open source release updates

Signed-off-by: Dean Troyer <dtroyer@gmail.com>

.gitignore

@@ -0,0 +1,4 @@
+*.swp
+/cgcs-centos-repo
+/cgcs-tis-repo
+/cgcs-3rd-party-repo

LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

addons/.gitignore

@@ -0,0 +1,3 @@
+/wr-avs
+/wr-cgcs
+

build-data/unbuilt_rpm_patterns

@@ -0,0 +1,19 @@
+[-]locale[-]
+[-]doc[-]
+[-]dbg[-]
+vswitch-staticdev
+guest-scale-agent-staticdev
+vim-spell
+openssh-server-sysvinit
+openstack-neutron-linuxbridge
+^libcacard-
+^kernel-bootwrapper
+^kernel-doc-
+^kernel-abi-whitelists
+^kernel-debug-
+^kernel-kdump
+^kernel-rt-bootwrapper
+^kernel-rt-doc-
+^kernel-rt-abi-whitelists
+^kernel-rt-debug-
+^kernel-rt-kdump

build-tools/Cached_Data.txt

@@ -0,0 +1,78 @@
+Data on an source rpm:
+
+   location:
+      ${MY_WORKSPACE}/${BUILD_TYPE}/rpmbuild/SPECS/${SRPM_FILE_NAME}/
+
+      files:
+         *.spec     # spec file found in the source rpm
+
+      subdirectories:
+         NAMES/     # Directory contains an emtpy file, where the file name 
+                    # is the name of the source rpm.
+
+         SERVICES/  # Directory contains zero or more emtpy files, where the 
+                    # file name is the name of the service provided by one 
+                    # or more of the rpms.
+   
+         BUILDS/    # Directory contains emtpy files, where the file name is 
+                    # the name of a binary rpm built from the source rpm.
+
+         BUILDS_VR/ # Directory contains emtpy files, where the file name is 
+                    # the name-verion-release of a binary rpm built from the 
+                    # source rpm.
+
+   location:
+      ${MY_WORKSPACE}/${BUILD_TYPE}/rpmbuild/SOURCES/${SRPM_FILE_NAME}/
+
+      files:
+         BIG        # if it exists, it contains one line, the numeric value 
+                    # extracted from build_srpms.data if the line 
+                    # BUILD_IS_BIG=### if present.  
+                    # This is the estimated filesystem size (GB) required to 
+                    # host a mock build of the package.
+                    # Note: not all parallel build environments are the same 
+                    # size.  The smallest build environmnet is 3 GB and this 
+                    # is sufficient for most packages.  Don't bother adding a 
+                    # BUILD_IS_BIG=### directive unless 3 gb is proven to be
+                    # insufficient.
+
+         SLOW       # if it exists, it contains one line, the numeric value i
+                    # extracted from build_srpms.data if the line 
+                    # BUILD_IS_SLOW=### if present.
+                    # This is the estimated build time (minutes) required to 
+                    # host perform a mock build of the package.
+                    # Note: Currently we only use this value as a boolean. 
+                    # Non-zero and we try to start the build of this package 
+                    # earlier rather than later.  Build times >= 3 minutes are 
+                    # worth anotating.  Else don't bother adding a 
+                    # BUILD_IS_SLOW=### directive
+e.g.
+
+cd $MY_WORKSPACE/std/rpmbuild/SPECS/openstack-cinder-9.1.1-0.tis.40.src.rpm
+find .
+./BUILDS
+./BUILDS/openstack-cinder
+./BUILDS/python-cinder
+./BUILDS/python-cinder-tests
+./NAMES
+./NAMES/openstack-cinder
+./SERVICES
+./SERVICES/cinder
+./BUILDS_VR
+./BUILDS_VR/openstack-cinder-9.1.1-0.tis.40
+./BUILDS_VR/python-cinder-9.1.1-0.tis.40
+./BUILDS_VR/python-cinder-tests-9.1.1-0.tis.40
+./openstack-cinder.spec
+
+
+e.g.
+cd $MY_WORKSPACE/std/rpmbuild/SOURCES/kernel-3.10.0-514.16.1.el7.29.tis.src.rpm
+find .
+./BIG
+./SLOW
+
+cat ./BIG
+8
+
+cat ./SLOW
+12

build-tools/audit-pkgs

@@ -0,0 +1,70 @@
+#!/bin/bash
+
+rpm_compare () {
+   local r="$1"
+   local r2="$2"
+   local line
+   local f=$(basename $r)
+   local f2=$(basename $r2)
+
+   rpm -q --dump --nosignature -p $r  | awk ' { print $1 "\n" $1 " " $5 " " $6 " " $7 " " $8 " " $9 " " $10 " " $11 } ' > /tmp/dump.new
+   rpm -q --dump --nosignature -p $r2 | awk ' { print $1 "\n" $1 " " $5 " " $6 " " $7 " " $8 " " $9 " " $10 " " $11 } ' > /tmp/dump.old
+   first_line=1
+   diff -y -W 200 --suppress-common-lines /tmp/dump.new /tmp/dump.old | grep '|' |
+   while read -r line; do
+      left=$(echo "$line" | awk -F '|' '{ print $1 }')
+      right=$(echo "$line" | awk -F '|' '{ print $2 }')
+      left_f=$(echo "$left" | awk '{ print $1 }')
+      right_f=$(echo "$right" | awk '{ print $1 }')
+      if [ "$left_f" != "$right_f" ];then
+         continue
+      fi
+      if  [ $first_line -eq 1 ]; then
+         echo ""
+         echo "$f   vs   $f2"
+         first_line=0
+      fi
+      echo "$line"
+   done
+}
+
+echo ""
+echo "======================================================"
+echo "Auditing built packages vs unpatched upstream packages"
+echo "======================================================"
+for r in $(find $MY_WORKSPACE/*/rpmbuild/RPMS -name '*.rpm' | grep -v '.src.rpm' | grep -v debuginfo); do
+   f=$(basename $r)
+   f2=$(echo $f | sed 's#[.]tis[.][0-9]*[.]#.#' | sed 's#[.]tis[.]#.#')
+   r2=$(find $MY_REPO/cgcs-centos-repo/Binary/ -name $f2)
+   if [ "$r2" == "" ]; then
+      # Probably one of our own
+      # echo "Couldn't find '$f2'"
+      continue
+   fi
+   rpm_compare "$r" "$r2"
+done
+
+echo ""
+echo "============================"
+echo "Auditing built for conflicts"
+echo "============================"
+grep 'conflicts with file from package' -r --binary-files=without-match $MY_WORKSPACE/*/results/ |
+while read -r line; do
+   w=$(echo "$line" | awk '{ print $8 }')".rpm"
+   w2=$(echo "$line" | awk '{ print $14 }')".rpm"
+   echo "$w $w2"
+done | sort --unique | sed 's#bash-completion-1:#bash-completion-#' |
+while read -r line2; do
+   f=$(echo "$line2" | awk '{ print $1 }')
+   f2=$(echo "$line2" | awk '{ print $2 }')
+   r=$(find $MY_REPO/cgcs-centos-repo/Binary/ $MY_WORKSPACE/*/rpmbuild/RPMS -name $f)
+   r2=$(find $MY_REPO/cgcs-centos-repo/Binary/ $MY_WORKSPACE/*/rpmbuild/RPMS -name $f2)
+   # echo ""
+   # echo "$f   vs   $f2"
+   # echo "$r   vs   $r2"
+   if [ "$r" != "" ] && [ "$r2" != "" ]; then
+      rpm_compare "$r" "$r2"
+   fi
+done
+
+

build-tools/branching/branch_and_tag.sh

@@ -0,0 +1,86 @@
+#!/bin/bash
+
+# The purpose of this script is to create branches and tags that follow a convention
+# If the desired branch already exists, it is skipped.
+# If the desired tag already exists, it is skipped.
+
+OLD_TAG=vCGCS_DEV_0018
+NEW_TAG=vCGCS_DEV_0019
+
+OLD_BRANCH=CGCS_DEV_0018
+NEW_BRANCH=CGCS_DEV_0019
+
+if [ -z "$MY_REPO" ]; then 
+  echo "MY_REPO is unset"
+  exit 1
+else
+  echo "MY_REPO is set to '$MY_REPO'"
+fi
+
+if [ -d "$MY_REPO" ]; then 
+  cd $MY_REPO
+  echo "checking out and pulling old branch"
+  wrgit checkout $OLD_BRANCH
+  if [ $? -ne 0 ]; then
+    echo "ERROR: wrgit checkout $OLD_BRANCH"
+    exit 1
+  fi
+
+  wrgit pull
+  if [ $? -ne 0 ]; then
+    echo "ERROR: wrgit pull"
+    exit 1
+  fi
+else
+  echo "Could not change to diectory '$MY_REPO'"
+  exit 1
+fi
+
+echo "Finding subgits"
+SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
+
+# Go through all subgits and create the NEW_BRANCH if it does not already exist
+# Go through all subgits and create the NEW_TAG if it does not already exist
+for subgit in $SUBGITS; do
+ echo ""
+ echo ""
+ pushd $subgit > /dev/null
+ git fetch
+ git fetch --tags
+ # check if destination branch already exists
+ echo "$subgit"
+ branch_check=`git branch -a --list $NEW_BRANCH`
+ if [ -z "$branch_check" ]
+ then
+   echo "Creating $NEW_BRANCH"
+   git checkout $OLD_BRANCH
+   git checkout -b $NEW_BRANCH
+   git push origin $NEW_BRANCH:$NEW_BRANCH
+ else
+   echo "$NEW_BRANCH already exists"
+ fi
+ tag_check=`git tag -l $NEW_TAG`
+ if [ -z "$tag_check" ]
+ then
+   echo "Creating $NEW_TAG"
+   # create tag
+   git checkout $NEW_BRANCH
+   git pull origin
+   git tag $NEW_TAG
+   git push origin $NEW_TAG
+ else
+   echo "$NEW_TAG already exists"
+ fi
+
+ popd > /dev/null
+done
+
+echo "All done.  branches and tags are pushed"
+
+
+
+
+
+
+
+

build-tools/branching/create_branches_and_tags.sh

@@ -0,0 +1,56 @@
+#!/bin/bash
+
+if [ x"$1" = x ] ; then
+    echo "ERROR: You must specify a name to create branches and tags"
+    exit 1
+fi
+branch=$1
+tag="v$branch"
+
+
+
+echo "Finding subgits"
+SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
+
+# Go through all subgits and create the branch and tag if they does not already exist
+for subgit in $SUBGITS; do
+   echo ""
+   echo ""
+   pushd $subgit > /dev/null
+
+   # check if destination branch already exists
+   echo "$subgit"
+   branch_check=`git branch -a --list $branch`
+   if [ -z "$branch_check" ]
+   then
+      echo "Creating branch $branch"
+      git checkout -b $branch
+      if [ $? != 0 ] ; then
+         echo "ERROR: Could not exec: git checkout -b $branch"
+         popd > /dev/null
+         exit 1
+      fi
+      # git push origin $branch:$branch
+   else
+      echo "Branch $branch already exists"
+      git checkout $branch
+   fi
+
+   tag_check=`git tag -l $tag`
+   if [ -z "$tag_check" ]
+   then
+      echo "Creating tag $tag"
+      git tag $tag
+      if [ $? != 0 ] ; then
+         echo "ERROR: Could not exec: git tag $tag"
+         popd > /dev/null
+         exit 1
+      fi
+      # git push origin $tag
+   else
+      echo "Tag $tag already exists"
+   fi
+  
+   popd > /dev/null
+done
+

build-tools/branching/create_tags.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+
+if [ x"$1" = x ] ; then
+    echo "ERROR: You must specify a name to create tags"
+    exit 1
+fi
+tag=$1
+
+
+echo "Finding subgits"
+SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
+
+# Go through all subgits and create the tag if it does not already exist
+for subgit in $SUBGITS; do
+   echo ""
+   echo ""
+   pushd $subgit > /dev/null
+
+   tag_check=`git tag -l $tag`
+   if [ -z "$tag_check" ]
+   then
+      echo "Creating tag $tag"
+      git tag $tag
+      if [ $? != 0 ] ; then
+         echo "ERROR: Could not exec: git tag $tag"
+         popd > /dev/null
+         exit 1
+      fi
+      # git push origin $tag
+   else
+      echo "Tag $tag already exists"
+   fi
+  
+   popd > /dev/null
+done
+

build-tools/branching/push_branches_tags.sh

@@ -0,0 +1,42 @@
+
+#!/bin/bash
+
+if [ x"$1" = x ] ; then
+    echo "ERROR: You must specify a name to create branches and tags"
+    exit 1
+fi
+branch=$1
+tag="v$branch"
+
+
+
+echo "Finding subgits"
+SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
+
+# Go through all subgits and create the branch and tag if they does not already exist
+for subgit in $SUBGITS; do
+   echo ""
+   echo ""
+   pushd $subgit > /dev/null
+
+   # check if destination branch already exists
+   echo "$subgit"
+   echo "Pushing branch $branch"
+   git push origin $branch:$branch
+   if [ $? != 0 ] ; then
+      echo "ERROR: Could not exec: git push origin $branch:$branch"
+      popd > /dev/null
+      exit 1
+   fi
+
+   echo "Pushing tag $tag"
+   git push origin $tag
+   if [ $? != 0 ] ; then
+      echo "ERROR: Could not exec: git push origin $tag"
+      popd > /dev/null
+      exit 1
+   fi
+  
+   popd > /dev/null
+done
+

build-tools/branching/push_tags.sh

@@ -0,0 +1,29 @@
+#!/bin/bash
+
+if [ x"$1" = x ] ; then
+    echo "ERROR: You must specify a name to push tags"
+    exit 1
+fi
+tag=$1
+
+
+echo "Finding subgits"
+SUBGITS=`find . -type d -name ".git" | sed "s%/\.git$%%"`
+
+# Go through all subgits and create the tag if it does not already exist
+for subgit in $SUBGITS; do
+   echo ""
+   echo ""
+   pushd $subgit > /dev/null
+
+   echo "Creating tag $tag"
+   git push origin $tag
+   if [ $? != 0 ] ; then
+      echo "ERROR: Could not exec: git push origin $tag"
+      popd > /dev/null
+      exit 1
+   fi
+  
+   popd > /dev/null
+done
+

build-tools/build-guest

@@ -0,0 +1,338 @@
+#!/bin/env bash
+
+#
+# Build the tis-centos-image.img or tis-centos-image-rt.img file
+#
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+# NOTE: TMP_DIR must end in '/'
+# NOTE: /tmp/ is now tmpfs like.  Can't be trusted across multiple mock commands
+# TMP_DIR=/tmp/
+TMP_DIR=/
+
+# Use RPMs from the std build only, for now
+export BUILD_TYPE=std
+export MY_BUILD_DIR_TOP=$MY_BUILD_DIR
+
+function init_vars {
+   # Output path (current dir unless MY_WORKSPACE defined)
+   OUTPUT_DIR="$PWD/export"
+   if [ ! -z "$MY_WORKSPACE" ] && [ -d "$MY_WORKSPACE" ] ; then
+       OUTPUT_DIR="$MY_WORKSPACE/export"
+       CGCS_REPO_DIR="$MY_WORKSPACE/rpmbuild/RPMS"
+   fi
+
+   if [ -n "$MY_GUEST_DIR" ]; then
+       GUEST_DIR=$MY_GUEST_DIR
+   else
+       GUEST_DIR=$MY_WORKSPACE/guest
+   fi
+
+   MOCK=/usr/bin/mock
+   if [ $VERBOSE -eq 0 ]; then
+       MOCK="$MOCK -q"
+   fi
+
+   # Path to guest configuration
+   GUEST_BUILD_DIR=$DIR/build_guest
+   GUEST_BUILD_CMD=$GUEST_BUILD_DIR/build-guest-image.py
+   if [ $VERBOSE -eq 1 ]; then
+       GUEST_BUILD_CMD="$GUEST_BUILD_CMD -x"
+   fi
+
+   if [ $BUILD_MODE == 'std' ]; then
+       OUTPUT_FILE=$OUTPUT_DIR/tis-centos-guest.img
+   elif [ $BUILD_MODE == 'rt' ]; then
+       OUTPUT_FILE=$OUTPUT_DIR/tis-centos-guest-rt.img
+   else
+       printf "   Error -- unknown BUILD_MODE '$BUILD_MODE'\n";
+       exit 1
+   fi
+}
+
+
+function check_vars {
+   # Where to store data
+   printf "Finding cgcs-root\n"
+   printf "  Checking \$MY_REPO (value \"$MY_REPO\")\n"
+
+   if [ ! -z "$MY_REPO" ] && [ -d "$MY_REPO" ] ; then
+      INTERNAL_REPO_ROOT=$MY_REPO
+      printf "  Found!\n"
+   fi
+
+   if [ -z "$INTERNAL_REPO_ROOT" ] ; then
+      printf "  No joy -- checking \$MY_REPO_ROOT_DIR (value \"$MY_REPO_ROOT_DIR\")\n"
+      if [ ! -z "$MY_REPO_ROOT_DIR" ] && [ -d "$MY_REPO_ROOT_DIR/cgcs-root" ] ; then
+          INTERNAL_REPO_ROOT=$MY_REPO_ROOT_DIR/cgcs-root
+          printf "  Found!\n"
+      fi
+   fi
+
+   if [ -z "$INTERNAL_REPO_ROOT" ] ; then
+      printf "  No joy -- checking for \$MY_WORKSPACE/cgcs-root\n"
+      if [ -d "$MY_WORKSPACE/cgcs-root" ] ; then
+          INTERNAL_REPO_ROOT=$MY_WORKSPACE/cgcs-root
+          printf "  Found!\n"
+      fi
+   fi
+
+   if [ -z "$INTERNAL_REPO_ROOT" ] ; then
+      printf "  Error -- could not locate cgcs-root repo.\n"
+      exit 1
+   fi
+
+   if [ "x$MY_BUILD_CFG" == "x" ];then
+       printf "  Error -- reqiure MY_BUILD_CFG to be defined.\n"
+       exit 1
+   fi
+
+   RELEASE_INFO=$INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/middleware/recipes-common/build-info/release-info.inc
+   export PLATFORM_RELEASE=$(source $RELEASE_INFO && echo $PLATFORM_RELEASE)
+}
+
+
+function create_rootfs {
+    printf "\nCreating guest file system\n"
+
+    mkdir -p $GUEST_DIR
+    if [ $? -ne 0 ]; then
+	printf "   Error -- Could not create $GUEST_DIR\n";
+	exit 1
+    fi
+
+    # Place build-time environment variables in mock configuration
+    GUEST_ENV="${MY_BUILD_ENVIRONMENT}-guest"
+    GUEST_CFG=$GUEST_DIR/$MY_BUILD_ENVIRONMENT_FILE
+
+    MY_BUILD_ENVIRONMENT=$GUEST_ENV ${DIR}/modify-build-cfg $GUEST_CFG
+    if [ $? -ne 0 ]; then
+	printf "   Error -- Could not update $GUEST_CFG\n";
+	exit 1
+    fi
+
+    # Setup mock directories for the guest
+    if [ -d /localdisk/loadbuild/mock ]; then
+	LNK=/localdisk/loadbuild/mock/$GUEST_ENV
+	if [ ! -L $LNK ]; then
+	    ln -s $GUEST_DIR $LNK
+	fi
+    fi
+
+    if [ -d /localdisk/loadbuild/mock-cache ]; then
+	mkdir -p $GUEST_DIR/cache
+	LNK=/localdisk/loadbuild/mock-cache/$GUEST_ENV
+	if [ ! -L $LNK ]; then
+	    ln -s $GUEST_DIR/cache $LNK
+	fi
+    fi
+
+    # Setup mock chroot environment
+    $MOCK -r $GUEST_CFG --clean && $MOCK -r $GUEST_CFG --init
+    if [ $? -ne 0 ]; then
+	printf "   Error -- Failed to setup guest mock chroot\n";
+	exit 1
+    fi
+
+    # Install the RPMs to the root filesystem
+    
+    # Note that the "rt" build needs access to both local-std and local-rt repos
+    local EXTRA_REPOS=""
+
+    if [ $BUILD_MODE == 'std' ]; then
+       INC_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/rpm-install-list.txt)
+       TIS_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/image.inc)
+    elif [ $BUILD_MODE == 'rt' ]; then
+       INC_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/rpm-install-list-rt.txt)
+       TIS_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/image-rt.inc)
+       EXTRA_REPOS="--enablerepo local-rt"
+    else
+       printf "   Error -- unknown BUILD_MODE '$BUILD_MODE'\n";
+       exit 1
+    fi
+
+    $MOCK -r $GUEST_CFG --install ${INC_RPM_LIST} ${TIS_RPM_LIST} ${EXTRA_REPOS}
+    if [ $? -ne 0 ]; then
+	printf "   Error -- Failed to install RPM packages\n";
+	exit 1
+    fi
+    
+    # Remove RPMs that are not required in image (pruned package list)
+    # NOTE: these are automatically installed from the mock init not
+    # through dependencies.
+    EXC_RPM_LIST=$(grep -v '^#' ${GUEST_BUILD_DIR}/rpm-remove-list.txt)
+
+    $MOCK -r $GUEST_CFG --remove ${EXC_RPM_LIST}
+    if [ $? -ne 0 ]; then
+	printf "   Error -- Failed to remove RPM packages\n";
+	exit 1
+    fi
+
+    printf "  Done\n"
+}
+
+
+function update_rootfs {
+    printf "\nCustomizing guest file system\n"
+
+    # Copy over skeleton configuration files
+    for GUEST_ROOTFS in $GUEST_BUILD_DIR/rootfs $GUEST_BUILD_DIR/rootfs-$BUILD_MODE;
+    do
+        for f in $(cd $GUEST_ROOTFS && find . -type f | cut -c3-);
+        do
+            echo "$MOCK -r $GUEST_CFG --copyin $GUEST_ROOTFS/$f $f"
+	    $MOCK -r $GUEST_CFG --copyin $GUEST_ROOTFS/$f $f
+	    if [ $? -ne 0 ]; then
+	        printf "   Error -- Failed to copyin file $f\n";
+	        exit 1
+	    fi
+        done
+    done
+
+    # Run the root file system setup script inside the chroot
+    ROOTFS_SETUP=rootfs-setup.sh
+    $MOCK -r $GUEST_CFG --copyin $GUEST_BUILD_DIR/$ROOTFS_SETUP $TMP_DIR && \
+    if [ $BUILD_MODE == 'rt' ]; then
+       ROOTFS_SETUP_CMD="$TMP_DIR$ROOTFS_SETUP --rt"
+    elif [ $BUILD_MODE == 'std' ]; then
+       ROOTFS_SETUP_CMD="$TMP_DIR$ROOTFS_SETUP --std"
+    else
+       ROOTFS_SETUP_CMD="$TMP_DIR$ROOTFS_SETUP"
+    fi
+    $MOCK -r $GUEST_CFG --chroot "$ROOTFS_SETUP_CMD"
+    if [ $? -ne 0 ]; then
+	printf "   Error -- Failed to run guest $ROOTFS_SETUP\n";
+	exit 1
+    fi
+    $MOCK -r $GUEST_CFG --chroot "rm -f $TMP_DIR$ROOTFS_SETUP"
+    if [ $? -ne 0 ]; then
+	printf "   Error -- Failed to delete $ROOTFS_SETUP from guest\n";
+	exit 1
+    fi
+
+    printf "  Done\n"
+}
+
+
+function build_image {
+    # Build the image
+    printf "\nBuilding guest image $OUTPUT_FILE\n"
+
+    mkdir -p $OUTPUT_DIR
+    if [ $? -ne 0 ]; then
+	printf "   Error -- Could not create $OUTPUT_DIR\n";
+	exit 1
+    fi
+
+    # Build guest rootfs archive
+    ROOTFS_SPACE=$((500*1024*1024))
+    ROOTFS_TAR=rootfs.tar
+    ROOTFS_EXCLUDE=rootfs-exclude.txt
+
+    $MOCK -r $GUEST_CFG --copyin $GUEST_BUILD_DIR/$ROOTFS_EXCLUDE $TMP_DIR
+    $MOCK -r $GUEST_CFG --chroot -- tar -cf $TMP_DIR$ROOTFS_TAR -X $TMP_DIR$ROOTFS_EXCLUDE --numeric-owner /
+    $MOCK -r $GUEST_CFG --copyout $TMP_DIR$ROOTFS_TAR $GUEST_DIR
+    $MOCK -r $GUEST_CFG --chroot -- rm -f $TMP_DIR$ROOTFS_TAR
+
+    $GUEST_BUILD_CMD -i $GUEST_DIR/$ROOTFS_TAR -o $OUTPUT_FILE -s $ROOTFS_SPACE
+    if [ $? -ne 0 ]; then
+	printf "   Error -- Failed to build guest image\n";
+	exit 1
+    fi
+
+    printf "  Done\n"
+}
+
+
+function clean_guest {
+    printf "\nCleaning the guest $GUEST_DIR\n"
+
+    if [ ! -e $GUEST_DIR ]; then	
+	printf "  Done...nothing to do\n";
+	exit 0
+    fi
+
+    # Place build-time environment variables in mock configuration
+    GUEST_ENV="${MY_BUILD_ENVIRONMENT}-guest"
+    GUEST_CFG=$GUEST_DIR/$MY_BUILD_ENVIRONMENT_FILE
+
+    if [ ! -e $GUEST_CFG ]; then
+	MY_BUILD_ENVIRONMENT=$GUEST_ENV ${DIR}/modify-build-cfg $GUEST_CFG
+	if [ $? -ne 0 ]; then
+	    printf "   Error -- Could not update $GUEST_CFG\n";
+	    exit 1
+	fi
+    fi
+
+    $MOCK -r $GUEST_CFG --clean
+    $MOCK -r $GUEST_CFG --scrub=cache
+
+    rm -rf $GUEST_DIR
+    if [ $? -ne 0 ]; then	
+	printf "   Error -- Failed to remove guest $GUEST_DIR\n";
+	exit 1
+    fi
+
+    printf "  Done\n"
+}
+
+#############################################
+# Main code
+#############################################
+
+usage () {
+    echo ""
+    echo "Usage: "
+    echo "   build-guest [--rt | --std] [--verbose]"
+    echo "   build-guest [--help]"
+    echo "   build-guest [--clean]"
+    echo ""
+}
+
+# Default argument values
+HELP=0
+CLEAN=0
+VERBOSE=0
+BUILD_MODE='std'
+
+# read the options
+TEMP=`getopt -o h --long clean --long rt --long  std --long verbose -n 'test.sh' -- "$@"`
+eval set -- "$TEMP"
+
+# extract options and their arguments into variables.
+while true ; do
+    case "$1" in
+        -h|--help) HELP=1 ; shift ;;
+        --clean) CLEAN=1 ; shift ;;
+        --verbose) VERBOSE=1 ; shift ;;
+        --rt) BUILD_MODE='rt' ; shift ;;
+        --std) BUILD_MODE='std' ; shift ;;
+        --) shift ; break ;;
+        *) echo "Internal error!" ; exit 1 ;;
+    esac
+done
+
+if [ $HELP -eq 1 ]; then
+   usage
+   exit 0
+fi
+
+(
+printf "\n*****************************\n"
+printf   "Create Titanium Cloud/CentOS Guest Image\n"
+printf   "*****************************\n\n"
+
+init_vars
+check_vars
+
+if [ $CLEAN -eq 1 ]; then
+   clean_guest
+   exit 0
+fi
+
+create_rootfs
+update_rootfs
+build_image
+
+) 2>&1 | stdbuf -o0 awk '{ print strftime("%H:%M:%S"), $0; fflush(); }' ; exit ${PIPESTATUS[0]}

build-tools/build-img

@@ -0,0 +1,94 @@
+#!/bin/bash
+
+# Build an IMG file capable of being booted in a virtual environment
+# The default settings are vda device which the Cumulus environment expects
+# and controller mode
+
+usage () {
+    echo ""
+    echo "Usage: "
+    echo "   build-img [--cpe] [--dest <filename>] [--part [1 | 2]]"
+    echo "   --dest <tis.img>"
+    echo "   --cpe  Boots in CPE mode. Default is controller mode."
+    echo ""
+}
+
+DEST_ISO=bootimage_auto.iso
+DEST_IMG=tis.img
+AUTO_MODE=controller
+HELP=0
+PART=0
+
+# read the options
+TEMP=`getopt -o hp:d: --long help,cpe,part:,dest: -n 'test.sh' -- "$@"`
+eval set -- "$TEMP"
+
+# extract options and their arguments into variables.
+while true ; do
+    case "$1" in
+        -h|--help) HELP=1 ; shift ;;
+        --cpe) AUTO_MODE=cpe;  shift ;;
+        -d | --dest) DEST_IMG="$2"; shift; shift ;;
+        -p | --part) PART="$2"; shift; shift ;;
+        --) shift ; break ;;
+        *) echo "Internal error!" ; exit 1 ;;
+    esac
+done
+
+if [ $HELP -eq 1 ]; then
+   usage
+   exit 0
+fi
+
+echo PART=$PART
+
+# Step 1: Build an ISO that autoboots 
+
+# Cumulus default device is vda
+if [ $PART -ne 2 ]; then
+   build-iso --file bootimage_auto.iso --auto $AUTO_MODE --device vda --cumulus
+fi
+
+# Step 2: Convert the ISO to IMG
+if [ $PART -ne 1 ]; then
+   if [ ! -e "/dev/loop-control"  -o ! -e "/dev/kvm" ]; then
+      CMD="cd $MY_WORKSPACE/export; \
+           $MY_REPO/addons/wr-cgcs/layers/cgcs/extras.ND/scripts/install_iso_to_disk_image.sh bootimage_auto.iso $DEST_IMG"
+
+      if [ "$HOSTNAME" == "yow-cgts3-centos7" ]; then
+         echo "Attempting to run kvm_iso_to_img on yow-cgts3-lx"
+         ssh -o StrictHostKeyChecking=no yow-cgts3-lx "$CMD"
+         if [ $? -ne 0 ]; then
+            echo "Failed to run update-efiboot-image on yow-cgts3-lx"
+         fi
+      fi
+
+      if [ "$HOSTNAME" == "yow-cgts2-centos7" ]; then
+         echo "Attempting to run kvm_iso_to_img on yow-cgts2-lx"
+         ssh -o StrictHostKeyChecking=no yow-cgts2-lx "$CMD"
+         if [ $? -ne 0 ]; then
+            echo "Failed to run update-efiboot-image on yow-cgts2-lx"
+         fi
+      fi
+
+      if [ ! -f "$MY_WORKSPACE/export/$DEST_IMG" ]; then
+         printf "\n"
+         printf "****************************************************************** \n"
+         printf "No kvm and/or loop device on this machine.  To complete the build \n"
+         printf "please copy '$MY_WORKSPACE/export/bootimage_auto.iso' to a machine \n"
+         printf "that supports kvm and loop devices and run ... \n"
+         printf "   $MY_REPO/addons/wr-cgcs/layers/cgcs/extras.ND/scripts/install_iso_to_disk_image.sh bootimage_auto.iso $DEST_IMG\n"
+         printf "****************************************************************** \n"
+         exit 1
+      fi
+   fi
+
+   if [ ! -f "$MY_WORKSPACE/export/$DEST_IMG" ]; then
+      (
+      cd $MY_WORKSPACE/export
+      $MY_REPO/addons/wr-cgcs/layers/cgcs/extras.ND/scripts/install_iso_to_disk_image.sh bootimage_auto.iso $DEST_IMG
+      exit $?
+      )
+   fi
+fi
+

build-tools/build-iso

@@ -0,0 +1,702 @@
+#!/bin/bash
+
+# Build the export/bootimage.iso file
+#
+# This script uses environment variables to determine the source of
+# packages, and bundles the packages into a bootable .iso
+#
+# It starts by building a basic "vanilla CentOS" ISO, and then adds our
+# packages to it.
+
+usage () {
+    echo ""
+    echo "Usage: "
+    echo "   build-iso [--auto <mode>] [--file <filename>] [--device <device>] [--skip-sign]"
+    echo "   --file <bootimage.iso> destination ISO file"
+    echo "   --auto <controller|cpe> Modify kickstart to auto-install controller or cpe mode"
+    echo "   --device <sda> Use a different boot/rootds device (default is sda)"
+    echo "   --skip-sign do not add file signature to RPMs"
+    echo ""
+}
+
+MY_YUM_CONF=""
+
+NPROCS=$(nproc)
+
+export MOCK=/usr/bin/mock
+
+CREATEREPO=$(which createrepo_c)
+if [ $? -ne 0 ]; then
+   CREATEREPO="createrepo"
+fi
+
+# TEMPORARY: Check for isohybrid now to give a warning about installing pkg
+if [ ! -f /usr/bin/isohybrid ]; then
+    echo "Missing required utility: /usr/bin/isohybrid"
+    echo "Installation of syslinux is required:"
+    echo "    sudo yum install -y syslinux"
+    exit 1
+fi
+
+function install_pkg_list {
+   local PKGLIST=$1
+   if [ "x$PKGLIST" == "x" ]; then
+       return 1
+   fi
+
+   OLD_PWD=$PWD
+
+   echo "Installing packages listed in $PKGLIST and dependancies"
+   \rm -f $OUTPUT_DIR/dist/report_deps.txt
+   $CREATEREPO $CGCS_REPO_DIR
+   $CREATEREPO $CGCS_RT_REPO_DIR
+
+   \cp -v $MY_YUM_CONF $OUTPUT_DIR
+
+   \cd $OUTPUT_DIST_DIR/isolinux/Packages
+   $INTERNAL_REPO_ROOT/build-tools/build_iso/cgts_deps.sh --deps=$PKGLIST
+   
+   if [ $? -ne 0 ]
+   then
+      echo "Could not install dependencies"
+      exit 1
+   fi
+   
+   # clean up
+   echo "Removing local-std yum repo  $CGCS_REPO_DIR/repodata"
+   echo "Removing local-rt yum repo   $CGCS_RT_REPO_DIR/repodata"
+
+   \cd $OLD_PWD
+}
+
+# Generate the report of where all packages come from
+function make_report {
+   local PKGLISTFILES=$@
+   if [ "x$PKGLISTFILES" == "x" ]; then
+       return 1
+   fi
+   echo "MAKING $REPORT_FILE"
+   echo "-----------------" >> $REPORT_FILE
+
+   echo "ISO REPORT" > $REPORT_FILE
+   date >>  $REPORT_FILE
+   echo "-----------------" >> $REPORT_FILE
+
+   echo " " >> $REPORT_FILE
+   echo "-----------------" >> $REPORT_FILE
+   echo "EXPLICIT INCLUDES" >> $REPORT_FILE
+   echo "-----------------" >> $REPORT_FILE
+   for PKGLIST in $PKGLISTFILES; do
+      while read PKG; do
+         PKG=`echo $PKG | sed "s/#.*//"`;
+         if [ "${PKG}x" != "x" ]; then 
+            echo $PKG  >> $REPORT_FILE
+         fi
+      done < $PKGLIST
+   done
+
+   echo " " >> $REPORT_FILE
+   echo "-----------------" >> $REPORT_FILE
+   echo " PACKAGES        " >> $REPORT_FILE
+   echo "-----------------" >> $REPORT_FILE
+   cat $BUILT_REPORT | sort | uniq >> $REPORT_FILE
+
+   echo " " >> $REPORT_FILE
+   echo "-----------------" >> $REPORT_FILE
+   echo " WARNINGS        " >> $REPORT_FILE
+   echo "-----------------" >> $REPORT_FILE
+   
+   # Note that the warnings file may have multiple lines for the same
+   # missing dependency.  A sort | uniq solves this so we don't duplicate
+   # warnings
+   cat $WARNINGS_REPORT | sort | uniq >> $REPORT_FILE
+
+   echo "ISO REPORT: $REPORT_FILE"
+}
+
+function init_vars {
+   #####################################
+   # Input definitions
+
+   # Where all CentOS packages live
+   # Where essential CentOS (minimal install) packages live
+   INTERNAL_REPO_ROOT=
+   # Where our own packages live
+   CGCS_REPO_DIR=$MY_WORKSPACE/std/rpmbuild/RPMS
+   CGCS_RT_REPO_DIR=$MY_WORKSPACE/rt/rpmbuild/RPMS
+
+   MY_YUM_CONF=$(create-yum-conf)
+   if [ $? -ne 0 ]; then
+      echo "ERROR: create-yum-conf failed"
+      exit 1
+   fi
+
+   DISTRO_REPO_DIR=$(for d in $(grep baseurl $MY_YUM_CONF | grep file: | awk -F : '{print $2}' | sed 's:///:/:g'); do if [ -d $d/images ]; then echo $d ;fi; done)
+
+   #####################################
+   # Output definitons
+
+   # where to put stuff (curent dir unless MY_WORKSPACE defined)
+   OUTPUT_DIR="$PWD/export"   
+   if [ ! -z "$MY_WORKSPACE" ] && [ -d "$MY_WORKSPACE" ] ; then
+       OUTPUT_DIR="$MY_WORKSPACE/export"
+       CGCS_REPO_DIR="$MY_WORKSPACE/std/rpmbuild/RPMS"
+       CGCS_RT_REPO_DIR="$MY_WORKSPACE/rt/rpmbuild/RPMS"
+   fi
+
+   # Directory in which to populate files to be distributed
+   if [ $CUMULUS -eq 0 ]; then
+      OUTPUT_DIST_DIR=$OUTPUT_DIR/dist
+   else
+      OUTPUT_DIST_DIR=$OUTPUT_DIR/dist-cumulus
+   fi
+
+   # Package disc image
+   OUTPUT_FILE=$OUTPUT_DIR/$DEST_FILE
+
+   # Generate an error if the output file is below this threshold
+   MINIMUM_EXPECTED_SIZE=500000000
+
+
+   # report variables
+   REPORT_FILE=$OUTPUT_DIR/report.txt
+   BUILT_REPORT=$OUTPUT_DIR/local.txt
+   CLOUD_REPORT=$OUTPUT_DIR/cloud.txt
+   CLOUD_COMMON_REPORT=$OUTPUT_DIR/cloudcommon.txt
+   CENTOS_REPORT=$OUTPUT_DIR/centos.txt
+   EPEL_REPORT=$OUTPUT_DIR/epel.txt
+   WARNINGS_REPORT=$OUTPUT_DIR/warnings.txt
+
+   \rm -f $REPORT_FILE
+   \rm -f $BUILT_REPORT
+   \rm -f $CLOUD_REPORT
+   \rm -f $CLOUD_COMMON_REPORT
+   \rm -f $CENTOS_REPORT
+   \rm -f $WARNINGS_REPORT
+}
+
+# check input variables
+function check_vars {
+   # Where to store data
+   printf "Finding cgcs-root\n"
+   printf "  Checking \$MY_REPO (value \"$MY_REPO\")\n"
+
+   if [ ! -z "$MY_REPO" ] && [ -d "$MY_REPO" ] ; then
+      INTERNAL_REPO_ROOT=$MY_REPO
+      printf "  Found!\n"
+   fi
+
+   if [ -z "$INTERNAL_REPO_ROOT" ] ; then
+      printf "  No joy -- checking \$MY_REPO_ROOT_DIR (value \"$MY_REPO_ROOT_DIR\")\n"
+      if [ ! -z "$MY_REPO_ROOT_DIR" ] && [ -d "$MY_REPO_ROOT_DIR/cgcs-root" ] ; then
+          INTERNAL_REPO_ROOT=$MY_REPO_ROOT_DIR/cgcs-root
+          printf "  Found!\n"
+      fi
+   fi
+
+   if [ -z "$INTERNAL_REPO_ROOT" ] ; then
+      printf "  No joy -- checking for \$MY_WORKSPACE/cgcs-root\n"
+      if [ -d "$MY_WORKSPACE/cgcs-root" ] ; then
+          INTERNAL_REPO_ROOT=$MY_WORKSPACE/cgcs-root
+          printf "  Found!\n"
+      fi
+   fi
+
+   if [ -z "$INTERNAL_REPO_ROOT" ] ; then
+      printf "  Error -- could not locate cgcs-root repo.\n"
+      exit 1
+   fi
+
+   printf "\nChecking that we can access $DISTRO_REPO_DIR\n"
+   if [ ! -d "$DISTRO_REPO_DIR" ] ; then
+      printf "  Error -- could not access $DISTRO_REPO_DIR\n"
+      exit 1
+   fi
+
+   if [ ! -e "$DISTRO_REPO_DIR/repodata" ] ; then
+      printf "  Error -- $DISTRO_REPO_DIR is there, but does not seem sane\n"
+   fi
+
+   printf "\nOkay, input looks fine...\n\n"
+   printf "Creating output directory $OUTPUT_DIST_DIR\n"
+   if [ $CLEAN_FLAG -eq 1 ]; then
+      echo "  Cleaning..."
+      if [ -e $OUTPUT_DIST_DIR ] ; then
+         chmod -R a+w $OUTPUT_DIST_DIR
+         \rm -rf $OUTPUT_DIST_DIR
+      fi
+      if [ -e $OUTPUT_DIST_DIR ] ; then
+         printf "Error: could not remove old $OUTPUT_DIST_DIR\n"
+         exit 1
+      fi
+   fi
+
+   \mkdir -p $OUTPUT_DIST_DIR
+   if [ ! -d $OUTPUT_DIST_DIR ] ; then
+      printf "Error: could not create $OUTPUT_DIST_DIR\n"
+      exit 1
+   fi
+
+   RELEASE_INFO=$INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/middleware/recipes-common/build-info/release-info.inc
+   export PLATFORM_RELEASE=$(source $RELEASE_INFO && echo $PLATFORM_RELEASE)
+
+   echo "  Done"
+   echo ""
+}
+
+function init_output_dir {
+   echo "Creating base output directory in $OUTPUT_DIST_DIR"
+   \mkdir -p $OUTPUT_DIST_DIR/isolinux/images
+   \mkdir -p $OUTPUT_DIST_DIR/isolinux/ks
+   \mkdir -p $OUTPUT_DIST_DIR/isolinux/LiveOS
+   \mkdir -p $OUTPUT_DIST_DIR/isolinux/Packages
+   \mkdir -p $OUTPUT_DIST_DIR/utils
+
+   \mkdir -p $OUTPUT_DIST_DIR/isolinux/EFI
+   # This directory will contains files required for the PXE network installer
+   \mkdir -p $OUTPUT_DIST_DIR/isolinux/pxeboot
+
+   echo "  Copying base files"
+
+   # Generate .discinfo file
+   date +%s.%N > $OUTPUT_DIST_DIR/isolinux/.discinfo
+   echo $PLATFORM_RELEASE >> $OUTPUT_DIST_DIR/isolinux/.discinfo
+   echo "x86_64" >> $OUTPUT_DIST_DIR/isolinux/.discinfo
+   
+   \cp -L -ru $DISTRO_REPO_DIR/isolinux/* $OUTPUT_DIST_DIR/isolinux/
+   \cp -L -ru $DISTRO_REPO_DIR/images/pxeboot $OUTPUT_DIST_DIR/isolinux/images/
+
+   echo "  Installing startup files"
+
+   \cp -L $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/centos.syslinux.cfg $OUTPUT_DIST_DIR/isolinux/syslinux.cfg
+   \cp -L $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/centos.syslinux.cfg $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
+   sed -i 's/wr_usb_boot/oe_iso_boot/' $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
+
+   # Modify the isolinux.cfg to auto install if requested
+   # Option 0 is Controller(serial). Option 2 is CPE serial.
+   if [ "$AUTO_INSTALL" == "controller" ] ; then
+       echo "Modifying ISO to auto-install controller load"
+       perl -p -i -e 's/timeout 0/timeout 1\ndefault 0/'  $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
+   elif [ "$AUTO_INSTALL" == "cpe" ] ; then
+       echo "Modifying ISO to auto-install CPE (combined load)"
+       perl -p -i -e 's/timeout 0/timeout 1\ndefault 2/'  $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
+   fi
+
+   # Modify the device if requested
+   if [ ! -z "$DEVICE" ] ; then
+       echo "Modifying ISO to use device $DEVICE"
+       perl -p -i -e "s/device=sda/device=${DEVICE}/g"  $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
+   fi
+
+
+
+   # Copy UEFI files 
+   \cp -L -ru $DISTRO_REPO_DIR/EFI/* $OUTPUT_DIST_DIR/isolinux/EFI/
+   \cp -L $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/grub.cfg $OUTPUT_DIST_DIR/isolinux/EFI/BOOT/grub.cfg
+   \cp -L $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/pxeboot_grub.cfg $OUTPUT_DIST_DIR/isolinux/pxeboot/pxeboot_grub.cfg
+
+   # Update the efiboot.img (See https://wiki.archlinux.org/index.php/Remastering_the_Install_ISO)
+   # We need to mount the image file, replace the grub.cfg file with the Titanium Cloud one, and unmount.
+   # Script update-efiboot-image will do this. If there is not loop device on the build machine
+   # then this script must be executed manually prior.
+
+   if [ ! -e "/dev/loop-control"  -a ! -f "$OUTPUT_DIR/efiboot.img" ]; then
+      CMD="export PROJECT=$PROJECT; \
+           export SRC_BUILD_ENVIRONMENT=$SRC_BUILD_ENVIRONMENT; \
+           export MY_BUILD_ENVIRONMENT=$MY_BUILD_ENVIRONMENT; \
+           export MY_BUILD_ENVIRONMENT_FILE=$MY_BUILD_ENVIRONMENT_FILE; \
+           export MY_BUILD_DIR=$MY_BUILD_DIR; \
+           export MY_WORKSPACE=$MY_WORKSPACE; \
+           export MY_REPO=$MY_REPO; \
+           export MY_BUILD_CFG=$MY_BUILD_CFG; \
+           export MY_MOCK_ROOT=$MY_MOCK_ROOT; \
+           export PATH=$MY_REPO/build-tools:\$PATH; \
+           update-efiboot-image"
+
+      if [ "$HOSTNAME" == "yow-cgts3-centos7" ]; then
+         echo "Attempting to run update-efiboot-image on yow-cgts3-lx"
+         ssh -o StrictHostKeyChecking=no yow-cgts3-lx "$CMD"
+         if [ $? -ne 0 ]; then
+            echo "Failed to run update-efiboot-image on yow-cgts3-lx"
+         fi
+      fi
+
+      if [ "$HOSTNAME" == "yow-cgts2-centos7" ]; then
+         echo "Attempting to run update-efiboot-image on yow-cgts2-lx"
+         ssh -o StrictHostKeyChecking=no yow-cgts2-lx "$CMD"
+         if [ $? -ne 0 ]; then
+            echo "Failed to run update-efiboot-image on yow-cgts2-lx"
+         fi
+      fi
+   fi
+
+   if [ ! -e "/dev/loop-control"  -a ! -f "$OUTPUT_DIR/efiboot.img" ]; then
+      printf "\n**************************************************************************************************** \n"
+      printf "No loop device on this machine. Please ensure $OUTPUT_DIR/efiboot.img \n"
+      printf "exist prior to executing build-iso by.  It can be created by running \n"
+      printf "   $INTERNAL_REPO_ROOT/build-tools/update-efiboot-image \n"
+      printf "on a machine that does support a loop device.  Please ensure all standard \n"
+      printf "build environment variables are defined (e.g. MY_REPO, MY_WORKSPACE, etc.). \n"
+      printf " \n"
+      printf "e.g. If building on yow-cgts3-centos7, you'll want to run the script on \n"
+      printf "     yow-cgts3-lx which shares the same file system, but supports loop devices \n"
+      printf "****************************************************************************************************** \n"
+      exit 1
+   fi
+
+   if [ -f "$OUTPUT_DIR/efiboot.img" ]; then
+
+      # The script update-efiboot-image was run outside the build-iso script, do nothing.
+      printf "  The image file $OUTPUT_DIR/efiboot.img already exist\n"
+   else
+      printf "  The image file $OUTPUT_DIR/efiboot.img does not exist \n"
+      if [ ! -f "$INTERNAL_REPO_ROOT/build-tools/update-efiboot-image" ]; then
+          printf "*** Error: script update-efiboot-image does not exist *** \n"
+          exit 1
+      fi
+
+      # Run the script
+      $INTERNAL_REPO_ROOT/build-tools/update-efiboot-image
+      RET=$?
+      if [ $RET != 0 ]; then
+          printf "*** Error: update-efiboot-image script returned failure $RET *** \n"
+          exit 1
+      fi
+
+   fi
+
+   \cp -L $OUTPUT_DIR/efiboot.img $OUTPUT_DIST_DIR/isolinux/images/
+   \rm -f $OUTPUT_DIR/efiboot.img
+
+   # Copy and set up pxeboot setup files
+   \cp $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/pxeboot_setup.sh $OUTPUT_DIST_DIR/isolinux/pxeboot_setup.sh
+   \cp $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/pxeboot.cfg $OUTPUT_DIST_DIR/isolinux/pxeboot/pxeboot.cfg
+   chmod +x $OUTPUT_DIST_DIR/isolinux/pxeboot_setup.sh
+
+   \rm -f $OUTPUT_DIST_DIR/comps.xml
+   \cp -L $INTERNAL_REPO_ROOT/build-tools/build_iso/comps.xml.gz $OUTPUT_DIST_DIR/
+   gunzip $OUTPUT_DIST_DIR/comps.xml.gz
+
+   TMP_DIR=$MY_WORKSPACE/tmp
+   \mkdir -p $TMP_DIR
+   TMPDIR=$TMP_DIR yum clean all -c $MY_YUM_CONF
+   \rm -rf $TMP_DIR/yum-$USER-*
+   echo "  Done"
+   echo ""
+}
+
+function final_touches {
+   OLD_PWD=$PWD
+
+   # Update the comps.xml
+   if [ ! -f $OUTPUT_DIST_DIR/comps.xml.bak ]; then
+      \cp $OUTPUT_DIST_DIR/comps.xml $OUTPUT_DIST_DIR/comps.xml.bak
+   fi
+
+   python $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/platform_comps.py \
+      --pkgdir $OUTPUT_DIST_DIR/isolinux/Packages \
+      --groups $OUTPUT_DIST_DIR/comps.xml
+   if [ $? -ne 0 ]; then
+      echo "Failed to update comps.xml"
+      exit 1
+   fi
+
+   # create the repo
+   \cd $OUTPUT_DIST_DIR/isolinux
+   $CREATEREPO -q -g ../comps.xml .
+   
+   # build the ISO
+   printf "Building image $OUTPUT_FILE\n"
+   \cd $OUTPUT_DIST_DIR
+   chmod 664 isolinux/isolinux.bin
+   mkisofs -o $OUTPUT_FILE \
+      -R -D -A 'oe_iso_boot' -V 'oe_iso_boot' \
+      -quiet \
+      -b isolinux.bin -c boot.cat -no-emul-boot \
+      -boot-load-size 4 -boot-info-table \
+      -eltorito-alt-boot \
+      -e images/efiboot.img \
+            -no-emul-boot \
+      isolinux/   
+
+   isohybrid --uefi $OUTPUT_FILE
+   implantisomd5 $OUTPUT_FILE
+
+   \cd $OLD_PWD
+}
+
+function extract_pkg_from_local_repo {
+   local repodir=$1
+   local pkgname=$2
+
+   local pkgfile=$(repoquery --repofrompath local,${repodir} --location -q ${pkgname})
+   if [ -z "${pkgfile}" ]; then
+      echo "Could not find package $pkgname in $repodir"
+      exit 1
+   fi
+
+   rpm2cpio ${pkgfile/file://} | cpio -idmv
+   if [ $? -ne 0 ]; then
+      echo "Failed to extract files from ${pkgfile/file://}"
+      exit 1
+   fi
+}
+
+function extract_installer_files {
+   # Changes to copied files here must also be reflected in patch-iso
+
+   CGCSDIR=$INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs
+   PKGDIR=$OUTPUT_DIST_DIR/isolinux/Packages
+
+   (
+   \cd $OUTPUT_DIR
+   \rm -rf kickstarts extra_cfgs kickstart.work
+   \mkdir kickstarts extra_cfgs kickstart.work
+
+   echo "Retrieving kickstarts..."
+
+   \cd kickstart.work
+
+   extract_pkg_from_local_repo ${CGCS_REPO_DIR} platform-kickstarts
+   extract_pkg_from_local_repo ${CGCS_REPO_DIR} platform-kickstarts-pxeboot
+   extract_pkg_from_local_repo ${CGCS_REPO_DIR} platform-kickstarts-extracfgs
+
+   \cp --preserve=all www/pages/feed/rel-*/*.cfg pxeboot/*.cfg ../kickstarts/ &&
+   \cp --preserve=all extra_cfgs/*.cfg ../extra_cfgs/
+   if [ $? -ne 0 ]; then
+      echo "Failed to copy extracted kickstarts"
+      exit 1
+   fi
+
+   \cd ..
+
+   # Copy kickstarts to ISO
+   \cp --preserve=all kickstarts/controller_ks.cfg $OUTPUT_DIST_DIR/isolinux/ks.cfg
+   # Modify the kickstart to shutdown instead of reboot if doing an auto install
+   if [ ! -z "$AUTO_INSTALL" ] ; then
+      sed -i 's/^reboot --eject/shutdown/' $OUTPUT_DIST_DIR/isolinux/ks.cfg
+   fi
+
+   \mv kickstarts/pxeboot* $OUTPUT_DIST_DIR/isolinux/pxeboot/
+   \cp --preserve=all kickstarts/* $OUTPUT_DIST_DIR/isolinux
+
+   # Update OAM interface for cumulus auto install
+   if [ $CUMULUS -eq 1 ]; then
+       # Cumulus wants tty1
+       perl -p -i -e 's/console=tty0/console=tty1/'  $OUTPUT_DIST_DIR/isolinux/isolinux.cfg
+
+       # CUMULUS setup scripts specify ens3 for OAM
+       OAM_IFNAME=ens3
+
+       cat <<EOM >> $OUTPUT_DIST_DIR/isolinux/ks.cfg
+%post
+#For cumulus tis on tis automated install
+cat << EOF > /etc/sysconfig/network-scripts/ifcfg-${OAM_IFNAME}
+IPADDR=10.10.10.3
+NETMASK=255.255.255.0
+BOOTPROTO=static
+ONBOOT=yes
+DEVICE=${OAM_IFNAME}
+MTU=1500
+GATEWAY=10.10.10.1
+EOF
+%end
+EOM
+   fi
+
+   # For PXE boot network installer
+
+   echo ${OUTPUT_DIST_DIR}/isolinux/Packages
+
+   local WORKDIR=pxe-network-installer.content
+   local ORIG_PWD=$PWD
+
+   \rm -rf $WORKDIR
+   \mkdir $WORKDIR
+   \cd $WORKDIR
+
+   extract_pkg_from_local_repo ${CGCS_REPO_DIR} pxe-network-installer
+   extract_pkg_from_local_repo ${CGCS_REPO_DIR} grub2-efi-pxeboot
+
+   \mkdir -p $OUTPUT_DIST_DIR/isolinux/pxeboot/EFI/centos/x86_64-efi
+
+   \cp --preserve=all pxeboot/pxelinux.0 pxeboot/menu.c32 pxeboot/chain.c32 $OUTPUT_DIST_DIR/isolinux/pxeboot &&
+   \cp --preserve=all pxeboot/EFI/centos/x86_64-efi/* $OUTPUT_DIST_DIR/isolinux/pxeboot/EFI/centos/x86_64-efi/ &&
+   \cp --preserve=all pxeboot/EFI/grubx64.efi $OUTPUT_DIST_DIR/isolinux/pxeboot/EFI/
+   if [ $? -ne 0 ]; then
+      echo "Error: Could not copy all files from installer"
+      exit 1
+   fi
+   
+   \cp --preserve=all www/pages/feed/rel-*/LiveOS/squashfs.img $OUTPUT_DIST_DIR/isolinux/LiveOS
+   if [ $? -ne 0 ]; then
+      echo "Error: Could not copy squashfs from LiveOS"
+      exit 1
+   fi
+
+
+   # Replace vmlinuz and initrd.img with our own pre-built ones
+   \rm -f \
+      $OUTPUT_DIST_DIR/isolinux/vmlinuz \
+      $OUTPUT_DIST_DIR/isolinux/images/pxeboot/vmlinuz \
+      $OUTPUT_DIST_DIR/isolinux/initrd.img \
+      $OUTPUT_DIST_DIR/isolinux/images/pxeboot/initrd.img
+   \cp --preserve=all pxeboot/rel-*/installer-bzImage_1.0 \
+      $OUTPUT_DIST_DIR/isolinux/vmlinuz &&
+   \cp --preserve=all pxeboot/rel-*/installer-bzImage_1.0 \
+      $OUTPUT_DIST_DIR/isolinux/images/pxeboot/vmlinuz &&
+   \cp --preserve=all pxeboot/rel-*/installer-intel-x86-64-initrd_1.0 \
+      $OUTPUT_DIST_DIR/isolinux/initrd.img &&
+   \cp --preserve=all pxeboot/rel-*/installer-intel-x86-64-initrd_1.0 \
+      $OUTPUT_DIST_DIR/isolinux/images/pxeboot/initrd.img
+
+   if [ $? -ne 0 ]; then
+      echo "Error: Failed to copy installer images"
+      exit 1
+   fi
+
+   \cd $ORIG_PWD
+   \rm -rf $WORKDIR
+   )
+   if [ $? -ne 0 ]; then
+      exit 1
+   fi
+}
+
+function setup_upgrades_files {
+   # Changes to copied files here must also be reflected in patch-iso
+
+   # Copy the upgrade files
+   UPGRADES_DIR="$OUTPUT_DIST_DIR/isolinux/upgrades"
+   \rm -rf $UPGRADES_DIR
+   \mkdir -p $UPGRADES_DIR
+   \cp $INTERNAL_REPO_ROOT/addons/wr-cgcs/layers/cgcs/mwa-beas/bsp-files/upgrades/*  $UPGRADES_DIR
+   sed -i "s/xxxSW_VERSIONxxx/${PLATFORM_RELEASE}/g" $UPGRADES_DIR/metadata.xml
+   chmod +x $UPGRADES_DIR/*.sh
+   # Write the version out (used in upgrade scripts - this is the same as SW_VERSION)
+   echo "VERSION=$PLATFORM_RELEASE" > $UPGRADES_DIR/version
+}
+
+function sign_iso {
+    # Sign the .iso with the developer private key
+    # Sigining with the formal key is only to be done for customer release
+    # builds
+    local isofilename=$(basename $OUTPUT_DIR/$DEST_FILE)
+    local isofilenoext="${isofilename%.*}"
+    openssl dgst -sha256 -sign ${MY_REPO}/build-tools/signing/dev-private-key.pem -binary -out $OUTPUT_DIR/$isofilenoext.sig $OUTPUT_DIR/$DEST_FILE
+}
+
+#############################################
+# Main code
+#############################################
+
+# Check args
+HELP=0
+CLEAN_FLAG=1 # TODO -- doesn't yet work without --clean
+DEST_FILE=bootimage.iso
+AUTO_FLAG=0
+AUTO_INSTALL=""
+CUMULUS=0
+SIGN_RPM_FILES=1
+DEVICE=""
+
+# read the options
+TEMP=`getopt -o hf:a:d: --long help,file:,auto:,device:,cumulus,clean,skip-sign -n 'test.sh' -- "$@"`
+eval set -- "$TEMP"
+
+# extract options and their arguments into variables.
+while true ; do
+    case "$1" in
+        -h|--help) HELP=1 ; shift ;;
+        --clean) CLEAN_FLAG=1 ; shift ;;
+        --skip-sign) SIGN_RPM_FILES=0 ; shift ;;
+        --cumulus) CUMULUS=1 ; shift ;;
+        -f | --file) DEST_FILE="$2"; shift; shift ;;
+        -d | --device) DEVICE="$2"; shift; shift ;;
+        -a | --auto) AUTO_FLAG=1; AUTO_INSTALL="$2"; shift; shift ;;
+        --) shift ; break ;;
+        *) echo "Internal error!" ; exit 1 ;;
+    esac
+done
+
+if [ $AUTO_FLAG -eq 1 ]; then
+    if [[ "$AUTO_INSTALL" != "controller" && "$AUTO_INSTALL" != "cpe" ]] ; then
+            echo "Unsupported --auto value: $AUTO_INSTALL"
+            exit 1
+    fi
+fi
+
+if [ $HELP -eq 1 ]; then
+   usage
+   exit 0
+fi
+
+
+(
+printf "\n*************************\n"
+printf   "Create Titanium Cloud/CentOS Boot CD\n"
+printf   "*************************\n\n"
+
+# Init variables
+init_vars
+check_vars
+PKGLIST_MINIMAL=$INTERNAL_REPO_ROOT/build-tools/build_iso/minimal_rpm_list.txt
+PKGLIST_TI=$INTERNAL_REPO_ROOT/build-tools/build_iso/image.inc
+PKGLIST_DEV=$INTERNAL_REPO_ROOT/build-tools/build_iso/image-dev.inc
+
+# Create skeleton build dir
+init_output_dir
+
+# Create the vanilla DVD
+echo "Copying vanilla CentOS RPMs"
+install_pkg_list $PKGLIST_MINIMAL
+
+# Find all CGCS packages
+# SAL exit 0
+echo "Installing Titanium Cloud packages"
+install_pkg_list $PKGLIST_TI
+if [ $? -eq 2 ]; then
+	exit 1
+fi
+if [ "x${RELEASE_BUILD}" == "x" ]; then
+    echo "Installing Titanium Cloud developer packages"
+    install_pkg_list ${PKGLIST_DEV}
+    if [ $? -eq 2 ]; then
+        exit 1
+    fi
+fi
+
+# Extract installer files
+extract_installer_files
+
+# Upgrades files
+setup_upgrades_files
+
+# add file signatures to all rpms
+if [ $SIGN_RPM_FILES -ne 0 ]; then
+    sign-rpms -d $OUTPUT_DIST_DIR/isolinux/Packages
+    if [ $? -ne 0 ] ; then
+	echo "failed to add file signatures to RPMs"
+	exit 1
+    fi
+fi
+
+# Finalize and build ISO
+final_touches
+
+# Sign the ISO
+sign_iso
+
+make_report $PKGLIST_MINIMAL $PKGLIST_TI
+
+# Check sanity
+FILESIZE=$(wc -c <"$OUTPUT_FILE")
+if [ $FILESIZE -ge $MINIMUM_EXPECTED_SIZE ]; then
+	printf "Done."
+        printf "Output file: $OUTPUT_FILE\n\n"
+else
+	printf "Output file $OUTPUT_FILE smaller than expected -- probable error\n\n"
+	exit 1
+fi
+
+) 2>&1 | stdbuf -o0 awk '{ print strftime("%H:%M:%S"), $0; fflush(); }' ; exit ${PIPESTATUS[0]}

build-tools/build-pkg-srpm

@@ -0,0 +1,63 @@
+#!/bin/bash
+
+# Available environment
+# SRC_BASE = absolute path to cgcs-root 
+# AVS_BASE = absolute path to AVS source
+# CGCS_BASE = absolute path to CGCS source
+# RPM_BUILD_BASE = Directory where the package .distro  directory can be found
+# SRPM_OUT = Directory into which SRC RPMS are copied in preparation for mock build
+# RPM_DIR = Directory into which binary RPMs are delivered by mock
+
+SRC_DIR="/sources"
+VERSION=$(grep '^Version:' PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
+TAR_NAME=$(grep '^Name:' PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
+CUR_DIR=`pwd`
+BUILD_DIR=".distro/centos7/rpmbuild"
+
+mkdir -p $BUILD_DIR/SRPMS
+
+TAR="$TAR_NAME-$VERSION.tar.gz"
+TAR_PATH="$BUILD_DIR/SOURCES/$TAR"
+
+TAR_NEEDED=0
+if [ -f $TAR_PATH ]; then
+    n=`find . -cnewer $TAR_PATH -and !  -path './.git*' \
+                                -and ! -path './build/*' \
+                                -and ! -path './.pc/*' \
+                                -and ! -path './patches/*' \
+                                -and ! -path './.distro/*' \
+                                -and ! -path './pbr-*.egg/*' \
+                                | wc -l`
+    if [ $n -gt 0 ]; then
+        TAR_NEEDED=1
+    fi
+else
+    TAR_NEEDED=1
+fi
+
+if [ $TAR_NEEDED -gt 0 ]; then
+    tar czvf  $TAR_PATH .$SRC_DIR --exclude '.git*' --exclude 'build' --exclude='.pc' --exclude='patches' --exclude='.distro' --exclude='pbr-*.egg' --transform "s,^\.$SRC_DIR,$TAR_NAME-$VERSION,"
+fi
+
+for SPEC in `ls $BUILD_DIR/SPECS`; do
+    SPEC_PATH="$BUILD_DIR/SPECS/$SPEC"
+    RELEASE=$(grep '^Release:' $SPEC_PATH | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
+    NAME=`echo $SPEC | sed 's/.spec$//'`
+    SRPM="$NAME-$VERSION-$RELEASE.src.rpm"
+    SRPM_PATH="$BUILD_DIR/SRPMS/$SRPM"
+
+    BUILD_NEEDED=0
+    if [ -f $SRPM_PATH ]; then
+        n=`find . -cnewer $SRPM_PATH | wc -l`
+        if [ $n -gt 0 ]; then
+            BUILD_NEEDED=1
+        fi
+    else
+        BUILD_NEEDED=1
+    fi
+
+    if [ $BUILD_NEEDED -gt 0 ]; then
+        rpmbuild -bs $SPEC_PATH --define="%_topdir $CUR_DIR/$BUILD_DIR" --define="_tis_dist .tis"
+    fi
+done
+

build-tools/build-pkgs

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# This program is a wrapper around build-pkgs-parallel and build-pkgs-serial
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+usage () {
+    echo ""
+    echo "Usage: "
+    echo "   Create source and binary rpms:"
+    echo "   build-pkgs [--serial] [args]"
+}
+
+SERIAL_FLAG=0
+
+for arg in "$@"; do
+    case "$1" in
+        --serial) SERIAL_FLAG=1 ;;
+    esac
+done
+
+which mock_tmpfs_umount >> /dev/null
+if [ $? -ne 0 ]; then
+    SERIAL_FLAG=1
+fi
+
+if [ $SERIAL_FLAG -eq 1 ]; then
+    echo "build-pkgs-serial $@"
+    build-pkgs-serial "$@"
+else
+    echo "build-pkgs-parallel $@"
+    build-pkgs-parallel "$@"
+fi
+

build-tools/build-pkgs-parallel

@@ -0,0 +1,370 @@
+#!/bin/bash
+
+# This program is a wrapper around build-srpms-parallel and build-rpms-parallel
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+usage () {
+    echo ""
+    echo "Usage: "
+    echo "   Create source and Binary rpms:"
+    echo "   Build optimizations (--no-descendants, --no-required, --no-build-info,"
+    echo "   --no-autoclean) are not recommended for the first build after a clone/pull,"
+    echo "   nor the final build prior to creating an iso or patch, but can be used"
+    echo "   for intermediate builds.  i.e. while debugging compilation failures." 
+    echo "      build-pkgs-parallel [--no-descendants] [--no-required] [--no-build-info] [--no-autoclean] [--careful] [--formal] [ list of package names ]"
+    echo ""
+    echo "   Delete source rpms, and the directories associated with it's creation:"
+    echo "   Note: does not clean an edit environment"
+    echo "      build-pkgs-parallel --clean [ list of package names ]"
+    echo ""
+    echo "   Extract an src.rpm into a pair of git trees to aid in editing it's contents,"
+    echo "   one for source code and one for metadata such as the spec file."
+    echo "   If --no-meta-patch is specified, then WRS patches are omitted."
+    echo "      build-pkgs-parallel --edit [--no-meta-patch] [ list of package names ]"
+    echo ""
+    echo "   Delete an edit environment"
+    echo "      build-pkgs-parallel --edit --clean [ list of package names ]"
+    echo ""
+    echo "   This help page"
+    echo "      build-pkgs-parallel [--help]"
+    echo ""
+}
+
+
+HELP=0
+CLEAN_FLAG=0
+EDIT_FLAG=0
+STD_BUILD=1
+RT_BUILD=1
+INSTALLER_BUILD=0
+
+# read the options
+TEMP=$(getopt -o h --long rt,std,installer,edit,no-meta-patch,no-descendants,no-required,no-build-info,no-autoclean,formal,careful,help,clean -n 'build-pkgs-parallel' -- "$@")
+if [ $? -ne 0 ]; then
+    usage
+    exit 0
+fi
+eval set -- "$TEMP"
+
+# extract options and their arguments into variables.
+EXTRA_ARGS_COMMON=""
+EXTRA_ARGS_SRPM=""
+EXTRA_ARGS_RPM=""
+while true ; do
+    case "$1" in
+        --no-descendants) EXTRA_ARGS_COMMON+=" --no-descendants" ; shift ;;
+        --formal)         EXTRA_ARGS_COMMON+=" --formal" ; shift ;;
+        --careful)        EXTRA_ARGS_RPM+=" --careful" ; shift ;;
+        --no-required)    EXTRA_ARGS_RPM+=" --no-required" ; shift ;;
+        --no-build-info)  EXTRA_ARGS_COMMON+=" --no-build-info" ; shift ;;
+        --no-autoclean)   EXTRA_ARGS_RPM+=" --no-autoclean" ; shift ;;
+        --no-meta-patch)  EXTRA_ARGS_SRPM+=" --no-meta-patch" ; shift ;;
+        -h|--help)        HELP=1 ; shift ;;
+        --clean)          CLEAN_FLAG=1 ; shift ;;
+        --edit)           EDIT_FLAG=1 ; EXTRA_ARGS_SRPM+=" --edit"; shift ;;
+        --rt)             STD_BUILD=0 ; shift ;;
+        --std)            RT_BUILD=0 ; shift ;;
+        --installer)      INSTALLER_BUILD=1 ; STD_BUILD=0 ; RT_BUILD=0 ; shift ;;
+        --)               shift ; break ;;
+        *)                usage; exit 1 ;;
+    esac
+done
+
+if [ $HELP -eq 1 ]; then
+    usage
+    exit 0
+fi
+
+function my_exit() {
+    build-rpms-parallel --std --tmpfs-clean
+    build-rpms-parallel --rt --tmpfs-clean
+}
+
+function my_sigint() {
+    echo "build-pkgs-parallel sigint"
+    pkill -SIGABRT -P $BASHPID &> /dev/null
+    echo "build-pkgs-parallel waiting"
+    wait
+    echo "build-pkgs-parallel wait complete"
+
+}
+
+function my_sighup() {
+    echo "build-pkgs-parallel sighup"
+    pkill -SIGABRT -P $BASHPID &> /dev/null
+    echo "build-pkgs-parallel waiting"
+    wait
+    echo "build-pkgs-parallel wait complete"
+}
+
+function my_sigabrt() {
+    echo "build-pkgs-parallel sigabrt"
+    pkill -SIGABRT -P $BASHPID &> /dev/null
+    echo "build-pkgs-parallel waiting"
+    wait
+    echo "build-pkgs-parallel wait complete"
+}
+
+function my_sigterm() {
+    echo "build-pkgs-parallel sigterm"
+    pkill -SIGABRT -P $BASHPID &> /dev/null
+    echo "build-pkgs-parallel waiting"
+    wait
+    echo "build-pkgs-parallel wait complete"
+}
+
+trap my_exit EXIT
+trap my_sigint INT
+trap my_sighup HUP
+trap my_sigabrt ABRT
+trap my_sigterm TERM
+
+TARGETS=" $@ "
+TARGETS_STD=" "
+TARGETS_RT=" "
+TARGETS_INSTALLER=" "
+TARGETS_MISC=" "
+
+find_targets () {
+   local centos_pkg_dirs=$1
+   local d=""
+   local d2=""
+   local g=""
+   local x=""
+   local name=""
+   local path=""
+   local RESULT=" "
+   local FOUND=0
+
+   for g in $(find $MY_REPO -type d -name .git); do
+      d=$(dirname $g)
+      if [ -f $d/$centos_pkg_dirs ]; then
+         for d2 in $(grep -v '^#' $d/$centos_pkg_dirs); do
+            name=""
+            if [ -f $d/$d2/centos/srpm_path ]; then
+                path=$(cat $d/$d2/centos/srpm_path | head -n 1 | sed "s#^mirror:CentOS/tis-r3-CentOS/mitaka#$MY_REPO/cgcs-centos-repo#" | sed "s#^mirror:#$MY_REPO/cgcs-centos-repo/#" | sed "s#^repo:#$MY_REPO/#" | sed "s#^3rd_party:#$MY_REPO/cgcs-3rd-party-repo/#" | sed "s#^Source/#$MY_REPO/cgcs-centos-repo/Source/#")
+                name=$(rpm -q --qf='%{NAME}' --nosignature -p $path)
+            else
+                path=$(find $d/$d2/centos/ -name '*.spec' | head -n 1)
+                if [[ ( -z "$path" ) &&  ( -f $d/$d2/centos/spec_path ) ]]; then
+                    path=$(find $MY_REPO/$(cat $d/$d2/centos/spec_path) -maxdepth 1 -name '*.spec' | head -n 1)
+                fi
+                if [ "$path" != "" ]; then
+                   name=$(spec_find_tag Name "$path" 2>> /dev/null)
+                fi
+            fi
+            if [ "$name" != "" ]; then
+               if [ "$BUILD_TYPE" == "rt" ]; then
+                  FOUND=0
+                  for x in $TARGETS; do
+                     if [ "${x: -3}" == "-rt" ]; then
+                        if [ "${name}" == "$x" ] || [ "${name}-rt" == "${x}" ]; then
+                           RESULT+="$x "
+                           FOUND=1
+                           break
+                        fi
+                     fi
+                  done
+                  if [ $FOUND -eq 0 ]; then
+                     for x in $TARGETS; do
+                        if [ "${name}" == "${x}-rt" ]; then
+                           RESULT+="$x-rt "
+                           FOUND=1
+                           break
+                        else
+                           if [ "${name}" == "$x" ] || [ "${name}-rt" == "${x}" ]; then
+                              RESULT+="$x "
+                              FOUND=1
+                              break
+                           fi
+                        fi
+                     done
+                  fi
+               else
+                  for x in $TARGETS; do
+                     if [ "${name}" == "$x" ]; then
+                         RESULT+="$x "
+                         FOUND=1
+                         break
+                     fi
+                  done
+               fi
+            fi
+         done
+      fi
+   done
+
+   echo "$RESULT"
+   return 0
+}
+
+echo "CLEAN_FLAG=$CLEAN_FLAG"
+echo "EDIT_FLAG=$EDIT_FLAG"
+
+if [ "x$TARGETS" != "x  " ]; then
+   source $MY_REPO/build-tools/spec-utils
+   TARGETS_STD="$(find_targets centos_pkg_dirs)"
+
+   BUILD_TYPE_SAVE="$BUILD_TYPE"
+   BUILD_TYPE="rt"
+   TARGETS_RT="$(find_targets centos_pkg_dirs_rt)"
+   BUILD_TYPE="$BUILD_TYPE_SAVE"
+
+   echo "TARGETS_STD=$TARGETS_STD"
+   echo "TARGETS_RT=$TARGETS_RT"
+
+   for x in $TARGETS; do
+       if [[ $TARGETS_STD == *" $x "* ]]
+       then
+           echo "found $x" >> /dev/null;
+       else
+           if [[ $TARGETS_RT == *" $x "* ]]
+           then
+               echo "found $x" >> /dev/null;
+           else
+               TARGETS_MISC+="$x "
+           fi
+       fi
+   done
+fi
+
+echo "EXTRA_ARGS_COMMON='$EXTRA_ARGS_COMMON'"
+echo "EXTRA_ARGS_SRPM='$EXTRA_ARGS_SRPM'"
+echo "EXTRA_ARGS_RPM='$EXTRA_ARGS_RPM'"
+echo "TARGETS='$TARGETS'"
+echo "TARGETS_STD='$TARGETS_STD'"
+echo "TARGETS_RT='$TARGETS_RT'"
+echo "TARGETS_MISC='$TARGETS_MISC'"
+
+if [ $CLEAN_FLAG -eq 1 ]; then
+   if [ $STD_BUILD -eq 1 ]; then
+      if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+         if [ $EDIT_FLAG -ne 1 ]; then
+            echo "$DIR/build-rpms-parallel --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_STD $TARGETS_MISC"
+            $DIR/build-rpms-parallel --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_STD $TARGETS_MISC || exit 1
+         fi
+      fi
+      if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+         echo "$DIR/build-srpms-parallel --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_STD $TARGETS_MISC"
+         $DIR/build-srpms-parallel --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_STD $TARGETS_MISC || exit 1
+      fi
+   fi
+   if [ $RT_BUILD -eq 1 ]; then
+      if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+         if [ $EDIT_FLAG -ne 1 ]; then
+            echo "$DIR/build-rpms-parallel --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_RT $TARGETS_MISC"
+            $DIR/build-rpms-parallel --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_RT $TARGETS_MISC || exit 1
+         fi
+      fi
+      if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+         echo "$DIR/build-srpms-parallel --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_RT $TARGETS_MISC"
+         $DIR/build-srpms-parallel --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_RT $TARGETS_MISC || exit 1
+      fi
+   fi
+   exit $?
+fi
+
+function launch_build()
+{
+   local build_type=$1
+   shift
+
+   local logfile=$MY_WORKSPACE/build-$build_type.log
+   local rc
+   local targets
+
+   if [ "$build_type" == "std" ]; then
+      targets="$TARGETS_STD $TARGETS_MISC"
+   else
+      if [ "$build_type" == "rt" ]; then
+         targets="$TARGETS_RT $TARGETS_MISC"
+      else
+         if [ "$build_type" == "installer" ]; then
+            targets="$TARGETS_INSTALLER $TARGETS_MISC"
+         else
+            targets="$TARGETS"
+         fi
+      fi
+   fi
+
+   echo "Launching $build_type build, logging to $logfile"
+   \rm $logfile
+
+   echo -e "\n######## $(date): Launching build-srpms-parallel --$build_type $EXTRA_ARGS $@\n" | tee --append $logfile
+   # No clean flag, call build-srpms-parallel followed by build-rpms-parallel
+   echo "$DIR/build-srpms-parallel --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets" | tee --append $logfile
+   $DIR/build-srpms-parallel --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets 2>&1 | tee --append $logfile
+   rc=${PIPESTATUS[0]}
+   if [ $rc -eq 0 ]; then
+      echo -e "\n######## $(date): build-srpm-parallel --$build_type was successful" | tee --append $logfile
+   else
+      echo -e "\n######## $(date): build-srpm-parallel --$build_type failed with rc=$rc" | tee --append $logfile
+      echo -e "\n$(date): build-srpm-parallel --$build_type failed with rc=$rc"
+      exit $rc
+   fi
+
+   if [ $EDIT_FLAG -ne 1 ]; then
+      echo -e "\n######## $(date): Launching build-rpms-parallel --$build_type $EXTRA_ARGS $@\n" | tee --append $logfile
+      echo "$DIR/build-rpms-parallel --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $targets" | tee --append $logfile
+      $DIR/build-rpms-parallel --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $targets 2>&1 | tee --append $logfile
+      rc=${PIPESTATUS[0]}
+      if [ $rc -eq 0 ]; then
+         echo -e "\n######## $(date): build-rpm-parallel --$build_type was successful" | tee --append $logfile
+      else
+         echo -e "\n######## $(date): build-rpm-parallel --$build_type failed with rc=$rc" | tee --append $logfile
+         echo -e "\n$(date): build-rpm-parallel --$build_type failed with rc=$rc"
+         exit $rc
+      fi
+   fi
+
+   echo -e "\n$(date): $build_type complete\n"
+   #exit $rc
+}
+
+function progbar()
+{
+   while :; do
+      for s in / - \\ \|; do
+         printf "\r$s"
+         sleep .5
+      done
+   done
+}
+
+# Create $MY_WORKSPACE if it doesn't exist already
+mkdir -p $MY_WORKSPACE
+
+if [ $STD_BUILD -eq 1 ]; then
+   if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+      launch_build std
+   else
+      echo "Skipping 'std' build, no valid targets in list: $TARGETS"
+   fi
+else
+   echo "Skipping 'std' build"
+fi
+if [ $RT_BUILD -eq 1 ]; then
+   if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+      launch_build rt
+   else
+      echo "Skipping 'rt' build, no valid targets in list: $TARGETS"
+   fi
+else
+   echo "Skipping 'rt' build"
+fi
+if [ $INSTALLER_BUILD -eq 1 ]; then
+   if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_INSTALLER" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+      launch_build installer
+   else
+      echo "Skipping 'installer' build, no valid targets in list: $TARGETS"
+   fi
+else
+   echo "Skipping 'installer' build"
+fi
+
+
+echo "All builds were successful"
+
+exit 0
+

build-tools/build-pkgs-serial

@@ -0,0 +1,379 @@
+#!/bin/bash
+
+# This program is a wrapper around build-srpms and build-rpms
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+usage () {
+    echo ""
+    echo "Usage: "
+    echo "   Create source and Binary rpms:"
+    echo "   Build optimizations (--no-descendants, --no-required, --no-build-info,"
+    echo "   --no-autoclean) are not recommended for the first build after a clone/pull,"
+    echo "   nor the final build prior to creating an iso or patch, but can be used"
+    echo "   for intermediate builds.  i.e. while debugging compilation failures." 
+    echo "      build-pkgs [--no-descendants] [--no-required] [--no-build-info] [--no-autoclean] [--careful] [--formal] [ list of package names ]"
+    echo ""
+    echo "   Delete source rpms, and the directories associated with it's creation:"
+    echo "   Note: does not clean an edit environment"
+    echo "      build-pkgs --clean [ list of package names ]"
+    echo ""
+    echo "   Extract an src.rpm into a pair of git trees to aid in editing it's contents,"
+    echo "   one for source code and one for metadata such as the spec file."
+    echo "   If --no-meta-patch is specified, then WRS patches are omitted."
+    echo "      build-pkgs --edit [--no-meta-patch] [ list of package names ]"
+    echo ""
+    echo "   Delete an edit environment"
+    echo "      build-pkgs --edit --clean [ list of package names ]"
+    echo ""
+    echo "   This help page"
+    echo "      build-pkgs [--help]"
+    echo ""
+}
+
+
+HELP=0
+CLEAN_FLAG=0
+EDIT_FLAG=0
+STD_BUILD=1
+RT_BUILD=1
+INSTALLER_BUILD=0
+
+# read the options
+TEMP=`getopt -o h --long serial,rt,std,edit,no-meta-patch,no-descendants,no-required,no-build-info,no-autoclean,formal,careful,help,clean -n 'build-pkgs' -- "$@"`
+if [ $? -ne 0 ]; then
+    usage
+    exit 0
+fi
+eval set -- "$TEMP"
+
+# extract options and their arguments into variables.
+EXTRA_ARGS_COMMON=""
+EXTRA_ARGS_SRPM=""
+EXTRA_ARGS_RPM=""
+while true ; do
+    case "$1" in
+        --no-descendants) EXTRA_ARGS_COMMON+=" --no-descendants" ; shift ;;
+        --formal)         EXTRA_ARGS_COMMON+=" --formal" ; shift ;;
+        --careful)        EXTRA_ARGS_RPM+=" --careful" ; shift ;;
+        --no-required)    EXTRA_ARGS_RPM+=" --no-required" ; shift ;;
+        --no-build-info)  EXTRA_ARGS_COMMON+=" --no-build-info" ; shift ;;
+        --no-autoclean)   EXTRA_ARGS_RPM+=" --no-autoclean" ; shift ;;
+        --no-meta-patch)  EXTRA_ARGS_SRPM+=" --no-meta-patch" ; shift ;;
+        -h|--help)        HELP=1 ; shift ;;
+        --clean)          CLEAN_FLAG=1 ; shift ;;
+        --edit)           EDIT_FLAG=1 ; EXTRA_ARGS_SRPM+=" --edit"; shift ;;
+        --rt)             STD_BUILD=0 ; shift ;;
+        --std)            RT_BUILD=0 ; shift ;;
+        --installer)      INSTALLER_BUILD=1 ; STD_BUILD=0 ; RT_BUILD=0 ; shift ;;
+        --serial)         shift ;;
+        --)               shift ; break ;;
+        *)                usage; exit 1 ;;
+    esac
+done
+
+if [ $HELP -eq 1 ]; then
+    usage
+    exit 0
+fi
+
+TARGETS=" $@ "
+TARGETS_STD=" "
+TARGETS_RT=" "
+TARGETS_INSTALLER=" "
+TARGETS_MISC=" "
+
+find_targets () {
+   local centos_pkg_dirs=$1
+   local d=""
+   local d2=""
+   local g=""
+   local x=""
+   local name=""
+   local path=""
+   local RESULT=" "
+   local FOUND=0
+
+   for g in $(find $MY_REPO -type d -name .git); do
+      d=$(dirname $g)
+      if [ -f $d/$centos_pkg_dirs ]; then
+         for d2 in $(grep -v '^#' $d/$centos_pkg_dirs); do
+            name=""
+            if [ -f $d/$d2/centos/srpm_path ]; then
+                path=$(cat $d/$d2/centos/srpm_path | head -n 1 | sed "s#^mirror:CentOS/tis-r3-CentOS/mitaka#$MY_REPO/cgcs-centos-repo#" | sed "s#^mirror:#$MY_REPO/cgcs-centos-repo/#" | sed "s#^repo:#$MY_REPO/#" | sed "s#^3rd_party:#$MY_REPO/cgcs-3rd-party-repo/#" | sed "s#^Source/#$MY_REPO/cgcs-centos-repo/Source/#")
+                name=$(rpm -q --qf='%{NAME}' --nosignature -p $path)
+            else
+                path=$(find $d/$d2/centos/ -name '*.spec' | head -n 1)
+                if [[ ( -z "$path" ) &&  ( -f $d/$d2/centos/spec_path ) ]]; then
+                    path=$(find $MY_REPO/$(cat $d/$d2/centos/spec_path) -maxdepth 1 -name '*.spec' | head -n 1)
+                fi
+                if [ "$path" != "" ]; then
+                   name=$(spec_find_tag Name "$path" 2>> /dev/null)
+                fi
+            fi
+            if [ "$name" != "" ]; then
+               if [ "$BUILD_TYPE" == "rt" ]; then
+                  FOUND=0
+                  for x in $TARGETS; do
+                     if [ "${x: -3}" == "-rt" ]; then
+                        if [ "${name}" == "$x" ] || [ "${name}-rt" == "${x}" ]; then
+                           RESULT+="$x "
+                           FOUND=1
+                           break
+                        fi
+                     fi
+                  done
+                  if [ $FOUND -eq 0 ]; then
+                     for x in $TARGETS; do
+                        if [ "${name}" == "${x}-rt" ]; then
+                           RESULT+="$x-rt "
+                           FOUND=1
+                           break
+                        else
+                           if [ "${name}" == "$x" ] || [ "${name}-rt" == "${x}" ]; then
+                              RESULT+="$x "
+                              FOUND=1
+                              break
+                           fi
+                        fi
+                     done
+                  fi
+               else
+                  for x in $TARGETS; do
+                     if [ "${name}" == "$x" ]; then
+                         RESULT+="$x "
+                         FOUND=1
+                         break
+                     fi
+                  done
+               fi
+            fi
+         done
+      fi
+   done
+
+   echo "$RESULT"
+   return 0
+}
+
+echo "CLEAN_FLAG=$CLEAN_FLAG"
+echo "EDIT_FLAG=$EDIT_FLAG"
+
+if [ "x$TARGETS" != "x  " ]; then
+   source $MY_REPO/build-tools/spec-utils
+   TARGETS_STD="$(find_targets centos_pkg_dirs)"
+
+   BUILD_TYPE_SAVE="$BUILD_TYPE"
+   BUILD_TYPE="rt"
+   TARGETS_RT="$(find_targets centos_pkg_dirs_rt)"
+   BUILD_TYPE="installer"
+   TARGETS_INSTALLER="$(find_targets centos_pkg_dirs_installer)"
+   BUILD_TYPE="$BUILD_TYPE_SAVE"
+
+   echo "TARGETS_STD=$TARGETS_STD"
+   echo "TARGETS_RT=$TARGETS_RT"
+   echo "TARGETS_INSTALLER=$TARGETS_INSTALLER"
+
+   for x in $TARGETS; do
+       if [[ $TARGETS_STD == *" $x "* ]]
+       then
+           echo "found $x" >> /dev/null;
+       else
+           if [[ $TARGETS_RT == *" $x "* ]]
+           then
+               echo "found $x" >> /dev/null;
+           else
+               if [[ $TARGETS_INSTALLER == *" $x "* ]]
+               then
+                   echo "found $x" >> /dev/null;
+                   INSTALLER_BUILD=1
+               else
+                   TARGETS_MISC+="$x "
+               fi
+           fi
+       fi
+   done
+fi
+
+echo "EXTRA_ARGS_COMMON='$EXTRA_ARGS_COMMON'"
+echo "EXTRA_ARGS_SRPM='$EXTRA_ARGS_SRPM'"
+echo "EXTRA_ARGS_RPM='$EXTRA_ARGS_RPM'"
+echo "TARGETS='$TARGETS'"
+echo "TARGETS_STD='$TARGETS_STD'"
+echo "TARGETS_RT='$TARGETS_RT'"
+echo "TARGETS_INSTALLER='$TARGETS_INSTALLER'"
+echo "TARGETS_MISC='$TARGETS_MISC'"
+
+if [ $CLEAN_FLAG -eq 1 ]; then
+   if [ $STD_BUILD -eq 1 ]; then
+      if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+         if [ $EDIT_FLAG -ne 1 ]; then
+            echo "$DIR/build-rpms-serial --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_STD $TARGETS_MISC"
+            $DIR/build-rpms-serial --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_STD $TARGETS_MISC || exit 1
+         fi
+      fi
+      if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+         echo "$DIR/build-srpms-serial --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_STD $TARGETS_MISC"
+         $DIR/build-srpms-serial --std --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_STD $TARGETS_MISC || exit 1
+      fi
+   fi
+   if [ $RT_BUILD -eq 1 ]; then
+      if [ "x$TARGETS" == "x" ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+         if [ $EDIT_FLAG -ne 1 ]; then
+            echo "$DIR/build-rpms-serial --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_RT $TARGETS_MISC"
+            $DIR/build-rpms-serial --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_RT $TARGETS_MISC || exit 1
+         fi
+      fi
+      if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+         echo "$DIR/build-srpms-serial --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_RT $TARGETS_MISC"
+         $DIR/build-srpms-serial --rt --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_RT $TARGETS_MISC || exit 1
+      fi
+   fi
+   if [ $INSTALLER_BUILD -eq 1 ]; then
+      if [ "x$TARGETS" == "x" ] || [ "$TARGETS_INSTALLER" != " " ]; then
+         if [ $EDIT_FLAG -ne 1 ]; then
+            echo "$DIR/build-rpms-serial --installer --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_INSTALLER"
+            $DIR/build-rpms-serial --installer --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $TARGETS_INSTALLER || exit 1
+         fi
+      fi
+      if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_INSTALLER" != " " ]; then
+         echo "$DIR/build-srpms-serial --installer --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_INSTALLER"
+         $DIR/build-srpms-serial --installer --clean $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $TARGETS_INSTALLER || exit 1
+      fi
+   fi
+   exit $?
+fi
+
+function launch_build()
+{
+   local build_type=$1
+   shift
+
+   local logfile=$MY_WORKSPACE/build-$build_type.log
+   local rc
+   local targets
+
+   if [ "$build_type" == "std" ]; then
+      targets="$TARGETS_STD $TARGETS_MISC"
+   else
+      if [ "$build_type" == "rt" ]; then
+         targets="$TARGETS_RT $TARGETS_MISC"
+      else
+         if [ "$build_type" == "installer" ]; then
+            targets="$TARGETS_INSTALLER"
+         else
+            targets="$TARGETS"
+         fi
+      fi
+   fi
+
+   echo "Launching $build_type build, logging to $logfile"
+
+   echo -e "\n######## $(date): Launching build-srpms-serial --$build_type $EXTRA_ARGS $@\n" | tee --append $logfile
+   # No clean flag, call build-srpms-serial followed by build-rpms-serial
+   echo "$DIR/build-srpms-serial --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets" | tee --append $logfile
+   $DIR/build-srpms-serial --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets 2>&1 | tee --append $logfile
+   rc=${PIPESTATUS[0]}
+   if [ $rc -eq 0 ]; then
+      echo -e "\n######## $(date): build-srpm-serial --$build_type was successful" | tee --append $logfile
+   else
+      echo -e "\n######## $(date): build-srpm-serial --$build_type failed with rc=$rc" | tee --append $logfile
+      echo -e "\n$(date): build-srpm-serial --$build_type failed with rc=$rc"
+      exit $rc
+   fi
+
+   if [ $EDIT_FLAG -ne 1 ]; then
+      echo -e "\n######## $(date): Launching build-rpms-serial --$build_type $EXTRA_ARGS $@\n" | tee --append $logfile
+      echo "$DIR/build-rpms-serial --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_SRPM $targets" | tee --append $logfile
+      $DIR/build-rpms-serial --$build_type $EXTRA_ARGS_COMMON $EXTRA_ARGS_RPM $targets 2>&1 | tee --append $logfile
+      rc=${PIPESTATUS[0]}
+      if [ $rc -eq 0 ]; then
+         echo -e "\n######## $(date): build-rpm-serial --$build_type was successful" | tee --append $logfile
+      else
+         echo -e "\n######## $(date): build-rpm-serial --$build_type failed with rc=$rc" | tee --append $logfile
+         echo -e "\n$(date): build-rpm-serial --$build_type failed with rc=$rc"
+         exit $rc
+      fi
+   fi
+
+   echo -e "\n$(date): $build_type complete\n"
+   #exit $rc
+}
+
+function progbar()
+{
+   while :; do
+      for s in / - \\ \|; do
+         printf "\r$s"
+         sleep .5
+      done
+   done
+}
+
+# Create $MY_WORKSPACE if it doesn't exist already
+mkdir -p $MY_WORKSPACE
+
+if [ $STD_BUILD -eq 1 ]; then
+   if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_STD" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+      launch_build std
+   else
+      echo "Skipping 'std' build, no valid targets in list: $TARGETS"
+   fi
+else
+   echo "Skipping 'std' build"
+fi
+if [ $RT_BUILD -eq 1 ]; then
+   if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_RT" != " " ] || [ "$TARGETS_MISC" != " " ]; then
+      launch_build rt
+   else
+      echo "Skipping 'rt' build, no valid targets in list: $TARGETS"
+   fi
+else
+   echo "Skipping 'rt' build"
+fi
+if [ $INSTALLER_BUILD -eq 1 ]; then
+   if [ "x$TARGETS" == "x  " ] || [ "$TARGETS_INSTALLER" != " " ]; then
+      launch_build installer
+   else
+      echo "Skipping 'installer' build, no valid targets in list: $TARGETS"
+   fi
+# else
+   # echo "Skipping 'installer' build"
+fi
+
+
+#progbar &
+
+#function killemall() {
+#    for pid in $(jobs -p)
+#    do
+#        pgid=$(echo -n $(ps -o pgid= $pid))
+#        if [ -n "$pgid" ]; then
+#            echo "Killing child progress group: kill -TERM -$pgid"
+#            kill -TERM -$pgid
+#        fi
+#    done
+#    echo "Bad mojo when you ctrl-C, so don't"
+#}
+
+#trap killemall INT
+
+## Wait for builds to complete
+#wait %1
+#STD_BUILD_RC=$?
+
+#wait %2
+#RT_BUILD_RC=$?
+
+# Kill the progress bar
+#kill %3
+
+#if [ $STD_BUILD_RC -ne 0 -o $RT_BUILD_RC -ne 0 ]; then
+#    echo "One or more builds failed"
+#    exit 1
+#fi
+
+echo "All builds were successful"
+
+exit 0
+

build-tools/build-pkgs4

@@ -0,0 +1 @@
+build-pkgs-parallel

build-tools/build-rpms

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# This program is a wrapper around build-rpms-parallel and build-rpms-serial
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+usage () {
+    echo ""
+    echo "Usage: "
+    echo "   Create binary rpms:"
+    echo "   build-rpms [--serial] [args]"
+}
+
+SERIAL_FLAG=0
+
+for arg in "$@"; do
+    case "$1" in
+        --serial) SERIAL_FLAG=1 ;;
+    esac
+done
+
+which mock_tmpfs_umount >> /dev/null
+if [ $? -ne 0 ]; then
+    SERIAL_FLAG=1
+fi
+
+if [ $SERIAL_FLAG -eq 1 ]; then
+    echo "build-rpms-serial $@"
+    build-rpms-serial "$@"
+else
+    echo "build-rpms-parallel $@"
+    build-rpms-parallel "$@"
+fi
+

build-tools/build-rpms4

@@ -0,0 +1 @@
+build-rpms-parallel

build-tools/build-sdk

@@ -0,0 +1,70 @@
+#!/bin/bash
+
+# We find SDK files in two ways -- one is that any package with an
+# "-cgts-sdk" component in the filename is deemed to have SDK content
+# (this works great for packages that produce installable content, as well
+# as SDK content which is placed in a separate -cgts-sdk package)
+#
+# The second way is explicitly name packages that contain SDK content.  This
+# works well for packages which only contain SDK content.  The other reason
+# that calling out packages explcitly may be of use is that some packages
+# (like restapi-doc) may not be built automatically because they get caught
+# in the blacklist filter (i.e. we normally don't build restapi-doc because
+# it contains "doc").
+
+EXPLICIT_PACKAGES=( \
+	wrs-branding \
+	wrs-heat-templates \
+	install-log-server \
+	restapi-doc \
+	remote-clients \
+	)
+
+extract_to_sdk () {
+	pushd $SDKTMP
+	rpm2cpio $1 | cpio -id
+	popd
+}
+
+SDKDIR=$MY_WORKSPACE/export/cgts-sdk
+SDKTMP=$MY_WORKSPACE/export/cgts-sdk_tmp
+SDKTMPFILE=$MY_WORKSPACE/export/sdk_tmp_file
+
+# These patterns match packages for which we find SDK tarballs
+PATTERNS=(wrs-branding*.x86_64.rpm \
+	wrs-heat-templates*.x86_64.rpm \
+	*cgts-sdk*.x86_64.rpm  \
+	install-log-server*.x86_64.rpm \
+	restapi-doc*.x86_64.rpm \
+	remote-clients*.x86_64.rpm )
+
+rm -f $SDKTMPFILE
+
+for pkg in "${EXPLICIT_PACKAGES[@]}"; do
+	ls $MY_WORKSPACE/std/rpmbuild/RPMS/$pkg*.x86_64.rpm > /dev/null
+	if [ $? -eq 2 ]; then # no page found...
+		build-rpms --std $pkg --no-descendants
+	fi
+done
+
+for pat in "${PATTERNS[@]}"; do
+	find $MY_WORKSPACE/std/rpmbuild/RPMS/$pat >> $SDKTMPFILE
+done
+
+rm -rf $SDKDIR $SDKTMP
+mkdir -p $SDKDIR
+mkdir -p $SDKTMP
+
+# extract the files
+while read rpm; do
+	extract_to_sdk $rpm
+done < $SDKTMPFILE
+
+find $SDKTMP -name "*.tgz" -exec cp -v {} $SDKDIR \;
+
+rm -f $SDKTMPFILE
+rm -rf $SDKTMP
+
+echo ""
+echo "Done.  SDK in $SDKDIR"
+echo ""

build-tools/build-srpms

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# This program is a wrapper around build-srpms-parallel and build-srpms-serial
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+usage () {
+    echo ""
+    echo "Usage: "
+    echo "   Create source rpms:"
+    echo "   build-srpms [--serial] [args]"
+}
+
+SERIAL_FLAG=0
+
+for arg in "$@"; do
+    case "$1" in
+        --serial) SERIAL_FLAG=1 ;;
+    esac
+done
+
+which mock_tmpfs_umount >> /dev/null
+if [ $? -ne 0 ]; then
+    SERIAL_FLAG=1
+fi
+
+if [ $SERIAL_FLAG -eq 1 ]; then
+    echo "build-srpms-serial $@"
+    build-srpms-serial "$@"
+else
+    echo "build-srpms-parallel $@"
+    build-srpms-parallel "$@"
+fi
+

build-tools/build-srpms4

@@ -0,0 +1 @@
+build-srpms-parallel

build-tools/build_guest/build-guest-image.py

@@ -0,0 +1,123 @@
+#!/usr/bin/env python
+
+#
+# Build a bootable guest image from the supplied rootfs archive
+#
+
+import getopt
+import guestfs
+import os
+import sys
+
+
+MBR_FILE='/usr/share/syslinux/mbr.bin'
+MBR_SIZE=440
+
+def build_image(inputfile, outputfile, extrasize, trace):
+    g = guestfs.GuestFS(python_return_dict=True)
+
+    # Set the trace flag so that we can see each libguestfs call.
+    if trace:
+        g.set_trace(1)
+
+    # Create a raw-format sparse disk image with padding of size
+    inputsize = os.path.getsize(inputfile)
+    g.disk_create(outputfile, "raw", inputsize + extrasize)
+
+    # Attach the new disk image to libguestfs.
+    g.add_drive_opts(outputfile, format="raw", readonly=0)
+
+    # Run the libguestfs back-end.
+    g.launch()
+
+    # Get the list of devices.  Because we only added one drive
+    # above, we expect that this list should contain a single
+    # element.
+    devices = g.list_devices()
+    assert(len(devices) == 1)
+
+    # Partition the disk as one single MBR partition.
+    g.part_disk(devices[0], "mbr")
+
+    # Get the list of partitions.  We expect a single element, which
+    # is the partition we have just created.
+    partitions = g.list_partitions()
+    assert(len(partitions) == 1)
+
+    # Create a filesystem on the partition.
+    # NOTE: extlinux does not support 64-bit file systems
+    g.mkfs("ext4", partitions[0], features="^64bit")
+
+    # Now mount the filesystem so that we can add files.
+    g.mount(partitions[0], "/")
+
+    # Upload file system files and directories.
+    g.tar_in(inputfile, "/")
+
+    # Install the boot loader
+    g.extlinux("/boot")
+
+    # Unmount the file systems.
+    g.umount_all();
+
+    # Write the master boot record.
+    with open(MBR_FILE, mode='rb') as mbr:
+        mbr_data = mbr.read()
+        assert(len(mbr_data) == MBR_SIZE)
+        g.pwrite_device(devices[0], mbr_data, 0)
+
+    # Mark the device as bootable.
+    g.part_set_bootable(devices[0], 1, 1)
+    
+    # Label the boot disk for root identification
+    g.set_label(partitions[0], "wrs_guest")
+
+    # Shutdown and close guest image
+    g.shutdown()
+    g.close()
+
+
+def exit_usage(result=0):
+    print('USAGE: -i <input-file> -o <output-file> [-s <extra-bytes>]')
+    sys.exit(result)
+
+
+def main(argv):
+    inputfile = None
+    outputfile = None
+    extrasize = None
+    trace = False
+
+    try:
+        opts, args = getopt.getopt(argv,"hxi:o:s:",
+                                   ["input=", "output=", "size="])
+    except getopt.GetoptError:
+        exit_usage(2)
+    for opt, arg in opts:
+        if opt == '-h':
+            exit_usage()
+        if opt == '-x':
+            trace = True
+        elif opt in ("-i", "--input"):
+            inputfile = arg
+        elif opt in ("-o", "--output"):
+            outputfile = arg
+        elif opt in ("-s", "--size"):
+            extrasize = int(arg)
+
+    if not inputfile:
+        print(stderr, "ERROR: missing input file")
+        exit_usage(-1)
+
+    if not outputfile:
+        print(stderr, "ERROR: missing output file")
+        exit_usage(-1)
+
+    if not extrasize:
+        extrasize = 0
+
+    build_image(inputfile, outputfile, extrasize, trace)
+
+
+if __name__ == "__main__":
+    main(sys.argv[1:])

build-tools/build_guest/image-rt.inc

@@ -0,0 +1,39 @@
+# List of CGTS packages to be included/installed in guest image
+# If these have dependencies, they will be pulled in automatically
+#
+build-info
+
+# custom cloud-init configuration
+
+# guest extensions
+guest-scale-agent
+guest-host-comm
+guest-client
+
+# Latest version of the i40e drivers from Intel.
+kmod-i40e-rt
+kmod-i40evf-rt
+
+# Latest version of the ixgbe drivers from Intel.
+kmod-ixgbe-rt
+kmod-ixgbevf-rt
+
+# Configure connectx pro devices to ethernet mode.
+# This is needed for PT devices
+# mlx4-config
+
+# Intel QuickAssist
+# qat16-guest
+# qat16-guest-app
+qat17-rt
+
+# TPM2 libs to enable vTPM on Guest
+tss2
+tpm2-tools
+kmod-tpm-rt
+
+# ima plugin for RPM
+rpm-plugin-systemd-inhibit
+
+# Add cfn-push-stats for heat demos
+heat-cfntools

build-tools/build_guest/image.inc

@@ -0,0 +1,44 @@
+# List of CGTS packages to be included/installed in guest image
+# If these have dependencies, they will be pulled in automatically
+#
+build-info
+
+# custom cloud-init configuration
+
+# guest extensions
+guest-scale-agent
+guest-host-comm
+guest-client
+
+# Latest version of the i40e drivers from Intel.
+kmod-i40e
+kmod-i40evf
+
+# Latest version of the ixgbe drivers from Intel.
+kmod-ixgbe
+kmod-ixgbevf
+
+# Intel QuickAssist
+# qat16-guest
+# qat16-guest-app
+qat17
+
+# TPM2 libs to enable vTPM on Guest
+tss2
+tpm2-tools
+kmod-tpm
+
+# This will help us have our automation debug TC failures when pings to VMs fail. 
+qemu-guest-agent
+
+# ima plugin for rpm
+rpm-plugin-systemd-inhibit
+
+# Add debugging tools
+perf
+zip
+unzip
+traceroute
+
+# Add cfn-push-stats for heat demos
+heat-cfntools

build-tools/build_guest/rootfs-exclude.txt

@@ -0,0 +1,13 @@
+# exclude special filesystems
+/builddir
+/dev/*
+/proc/*
+/tmp/*
+/sys/*
+/root/rootfs.tar
+
+# exclude local repo yum configuration
+/etc/yum/yum.conf
+
+# omit platform hooks to check install uuid
+/etc/dhcp/dhclient-enter-hooks

build-tools/build_guest/rootfs-rt/boot/extlinux.conf

@@ -0,0 +1,7 @@
+SERIAL 0 115200
+
+DEFAULT linux
+LABEL linux
+  KERNEL vmlinuz
+  INITRD initramfs.img
+  APPEND rw root=LABEL=wrs_guest clocksource=pit console=tty0 console=ttyS0 biosdevname=0 net.ifnames=0 no_timer_check audit=0 cgroup_disable=memory isolcpus=1-3 irqaffinity=0  nmi_watchdog=0 softlockup_panic=0 intel_idle.max_cstate=0 processor.max_cstate=1 idle=poll

build-tools/build_guest/rootfs-setup.sh

@@ -0,0 +1,90 @@
+#!/bin/bash
+
+BUILD_MODE=''
+if [ "$1" == "--rt" ]; then
+   BUILD_MODE="rt"
+fi
+if [ "$1" == "--std" ]; then
+   BUILD_MODE="std"
+fi
+
+# Setup boot directory for syslinux configuration (/boot/extlinux.conf)
+ln -s $(ls /boot/vmlinuz-*.x86_64 | head -1) /boot/vmlinuz
+ln -s $(ls /boot/initramfs-*.x86_64.img | head -1) /boot/initramfs.img
+
+# Setup root and wrsroot users
+usermod -p $(openssl passwd -1 root) root
+useradd -p $(openssl passwd -1 wrsroot) wrsroot
+
+# Enable SUDO access for wrsroot
+echo "wrsroot ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
+
+# Enable remote root login to permit automated tools to run privileged commands
+sed -i 's%^#\(PermitRootLogin \)%\1%' /etc/ssh/sshd_config
+sed -i 's#^\(PermitRootLogin \).*#\1yes#' /etc/ssh/sshd_config
+
+# Enable password login to permit automated tools to run commands
+sed -i 's%^#\(PasswordAuthentication \)%\1%' /etc/ssh/sshd_config
+sed -i 's#^\(PasswordAuthentication \).*#\1yes#' /etc/ssh/sshd_config
+
+# Disable PAM authentication
+sed -i 's#^\(UsePAM \).*#\1no#' /etc/ssh/sshd_config
+
+# Prevent cloud_init for reverting our changes
+sed -i 's#^\(ssh_pwauth:\).*#\1 1#' /etc/cloud/cloud.cfg
+sed -i 's#^\(disable_root:\).*#\1 0#' /etc/cloud/cloud.cfg
+
+# Setup SSHD to mark packets for QoS processing in the host (this seems to
+# be broken in our version of SSHd so equivalent iptables rules are being
+# added to compensate.
+echo "IPQoS cs7" >> /etc/ssh/sshd_config
+
+# Disable reverse path filtering to permit traffic testing from
+# foreign routes.
+sed -i 's#^\(net.ipv4.conf.*.rp_filter=\).*#\10#' /etc/sysctl.conf
+
+# Change /etc/rc.local to touch a file to indicate that the init has
+# completed.  This is required by the AVS vbenchmark tool so that it knows
+# that the VM is ready to run.  This was added because VM instances take a
+# long time (2-3 minutes) to resize their filesystem when run on a system with
+# HDD instead of SSD.
+chmod +x /etc/rc.d/rc.local
+echo "touch /var/run/.init-complete" >> /etc/rc.local
+
+if [ "$BUILD_MODE" == "rt" ]; then
+   # Adjust system tuning knobs during init when using rt kernel (CGTS-7047)
+   echo "echo 1 > /sys/devices/virtual/workqueue/cpumask" >> /etc/rc.local
+   echo "echo 1 > /sys/bus/workqueue/devices/writeback/cpumask" >> /etc/rc.local
+   echo "echo -1 > /proc/sys/kernel/sched_rt_runtime_us" >> /etc/rc.local
+   echo "echo 0 > /proc/sys/kernel/timer_migration" >> /etc/rc.local
+   echo "echo 10 > /proc/sys/vm/stat_interval" >> /etc/rc.local
+fi
+
+# Disable audit service by default
+# With this enabled, it causes system delays when running at maximum
+# capacity that impacts the traffic processing enough to cause unclean
+# traffic runs when doing benchmark tests.
+systemctl disable auditd
+
+if [ "$BUILD_MODE" == "rt" ]; then
+   # Additional services to disable on rt guest (CGTS-7047)
+   systemctl disable polkit.service
+   systemctl disable tuned.service
+fi
+
+# Clean the yum cache.  We don't want to maintain it on the guest file system.
+yum clean all
+
+# update /etc/rsyslog.conf to have OmitLocalLogging off
+sed -i 's#OmitLocalLogging on#OmitLocalLogging off#g' /etc/rsyslog.conf
+
+# select correct kernel and initrd
+if [ "$BUILD_MODE" == "rt" ]; then
+   PATTERN=$(rpm -q --qf '%{VERSION}-%{RELEASE}' kernel-rt)
+else
+   PATTERN=$(rpm -q --qf '%{VERSION}-%{RELEASE}' kernel)
+fi
+cd /boot
+rm -f vmlinuz initramfs.img
+ln -s $(ls -1 vmlinuz-$PATTERN*) vmlinuz
+ln -s $(ls -1 initramfs-$PATTERN*img) initramfs.img

build-tools/build_guest/rootfs-std/boot/extlinux.conf

@@ -0,0 +1,7 @@
+SERIAL 0 115200
+
+DEFAULT linux
+LABEL linux
+  KERNEL vmlinuz
+  INITRD initramfs.img
+  APPEND rw root=LABEL=wrs_guest clocksource=pit console=tty0 console=ttyS0 biosdevname=0 net.ifnames=0 no_timer_check

build-tools/build_guest/rootfs/etc/cloud/cloud.cfg.d/99_wrs-datasources.cfg

@@ -0,0 +1,18 @@
+# Override the datasource list to use only those that are expected (and needed)
+# to work in our lab environment.
+#
+datasource_list:
+ - NoCloud
+ - ConfigDrive
+ - Ec2
+ - None
+
+# Adjust the Ec2 max_wait to be 30 seconds instead of the default 120 seconds,
+# and set the list of URLs to be the only one that we expect to work in our lab
+# environment so that we avoid DNS lookup failures for alternate choices.
+#
+datasource:
+  Ec2:
+    timeout: 10
+    max_wait: 30
+    metadata_urls: ['http://169.254.169.254']

build-tools/build_guest/rootfs/etc/dhcp/dhclient.conf

@@ -0,0 +1,21 @@
+## Use a CID based on the hardware address for both IPv4 and IPv6.  This mostly
+## useful for IPv6 to ensure that the client is not using a random DUID for the
+## CID on each reboot.
+send dhcp6.client-id = concat(00:03:00, hardware);
+send dhcp-client-identifier = concat(00:03:00, hardware);
+
+## Defaults for all interfaces
+request interface-mtu, subnet-mask, broadcast-address, time-offset,
+    classless-static-routes;
+
+interface "eth0" {
+    ## Override for eth0 to add requests for attributes that we only care to 
+    ## configure for our primary network interface
+    request interface-mtu, subnet-mask, broadcast-address, time-offset,
+        domain-name, domain-name-servers, host-name,
+        classless-static-routes, routers;
+}
+
+timeout 15;
+
+retry 5;

build-tools/build_guest/rootfs/etc/iptables.rules

@@ -0,0 +1,12 @@
+*mangle
+:PREROUTING ACCEPT [0:0]
+:INPUT ACCEPT [0:0]
+:FORWARD ACCEPT [0:0]
+:OUTPUT ACCEPT [0:0]
+:POSTROUTING ACCEPT [0:0]
+-A OUTPUT -o eth0 -p tcp --sport 22 -j DSCP --set-dscp-class CS7
+-A OUTPUT -o eth0 -p tcp --dport 22 -j DSCP --set-dscp-class CS7
+-A OUTPUT -o eth0 -p udp --sport 67:68 -j DSCP --set-dscp-class CS7
+-A OUTPUT -o eth0 -p udp --dport 67:68 -j DSCP --set-dscp-class CS7
+-A OUTPUT -o eth0 -d 169.254.169.254 -j DSCP --set-dscp-class CS7
+COMMIT

build-tools/build_guest/rootfs/etc/modprobe.d/floppy.conf

@@ -0,0 +1 @@
+blacklist floppy

build-tools/build_guest/rootfs/etc/modprobe.d/wrs_avp.conf

@@ -0,0 +1 @@
+options wrs_avp kthread_cpulist=0-7 kthread_policy=0

build-tools/build_guest/rootfs/etc/modules-load.d/wrs_avp.conf

@@ -0,0 +1 @@
+wrs_avp

build-tools/build_guest/rootfs/etc/sysconfig/network-scripts/ifcfg-eth0

@@ -0,0 +1,8 @@
+DEVICE=eth0 
+BOOTPROTO=dhcp 
+ONBOOT=yes
+TYPE=Ethernet
+USERCTL=yes
+PEERDNS=yes
+IPV6INIT=no
+PERSISTENT_DHCLIENT=1

build-tools/build_guest/rootfs/etc/udev/rules.d/65-renumber-net.rules

@@ -0,0 +1,4 @@
+# Renames interfaces to be sequential ethX interface names regardless of interface type
+# This is required to avoid a kernel host patch that starts number at 1000 and to
+# override slot specific naming for non-kernel interfaces.
+ACTION=="add", SUBSYSTEM=="net", DRIVERS=="?*", ATTR{type}=="1", KERNEL=="eth?*" PROGRAM=="/usr/lib/udev/renumber_device", NAME="$result"

build-tools/build_guest/rootfs/usr/lib/udev/renumber_device

@@ -0,0 +1,12 @@
+#!/bin/bash
+# Renames interfaces to be sequential ethX interface names regardless of interface type
+# This is required to avoid a kernel host patch that starts number at 1000 and to
+# override slot specific naming for non-kernel interfaces.
+
+# The ifindex for the first interface that is not 'lo' will be 2.
+# Therefore adjust the numbering to start at 0 for eth0..ethN naming
+
+INDEX=$(($IFINDEX-2))
+echo "eth$INDEX"
+
+exit 0

build-tools/build_guest/rpm-install-list-rt.txt

@@ -0,0 +1,305 @@
+# list of standard packages to include in the guest image
+acl
+acpid
+audit
+audit-libs
+audit-libs-python
+authconfig
+basesystem
+bash
+bind-libs-lite
+bind-license
+binutils
+bonnie++
+bridge-utils
+btrfs-progs
+bzip2-libs
+ca-certificates
+centos-logos
+centos-release
+checkpolicy
+chkconfig
+chrony
+cloud-init
+cloud-utils-growpart
+coreutils
+cpio
+cracklib
+cracklib-dicts
+cronie
+cronie-anacron
+crontabs
+cryptsetup-libs
+curl
+cyrus-sasl-lib
+dbus
+dbus-glib
+dbus-libs
+dbus-python
+device-mapper
+device-mapper-libs
+dhclient
+dhcp-common
+dhcp-libs
+diffutils
+dmidecode
+dnsmasq
+dracut
+dracut-config-generic
+dracut-config-rescue
+dracut-network
+e2fsprogs
+e2fsprogs-libs
+elfutils-libelf
+elfutils-libs
+ethtool
+expat
+file
+file-libs
+filesystem
+findutils
+fipscheck
+fipscheck-lib
+freetype
+gawk
+gdbm
+gettext
+gettext-libs
+glib2
+glibc
+glibc-common
+glib-networking
+gmp
+gnupg2
+gnutls
+gobject-introspection
+gpgme
+grep
+groff-base
+grub2
+grub2-tools
+grubby
+gsettings-desktop-schemas
+gssproxy
+gzip
+hardlink
+hostname
+info
+initscripts
+iperf3
+iproute
+iptables
+iputils
+irqbalance
+iwl7265-firmware
+jansson
+jbigkit-libs
+json-c
+kbd
+kbd-legacy
+kbd-misc
+kernel-rt
+kernel-rt-tools
+kernel-rt-tools-libs
+kexec-tools
+keyutils
+keyutils-libs
+kmod
+kmod-libs
+kpartx
+krb5-libs
+less
+libacl
+libassuan
+libattr
+libbasicobjects
+libblkid
+libcap
+libcap-ng
+libcgroup
+libcollection
+libcom_err
+libcroco
+libcurl
+libdaemon
+libdb
+libdb-utils
+libedit
+libestr
+libevent
+libffi
+libgcc
+libgcrypt
+libgomp
+libgpg-error
+libgudev1
+libidn
+libini_config
+libjpeg-turbo
+libmnl
+libmodman
+libmount
+libndp
+libnetfilter_conntrack
+libnfnetlink
+libnfsidmap
+libnl3
+libnl3-cli
+libpath_utils
+libpcap
+libpipeline
+libproxy
+libpwquality
+libref_array
+libselinux
+libselinux-python
+libselinux-utils
+libsemanage
+libsemanage-python
+libsepol
+libsoup
+libss
+libssh2
+libstdc++
+libsysfs
+libtalloc
+libtasn1
+libteam
+libtevent
+libtiff
+libtirpc
+libunistring
+libuser
+libutempter
+libuuid
+libverto
+libverto-tevent
+libwebp
+libxml2
+libyaml
+logrotate
+lua
+lzo
+make
+man-db
+mariadb-libs
+microcode_ctl
+mozjs17
+ncurses
+ncurses-base
+ncurses-libs
+nettle
+net-tools
+newt
+newt-python
+nfs-utils
+nspr
+nss
+nss-softokn
+nss-softokn-freebl
+nss-sysinit
+nss-tools
+nss-util
+numactl-libs
+openssh
+openssh-clients
+openssh-server
+openssl
+openssl-libs
+os-prober
+p11-kit
+p11-kit-trust
+pam
+parted
+passwd
+pciutils
+pciutils-libs
+pcre
+pinentry
+pkgconfig
+policycoreutils
+policycoreutils-python
+polkit
+polkit-pkla-compat
+popt
+postfix
+ppp
+procps-ng
+pth
+pygobject3-base
+pygpgme
+pyliblzma
+python
+python-backports
+python-backports-ssl_match_hostname
+python-chardet
+python-cheetah
+python-configobj
+python-decorator
+python-iniparse
+python-IPy
+python-jsonpatch
+python-jsonpointer
+python-kitchen
+python-libs
+python-markdown
+python-perf
+python-pillow
+python-prettytable
+python-pycurl
+python-pygments
+python-pyudev
+python-requests
+python2-six
+python-urlgrabber
+python-urllib3
+pyxattr
+PyYAML
+qrencode-libs
+quota
+quota-nls
+rdma
+readline
+rootfiles
+rpcbind
+rpm
+rpm-build-libs
+rpm-libs
+rpm-python
+rsync
+rsyslog
+sed
+rt-setup
+rtctl
+shadow-utils
+shared-mime-info
+slang
+snappy
+sqlite
+sudo
+systemd
+systemd-libs
+systemd-sysv
+sysvinit-tools
+tar
+tcpdump
+tcp_wrappers
+tcp_wrappers-libs
+teamd
+trousers
+tuned
+tzdata
+ustr
+util-linux
+vim-minimal
+virt-what
+wget
+which
+wpa_supplicant
+xz
+xz-libs
+yum
+yum-metadata-parser
+yum-plugin-fastestmirror
+yum-utils
+zlib

build-tools/build_guest/rpm-install-list.txt

@@ -0,0 +1,303 @@
+# list of standard packages to include in the guest image
+acl
+acpid
+audit
+audit-libs
+audit-libs-python
+authconfig
+basesystem
+bash
+bind-libs-lite
+bind-license
+binutils
+bonnie++
+bridge-utils
+btrfs-progs
+bzip2-libs
+ca-certificates
+centos-logos
+centos-release
+checkpolicy
+chkconfig
+chrony
+cloud-init
+cloud-utils-growpart
+coreutils
+cpio
+cracklib
+cracklib-dicts
+cronie
+cronie-anacron
+crontabs
+cryptsetup-libs
+curl
+cyrus-sasl-lib
+dbus
+dbus-glib
+dbus-libs
+dbus-python
+device-mapper
+device-mapper-libs
+dhclient
+dhcp-common
+dhcp-libs
+diffutils
+dmidecode
+dnsmasq
+dracut
+dracut-config-generic
+dracut-config-rescue
+dracut-network
+e2fsprogs
+e2fsprogs-libs
+elfutils-libelf
+elfutils-libs
+ethtool
+expat
+file
+file-libs
+filesystem
+findutils
+fipscheck
+fipscheck-lib
+freetype
+gawk
+gdbm
+gettext
+gettext-libs
+glib2
+glibc
+glibc-common
+glib-networking
+gmp
+gnupg2
+gnutls
+gobject-introspection
+gpgme
+grep
+groff-base
+grub2
+grub2-tools
+grubby
+gsettings-desktop-schemas
+gssproxy
+gzip
+hardlink
+hostname
+info
+initscripts
+iperf3
+iproute
+iptables
+iputils
+irqbalance
+iwl7265-firmware
+jansson
+jbigkit-libs
+json-c
+kbd
+kbd-legacy
+kbd-misc
+kernel
+kernel-tools
+kernel-tools-libs
+kexec-tools
+keyutils
+keyutils-libs
+kmod
+kmod-libs
+kpartx
+krb5-libs
+less
+libacl
+libassuan
+libattr
+libbasicobjects
+libblkid
+libcap
+libcap-ng
+libcgroup
+libcollection
+libcom_err
+libcroco
+libcurl
+libdaemon
+libdb
+libdb-utils
+libedit
+libestr
+libevent
+libffi
+libgcc
+libgcrypt
+libgomp
+libgpg-error
+libgudev1
+libidn
+libini_config
+libjpeg-turbo
+libmnl
+libmodman
+libmount
+libndp
+libnetfilter_conntrack
+libnfnetlink
+libnfsidmap
+libnl3
+libnl3-cli
+libpath_utils
+libpcap
+libpipeline
+libproxy
+libpwquality
+libref_array
+libselinux
+libselinux-python
+libselinux-utils
+libsemanage
+libsemanage-python
+libsepol
+libsoup
+libss
+libssh2
+libstdc++
+libsysfs
+libtalloc
+libtasn1
+libteam
+libtevent
+libtiff
+libtirpc
+libunistring
+libuser
+libutempter
+libuuid
+libverto
+libverto-tevent
+libwebp
+libxml2
+libyaml
+logrotate
+lua
+lzo
+make
+man-db
+mariadb-libs
+microcode_ctl
+mozjs17
+ncurses
+ncurses-base
+ncurses-libs
+nettle
+net-tools
+newt
+newt-python
+nfs-utils
+nspr
+nss
+nss-softokn
+nss-softokn-freebl
+nss-sysinit
+nss-tools
+nss-util
+numactl-libs
+openssh
+openssh-clients
+openssh-server
+openssl
+openssl-libs
+os-prober
+p11-kit
+p11-kit-trust
+pam
+parted
+passwd
+pciutils
+pciutils-libs
+pcre
+pinentry
+pkgconfig
+policycoreutils
+policycoreutils-python
+polkit
+polkit-pkla-compat
+popt
+postfix
+ppp
+procps-ng
+pth
+pygobject3-base
+pygpgme
+pyliblzma
+python
+python-backports
+python-backports-ssl_match_hostname
+python-chardet
+python-cheetah
+python-configobj
+python-decorator
+python-iniparse
+python-IPy
+python-jsonpatch
+python-jsonpointer
+python-kitchen
+python-libs
+python-markdown
+python-perf
+python-pillow
+python-prettytable
+python-pycurl
+python-pygments
+python-pyudev
+python-requests
+python2-six
+python-urlgrabber
+python-urllib3
+pyxattr
+PyYAML
+qrencode-libs
+quota
+quota-nls
+rdma
+readline
+rootfiles
+rpcbind
+rpm
+rpm-build-libs
+rpm-libs
+rpm-python
+rsync
+rsyslog
+sed
+setup
+shadow-utils
+shared-mime-info
+slang
+snappy
+sqlite
+sudo
+systemd
+systemd-libs
+systemd-sysv
+sysvinit-tools
+tar
+tcpdump
+tcp_wrappers
+tcp_wrappers-libs
+teamd
+trousers
+tzdata
+ustr
+util-linux
+vim-enhanced
+virt-what
+wget
+which
+wpa_supplicant
+xz
+xz-libs
+yum
+yum-metadata-parser
+yum-plugin-fastestmirror
+yum-utils
+zlib

build-tools/build_guest/rpm-remove-list.txt

@@ -0,0 +1,7 @@
+# list of packages to be excluded from guest image
+cpp
+gcc
+gcc-c++
+gdb
+linux-firmware
+rpm-build

build-tools/build_iso/anaconda-ks.cfg

@@ -0,0 +1,40 @@
+#version=DEVEL
+# System authorization information
+auth --enableshadow --passalgo=sha512
+# Use CDROM installation media
+cdrom
+# Use graphical install
+graphical
+# Run the Setup Agent on first boot
+firstboot --enable
+ignoredisk --only-use=sda
+# Keyboard layouts
+keyboard --vckeymap=us --xlayouts='us'
+# System language
+lang en_US.UTF-8
+
+# Network information
+network  --bootproto=dhcp --device=enp0s3 --onboot=off --ipv6=auto
+network  --bootproto=static --device=enp0s8 --ip=10.10.10.10 --netmask=255.255.255.0 --ipv6=auto --activate
+network --device=lo  --hostname=localhost.localdomain
+
+#Root password
+rootpw --lock
+# System timezone
+timezone America/New_York --isUtc
+user --groups=wheel --name=wrsroot --password=$6$c3gaCcJlh.rp//Yx$/mIjNNoUDS1qZldBL29YSJdsA9ttPA/nXN1CPsIcCmionXC22APT3IoRSd9j5dPiZoviDdQf7YxLsOYdieOQr/ --iscrypted --gecos="wrsroot"
+# System bootloader configuration
+bootloader --location=mbr --boot-drive=sda
+autopart --type=lvm
+# Partition clearing information
+clearpart --all --initlabel --drives=sda
+
+%packages
+@^minimal
+@core
+
+%end
+
+%addon com_redhat_kdump --disable --reserve-mb='auto'
+
+%end

build-tools/build_iso/cgts_deps.sh

@@ -0,0 +1,296 @@
+#!/bin/env bash
+
+# Here's the score, kids.  There are a few different places from which we can
+# get packages.  In priority order, they are:
+#
+# The CGTS packages we've built ourselves
+# The CGTS packages that Jenkins has built (coming soon to a script near you)
+# The CentOS packages in various repos
+#    - Base OS
+#    - OpenStack Repos
+# EPEL (Extra Packages for Enterprise Linux)
+#
+# This script can function in two ways:
+#   If you specify a filename, it assumes the file is a list of packages you
+#      want to install, or dependencies you want to meet.  It installs whatever
+#      is in the list into current directory.  Failure to find a dependency
+#      results in a return code of 1
+#
+#   If no file is specified, we generate a file ($DEPLISTFILE) of dependencies
+#      based on current directory
+#
+# We then continuously loop through generating new dependencies and installing
+#  them until either all dependencies are met, or we cannot install anymore
+#
+# We also log where dependencies were installed from into
+#   export/dist/report_deps.txt
+#
+
+# This function generates a simple file of dependencies we're trying to resolve
+function generate_dep_list {
+   TMP_RPM_DB=$(mktemp -d $(pwd)/tmp_rpm_db_XXXXXX)
+   mkdir -p $TMP_RPM_DB
+   rpm --initdb --dbpath $TMP_RPM_DB
+   rpm --dbpath $TMP_RPM_DB --test -Uvh --replacefiles '*.rpm' >> $DEPDETAILLISTFILE 2>&1
+   rpm --dbpath $TMP_RPM_DB --test -Uvh --replacefiles '*.rpm' 2>&1 \
+      | grep -v "error:" \
+      | grep -v "warning:" \
+      | grep -v "Preparing..." \
+      | sed "s/ is needed by.*$//" | sed "s/ >=.*$//" | sort -u > $DEPLISTFILE
+   \rm -rf $TMP_RPM_DB
+}
+
+# Takes a list of requirements (either explcit package name, or capabilities
+# to provide) and install packages to meet those dependancies
+#
+# We take the list of requirements and first try to look them up based on
+# package name.  If we can't find a package with the name of the requirement,
+# we use --whatprovides to complete the lookup.
+#
+# The reason for this initial name-based attempt is that a couple of funky
+# packages (notably -devel packages) have "Provides:" capabilities which
+# conflict with named packages.  So if explictly say we want "xyz" then we'll
+# install the "xyz" package, rather than "something-devel" which has "xyz"
+# capabilities.
+function install_deps {
+	local DEP_LIST=""
+	local DEP_LIST_FILE="$1"
+
+	# Temporary files are used in a few different ways
+	# Here we essenitally create variable aliases to make it easier to read
+	# the script
+	local UNSORTED_PACKAGES=$TMPFILE
+	local SORTED_PACKAGES=$TMPFILE1
+	local UNRESOLVED_PACKAGES=$TMPFILE2
+
+	rm -f $UNSORTED_PACKAGES
+
+	while read DEP
+	do
+		DEP_LIST="${DEP_LIST} ${DEP}"
+	done < $DEP_LIST_FILE
+
+	echo "Debug: List of deps to resolve: ${DEP_LIST}"
+
+	if [ -z "${DEP_LIST}" ]
+	then
+		return 0
+	fi
+
+	# go through each repo and convert deps to packages based on package name
+	for REPOID in `grep  '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
+		echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch ${DEP_LIST} --qf='%{name}'"
+		TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --qf='%{name}' ${DEP_LIST} | sed "s/kernel-debug/kernel/g" >> $UNSORTED_PACKAGES
+                \rm -rf $TMP_DIR/yum-$USER-*
+	done
+	sort $UNSORTED_PACKAGES -u > $SORTED_PACKAGES
+
+	# figure out any dependancies which could not be resolved based on
+	# package name.  We use --whatpovides to deal with this
+	#
+	# First, we build a new DEP_LIST based on what was NOT found in
+	# search-by-name attempt
+	sort $DEP_LIST_FILE -u > $TMPFILE
+	comm -2 -3 $TMPFILE $SORTED_PACKAGES > $UNRESOLVED_PACKAGES
+
+	# If there are any requirements not resolved, look up the packages with
+	# --whatprovides
+	if [ -s $UNRESOLVED_PACKAGES ]; then
+		DEP_LIST=""
+		\cp $SORTED_PACKAGES $UNSORTED_PACKAGES
+		while read DEP
+		do
+			DEP_LIST="${DEP_LIST} ${DEP}"
+		done < $UNRESOLVED_PACKAGES
+
+                DEP_LIST=$(echo "$DEP_LIST" | sed 's/^ //g')
+                if [ "$DEP_LIST" != "" ]; then
+
+		    for REPOID in `grep  '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
+			echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --whatprovides ${DEP_LIST} --qf='%{name}'"
+			TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --qf='%{name}' --whatprovides ${DEP_LIST} | sed "s/kernel-debug/kernel/g" >> $UNSORTED_PACKAGES
+                        \rm -rf $TMP_DIR/yum-$USER-*
+		    done
+                fi
+
+		sort -u $UNSORTED_PACKAGES > $SORTED_PACKAGES
+	fi
+
+	# clean up
+	\rm -f $UNSORTED_PACKAGES $UNRESOLVED_PACKAGES
+
+	# We now have, in SORTED_PACKAGES, a list of all packages that we need to install
+	# to meet our dependancies
+	DEP_LIST=" "
+	while read DEP
+	do
+		DEP_LIST="${DEP_LIST}${DEP} "
+	done < $SORTED_PACKAGES
+	rm $SORTED_PACKAGES
+
+	# go through each repo and install packages
+	local TARGETS=${DEP_LIST}
+	echo "Debug: Resolved list of deps to install: ${TARGETS}"
+	local UNRESOLVED
+	for REPOID in `grep  '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
+		UNRESOLVED="$TARGETS"
+           
+		if [[ ! -z "${TARGETS// }" ]]; then
+			REPO_PATH=$(cat $YUM | sed -n "/^\[$REPOID\]\$/,\$p" | grep '^baseurl=' | head -n 1 | awk -F 'file://' '{print $2}' | sed 's:/$::')
+			>&2  echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf='%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}'"
+			TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf="%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}" | sort -r -V > $TMPFILE
+                        \rm -rf $TMP_DIR/yum-$USER-*
+
+			while read STR
+			do
+				>&2 echo "STR=$STR"
+				if [ "x$STR" == "x" ]; then
+					continue
+				fi
+
+				PKG=`echo $STR | cut -d " " -f 1`
+				PKG_FILE=`echo $STR | cut -d " " -f 2`
+				PKG_REL_PATH=`echo $STR | cut -d " " -f 3`
+				PKG_PATH="${REPO_PATH}/${PKG_REL_PATH}"
+
+				>&2 echo "Installing PKG=$PKG PKG_FILE=$PKG_FILE PKG_REL_PATH=$PKG_REL_PATH PKG_PATH=$PKG_PATH from repo $REPOID"
+				cp $PKG_PATH .
+				if [ $? -ne 0 ]
+				then
+					>&2 echo "  Here's what I have to work with..."
+					>&2 echo "  TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $PKG --qf=\"%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}\""
+					>&2 echo "  PKG=$PKG PKG_FILE=$PKG_FILE REPO_PATH=$REPO_PATH PKG_REL_PATH=$PKG_REL_PATH PKG_PATH=$PKG_PATH"
+				fi
+
+				echo $UNRESOLVED | grep $PKG >> /dev/null
+				if [ $? -eq 0 ]; then
+					echo "$PKG found in $REPOID as $PKG" >> $BUILT_REPORT
+					echo "$PKG_PATH" >> $BUILT_REPORT
+					UNRESOLVED=$(echo "$UNRESOLVED" | sed "s# $PKG # #g")
+				else
+					echo "$PKG satisfies unknown target in $REPOID" >> $BUILT_REPORT
+					echo "  but it doesn't match targets, $UNRESOLVED" >> $BUILT_REPORT
+					echo "  path $PKG_PATH" >> $BUILT_REPORT
+					FOUND_UNKNOWN=1
+				fi
+			done < $TMPFILE #<<< "$(TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf=\"%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}\" | sort -r -V)"
+                        \rm -rf $TMP_DIR/yum-$USER-*
+			TARGETS="$UNRESOLVED"
+		fi
+	done
+	>&2 echo "Debug: Packages still unresolved: $UNRESOLVED"
+	echo "Debug: Packages still unresolved: $UNRESOLVED" >> $WARNINGS_REPORT
+	echo "Debug: Packages still unresolved: $UNRESOLVED" >> $BUILT_REPORT
+	>&2 echo ""
+}
+
+function check_all_explicit_deps_installed
+{
+
+	PKGS_TO_CHECK=" "
+	while read PKG_TO_ADD
+	do
+		PKGS_TO_CHECK="$PKGS_TO_CHECK ${PKG_TO_ADD}"
+	done < $DEPLISTFILE
+	rpm -qp $MY_WORKSPACE/export/dist/isolinux/Packages/*.rpm --qf="%{name}\n" --nosignature > $TMPFILE
+
+	while read INSTALLED_PACKAGE
+	do
+		echo $PKGS_TO_CHECK | grep -q "${INSTALLED_PACKAGE}"
+		if [ $? -eq 0 ]; then
+			PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/^${INSTALLED_PACKAGE} //"`
+			PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/ ${INSTALLED_PACKAGE} / /"`
+			PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/ ${INSTALLED_PACKAGE}\$//"`
+			PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/^${INSTALLED_PACKAGE}\$//"`
+		fi
+	done < $TMPFILE
+
+	if [ -z "$PKGS_TO_CHECK" ]
+	then
+		>&2 echo "All explicitly specified packages resolved!"
+	else
+		>&2 echo "Could not resolve packages: $PKGS_TO_CHECK"
+		return 1
+	fi
+	return 0
+}
+
+ATTEMPTED=0
+DISCOVERED=0
+OUTPUT_DIR=$MY_WORKSPACE/export
+TMP_DIR=$MY_WORKSPACE/tmp
+YUM=$OUTPUT_DIR/yum.conf
+DEPLISTFILE=$OUTPUT_DIR/deps.txt
+DEPDETAILLISTFILE=$OUTPUT_DIR/deps_detail.txt
+
+BUILT_REPORT=$OUTPUT_DIR/local.txt
+WARNINGS_REPORT=$OUTPUT_DIR/warnings.txt
+LAST_TEST=$OUTPUT_DIR/last_test.txt
+TMPFILE=$OUTPUT_DIR/cgts_deps_tmp.txt
+TMPFILE1=$OUTPUT_DIR/cgts_deps_tmp1.txt
+TMPFILE2=$OUTPUT_DIR/cgts_deps_tmp2.txt
+
+touch "$BUILT_REPORT"
+touch "$WARNINGS_REPORT"
+
+for i in "$@"
+do
+case $i in
+    -d=*|--deps=*)
+    DEPS="${i#*=}"
+    shift # past argument=value
+    ;;
+esac
+done
+
+mkdir -p $TMP_DIR
+
+rm -f "$DEPDETAILLISTFILE"
+# FIRST PASS we are being given a list of REQUIRED dependencies
+if [ "${DEPS}x" != "x" ]; then
+    cat $DEPS | grep -v "^#" | sed '/^\s*$/d' > $DEPLISTFILE
+    install_deps $DEPLISTFILE
+    if [ $? -ne 0 ]; then
+        exit 1
+    fi
+fi
+
+# check that we resolved them all
+check_all_explicit_deps_installed
+if [ $? -ne 0 ]; then
+	>&2 echo "Error -- could not install all explicitly listed packages"
+	exit 1
+fi
+
+ALL_RESOLVED=0
+
+while [ $ALL_RESOLVED -eq 0 ]; do
+	cp $DEPLISTFILE $DEPLISTFILE.old
+	generate_dep_list
+	if [ ! -s $DEPLISTFILE ]; then
+		# no more dependencies!
+		ALL_RESOLVED=1
+	else
+		DIFFLINES=`diff $DEPLISTFILE.old $DEPLISTFILE | wc -l`
+		if [ $DIFFLINES -eq 0 ]; then
+			>&2 echo "Warning: Infinite loop detected in dependency resolution.  See $DEPLISTFILE for details -- exiting"
+			>&2 echo "These RPMS had problems (likely version conflicts)"
+			>&2 cat  $DEPLISTFILE
+
+			echo "Warning: Infinite loop detected in dependency resolution See $DEPLISTFILE for details -- exiting" >> $WARNINGS_REPORT
+			echo "These RPMS had problems (likely version conflicts)" >> $WARNINGS_REPORT
+			cat  $DEPLISTFILE >> $WARNINGS_REPORT
+
+			date > $LAST_TEST
+
+			rm -f $DEPLISTFILE.old
+			exit 1 # nothing fixed
+		fi
+		install_deps $DEPLISTFILE
+		if [ $? -ne 0 ]; then
+			exit 1
+		fi
+	fi
+done
+
+exit 0

build-tools/build_iso/gather_packages.pl

@@ -0,0 +1,122 @@
+#!/usr/bin/perl
+
+# Copy/pasted from http://www.smorgasbork.com/content/gather_packages.txt
+# As referenced by http://www.smorgasbork.com/2012/01/04/building-a-custom-centos-7-kickstart-disc-part-2/
+
+use XML::Simple;
+
+my ($comps_file, $rpm_src_path, $rpm_dst_path, $arch, @extra_groups_and_packages) = @ARGV;
+
+if (!-e $comps_file)
+{
+    print_usage ("Can't find '$comps_file'");
+}
+if (!-e $rpm_src_path)
+{
+    print_usage ("RPM source path '$rpm_src_path' does not exist");
+}
+if (!-e $rpm_dst_path)
+{
+    print_usage ("RPM destination path '$rpm_dst_path' does not exist");
+}
+if (!$arch)
+{
+    print_usage ("Architecture not specified");
+}
+
+#### we always gather core and base; note that for CentOS 7, we also need
+#### to include the grub2 package, or installation will fail
+@desired_groups = ('core', 'base', 'grub2');
+foreach (@extra_groups_and_packages)
+{
+    push (@desired_groups, $_);
+}
+
+$regex = '^(' . join ('|', @desired_groups) . ')$';
+
+print "reading $comps_file...\n";
+print "getting RPMs from $rpm_src_path...\n";
+
+$xml = new XML::Simple;
+$comps = $xml->XMLin($comps_file);
+
+$cmd = "rm $rpm_dst_path/*";
+print "$cmd\n";
+`$cmd`;
+
+%copied_groups = {};
+%copied_packages = {};
+
+foreach $group (@{$comps->{group}})
+{
+    $id = $group->{id};
+    if ($id !~ m#$regex#)
+    {
+        next;
+    }
+
+    print "#### group \@$id\n";
+    $packagelist = $group->{packagelist};
+    foreach $pr (@{$packagelist->{packagereq}})
+    {
+        if ($pr->{type} eq 'optional' || $pr->{type} eq 'conditional')
+        {
+            next;
+        }
+
+        $cmd = "cp $rpm_src_path/" . $pr->{content} . "-[0-9]*.$arch.rpm"
+                . " $rpm_src_path/" . $pr->{content} . "-[0-9]*.noarch.rpm $rpm_dst_path";
+        print "$cmd\n";
+        `$cmd 2>&1`;
+
+        $copied_packages{$pr->{content}} = 1;
+    }
+
+    $copied_groups{$group} = 1;
+}
+
+#### assume that any strings that weren't matched in the comps file's group list
+#### are actually packages
+
+foreach $group (@desired_groups)
+{
+    if ($copied_groups{$group})
+    {
+        next;
+    }
+
+    $cmd = "cp $rpm_src_path/" . $group . "-[0-9]*.$arch.rpm"
+            . " $rpm_src_path/" . $group . "-[0-9]*.noarch.rpm $rpm_dst_path";
+    print "$cmd\n";
+    `$cmd 2>&1`;
+}
+
+sub print_usage
+{
+    my ($msg) = @_;
+
+    ($msg) && print "$msg\n\n";
+
+    print <<__TEXT__;
+
+parse_comps.pl comps_file rpm_src_path arch [xtra_grps_and_pkgs]
+
+    comps_file           the full path to the comps.xml file (as provided 
+                         in the original distro
+
+    rpm_src_path         the full path to the directory of all RPMs from 
+                         the distro
+
+    rpm_dst_path         the full path to the directory where you want
+                         to save the RPMs for your kickstart
+
+    arch                 the target system architecture (e.g. x86_64)
+
+    xtra_grps_and_pkgs   a list of extra groups and packages, separated by spaces
+
+
+__TEXT__
+
+    exit;
+}
+

build-tools/build_iso/image-dev.inc

@@ -0,0 +1,4 @@
+# The following packages will not be included in the customer ISO
+#
+# They are exceptional packages only to be included in developer builds
+enable-dev-patch

build-tools/build_iso/image.inc

@@ -0,0 +1,364 @@
+# List of CGTS packages to be included/installed in ISO
+# If these have dependencies, they will be pulled in automatically
+#
+acpid
+audit
+build-info
+bash
+ceph-manager
+cgcs-patch-agent
+cgcs-patch-controller
+cgts-mtce-compute
+cgts-mtce-control
+cgts-mtce-storage
+computeconfig
+computeconfig-standalone
+computeconfig-subfunction
+compute-huge
+dnsmasq
+dnsmasq-utils
+dpkg
+filesystem-scripts
+fm-api
+gdb
+grub2
+grub2-efi
+grub2-efi-modules
+grub2-tools
+haproxy
+io-monitor
+iperf3
+isomd5sum
+iscsi-initiator-utils
+iscsi-initiator-utils-iscsiuio
+kmod-drbd
+libevent
+libtrap-handler
+libvirt-docs
+libvirt-python
+logrotate
+lshell
+net-snmp-libs
+net-snmp-python
+net-snmp-utils
+nfv
+nfv-common
+nfv-plugins
+nfv-tools
+nfv-vim
+nfv-client
+nova-utils
+ntpdate
+openldap
+openldap-clients
+openldap-servers
+openssh
+openssh-clients
+openssh-server
+openstack-ras
+pam
+parted
+platform-util
+platform-util-noncontroller
+python-cephclient
+python-keyring
+qemu-kvm-tools-ev
+rsync
+sm-client
+sm-tools
+snmp-ext
+storageconfig
+sysinv
+sysinv-agent
+tsconfig
+tboot
+rabbitmq-server
+cgts-client
+python-django-horizon
+libvirt
+libvirt-client
+libvirt-daemon
+libvirt-daemon-config-network
+libvirt-daemon-config-nwfilter
+libvirt-daemon-driver-lxc
+libvirt-daemon-driver-network
+libvirt-daemon-driver-nodedev
+libvirt-daemon-driver-nwfilter
+libvirt-daemon-driver-qemu
+libvirt-daemon-driver-secret
+libvirt-daemon-driver-storage
+nova-api-proxy
+configutilities
+controllerconfig
+fm-common
+fm-mgr
+logmgmt
+cgts-mtce-common
+cgts-mtce-common-pmon
+cgts-mtce-common-rmon
+cgts-mtce-common-hostw
+cgts-mtce-common-hwmon
+cgts-mtce-common-guestAgent
+cgts-mtce-common-guestServer
+cgcs-patch
+patch-alarm
+# patch-boot-args is not ported to centos yet
+wrs-ssl
+tss2
+tpm2-tools
+tpm2-openssl-engine
+libtpms
+swtpm
+swtpm-tools
+swtpm-cuse
+monitor-tools
+python2-aodhclient
+openstack-aodh-api
+openstack-aodh-evaluator
+openstack-aodh-expirer
+openstack-aodh-listener
+openstack-aodh-notifier
+python-ceilometer
+#openstack-ceilometer-alarm provided in openstack-aodh-compat package
+openstack-aodh-compat
+openstack-ceilometer-api
+openstack-ceilometer-collector
+openstack-ceilometer-common
+openstack-ceilometer-ipmi
+openstack-ceilometer-notification
+openstack-ceilometer-polling
+openstack-cinder
+openstack-glance
+openstack-heat-api
+openstack-heat-api-cfn
+openstack-heat-api-cloudwatch
+openstack-heat-common
+openstack-heat-engine
+openstack-ironic-api
+openstack-ironic-common
+openstack-ironic-conductor
+python2-ironicclient
+# doc generation turned off in Mitaka by default
+#python-django-horizon-doc
+openstack-keystone
+openstack-murano-api
+openstack-murano-cf-api
+openstack-murano-common
+openstack-murano-doc
+openstack-murano-engine
+openstack-murano-ui
+openstack-murano-ui-doc
+python2-muranoclient
+python-muranoclient-doc
+openstack-magnum-api
+openstack-magnum-common
+openstack-magnum-conductor
+openstack-magnum-doc
+openstack-magnum-ui
+python2-magnumclient
+python-magnum
+python-magnumclient-doc
+python-magnumclient-tests
+python-magnum-tests
+openstack-neutron
+openstack-neutron-common
+openstack-neutron-sriov-nic-agent
+openstack-neutron-ml2
+openstack-neutron-openvswitch
+python-neutron
+openstack-nova-api
+openstack-nova-cells
+openstack-nova-cert
+openstack-nova-common
+openstack-nova-compute
+openstack-nova-conductor
+openstack-nova-console
+openstack-nova-network
+openstack-nova-novncproxy
+# openstack-nova-objectstore removed in Mitaka
+#openstack-nova-objectstore
+openstack-nova-scheduler
+openstack-nova-serialproxy
+openstack-nova-spicehtml5proxy
+openstack-nova-placement-api
+python-nova
+python2-novaclient
+python2-openstackclient
+python2-oslo-log
+python2-django-openstack-auth
+python2-six
+#base-files is not yet ported to CENTOS
+cgcs-users
+namespace-utils
+nss-pam-ldapd
+# thin-provisioning-tools is not yet ported to CENTOS
+# cluster-resource-agents is not yet ported to CENTOS
+dhcp
+nfs-utils
+initscripts
+# note -- the "systemd" package provides udev
+systemd
+# udev-extraconf  is not yet ported to CENTOS
+# cdrkit is only available as a src.rpm
+python-d2to1
+# eucatools  is not yet ported to CENTOS
+facter
+hiera
+# libaugesas-ruby is not yet ported to CENTOS
+nfscheck
+python2-pecan
+# pseudo not available
+# ps-byte-compile is not yet ported to CENTOS
+python
+python-configobj
+python-pep8
+# python-pyflakes is not yet ported to CENTOS
+python2-rsa
+# python-xmlrpclib is not yet ported to CENTOS
+ruby-shadow
+swig
+syslinux
+iotop
+iptables
+iptables-services
+iptables-utils
+ldapscripts
+lighttpd
+lighttpd-fastcgi
+lighttpd-mod_geoip
+lighttpd-mod_mysql_vhost
+novnc
+ntp
+ntp-perl
+procps-ng
+python-daemon
+python2-gunicorn
+python-pyudev
+python-smartpm
+# shadow is not yet ported to CENTOS
+shadow-utils
+syslog-ng
+syslog-ng-libdbi
+drbd
+drbd-bash-completion
+# drbd-debuginfo
+drbd-heartbeat
+# drbd-kernel-debuginfo
+drbd-pacemaker
+drbd-udev
+drbd-utils
+# drbd-tools is not yet ported to CENTOS
+# nfv-docs  should not be included
+# nfv-tests should not be included
+curl
+lvm2
+# tgt is ported, but not yet added here
+sm
+sm-common
+sm-common-libs
+sm-db
+sm-api
+sm-eru
+time
+puppet
+puppet-manifests
+openstack-dashboard
+# openstack-dashboard-theme TODO: fix build-iso
+dhclient
+postgresql
+postgresql-server
+postgresql-contrib
+python-psycopg2
+setup
+targetcli
+sudo
+pxe-network-installer
+strace
+resource-agents
+lldpd
+wget
+bind-utils
+selinux-policy
+pm-utils
+centos-release
+tcpdump
+config-gate
+config-gate-compute
+sysstat
+smartmontools
+collector
+io-scheduler
+fm-doc
+vm-topology
+python2-wsme
+ceph
+ceph-common
+ceph-fuse
+ceph-radosgw
+libcephfs1
+python-ceph-compat
+python-cephfs
+socat
+irqbalance
+kmod-e1000e
+kmod-i40e
+kmod-ixgbe
+kmod-ixgbevf
+kmod-tpm
+kmod-integrity
+vim-enhanced
+wrs-heat-templates
+# heat-contrib-nova-flavor is now included with openstack-heat
+tis-extensions
+tis-extensions-controller
+qat17
+mlx4-config
+host-guest-comm
+guest-scale-helper
+python-networking-odl
+qemu-kvm-ev
+qemu-img-ev
+# for realtime kernel
+kernel-rt
+kernel-rt-kvm
+kmod-e1000e-rt
+kmod-i40e-rt
+kmod-ixgbe-rt
+kmod-tpm-rt
+kmod-integrity-rt
+rtctl
+rt-setup
+qat17-rt
+kernel-rt-tools
+# For low-latency compute
+kmod-drbd-rt
+snmp-audittrail
+OVMF
+# neutron bgp
+python2-networking-bgpvpn
+python-networking-bgpvpn-dashboard
+python-networking-bgpvpn-heat
+python2-neutron-dynamic-routing
+python2-ryu
+python-ryu-common
+openvswitch
+python2-networking-sfc
+openstack-panko-api
+openstack-panko-common
+python-panko
+python2-pankoclient
+# distributed cloud
+distributedcloud-dcmanager
+distributedcloud-client-dcmanagerclient
+distributedcloud-dcorch
+# ima plugin for RPM
+rpm-plugin-systemd-inhibit
+platform-kickstarts
+python-lefthandclient
+python-3parclient
+
+# Add debugging tools
+perf
+zip
+unzip
+traceroute

build-tools/build_iso/isolinux.cfg

@@ -0,0 +1,125 @@
+default vesamenu.c32
+timeout 600
+
+display boot.msg
+
+# Clear the screen when exiting the menu, instead of leaving the menu displayed.
+# For vesamenu, this means the graphical background is still displayed without
+# the menu itself for as long as the screen remains in graphics mode.
+menu clear
+menu background splash.png
+menu title CentOS 7
+menu vshift 8
+menu rows 18
+menu margin 8
+#menu hidden
+menu helpmsgrow 15
+menu tabmsgrow 13
+
+# Border Area
+menu color border * #00000000 #00000000 none
+
+# Selected item
+menu color sel 0 #ffffffff #00000000 none
+
+# Title bar
+menu color title 0 #ff7ba3d0 #00000000 none
+
+# Press [Tab] message
+menu color tabmsg 0 #ff3a6496 #00000000 none
+
+# Unselected menu item
+menu color unsel 0 #84b8ffff #00000000 none
+
+# Selected hotkey
+menu color hotsel 0 #84b8ffff #00000000 none
+
+# Unselected hotkey
+menu color hotkey 0 #ffffffff #00000000 none
+
+# Help text
+menu color help 0 #ffffffff #00000000 none
+
+# A scrollbar of some type? Not sure.
+menu color scrollbar 0 #ffffffff #ff355594 none
+
+# Timeout msg
+menu color timeout 0 #ffffffff #00000000 none
+menu color timeout_msg 0 #ffffffff #00000000 none
+
+# Command prompt text
+menu color cmdmark 0 #84b8ffff #00000000 none
+menu color cmdline 0 #ffffffff #00000000 none
+
+# Do not display the actual menu unless the user presses a key. All that is displayed is a timeout message.
+
+menu tabmsg Press Tab for full configuration options on menu items.
+
+menu separator # insert an empty line
+menu separator # insert an empty line
+
+label tis
+  menu label ^Install Titanium Cloud
+  menu default
+  kernel vmlinuz
+  append initrd=initrd.img inst.ks=cdrom:/dev/cdrom:/ks/ks.cfg
+
+label linux
+  menu label ^Install CentOS 7
+  kernel vmlinuz
+  append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 quiet
+
+label check
+  menu label Test this ^media & install CentOS 7
+  kernel vmlinuz
+  append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rd.live.check quiet
+
+menu separator # insert an empty line
+
+# utilities submenu
+menu begin ^Troubleshooting
+  menu title Troubleshooting
+
+label vesa
+  menu indent count 5
+  menu label Install CentOS 7 in ^basic graphics mode
+  text help
+	Try this option out if you're having trouble installing
+	CentOS 7.
+  endtext
+  kernel vmlinuz
+  append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 xdriver=vesa nomodeset quiet
+
+label rescue
+  menu indent count 5
+  menu label ^Rescue a CentOS system
+  text help
+	If the system will not boot, this lets you access files
+	and edit config files to try to get it booting again.
+  endtext
+  kernel vmlinuz
+  append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rescue quiet
+
+label memtest
+  menu label Run a ^memory test
+  text help
+	If your system is having issues, a problem with your
+	system's memory may be the cause. Use this utility to
+	see if the memory is working correctly.
+  endtext
+  kernel memtest
+
+menu separator # insert an empty line
+
+label local
+  menu label Boot from ^local drive
+  localboot 0xffff
+
+menu separator # insert an empty line
+menu separator # insert an empty line
+
+label returntomain
+  menu label Return to ^main menu
+  menu exit
+
+menu end

build-tools/build_iso/ks.cfg

@@ -0,0 +1,36 @@
+install
+text
+lang en_US.UTF-8
+keyboard us
+reboot --eject
+firstboot --enable
+auth --enableshadow --passalgo=sha512
+
+# Network information
+network  --bootproto=dhcp --device=enp0s3 --onboot=on --ipv6=auto --activate
+network  --bootproto=static --device=enp0s8 --ip=10.10.10.12 --netmask=255.255.255.0 --ipv6=auto --activate
+network --device=lo  --hostname=localhost.localdomain
+
+rootpw --lock
+timezone America/New_York --isUtc
+user --groups=wheel --name=wrsroot --password=$6$c3gaCcJlh.rp//Yx$/mIjNNoUDS1qZldBL29YSJdsA9ttPA/nXN1CPsIcCmionXC22APT3IoRSd9j5dPiZoviDdQf7YxLsOYdieOQr/ --iscrypted --gecos="wrsroot"
+
+# System bootloader configuration
+#bootloader --location=mbr --boot-drive=sda
+
+autopart --type=lvm
+# Partition clearing information
+clearpart --all --initlabel --drives=sda
+
+cdrom
+#repo --name=base --baseurl=http://mirror.cogentco.com/pub/linux/centos/7/os/x86_64/
+#url --url="http://mirror.cogentco.com/pub/linux/centos/7/os/x86_64/"
+
+%packages --nobase --ignoremissing
+@^minimal
+@core
+kexec-tools
+net-tools
+# CGTS packages
+# end CGTS packages
+%end

build-tools/build_iso/minimal_rpm_list.txt

@@ -0,0 +1,258 @@
+acl
+alsa-lib
+audit
+audit-libs
+authconfig
+basesystem
+bind-libs-lite
+bind-license
+binutils
+biosdevname
+btrfs-progs
+bzip2-libs
+ca-certificates
+centos-logos
+chkconfig
+coreutils
+cpio
+cracklib
+cracklib-dicts
+cronie
+cronie-anacron
+crontabs
+cryptsetup
+cryptsetup-libs
+curl
+cyrus-sasl-lib
+dbus
+dbus-glib
+dbus-libs
+dbus-python
+device-mapper
+device-mapper-event
+device-mapper-event-libs
+device-mapper-libs
+device-mapper-multipath
+device-mapper-multipath-libs
+device-mapper-persistent-data
+diffutils
+dmidecode
+dosfstools
+dracut
+dracut-config-rescue
+dracut-network
+e2fsprogs
+e2fsprogs-libs
+ebtables
+efibootmgr
+efivar-libs
+elfutils-libelf
+elfutils-libs
+ethtool
+expat
+file
+file-libs
+filesystem
+findutils
+fipscheck
+fipscheck-lib
+firewalld
+freetype
+gawk
+gdbm
+gettext
+gettext-libs
+glib2
+glibc
+glibc-common
+glib-networking
+gmp
+gnupg2
+gnutls
+gobject-introspection
+gpgme
+grep
+groff-base
+grub2
+grub2-efi
+grub2-tools
+grubby
+gsettings-desktop-schemas
+gzip
+hardlink
+hostname
+hwdata
+info
+iproute
+iprutils
+iputils
+jansson
+json-c
+kbd
+kbd-legacy
+kbd-misc
+kernel-tools
+kernel-tools-libs
+kexec-tools
+keyutils-libs
+kmod
+kmod-libs
+kpartx
+krb5-libs
+less
+libacl
+libaio
+libassuan
+libattr
+libblkid
+libcap
+libcap-ng
+libcom_err
+libconfig
+libcroco
+libcurl
+libdaemon
+libdb
+libdb-utils
+libdrm
+libedit
+libestr
+libffi
+libgcc
+libgcrypt
+libgomp
+libgpg-error
+libgudev1
+libidn
+libmnl
+libmodman
+libmount
+libndp
+libnetfilter_conntrack
+libnfnetlink
+libnl
+libnl3
+libnl3-cli
+libpcap
+libpciaccess
+libpipeline
+libproxy
+libpwquality
+libreport-filesystem
+libselinux
+libselinux-python
+libselinux-utils
+libsemanage
+libsepol
+libss
+libssh2
+libstdc++
+libsysfs
+libtasn1
+libteam
+libunistring
+libuser
+libutempter
+libuuid
+libverto
+libxml2
+libxslt
+linux-firmware
+lldpad
+lsscsi
+lua
+lvm2
+lvm2-libs
+lzo
+make
+man-db
+mariadb-libs
+mdadm
+microcode_ctl
+mokutil
+mozjs17
+ncurses
+ncurses-base
+ncurses-libs
+nettle
+newt
+newt-python
+nspr
+nss
+nss-softokn
+nss-softokn-freebl
+nss-sysinit
+nss-tools
+nss-util
+numactl-libs
+openscap
+openscap-scanner
+openssl
+openssl-libs
+os-prober
+p11-kit
+p11-kit-trust
+passwd
+pciutils-libs
+pcre
+pinentry
+pkgconfig
+policycoreutils
+popt
+procps-ng
+pth
+pygobject3-base
+pygpgme
+pyliblzma
+python
+python-backports
+python-backports-ssl_match_hostname
+python-configobj
+python-decorator
+python-iniparse
+python-libs
+python-perf
+python-pycurl
+python-pyudev
+python2-setuptools
+python-slip
+python-slip-dbus
+python-urlgrabber
+pyxattr
+qrencode-libs
+rdma
+readline
+rootfiles
+rpm
+rpm-build-libs
+rpm-libs
+rpm-python
+sed
+shared-mime-info
+shim
+shim-unsigned
+slang
+snappy
+sqlite
+systemd
+systemd-libs
+systemd-sysv
+sysvinit-tools
+tar
+tcp_wrappers-libs
+teamd
+time
+trousers
+tzdata
+ustr
+util-linux
+virt-what
+which
+xfsprogs
+xml-common
+xz
+xz-libs
+zlib
+lksctp-tools
+boost-thread
+boost-system

build-tools/build_iso/openstack_kilo.txt

@@ -0,0 +1,2 @@
+# Files copied in from /import/mirrors/CentOS/7.2.1511/cloud/x86_64/openstack-kilo
+

build-tools/build_minimal_iso/README

@@ -0,0 +1,112 @@
+This document describes how to generate a DVD image (.iso) which installs
+a minimal CentOS installation where the entirety of the installed system is
+build from the provided source.
+
+There are three parts to this document:
+  How to build binary RPMs from source RPMS
+  How to build the install disk from the binary RPMS
+  How to install the minimal system
+
+-------------------------------------------------------------------------------
+How to build the binary RPMs from the source RPMS
+-------------------------------------------------------------------------------
+
+(note - building the binary RPMs is expected to take a long time, ~ 15 hours
+on a typical system)
+
+The source RPMs in the "srcs" subdirectory are compiled in an environment
+called "mock" which builds each package in a chroot jail to ensure the output
+is not influenced by the build system.  Mock is controlled by a config file.
+The example srcs/build.cfg is provided as a starting point, however it does
+to be adjusted for your build environment.  In particular, the paths and repo
+locations need to be configured for your system.  It is highly recommended that
+a local mirror of the CentOS repos be used for speed.  The example config file
+is configured to use an localhost http mirror of the CentOS repos.
+
+To build the binary RPMs from the source RPMs change to the "srcs" subdirectory
+and execute the "build.sh" script.
+
+# cd srcs
+# ./build.sh
+
+This will use build.cfg and mock to compile every source RPM listed in list.txt.
+The output binary RPMs will be in srcs/results.  There will also be success.txt
+and fail.txt files which list any RPMs that failed to build.  Debugging why RPMs
+fail to build is beyond the scope of this document, however be aware that RPMs
+often fail in the "check" phase of the build (i.e. the package compiled fine
+but tests failed).  Notably, the python package may fail due to a "test_nis"
+failure, and the "attr" and "e2fsprogs" packages may or may not fail depending
+on the host file system used for compilation.  These failures may or may not be
+false positives (for example, the mock environment does not have NIS configured
+which is why python's test_nis reports a failure -- the code is actually fine,
+we just can't run the test in the mock environment).
+
+To disable the check phase, add the line
+
+config_opts['rpmbuild_opts'] = '--nocheck'
+
+to build.cfg.  You can then run build.sh again with list.txt containing
+packages which failed:
+
+# cp list.txt list.txt.orig
+# cp fail.txt list.txt
+# ./build.sh
+
+-------------------------------------------------------------------------------
+How to build the install disk from the binary RPMS
+-------------------------------------------------------------------------------
+
+Once srcs/results is populated with binary RPMs, an installation disk can be
+built.  Edit the yum.conf file and place an (arbitrary) path for yum log and
+cache locations, and make sure that the repository path points to srcs/results.
+Run the build_centos.sh script to build the installation DVD:
+
+# ./build_centos.sh
+
+Scroll up the output to the top of the "Spawning worker" messages.  You should
+observe a line indicating that there are no remaining unresolved dependencies:
+
+...
+Installing PKG=dhcp-common PKG_FILE=dhcp-common-4.2.5-42.el7.centos.tis.1.x86_64.rpm PKG_REL_PATH=dhcp-common-4.2.5-42.el7.centos.tis.1.x86_64.rpm PKG_PATH=/localdisk/loadbuild/jmckenna/centos/srcs/results/dhcp-common-4.2.5-42.el7.centos.tis.1.x86_64.rpm from repo local-std
+dhcp-common
+Debug: Packages still unresolved:
+
+Spawning worker 0 with 4 pkgs
+Spawning worker 1 with 4 pkgs
+Spawning worker 2 with 4 pkgs
+...
+
+This is your confirmation that all required pacakges were found and installed
+on the ISO.  You should also now see a new file called "centosIso.iso":
+
+# ls -l centosIso.iso
+
+-------------------------------------------------------------------------------
+How to install the minimal system
+-------------------------------------------------------------------------------
+
+The centosIso.iso file can be burned to a DVD or booted in a virtual
+environment.  It is configured to self-install on boot.  After installation,
+a user with sudo access must be created manually.  The system can then be
+booted.
+
+Power the system on with the DVD inserted.  A system install will take place
+(takes approximately 2 minutes).  The system will then report an error and
+ask you if you wish to report a bug, debug, or quit.  Hit control-alt-F2 to
+switch to a terminal window.  Enter the following commands to change to the
+installed system root, and create a (wrsroot) with sudo access:
+
+cd /mnt/sysimage
+chroot .
+groupadd -r wrs
+groupadd -f -g 345 wrs_protected
+useradd -m -g wrs -G root,wrs_protected,wheel -d /home/wrsroot -p cBglipPpsKwBQ -s /bin/sh wrsroot
+exit
+
+Change back to the main window with control-alt-F1.
+Hit 3 <enter> (the "Quit" option).  The system will reboot (make sure you eject
+the DVD or use your BIOS to boot from hard disk rather than DVD; the installer
+will re-run if the DVD boots again).
+
+You can log into the system as user "wrsroot" with password "wrsroot".
+

build-tools/build_minimal_iso/README.2

@@ -0,0 +1,5 @@
+The files in this directory are to be used as described at
+http://twiki.wrs.com/PBUeng/DeliveryExtras#Minimal_CentOS_install
+
+They include the scripts (and customer README) for building a minimual
+CentOS ISO from our modified sources.

build-tools/build_minimal_iso/build.cfg

@@ -0,0 +1,108 @@
+config_opts['root'] = 'jmckenna-centos/mock'
+config_opts['target_arch'] = 'x86_64'
+config_opts['legal_host_arches'] = ('x86_64',)
+config_opts['chroot_setup_cmd'] = 'install @buildsys-build'
+config_opts['dist'] = 'el7'  # only useful for --resultdir variable subst
+config_opts['releasever'] = '7'
+
+config_opts['yum.conf'] = """
+[main]
+keepcache=1
+debuglevel=2
+reposdir=/dev/null
+logfile=/var/log/yum.log
+retries=20
+obsoletes=1
+gpgcheck=0
+assumeyes=1
+syslog_ident=mock
+syslog_device=
+
+# repos
+[my-build]
+name=my-build
+baseurl=http://127.0.0.1:8088/localdisk/loadbuild/centos/src/results
+enabled=1
+skip_if_unavailable=1
+metadata_expire=0
+
+[base]
+name=CentOS-$releasever - Base
+#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
+#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
+baseurl=http://127.0.0.1:8088/CentOS/7.2.1511/os/$basearch/
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+#released updates
+[updates]
+name=CentOS-$releasever - Updates
+#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra
+#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
+baseurl=http://127.0.0.1:8088/CentOS/7.2.1511/updates/$basearch/
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+#additional packages that may be useful
+[extras]
+name=CentOS-$releasever - Extras
+#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra
+#baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
+baseurl=http://127.0.0.1:8088/CentOS/7.2.1511/extras/$basearch/
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+#additional packages that extend functionality of existing packages
+[centosplus]
+name=CentOS-$releasever - Plus
+#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus&infra=$infra
+#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/
+baseurl=http://127.0.0.1:8088/CentOS/7.2.1511/centosplus/$basearch/
+gpgcheck=1
+enabled=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+[epel]
+name=Extra Packages for Enterprise Linux 7 - $basearch
+baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch
+#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch
+failovermethod=priority
+enabled=1
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
+
+[epel-debuginfo]
+name=Extra Packages for Enterprise Linux 7 - $basearch - Debug
+baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch/debug
+#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-7&arch=$basearch
+failovermethod=priority
+enabled=0
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
+gpgcheck=1
+
+[epel-source]
+name=Extra Packages for Enterprise Linux 7 - $basearch - Source
+baseurl=http://download.fedoraproject.org/pub/epel/7/SRPMS
+#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-source-7&arch=$basearch
+failovermethod=priority
+enabled=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
+gpgcheck=1
+
+
+"""
+config_opts['environment']['BUILD_BY'] = 'jmckenna'
+config_opts['environment']['BUILD_DATE'] = '2016-10-31 14:27:28 -0400'
+config_opts['environment']['REPO'] = '/localdisk/designer/jmckenna/dev0019/cgcs-root'
+config_opts['environment']['WRS_GIT_BRANCH'] = 'CGCS_DEV_0019'
+config_opts['environment']['CGCS_GIT_BRANCH'] = 'CGCS_DEV_0019'
+config_opts['macros']['%_no_cgcs_license_check'] = '1'
+config_opts['macros']['%_tis_build_type'] = 'std'
+config_opts['chroot_setup_cmd'] = 'install @buildsys-build pigz lbzip2 yum shadow-utils rpm-build lbzip2 gcc glibc-headers make gcc-c++ java-devel'
+config_opts['macros']['%__gzip'] = '/usr/bin/pigz'
+config_opts['macros']['%__bzip2'] = '/usr/bin/lbzip2'
+config_opts['macros']['%_patch_confdir'] = '%{_sysconfdir}/patching'
+config_opts['macros']['%_patch_scripts'] = '%{_patch_confdir}/patch-scripts'
+config_opts['macros']['%_runtime_patch_scripts'] = '/run/patching/patch-scripts'
+config_opts['macros']['%_tis_dist'] = '.tis'
+#config_opts['rpmbuild_opts'] = '--nocheck'

build-tools/build_minimal_iso/build.sh

@@ -0,0 +1,45 @@
+#!/bin/sh
+
+CREATEREPO=$(which createrepo_c)
+if [ $? -ne 0 ]; then
+   CREATEREPO="createrepo"
+fi
+
+# If a file listed in list.txt is missing, this function attempts to find the
+# RPM and copy it to the local directory.  This should not be required normally
+# and is only used when collecting the source RPMs initially.
+function findSrc {
+	local lookingFor=$1
+	find $MY_REPO/cgcs-centos-repo/Source -name $lookingFor | xargs -I '{}' cp '{}' . 
+	find $MY_REPO/cgcs-tis-repo/Source -name $lookingFor | xargs -I '{}' cp '{}' . 
+	find $MY_WORKSPACE/std/rpmbuild/SRPMS -name $lookingFor | xargs -I '{}' cp '{}' .
+}
+
+rm -f success.txt
+rm -f fail.txt
+rm -f missing.txt
+mkdir -p results
+infile=list.txt
+
+while read p; do
+
+	if [ ! -f "$p" ]; then
+		findSrc $p
+		if [ ! -f "$p" ]; then
+			echo "couldn't find" >> missing.txt
+			echo "couldn't find $p" >> missing.txt
+			continue
+		fi
+		echo "found $p"
+	fi
+	
+	mock -r build.cfg $p --resultdir=results --no-clean
+	if [ $? -eq 0 ]; then
+		echo "$p" >> success.txt
+		cd results
+		$CREATEREPO .
+		cd ..
+	else
+		echo "$p" >> fail.txt
+	fi
+done < $infile

build-tools/build_minimal_iso/build_centos.sh

@@ -0,0 +1,62 @@
+#!/bin/bash
+
+# Build a basic CentOS system
+
+CREATEREPO=$(which createrepo_c)
+if [ $? -ne 0 ]; then
+   CREATEREPO="createrepo"
+fi
+
+function final_touches {
+   # create the repo
+   cd ${ROOTDIR}/${DEST}/isolinux
+   $CREATEREPO -g ../comps.xml .
+   
+   # build the ISO
+   printf "Building image $OUTPUT_FILE\n"
+   cd ${ROOTDIR}/${DEST}
+   chmod 664 isolinux/isolinux.bin
+   mkisofs -o $OUTPUT_FILE \
+      -R -D -A 'oe_iso_boot' -V 'oe_iso_boot' \
+      -b isolinux.bin -c boot.cat -no-emul-boot \
+      -boot-load-size 4 -boot-info-table \
+      -eltorito-alt-boot \
+      -e images/efiboot.img \
+            -no-emul-boot \
+      isolinux/   
+
+   isohybrid --uefi $OUTPUT_FILE
+   implantisomd5 $OUTPUT_FILE
+
+   cd $ROOTDIR
+}
+
+function setup_disk {
+	tar xJf emptyInstaller.tar.xz
+	mkdir ${DEST}/isolinux/Packages
+}
+
+function install_packages {
+	cd ${DEST}/isolinux/Packages
+	ROOT=${ROOTDIR} ../../../cgts_deps.sh --deps=../../../${MINIMAL}
+	cd ${ROOTDIR}
+}
+
+
+ROOTDIR=$PWD
+INSTALLER_SRC=basicDisk
+DEST=newDisk
+PKGS_DIR=all_rpms
+MINIMAL=minimal_rpm_list.txt
+OUTPUT_FILE=${ROOTDIR}/centosIso.iso
+
+# Make a basic install disk (no packages, at this point)
+rm -rf ${DEST}
+setup_disk
+
+# install the packages (initially from minimal list, then resolve deps)
+install_packages
+
+# build the .iso
+final_touches
+

build-tools/build_minimal_iso/cgts_deps.sh

@@ -0,0 +1,222 @@
+#!/bin/env bash
+function generate_dep_list {
+   TMP_RPM_DB=$(mktemp -d $(pwd)/tmp_rpm_db_XXXXXX)
+   mkdir -p $TMP_RPM_DB
+   rpm --initdb --dbpath $TMP_RPM_DB
+   rpm --dbpath $TMP_RPM_DB --test -Uvh --replacefiles '*.rpm' >> $DEPDETAILLISTFILE 2>&1
+   rpm --dbpath $TMP_RPM_DB --test -Uvh --replacefiles '*.rpm' 2>&1 \
+      | grep -v "error:" \
+      | grep -v "warning:" \
+      | grep -v "Preparing..." \
+      | sed "s/ is needed by.*$//" | sed "s/ >=.*$//" | sort -u > $DEPLISTFILE
+   rm -rf $TMP_RPM_DB
+}
+
+function install_deps {
+	local DEP_LIST=""
+	local DEP_LIST_FILE="$1"
+
+	rm -f $TMPFILE
+
+	while read DEP
+	do
+		DEP_LIST="${DEP_LIST} ${DEP}"
+	done < $DEP_LIST_FILE
+
+	echo "Debug: List of deps to resolve: ${DEP_LIST}"
+
+	if [ -z "${DEP_LIST}" ]
+	then
+		return 0
+	fi
+
+	# go through each repo and convert deps to packages
+
+	for REPOID in `grep  '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
+		echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --whatprovides ${DEP_LIST} --qf='%{name}'"
+		TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --qf='%{name}' --whatprovides ${DEP_LIST} | sed "s/kernel-debug/kernel/g" >> $TMPFILE
+                \rm -rf $TMP_DIR/yum-$USER-*
+	done
+	sort $TMPFILE -u > $TMPFILE1
+	rm $TMPFILE
+
+	DEP_LIST=""
+	while read DEP
+	do
+		DEP_LIST="${DEP_LIST} ${DEP}"
+	done < $TMPFILE1
+	rm $TMPFILE1
+
+	# next go through each repo and install packages
+	local TARGETS=${DEP_LIST}
+	echo "Debug: Resolved list of deps to install: ${TARGETS}"
+	local UNRESOLVED
+	for REPOID in `grep  '^[[].*[]]$' $YUM | grep -v '[[]main[]]' | awk -F '[][]' '{print $2 }'`; do
+		UNRESOLVED=" $TARGETS "
+           
+		if [[ ! -z "${TARGETS// }" ]]; then
+			REPO_PATH=$(cat $YUM | sed -n "/^\[$REPOID\]\$/,\$p" | grep '^baseurl=' | head -n 1 | awk -F 'file://' '{print $2}' | sed 's:/$::')
+			>&2  echo "TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf='%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}'"
+			TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf="%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}" | sort -r -V >> $TMPFILE
+                        \rm -rf $TMP_DIR/yum-$USER-*
+
+			while read STR
+			do
+				>&2 echo "STR=$STR"
+				if [ "x$STR" == "x" ]; then
+					continue
+				fi
+
+				PKG=`echo $STR | cut -d " " -f 1`
+				PKG_FILE=`echo $STR | cut -d " " -f 2`
+				PKG_REL_PATH=`echo $STR | cut -d " " -f 3`
+				PKG_PATH="${REPO_PATH}/${PKG_REL_PATH}"
+
+				>&2 echo "Installing PKG=$PKG PKG_FILE=$PKG_FILE PKG_REL_PATH=$PKG_REL_PATH PKG_PATH=$PKG_PATH from repo $REPOID"
+				cp $PKG_PATH .
+				if [ $? -ne 0 ]
+				then
+					>&2 echo "  Here's what I have to work with..."
+					>&2 echo "  TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $PKG --qf=\"%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}\""
+					>&2 echo "  PKG=$PKG PKG_FILE=$PKG_FILE REPO_PATH=$REPO_PATH PKG_REL_PATH=$PKG_REL_PATH PKG_PATH=$PKG_PATH"
+				fi
+
+				echo $UNRESOLVED | grep $PKG
+				echo $UNRESOLVED | grep $PKG >> /dev/null
+				if [ $? -eq 0 ]; then
+					echo "$PKG found in $REPOID as $PKG" >> $BUILT_REPORT
+					echo "$PKG_PATH" >> $BUILT_REPORT
+					UNRESOLVED=$(echo "$UNRESOLVED" | sed "s# $PKG # #g")
+				else
+					echo "$PKG satisfies unknown target in $REPOID" >> $BUILT_REPORT
+					echo "  but it doesn't match targets, $UNRESOLVED" >> $BUILT_REPORT
+					echo "  path $PKG_PATH" >> $BUILT_REPORT
+					FOUND_UNKNOWN=1
+				fi
+			done < $TMPFILE #<<< "$(TMPDIR=$TMP_DIR repoquery -c $YUM --repoid=$REPOID --arch=x86_64,noarch --resolve $TARGETS --qf=\"%{name} %{name}-%{version}-%{release}.%{arch}.rpm %{relativepath}\" | sort -r -V)"
+                        \rm -rf $TMP_DIR/yum-$USER-*
+			TARGETS="$UNRESOLVED"
+		fi
+	done
+	>&2 echo "Debug: Packages still unresolved: $UNRESOLVED"
+	echo "Debug: Packages still unresolved: $UNRESOLVED" >> $WARNINGS_REPORT
+	echo "Debug: Packages still unresolved: $UNRESOLVED" >> $BUILT_REPORT
+	>&2 echo ""
+}
+
+function check_all_explicit_deps_installed
+{
+
+	PKGS_TO_CHECK=" "
+	while read PKG_TO_ADD
+	do
+		PKGS_TO_CHECK="$PKGS_TO_CHECK ${PKG_TO_ADD}"
+	done < $DEPLISTFILE
+	rpm -qp ${INSTALLDIR}/*.rpm --qf="%{name}\n" > $TMPFILE
+	
+	echo "checking... $PKGS_TO_CHECK vs ${INSTALLED_PACKAGE}"
+
+	while read INSTALLED_PACKAGE
+	do
+		echo $PKGS_TO_CHECK | grep -q "${INSTALLED_PACKAGE}"
+		if [ $? -eq 0 ]; then
+			PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/^${INSTALLED_PACKAGE} //"`
+			PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/ ${INSTALLED_PACKAGE} / /"`
+			PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/ ${INSTALLED_PACKAGE}\$//"`
+			PKGS_TO_CHECK=`echo $PKGS_TO_CHECK | sed "s/^${INSTALLED_PACKAGE}\$//"`
+		fi
+	done < $TMPFILE
+
+	if [ -z "$PKGS_TO_CHECK" ]
+	then
+		>&2 echo "All explicitly specified packages resolved!"
+	else
+		>&2 echo "Could not resolve packages: $PKGS_TO_CHECK"
+		return 1
+	fi
+	return 0
+}
+
+if [ "x${ROOT}" == "x" ]; then
+	ROOT=/localdisk/loadbuild/centos
+fi
+
+ATTEMPTED=0
+DISCOVERED=0
+OUTPUT_DIR=${ROOT}/newDisk
+YUM=${ROOT}/yum.conf
+TMP_DIR=${ROOT}/tmp
+DEPLISTFILE=${ROOT}/deps.txt
+DEPDETAILLISTFILE=${ROOT}/deps_detail.txt
+INSTALLDIR=${ROOT}/newDisk/isolinux/Packages
+
+BUILT_REPORT=${ROOT}/local.txt
+WARNINGS_REPORT=${ROOT}/warnings.txt
+LAST_TEST=${ROOT}/last_test.txt
+TMPFILE=${ROOT}/cgts_deps_tmp.txt
+TMPFILE1=${ROOT}/cgts_deps_tmp1.txt
+
+touch "$BUILT_REPORT"
+touch "$WARNINGS_REPORT"
+
+for i in "$@"
+do
+case $i in
+    -d=*|--deps=*)
+    DEPS="${i#*=}"
+    shift # past argument=value
+    ;;
+esac
+done
+
+mkdir -p $TMP_DIR
+
+rm -f "$DEPDETAILLISTFILE"
+# FIRST PASS we are being given a list of REQUIRED dependencies
+if [ "${DEPS}x" != "x" ]; then
+    cat $DEPS | grep -v "^#" > $DEPLISTFILE
+    install_deps $DEPLISTFILE
+    if [ $? -ne 0 ]; then
+        exit 1
+    fi
+fi
+
+# check that we resolved them all
+check_all_explicit_deps_installed
+if [ $? -ne 0 ]; then
+	>&2 echo "Error -- could not install all explicitly listed packages"
+	exit 1
+fi
+
+ALL_RESOLVED=0
+
+while [ $ALL_RESOLVED -eq 0 ]; do
+	cp $DEPLISTFILE $DEPLISTFILE.old
+	generate_dep_list
+	if [ ! -s $DEPLISTFILE ]; then
+		# no more dependencies!
+		ALL_RESOLVED=1
+	else
+		DIFFLINES=`diff $DEPLISTFILE.old $DEPLISTFILE | wc -l`
+		if [ $DIFFLINES -eq 0 ]; then
+			>&2 echo "Warning: Infinite loop detected in dependency resolution.  See $DEPLISTFILE for details -- exiting"
+			>&2 echo "These RPMS had problems (likely version conflicts)"
+			>&2 cat  $DEPLISTFILE
+
+			echo "Warning: Infinite loop detected in dependency resolution See $DEPLISTFILE for details -- exiting" >> $WARNINGS_REPORT
+			echo "These RPMS had problems (likely version conflicts)" >> $WARNINGS_REPORT
+			cat  $DEPLISTFILE >> $WARNINGS_REPORT
+
+			date > $LAST_TEST
+
+			rm -f $DEPLISTFILE.old
+			exit 1 # nothing fixed
+		fi
+		install_deps $DEPLISTFILE
+		if [ $? -ne 0 ]; then
+			exit 1
+		fi
+	fi
+done
+
+exit 0

build-tools/build_minimal_iso/yum.conf

@@ -0,0 +1,22 @@
+
+[main]
+cachedir=/localdisk/loadbuild/jmckenna/centos/yum/cache
+keepcache=1
+debuglevel=2
+reposdir=/dev/null
+logfile=/localdisk/loadbuild/jmckenna/centos/yum/yum.log
+retries=20
+obsoletes=1
+gpgcheck=0
+assumeyes=1
+syslog_ident=mock
+syslog_device=
+
+# repos
+[local-std]
+name=local-std
+baseurl=file:///localdisk/loadbuild/jmckenna/centos/srcs/results
+enabled=1
+skip_if_unavailable=1
+metadata_expire=0
+

build-tools/classify

@@ -0,0 +1,67 @@
+classify () {
+   local pkg_dir="$1"
+
+   if [ -f $pkg_dir/centos/srpm_path ]; then
+      # echo "srpm + patch: $(basename $(cat $pkg_dir/centos/srpm_path | head -n 1))"
+      echo "srpm + patches"
+   elif [ -f $pkg_dir/centos/*.spec ]; then
+      if [ -f $pkg_dir/centos/build_srpm ]; then
+         # echo "spec + custom_script: $pkg_dir"         
+         echo "spec + custom_script"         
+      elif [ -f $pkg_dir/centos/build_srpm.data ]; then
+         local ALLOW_EMPTY_RPM=""
+         local COPY_LIST=""
+         local SRC_DIR=""
+         local PKG_BASE="$pkg_dir"
+         source $pkg_dir/centos/build_srpm.data
+
+         if [ "" != "$SRC_DIR" ] ; then
+            # echo "spec + src_dir: $pkg_dir/$SRC_DIR"
+            echo "spec + src_dir"
+         elif [ "" != "$COPY_LIST" ] ; then
+            local TARBALL=""
+            for f in $COPY_LIST; do
+               case $f in
+                  *.tar.gz)  TARBALL=$f ;;
+                  *.tgz)     TARBALL=$f ;;
+                  *.tar.bz2) TARBALL=$f ;;
+                  *.tar.xz)  TARBALL=$f ;;
+                  *.tar)     TARBALL=$f ;;
+               esac
+            done
+            if [ "" != "$TARBALL" ]; then
+               # echo "spec + tarball: $pkg_dir/$TARBALL"
+               echo "spec + tarball"
+            else
+               # echo "spec + files: $pkg_dir"
+               echo "spec + files"
+            fi
+         elif [ "$ALLOW_EMPTY_RPM" == "true" ] ; then
+            # echo "spec + empty: $pkg_dir"
+            echo "spec + empty"
+         else
+            # echo "spec + build_srpm.data + unknown: $pkg_dir"
+            # cat $pkg_dir/centos/build_srpm.data
+            echo "spec + build_srpm.data + unknown"
+         fi
+      else
+         # echo "spec + unknown: $pkg_dir"
+         echo "spec + unknown"
+      fi
+   else
+      # echo "unknown: $pkg_dir"
+      echo "unknown"
+   fi
+}
+
+# for g in $(find $MY_REPO -type d -name .git); do
+#     d=$(dirname $g)
+#     c="$d/centos_pkg_dirs"
+#     if [ -f $c ]; then
+#        for d2 in $(cat $c); do
+#           pkg_dir="$d/$d2"
+#           echo "$(classify $pkg_dir): $pkg_dir"
+#        done
+#     fi
+# done
+

build-tools/create-cgcs-centos-repo

@@ -0,0 +1,54 @@
+#!/bin/bash
+
+DEST_DIR="$MY_REPO/cgcs-centos-repo"
+ORIGIN="$DEST_DIR/origin"
+
+CREATEREPO=$(which createrepo_c)
+if [ $? -ne 0 ]; then
+   CREATEREPO="createrepo"
+fi
+
+if [ ! -f $ORIGIN ]; then
+   echo "ERROR: file not found '$ORIGIN'"
+   exit -1
+fi
+
+ORIGIN_DIR=$(cat $MY_REPO/cgcs-centos-repo/origin | grep -v '^#' | head -n 1)
+
+if [ ! -d $ORIGIN_DIR ]; then
+   echo "ERROR: directory not found '$ORIGIN_DIR'"
+   exit -1
+fi
+
+echo "ORIGIN_DIR=$ORIGIN_DIR"
+for d in $(find $ORIGIN_DIR -type d | tail -n +2); do
+   RELATIVE_DIR=$(echo $d | sed "s#^$ORIGIN_DIR/##")
+   if [ -d $DEST_DIR/$RELATIVE_DIR ]; then
+      rm -rf $DEST_DIR/$RELATIVE_DIR/*
+   fi
+   mkdir -p $DEST_DIR/$RELATIVE_DIR
+done
+
+for d in $(find $ORIGIN_DIR -type d | tail -n +2); do
+   for f in $(find $d -maxdepth 1 -type f); do
+      RELATIVE_FILE=$(echo $f | sed "s#^$ORIGIN_DIR/##")
+      if [ -e "$DEST_DIR/$RELATIVE_FILE" ]; then
+         rm -f "$DEST_DIR/$RELATIVE_FILE"
+      fi
+
+      ln -s $f "$DEST_DIR/$RELATIVE_FILE"
+   done
+done
+
+for d in `find -L $DEST_DIR -type d -name repodata`; do
+   (cd $d/..
+    cp -L -f repodata/*comps*xml comps.xml
+    rm -rf repodata
+    if [ -f comps.xml ]; then
+       $CREATEREPO -g comps.xml --workers $(cat /usr/bin/nproc) $(pwd)
+    else
+       $CREATEREPO --workers $(cat /usr/bin/nproc) $(pwd)
+    fi
+   )
+done
+

build-tools/create-cgcs-tis-repo

@@ -0,0 +1,55 @@
+#!/bin/bash
+
+DEST_DIR="$MY_REPO/cgcs-tis-repo"
+ORIGIN="$DEST_DIR/origin"
+
+CREATEREPO=$(which createrepo_c)
+if [ $? -ne 0 ]; then
+   CREATEREPO="createrepo"
+fi
+
+
+if [ ! -f $ORIGIN ]; then
+   echo "ERROR: file not found '$ORIGIN'"
+   exit -1
+fi
+
+ORIGIN_DIR=$(cat $MY_REPO/cgcs-tis-repo/origin | grep -v '^#' | head -n 1)
+
+if [ ! -d $ORIGIN_DIR ]; then
+   echo "ERROR: directory not found '$ORIGIN_DIR'"
+   exit -1
+fi
+
+echo "ORIGIN_DIR=$ORIGIN_DIR"
+for d in $(find $ORIGIN_DIR -type d | tail -n +2); do
+   RELATIVE_DIR=$(echo $d | sed "s#^$ORIGIN_DIR/##")
+   if [ -d $DEST_DIR/$RELATIVE_DIR ]; then
+      rm -rf $DEST_DIR/$RELATIVE_DIR/*
+   fi
+   echo "mkdir -p $RELATIVE_DIR"
+   mkdir -p $DEST_DIR/$RELATIVE_DIR
+done
+
+for d in $(find $ORIGIN_DIR -type d | tail -n +2); do
+   for f in $(find $d -maxdepth 1 -type f); do
+      RELATIVE_FILE=$(echo $f | sed "s#^$ORIGIN_DIR/##")
+      if [ -e "$DEST_DIR/$RELATIVE_FILE" ]; then
+         rm -f "$DEST_DIR/$RELATIVE_FILE"
+      fi
+
+      ln -s $f "$DEST_DIR/$RELATIVE_FILE"
+   done
+done
+
+for d in `find -L $DEST_DIR -type d -name repodata`; do
+   (cd $d/..
+    rm -rf repodata
+    if [ -f comps.xml ]; then
+       $CREATEREPO -g comps.xml `pwd`
+    else
+       $CREATEREPO `pwd`
+    fi
+   )
+done
+

build-tools/create-yum-conf

@@ -0,0 +1,59 @@
+#!/bin/bash
+
+if [ "$MY_WORKSPACE" == "" ]; then
+   echo "ERROR: MY_WORKSPACE not defined"
+   exit 1;
+fi
+
+if [ "$MY_REPO" == "" ]; then
+   echo "ERROR: MY_REPO not defined"
+   exit 1;
+fi
+
+if [ "$MY_BUILD_ENVIRONMENT" == "" ]; then
+   echo "ERROR: MY_BUILD_ENVIRONMENT not defined"
+   exit 1;
+fi
+
+if [ "$MY_BUILD_DIR" == "" ]; then
+   echo "ERROR: MY_BUILD_DIR not defined"
+   exit 1;
+fi
+
+MY_YUM_CONF="$MY_WORKSPACE/yum.conf"
+MOCK_CFG_PROTO="$MY_REPO/cgcs-centos-repo/mock.cfg.proto"
+YUM_DIR="$MY_WORKSPACE/yum"
+YUM_CACHE="$YUM_DIR/cache"
+
+if [ -f "$MOCK_CFG_PROTO" ]; then
+    if [ -f "$MY_YUM_CONF" ]; then
+        N=$(find $MOCK_CFG_PROTO $MY_REPO/build-tools/create-yum-conf -cnewer $MY_YUM_CONF | wc -l) 
+        if [ $N -gt 0 ]; then
+            # New inputs, remove to force regeneration of yum.conf
+            \rm -f "$MY_YUM_CONF"
+        fi
+    fi
+fi
+   
+if [ ! -f "$MY_YUM_CONF" ]; then
+    if [ -f "$MOCK_CFG_PROTO" ]; then
+        mock_cfg_to_yum_conf.py "$MOCK_CFG_PROTO"      > "$MY_YUM_CONF"
+        sed -i "s%\[main\]%&\ncachedir=$YUM_CACHE%"      "$MY_YUM_CONF"
+        sed -i "s%logfile=.*%logfile=$YUM_DIR/yum.log%"  "$MY_YUM_CONF"
+        sed -i "s%LOCAL_BASE%file://%g"                  "$MY_YUM_CONF"
+        sed -i "s%MIRROR_BASE%file:///import/mirrors%g"  "$MY_YUM_CONF"
+        sed -i "s%BUILD_ENV%$MY_BUILD_ENVIRONMENT%g"     "$MY_YUM_CONF"
+        sed -i "s%/MY_BUILD_DIR%$MY_BUILD_DIR%g"         "$MY_YUM_CONF"
+        sed -i "s%/MY_REPO_DIR%$MY_REPO%g"               "$MY_YUM_CONF"
+    else
+        echo "ERROR: Could not find yum.conf or MOCK_CFG_PROTO"
+        exit 1
+    fi
+fi
+
+if [ ! -d "$YUM_CACHE" ]; then
+    mkdir -p "$YUM_CACHE"
+fi
+
+echo "$MY_YUM_CONF"
+exit 0

build-tools/create_dependancy_cache.py

@@ -0,0 +1,674 @@
+#!/usr/bin/python
+
+import xml.etree.ElementTree as ET
+import fnmatch
+import os
+import gzip
+import getopt
+import sys
+import string
+
+
+ns = { 'root': 'http://linux.duke.edu/metadata/common',
+       'filelists': 'http://linux.duke.edu/metadata/filelists',
+       'rpm': 'http://linux.duke.edu/metadata/rpm' }
+
+build_types=['std', 'rt']
+rpm_types=['RPM', 'SRPM']
+default_arch = 'x86_64'
+default_arch_list = [ 'x86_64', 'noarch' ]
+default_arch_by_type = {'RPM': [ 'x86_64', 'noarch' ],
+                        'SRPM': [ 'src' ]
+                       }
+
+repodata_dir="/export/jenkins/mirrors"
+if not os.path.isdir(repodata_dir):
+    repodata_dir="/import/mirrors"
+    if not os.path.isdir(repodata_dir):
+        print("ERROR: directory not found %s" % repodata_dir)
+        sys.exit(1)
+
+publish_cache_dir="%s/cgcs-tis-repo/dependancy-cache" % os.environ['MY_REPO']
+centos_repo_dir="%s/cgcs-centos-repo" % os.environ['MY_REPO']
+third_party_repo_dir="%s/cgcs-3rd-party-repo" % os.environ['MY_REPO']
+tis_repo_dir="%s/cgcs-tis-repo" % os.environ['MY_REPO']
+workspace_repo_dirs={}
+for rt in rpm_types:
+   workspace_repo_dirs[rt]={}
+   for bt in build_types:
+      workspace_repo_dirs[rt][bt]="%s/%s/rpmbuild/%sS" % (os.environ['MY_WORKSPACE'], bt, rt)
+
+if not os.path.isdir(os.environ['MY_REPO']):
+   print("ERROR: directory not found MY_REPO=%s" % os.environ['MY_REPO'])
+   sys.exit(1)
+
+if not os.path.isdir(centos_repo_dir):
+   print("ERROR: directory not found %s" % centos_repo_dir)
+   sys.exit(1)
+
+if not os.path.isdir(third_party_repo_dir):
+   print("ERROR: directory not found %s" % third_party_repo_dir)
+   sys.exit(1)
+
+if not os.path.isdir(tis_repo_dir):
+   print("ERROR: directory not found %s" % tis_repo_dir)
+   sys.exit(1)
+
+# bin_rpm_mirror_roots = ["%s/fedora/epel/7" % repodata_dir, 
+#                         "%s/CentOS/7.2.1511" % repodata_dir, 
+#                         "%s/CentOS/tis-r3/" % repodata_dir ]
+
+# src_rpm_mirror_roots = ["%s/fedora/dl.fedoraproject.org/pub/epel/7/SRPMS" % repodata_dir, 
+#                         "%s/CentOS/vault.centos.org/7.2.1511" % repodata_dir, 
+#                         "%s/CentOS/tis-r3/Source" % repodata_dir ]
+
+bin_rpm_mirror_roots = ["%s/Binary" % centos_repo_dir,
+                        "%s/Binary" % third_party_repo_dir ]
+
+src_rpm_mirror_roots = ["%s/Source" % centos_repo_dir,
+                        "%s/Source" % third_party_repo_dir ]
+
+for bt in build_types:
+   bin_rpm_mirror_roots.append(workspace_repo_dirs['RPM'][bt])
+   src_rpm_mirror_roots.append(workspace_repo_dirs['SRPM'][bt])
+
+short_options=''
+long_options=[ 'cache_dir=' ]
+
+options, remainder = getopt.getopt(sys.argv[1:], short_options, long_options)
+
+for opt, arg in options:
+    if opt in ('--cache_dir'):
+        publish_cache_dir = arg
+
+if not os.path.isdir(publish_cache_dir):
+    print("ERROR: directory not found %s" % publish_cache_dir)
+    sys.exit(1)
+
+# The Main data structure
+pkg_data={}
+
+for rpm_type in rpm_types:
+    pkg_data[rpm_type]={}
+
+    # map provided_name -> pkg_name
+    pkg_data[rpm_type]['providers']={}
+
+    # map pkg_name -> required_names ... could be a pkg, capability or file
+    pkg_data[rpm_type]['requires']={}
+
+    # map file_name -> pkg_name
+    pkg_data[rpm_type]['file_owners']={}
+
+    # map pkg_name -> file_name
+    pkg_data[rpm_type]['files']={}
+
+    # map pkg_name -> required_pkg_names ... only pkg names, and only direct requirement
+    pkg_data[rpm_type]['pkg_direct_requires']={}
+
+    # map pkg_name -> required_pkg_names ... only pkg names, but this is the transitive list of all requirements
+    pkg_data[rpm_type]['pkg_transitive_requires']={}
+
+    # map pkg_name -> descendant_pkgs ... only packages the directly require this package
+    pkg_data[rpm_type]['pkg_direct_descendants']={}
+
+    # map pkg_name -> descendant_pkgs ... packages that have a transitive requiremant on this package
+    pkg_data[rpm_type]['pkg_transitive_descendants']={}
+
+    # Map package name to a source rpm file name
+    pkg_data[rpm_type]['sourcerpm']={}
+    pkg_data[rpm_type]['binrpm']={}
+
+    # Map file name to package name
+    pkg_data[rpm_type]['fn_to_name']={}
+
+pkg_data['SRPM']['pkg_direct_requires_rpm']={}
+pkg_data['SRPM']['pkg_transitive_requires_rpm']={}
+
+
+# Return a list of file paths, starting in 'dir', matching 'pattern'
+#    dir= directory to search under
+#    pattern= search for file or directory matching pattern, wildcards allowed
+#    recursive_depth= how many levels of directory before giving up
+def file_search(dir, pattern, recursive_depth=0):
+    match_list = []
+    new_depth = recursive_depth - 1
+    # print "file_search(%s,%s,%s)" % (dir, pattern, recursive_depth)
+    for file in os.listdir(dir):
+        path = "%s/%s" % (dir, file)
+        if fnmatch.fnmatch(file, pattern):
+            print path
+            match_list.append(path)
+        elif (recursive_depth > 0) and os.path.isdir(path):
+            sub_list = []
+            sub_list = file_search(path, pattern, recursive_depth=new_depth)
+            match_list.extend(sub_list)
+    return match_list
+
+# Return the list of .../repodate/*primary.xml.gz files
+#    rpm_type= 'RPM' or 'SRPM'
+#    arch= e.g. x86_64, only relevant of rpm_type=='RPM'
+def get_repo_primary_data_list(rpm_type='RPM', arch_list=default_arch_list):
+    rpm_repo_roots = []
+    rpm_repodata_roots = []
+    repodata_list = []
+
+    if rpm_type == 'RPM':
+        for d in bin_rpm_mirror_roots:
+            sub_list = file_search(d, 'repodata', 25)
+            rpm_repodata_roots.extend(sub_list)
+    elif rpm_type == 'SRPM':
+        for d in src_rpm_mirror_roots:
+            sub_list = file_search(d, 'repodata', 5)
+            rpm_repodata_roots.extend(sub_list)
+    else:
+        print "invalid rpm_type '%s', valid types are %s" % (rpm_type, str(rpm_types))
+        return repodata_list
+
+    for d in rpm_repodata_roots:
+       sub_list = file_search(d, '*primary.xml.gz', 2)
+       repodata_list.extend(sub_list)
+   
+    return repodata_list
+
+
+# Return the list of .../repodate/*filelists.xml.gz files
+#    rpm_type= 'RPM' or 'SRPM'
+#    arch= e.g. x86_64, only relevant of rpm_type=='RPM'
+def get_repo_filelists_data_list(rpm_type='RPM', arch_list=default_arch_list):
+    rpm_repo_roots = []
+    rpm_repodata_roots = []
+    repodata_list = []
+
+    if rpm_type == 'RPM':
+        for d in bin_rpm_mirror_roots:
+            sub_list = file_search(d, 'repodata', 25)
+            rpm_repodata_roots.extend(sub_list)
+    elif rpm_type == 'SRPM':
+        for d in src_rpm_mirror_roots:
+            sub_list = file_search(d, 'repodata', 5)
+            rpm_repodata_roots.extend(sub_list)
+    else:
+        print "invalid rpm_type '%s', valid types are %s" % (rpm_type, str(rpm_types))
+        return repodata_list
+
+    for d in rpm_repodata_roots:
+       sub_list = file_search(d, '*filelists.xml.gz', 2)
+       repodata_list.extend(sub_list)
+
+    return repodata_list
+
+
+
+# Process a list of repodata files (*filelists.xml.gz) and extract package data.
+# Data is saved to the global 'pkg_data'.
+def read_data_from_repodata_filelists_list(repodata_list, rpm_type='RPM', arch=default_arch):
+    for repodata_path in repodata_list:
+        read_data_from_filelists_xml_gz(repodata_path, rpm_type=rpm_type, arch=arch)
+
+# Process a single repodata file (*filelists.xml.gz) and extract package data.
+# Data is saved to the global 'pkg_data'.
+def read_data_from_filelists_xml_gz(repodata_path, rpm_type='RPM', arch=default_arch):
+    # print "repodata_path=%s" % repodata_path
+    infile = gzip.open(repodata_path)
+    root = ET.parse(infile).getroot()
+    for pkg in root.findall('filelists:package', ns):
+        name=pkg.get('name')
+        pkg_arch=pkg.get('arch')
+
+        version=""
+        release=""
+
+        if arch is not None:
+            if pkg_arch is None:
+                continue
+            if pkg_arch != arch:
+                continue
+
+        v=pkg.find('filelists:version', ns)
+        if v is not None:
+            version=v.get('ver')
+            release=v.get('rel')
+        else:
+            print "%s: %s.%s has no 'filelists:version'" % (repodata_path, name, pkg_arch)
+
+        # print "%s  %s  %s  %s  " % (name, pkg_arch, version,  release)
+
+        for f in pkg.findall('filelists:file', ns):
+            fn=f.text
+            # print "   fn=%s -> plg=%s" % (fn, name)
+            if not name in pkg_data[rpm_type]['files']:
+                pkg_data[rpm_type]['files'][name]=[]
+            pkg_data[rpm_type]['files'][name].append(fn)
+            if not fn in pkg_data[rpm_type]['file_owners']:
+                pkg_data[rpm_type]['file_owners'][fn]=[]
+            pkg_data[rpm_type]['file_owners'][fn]=name
+
+
+
+
+
+# Process a list of repodata files (*primary.xml.gz) and extract package data.
+# Data is saved to the global 'pkg_data'.
+def read_data_from_repodata_primary_list(repodata_list, rpm_type='RPM', arch=default_arch):
+    for repodata_path in repodata_list:
+        read_data_from_primary_xml_gz(repodata_path, rpm_type=rpm_type, arch=arch)
+
+# Process a single repodata file (*primary.xml.gz) and extract package data.
+# Data is saved to the global 'pkg_data'.
+def read_data_from_primary_xml_gz(repodata_path, rpm_type='RPM', arch=default_arch):
+    # print "repodata_path=%s" % repodata_path
+    infile = gzip.open(repodata_path)
+    root = ET.parse(infile).getroot()
+    for pkg in root.findall('root:package', ns):
+        name=pkg.find('root:name', ns).text
+        pkg_arch=pkg.find('root:arch', ns).text
+        version=""
+        release=""
+        license=""
+        sourcerpm=""
+
+        if arch is not None:
+            if pkg_arch is None:
+                continue
+            if pkg_arch != arch:
+                continue
+
+        pkg_data[rpm_type]['providers'][name]=name
+        pkg_data[rpm_type]['files'][name]=[]
+        pkg_data[rpm_type]['requires'][name] = []
+        pkg_data[rpm_type]['requires'][name].append(name)
+
+        url=pkg.find('root:url', ns).text
+        v=pkg.find('root:version', ns)
+        if v is not None:
+            version=v.get('ver')
+            release=v.get('rel')
+        else:
+            print "%s: %s.%s has no 'root:version'" % (repodata_path, name, pkg_arch)
+
+        fn="%s-%s-%s.%s.rpm" % (name, version, release, arch)
+        pkg_data[rpm_type]['fn_to_name'][fn]=name
+
+        # SAL print "%s  %s  %s  %s  " % (name, pkg_arch, version,  release)
+        print "%s  %s  %s  %s  " % (name, pkg_arch, version,  release)
+        f=pkg.find('root:format', ns)
+        if f is not None:
+            license=f.find('rpm:license', ns).text
+            sourcerpm=f.find('rpm:sourcerpm', ns).text
+            if sourcerpm != "":
+                pkg_data[rpm_type]['sourcerpm'][name] = sourcerpm
+            # SAL print "--- requires ---"
+            print "--- requires ---"
+            r=f.find('rpm:requires', ns)
+            if r is not None:
+                for rr in r.findall('rpm:entry', ns):
+                    required_name=rr.get('name')
+                    # SAL print "    %s" % required_name
+                    print "    %s" % required_name
+                    pkg_data[rpm_type]['requires'][name].append(required_name)
+            else:
+                print "%s: %s.%s has no 'rpm:requires'" % (repodata_path, name, pkg_arch)
+            # print "--- provides ---"
+            p=f.find('rpm:provides', ns)
+            if p is not None:
+                for pp in p.findall('rpm:entry', ns):
+                    provided_name=pp.get('name')
+                    # print "    %s" % provided_name
+                    if name == "kernel-rt" and provided_name in pkg_data[rpm_type]['providers'] and pkg_data[rpm_type]['providers'][provided_name] == "kernel":
+                        continue
+                    if name.startswith('kernel-rt'):
+                        alt_name=string.replace(name, 'kernel-rt', 'kernel')
+                        if provided_name in pkg_data[rpm_type]['providers'] and pkg_data[rpm_type]['providers'][provided_name] == alt_name:
+                            continue
+                    pkg_data[rpm_type]['providers'][provided_name]=name
+            else:
+                print "%s: %s.%s has no 'rpm:provides'" % (repodata_path, name, pkg_arch)
+            # print "--- files ---"
+            for fn in f.findall('root:file', ns):
+               file_name=fn.text
+               # print "    %s" % file_name
+               pkg_data[rpm_type]['files'][name].append(file_name)
+               if name == "kernel-rt" and file_name in pkg_data[rpm_type]['file_owners'] and pkg_data[rpm_type]['file_owners'][file_name] == "kernel":
+                   continue
+               if name.startswith('kernel-rt'):
+                   alt_name=string.replace(name, 'kernel-rt', 'kernel')
+                   if provided_name in pkg_data[rpm_type]['file_owners'] and pkg_data[rpm_type]['file_owners'][file_name] == alt_name:
+                       continue
+               pkg_data[rpm_type]['file_owners'][file_name]=name
+        else:
+            print "%s: %s.%s has no 'root:format'" % (repodata_path, name, pkg_arch)
+        # print "%s  %s  %s  %s  %s" % (name, pkg_arch, version,  release, license)
+    infile.close
+    
+def calulate_all_direct_requires_and_descendants(rpm_type='RPM'):
+    # print "calulate_all_direct_requires_and_descendants rpm_type=%s" % rpm_type
+    for name in pkg_data[rpm_type]['requires']:
+        calulate_pkg_direct_requires_and_descendants(name, rpm_type=rpm_type)
+
+def calulate_pkg_direct_requires_and_descendants(name, rpm_type='RPM'):
+    print "SAL: %s needs:" % name
+    if not rpm_type in pkg_data:
+        print "Error: unknown rpm_type '%s'" % rpm_type
+        return
+
+    if not name in pkg_data[rpm_type]['requires']:
+        print "Note: No requires data for '%s'" % name
+        return
+
+    for req in pkg_data[rpm_type]['requires'][name]:
+        pro = '???'
+        if rpm_type == 'RPM':
+            if req in pkg_data[rpm_type]['providers']:
+                pro = pkg_data[rpm_type]['providers'][req]
+            elif req in pkg_data[rpm_type]['file_owners']:
+                pro = pkg_data[rpm_type]['file_owners'][req]
+            else:
+                pro = '???'
+                print "package %s has unresolved requirement '%s'" % (name, req)
+        else:
+            #  i.e. rpm_type == 'SRPM'
+            rpm_pro = '???'
+            if req in pkg_data['RPM']['providers']:
+                rpm_pro = pkg_data['RPM']['providers'][req]
+            elif req in pkg_data['RPM']['file_owners']:
+                rpm_pro = pkg_data['RPM']['file_owners'][req]
+            else:
+                rpm_pro = '???'
+                print "package %s has unresolved requirement '%s'" % (name, req)
+
+            if rpm_pro is not None and rpm_pro != '???':
+                if not name in pkg_data[rpm_type]['pkg_direct_requires_rpm']:
+                    pkg_data[rpm_type]['pkg_direct_requires_rpm'][name] = []
+                if not rpm_pro in pkg_data[rpm_type]['pkg_direct_requires_rpm'][name]:
+                    pkg_data[rpm_type]['pkg_direct_requires_rpm'][name].append(rpm_pro)
+
+                if rpm_pro in pkg_data['RPM']['sourcerpm']:
+                    fn = pkg_data['RPM']['sourcerpm'][rpm_pro]
+                    if fn in pkg_data['SRPM']['fn_to_name']:
+                        pro = pkg_data['SRPM']['fn_to_name'][fn]
+                    else:
+                        pro = '???'
+                        print "package %s requires srpm file name %s" % (name,fn)
+                else:
+                    pro = '???'
+                    print "package %s requires rpm %s, but that rpm has no known srpm" % (name,rpm_pro)
+
+        if pro is not None and pro != '???':
+            if not name in pkg_data[rpm_type]['pkg_direct_requires']:
+                pkg_data[rpm_type]['pkg_direct_requires'][name] = []
+            if not pro in pkg_data[rpm_type]['pkg_direct_requires'][name]:
+                pkg_data[rpm_type]['pkg_direct_requires'][name].append(pro)
+            if not pro in pkg_data[rpm_type]['pkg_direct_descendants']:
+                pkg_data[rpm_type]['pkg_direct_descendants'][pro] = []
+            if not name in pkg_data[rpm_type]['pkg_direct_descendants'][pro]:
+                pkg_data[rpm_type]['pkg_direct_descendants'][pro].append(name)
+
+        print "SAL:    %s -> %s" % (req, pro)
+
+
+
+def calulate_all_transitive_requires(rpm_type='RPM'):
+    for name in pkg_data[rpm_type]['pkg_direct_requires']:
+        calulate_pkg_transitive_requires(name, rpm_type=rpm_type)
+
+def calulate_pkg_transitive_requires(name, rpm_type='RPM'):
+    if not rpm_type in pkg_data:
+        print "Error: unknown rpm_type '%s'" % rpm_type
+        return
+
+    if not name in pkg_data[rpm_type]['pkg_direct_requires']:
+        print "Note: No direct_requires data for '%s'" % name
+        return
+
+    pkg_data[rpm_type]['pkg_transitive_requires'][name]=[]
+    if rpm_type != 'RPM':
+        pkg_data[rpm_type]['pkg_transitive_requires_rpm'][name]=[]
+    unresolved = []
+    unresolved.append(name)
+
+    while unresolved:
+        n = unresolved.pop(0)
+        # print "%s: remove %s" % (name, n)
+        if rpm_type == 'RPM':
+            direct_requires='pkg_direct_requires'
+            transitive_requires='pkg_transitive_requires'
+        else:
+            direct_requires='pkg_direct_requires_rpm'
+            transitive_requires='pkg_transitive_requires_rpm'
+        if n in pkg_data[rpm_type][direct_requires]:
+            for r in pkg_data[rpm_type][direct_requires][n]:
+                if r != name:
+                    if not r in pkg_data[rpm_type][transitive_requires][name]:
+                        pkg_data[rpm_type][transitive_requires][name].append(r)
+                        if r in pkg_data['RPM']['pkg_transitive_requires']:
+                            for r2 in pkg_data['RPM']['pkg_transitive_requires'][r]:
+                                if r2 != name:
+                                    if not r2 in pkg_data[rpm_type][transitive_requires][name]:
+                                        pkg_data[rpm_type][transitive_requires][name].append(r2)
+                        else:
+                            if rpm_type == 'RPM':
+                                unresolved.append(r)
+                            else:
+                                print "WARNING: calulate_pkg_transitive_requires: can't append rpm to SRPM list, name=%s, r=%s" % (name, r)
+                            # print "%s: add %s" % (name, r)
+    if rpm_type != 'RPM':
+        for r in pkg_data[rpm_type]['pkg_transitive_requires_rpm'][name]:
+            if r in pkg_data['RPM']['sourcerpm']:
+                fn = pkg_data['RPM']['sourcerpm'][r]
+                if fn in pkg_data['SRPM']['fn_to_name']:
+                    s = pkg_data['SRPM']['fn_to_name'][fn]
+                    pkg_data[rpm_type]['pkg_transitive_requires'][name].append(s)
+                else:
+                    print "package %s requires srpm file name %s, but srpm name is not known" % (name, fn)
+            else:
+                print "package %s requires rpm %s, but that rpm has no known srpm" % (name, r)
+
+def calulate_all_transitive_descendants(rpm_type='RPM'):
+    for name in pkg_data[rpm_type]['pkg_direct_descendants']:
+        calulate_pkg_transitive_descendants(name, rpm_type=rpm_type)
+
+def calulate_pkg_transitive_descendants(name, rpm_type='RPM'):
+    if not rpm_type in pkg_data:
+        print "Error: unknown rpm_type '%s'" % rpm_type
+        return
+
+    if not name in pkg_data[rpm_type]['pkg_direct_descendants']:
+        print "Note: No direct_requires data for '%s'" % name
+        return
+
+    pkg_data[rpm_type]['pkg_transitive_descendants'][name]=[]
+    unresolved = []
+    unresolved.append(name)
+
+    while unresolved:
+        n = unresolved.pop(0)
+        # print "%s: remove %s" % (name, n)
+        if n in pkg_data[rpm_type]['pkg_direct_descendants']:
+            for r in pkg_data[rpm_type]['pkg_direct_descendants'][n]:
+                if r != name:
+                    if not r in pkg_data[rpm_type]['pkg_transitive_descendants'][name]:
+                        pkg_data[rpm_type]['pkg_transitive_descendants'][name].append(r)
+                        if r in pkg_data[rpm_type]['pkg_transitive_descendants']:
+                            for n2 in pkg_data[rpm_type]['pkg_transitive_descendants'][r]:
+                                if n2 != name:
+                                    if not n2 in pkg_data[rpm_type]['pkg_transitive_descendants'][name]:
+                                        pkg_data[rpm_type]['pkg_transitive_descendants'][name].append(n2)
+                        else:
+                            unresolved.append(r)
+                            # print "%s: add %s" % (name, r)
+
+def create_dest_rpm_data():
+    for name in sorted(pkg_data['RPM']['sourcerpm']):
+        fn=pkg_data['RPM']['sourcerpm'][name]
+        if fn in pkg_data['SRPM']['fn_to_name']:
+            sname = pkg_data['SRPM']['fn_to_name'][fn]
+            if not sname in pkg_data['SRPM']['binrpm']:
+                pkg_data['SRPM']['binrpm'][sname]=[]
+            pkg_data['SRPM']['binrpm'][sname].append(name)
+
+def create_cache(cache_dir):
+    for rpm_type in rpm_types:
+        print ""
+        print "==== %s ====" % rpm_type
+        print ""
+        rpm_repodata_primary_list = get_repo_primary_data_list(rpm_type=rpm_type, arch_list=default_arch_by_type[rpm_type])
+        for arch in default_arch_by_type[rpm_type]:
+            read_data_from_repodata_primary_list(rpm_repodata_primary_list, rpm_type=rpm_type, arch=arch)
+        rpm_repodata_filelists_list = get_repo_filelists_data_list(rpm_type=rpm_type, arch_list=default_arch_by_type[rpm_type])
+        for arch in default_arch_by_type[rpm_type]:
+            read_data_from_repodata_filelists_list(rpm_repodata_filelists_list, rpm_type=rpm_type, arch=arch)
+        calulate_all_direct_requires_and_descendants(rpm_type=rpm_type)
+        calulate_all_transitive_requires(rpm_type=rpm_type)
+        calulate_all_transitive_descendants(rpm_type=rpm_type)
+
+        cache_name="%s/%s-direct-requires" % (cache_dir, rpm_type)
+        f=open(cache_name, "w")
+        for name in sorted(pkg_data[rpm_type]['pkg_direct_requires']):
+            print "%s needs %s" % (name, pkg_data[rpm_type]['pkg_direct_requires'][name])
+            f.write("%s;" % name)
+            first=True
+            for req in sorted(pkg_data[rpm_type]['pkg_direct_requires'][name]):
+                if first:
+                    first=False
+                    f.write("%s" % req)
+                else:
+                    f.write(",%s" % req)
+            f.write("\n")
+        f.close()
+
+        cache_name="%s/%s-direct-descendants" % (cache_dir, rpm_type)
+        f=open(cache_name, "w")
+        for name in sorted(pkg_data[rpm_type]['pkg_direct_descendants']):
+            print "%s informs %s" % (name, pkg_data[rpm_type]['pkg_direct_descendants'][name])
+            f.write("%s;" % name)
+            first=True
+            for req in sorted(pkg_data[rpm_type]['pkg_direct_descendants'][name]):
+                if first:
+                    first=False
+                    f.write("%s" % req)
+                else:
+                    f.write(",%s" % req)
+            f.write("\n")
+        f.close()
+
+        cache_name="%s/%s-transitive-requires" % (cache_dir, rpm_type)
+        f=open(cache_name, "w")
+        for name in sorted(pkg_data[rpm_type]['pkg_transitive_requires']):
+            f.write("%s;" % name)
+            first=True
+            for req in sorted(pkg_data[rpm_type]['pkg_transitive_requires'][name]):
+                if first:
+                    first=False
+                    f.write("%s" % req)
+                else:
+                    f.write(",%s" % req)
+            f.write("\n")
+        f.close()
+
+        cache_name="%s/%s-transitive-descendants" % (cache_dir, rpm_type)
+        f=open(cache_name, "w")
+        for name in sorted(pkg_data[rpm_type]['pkg_transitive_descendants']):
+            f.write("%s;" % name)
+            first=True
+            for req in sorted(pkg_data[rpm_type]['pkg_transitive_descendants'][name]):
+                if first:
+                    first=False
+                    f.write("%s" % req)
+                else:
+                    f.write(",%s" % req)
+            f.write("\n")
+        f.close()
+
+        if rpm_type != 'RPM':
+            cache_name="%s/%s-direct-requires-rpm" % (cache_dir, rpm_type)
+            f=open(cache_name, "w")
+            for name in sorted(pkg_data[rpm_type]['pkg_direct_requires_rpm']):
+                print "%s needs rpm %s" % (name, pkg_data[rpm_type]['pkg_direct_requires_rpm'][name])
+                f.write("%s;" % name)
+                first=True
+                for req in sorted(pkg_data[rpm_type]['pkg_direct_requires_rpm'][name]):
+                    if first:
+                        first=False
+                        f.write("%s" % req)
+                    else:
+                        f.write(",%s" % req)
+                f.write("\n")
+            f.close()
+
+            cache_name="%s/%s-transitive-requires-rpm" % (cache_dir, rpm_type)
+            f=open(cache_name, "w")
+            for name in sorted(pkg_data[rpm_type]['pkg_transitive_requires_rpm']):
+                f.write("%s;" % name)
+                first=True
+                for req in sorted(pkg_data[rpm_type]['pkg_transitive_requires_rpm'][name]):
+                    if first:
+                        first=False
+                        f.write("%s" % req)
+                    else:
+                        f.write(",%s" % req)
+                f.write("\n")
+            f.close()
+
+    cache_name="%s/rpm-to-srpm" % cache_dir
+    f=open(cache_name, "w")
+    for name in sorted(pkg_data['RPM']['sourcerpm']):
+        f.write("%s;" % name)
+        fn=pkg_data['RPM']['sourcerpm'][name]
+        if fn in pkg_data['SRPM']['fn_to_name']:
+            sname = pkg_data['SRPM']['fn_to_name'][fn]
+            f.write("%s" % sname)
+        f.write("\n")
+    f.close()
+
+    create_dest_rpm_data()
+    cache_name="%s/srpm-to-rpm" % cache_dir
+    f=open(cache_name, "w")
+    for name in sorted(pkg_data['SRPM']['binrpm']):
+        f.write("%s;" % name)
+        first=True
+        for bname in sorted(pkg_data['SRPM']['binrpm'][name]):
+            if first:
+                first=False
+                f.write("%s" % bname)
+            else:
+                f.write(",%s" % bname)
+        f.write("\n")
+    f.close()
+
+
+    
+def test():
+    for rpm_type in rpm_types:
+        print ""
+        print "==== %s ====" % rpm_type
+        print ""
+        rpm_repodata_primary_list = get_repo_primary_data_list(rpm_type=rpm_type, arch_list=default_arch_by_type[rpm_type])
+        for arch in default_arch_by_type[rpm_type]:
+            read_data_from_repodata_primary_list(rpm_repodata_primary_list, rpm_type=rpm_type, arch=arch)
+        rpm_repodata_filelists_list = get_repo_filelists_data_list(rpm_type=rpm_type, arch_list=default_arch_by_type[rpm_type])
+        for arch in default_arch_by_type[rpm_type]:
+            read_data_from_repodata_filelists_list(rpm_repodata_filelists_list, rpm_type=rpm_type, arch=arch)
+        calulate_all_direct_requires_and_descendants(rpm_type=rpm_type)
+        calulate_all_transitive_requires(rpm_type=rpm_type)
+        calulate_all_transitive_descendants(rpm_type=rpm_type)
+
+        for name in pkg_data[rpm_type]['pkg_direct_requires']:
+            print "%s needs %s" % (name, pkg_data[rpm_type]['pkg_direct_requires'][name])
+
+        for name in pkg_data[rpm_type]['pkg_direct_descendants']:
+            print "%s informs %s" % (name, pkg_data[rpm_type]['pkg_direct_descendants'][name])
+
+        for name in pkg_data[rpm_type]['pkg_transitive_requires']:
+            print "%s needs %s" % (name, pkg_data[rpm_type]['pkg_transitive_requires'][name])
+            print ""
+     
+        for name in pkg_data[rpm_type]['pkg_transitive_descendants']:
+            print "%s informs %s" % (name, pkg_data[rpm_type]['pkg_transitive_descendants'][name])
+            print ""
+
+
+if os.path.isdir(publish_cache_dir):
+   create_cache(publish_cache_dir)
+else:
+   print "ERROR: Directory not found '%s" % publish_cache_dir

build-tools/default_build_srpm

@@ -0,0 +1,264 @@
+#!/bin/bash
+# set -x
+
+source "$SRC_BASE/build-tools/spec-utils"
+source "$SRC_BASE/build-tools/srpm-utils"
+
+CUR_DIR=`pwd`
+BUILD_DIR="$RPMBUILD_BASE"
+
+if [ "x$DATA" == "x" ]; then
+   echo "ERROR: default_build_srpm (${LINENO}): Environment variable 'DATA' not defined."
+   exit 1
+fi
+
+srpm_source_build_data $DATA
+if [ $? -ne 0 ]; then
+    echo "ERROR: default_build_srpm (${LINENO}): Failed to source build data from $DATA"
+    exit 1
+fi
+
+if [ "x$VERSION" == "x" ]; then
+    for SPEC in `find $SPECS_BASE -name '*.spec' | sort -V`; do
+       SPEC_PATH="$SPEC"
+
+       VERSION_DERIVED=`spec_evaluate '%{version}' "$SPEC_PATH" 2>> /dev/null`
+       if [ $? -ne 0 ]; then
+           echo "ERROR: default_build_srpm (${LINENO}): '%{version}' not found in '$PKG_BASE/$SPEC_PATH'"
+           VERSION_DERIVED=""
+       fi
+
+       if [ "x$VERSION_DERIVED" != "x" ]; then
+          if [ "x$VERSION" == "x" ]; then
+             VERSION=$VERSION_DERIVED
+          else
+             if [ "x$SRC_DIR" != "x" ]; then
+                echo "ERROR: default_build_srpm (${LINENO}): multiple spec files found, can't set VERSION automatically"
+                exit 1
+             fi
+          fi
+       fi
+    done
+
+    if [ "x$VERSION" == "x" ]; then
+       if [ -f $SRC_DIR/PKG-INFO ]; then
+          VERSION=$(grep '^Version:' $SRC_DIR/PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
+       fi
+    fi
+
+    if [ "x$VERSION" != "x" ]; then
+        echo "Derived VERSION=$VERSION"
+    else
+        echo "ERROR: default_build_srpm (${LINENO}): Failed to derive a good VERSION from SPEC file, and none provided."
+        exit 1
+    fi
+fi
+
+if [ "x$TAR_NAME" == "x" ]; then
+    for SPEC in `find $SPECS_BASE -name '*.spec' | sort -V`; do
+       SPEC_PATH="$SPEC"
+
+       SERVICE=`spec_find_global service "$SPEC_PATH" 2>> /dev/null`
+       if [ $? -eq 0 ]; then
+          if [ "x$TAR_NAME" == "x" ]; then
+             TAR_NAME=$SERVICE
+          else
+             if [ "x$SRC_DIR" != "x" ]; then
+                echo "ERROR: default_build_srpm (${LINENO}): multiple spec files found, can't set TAR_NAME automatically"
+                exit 1
+             fi
+          fi
+       else
+          NAME=`spec_find_tag Name "$SPEC_PATH" 2>> /dev/null`
+          if [ $? -eq 0 ]; then
+             if [ "x$TAR_NAME" == "x" ]; then
+                TAR_NAME=$NAME
+             else
+                if [ "x$SRC_DIR" != "x" ]; then
+                   echo "ERROR: default_build_srpm (${LINENO}): multiple spec files found, can't set TAR_NAME automatically"
+                   exit 1
+                fi
+             fi
+          else
+             echo "WARNING: default_build_srpm (${LINENO}): 'Name' not found in '$SPEC_PATH'"
+             NAME=""
+          fi
+       fi
+    done
+
+    if [ "x$TAR_NAME" == "x" ]; then
+        if [ -f $SRC_DIR/PKG-INFO ]; then
+            TAR_NAME=$(grep '^Name:' $SRC_DIR/PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
+        fi
+    fi
+
+    if [ "x$TAR_NAME" != "x" ]; then
+        echo "Derived TAR_NAME=$TAR_NAME"
+    else
+        echo "ERROR: default_build_srpm (${LINENO}): Failed to derive a good TAR_NAME from SPEC file, and none provided."
+        exit 1
+    fi
+fi
+
+if [ "x$TAR" == "x" ]; then
+    TAR="$TAR_NAME-$VERSION.tar.gz"
+fi
+
+SOURCE_PATH="$BUILD_DIR/SOURCES"
+TAR_PATH="$SOURCE_PATH/$TAR"
+STAGING=""
+
+if [ "x$COPY_LIST_TO_TAR" != "x" ] || [ "x$EXCLUDE_LIST_FROM_TAR" != "x" ]; then
+	STAGING="$BUILD_DIR/staging"
+	mkdir -p $STAGING
+fi
+
+mkdir -p "$BUILD_DIR/SRPMS"
+mkdir -p "$SOURCE_PATH"
+
+if [ "x$SRC_DIR" == "x" -a "x$COPY_LIST" == "x" -a "$ALLOW_EMPTY_RPM" != "true" ]; then
+   echo "ERROR: default_build_srpm (${LINENO}): '$PWD/$DATA' failed to provide at least one of 'SRC_DIR' or 'COPY_LIST'"
+   exit 1
+fi
+
+if [ "x$SRC_DIR" != "x" ]; then
+   if [ ! -d "$SRC_DIR" ]; then
+      echo "ERROR: default_build_srpm (${LINENO}): directory not found: '$SRC_DIR'"
+      exit 1
+   fi
+fi
+
+if [ "x$COPY_LIST" != "x" ]; then
+   echo "COPY_LIST: $COPY_LIST"
+   for p in $COPY_LIST; do
+      # echo "COPY_LIST: $p"
+      \cp -L -u -r -v $p $SOURCE_PATH
+      if [ $? -ne 0 ]; then
+         echo "ERROR: default_build_srpm (${LINENO}): COPY_LIST: file not found: '$p'"
+         exit 1
+      fi
+   done
+fi
+
+if [ "x$STAGING" != "x" ]; then
+   \cp -L -u -r -v $SRC_DIR $STAGING
+   echo "COPY_LIST_TO_TAR: $COPY_LIST_TO_TAR"
+   for p in $COPY_LIST_TO_TAR; do
+      # echo "COPY_LIST_TO_TAR: $p"
+      \cp -L -u -r -v $p $STAGING/$SRC_DIR
+      if [ $? -ne 0 ]; then
+         echo "ERROR: default_build_srpm (${LINENO}): COPY_LIST_TO_TAR: file not found: '$p'"
+         exit 1
+      fi
+   done   
+   echo "EXCLUDE_LIST_FROM_TAR: $EXCLUDE_LIST_FROM_TAR"
+   for p in $EXCLUDE_LIST_FROM_TAR; do
+      # echo "EXCLUDE_LIST_FROM_TAR: $p"
+      echo "rm -rf $STAGING/$SRC_DIR/$p"
+      \rm -rf $STAGING/$SRC_DIR/$p
+      if [ $? -ne 0 ]; then
+         echo "ERROR: default_build_srpm (${LINENO}): EXCLUDE_LIST_FROM_TAR: could not remove file: '$p'"
+         exit 1
+      fi
+   done   
+   
+fi
+
+TRANSFORM=`echo "$SRC_DIR" | sed 's/^\./\\./' | sed 's:^/::'`
+
+if [ "x$STAGING" != "x" ]; then
+	pushd $STAGING
+fi
+
+TAR_NEEDED=0
+if [ "x$SRC_DIR" != "x" ]; then
+    echo "SRC_DIR=$SRC_DIR"
+    if [ -f $TAR_PATH ]; then
+        n=`find . -cnewer $TAR_PATH -and ! -path './.git*' \
+                                    -and ! -path './build/*' \
+                                    -and ! -path './.pc/*' \
+                                    -and ! -path './patches/*' \
+                                    -and ! -path "./$DISTRO/*" \
+                                    -and ! -path './pbr-*.egg/*' \
+                                    | wc -l`
+        if [ $n -gt 0 ]; then
+            TAR_NEEDED=1
+        fi
+    else
+        TAR_NEEDED=1
+    fi
+fi
+
+if [ $TAR_NEEDED -gt 0 ]; then
+    echo "Creating tar file: $TAR_PATH ..."
+    tar czf  $TAR_PATH $SRC_DIR --exclude '.git*' --exclude 'build' --exclude='.pc' --exclude='patches' --exclude="$SRC_DIR/$DISTRO" --exclude='pbr-*.egg' --transform "s,^$TRANSFORM,$TAR_NAME-$VERSION,"
+    if [ $? -ne 0 ]; then
+		if [ "x$STAGING" != "x" ]; then
+			popd
+		fi
+    
+        echo "ERROR: default_build_srpm (${LINENO}): failed to create tar file, cmd: tar czf  $TAR_PATH $SRC_DIR --exclude '.git*' --exclude 'build' --exclude='.pc' --exclude='patches' --exclude="$SRC_DIR/$DISTRO" --exclude='pbr-*.egg' --transform \"s,^$TRANSFORM,$TAR_NAME-$VERSION,\""
+        exit 1
+    fi
+    echo "Created tar file: $TAR_PATH"
+else
+    echo "Tar file not needed."
+fi
+
+if [ "x$STAGING" != "x" ]; then
+	popd
+fi
+
+if [ ! -d $BUILD_DIR/SPECS ]; then
+    echo "Spec directory '$BUILD_DIR/SPECS' does not exist"
+    exit 1
+fi
+
+if [ $(ls -1 $BUILD_DIR/SPECS/*.spec | wc -l) -eq 0 ]; then
+    echo "No spec files found in spec directory '$BUILD_DIR/SPECS'"
+    exit 1
+fi
+
+for SPEC in `ls -1 $BUILD_DIR/SPECS`; do
+    SPEC_PATH="$BUILD_DIR/SPECS/$SPEC"
+    RELEASE=`spec_find_tag Release "$SPEC_PATH" 2>> /dev/null`
+    if [ $? -ne 0 ]; then
+        echo "ERROR: default_build_srpm (${LINENO}): 'Release' not found in '$SPEC_PATH'"
+    fi
+    NAME=`spec_find_tag Name "$SPEC_PATH" 2>> /dev/null`
+    if [ $? -ne 0 ]; then
+        echo "ERROR: default_build_srpm (${LINENO}): 'Name' not found in '$SPEC_PATH'"
+    fi
+    SRPM="$NAME-$VERSION-$RELEASE.src.rpm"
+    SRPM_PATH="$BUILD_DIR/SRPMS/$SRPM"
+
+    spec_validate_tis_release $SPEC_PATH
+    if [ $? -ne 0 ]; then
+        echo "TIS Validation of $SPEC_PATH failed"
+        exit 1
+    fi
+
+    BUILD_NEEDED=0
+    if [ -f $SRPM_PATH ]; then
+        n=`find . -cnewer $SRPM_PATH | wc -l`
+        if [ $n -gt 0 ]; then
+            BUILD_NEEDED=1
+        fi
+    else
+        BUILD_NEEDED=1
+    fi
+
+    if [ $BUILD_NEEDED -gt 0 ]; then
+        echo "SPEC file: $SPEC_PATH"
+        echo "SRPM build directory: $BUILD_DIR"
+        echo "TIS_PATCH_VER: $TIS_PATCH_VER"
+
+        sed -i -e "1 i%define _tis_build_type $BUILD_TYPE" $SPEC_PATH
+        sed -i -e "1 i%define tis_patch_ver $TIS_PATCH_VER" $SPEC_PATH
+        rpmbuild -bs $SPEC_PATH --define="%_topdir $BUILD_DIR"  --undefine=dist --define="_tis_dist .tis"
+    else
+        echo "SRPM build not needed"
+    fi
+done
+
+

build-tools/find_klm

@@ -0,0 +1,59 @@
+#!/bin/bash
+
+for r in $(find $MY_WORKSPACE/*/rpmbuild/RPMS -name '*.rpm'); do
+   f=$(basename $r)
+   find  $MY_WORKSPACE/export/dist/isolinux/Packages | grep $f >> /dev/null
+   if [ $? -ne 0 ]; then
+         continue
+   fi
+   n=$(rpm -q --qf='%{NAME}\n' -p $r)
+   d=$(dirname $r)
+# echo "f=$f"
+   for f in $(rpm -q -p -l $r | grep '[.]ko$' | head -n 1); do
+      FOUND=0 
+      s=$(rpm -q --info -p $r | grep 'Source RPM  :' | awk -F: '{print $2}' | tr -d '[[:space:]]')
+      NAME=$(rpm -q --qf='%{NAME}\n' -p $d/$s)
+# echo "NAME=$NAME"
+      for s2 in $(find $MY_WORKSPACE/*/rpmbuild/SRPMS -name "$NAME-[0-9]*.src.rpm"); do
+         NAME2=$(rpm -q --qf='%{NAME}\n' -p $s2)
+# echo "NAME2=$NAME2"
+         if [ "${NAME}" == "${NAME2}" ]; then
+            echo $NAME | grep '[-]rt' >> /dev/null
+            if [ $? -ne 0 ]; then
+               echo $NAME
+               FOUND=1
+               break
+            fi
+            # SIMPLE_NAME=$(echo $NAME | sed 's#-kmod##' | sed 's#-kernel##' | sed 's#^kernel$#linux#'   | sed 's#^kernel-rt$#linux-rt#')
+            SIMPLE_NAME=$(echo $NAME | sed 's#^kernel$#linux#'   | sed 's#^kernel-rt$#linux-rt#')
+# echo "SIMPLE_NAME=$SIMPLE_NAME"
+            grep "[/]$SIMPLE_NAME$" $(for g in $(find $MY_REPO -type d -name .git); do d=$(dirname $g); find $d -name 'centos_pkg_dirs*'; done) >> /dev/null
+            if [ $? -eq 0 ]; then
+               echo $NAME
+               FOUND=1
+               break
+            fi
+            SIMPLE_NAME=$(echo $NAME | sed 's#-rt$##' )
+# echo "SIMPLE_NAME=$SIMPLE_NAME"
+            grep "[/]$SIMPLE_NAME$" $(for g in $(find $MY_REPO -type d -name .git); do d=$(dirname $g); find $d -name 'centos_pkg_dirs*'; done) >> /dev/null
+            if [ $? -eq 0 ]; then
+               echo $SIMPLE_NAME
+               FOUND=1
+               break
+            fi
+            SIMPLE_NAME2=$(echo $SIMPLE_NAME | sed 's#-kmod##' )
+# echo "SIMPLE_NAME2=$SIMPLE_NAME2"
+            grep "[/-]$SIMPLE_NAME2$" $(for g in $(find $MY_REPO -type d -name .git); do d=$(dirname $g); find $d -name 'centos_pkg_dirs*'; done) >> /dev/null
+            if [ $? -eq 0 ]; then
+               echo $SIMPLE_NAME
+               FOUND=1
+               break
+            fi
+         fi
+       done
+       if [ $FOUND -eq 1 ]; then
+          break
+       fi
+   done
+# done
+done | sort --unique

build-tools/find_patched_srpms_needing_upgrade

@@ -0,0 +1,54 @@
+#!/bin/bash
+
+for f in `find $MY_REPO -name srpm_path`; do
+   orig_line=`cat $f`
+   first=`echo $orig_line | awk -F : '{print $1}'`
+   orig_path="/import/mirrors/$orig_line"
+   if [ "$first" == "mirror" ]; then
+      orig_path="/import/mirrors/"$(echo $orig_line | awk -F : '{print $2}');
+   fi
+   if [ "$first" == "repo" ]; then
+      orig_path="$MY_REPO/"$(echo $orig_line | awk -F : '{print $2}')
+      continue
+   fi
+
+   if [ ! -f $orig_path ]; then
+      echo "ERROR: bad srpm path: '$orig_path' derived from '$f'"
+      exit 1
+   fi
+
+   orig_dir=$(dirname $orig_path)
+   repodata_dir=$orig_dir/repodata
+   if [ ! -d $repodata_dir ]; then
+      repodata_dir=$orig_dir/../repodata
+      if [ ! -d $repodata_dir ]; then
+         repodata_dir=$orig_dir/../../repodata
+         if [ ! -d $repodata_dir ]; then
+            echo "ERROR: couldn't find repodata for '$orig_path'"
+            exit 1
+         fi
+      fi
+   fi
+
+   # echo "'$orig_path' -> '$repodata_dir'"
+   name=$(rpm -q --queryformat '%{NAME}\n' -p $orig_path 2>> /dev/null)
+   version=$(rpm -q --queryformat '%{VERSION}\n' -p $orig_path 2>> /dev/null)
+   release=$(rpm -q --queryformat '%{RELEASE}\n' -p $orig_path 2>> /dev/null)
+   orig_name=$(basename $orig_path)
+   best_name="$orig_name"
+   for n in `find $orig_dir -name $name-*`; do
+      if [ "$n" != "$orig_path" ]; then
+         new_name=$(rpm -q --queryformat '%{NAME}\n' -p $n)
+         if [ "$name" == "$new_name" ]; then
+            rpmdev-vercmp $(basename $n) $best_name >> /dev/null
+            if [ $? -eq 11 ]; then
+               best_name=$(basename $n)
+            fi
+         fi
+      fi
+   done
+   if [ "$best_name" != "$orig_name" ]; then
+      echo "$f: $orig_name ==> $best_name"
+   fi
+done
+

build-tools/ip_report.py

@@ -0,0 +1,517 @@
+#!/usr/bin/python
+
+import csv
+import os
+import rpm
+import shutil
+import subprocess
+import sys
+import getopt
+
+
+class BinPackage(object):
+    def __init__(self, path, ts):
+        fdno = os.open(path, os.O_RDONLY)
+        hdr = ts.hdrFromFdno(path)
+        os.close(fdno)
+
+        self.source = hdr[rpm.RPMTAG_SOURCERPM]
+        self.desc = hdr[rpm.RPMTAG_DESCRIPTION].replace('\n', ' ')
+        self.dirname = os.path.dirname(path)
+        self.filename = os.path.basename(path)
+        self.path = path
+        self.kernel_module = False
+        self.name = hdr[rpm.RPMTAG_NAME]
+
+        # Does the package contain kernel modules?
+        for filename in hdr[rpm.RPMTAG_BASENAMES]:
+            assert isinstance(filename, basestring)
+            if filename.endswith('.ko'):
+                self.kernel_module = True
+                break
+
+
+class SrcPackage(object):
+    def __init__(self, path=None):
+        self.bin_pkg = None
+        self.original_src = None
+        self.sha = 'SHA'
+        if path is None:
+            self.filename = None
+            self.path = None
+        else:
+            self.filename = os.path.basename(path)
+            self.path = path
+            ts = rpm.TransactionSet()
+            ts.setVSFlags(rpm._RPMVSF_NODIGESTS | rpm._RPMVSF_NOSIGNATURES)
+            fdno = os.open(self.path, os.O_RDONLY)
+            hdr = ts.hdrFromFdno(self.path)
+            os.close(fdno)
+            self.desc = hdr[rpm.RPMTAG_DESCRIPTION].replace('\n', ' ')
+            self.version = hdr[rpm.RPMTAG_VERSION] + '-' + hdr[rpm.RPMTAG_RELEASE]
+            self.licences = hdr[rpm.RPMTAG_LICENSE]
+            self.name = hdr[rpm.RPMTAG_NAME]
+            self.url = hdr[rpm.RPMTAG_URL]
+
+        self.modified = None
+        self.kernel_module = False
+        self.disclosed_by = 'Jason McKenna'
+        self.shipped_as = 'Binary'
+        self.origin = 'Unknown'
+        self.notes = ''
+        self.wrs = False
+
+    def __lt__(self, other):
+        me = self.name.lower()
+        them = other.name.lower()
+        if me == them:
+            return self.name < other.name
+        else:
+            return me < them
+
+
+class IPReport(object):
+    __KNOWN_PATHS = [
+        # CentOS 7.4
+        ['/import/mirrors/CentOS/7.4.1708/os/Source/SPackages',
+         'http://vault.centos.org/7.4.1708/os/Source/SPackages'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.4.1708/updates/Source/SPackages',
+         'http://vault.centos.org/7.4.1708/updates/Source/SPackages'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.4.1708/cloud/Source/openstack-newton/common',
+         'http://vault.centos.org/7.4.1708/cloud/Source/openstack-newton/common'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.4.1708/cloud/Source/openstack-newton',
+         'http://vault.centos.org/7.4.1708/cloud/Source/openstack-newton'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.4.1708/cloud/Source/openstack-mitaka/common',
+         'http://vault.centos.org/7.4.1708/cloud/Source/openstack-mitaka/common'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.4.1708/cloud/Source/openstack-mitaka',
+         'http://vault.centos.org/7.4.1708/cloud/Source/openstack-mitaka'],
+        ['/import/mirrors/CentOS/7.4.1708/extras/Source/SPackages',
+         'http://vault.centos.org/7.4.1708/extras/Source/SPackages'],
+        # CentOS 7.3
+        ['/import/mirrors/CentOS/7.3.1611/os/Source/SPackages',
+         'http://vault.centos.org/7.3.1611/os/Source/SPackages'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.3.1611/updates/Source/SPackages',
+         'http://vault.centos.org/7.3.1611/updates/Source/SPackages'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.3.1611/cloud/Source/openstack-newton/common',
+         'http://vault.centos.org/7.3.1611/cloud/Source/openstack-newton/common'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.3.1611/cloud/Source/openstack-newton',
+         'http://vault.centos.org/7.3.1611/cloud/Source/openstack-newton'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.3.1611/cloud/Source/openstack-mitaka/common',
+         'http://vault.centos.org/7.3.1611/cloud/Source/openstack-mitaka/common'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.3.1611/cloud/Source/openstack-mitaka',
+         'http://vault.centos.org/7.3.1611/cloud/Source/openstack-mitaka'],
+        ['/import/mirrors/CentOS/7.3.1611/extras/Source/SPackages',
+         'http://vault.centos.org/7.3.1611/extras/Source/SPackages'],
+        # CentOS 7.2
+        ['/import/mirrors/CentOS/7.2.1511/os/Source/SPackages', 'http://vault.centos.org/7.2.1511/os/Source/SPackages'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.2.1511/updates/Source/SPackages',
+         'http://vault.centos.org/7.2.1511/updates/Source/SPackages'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.2.1511/cloud/Source/openstack-mitaka/common',
+         'http://vault.centos.org/7.2.1511/cloud/Source/openstack-mitaka/common'],
+        ['/import/mirrors/CentOS/vault.centos.org/7.2.1511/cloud/Source/openstack-mitaka',
+         'http://vault.centos.org/7.2.1511/cloud/Source/openstack-mitaka'],
+        ['/import/mirrors/CentOS/7.2.1511/extras/Source/SPackages',
+         'http://vault.centos.org/7.2.1511/extras/Source/SPackages'],
+        ['/import/mirrors/CentOS/tis-r4-CentOS/newton/Source', 'Unknown'],
+        ['/import/mirrors/CentOS/tis-r4-CentOS/tis-r4-3rd-Party', 'Unknown']
+
+        ]
+
+    def __init__(self, workspace=None, repo=None):
+        self.workspace = None
+        self.repo = None
+        self.shipped_binaries = list()
+        self.built_binaries = list()
+        self.check_env()
+        if workspace is not None:
+            self.workspace = workspace
+        if repo is not None:
+            self.repo = repo
+
+        # Generate a list of binaries that we shipped
+        for filename in os.listdir(self.workspace + '/export/dist/isolinux/Packages'):
+            if filename.endswith('rpm'):
+                self.shipped_binaries.append(filename)
+
+        # Generate a list of binaries that we built ourselves
+        for build in ['rt', 'std']:
+            for filename in os.listdir(self.workspace + '/' + build + '/rpmbuild/RPMS/'):
+                if filename.endswith('rpm'):
+                    self.built_binaries.append(filename)
+
+        print ('Looking up packages for which we have source...')
+        self.original_src_pkgs = dict()
+        self.build_original_src_pkgs()
+        print ('Looking up packages we built...')
+        self.built_src_pkgs = dict()
+        self.build_built_src_pkgs()
+        print ('Looking up packages we built...')
+        self.hardcoded_lookup_dict = dict()
+        self.build_hardcoded_lookup_dict()
+
+    def build_hardcoded_lookup_dict(self):
+        with open(self.repo + '/build-tools/source_lookup.txt', 'r') as lookup_file:
+            for line in lookup_file:
+                line = line.rstrip()
+                words = line.split()
+                if (words is not None) and (len(words) >= 2):
+                    self.hardcoded_lookup_dict[words[1]] = (words[0], False)
+
+        with open(self.repo + '/build-tools/wrs_orig.txt', 'r') as lookup_file:
+            for line in lookup_file:
+                line = line.rstrip()
+                words = line.split()
+                if (words is not None) and (len(words) >= 1):
+                    self.hardcoded_lookup_dict[words[0]] = ('No download', True)
+
+    @staticmethod
+    def path_to_origin(filepath):
+        for path in IPReport.__KNOWN_PATHS:
+            if filepath.startswith(path[0]) and (not path[1].lower().startswith('unknown')):
+                return path[1] + '/' + os.path.basename(filepath)
+        return 'Unknown'
+
+    def hardcoded_lookup(self, package_name):
+        if package_name in self.hardcoded_lookup_dict.keys():
+            return self.hardcoded_lookup_dict[package_name]
+        return None, False
+
+    def check_env(self):
+        if 'MY_WORKSPACE' in os.environ:
+            self.workspace = os.environ['MY_WORKSPACE']
+        else:
+            print 'Could not find $MY_WORKSPACE'
+            raise IOError('Could not fine $MY_WORKSPACE')
+
+        if 'MY_REPO' in os.environ:
+            self.repo = os.environ['MY_REPO']
+        else:
+            print 'Could not find $MY_REPO'
+            raise IOError('Could not fine $MY_REPO')
+
+    def do_bin_pkgs(self):
+        print ('Gathering binary package information')
+        self.read_bin_pkgs()
+
+    def read_bin_pkgs(self):
+        self.bin_pkgs = list()
+        ts = rpm.TransactionSet()
+        ts.setVSFlags(rpm._RPMVSF_NODIGESTS | rpm._RPMVSF_NOSIGNATURES)
+        for filename in self.shipped_binaries:
+            if filename.endswith('rpm'):
+                bin_pkg = BinPackage(self.workspace + '/export/dist/isolinux/Packages/' + filename, ts)
+                self.bin_pkgs.append(bin_pkg)
+
+    def do_src_report(self, copy_packages=False, do_wrs=True, delta_file=None, output_path=None, strip_unchanged=False):
+        self.bin_to_src()
+        self.src_pkgs.sort()
+
+        if delta_file is not None:
+            self.delta(delta_file)
+
+        if output_path is None:
+            output_path = self.workspace + '/export/ip_report'
+
+        # Create output dir (if required)
+        if not os.path.exists(output_path):
+            os.makedirs(output_path)
+
+        # Create paths for RPMs (if required)
+        if copy_packages:
+            if not os.path.exists(output_path + '/non_wrs'):
+                shutil.rmtree(output_path + '/non_wrs', True)
+                os.makedirs(output_path + '/non_wrs')
+            if do_wrs:
+                shutil.rmtree(output_path + '/wrs', True)
+                os.makedirs(output_path + '/wrs')
+
+        with open(output_path + '/srcreport.csv', 'wb') as src_report_file:
+            src_report_writer = csv.writer(src_report_file)
+
+            # Write header row
+            src_report_writer.writerow(
+                ['Package File', 'File Name', 'Package Name', 'Version', 'SHA1', 'Disclosed By',
+                 'Description', 'Part Of (Runtime, Host, Both)', 'Modified (Yes, No)', 'Hardware Interfacing (Yes, No)',
+                 'License(s) Found', 'Package Download URL', 'Kernel module', 'Notes'])
+
+            for src_pkg in self.src_pkgs:
+                if src_pkg.modified:
+                    modified_string = 'Yes'
+                else:
+                    modified_string = 'No'
+                if src_pkg.kernel_module:
+                    kmod_string = 'Yes'
+                else:
+                    kmod_string = 'No'
+
+                # Copy the pacakge and get the SHA
+                if copy_packages:
+                    if src_pkg.wrs is False:
+                        shutil.copyfile(src_pkg.path, output_path + '/non_wrs/' + src_pkg.filename)
+                        shasumout = subprocess.check_output(
+                            ['shasum', output_path + '/non_wrs/' + src_pkg.filename]).split()[0]
+                        src_pkg.sha = shasumout
+                        if strip_unchanged and (src_pkg.notes.lower().startswith('unchanged')):
+                            os.remove(output_path + '/non_wrs/' + src_pkg.filename)
+                    else:
+                        if do_wrs:
+                            shutil.copyfile(src_pkg.path, output_path + '/wrs/' + src_pkg.filename)
+                            shasumout = subprocess.check_output(
+                                ['shasum', output_path + '/wrs/' + src_pkg.filename]).split()[0]
+                            src_pkg.sha = shasumout
+                            if strip_unchanged and (src_pkg.notes.lower().startswith('unchanged')):
+                                os.remove(output_path + '/wrs/' + src_pkg.filename)
+
+                if do_wrs or (src_pkg.wrs is False):
+                    src_report_writer.writerow(
+                        [src_pkg.filename, src_pkg.name, src_pkg.version, src_pkg.sha, src_pkg.disclosed_by,
+                         src_pkg.desc, 'Runtime', src_pkg.shipped_as, modified_string, 'No', src_pkg.licences,
+                         src_pkg.origin, kmod_string, src_pkg.notes])
+                    if 'unknown' in src_pkg.origin.lower():
+                        print (
+                        'Warning: Could not determine origin of ' + src_pkg.name + '.  Please investigate/populate manually')
+
+    def bin_to_src(self):
+        self.src_pkgs = list()
+        src_pkg_names = list()
+        for bin_pkg in self.bin_pkgs:
+            if src_pkg_names.__contains__(bin_pkg.source):
+                if bin_pkg.kernel_module:
+                    for src_pkg in self.src_pkgs:
+                        if src_pkg.filename == bin_pkg.source:
+                            src_pkg.kernel_module = True
+                            break
+
+                continue
+
+            # if we reach here, then the source package is not yet in our db.
+            # we first search for the source package in the built-rpms
+            if 'shim-signed' in bin_pkg.source:
+                for tmp in self.built_src_pkgs:
+                    if 'shim-signed' in tmp:
+                        print ('shim-signed hack -- ' + bin_pkg.source + ' to ' + tmp)
+                        bin_pkg.source = tmp
+                        break
+            if 'shim-unsigned' in bin_pkg.source:
+                for tmp in self.built_src_pkgs:
+                    if 'shim-0' in tmp:
+                        print ('shim-unsigned hack -- ' + bin_pkg.source + ' to ' + tmp)
+                        bin_pkg.source = tmp
+                        break
+            if 'grub2-efi-pxeboot' in bin_pkg.source:
+                for tmp in self.built_src_pkgs:
+                    if 'grub2-2' in tmp:
+                        print ('grub2-efi-pxeboot hack -- ' + bin_pkg.source + ' to ' + tmp)
+                        bin_pkg.source = tmp
+                        break
+
+            if bin_pkg.source in self.built_src_pkgs:
+                src_pkg = self.built_src_pkgs[bin_pkg.source]
+                src_pkg.modified = True
+
+                # First guess, we see if there's an original source with the source package name
+                # (this is 99% of the cases)
+                src_pkg_orig_name = src_pkg.name
+                if src_pkg_orig_name in self.original_src_pkgs:
+                    src_pkg.original_src = self.original_src_pkgs[src_pkg_orig_name]
+                    src_pkg.origin = src_pkg.original_src.origin
+
+            else:
+                src_pkg_path = self.locate_in_mirror(bin_pkg.source)
+                if not os.path.isabs(src_pkg_path):
+                    continue
+                src_pkg = SrcPackage(src_pkg_path)
+                src_pkg.origin = IPReport.path_to_origin(src_pkg_path)
+                src_pkg.modified = False
+
+            if bin_pkg.kernel_module:
+                src_pkg.kernel_module = True
+
+            src_pkg_names.append(bin_pkg.source)
+            self.src_pkgs.append(src_pkg)
+
+            if src_pkg.origin.lower() == 'unknown':
+                if 'windriver' in src_pkg.licences.lower():
+                    src_pkg.origin = 'No download'
+                else:
+                    if src_pkg.url is not None:
+                        src_pkg.origin = src_pkg.url
+
+            if 'unknown' in src_pkg.origin.lower():
+                (orig, is_wrs) = self.hardcoded_lookup(src_pkg.name)
+                if orig is not None:
+                    src_pkg.origin = orig
+                    src_pkg.wrs = is_wrs
+
+            if (src_pkg.origin.lower() == 'no download') and ('windriver' in src_pkg.licences.lower()):
+                src_pkg.wrs = True
+
+    def locate_in_mirror(self, filename):
+        """ takes an RPM filename and finds the full path of the file """
+
+        fullpath = None
+
+        filename = filename.replace('mirror:', self.repo + '/cgcs-centos-repo/')
+        filename = filename.replace('repo:', self.repo + '/')
+        filename = filename.replace('3rd_party:', self.repo + '/cgcs-3rd-party-repo/')
+
+        # At this point, filename could be a complete path (incl symlink), or just a filename
+        best_guess = filename
+        filename = os.path.basename(filename)
+
+        for path in IPReport.__KNOWN_PATHS:
+            if os.path.exists(path[0] + '/' + filename):
+                fullpath = path[0] + '/' + filename
+                break
+
+        if fullpath is not None:
+            return fullpath
+        else:
+            return best_guess
+
+    def build_original_src_pkgs(self):
+        for root, dirs, files in os.walk(self.repo):
+            for name in files:
+                if name == 'srpm_path':
+                    with open(os.path.join(root, 'srpm_path'), 'r') as srpm_path_file:
+                        original_srpm_file = srpm_path_file.readline().rstrip()
+                        original_src_pkg_path = self.locate_in_mirror(original_srpm_file)
+                        original_src_pkg = SrcPackage(original_src_pkg_path)
+                        original_src_pkg.origin = IPReport.path_to_origin(original_src_pkg_path)
+                        self.original_src_pkgs[original_src_pkg.name] = original_src_pkg
+
+    def build_built_src_pkgs(self):
+        """ Create a dict of any source package that we built ourselves """
+        for build in ['std', 'rt']:
+            for root, dirs, files in os.walk(self.workspace + '/' + build + '/rpmbuild/SRPMS'):
+                for name in files:
+                    if name.endswith('.src.rpm'):
+                        built_src_pkg = SrcPackage(os.path.join(root, name))
+                        self.built_src_pkgs[built_src_pkg.filename] = built_src_pkg
+
+    def delta(self, orig_report):
+        if orig_report is None:
+            return
+        delta_src_pkgs = self.read_last_report(orig_report)
+
+        for pkg in self.src_pkgs:
+            if pkg.name in delta_src_pkgs:
+                old_pkg = delta_src_pkgs[pkg.name]
+                if old_pkg.version == pkg.version:
+                    pkg.notes = 'Unchanged'
+                else:
+                    pkg.notes = 'New version'
+            else:
+                pkg.notes = 'New package'
+
+    def read_last_report(self, orig_report):
+        orig_pkg_dict = dict()
+        with open(orig_report, 'rb') as orig_report_file:
+            orig_report_reader = csv.reader(orig_report_file)
+            doneHeader = False
+            for row in orig_report_reader:
+                if (not doneHeader) and ('package file name' in row[0].lower()):
+                    doneHeader = True
+                    continue
+                doneHeader = True
+                orig_pkg = SrcPackage()
+                orig_pkg.filename = row[0]
+                orig_pkg.name = row[1]
+                orig_pkg.version = row[2]
+                # sha = row[3]
+                orig_pkg.disclosed_by = row[4]
+                orig_pkg.desc = row[5]
+                # runtime = row[6]
+                orig_pkg.shipped_as = row[7]
+                if row[8].lower is 'yes':
+                    orig_pkg.modified = True
+                else:
+                    orig_pkg.modifed = False
+                # hardware interfacing = row[9]
+                orig_pkg.licences = row[10]
+                orig_pkg.origin = row[11]
+                if row[12].lower is 'yes':
+                    orig_pkg.kernel_module = True
+                else:
+                    orig_pkg.kernel_module = False
+                orig_pkg_dict[orig_pkg.name] = orig_pkg
+
+        return orig_pkg_dict
+
+
+def main(argv):
+    # handle command line arguments
+    # -h/--help       -- help
+    # -n/--no-copy    -- do not copy files (saves time)
+    # -d/--delta=     -- compare with an ealier report
+    # -o/--output=    -- output report/binaries to specified path
+    # -w/--workspace= -- use specified workspace instead of $WORKSPACE
+    # -r/--repo=      -- use sepeciied repo instead of $MY_REPO
+    # -s              -- strip (remove) unchanged packages from copy out directory
+
+    try:
+        opts, args = getopt.getopt(argv, "hnd:o:w:r:s",
+                                   ["delta=", "help", "no-copy", "workspace=", "repo=", "output=", "--strip"])
+    except getopt.GetoptError:
+        # todo - output help
+        sys.exit(2)
+    delta_file = None
+    do_copy = True
+    workspace = None
+    repo = None
+    output_path = None
+    strip_unchanged = False
+
+    for opt, arg in opts:
+        if opt in ('-h', '--help'):
+            print 'usage:'
+            print ' ip_report.py [options]'
+            print ' Creates and IP report in $MY_WORKSPACE/export/ip_report '
+            print ' Source RPMs (both Wind River and non WR) are placed in subdirs within that path'
+            print ''
+            print 'Options:'
+            print '  -h/--help                - this help'
+            print '  -d <file>/--delta=<file> - create "notes" field, comparing report with a previous report'
+            print '  -n/--no-copy             - do not copy files into subdirs (this is faster, but means you'
+            print '                             don\'t get SHA sums for files)'
+            print '  -w <path>/--workspace=<path> - use the specified path as workspace, instead of $MY_WORKSPACE'
+            print '  -r <path>/--repo=<path>  - use the specified path as repo, instead of $MY_REPO'
+            print '  -o <path>/--output=<path> - output to specified path (instead of $MY_WORKSPACE/export/ip_report)'
+            print '  -s/--strip               - strip (remove) unchanged files if copied'
+            exit()
+        elif opt in ('-d', '--delta'):
+            delta_file = os.path.normpath(arg)
+            delta_file = os.path.expanduser(delta_file)
+            if not os.path.exists(delta_file):
+                print 'Cannot locate ' + delta_file
+                exit(1)
+        elif opt in ('-w', '--workspace'):
+            workspace = os.path.normpath(arg)
+            workspace = os.path.expanduser(workspace)
+        elif opt in ('-r', '--repo'):
+            repo = os.path.normpath(arg)
+            repo = os.path.expanduser(repo)
+        elif opt in ('-o', '--output'):
+            output_path = os.path.normpath(arg)
+            output_path = os.path.expanduser(output_path)
+        elif opt in ('-n', '--no-copy'):
+            do_copy = False
+        elif opt in ('-s', '--strip-unchanged'):
+            strip_unchanged = True
+
+    print ('Doing IP report')
+    if delta_file is not None:
+        print 'Delta from ' + delta_file
+    else:
+        print 'No delta specified'
+    ip_report = IPReport(workspace=workspace, repo=repo)
+
+    ip_report.do_bin_pkgs()
+    ip_report.do_src_report(copy_packages=do_copy,
+                            delta_file=delta_file,
+                            output_path=output_path,
+                            strip_unchanged=strip_unchanged)
+
+
+if __name__ == "__main__":
+    main(sys.argv[1:])

build-tools/make-installer-images.sh

@@ -0,0 +1,244 @@
+#!/bin/bash -e
+## this script is called by "update-pxe-network-installer" and run in "sudo"
+## created by Yong Hu (yong.hu@intel.com), 05/24/2018
+
+function clean_rootfs() {
+    rootfs_dir=$1
+   echo "--> remove old files in original rootfs"
+    conf="$(ls ${rootfs_dir}/etc/ld.so.conf.d/kernel-*.conf)"
+    echo "conf basename = $(basename $conf)"
+    old_version="tbd"
+    if [ -f $conf ];then
+        old_version="$(echo $(basename $conf) | rev | cut -d'.' -f2- | rev | cut -d'-' -f2-)"
+    fi
+    echo "old version is $old_version"
+    # remove old files in original initrd.img
+    # do this in chroot to avoid accidentialy wrong operations on host root
+chroot $rootfs_dir /bin/bash -x <<EOF
+        rm -rf ./boot/ ./etc/modules-load.d/ 
+        if [ -n $old_version ] &&  [ -f ./etc/ld.so.conf.d/kernel-${old_version}.conf ]; then
+            rm -rf ./etc/ld.so.conf.d/kernel-${old_version}.conf 
+            rm -rf ./lib/modules/${old_version}
+        fi
+	if [ -d ./usr/lib64/python2.7/site-packages/pyanaconda/ ];then 
+            rm -rf usr/lib64/python2.7/site-packages/pyanaconda/
+        fi
+        if [ -d ./usr/lib64/python2.7/site-packages/rpm/ ];then
+            rm -rf usr/lib64/python2.7/site-packages/rpm/
+        fi
+        #find old .pyo files and delete them
+        all_pyo="`find ./usr/lib64/python2.7/site-packages/pyanaconda/ usr/lib64/python2.7/site-packages/rpm/ -name *.pyo`"
+        if [ -n $all ]; then
+            for pyo in $all_pyo;do
+                rm -f $pyo
+            done
+        fi
+        exit
+EOF
+    #back to previous folder
+}
+
+
+echo "This script makes new initrd.img, vmlinuz and squashfs.img."
+echo "NOTE: it has to be executed with *root*!"
+
+if [ $# -lt 2 ];then
+    echo "$0 <work_dir> <kernel_mode>"
+    echo "kernel_mode: std or rt"
+    exit -1;
+fi
+
+work_dir=$1
+mode=$2
+output_dir=$work_dir/output
+if [ ! -d $output_dir ];then mkdir -p $output_dir; fi
+
+if [ "$mode" != "std" ] && [ "$mode" != "rt" ]; then
+    echo "ERROR: wrong kernel mode, must be std or rt"
+    exit -1
+fi
+
+timestamp=$(date +%F_%H%M)
+
+echo "---------------- start to make new initrd.img and vmlinuz -------------"
+ORIG_INITRD=$work_dir/orig/initrd.img
+if [ ! -f $ORIG_INITRD ];then
+    echo "ERROR: $ORIG_INITRD does NOT exist!"
+    exit -1
+fi
+
+kernel_rpms_dir=$work_dir/kernel-rpms
+if [ ! -d $kernel_rpms_dir ];then
+    echo "ERROR: $kernel_rpms_dir does NOT exist!"
+    exit -1
+fi
+
+initrd_root=$work_dir/initrd.work
+if [ -d $initrd_root ];then
+    rm -rf $initrd_root
+fi
+mkdir -p $initrd_root
+
+cd $initrd_root
+# uncompress initrd.img
+echo "--> uncompress original initrd.img"
+/usr/bin/xzcat $ORIG_INITRD | cpio -i
+
+echo "--> clean up $initrd_root"
+clean_rootfs $initrd_root
+
+echo "--> extract files from new kernel and its modular rpms to initrd root"
+for kf in $kernel_rpms_dir/$mode/*.rpm ; do rpm2cpio $kf | cpio -idu; done
+
+# by now new kernel and its modules exist!
+# find new kernel in /boot/
+echo "--> get new kernel image: vmlinuz"
+new_kernel="$(ls ./boot/vmlinuz-*)"
+echo $new_kernel
+if [ -f $new_kernel ];then
+    #copy out the new kernel
+    if [ $mode == "std" ];then
+        if [ -f $output_dir/new-vmlinuz ]; then
+             mv -f $output_dir/new-vmlinuz $output_dir/vmlinuz-bakcup-$timestamp
+        fi
+        cp -f $new_kernel $output_dir/new-vmlinuz
+    else
+        if [ -f $output_dir/new-vmlinuz-rt ]; then
+             mv -f $output_dir/new-vmlinuz-rt $output_dir/vmlinuz-rt-bakcup-$timestamp
+        fi
+        cp -f $new_kernel $output_dir/new-vmlinuz-rt
+    fi
+    kernel_name=$(basename $new_kernel)
+    new_ver=$(echo $kernel_name | cut -d'-' -f2-)
+    echo $new_ver
+else
+    echo "ERROR: new kernel is NOT found!"
+    exit -1
+fi
+
+echo "-->check module dependencies in new initrd.img in chroot context"
+chroot $initrd_root /bin/bash -x <<EOF
+/usr/sbin/depmod -aeF "/boot/System.map-$new_ver" "$new_ver"
+if [ $? == 0 ]; then echo "module dependencies are satisfied!" ; fi
+## Remove the bisodevname package!
+rm -f ./usr/lib/udev/rules.d/71-biosdevname.rules ./usr/sbin/biosdevname
+exit
+EOF
+
+echo "--> Rebuild the initrd"
+if [ -f $output_dir/new-initrd.img ]; then
+    mv -f $output_dir/new-initrd.img $output_dir/initrd.img-bakcup-$timestamp
+fi
+find . | cpio -o -H newc | xz --check=crc32 --x86 --lzma2=dict=512KiB > $output_dir/new-initrd.img
+if [ $? != 0 ];then
+    echo "ERROR: failed to create new initrd.img"
+    exit -1
+fi
+
+cd $work_dir
+
+if [ -f $output_dir/new-initrd.img ];then
+    ls -l $output_dir/new-initrd.img
+else
+    echo "ERROR: new-initrd.img is not generated!"
+    exit -1
+fi
+
+if [ -f $output_dir/new-vmlinuz ];then
+    ls -l $output_dir/new-vmlinuz
+else
+    echo "ERROR: new-vmlinuz is not generated!"
+    exit -1
+fi
+
+echo "---------------- start to make new squashfs.img -------------"
+ORIG_SQUASHFS=$work_dir/orig/squashfs.img
+if [ ! -f $ORIG_SQUASHFS ];then
+    echo "ERROR: $ORIG_SQUASHFS does NOT exist!"
+    exit -1
+fi
+
+rootfs_rpms_dir=$work_dir/rootfs-rpms
+if [ ! -d $rootfs_rpms_dir ];then
+    echo "ERROR: $rootfs_rpms_dir does NOT exist!"
+    exit -1
+fi
+
+# make squashfs.mnt and ready and umounted
+if [ ! -d $work_dir/squashfs.mnt ];then 
+    mkdir -p $work_dir/squashfs.mnt
+else
+    # in case it was mounted previously
+    mnt_path=$(mount | grep "squashfs.mnt" | cut -d' ' -f3-3)
+    if [ x"$mnt_path" != "x" ] &&  [ "$(basename $mnt_path)" == "squashfs.mnt" ];then
+        umount $work_dir/squashfs.mnt
+    fi
+fi
+
+# make squashfs.work ready and umounted
+squashfs_root="$work_dir/squashfs.work"
+# Now mount the rootfs.img file:
+if [ ! -d $squashfs_root ];then
+    mkdir -p $squashfs_root
+else
+    # in case it was mounted previously
+    mnt_path=$(mount | grep "$(basename $squashfs_root)" | cut -d' ' -f3-3)
+    if [ x"$mnt_path" != "x" ] &&  [ "$(basename $mnt_path)" == "$(basename $squashfs_root)" ];then
+        umount $squashfs_root
+    fi
+fi
+
+echo $ORIG_SQUASHFS
+mount -o loop -t squashfs $ORIG_SQUASHFS $work_dir/squashfs.mnt
+
+if [ ! -d ./LiveOS ];then mkdir -p ./LiveOS ; fi
+
+echo "--> copy rootfs.img from original squashfs.img to LiveOS folder"
+cp -f ./squashfs.mnt/LiveOS/rootfs.img ./LiveOS/.
+
+echo "--> done to copy rootfs.img, umount squashfs.mnt"
+umount ./squashfs.mnt
+
+echo "--> mount rootfs.img into $squashfs_root"
+mount -o loop LiveOS/rootfs.img $squashfs_root
+
+echo "--> clean up ./squashfs-rootfs from original squashfs.img in chroot context"
+clean_rootfs $squashfs_root
+
+cd $squashfs_root
+echo "--> extract files from rootfs-rpms to squashfs root"
+for ff in $rootfs_rpms_dir/*.rpm ; do rpm2cpio $ff | cpio -idu; done
+
+echo "--> extract files from kernel and its modular rpms to squashfs root"
+for kf in $kernel_rpms_dir/$mode/*.rpm ; do rpm2cpio $kf | cpio -idu; done
+
+echo "-->check module dependencies in new squashfs.img in chroot context"
+#we are using the same new  kernel-xxx.rpm, so the $new_ver is the same
+chroot $squashfs_root /bin/bash -x <<EOF
+/usr/sbin/depmod -aeF "/boot/System.map-$new_ver" "$new_ver"
+if [ $? == 0 ]; then echo "module dependencies are satisfied!" ; fi
+## Remove the bisodevname package!
+rm -f ./usr/lib/udev/rules.d/71-biosdevname.rules ./usr/sbin/biosdevname
+exit
+EOF
+
+# come back to the original work dir
+cd $work_dir
+
+echo "--> unmount $squashfs_root"
+umount $squashfs_root
+#rename the old version
+if [ -f $output_dir/new-squashfs.img ]; then
+    mv -f $output_dir/new-squashfs.img $output_dir/squashfs.img-backup-$timestamp
+fi 
+
+echo "--> make the new squashfs image"
+mksquashfs LiveOS $output_dir/new-squashfs.img -keep-as-directory -comp xz -b 1M
+if [ $? == 0 ];then
+    ls -l $output_dir/new-squashfs.img
+else
+    echo "ERROR: failed to make a new squashfs.img"
+    exit -1
+fi
+
+echo "--> done successfully!"

build-tools/mirror_rebase/copy_external_mirror_to_tis_mirror

@@ -0,0 +1,229 @@
+#!/bin/bash
+
+if [ "$USER" != "jenkins" ]; then
+   echo "ERROR: only jenkins should run this"
+   return 1
+fi
+
+CENTOS_BASE_VERSION=7.2.1511
+EPEL_BASE_VERSION=7
+BIN_ARCH=x86_64
+
+BIN_DIRS="/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/updates/x86_64
+          /export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/cloud/x86_64/openstack-mitaka
+          /export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/os/x86_64
+          /export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/extras/x86_64
+          /export/jenkins/mirrors/fedora/epel/$EPEL_BASE_VERSION/x86_64
+         "
+SRC_DIRS="/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/updates/Source
+          /export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/cloud/Source/openstack-mitaka
+          /export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/os/Source
+          /export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/extras/Source
+          /export/jenkins/mirrors/fedora/dl.fedoraproject.org/pub/epel/$EPEL_BASE_VERSION/SRPMS
+         "
+TESTING_BIN_DIRS="/export/jenkins/mirrors/fedora/epel/testing/7/x86_64
+                 "
+TESTING_SRC_DIRS="/export/jenkins/mirrors/dl.fedoraproject.org/pub/epel/testing/7/SRPMS
+                 "
+
+BIN_ARCH=x86_64
+
+NEW_DIR="/export/jenkins/mirrors/CentOS/tis-r3-CentOS/mitaka"
+BACKUP_DIR="/export/jenkins/mirrors/CentOS/tis-r3-CentOS/mitaka.old"
+
+CREATEREPO=$(which createrepo_c)
+if [ $? -ne 0 ]; then
+   CREATEREPO="createrepo"
+fi
+
+if [ -d $BACKUP_DIR ]; then
+   rm -rf $BACKUP_DIR
+fi
+cp -r $NEW_DIR $BACKUP_DIR
+
+BIN_DEST="$NEW_DIR/Binary"
+SRC_DEST="$NEW_DIR/Source"
+LOG_DEST="$NEW_DIR/Data"
+DATA_DEST="$NEW_DIR/Data"
+
+mkdir -p $BIN_DEST $SRC_DEST $LOG_DEST
+
+WHITE_LIST_FILE="$DATA_DIR/white_list.txt"
+BLACK_LIST_FILE="$DATA_DIR/black_list.txt"
+OBSOLETE_LOG="$LOG_DEST/obsolete.log"
+NO_SOURCERPM_LOG="$LOG_DEST/no_sourcerpm.log"
+SOURCERPM_NOT_FOUND_LOG="$LOG_DEST/sourcerpm_not_found.log"
+BIN_COPY_LOG="$LOG_DEST/bin_copy.log"
+SRC_COPY_LOG="$LOG_DEST/src_copy.log"
+NO_INSTALLED_COPY="$LOG_DEST/installed_copy.log"
+BLACKLISTED_LOG="$LOG_DEST/blacklisted.log"
+WHITELISTED_LOG="$LOG_DEST/whitelisted.log"
+
+rm -f $OBSOLETE_LOG $NO_SOURCERPM_LOG $SOURCERPM_NOT_FOUND_LOG $BIN_COPY_LOG $SRC_COPY_LOG $NO_INSTALLED_COPY $BLACKLISTED_LOG $WHITELISTED_LOG
+
+LAST_INSTALLED_NAME=""
+LAST_NAME=""
+NAME=""
+
+# The following is similar to a find for *.rpm files,
+# but we transform the path into <name_and_version>#<filename>#<directory_name> .
+# Then do a reverse 'version' sort, so that the newest version
+# of the rpm appears first, no matter what directory it originates from.
+for dat in $((for bd in `echo $BIN_DIRS`; do
+                 for br in $(find $bd/ -name '*.rpm' ); do
+                    d=$(dirname $br)
+                    b=$(basename $br)
+                    s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+                    echo "$s#$b#$d"
+                 done
+              done) | sort -r -V)
+do
+   b=$(echo "$dat" | awk -F '#' '{ print $2 }')
+   d=$(echo "$dat" | awk -F '#' '{ print $3 }')
+   br="$d/$b"
+   echo $br
+
+   MATCH=$(grep "^$b" $BLACK_LIST_FILE || true)
+   if [ "$MATCH" != "" ]; then
+      echo "$bs" >> $BLACKLISTED_LOG
+      continue
+   fi
+
+   SOURCERPM=$(rpm -q --queryformat='%{SOURCERPM}' -p $br --nosignature)
+   if [ "x$SOURCERPM" != "x" ]; then
+      MATCH=$(grep "^$SOURCERPM" $BLACK_LIST_FILE || true)
+      if [ "$MATCH" != "" ]; then
+         echo "$bs" >> $BLACKLISTED_LOG
+         echo "$SOURCERPM" >> $BLACKLISTED_LOG
+         continue
+      fi
+   fi
+
+   ARCH=$(rpm -q --queryformat='%{ARCH}' -p $br --nosignature)
+   LAST_NAME="$NAME"
+   NAME=$(rpm -q --queryformat='%{NAME}' -p $br --nosignature)
+   if [ "$NAME" != "$LAST_NAME" ] && [ "$LAST_NAME" != "$LAST_INSTALLED_NAME" ]; then
+      echo "$LAST_NAME" >> $NO_INSTALLED_COPY
+   fi
+   if [ "$ARCH" == "$BIN_ARCH" ] || [ "$ARCH" == "noarch" ]; then
+      if [ "$NAME" != "$LAST_INSTALLED_NAME" ]; then
+         if [ "x$SOURCERPM" != "x" ]; then
+            bs=$(find $SRC_DIRS -name $SOURCERPM | head -n 1)
+            if [ "x$bs" != "x" ]; then
+               mkdir -p $BIN_DEST/$ARCH
+               if [ ! -f $BIN_DEST/$ARCH/$b ]; then
+                  cp -v $br $BIN_DEST/$ARCH/
+                  echo "$br" >> $BIN_COPY_LOG
+               fi
+               if [ ! -f $SRC_DEST/$SOURCERPM ]; then
+                  cp -v $bs $SRC_DEST/
+                  echo "$bs" >> $SRC_COPY_LOG
+               fi
+               LAST_INSTALLED_NAME=$NAME
+            else
+               echo "$SOURCERPM not found"
+               echo "$br" >> $SOURCERPM_NOT_FOUND_LOG
+            fi
+         else
+            echo "no SOURCERPM for $br"
+            echo "$br" >> $NO_SOURCERPM_LOG
+         fi
+      else
+         echo "$br is obsolete"
+         echo "$br" >> $OBSOLETE_LOG
+      fi
+   fi
+done
+
+for dat in $((for bd in $(echo $BIN_DIRS; echo $TESTING_BIN_DIRS); do
+                 for br in $(find $bd/ -name '*.rpm'); do
+                    d=$(dirname $br)
+                    b=$(basename $br)
+                    s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+                    echo "$s#$b#$d"
+                 done
+              done) | sort -r -V)
+do
+   b=$(echo "$dat" | awk -F '#' '{ print $2 }')
+   d=$(echo "$dat" | awk -F '#' '{ print $3 }')
+   br="$d/$b"
+   echo $br
+
+   MATCH=$(grep "^$b" $WHITE_LIST_FILE || true)
+   if [ "$MATCH" != "" ]; then
+      echo "$bs" >> $WHITELISTED_LOG
+   else
+      continue
+   fi
+   
+   SOURCERPM=$(rpm -q --queryformat='%{SOURCERPM}' -p $br --nosignature)
+   if [ "x$SOURCERPM" != "x" ]; then
+      grep "^$SOURCERPM" $WHITE_LIST_FILE >> /dev/null || true
+      if [ $? -eq 0 ]; then
+         echo "$bs" >> $WHITELISTED_LOG
+         echo "$SOURCERPM" >> $WHITELISTED_LOG
+      else
+         continue
+      fi
+   fi
+   
+   ARCH=$(rpm -q --queryformat='%{ARCH}' -p $br --nosignature)
+   LAST_NAME="$NAME"
+   NAME=$(rpm -q --queryformat='%{NAME}' -p $br --nosignature)
+   if [ "$NAME" != "$LAST_NAME" ] && [ "$LAST_NAME" != "$LAST_INSTALLED_NAME" ]; then
+      echo "$LAST_NAME" >> $NO_INSTALLED_COPY
+   fi
+   if [ "$ARCH" == "$BIN_ARCH" ] || [ "$ARCH" == "noarch" ]; then
+      if [ "$NAME" != "$LAST_INSTALLED_NAME" ]; then
+         if [ "x$SOURCERPM" != "x" ]; then
+            bs=$(find $SRC_DIRS $TESTING_SRC_DIRS -name $SOURCERPM | head -n 1)
+            if [ "x$bs" != "x" ]; then
+               mkdir -p $BIN_DEST/$ARCH
+               if [ ! -f $BIN_DEST/$ARCH/$b ]; then
+                  cp -v $br $BIN_DEST/$ARCH/
+                  echo "$br" >> $BIN_COPY_LOG
+               fi
+               if [ ! -f $SRC_DEST/$SOURCERPM ]; then
+                  cp -v $bs $SRC_DEST/
+                  echo "$bs" >> $SRC_COPY_LOG
+               fi
+               LAST_INSTALLED_NAME=$NAME
+            else
+               echo "$SOURCERPM not found"
+               echo "$br" >> $SOURCERPM_NOT_FOUND_LOG
+            fi
+         else
+            echo "no SOURCERPM for $br"
+            echo "$br" >> $NO_SOURCERPM_LOG
+         fi
+      else
+         echo "$br is obsolete"
+         echo "$br" >> $OBSOLETE_LOG
+      fi
+   fi
+done
+
+
+repodata_update () {
+   DIR=${1}
+   (
+   cd $DIR
+   for d in `find -L . -type d -name repodata`; do
+      (cd $d/..
+       for c in $(find repodata -name '*comps*xml'); do
+          mv -f $c comps.xml
+       done
+       rm -rf repodata
+       if [ -f comps.xml ]; then
+          $CREATEREPO -g comps.xml --workers $(cat /usr/bin/nproc) $(pwd)
+       else
+          $CREATEREPO --workers $(cat /usr/bin/nproc) $(pwd)
+       fi
+      )
+   done
+   )
+return 0
+}
+
+repodata_update $SRC_DEST
+repodata_update $BIN_DEST

build-tools/mirror_rebase/copy_external_mirror_to_tis_mirror.old

@@ -0,0 +1,216 @@
+#!/bin/bash
+
+if [ "$USER" != "jenkins" ]; then
+   echo "ERROR: only jenkins should run this"
+   return 1
+fi
+
+CENTOS_BASE_VERSION=7.2.1511
+EPEL_BASE_VERSION=7
+BIN_ARCH=x86_64
+
+BIN_DIRS="/export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/updates/x86_64
+          /export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/cloud/x86_64/openstack-mitaka
+          /export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/os/x86_64
+          /export/jenkins/mirrors/CentOS/$CENTOS_BASE_VERSION/extras/x86_64
+          /export/jenkins/mirrors/fedora/epel/$EPEL_BASE_VERSION/x86_64
+         "
+SRC_DIRS="/export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/updates/Source
+          /export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/cloud/Source/openstack-mitaka
+          /export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/os/Source
+          /export/jenkins/mirrors/CentOS/vault.centos.org/$CENTOS_BASE_VERSION/extras/Source
+          /export/jenkins/mirrors/fedora/dl.fedoraproject.org/pub/epel/$EPEL_BASE_VERSION/SRPMS
+         "
+TESTING_BIN_DIRS="/export/jenkins/mirrors/fedora/epel/testing/7/x86_64
+                 "
+TESTING_SRC_DIRS="/export/jenkins/mirrors/dl.fedoraproject.org/pub/epel/testing/7/SRPMS
+                 "
+
+ORIG_DIR="/export/jenkins/mirrors/CentOS/tis-r3-CentOS/mitaka"
+NEW_DIR="/export/jenkins/mirrors/CentOS/tis-r3-CentOS/mitaka-2"
+cp -r $ORIG_DIR $NEW_DIR
+
+BIN_DEST="$NEW_DIR/Binary"
+SRC_DEST="$NEW_DIR/Source"
+LOG_DEST="$NEW_DIR/Data"
+DATA_DEST="$NEW_DIR/Data"
+
+rm -rf $LOG_DEST
+mkdir -p $BIN_DEST $SRC_DEST $LOG_DEST
+
+WHITE_LIST_FILE="$DATA_DIR/white_list.txt"
+BLACK_LIST_FILE="$DATA_DIR/black_list.txt"
+OBSOLETE_LOG="$LOG_DEST/obsolete.log"
+NO_SOURCERPM_LOG="$LOG_DEST/no_sourcerpm.log"
+SOURCERPM_NOT_FOUND_LOG="$LOG_DEST/sourcerpm_not_found.log"
+BIN_COPY_LOG="$LOG_DEST/bin_copy.log"
+SRC_COPY_LOG="$LOG_DEST/src_copy.log"
+NO_INSTALLED_COPY="$LOG_DEST/installed_copy.log"
+BLACKLISTED_LOG="$LOG_DEST/blacklisted.log"
+WHITELISTED_LOG="$LOG_DEST/whitelisted.log"
+
+rm -f $OBSOLETE_LOG $NO_SOURCERPM_LOG $SOURCERPM_NOT_FOUND_LOG $BIN_COPY_LOG $SRC_COPY_LOG $NO_INSTALLED_COPY $BLACKLISTED_LOG $WHITELISTED_LOG
+
+LAST_INSTALLED_NAME=""
+LAST_NAME=""
+NAME=""
+
+# The following is similar to a find for *.rpm files,
+# but we transform the path into <name_and_version>#<filename>#<directory_name> .
+# Then do a reverse 'version' sort, so that the newest version
+# of the rpm appears first, no matter what directory it originates from.
+for dat in $((for bd in `echo $BIN_DIRS`; do
+                 for br in $(find $bd/ -name '*.rpm' ); do
+                    d=$(dirname $br)
+                    b=$(basename $br)
+                    s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+                    echo "$s#$b#$d"
+                 done
+              done) | sort -r -V)
+do
+   b=$(echo "$dat" | awk -F '#' '{ print $2 }')
+   d=$(echo "$dat" | awk -F '#' '{ print $3 }')
+   br="$d/$b"
+   echo $br
+
+   MATCH=$(grep "^$b" $BLACK_LIST_FILE || true)
+   if [ "$MATCH" != "" ]; then
+      echo "$bs" >> $BLACKLISTED_LOG
+      continue
+   fi
+
+   SOURCERPM=$(rpm -q --queryformat='%{SOURCERPM}' -p $br --nosignature)
+   if [ "x$SOURCERPM" != "x" ]; then
+      MATCH=$(grep "^$SOURCERPM" $BLACK_LIST_FILE || true)
+      if [ "$MATCH" != "" ]; then
+         echo "$bs" >> $BLACKLISTED_LOG
+         echo "$SOURCERPM" >> $BLACKLISTED_LOG
+         continue
+      fi
+   fi
+
+   ARCH=$(rpm -q --queryformat='%{ARCH}' -p $br --nosignature)
+   LAST_NAME="$NAME"
+   NAME=$(rpm -q --queryformat='%{NAME}' -p $br --nosignature)
+   if [ "$NAME" != "$LAST_NAME" ] && [ "$LAST_NAME" != "$LAST_INSTALLED_NAME" ]; then
+      echo "$LAST_NAME" >> $NO_INSTALLED_COPY
+   fi
+   if [ "$ARCH" == "$BIN_ARCH" ] || [ "$ARCH" == "noarch" ]; then
+      if [ "$NAME" != "$LAST_INSTALLED_NAME" ]; then
+         if [ "x$SOURCERPM" != "x" ]; then
+            bs=$(find $SRC_DIRS -name $SOURCERPM | head -n 1)
+            if [ "x$bs" != "x" ]; then
+               mkdir -p $BIN_DEST/$ARCH
+               if [ ! -f $BIN_DEST/$ARCH/$b ]; then
+                  cp -v $br $BIN_DEST/$ARCH/
+                  echo "$br" >> $BIN_COPY_LOG
+               fi
+               if [ ! -f $SRC_DEST/$SOURCERPM ]; then
+                  cp -v $bs $SRC_DEST/
+                  echo "$bs" >> $SRC_COPY_LOG
+               fi
+               LAST_INSTALLED_NAME=$NAME
+            else
+               echo "$SOURCERPM not found"
+               echo "$br" >> $SOURCERPM_NOT_FOUND_LOG
+            fi
+         else
+            echo "no SOURCERPM for $br"
+            echo "$br" >> $NO_SOURCERPM_LOG
+         fi
+      else
+         echo "$br is obsolete"
+         echo "$br" >> $OBSOLETE_LOG
+      fi
+   fi
+done
+
+for dat in $((for bd in $(echo $BIN_DIRS; echo $TESTING_BIN_DIRS); do
+                 for br in $(find $bd/ -name '*.rpm'); do
+                    d=$(dirname $br)
+                    b=$(basename $br)
+                    s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+                    echo "$s#$b#$d"
+                 done
+              done) | sort -r -V)
+do
+   b=$(echo "$dat" | awk -F '#' '{ print $2 }')
+   d=$(echo "$dat" | awk -F '#' '{ print $3 }')
+   br="$d/$b"
+   echo $br
+
+   MATCH=$(grep "^$b" $WHITE_LIST_FILE || true)
+   if [ "$MATCH" != "" ]; then
+      echo "$bs" >> $WHITELISTED_LOG
+   else
+      continue
+   fi
+   
+   SOURCERPM=$(rpm -q --queryformat='%{SOURCERPM}' -p $br --nosignature)
+   if [ "x$SOURCERPM" != "x" ]; then
+      grep "^$SOURCERPM" $WHITE_LIST_FILE >> /dev/null || true
+      if [ $? -eq 0 ]; then
+         echo "$bs" >> $WHITELISTED_LOG
+         echo "$SOURCERPM" >> $WHITELISTED_LOG
+      else
+         continue
+      fi
+   fi
+   
+   ARCH=$(rpm -q --queryformat='%{ARCH}' -p $br --nosignature)
+   LAST_NAME="$NAME"
+   NAME=$(rpm -q --queryformat='%{NAME}' -p $br --nosignature)
+   if [ "$NAME" != "$LAST_NAME" ] && [ "$LAST_NAME" != "$LAST_INSTALLED_NAME" ]; then
+      echo "$LAST_NAME" >> $NO_INSTALLED_COPY
+   fi
+   if [ "$ARCH" == "$BIN_ARCH" ] || [ "$ARCH" == "noarch" ]; then
+      if [ "$NAME" != "$LAST_INSTALLED_NAME" ]; then
+         if [ "x$SOURCERPM" != "x" ]; then
+            bs=$(find $SRC_DIRS $TESTING_SRC_DIRS -name $SOURCERPM | head -n 1)
+            if [ "x$bs" != "x" ]; then
+               mkdir -p $BIN_DEST/$ARCH
+               if [ ! -f $BIN_DEST/$ARCH/$b ]; then
+                  cp -v $br $BIN_DEST/$ARCH/
+                  echo "$br" >> $BIN_COPY_LOG
+               fi
+               if [ ! -f $SRC_DEST/$SOURCERPM ]; then
+                  cp -v $bs $SRC_DEST/
+                  echo "$bs" >> $SRC_COPY_LOG
+               fi
+               LAST_INSTALLED_NAME=$NAME
+            else
+               echo "$SOURCERPM not found"
+               echo "$br" >> $SOURCERPM_NOT_FOUND_LOG
+            fi
+         else
+            echo "no SOURCERPM for $br"
+            echo "$br" >> $NO_SOURCERPM_LOG
+         fi
+      else
+         echo "$br is obsolete"
+         echo "$br" >> $OBSOLETE_LOG
+      fi
+   fi
+done
+
+
+TMP_DIR=$(mktemp -d /tmp/copy_external_mirror_to_tis_mirror_XXXXXX)
+if [ $? -ne 0 ]; then
+   echo "Failed to create temporary directory"
+   return 1
+fi
+
+(cd $ORIG_DIR; find . | sort -V > $TMP_DIR/pre)
+(cd $NEW_DIR; find . | sort -V > $TMP_DIR/post)
+echo "Listing deletions"
+diff $TMP_DIR/pre $TMP_DIR/post | grep -v '^< ./Data/' | grep '^<'
+if [ $? -eq 0 ]; then
+   echo
+   echo "Cowardly refusing to alter $ORIG_DIR due to deletions: please see $NEW_DIR"
+   return 1
+fi
+
+mv -f $ORIG_DIR $ORIG_DIR.old
+mv -f $NEW_DIR $ORIG_DIR
+rm -rf $TMP_DIR
+return 0

build-tools/mirror_rebase/link_cgcs_centos_repo

@@ -0,0 +1,225 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
+#
+# This step updates Binary links, and adds Source links
+#
+
+MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
+BIN_ROOT=$MIRROR_ROOT/Binary
+SRC_ROOT=$MIRROR_ROOT/Source
+BLACK_LIST_FILE=$MIRROR_ROOT/Data/black_list.txt
+WHITE_LIST_FILE=$MIRROR_ROOT/Data/white_list.txt
+
+cd $MY_REPO/cgcs-centos-repo
+
+if [ $? -ne 0 ]; then
+   echo 'ERROR: failed to cd to $MY_REPO/cgcs-centos-repo'
+   return 1
+fi
+
+names=' '
+snames=' '
+
+do_work () {
+   dat=${1}
+
+   b=$(echo "$dat" | awk -F '#' '{ print $2 }')
+   d=$(echo "$dat" | awk -F '#' '{ print $3 }')
+   r="$d/$b"
+
+   DEBUG_INFO=0
+   MATCH=$(echo $b | grep '[-]debuginfo-')
+   if [ "$MATCH" != "" ]; then
+      DEBUG_INFO=1
+   fi
+
+   MATCH=$(grep "^$b" $BLACK_LIST_FILE || true)
+   if [ "$MATCH" != "" ]; then
+      echo "NOTE: '$b' is black listed"
+      continue
+   fi
+
+   if [ $DEBUG_INFO -eq 1 ]; then
+      sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM  : ' | sed 's#^Source RPM  : ##')
+      if [ "x$sb" == "x" ]; then
+         echo "ERROR: no source rpm listed for '$b'"
+         continue
+      fi
+      s=$(find Source -name "$sb")
+      if [ "x$s" == "x" ]; then
+         echo "NOTE: no source rpm '$sb' found for '$b'"
+         continue
+      fi
+   fi
+
+   name=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r)
+
+   MATCH=$(grep "^$b" $WHITE_LIST_FILE || true)
+   if [ "$MATCH" == "" ]; then
+      # Not white listed, check for python2 alternative
+      # python-rpm-macros-3-6.1 is a notable case white_list case... 
+      # We need BOTH python-rpm-macros-3-6.1 and python2-rpm-macros-3-6.1
+      #  so substituting python-rpm-macros-3-6.1 with python2-rpm-macros-3-6.1 is an error
+
+      altname=$(echo $name | sed 's#^python-#python2-#')
+      if [ "$altname" != "$name" ]; then
+         # look for python2 alternative
+         sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM  : ' | sed 's#^Source RPM  : ##')
+
+         if [ "x$sb" != "x" ]; then
+            MATCH=$(grep "^$sb" $BLACK_LIST_FILE || true)
+            if [ "$MATCH" != "" ]; then
+               echo "NOTE: '$sb' is black listed, so '$b' is ignored"
+               continue
+            fi
+         fi
+
+         for dat2 in $(for br in $(find $BIN_ROOT -name "$altname-*.rpm" | grep -v '.src.rpm$' | grep -v '[-]debuginfo-'); do
+                         ddd=$(dirname $br)
+                         bbb=$(basename $br)
+                         sss=$(echo $bbb | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+                         echo "$sss#$bbb#$ddd"
+                     done | sort -r -V)
+         do
+             b2=$(echo "$dat2" | awk -F '#' '{ print $2 }')
+             d2=$(echo "$dat2" | awk -F '#' '{ print $3 }')
+             r2="$d2/$b2"
+             name2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r2)
+             if [ "$name2" != "$altname" ]; then
+                continue
+             fi
+             sb2=$(rpm -q --info --nosignature -p $r2 | grep '^Source RPM  : ' | sed 's#^Source RPM  : ##')
+             # if [ "$sb" == "$sb2" ]; then
+             #   continue
+             # fi
+
+             sbs=$(echo $sb | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+             sbs2=$(echo $sb2 | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+             newer=$((echo $sbs; echo $sbs2) | sort -r -V | head -n 1)
+             if [ "$sbs" != "$sbs2" ]; then
+                if [ "$newer" == "$sbs2" ]; then
+                   # swap alternate for original
+                   for link in $(find Binary -name $b); do
+                      echo "SUGGEST: rm $link"
+                      git rm -f $link
+                   done
+   
+                   r=$r2
+                   name=$name2
+                   b=$b2
+                   break
+                fi
+             fi
+         done
+      fi
+   fi
+
+   echo "$names" | grep " $name " >> /dev/null
+   if [ $? -ne 0 ]; then
+      sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM  : ' | sed 's#^Source RPM  : ##')
+      if [ "x$sb" == "x" ]; then
+         echo "ERROR: no source rpm listed for '$b'"
+         continue
+      fi
+      s=$(find $SRC_ROOT -name "$sb")
+      if [ "x$s" == "x" ]; then
+         echo "ERROR: no source rpm '$sb' found for '$b'"
+         continue
+      fi
+
+      if [ "x$sb" != "x" ]; then
+         MATCH=$(grep "^$sb" $BLACK_LIST_FILE || true)
+         if [ "$MATCH" != "" ]; then
+            echo "NOTE: '$sb' is black listed, so '$b' is ignored"
+            continue
+         fi
+      fi
+
+      sname=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $s)
+
+      lb=$(find Binary -name "$b")
+      if [ "x$lb" == "x" ]; then
+          echo "MISSING: '$b'"
+          link=$(echo $r | sed "s#^$MIRROR_ROOT/##")
+          echo "SUGGEST: ln -s $r $link"
+          ln -s $r $link
+          git add $link
+      else
+          echo "OK: '$b'"
+      fi
+
+      for r2 in $(find Binary -name "$name-*.rpm"); do
+          b2=$(basename $r2)
+          if [ "$b" != "$b2" ]; then
+             name2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r2)
+             if [ "$name" == "$name2" ]; then
+                MATCH=$(grep "^$b2" $WHITE_LIST_FILE || true)
+                if [ "$MATCH" != "" ]; then
+                   link=$(echo $r2 | sed "s#^$MIRROR_ROOT/##")
+                   echo "SUGGEST: rm $link"
+                   git rm -f $link
+                fi
+             fi
+          fi
+      done
+
+      if [ $DEBUG_INFO -eq 0 ]; then
+         # Not a debuginfo therefore we can pull in new src.rpm
+         names="${names}${name} "
+         lsb=$(find Source -name "$sb")
+         if [ "x$lsb" == "x" ]; then
+             echo "MISSING: '$sb'"
+             link=$(echo $s | sed "s#^$MIRROR_ROOT/##")
+             echo "SUGGEST: ln -s $s $link"
+             ln -s $s $link
+             git add $link
+         else
+             echo "OK: '$sb'"
+         fi
+
+         echo "$names" | grep " $name " >> /dev/null
+         if [ $? -ne 0 ]; then
+            for s2 in $(find Source -name "$sname-*.rpm"); do
+                sb2=$(basename $s2)
+                if [ "$sb" != "$sb2" ]; then
+                   sname2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $s2)
+                   if [ "$sname" == "$sname2" ]; then
+                      MATCH=$(grep "^$sb2" $WHITE_LIST_FILE || true)
+                      if [ "$MATCH" != "" ]; then
+                         link=$(echo $s2 | sed "s#^$MIRROR_ROOT/##")
+                         echo "SUGGEST: rm $link"
+                         git rm -f $link
+                      fi
+                   fi
+                fi
+            done
+         fi
+         snames="${snames}${sname} "
+      fi
+   fi
+}
+
+
+for dat in $(for br in $(find $BIN_ROOT -name '*.rpm' | grep -v '.src.rpm$' | grep -v '[-]debuginfo-'); do
+              d=$(dirname $br)
+              b=$(basename $br)
+              s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+              echo "$s#$b#$d"
+            done | sort -r -V)
+do
+   do_work ${dat}
+done
+
+for dat in $(for br in $(find $BIN_ROOT -name '*.rpm' | grep -v '.src.rpm$' | grep '[-]debuginfo-'); do
+              d=$(dirname $br)
+              b=$(basename $br)
+              s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+              echo "$s#$b#$d"
+            done | sort -r -V)
+do
+   do_work ${dat}
+done

build-tools/mirror_rebase/link_cgcs_centos_repo_2

@@ -0,0 +1,82 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
+#
+# This step removes obsolete or broken Source links
+#
+
+MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
+BIN_ROOT=$MIRROR_ROOT/Binary
+SRC_ROOT=$MIRROR_ROOT/Source
+
+cd $MY_REPO/cgcs-centos-repo
+
+if [ $? -ne 0 ]; then
+   echo 'ERROR: failed to cd to $MY_REPO/cgcs-centos-repo'
+   return 1
+fi
+
+
+# Clean broken and obsolete srpm links
+snames=" "
+for dat in $(for br in $(find Source -name '*.src.rpm'); do
+              d=$(dirname $br)
+              b=$(basename $br)
+              s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+              echo "$s#$b#$d"
+            done | sort -r -V)
+do
+   sb=$(echo "$dat" | awk -F '#' '{ print $2 }')
+   d=$(echo "$dat" | awk -F '#' '{ print $3 }')
+   s="$d/$sb"
+
+   if [ ! -f $s ]; then
+      continue
+   fi
+
+   link=$(readlink $s)
+   if [ ! -f $link ]; then
+      echo "ERROR: '$sb' link to non-existant file '$link'"
+      echo "SUGGEST: rm $s"
+      git rm -f $s
+      continue
+   fi
+
+   echo $link | grep "$MIRROR_ROOT" >> /dev/null
+   if [ $? -ne 0 ]; then
+      echo "ERROR: '$sb' links to unexpected file '$link'"
+      echo "SUGGEST: rm $s"
+      git rm -f $s
+      continue
+   fi
+
+   sname=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $s)
+   echo "$snames" | grep " $sname " >> /dev/null
+   if [ $? -ne 0 ]; then
+      if [ "x$sname" != "x" ]; then
+          for s2 in $(find Source -name "$sname-*.src.rpm"); do
+              sb2=$(basename $s2)
+              if [ "$sb" != "$sb2" ]; then
+                 sname2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $s2)
+                 if [ "$sname" == "$sname2" ]; then
+                    link=$(echo $s2 | sed "s#^$MIRROR_ROOT/##")
+                    echo "SUGGEST: rm $link, due to $sb"
+                    git rm -f $link
+
+                    for r3 in $(find Binary -name "$sname-*.rpm"); do
+                       sb3=$(rpm -q --info --nosignature -p $r3 | grep '^Source RPM  : ' | sed 's#^Source RPM  : ##')
+                       if [ "$sb3" == "$sb2" ]; then
+                          echo "SUGGEST: rm $r3, due to $sb2"
+                       fi
+                    done
+                 fi
+              fi
+          done
+      fi
+      snames="${snames}${sname} "
+   fi
+done
+

build-tools/mirror_rebase/link_cgcs_centos_repo_3

@@ -0,0 +1,40 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
+#
+# This step removes broken Binary links
+#
+
+MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
+
+cd $MY_REPO/cgcs-centos-repo
+
+if [ $? -ne 0 ]; then
+   echo 'ERROR: failed to cd to $MY_REPO/cgcs-centos-repo'
+   return 1
+fi
+
+# Clean broken rpm links
+for r in $(find Binary -name '*.rpm' | grep -v '.src.rpm$' | sort -r -V); do
+   b=$(basename $r)
+
+   link=$(readlink $r)
+   if [ ! -f $link ]; then
+      echo "ERROR: '$b' link to non-existant file '$link'"
+      echo "SUGGEST: rm $r"
+      git rm -f $r
+      continue
+   fi
+
+   echo $link | grep "$MIRROR_ROOT" >> /dev/null
+   if [ $? -ne 0 ]; then
+      echo "ERROR: '$b' links to unexpected file '$link'"
+      echo "SUGGEST: rm $r"
+      git rm -f $r
+      continue
+   fi
+done
+

build-tools/mirror_rebase/link_cgcs_centos_repo_4

@@ -0,0 +1,176 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
+#
+# This step removes obsolete Binary links
+#
+
+cd $MY_REPO/cgcs-centos-repo
+MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
+BIN_ROOT=$MIRROR_ROOT/Binary
+SRC_ROOT=$MIRROR_ROOT/Source
+
+cd $MY_REPO/cgcs-centos-repo
+
+if [ $? -ne 0 ]; then
+   echo 'ERROR: failed to cd to $MY_REPO/cgcs-centos-repo'
+   return 1
+fi
+
+for dat in $(for br in $(find Binary -name '*.rpm' | grep -v '.src.rpm$' ); do
+              d=$(dirname $br)
+              b=$(basename $br)
+              s=$(echo $b | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+              echo "$s#$b#$d"
+            done | sort -r -V)
+do
+   b=$(echo "$dat" | awk -F '#' '{ print $2 }')
+   d=$(echo "$dat" | awk -F '#' '{ print $3 }')
+   r="$d/$b"
+   name=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r)
+
+   link=$(readlink $r)
+   sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM  : ' | sed 's#^Source RPM  : ##')
+   if [ "x$sb" == "x" ]; then
+      echo "ERROR: no source rpm listed for '$b'"
+      continue
+   fi
+   s=$(find Source -name "$sb")
+   if [ "x$s" == "x" ]; then
+      DELETED=0
+      altname=$(echo $name | sed 's#^python-#python2-#')
+      if [ "$altname" != "$name" ]; then
+         # look for python2 alternative
+
+         for dat2 in $(for br in $(find $BIN_ROOT -name "$altname-*.rpm" | grep -v '.src.rpm$' | grep -v '[-]debuginfo-'); do
+                         ddd=$(dirname $br)
+                         bbb=$(basename $br)
+                         sss=$(echo $bbb | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+                         echo "$sss#$bbb#$ddd"
+                     done | sort -r -V)
+         do
+             b2=$(echo "$dat2" | awk -F '#' '{ print $2 }')
+             d2=$(echo "$dat2" | awk -F '#' '{ print $3 }')
+             r2="$d2/$b2"
+
+             name2=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $r2)
+             if [ "$name2" != "$altname" ]; then
+                continue
+             fi
+             sb2=$(rpm -q --info --nosignature -p $r2 | grep '^Source RPM  : ' | sed 's#^Source RPM  : ##')
+             if [ "$sb" == "$sb2" ]; then
+                continue
+             fi
+
+             sbs=$(echo $sb | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+             sbs2=$(echo $sb2 | sed -e 's#.centos.#.#' -e 's#.el7.#.#'  -e 's#.el7_##' -e 's#.rpm$##' -e 's#.src$##' -e 's#.noarch$##' -e 's#.x86_64$##')
+             newer=$((echo $sbs; echo $sbs2) | sort -r -V | head -n 1)
+             if [ "$sbs" != "$sbs2" ]; then
+                if [ "$newer" == "$sbs2" ]; then
+                   # swap alternate for original
+                   echo "SUGGEST: rm $r"
+                   git rm -f $r
+   
+                   DELETED=1
+                fi
+             fi
+         done
+      fi
+
+      if [ $DELETED -eq 0 ]; then
+         echo "ERROR: no source rpm '$sb' found for '$b'"
+         echo "SUGGEST: rm $r"
+         git rm -f $r
+         continue
+      fi
+   fi
+done
+
+#
+# The following would delete all binary rpms that Titanium Cloud would otherwise compile.
+# However for bootstrapping the build there are some packages that we must have...
+#    e.g. bash, kernel-headers, ....
+# So we will need something smarter than to just delete everything.
+#
+# TMP_DIR=$(mktemp -d /tmp/link_cgcs_centos_repo_XXXXXX)
+#
+# BUILT_SRPMS_FILE=$TMP_DIR/built_srpms_file
+# 
+# for r in $(for c in $(find $MY_REPO -type d -name centos); do
+#               for sp in $(find $c -name srpm_path); do
+#                  echo "$sp: $(cat $sp)"
+#               done
+#            done | grep 'mirror:' | awk -F ' ' '{ print $2 }') ; do
+#    b=$(basename $r)
+#    s=$(find $MY_REPO/cgcs-centos-repo/Source/ -name $b)
+#    n=$(rpm -q --qf '%{NAME}' --nosignature -p $s)
+#    echo "$n:$b" >> $BUILT_SRPMS_FILE
+# done
+# 
+# cd $MY_REPO/cgcs-centos-repo
+# 
+# for r in $(find Binary -name '*.rpm'); do
+#    b=$(basename $r)
+#    sb=$(rpm -q --info --nosignature -p $r | grep '^Source RPM  :' | sed 's#^Source RPM  : ##')
+#    if [ "x$sb" != "x" ]; then
+#       s=$(find Source/ -name $sb)
+#       if [ "x$s" != "x" ]; then
+#          n=$(rpm -q --qf '%{NAME}' --nosignature -p $s)
+#          grep "^$n:" $BUILT_SRPMS_FILE
+#          if [ $? -eq 0 ]; then
+#             git rm -f $r
+#          fi
+#       fi
+#    fi
+# done
+# 
+# \rm $BUILT_SRPMS_FILE
+# rmdir $TMP_DIR
+
+TMP_DIR=$(mktemp -d /tmp/link_cgcs_centos_repo_XXXXXX)
+EXCLUDE_LIST=$TMP_DIR/exclude_list
+
+# List od packages we compile from scratch, not from centos srpm, goes to $EXCLUDE_LIST
+for g in $(find $MY_REPO -type d -name .git); do 
+   d=$(dirname $g)
+   for cpd in $(find $d -maxdepth 1 -name 'centos_pkg_dir*'); do 
+      (
+       cd $d
+       for pd in $(cat $cpd); do 
+          (
+           cd $pd/centos
+           if [ -f srpm_path ]; then 
+              continue
+           fi
+           for spec in $(find . -name '*.spec'); do 
+              n=$(spec_find_tag 'name' $spec '/tmp/' '0' 2> /dev/null)
+              echo "$spec: $n"
+              echo $n >> $EXCLUDE_LIST
+           done
+          )
+       done
+      )
+   done
+done
+
+cd $MY_REPO/cgcs-centos-repo
+for r in $(find Binary  -name '*.rpm'); do 
+   s=$(rpm -q --info --nosignature -p $r | grep '^Source RPM' | sed 's#^Source RPM ..##')
+   n=$(rpm -q --nosignature --qf '%{NAME}' -p $(find Source -name $s) )
+   grep "^$n$" $EXCLUDE_LIST
+   if [ $? -eq 0 ]; then 
+      echo "rm $r"
+      git rm $r
+   fi
+done
+
+\rm -r $TMP_DIR
+
+echo "REMINDER: inspect changes in $MY_REPO/cgcs-centos-repo and commit with ..."
+echo '   TIMESTAMP=$(date +"%Y-%m-%d")'
+echo '   git commit -m "JENKINS: repo update $TIMESTAMP"'
+
+

build-tools/mirror_rebase/link_cgcs_centos_repo_5

@@ -0,0 +1,94 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
+#
+# Search for tis patched centos src.rpm's that have been upversioned
+#
+
+cd $MY_REPO/cgcs-centos-repo
+# OLD_MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/mitaka
+OLD_MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
+# MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/newton
+OLD_THIRD_MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/tis-r4-3rd-Party
+# THIRD_MIRROR_ROOT=/import/mirrors/CentOS/tis-r5-CentOS/tis-r4-3rd-Party
+# BIN_ROOT=$MIRROR_ROOT/Binary
+# SRC_ROOT=$MIRROR_ROOT/Source
+UPVERSION_LOG=$MY_WORKSPACE/upversion.log
+
+REPO_DOWNLOADS_ROOT="$MY_REPO"
+NEW_MIRROR_ROOT="$MY_REPO/cgcs-centos-repo"
+THIRD_PARTY_ROOT="$MY_REPO/cgcs-3rd-party-repo"
+
+
+if [ -f $UPVERSION_LOG ]; then
+   rm -f $UPVERSION_LOG
+fi
+
+cd $MY_REPO
+
+for g in $(find $MY_REPO -type d -name .git); do 
+   d=$(dirname $g)
+   for pf in $(find $d -maxdepth 1 -name 'centos_pkg_dirs*'); do 
+   if [ -f $pf ]; then
+      for p in $(cat $pf); do
+         pkg_dir="$d/$p"
+         sf="$pkg_dir/centos/srpm_path"
+         if [ -f  $sf ]; then
+            for s in $(grep  '^[^#]' $sf); do
+               ORIG_SRPM_PATH=""
+               # absolute path source rpms
+               echo "$s" | grep "^/" >/dev/null && ORIG_SRPM_PATH=$s
+
+               if [ "${ORIG_SRPM_PATH}x" == "x" ]; then
+                  # handle repo: definitions
+                  echo "$s" | grep "^repo:" >/dev/null && ORIG_SRPM_PATH=$(echo $s | sed "s%^repo:%$REPO_DOWNLOADS_ROOT/%")
+               fi
+
+               if [ "${ORIG_SRPM_PATH}x" == "x" ]; then
+                  # handle 3rd_party: definitions
+                  echo "$s" | grep "^3rd_party:" >/dev/null && ORIG_SRPM_PATH=$(echo $s | sed "s%^3rd_party:%$THIRD_PARTY_ROOT/%")
+               fi
+
+               if [ "${ORIG_SRPM_PATH}x" == "x" ]; then
+                  # handle mirror: definitions
+                  # SAL TEMPORARY echo "$s" | grep "^mirror:" >/dev/null && ORIG_SRPM_PATH=`echo $s | sed "s%^mirror:%$MIRROR_ROOT/%"`
+                  echo "$s" | grep "^mirror:" >/dev/null && ORIG_SRPM_PATH=$(echo $s | sed "s%^mirror:%$NEW_MIRROR_ROOT/%" | sed "s#CentOS/tis-r4-CentOS/kilo/##" | sed "s#CentOS/tis-r4-CentOS/mitaka/##" | sed "s#CentOS/tis-r4-CentOS/newton/##")
+               fi
+
+               if [ "${ORIG_SRPM_PATH}x" == "x" ]; then
+                  # we haven't found a valid prefix yet, so assume it's a legacy
+                  # file (mirror: interpretation)
+                  ORIG_SRPM_PATH="$NEW_MIRROR_ROOT/$s"
+               fi
+
+               if [ ! -f $ORIG_SRPM_PATH ]; then
+                  b=$(basename "$ORIG_SRPM_PATH")
+                  old_srpm=$(find $OLD_MIRROR_ROOT $OLD_THIRD_MIRROR_ROOT -name $b | head -n 1)
+                  old_name=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $old_srpm)
+                  if [ "$old_name" == "" ]; then
+                     echo "FAILED to find name for '$b', ORIG_SRPM_PATH='$ORIG_SRPM_PATH'"
+                     exit 1
+                  fi
+                  NEW_SRPM_PATH=""
+                  for new_srpm in $(find $NEW_MIRROR_ROOT/Source $THIRD_PARTY_ROOT/Source -name "$old_name-[0-9]*.src.rpm"); do
+                      new_name=$(rpm -q --nosignature --queryformat '%{NAME}\n' -p $new_srpm)
+                      if [ "$new_name" == "$old_name" ]; then
+                         NEW_SRPM_PATH=$new_srpm
+                         break
+                      fi
+                  done
+                  nb=$(basename $NEW_SRPM_PATH)
+                  echo "FIX: '$sf' : '$b' -> '$nb'"
+                  echo "$old_name#$sf#$s#$b#$nb" >> $UPVERSION_LOG
+               fi
+
+            done
+         fi
+      done
+   fi
+   done
+done
+

build-tools/mirror_rebase/link_cgcs_centos_repo_6

@@ -0,0 +1,91 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
+#
+# start an edit session for packages to be upgraded - pre upgrade version
+#
+
+UPVERSION_LOG=$MY_WORKSPACE/upversion.log
+if [ "x$WORKING_BRANCH" == "x" ]; then
+   WORKING_BRANCH=CGCS_DEV_0029_rebase_7_4
+fi
+
+if [ ! -f $UPVERSION_LOG ]; then
+    echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
+fi
+
+
+# One step back to see the old symlinks
+cd $MY_REPO/cgcs-3rd-party-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$(pwd)'"
+   exit 1
+fi
+
+git checkout HEAD^
+
+cd $MY_REPO/cgcs-centos-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$(pwd)'"
+   exit 1
+fi
+
+git checkout HEAD^
+
+#
+#
+#
+FAILED=""
+for dat in $(cat $UPVERSION_LOG); do
+   name=$(echo $dat | awk -F '#' '{print $1}')
+   srpm_path=$(echo $dat | awk -F '#' '{print $2}')
+   old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
+   new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
+
+   echo "$name  $old_src_rpm  $new_src_rpm"
+
+   if [ "$name" == "kernel" ]; then
+      build-pkgs --std --edit --clean $name
+   elif [ "$name" == "kernel-rt" ]; then
+      build-pkgs --rt --edit --clean $name
+   else
+      build-pkgs --edit --clean $name
+   fi
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed cmd 'build-pkgs --edit --clean $name'"
+      FAILED="$name $FAILED"
+      break
+   fi
+   echo "$?   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
+   if [ "$name" == "kernel" ]; then
+      build-pkgs --std --edit $name
+   elif [ "$name" == "kernel-rt" ]; then
+      build-pkgs --rt --edit $name
+   else
+      build-pkgs --edit $name
+   fi
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed cmd 'build-pkgs --edit $name'"
+      FAILED="$name $FAILED"
+      break
+   fi
+   echo "$?   <=<=<=<=<=<=<=<=<=<=<=<=<=<=<=<="
+done
+
+cd $MY_REPO/cgcs-3rd-party-repo
+git checkout $WORKING_BRANCH
+
+cd $MY_REPO/cgcs-centos-repo
+git checkout $WORKING_BRANCH
+
+if [ "$FAILED" != "" ]; then
+   echo "Failed build-pkgs --edit for ... $FAILED"
+   exit 1
+fi
+
+

build-tools/mirror_rebase/link_cgcs_centos_repo_7

@@ -0,0 +1,84 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
+#
+# Update srpm_path for packages to be upgraded
+#
+
+UPVERSION_LOG=$MY_WORKSPACE/upversion.log
+if [ "x$ORIGIN_BRANCH" == "x" ]; then
+   ORIGIN_BRANCH=CGCS_DEV_0029
+fi
+if [ "x$WORKING_BRANCH" == "x" ]; then
+   WORKING_BRANCH=CGCS_DEV_0029_rebase_7_4
+fi
+
+if [ ! -f $UPVERSION_LOG ]; then
+    echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
+fi
+
+
+# One step back to see the old symlinks
+cd $MY_REPO
+
+FAILED=""
+for dat in $(cat $UPVERSION_LOG); do
+   name=$(echo $dat | awk -F '#' '{print $1}')
+   srpm_path=$(echo $dat | awk -F '#' '{print $2}')
+   old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
+   new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
+
+   (
+   cd $(dirname $srpm_path)
+   CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
+   if [ "$CURRENT_BRANCH" != "$WORKING_BRANCH" ]; then
+      git checkout $WORKING_BRANCH
+      if [ $? -ne 0 ]; then
+         git checkout $ORIGIN_BRANCH
+         if [ $? -ne 0 ]; then
+            echo "ERROR: Can't checkout branch '$ORIGIN_BRANCH' in directory '$(pwd)'"
+            exit 1
+         fi
+
+         git checkout -b $WORKING_BRANCH
+         if [ $? -ne 0 ]; then
+            echo "ERROR: failed to 'git checkout -b $WORKING_BRANCH' from '$(pwd)'"
+            exit 1
+         else
+            echo "created branch '$WORKING_BRANCH' at '$(pwd)'"
+         fi
+      fi
+   fi
+
+   sed -i "s#$old_src_rpm#$new_src_rpm#" $srpm_path
+   if [ $? -ne 0 ]; then
+      echo "ERROR: sed failed '$old_src_rpm' -> '$new_src_rpm'"
+      exit 1
+   else
+      echo "updated $srpm_path: '$old_src_rpm' -> '$new_src_rpm'"
+   fi
+
+   exit 0
+   )
+
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed while working on package '$name' at '$srpm_path'"
+      exit 1
+   fi
+done
+
+echo ""
+for d in $(for dat in $(cat $UPVERSION_LOG); do srpm_path=$(echo $dat | awk -F '#' '{print $2}'); ( cd $(dirname $srpm_path); git rev-parse --show-toplevel ); done | sort --unique); do
+   (
+    cd $d
+    echo "cd $d"
+    for f in $(git status --porcelain | grep 'srpm_path$' | awk '{print $2}'); do 
+        echo "git add $f";
+    done
+    echo "git commit -m 'srpm_path updates for centos rebase'"
+   )
+done
+echo ""

build-tools/mirror_rebase/link_cgcs_centos_repo_8

@@ -0,0 +1,65 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
+#
+# Start an edit session for packages to be upgraded - post upgrade version
+#
+
+UPVERSION_LOG=$MY_WORKSPACE/upversion.log
+if [ "x$WORKING_BRANCH" == "x" ]; then
+   WORKING_BRANCH=CGCS_DEV_0029_rebase_7_4
+fi
+
+if [ ! -f $UPVERSION_LOG ]; then
+    echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
+fi
+
+
+# Restore new symlinks
+cd $MY_REPO/cgcs-3rd-party-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-3rd-party-repo'"
+   exit 1
+fi
+
+cd $MY_REPO/cgcs-centos-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
+   exit 1
+fi
+
+FAILED=""
+for dat in $(cat $UPVERSION_LOG); do
+   name=$(echo $dat | awk -F '#' '{print $1}')
+   srpm_path=$(echo $dat | awk -F '#' '{print $2}')
+   old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
+   new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
+
+   echo "$name  $old_src_rpm  $new_src_rpm"
+
+   if [ "$name" == "kernel" ]; then
+      build-pkgs --std --edit $name --no-meta-patch
+   elif [ "$name" == "kernel-rt" ]; then
+      build-pkgs --rt --edit $name --no-meta-patch
+   else
+      build-pkgs --edit $name --no-meta-patch
+   fi
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed cmd 'build-pkgs --edit $name'"
+      FAILED="$name $FAILED"
+      break
+   fi
+   echo "$?   <=<=<=<=<=<=<=<=<=<=<=<=<=<=<=<="
+done
+
+if [ "$FAILED" != "" ]; then
+   echo "Failed build-pkgs --edit for ... $FAILED"
+   exit 1
+fi
+
+

build-tools/mirror_rebase/link_cgcs_centos_repo_9

@@ -0,0 +1,346 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r5-CentOS/newton
+#
+# Search for tis patched centos src.rpm's that have been upversioned
+#
+
+UPVERSION_LOG=$MY_WORKSPACE/upversion.log
+if [ "x$WORKING_BRANCH" == "x" ]; then
+   WORKING_BRANCH=CGCS_DEV_0029_rebase_7_4
+fi
+
+if [ ! -f $UPVERSION_LOG ]; then
+    echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
+fi
+
+if [ "$DISPLAY" == "" ]; then
+   echo "ERROR: X-Windows 'DISPLAY' variable not set. This script needs to open pop-up windows."
+   usage
+   exit 1
+fi
+
+# restore new symlinks
+cd $MY_REPO/cgcs-3rd-party-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-3rd-party-repo'"
+   exit 1
+fi
+
+cd $MY_REPO/cgcs-centos-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
+   exit 1
+fi
+
+FAILED=""
+build_types="std rt"
+for dat in $(cat $UPVERSION_LOG); do
+   (
+   name=$(echo $dat | awk -F '#' '{print $1}')
+   srpm_path=$(echo $dat | awk -F '#' '{print $2}')
+   old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
+   new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
+
+   PKG_DIR=$(dirname $(dirname $srpm_path))
+   OLD_BRANCH=$(echo $old_src_rpm | sed 's#[.]src[.]rpm$##')
+   NEW_BRANCH=$(echo $new_src_rpm | sed 's#[.]src[.]rpm$##')
+   
+   WORK_META_DIR=""
+   for dd in $build_types; do
+      WORK_META_DIR=$MY_WORKSPACE/$dd/srpm_work/$name/rpmbuild
+      echo "WORK_META_DIR=$WORK_META_DIR"
+      if [ -d $WORK_META_DIR ]; then
+         break;
+      else
+         WORK_META_DIR=""
+      fi
+   done
+   if [ "$WORK_META_DIR" == "" ]; then
+      echo "ERROR: failed to find srpm_work directory for '$name'"
+      exit 1
+   fi
+
+   # WORK_SRC_DIR=$(dirname $(find $MY_WORKSPACE/srpm_work/$name/gits/ -type d -name .git))
+   NEW_WORK_SRC_DIR=""
+   OLD_WORK_SRC_DIR=""
+   for dd in $build_types; do
+      for g in $(find $MY_WORKSPACE/$dd/srpm_work/$name/gits/ -type d -name .git); do
+         d=$(dirname $g)
+         if [ -d $d ]; then
+            cd $d;
+            git tag | grep pre_wrs_ >> /dev/null
+            if [ $? -ne 0 ]; then
+               continue
+            fi
+            git checkout $OLD_BRANCH 2>> /dev/null
+            if [ $? -eq 0 ]; then
+               OLD_WORK_SRC_DIR=$d
+            fi
+            git checkout $NEW_BRANCH  2>> /dev/null
+            if [ $? -eq 0 ]; then
+               NEW_WORK_SRC_DIR=$d
+            fi
+         fi
+      done
+   done
+   if [ "$WORK_META_DIR" == "" ]; then
+      echo "ERROR: failed to find srpm_work directory for '$name'"
+      exit 1
+   fi
+
+   echo "$name  $old_src_rpm  $new_src_rpm"
+   echo "PKG_DIR=$PKG_DIR"
+   echo "OLD_BRANCH=$OLD_BRANCH"
+   echo "NEW_BRANCH=$NEW_BRANCH"
+   echo "WORK_META_DIR=$WORK_META_DIR"
+   echo "OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
+   echo "NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
+   echo ""
+
+   (
+   cd $WORK_META_DIR
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to cd to WORK_META_DIR=$WORK_META_DIR"
+      exit 1
+   fi
+   echo "--- old meta git log (oldest to newest) ---"
+   git checkout $OLD_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH"
+      exit 1
+   fi
+   (git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit ; echo "") | sed '/^$/d' | tac
+   PATCH_COMMIT_LIST=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
+   echo "--- new meta git log (oldest to newest) ---"
+   git checkout $NEW_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH"
+      exit 1
+   fi
+   (git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit ; echo "") | sed '/^$/d' | tac
+   REFERENCE_COMMIT=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | head -n 1 | awk '{ print $2 }')
+   echo ""
+
+   for COMMIT in ${PATCH_COMMIT_LIST}; do
+      echo "git cherry-pick $COMMIT"
+      git --no-pager show "$COMMIT"
+      git cherry-pick "$COMMIT"
+      if [ $? -ne 0 ]; then
+         echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files" 
+         git status --porcelain | grep '^UU ' | awk '{ print $2}'
+         echo "pwd=$(pwd)"
+         # gitk &
+         echo "git mergetool --no-prompt"
+         git mergetool --no-prompt
+         # for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
+         #    xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
+         #    if [ $? -ne 0 ]; then
+         #       echo "ERROR: problem launching editor on "
+         #       exit 1
+         #    fi
+         # done
+         echo "git cherry-pick --continue"
+         git cherry-pick --continue
+      fi
+   done
+
+   PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
+      exit 1
+   fi
+   for PATCH_FILE in ${PATCH_LIST}; do
+      PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
+      echo "$PATCH_FILE -> $PATCH_TARGET"
+      N=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*" | wc -l)
+      if [ $N -eq 1 ]; then
+          PATCH_DEST=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*")
+          echo "cp -f $PATCH_FILE $PATCH_DEST"
+          \cp -f  $PATCH_FILE  $PATCH_DEST
+          if [ $? -ne 0 ]; then
+             echo "ERROR: copy failed $WORK_META_DIR/$PATCH_FILE -> $PATCH_DEST"
+             exit 1
+          fi
+      else
+          echo "ERROR: Don't know what destination file name to use for patch '$WORK_META_DIR/$PATCH_FILE' derived from commit $COMMIT, and to be copied to '$PKG_DIR/centos/meta_patches'"
+      fi
+   done
+
+   echo ""
+   echo ""
+   )
+
+   if [ $? -ne 0 ]; then
+      FAILED=$name
+      break
+   fi
+
+   (
+   echo "--- old git log (oldest to newest) ---"
+   cd $OLD_WORK_SRC_DIR
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to cd to OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
+      exit 1
+   fi
+
+   git checkout $OLD_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH in directory '$OLD_WORK_SRC_DIR'"
+      exit 1
+   fi
+
+   (git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit; echo "") | sed '/^$/d' | tac
+   PATCH_COMMIT_LIST=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit; echo "") | sed '/^$/d' | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
+
+   echo "--- new git log (oldest to newest) ---"
+   cd $NEW_WORK_SRC_DIR
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to cd to NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
+      exit 1
+   fi
+
+   git checkout $NEW_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH in directory '$NEW_WORK_SRC_DIR'"
+      exit 1
+   fi
+
+   (git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit; echo "") | sed '/^$/d' | tac
+   REFERENCE_COMMIT=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | head -n 1 | awk '{ print $2 }')
+   echo ""
+
+   if [ "$OLD_WORK_SRC_DIR" == "$NEW_WORK_SRC_DIR" ]; then
+      for COMMIT in ${PATCH_COMMIT_LIST}; do
+         echo "git cherry-pick $COMMIT"
+         git --no-pager show "$COMMIT"
+         git cherry-pick "$COMMIT"
+         if [ $? -ne 0 ]; then
+            echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files" 
+            git status --porcelain | grep '^UU ' | awk '{ print $2}'
+            echo "pwd=$(pwd)"
+            # gitk &
+            echo "git mergetool --no-prompt"
+            git mergetool --no-prompt
+            # for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
+            #    xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
+            #    if [ $? -ne 0 ]; then
+            #       echo "ERROR: problem launching editor on "
+            #       exit 1
+            #    fi
+            # done
+            echo "git cherry-pick --continue"
+            git cherry-pick --continue
+         fi
+      done
+   else
+      cd $OLD_WORK_SRC_DIR
+      PATCH_LIST=$(git format-patch -n pre_wrs_$OLD_BRANCH)
+      if [ $? -ne 0 ]; then
+         echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=pre_wrs_$OLD_BRANCH"
+         exit 1
+      fi
+      cd $NEW_WORK_SRC_DIR
+      for PATCH_FILE in ${PATCH_LIST}; do
+         echo "=== Apply $PATCH_FILE ==="
+         cat $OLD_WORK_SRC_DIR/$PATCH_FILE
+         cat $OLD_WORK_SRC_DIR/$PATCH_FILE | patch -p1
+         if [ $? -ne 0 ]; then
+            for REJECT in $(find . -name '*.rej'); do
+               FILE_NAME=$(echo $REJECT | sed 's#.rej$##')
+               cd $OLD_WORK_SRC_DIR
+               gitk $FILE_NAME &
+               cd $NEW_WORK_SRC_DIR
+               if [ -f $FILE_NAME ] && [ -f $FILE_NAME.orig ]; then
+                  \cp -f $FILE_NAME.orig $FILE_NAME
+                  xterm -e "vi $FILE_NAME $REJECT"
+                  rm -f $REJECT
+                  rm -f $FILE_NAME.orig
+               fi
+            done
+         fi
+
+         git add --all
+         MSG=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
+         git commit -m "WRS: $MSG"
+      done
+      
+   fi
+
+   PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
+      exit 1
+   fi
+   for PATCH_FILE in ${PATCH_LIST}; do
+      PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch[0-9]*-//' | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch//' | sed 's/.patch$//')
+      echo "$PATCH_FILE -> $PATCH_TARGET"
+      PKG_PATCH_DIR="$PKG_DIR/centos/patches"
+      N=0
+      if [ -d  "$PKG_PATCH_DIR" ]; then
+         N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
+      fi
+      if [ $N -ne 1 ]; then
+         PKG_PATCH_DIR="$PKG_DIR"
+         if [ -d  "$PKG_PATCH_DIR" ]; then
+            N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
+         fi
+      fi
+      echo "N=$N"
+      echo "PKG_PATCH_DIR=$PKG_PATCH_DIR"
+
+      if [ $N -eq 1 ]; then
+          PATCH_DEST=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]')
+          echo "meld $PATCH_FILE -> $PATCH_DEST"
+          meld  $PATCH_FILE  $PATCH_DEST
+          if [ $? -ne 0 ]; then
+             echo "ERROR: meld failed $WORK_SRC_DIR/$PATCH_FILE -> $PATCH_DEST"
+             exit 1
+          fi
+      else
+          echo "ERROR: Don't know what destination file name to use for patch '$OLD_WORK_SRC_DIR/$PATCH_FILE', and to be copied to '$PKG_PATCH_DIR'"
+      fi
+   done
+
+   echo ""
+   echo ""
+   )
+
+   if [ $? -ne 0 ]; then
+      FAILED=$name
+      break
+   fi
+
+   )
+
+
+done
+
+if [ "$FAILED" != "" ]; then
+   echo "Failed for ... $FAILED"
+   exit 1
+fi
+
+echo ""
+for d in $(for dat in $(cat $UPVERSION_LOG); do srpm_path=$(echo $dat | awk -F '#' '{print $2}'); ( cd $(dirname $srpm_path); git rev-parse --show-toplevel ); done | sort --unique); do
+   (
+    cd $d
+    echo "cd $d"
+    for f in $(git status --porcelain | awk '{print $2}'); do 
+        echo "git add $f"; 
+    done
+    if [ "$PATCH_ID" == "" ]; then
+       echo "git commit -m 'rebased patches'"
+    else
+       echo "git commit -m 'rebased patches'"
+    fi
+   )
+done
+echo ""
+
+

build-tools/mirror_rebase/tarball_upgrade

@@ -0,0 +1,330 @@
+#!/bin/bash
+
+#
+# Part of the monthly mirror update
+#
+# Update symlinks in cgcs-centos-repo to point to the latest version of packages in /import/mirrors/CentOS/tis-r4-CentOS/newton
+#
+# Search for tis patched centos src.rpm's that have been upversioned
+#
+
+UPVERSION_LOG=$MY_WORKSPACE/upversion.log
+if [ "x$WORKING_BRANCH" == "x" ]; then
+   WORKING_BRANCH=CGCS_DEV_0026_may_rebase
+fi
+
+if [ ! -f $UPVERSION_LOG ]; then
+    echo "ERROR: Can't find UPVERSION_LOG at '$UPVERSION_LOG'"
+fi
+
+if [ "$DISPLAY" == "" ]; then
+   echo "ERROR: X-Windows 'DISPLAY' variable not set. This script needs to open pop-up windows."
+   usage
+   exit 1
+fi
+
+# restore new symlinks
+cd $MY_REPO/cgcs-3rd-party-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-3rd-party-repo'"
+   exit 1
+fi
+
+cd $MY_REPO/cgcs-centos-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
+   exit 1
+fi
+
+FAILED=""
+build_types="std rt"
+for dat in $(cat $UPVERSION_LOG); do
+   (
+   name=$(echo $dat | awk -F '#' '{print $1}')
+   srpm_path=$(echo $dat | awk -F '#' '{print $2}')
+   old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
+   new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
+
+   PKG_DIR=$(dirname $(dirname $srpm_path))
+   OLD_BRANCH=$(echo $old_src_rpm | sed 's#[.]src[.]rpm$##')
+   NEW_BRANCH=$(echo $new_src_rpm | sed 's#[.]src[.]rpm$##')
+   
+   WORK_META_DIR=""
+   for dd in $build_types; do
+      WORK_META_DIR=$MY_WORKSPACE/$dd/srpm_work/$name/rpmbuild
+      echo "WORK_META_DIR=$WORK_META_DIR"
+      if [ -d $WORK_META_DIR ]; then
+         break;
+      else
+         WORK_META_DIR=""
+      fi
+   done
+   if [ "$WORK_META_DIR" == "" ]; then
+      echo "ERROR: failed to find srpm_work directory for '$name'"
+      exit 1
+   fi
+
+   # WORK_SRC_DIR=$(dirname $(find $MY_WORKSPACE/srpm_work/$name/gits/ -type d -name .git))
+   NEW_WORK_SRC_DIR=""
+   OLD_WORK_SRC_DIR=""
+   for dd in $build_types; do
+      for g in $(find $MY_WORKSPACE/$dd/srpm_work/$name/gits/ -type d -name .git); do
+         d=$(dirname $g)
+         if [ -d $d ]; then
+            cd $d;
+            git tag | grep pre_wrs_ >> /dev/null
+            if [ $? -ne 0 ]; then
+               continue
+            fi
+            git checkout $OLD_BRANCH 2>> /dev/null
+            if [ $? -eq 0 ]; then
+               OLD_WORK_SRC_DIR=$d
+            fi
+            git checkout $NEW_BRANCH  2>> /dev/null
+            if [ $? -eq 0 ]; then
+               NEW_WORK_SRC_DIR=$d
+            fi
+         fi
+      done
+   done
+   if [ "$WORK_META_DIR" == "" ]; then
+      echo "ERROR: failed to find srpm_work directory for '$name'"
+      exit 1
+   fi
+
+   echo "$name  $old_src_rpm  $new_src_rpm"
+   echo "PKG_DIR=$PKG_DIR"
+   echo "OLD_BRANCH=$OLD_BRANCH"
+   echo "NEW_BRANCH=$NEW_BRANCH"
+   echo "WORK_META_DIR=$WORK_META_DIR"
+   echo "OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
+   echo "NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
+   echo ""
+
+   (
+   cd $WORK_META_DIR
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to cd to WORK_META_DIR=$WORK_META_DIR"
+      exit 1
+   fi
+   echo "--- old meta git log (oldest to newest) ---"
+   git checkout $OLD_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH"
+      exit 1
+   fi
+   (git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit ; echo "") | sed '/^$/d' | tac
+   PATCH_COMMIT_LIST=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
+   echo "--- new meta git log (oldest to newest) ---"
+   git checkout $NEW_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH"
+      exit 1
+   fi
+   (git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit ; echo "") | sed '/^$/d' | tac
+   REFERENCE_COMMIT=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | head -n 1 | awk '{ print $2 }')
+   echo ""
+
+   for COMMIT in ${PATCH_COMMIT_LIST}; do
+      echo "git cherry-pick $COMMIT"
+      git --no-pager show "$COMMIT"
+      git cherry-pick "$COMMIT"
+      if [ $? -ne 0 ]; then
+         echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files" 
+         git status --porcelain | grep '^UU ' | awk '{ print $2}'
+         echo "pwd=$(pwd)"
+         # gitk &
+         echo "git mergetool --no-prompt"
+         git mergetool --no-prompt
+         # for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
+         #    xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
+         #    if [ $? -ne 0 ]; then
+         #       echo "ERROR: problem launching editor on "
+         #       exit 1
+         #    fi
+         # done
+         echo "git cherry-pick --continue"
+         git cherry-pick --continue
+      fi
+   done
+
+   PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
+      exit 1
+   fi
+   for PATCH_FILE in ${PATCH_LIST}; do
+      PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
+      echo "$PATCH_FILE -> $PATCH_TARGET"
+      N=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*" | wc -l)
+      if [ $N -eq 1 ]; then
+          PATCH_DEST=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*")
+          echo "cp -f $PATCH_FILE $PATCH_DEST"
+          \cp -f  $PATCH_FILE  $PATCH_DEST
+          if [ $? -ne 0 ]; then
+             echo "ERROR: copy failed $WORK_META_DIR/$PATCH_FILE -> $PATCH_DEST"
+             exit 1
+          fi
+      else
+          echo "ERROR: Don't know what destination file name to use for patch '$WORK_META_DIR/$PATCH_FILE' derived from commit $COMMIT, and to be copied to '$PKG_DIR/centos/meta_patches'"
+      fi
+   done
+
+   echo ""
+   echo ""
+   )
+
+   if [ $? -ne 0 ]; then
+      FAILED=$name
+      break
+   fi
+
+   (
+   echo "--- old git log (oldest to newest) ---"
+   cd $OLD_WORK_SRC_DIR
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to cd to OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
+      exit 1
+   fi
+
+   git checkout $OLD_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH in directory '$OLD_WORK_SRC_DIR'"
+      exit 1
+   fi
+
+   (git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit; echo "") | sed '/^$/d' | tac
+   PATCH_COMMIT_LIST=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit; echo "") | sed '/^$/d' | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
+
+   echo "--- new git log (oldest to newest) ---"
+   cd $NEW_WORK_SRC_DIR
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to cd to NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
+      exit 1
+   fi
+
+   git checkout $NEW_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH in directory '$NEW_WORK_SRC_DIR'"
+      exit 1
+   fi
+
+   (git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit; echo "") | sed '/^$/d' | tac
+   REFERENCE_COMMIT=$((git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit ; echo "") | sed '/^$/d' | head -n 1 | awk '{ print $2 }')
+   echo ""
+
+   if [ "$OLD_WORK_SRC_DIR" == "$NEW_WORK_SRC_DIR" ]; then
+      for COMMIT in ${PATCH_COMMIT_LIST}; do
+         echo "git cherry-pick $COMMIT"
+         git --no-pager show "$COMMIT"
+         git cherry-pick "$COMMIT"
+         if [ $? -ne 0 ]; then
+            echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files" 
+            git status --porcelain | grep '^UU ' | awk '{ print $2}'
+            echo "pwd=$(pwd)"
+            # gitk &
+            echo "git mergetool --no-prompt"
+            git mergetool --no-prompt
+            # for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
+            #    xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
+            #    if [ $? -ne 0 ]; then
+            #       echo "ERROR: problem launching editor on "
+            #       exit 1
+            #    fi
+            # done
+            echo "git cherry-pick --continue"
+            git cherry-pick --continue
+         fi
+      done
+   else
+      cd $OLD_WORK_SRC_DIR
+      PATCH_LIST=$(git format-patch -n pre_wrs_$OLD_BRANCH)
+      if [ $? -ne 0 ]; then
+         echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=pre_wrs_$OLD_BRANCH"
+         exit 1
+      fi
+      cd $NEW_WORK_SRC_DIR
+      for PATCH_FILE in ${PATCH_LIST}; do
+         echo "=== Apply $PATCH_FILE ==="
+         cat $OLD_WORK_SRC_DIR/$PATCH_FILE
+         cat $OLD_WORK_SRC_DIR/$PATCH_FILE | patch -p1
+         if [ $? -ne 0 ]; then
+            for REJECT in $(find . -name '*.rej'); do
+               FILE_NAME=$(echo $REJECT | sed 's#.rej$##')
+               cd $OLD_WORK_SRC_DIR
+               gitk $FILE_NAME &
+               cd $NEW_WORK_SRC_DIR
+               if [ -f $FILE_NAME ] && [ -f $FILE_NAME.orig ]; then
+                  \cp -f $FILE_NAME.orig $FILE_NAME
+                  xterm -e "vi $FILE_NAME $REJECT"
+                  rm -f $REJECT
+                  rm -f $FILE_NAME.orig
+               fi
+            done
+         fi
+
+         git add --all
+         MSG=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
+         git commit -m "WRS: $MSG"
+      done
+      
+   fi
+
+   PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
+      exit 1
+   fi
+   for PATCH_FILE in ${PATCH_LIST}; do
+      PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch[0-9]*-//' | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch//' | sed 's/.patch$//')
+      echo "$PATCH_FILE -> $PATCH_TARGET"
+      PKG_PATCH_DIR="$PKG_DIR/centos/patches"
+      N=0
+      if [ -d  "$PKG_PATCH_DIR" ]; then
+         N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
+      fi
+      if [ $N -ne 1 ]; then
+         PKG_PATCH_DIR="$PKG_DIR"
+         if [ -d  "$PKG_PATCH_DIR" ]; then
+            N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
+         fi
+      fi
+      echo "N=$N"
+      echo "PKG_PATCH_DIR=$PKG_PATCH_DIR"
+
+      if [ $N -eq 1 ]; then
+          PATCH_DEST=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]')
+          echo "meld $PATCH_FILE -> $PATCH_DEST"
+          meld  $PATCH_FILE  $PATCH_DEST
+          if [ $? -ne 0 ]; then
+             echo "ERROR: meld failed $WORK_SRC_DIR/$PATCH_FILE -> $PATCH_DEST"
+             exit 1
+          fi
+      else
+          echo "ERROR: Don't know what destination file name to use for patch '$OLD_WORK_SRC_DIR/$PATCH_FILE', and to be copied to '$PKG_PATCH_DIR'"
+      fi
+   done
+
+   echo ""
+   echo ""
+   )
+
+   if [ $? -ne 0 ]; then
+      FAILED=$name
+      break
+   fi
+
+   )
+
+
+done
+
+if [ "$FAILED" != "" ]; then
+   echo "Failed for ... $FAILED"
+   exit 1
+fi
+
+
+

build-tools/mk/_sign_pkgs.mk

@@ -0,0 +1,31 @@
+
+#
+# this makefile is used by the build-iso process to add file signature to all rpms
+# 
+# it requires a private key, passed as the variable KEY
+
+PKGS_LIST := $(wildcard *.rpm)
+
+# we need to skip the signature of some packages that
+# might be installed in file systems that do not support extended attributes
+# in the case of shim- and grub2-efi-, the UEFI configuration installs them in a VFAT file system
+PKGS_TO_SKIP := $(wildcard grub2-efi-[0-9]*.x86_64.rpm shim-[0-9]*.x86_64.rpm)
+
+PKGS_TO_SIGN = $(filter-out $(PKGS_TO_SKIP),$(PKGS_LIST))
+
+define _pkg_sign_tmpl
+
+_sign_$1 :
+	@ rpmsign --signfiles --fskpath=$(KEY) $1
+	@ chown mockbuild $1
+	@ chgrp users $1
+
+sign : _sign_$1
+
+endef
+
+sign :
+	@echo signed all packages
+
+$(foreach file,$(PKGS_TO_SIGN),$(eval $(call _pkg_sign_tmpl,$(file))))
+

build-tools/mock_cfg_to_yum_conf.py

@@ -0,0 +1,9 @@
+#!/usr/bin/python
+
+import sys
+
+FN=sys.argv[1]
+variables={}
+variables['config_opts']={}
+execfile( FN, variables )
+print variables['config_opts']['yum.conf']

build-tools/modify-build-cfg

@@ -0,0 +1,130 @@
+#!/bin/sh
+
+# This script modifies a mock configuration file (typically $MY_BUILD_CFG)
+# to add build time environment variables to the mock environment (things
+# like what branch we're building on, etc).
+#
+# For reasons of security, the host environment variables cannot normally be
+# passed through to the mock environment, so this scripts sets the variables
+# to literal values.
+#
+# usage: modify-build-cfg [file.cfg]
+#
+
+MOCK_CFG_PROTO="$MY_REPO/cgcs-centos-repo/mock.cfg.proto"
+if [ ! -f "$MOCK_CFG_PROTO" ]; then
+   echo "ERROR: Couldn't find mock config prototype at '$MOCK_CFG_PROTO'"
+   exit 1
+fi
+
+if [ "${1}x" == "x" ]; then
+	FILE=$MY_BUILD_CFG
+else
+	FILE=$1
+fi
+
+if [ -f $MOCK_CFG_PROTO ]; then
+   if [ -f $FILE ]; then
+      NEWER=$(find "$MOCK_CFG_PROTO" -newer "$FILE")
+      if [ "x$NEWER" != "x" ]; then
+         \rm -f -v "$FILE"
+      fi
+   fi
+fi
+
+if [ ! -f $FILE ]; then
+   if [ -z $MY_BUILD_ENVIRONMENT ] || [ -z $MY_BUILD_DIR ] || [ -z $MY_REPO ]; then
+      echo "Can't create $FILE without MY_BUILD_ENVIRONMENT, MY_BUILD_DIR and MY_REPO environment variables"
+      exit 1
+   fi
+
+   echo "Recreating $FILE"
+   \cp -f -v "$MOCK_CFG_PROTO" "$FILE"
+   if [ $? -ne 0 ]; then
+      echo "Couldn't find config file '$FILE', nor construct it from '$MOCK_CFG_PROTO'"
+      exit 1
+   fi
+
+   sed -i "s%LOCAL_BASE%http://127.0.0.1:8088%g"  "$FILE"
+   sed -i "s%MIRROR_BASE%http://127.0.0.1:8088%g" "$FILE"
+   sed -i "s%BUILD_ENV%$MY_BUILD_ENVIRONMENT%g"   "$FILE"
+   sed -i "s%/MY_BUILD_DIR%$MY_BUILD_DIR_TOP%g"   "$FILE"
+   sed -i "s%/MY_REPO_DIR%$MY_REPO%g"             "$FILE"
+
+   # Disable all local-* repos for the build-types other than the current one
+   for bt in std rt; do
+      if [ "$bt" != "$BUILD_TYPE" ]; then
+         # Use the range of lines starting with pattern [local-$bt] until the next line starting with []
+         sed -i "/^\[local-$bt\]/,/^\[/ s/enabled=1/enabled=0/" $FILE
+      fi
+   done
+fi
+
+
+# Add environment variables to mock config if they don't exist
+grep -q "config_opts\['environment'\]\['BUILD_BY'\]" $FILE || \
+    echo "config_opts['environment']['BUILD_BY']" >> $FILE
+
+grep -q "config_opts\['environment'\]\['BUILD_DATE'\]" $FILE || \
+    echo "config_opts['environment']['BUILD_DATE']" >> $FILE
+
+grep -q "config_opts\['environment'\]\['REPO'\]" $FILE || \
+    echo "config_opts['environment']['REPO']" >> $FILE
+
+grep -q "config_opts\['environment'\]\['WRS_GIT_BRANCH'\]" $FILE || \
+    echo "config_opts['environment']['WRS_GIT_BRANCH']" >> $FILE
+
+grep -q "config_opts\['environment'\]\['CGCS_GIT_BRANCH'\]" $FILE || \
+    echo "config_opts['environment']['CGCS_GIT_BRANCH']" >> $FILE
+
+if [ -z $FORMAL_BUILD ]; then 
+    grep -q "config_opts\['macros'\]\['%_no_cgcs_license_check'\] = '1'" $FILE || \
+	echo "config_opts['macros']['%_no_cgcs_license_check'] = '1'" >> $FILE
+else
+    sed -i "/config_opts\['macros'\]\['%_no_cgcs_license_check'\] = '1'/d" $FILE
+fi
+
+grep -q "config_opts\['macros'\]\['%_tis_build_type'\] = '$BUILD_TYPE'" $FILE || \
+    echo "config_opts['macros']['%_tis_build_type'] = '$BUILD_TYPE'" >> $FILE
+
+if [ -f /usr/lib64/nosync/nosync.so ]; then
+    grep -q "config_opts\['nosync'\] = True" $FILE || \
+        echo "config_opts['nosync'] = True" >> $FILE
+fi
+
+grep -q "config_opts\['chroot_setup_cmd'\] = 'install @buildsys-build pigz lbzip2 yum'" $FILE || \
+    echo "config_opts['chroot_setup_cmd'] = 'install @buildsys-build pigz lbzip2 yum'" >> $FILE
+
+#
+# Read macros from tis.macros to add to the build config file,
+# for use in RPM spec files
+#
+RPM_MACROS=$MY_REPO/build-tools/tis.macros
+sed 's/#.*//' $RPM_MACROS | grep '=' | while IFS='=' read name value; do
+    # Check if the entry already exists. If so, go to next line
+    grep -q "^config_opts\['macros'\]\['${name}'\] = '${value}'$" $FILE && continue
+
+    # Update or add the entry
+    grep -q "^config_opts\['macros'\]\['${name}'\]" $FILE
+    if [ $? -eq 0 ]; then
+        sed -i -r "s#^(config_opts\['macros'\]\['${name}'\]).*#\1 = '${value}'#" $FILE
+    else
+        echo "config_opts['macros']['${name}'] = '${value}'" >> $FILE
+    fi
+done
+
+# okay, now we have lines for each env var.  Generate the correct values
+
+BUILD_DATE=`date "+%F %T %z"`
+CGCS_GIT_BRANCH=`cd $MY_REPO/addons/wr-cgcs/layers/cgcs/; git rev-parse --abbrev-ref HEAD`
+WRS_GIT_BRANCH=`cd $MY_REPO; git rev-parse --abbrev-ref HEAD`
+REPO=$MY_REPO
+
+# Finally, our good friend sed will place the values in the mock config file
+sed -i \
+    -e "s#config_opts\['environment'\]\['BUILD_BY'\].*#config_opts\['environment'\]\['BUILD_BY'\] = '$USER'#" \
+    -e "s#config_opts\['environment'\]\['BUILD_DATE'\].*#config_opts\['environment'\]\['BUILD_DATE'\] = '$BUILD_DATE'#" \
+    -e "s#config_opts\['environment'\]\['REPO'\].*#config_opts\['environment'\]\['REPO'\] = '$REPO'#" \
+    -e "s#config_opts\['environment'\]\['WRS_GIT_BRANCH'\].*#config_opts\['environment'\]\['WRS_GIT_BRANCH'\] = '$WRS_GIT_BRANCH'#" \
+    -e "s#config_opts\['environment'\]\['CGCS_GIT_BRANCH'\].*#config_opts\['environment'\]\['CGCS_GIT_BRANCH'\] = '$CGCS_GIT_BRANCH'#" \
+    $FILE

build-tools/patch-iso

@@ -0,0 +1,320 @@
+#!/bin/bash
+#
+# Utility for adding patches to an unpatched ISO
+#
+
+if [ -z "${MY_REPO}" ]; then
+    echo "Required environment variable MY_REPO is not set"
+    exit 1
+fi
+
+SETUP_PATCH_REPO=${MY_REPO}/addons/wr-cgcs/layers/cgcs/extras.ND/scripts/setup_patch_repo.sh
+if [ ! -x ${SETUP_PATCH_REPO} ]; then
+    echo "Cannot find or execute ${SETUP_PATCH_REPO}"
+    exit 1
+fi
+
+REPO_UPGRADES_DIR=${MY_REPO}/addons/wr-cgcs/layers/cgcs/common-bsp/files/upgrades
+RELEASE_INFO=${MY_REPO}/addons/wr-cgcs/layers/cgcs/middleware/recipes-common/build-info/release-info.inc
+PLATFORM_RELEASE=$(source $RELEASE_INFO && echo $PLATFORM_RELEASE)
+
+function usage() {
+    echo ""
+    echo "Usage: "
+    echo "   $(basename $0) -i <input bootimage.iso> -o <output bootimage.iso> [ -u ] <patch> ..."
+    echo "        -i <file>: Specify input ISO file"
+    echo "        -o <file>: Specify output ISO file"
+    echo "        -u       : Update with upgrades files from ${REPO_UPGRADES_DIR}"
+    echo ""
+}
+
+function extract_pkg_from_patch_repo() {
+   local repodir=${BUILDDIR}/patches
+   local pkgname=$1
+
+   local pkgfile=$(repoquery --repofrompath local,${repodir} --location -q ${pkgname})
+   if [ -z "${pkgfile}" ]; then
+      return 1
+   fi
+
+   rpm2cpio ${pkgfile/file://} | cpio -idmv
+   if [ $? -ne 0 ]; then
+      echo "Failed to extract $pkgname files from ${pkgfile/file://}"
+      exit 1
+   fi
+}
+
+declare INPUT_ISO=
+declare OUTPUT_ISO=
+declare ORIG_PWD=$PWD
+declare DO_UPGRADES=1
+
+while getopts "i:o:u" opt; do
+    case $opt in
+        i)
+            INPUT_ISO=$OPTARG
+            ;;
+        o)
+            OUTPUT_ISO=$OPTARG
+            ;;
+        u)
+            DO_UPGRADES=0
+            ;;
+        *)
+            usage
+            exit 1
+            ;;
+    esac
+done
+
+if [ -z "$INPUT_ISO" -o -z "$OUTPUT_ISO" ]; then
+    usage
+    exit 1
+fi
+
+if [ ! -f ${INPUT_ISO} ]; then
+    echo "Input file does not exist: ${INPUT_ISO}"
+    exit 1
+fi
+
+if [ -f ${OUTPUT_ISO} ]; then
+    echo "Output file already exists: ${OUTPUT_ISO}"
+    exit 1
+fi
+
+shift $((OPTIND-1))
+
+if [ $# -le 0 ]; then
+    usage
+    exit
+fi
+
+for pf in $@; do
+    if [ ! -f $pf ]; then
+        echo "Patch file $pf does not exist"
+        exit 1
+    fi
+
+    if [[ ! $pf =~ \.patch$ ]]; then
+        echo "Specified file $pf does not have .patch extension"
+        exit 1
+    fi
+done
+
+declare MNTDIR=
+declare BUILDDIR=
+declare WORKDIR=
+
+function cleanup() {
+    if [ -n "$MNTDIR" -a -d "$MNTDIR" ]; then
+        guestunmount $MNTDIR
+        \rmdir $MNTDIR
+    fi
+
+    if [ -n "$BUILDDIR" -a -d "$BUILDDIR" ]; then
+        \rm -rf $BUILDDIR
+    fi
+
+    if [ -n "$WORKDIR" -a -d "$WORKDIR" ]; then
+        \rm -rf $WORKDIR
+    fi
+}
+
+trap cleanup EXIT
+
+MNTDIR=$(mktemp -d -p $PWD patchiso_mnt_XXXXXX)
+if [ -z "${MNTDIR}" -o ! -d ${MNTDIR} ]; then
+    echo "Failed to create mntdir. Aborting..."
+    exit $rc
+fi
+
+BUILDDIR=$(mktemp -d -p $PWD patchiso_build_XXXXXX)
+if [ -z "${BUILDDIR}" -o ! -d ${BUILDDIR} ]; then
+    echo "Failed to create builddir. Aborting..."
+    exit $rc
+fi
+
+# Mount the ISO
+guestmount -a ${INPUT_ISO} -m /dev/sda1 --ro ${MNTDIR}
+rc=$?
+if [ $rc -ne 0 ]; then
+    echo "Call to guestmount failed with rc=$rc. Aborting..."
+    exit $rc
+fi
+
+rsync -a ${MNTDIR}/ ${BUILDDIR}/
+rc=$?
+if [ $rc -ne 0 ]; then
+    echo "Call to rsync ISO content. Aborting..."
+    exit $rc
+fi
+
+guestunmount ${MNTDIR}
+\rmdir ${MNTDIR}
+
+# Setup the patch repo
+${SETUP_PATCH_REPO} -o ${BUILDDIR}/patches $@
+rc=$?
+if [ $rc -ne 0 ]; then
+    echo "Call to $(basename ${SETUP_PATCH_REPO}) failed with rc=$rc. Aborting..."
+    exit $rc
+fi
+
+# Look for components that need modification
+#extract_pkg_from_patch_repo
+WORKDIR=$(mktemp -d -p $PWD patchiso_work_XXXXXX)
+if [ -z "${WORKDIR}" -o ! -d ${WORKDIR} ]; then
+    echo "Failed to create workdir. Aborting..."
+    exit $rc
+fi
+
+\cd ${WORKDIR}
+\mkdir extract
+\cd extract
+
+# Changes to copied files here must also be reflected in build-iso
+
+extract_pkg_from_patch_repo platform-kickstarts
+if [ $? -eq 0 ]; then
+    # Replace files
+    \rm -f ${BUILDDIR}/*ks.cfg &&
+    \cp --preserve=all www/pages/feed/rel-*/*.cfg ${BUILDDIR}/ &&
+    \cp --preserve=all ${BUILDDIR}/controller_ks.cfg ${BUILDDIR}/ks.cfg
+    if [ $? -ne 0 ]; then
+        echo "Failed to copy extracted kickstarts"
+        exit 1
+    fi
+fi
+\cd ${WORKDIR}
+\rm -rf extract
+
+\mkdir extract
+\cd extract
+extract_pkg_from_patch_repo platform-kickstarts-pxeboot
+if [ $? -eq 0 ]; then
+    # Replace files
+    \rm -f ${BUILDDIR}/pxeboot/pxeboot_controller.cfg \
+        ${BUILDDIR}/pxeboot/pxeboot_smallsystem.cfg \
+        ${BUILDDIR}/pxeboot/pxeboot_smallsystem_lowlatency.cfg &&
+    \cp --preserve=all pxeboot/* ${BUILDDIR}/pxeboot/
+    if [ $? -ne 0 ]; then
+        echo "Failed to copy extracted pxeboot kickstarts"
+        exit 1
+    fi
+fi
+\cd ${WORKDIR}
+\rm -rf extract
+
+\mkdir extract
+\cd extract
+extract_pkg_from_patch_repo pxe-network-installer
+if [ $? -eq 0 ]; then
+    # Replace files
+    \rm -f ${BUILDDIR}/pxeboot/pxelinux.0 \
+        ${BUILDDIR}/pxeboot/menu.c32 \
+        ${BUILDDIR}/pxeboot/chain.c32 &&
+    \cp --preserve=all pxeboot/pxelinux.0 pxeboot/menu.c32 pxeboot/chain.c32 ${BUILDDIR}/pxeboot/
+    if [ $? -ne 0 ]; then
+        echo "Error: Could not copy all files from installer"
+        exit 1
+    fi
+
+    for f in pxeboot/EFI/centos/x86_64-efi/*; do
+        \rm -f ${BUILDDIR}/${f}
+    done
+    \cp --preserve=all pxeboot/EFI/centos/x86_64-efi/* ${BUILDDIR}/pxeboot/EFI/centos/x86_64-efi/
+    if [ $? -ne 0 ]; then
+        echo "Error: Could not copy all files from installer"
+        exit 1
+    fi
+
+    \rm -f ${BUILDDIR}/LiveOS/squashfs.img &&
+    \cp --preserve=all www/pages/feed/rel-*/LiveOS/squashfs.img ${BUILDDIR}/LiveOS/
+    if [ $? -ne 0 ]; then
+        echo "Error: Could not copy squashfs from LiveOS"
+        exit 1
+    fi
+
+    # Replace vmlinuz and initrd.img with our own pre-built ones
+    \rm -f \
+        ${BUILDDIR}/vmlinuz \
+        ${BUILDDIR}/images/pxeboot/vmlinuz \
+        ${BUILDDIR}/initrd.img \
+        ${BUILDDIR}/images/pxeboot/initrd.img &&
+    \cp --preserve=all pxeboot/rel-*/installer-bzImage_1.0 \
+        ${BUILDDIR}/vmlinuz &&
+    \cp --preserve=all pxeboot/rel-*/installer-bzImage_1.0 \
+        ${BUILDDIR}/images/pxeboot/vmlinuz &&
+    \cp --preserve=all pxeboot/rel-*/installer-intel-x86-64-initrd_1.0 \
+        ${BUILDDIR}/initrd.img &&
+    \cp --preserve=all pxeboot/rel-*/installer-intel-x86-64-initrd_1.0 \
+        ${BUILDDIR}/images/pxeboot/initrd.img
+    if [ $? -ne 0 ]; then
+        echo "Error: Failed to copy installer images"
+        exit 1
+    fi
+fi
+\cd ${WORKDIR}
+\rm -rf extract
+
+\mkdir extract
+\cd extract
+extract_pkg_from_patch_repo grub2-efi-pxeboot
+if [ $? -eq 0 ]; then
+    # Replace files
+    \rm -f ${BUILDDIR}/pxeboot/EFI/grubx64.efi &&
+    \cp --preserve=all pxeboot/EFI/grubx64.efi ${BUILDDIR}/pxeboot/EFI/
+    if [ $? -ne 0 ]; then
+        echo "Error: Failed to copy grub2-efi-pxeboot files"
+        exit 1
+    fi
+fi
+\cd ${WORKDIR}
+\rm -rf extract
+
+\cd ${ORIG_PWD}
+
+if [ ${DO_UPGRADES} -eq 0 ]; then
+    # Changes to copied files here must also be reflected in build-iso
+
+    echo "Updating upgrade support files"
+    ISO_UPGRADES_DIR="${BUILDDIR}/upgrades"
+    \rm -rf ${ISO_UPGRADES_DIR}
+    \mkdir ${ISO_UPGRADES_DIR}
+    \cp ${REPO_UPGRADES_DIR}/* ${ISO_UPGRADES_DIR}
+    sed -i "s/xxxSW_VERSIONxxx/${PLATFORM_RELEASE}/g" ${ISO_UPGRADES_DIR}/metadata.xml
+    chmod +x ${ISO_UPGRADES_DIR}/*.sh
+    # Write the version out (used in upgrade scripts - this is the same as SW_VERSION)
+    echo "VERSION=$PLATFORM_RELEASE" > ${ISO_UPGRADES_DIR}/version
+fi
+
+# Rebuild the ISO
+mkisofs -o ${OUTPUT_ISO} \
+    -R -D -A 'oe_iso_boot' -V 'oe_iso_boot' \
+    -quiet \
+    -b isolinux.bin -c boot.cat -no-emul-boot \
+    -boot-load-size 4 -boot-info-table \
+    -eltorito-alt-boot \
+    -e images/efiboot.img \
+    -no-emul-boot \
+    ${BUILDDIR}
+
+isohybrid --uefi ${OUTPUT_ISO}
+implantisomd5 ${OUTPUT_ISO}
+
+# Sign the .iso with the developer private key
+# Signing with the formal key is only to be done for customer release
+# and is a manual step afterwards, as with the GA ISO
+openssl dgst -sha256 \
+    -sign ${MY_REPO}/build-tools/signing/dev-private-key.pem \
+    -binary \
+    -out ${OUTPUT_ISO/%.iso/.sig} \
+    ${OUTPUT_ISO}
+rc=$?
+if [ $rc -ne 0 ]; then
+    echo "Call to $(basename ${SETUP_PATCH_REPO}) failed with rc=$rc. Aborting..."
+    exit $rc
+fi
+
+echo "Patched ISO: ${OUTPUT_ISO}"
+

build-tools/patch_rebase_1

@@ -0,0 +1,130 @@
+#!/bin/bash
+
+#
+# Start an edit session for packages to be upgraded - pre upgrade version
+#
+
+usage () {
+    echo ""
+    echo "Step 1: Start an edit session for packages to be upgraded - pre upgrade version"
+    echo ""
+    echo "Usage: "
+    echo "   patch_rebase_1 [--origin_branch <branch>] [--working_branch <branch>] [--upversion_data <file>]"
+    echo ""
+    echo "Assumes cgcs-centos-repo already has a working_branch commit that sets the new symlinks."
+    echo ""
+    echo "The upversion_data file has data on all the src.rpm being updated in the format:"
+    echo "  export UPVERSION_DATA=$MY_WORKSPACE/upversion.log"
+    echo "  PKG=lighttpd"
+    echo "  OLD_SRC_RPM=lighttpd-1.4.41-1.el7.src.rpm"
+    echo "  NEW_SRC_RPM=lighttpd-1.4.41-2.el7.src.rpm"
+    echo "  SRPM_PATH=$MY_REPO/addons/wr-cgcs/layers/cgcs/recipes-extended/lighttpd/centos/srpm_path"
+    echo "  echo \"\$PKG#\$SRPM_PATH##\$OLD_SRC_RPM#\$NEW_SRC_RPM\" > UPVERSION_DATA"
+    echo ""
+}
+
+TEMP=`getopt -o h --long origin_branch:,working_branch:,upversion_data:,help -n 'test.sh' -- "$@"`
+eval set -- "$TEMP"
+
+ORIGIN_BRANCH=""
+WORKING_BRANCH=""
+UPVERSION_LOG=""
+HELP=0
+
+while true ; do
+    case "$1" in
+        --origin_branch)  shift ; ORIGIN_BRANCH="$1" ; shift ;;
+        --working_branch) shift ; WORKING_BRANCH="$1" ; shift ;;
+        --upversion_data) shift ; UPVERSION_LOG="$1" ; shift ;;
+        -h|--help)        HELP=1 ; shift ;;
+        --)               shift ; break ;;
+        *)                usage; exit 1 ;;
+    esac
+done
+
+if [ $HELP -eq 1 ]; then
+    usage
+    exit 0
+fi
+
+if [ "$UPVERSION_LOG" == "" ]; then
+   UPVERSION_LOG=$UPVERSION_DATA
+fi
+
+if [ "$UPVERSION_LOG" == "" ]; then
+   echo "ERROR: please specify location of upversion data"
+   usage
+   exit 1
+fi
+
+if [ ! -f "$UPVERSION_LOG" ]; then
+   echo "File not found: '$UPVERSION_LOG'"
+   exit 1
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
+   ORIGIN_BRANCH=$PATCH_SOURCE_BRANCH
+   WORKING_BRANCH=$MY_PATCH_BRANCH
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
+   ORIGIN_BRANCH=$SOURCE_BRANCH
+   WORKING_BRANCH=$MY_BRANCH
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ]; then
+   echo "ERROR: please specify a origin branch"
+   usage
+   exit 1
+fi
+
+if [ "$WORKING_BRANCH" == "" ]; then
+   echo "ERROR: please specify a working branch"
+   usage
+   exit 1
+fi
+
+# One step back to see the old symlinks
+cd $MY_REPO/cgcs-centos-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$(pwd)'"
+   exit 1
+fi
+
+git checkout HEAD^
+
+FAILED=""
+for dat in $(cat $UPVERSION_LOG); do
+   name=$(echo $dat | awk -F '#' '{print $1}')
+   srpm_path=$(echo $dat | awk -F '#' '{print $2}')
+   old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
+   new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
+
+   echo "$name  $old_src_rpm  $new_src_rpm"
+
+   build-pkgs --edit --clean $name
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed cmd 'build-pkgs --edit --clean $name'"
+      FAILED="$name $FAILED"
+      break
+   fi
+   echo "$?   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
+   build-pkgs --edit $name
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed cmd 'build-pkgs --edit $name'"
+      FAILED="$name $FAILED"
+      break
+   fi
+   echo "$?   <=<=<=<=<=<=<=<=<=<=<=<=<=<=<=<="
+done
+
+cd $MY_REPO/cgcs-centos-repo
+git checkout $WORKING_BRANCH
+
+if [ "$FAILED" != "" ]; then
+   echo "Failed build-pkgs --edit for ... $FAILED"
+   exit 1
+fi
+
+

build-tools/patch_rebase_2

@@ -0,0 +1,148 @@
+#!/bin/bash
+
+#
+# Update srpm_path for packages to be upgraded
+#
+
+usage () {
+    echo ""
+    echo "Step 2: Update srpm_path for packages to be upgraded"
+    echo ""
+    echo "Usage: "
+    echo "   patch_rebase_2 [--origin_branch <branch>] [--working_branch <branch>] [--upversion_data <file>]"
+    echo ""
+    echo "Assumes cgcs-centos-repo already has a working_branch commit that sets the new symlinks."
+    echo ""
+    echo "The upversion_data file has data on all the src.rpm being updated in the format:"
+    echo "  export UPVERSION_DATA=$MY_WORKSPACE/upversion.log"
+    echo "  PKG=lighttpd"
+    echo "  OLD_SRC_RPM=lighttpd-1.4.41-1.el7.src.rpm"
+    echo "  NEW_SRC_RPM=lighttpd-1.4.41-2.el7.src.rpm"
+    echo "  SRPM_PATH=$MY_REPO/addons/wr-cgcs/layers/cgcs/recipes-extended/lighttpd/centos/srpm_path"
+    echo "  echo \"\$PKG#\$SRPM_PATH##\$OLD_SRC_RPM#\$NEW_SRC_RPM\" > UPVERSION_DATA"
+    echo ""
+}
+
+
+TEMP=`getopt -o h --long origin_branch:,working_branch:,upversion_data:,help -n 'test.sh' -- "$@"`
+eval set -- "$TEMP"
+
+ORIGIN_BRANCH=""
+WORKING_BRANCH=""
+UPVERSION_LOG=""
+HELP=0
+
+while true ; do
+    case "$1" in
+        --origin_branch)  shift ; ORIGIN_BRANCH="$1" ; shift ;;
+        --working_branch) shift ; WORKING_BRANCH="$1" ; shift ;;
+        --upversion_data) shift ; UPVERSION_LOG="$1" ; shift ;;
+        -h|--help)        HELP=1 ; shift ;;
+        --)               shift ; break ;;
+        *)                usage; exit 1 ;;
+    esac
+done
+
+if [ $HELP -eq 1 ]; then
+    usage
+    exit 0
+fi
+
+if [ "$UPVERSION_LOG" == "" ]; then
+   UPVERSION_LOG=$UPVERSION_DATA
+fi
+
+if [ "$UPVERSION_LOG" == "" ]; then
+   echo "ERROR: please specify location of upversion data"
+   usage
+   exit 1
+fi
+
+if [ ! -f "$UPVERSION_LOG" ]; then
+   echo "File not found: '$UPVERSION_LOG'"
+   exit 1
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
+   ORIGIN_BRANCH=$PATCH_SOURCE_BRANCH
+   WORKING_BRANCH=$MY_PATCH_BRANCH
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
+   ORIGIN_BRANCH=$SOURCE_BRANCH
+   WORKING_BRANCH=$MY_BRANCH
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ]; then
+   echo "ERROR: please specify a origin branch"
+   usage
+   exit 1
+fi
+
+if [ "$WORKING_BRANCH" == "" ]; then
+   echo "ERROR: please specify a working branch"
+   usage
+   exit 1
+fi
+
+# One step back to see the old symlinks
+cd $MY_REPO
+
+FAILED=""
+for dat in $(cat $UPVERSION_LOG); do
+   name=$(echo $dat | awk -F '#' '{print $1}')
+   srpm_path=$(echo $dat | awk -F '#' '{print $2}')
+   old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
+   new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
+
+   (
+   cd $(dirname $srpm_path)
+   CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
+   if [ "$CURRENT_BRANCH" != "$WORKING_BRANCH" ]; then
+      git checkout $WORKING_BRANCH
+      if [ $? -ne 0 ]; then
+         git checkout $ORIGIN_BRANCH
+         if [ $? -ne 0 ]; then
+            echo "ERROR: Can't checkout branch '$ORIGIN_BRANCH' in directory '$(pwd)'"
+            exit 1
+         fi
+
+         git checkout -b $WORKING_BRANCH
+         if [ $? -ne 0 ]; then
+            echo "ERROR: failed to 'git checkout -b $WORKING_BRANCH' from '$(pwd)'"
+            exit 1
+         else
+            echo "created branch '$WORKING_BRANCH' at '$(pwd)'"
+         fi
+      fi
+   fi
+
+   sed -i "s#$old_src_rpm#$new_src_rpm#" $srpm_path
+   if [ $? -ne 0 ]; then
+      echo "ERROR: sed failed '$old_src_rpm' -> '$new_src_rpm'"
+      exit 1
+   else
+      echo "updated $srpm_path: '$old_src_rpm' -> '$new_src_rpm'"
+   fi
+
+   exit 0
+   )
+
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed while working on package '$name' at '$srpm_path'"
+      exit 1
+   fi
+done
+
+echo ""
+for d in $(for dat in $(cat $UPVERSION_LOG); do srpm_path=$(echo $dat | awk -F '#' '{print $2}'); ( cd $(dirname $srpm_path); git rev-parse --show-toplevel ); done | sort --unique); do
+   (
+    cd $d
+    echo "cd $d"
+    for f in $(git status --porcelain | grep 'srpm_path$' | awk '{print $2}'); do 
+        echo "git add $f";
+    done
+    echo "git commit -m 'srpm_path updates for patch $PATCH_ID'"
+   )
+done
+echo ""

build-tools/patch_rebase_3

@@ -0,0 +1,119 @@
+#!/bin/bash
+
+#
+# Start an edit session for packages to be upgraded - post upgrade version
+#
+
+usage () {
+    echo ""
+    echo "Step 3: Start an edit session for packages to be upgraded - post upgrade version"
+    echo ""
+    echo "Usage: "
+    echo "   patch_rebase_3 [--origin_branch <branch>] [--working_branch <branch>] [--upversion_data <file>]"
+    echo ""
+    echo "Assumes cgcs-centos-repo already has a working_branch commit that sets the new symlinks."
+    echo ""
+    echo "The upversion_data file has data on all the src.rpm being updated in the format:"
+    echo "  export UPVERSION_DATA=$MY_WORKSPACE/upversion.log"
+    echo "  PKG=lighttpd"
+    echo "  OLD_SRC_RPM=lighttpd-1.4.41-1.el7.src.rpm"
+    echo "  NEW_SRC_RPM=lighttpd-1.4.41-2.el7.src.rpm"
+    echo "  SRPM_PATH=$MY_REPO/addons/wr-cgcs/layers/cgcs/recipes-extended/lighttpd/centos/srpm_path"
+    echo "  echo \"\$PKG#\$SRPM_PATH##\$OLD_SRC_RPM#\$NEW_SRC_RPM\" > UPVERSION_DATA"
+    echo ""
+}
+
+
+TEMP=`getopt -o h --long origin_branch:,working_branch:,upversion_data:,help -n 'test.sh' -- "$@"`
+eval set -- "$TEMP"
+
+ORIGIN_BRANCH=""
+WORKING_BRANCH=""
+UPVERSION_LOG=""
+HELP=0
+
+while true ; do
+    case "$1" in
+        --origin_branch)  shift ; ORIGIN_BRANCH="$1" ; shift ;;
+        --working_branch) shift ; WORKING_BRANCH="$1" ; shift ;;
+        --upversion_data) shift ; UPVERSION_LOG="$1" ; shift ;;
+        -h|--help)        HELP=1 ; shift ;;
+        --)               shift ; break ;;
+        *)                usage; exit 1 ;;
+    esac
+done
+
+if [ $HELP -eq 1 ]; then
+    usage
+    exit 0
+fi
+
+if [ "$UPVERSION_LOG" == "" ]; then
+   UPVERSION_LOG=$UPVERSION_DATA
+fi
+
+if [ "$UPVERSION_LOG" == "" ]; then
+   echo "ERROR: please specify location of upversion data"
+   usage
+   exit 1
+fi
+
+if [ ! -f "$UPVERSION_LOG" ]; then
+   echo "File not found: '$UPVERSION_LOG'"
+   exit 1
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
+   ORIGIN_BRANCH=$PATCH_SOURCE_BRANCH
+   WORKING_BRANCH=$MY_PATCH_BRANCH
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
+   ORIGIN_BRANCH=$SOURCE_BRANCH
+   WORKING_BRANCH=$MY_BRANCH
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ]; then
+   echo "ERROR: please specify a origin branch"
+   usage
+   exit 1
+fi
+
+if [ "$WORKING_BRANCH" == "" ]; then
+   echo "ERROR: please specify a working branch"
+   usage
+   exit 1
+fi
+
+# One step back to see the old symlinks
+cd $MY_REPO/cgcs-centos-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
+   exit 1
+fi
+
+FAILED=""
+for dat in $(cat $UPVERSION_LOG); do
+   name=$(echo $dat | awk -F '#' '{print $1}')
+   srpm_path=$(echo $dat | awk -F '#' '{print $2}')
+   old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
+   new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
+
+   echo "$name  $old_src_rpm  $new_src_rpm"
+
+   build-pkgs --edit $name --no-meta-patch
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed cmd 'build-pkgs --edit $name'"
+      FAILED="$name $FAILED"
+      break
+   fi
+   echo "$?   <=<=<=<=<=<=<=<=<=<=<=<=<=<=<=<="
+done
+
+if [ "$FAILED" != "" ]; then
+   echo "Failed build-pkgs --edit for ... $FAILED"
+   exit 1
+fi
+
+

build-tools/patch_rebase_4

@@ -0,0 +1,403 @@
+#!/bin/bash
+
+#
+# Migrate Titanium Cloud patches to the new package version
+#
+
+usage () {
+    echo ""
+    echo "Step 4: Migrate Titanium Cloud patches to the new package version"
+    echo ""
+    echo "Usage: "
+    echo "   patch_rebase_4 [--origin_branch <branch>] [--working_branch <branch>] [--upversion_data <file>]"
+    echo ""
+    echo "Assumes cgcs-centos-repo already has a working_branch commit that sets the new symlinks."
+    echo ""
+    echo "The upversion_data file has data on all the src.rpm being updated in the format:"
+    echo "  export UPVERSION_DATA=$MY_WORKSPACE/upversion.log"
+    echo "  PKG=lighttpd"
+    echo "  OLD_SRC_RPM=lighttpd-1.4.41-1.el7.src.rpm"
+    echo "  NEW_SRC_RPM=lighttpd-1.4.41-2.el7.src.rpm"
+    echo "  SRPM_PATH=$MY_REPO/addons/wr-cgcs/layers/cgcs/recipes-extended/lighttpd/centos/srpm_path"
+    echo "  echo \"\$PKG#\$SRPM_PATH##\$OLD_SRC_RPM#\$NEW_SRC_RPM\" > UPVERSION_DATA"
+    echo ""
+}
+
+
+TEMP=`getopt -o h --long origin_branch:,working_branch:,upversion_data:,help -n 'test.sh' -- "$@"`
+eval set -- "$TEMP"
+
+ORIGIN_BRANCH=""
+WORKING_BRANCH=""
+UPVERSION_LOG=""
+HELP=0
+
+while true ; do
+    case "$1" in
+        --origin_branch)  shift ; ORIGIN_BRANCH="$1" ; shift ;;
+        --working_branch) shift ; WORKING_BRANCH="$1" ; shift ;;
+        --upversion_data) shift ; UPVERSION_LOG="$1" ; shift ;;
+        -h|--help)        HELP=1 ; shift ;;
+        --)               shift ; break ;;
+        *)                usage; exit 1 ;;
+    esac
+done
+
+if [ $HELP -eq 1 ]; then
+    usage
+    exit 0
+fi
+
+if [ "$UPVERSION_LOG" == "" ]; then
+   UPVERSION_LOG=$UPVERSION_DATA
+fi
+
+if [ "$UPVERSION_LOG" == "" ]; then
+   echo "ERROR: please specify location of upversion data"
+   usage
+   exit 1
+fi
+
+if [ ! -f "$UPVERSION_LOG" ]; then
+   echo "File not found: '$UPVERSION_LOG'"
+   exit 1
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
+   ORIGIN_BRANCH=$PATCH_SOURCE_BRANCH
+   WORKING_BRANCH=$MY_PATCH_BRANCH
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ] && [ "$WORKING_BRANCH" == "" ]; then
+   ORIGIN_BRANCH=$SOURCE_BRANCH
+   WORKING_BRANCH=$MY_BRANCH
+fi
+
+if [ "$ORIGIN_BRANCH" == "" ]; then
+   echo "ERROR: please specify a origin branch"
+   usage
+   exit 1
+fi
+
+if [ "$WORKING_BRANCH" == "" ]; then
+   echo "ERROR: please specify a working branch"
+   usage
+   exit 1
+fi
+
+if [ "$DISPLAY" == "" ]; then
+   echo "ERROR: X-Windows 'DISPLAY' variable not set. This script needs to open pop-up windows."
+   usage
+   exit 1
+fi
+
+# One step back to see the old symlinks
+cd $MY_REPO/cgcs-centos-repo
+git checkout $WORKING_BRANCH
+if [ $? != 0 ]; then
+   echo "ERROR: Can't checkout branch '$WORKING_BRANCH' in directory '$MY_REPO/cgcs-centos-repo'"
+   exit 1
+fi
+
+FAILED=""
+build_types="std rt"
+for dat in $(cat $UPVERSION_LOG); do
+   (
+   name=$(echo $dat | awk -F '#' '{print $1}')
+   srpm_path=$(echo $dat | awk -F '#' '{print $2}')
+   old_src_rpm=$(echo $dat | awk -F '#' '{print $4}')
+   new_src_rpm=$(echo $dat | awk -F '#' '{print $5}')
+
+   PKG_DIR=$(dirname $(dirname $srpm_path))
+   OLD_BRANCH=$(echo $old_src_rpm | sed 's#[.]src[.]rpm$##')
+   NEW_BRANCH=$(echo $new_src_rpm | sed 's#[.]src[.]rpm$##')
+   
+   WORK_META_DIR=""
+   for dd in $build_types; do
+      WORK_META_DIR=$MY_WORKSPACE/$dd/srpm_work/$name/rpmbuild
+      echo "WORK_META_DIR=$WORK_META_DIR"
+      if [ -d $WORK_META_DIR ]; then
+         break;
+      else
+         WORK_META_DIR=""
+      fi
+   done
+   if [ "$WORK_META_DIR" == "" ]; then
+      echo "ERROR: failed to find srpm_work directory for '$name'"
+      exit 1
+   fi
+
+   # WORK_SRC_DIR=$(dirname $(find $MY_WORKSPACE/srpm_work/$name/gits/ -type d -name .git))
+   NEW_WORK_SRC_DIR=""
+   OLD_WORK_SRC_DIR=""
+   for dd in $build_types; do
+      for g in $(find $MY_WORKSPACE/$dd/srpm_work/$name/gits/ -type d -name .git); do
+         d=$(dirname $g)
+         if [ -d $d ]; then
+            cd $d;
+            git tag | grep pre_wrs_ >> /dev/null
+            if [ $? -ne 0 ]; then
+               continue
+            fi
+            git checkout $OLD_BRANCH 2>> /dev/null
+            if [ $? -eq 0 ]; then
+               OLD_WORK_SRC_DIR=$d
+            fi
+            git checkout $NEW_BRANCH  2>> /dev/null
+            if [ $? -eq 0 ]; then
+               NEW_WORK_SRC_DIR=$d
+            fi
+         fi
+      done
+   done
+   if [ "$WORK_META_DIR" == "" ]; then
+      echo "ERROR: failed to find srpm_work directory for '$name'"
+      exit 1
+   fi
+
+   echo "$name  $old_src_rpm  $new_src_rpm"
+   echo "PKG_DIR=$PKG_DIR"
+   echo "OLD_BRANCH=$OLD_BRANCH"
+   echo "NEW_BRANCH=$NEW_BRANCH"
+   echo "WORK_META_DIR=$WORK_META_DIR"
+   echo "OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
+   echo "NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
+   echo ""
+
+   (
+   cd $WORK_META_DIR
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to cd to WORK_META_DIR=$WORK_META_DIR"
+      exit 1
+   fi
+   echo "--- old meta git log (oldest to newest) ---"
+   git checkout $OLD_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH"
+      exit 1
+   fi
+   git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit | tac
+   PATCH_COMMIT_LIST=$(git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
+   echo "--- new meta git log (oldest to newest) ---"
+   git checkout $NEW_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH"
+      exit 1
+   fi
+   git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit | tac
+   REFERENCE_COMMIT=$(git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit | head -n 1 | awk '{ print $2 }')
+   echo ""
+
+   for COMMIT in ${PATCH_COMMIT_LIST}; do
+      echo "git cherry-pick $COMMIT"
+      git cherry-pick "$COMMIT"
+      if [ $? -ne 0 ]; then
+         echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files" 
+         git status --porcelain | grep '^UU ' | awk '{ print $2}'
+         echo "pwd=$(pwd)"
+         # gitk &
+         echo "git mergetool --no-prompt"
+         git mergetool --no-prompt
+         # for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
+         #    xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
+         #    if [ $? -ne 0 ]; then
+         #       echo "ERROR: problem launching editor on "
+         #       exit 1
+         #    fi
+         # done
+         echo "git cherry-pick --continue"
+         git cherry-pick --continue
+      fi
+   done
+
+   PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
+      exit 1
+   fi
+   for PATCH_FILE in ${PATCH_LIST}; do
+      PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
+      echo "$PATCH_FILE -> $PATCH_TARGET"
+      N=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*" | wc -l)
+      if [ $N -eq 1 ]; then
+          PATCH_DEST=$(find "$PKG_DIR/centos/meta_patches" -name "$PATCH_TARGET*")
+          echo "cp -f $PATCH_FILE $PATCH_DEST"
+          \cp -f  $PATCH_FILE  $PATCH_DEST
+          if [ $? -ne 0 ]; then
+             echo "ERROR: copy failed $WORK_META_DIR/$PATCH_FILE -> $PATCH_DEST"
+             exit 1
+          fi
+      else
+          echo "ERROR: Don't know what destination file name to use for patch '$WORK_META_DIR/$PATCH_FILE' derived from commit $COMMIT, and to be copied to '$PKG_DIR/centos/meta_patches'"
+      fi
+   done
+
+   echo ""
+   echo ""
+   )
+
+   if [ $? -ne 0 ]; then
+      FAILED=$name
+      break
+   fi
+
+   (
+   echo "--- old git log (oldest to newest) ---"
+   cd $OLD_WORK_SRC_DIR
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to cd to OLD_WORK_SRC_DIR=$OLD_WORK_SRC_DIR"
+      exit 1
+   fi
+
+   git checkout $OLD_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout OLD_BRANCH=$OLD_BRANCH in directory '$OLD_WORK_SRC_DIR'"
+      exit 1
+   fi
+
+   git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit | tac
+   PATCH_COMMIT_LIST=$(git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit | tac | grep WRS: | grep -v 'WRS: COPY_LIST content' | awk '{ print $2 }')
+
+   echo "--- new git log (oldest to newest) ---"
+   cd $NEW_WORK_SRC_DIR
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to cd to NEW_WORK_SRC_DIR=$NEW_WORK_SRC_DIR"
+      exit 1
+   fi
+
+   git checkout $NEW_BRANCH
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git checkout NEW_BRANCH=$NEW_BRANCH in directory '$NEW_WORK_SRC_DIR'"
+      exit 1
+   fi
+
+   git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit | tac
+   REFERENCE_COMMIT=$(git log --graph --pretty=format:'%h -%d %s (%ci) <%an>' --abbrev-commit | head -n 1 | awk '{ print $2 }')
+   echo ""
+
+   if [ "$OLD_WORK_SRC_DIR" == "$NEW_WORK_SRC_DIR" ]; then
+      for COMMIT in ${PATCH_COMMIT_LIST}; do
+         echo "git cherry-pick $COMMIT"
+         git cherry-pick "$COMMIT"
+         if [ $? -ne 0 ]; then
+            echo "WARNING: 'git cherry-pick $COMMIT' found merge conflicts. Please fix these files" 
+            git status --porcelain | grep '^UU ' | awk '{ print $2}'
+            echo "pwd=$(pwd)"
+            # gitk &
+            echo "git mergetool --no-prompt"
+            git mergetool --no-prompt
+            # for FILE_NAME in $(git status --porcelain | grep '^UU ' | awk '{ print $2}'); do
+            #    xterm -e "vi $FILE_NAME -c '/[<=>][<=>][<=>][<=>]'"
+            #    if [ $? -ne 0 ]; then
+            #       echo "ERROR: problem launching editor on "
+            #       exit 1
+            #    fi
+            # done
+            echo "git cherry-pick --continue"
+            git cherry-pick --continue
+         fi
+      done
+   else
+      cd $OLD_WORK_SRC_DIR
+      PATCH_LIST=$(git format-patch -n pre_wrs_$OLD_BRANCH)
+      if [ $? -ne 0 ]; then
+         echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=pre_wrs_$OLD_BRANCH"
+         exit 1
+      fi
+      cd $NEW_WORK_SRC_DIR
+      for PATCH_FILE in ${PATCH_LIST}; do
+         cat $OLD_WORK_SRC_DIR/$PATCH_FILE | patch -p1
+         if [ $? -ne 0 ]; then
+            for REJECT in $(find . -name '*.rej'); do
+               FILE_NAME=$(echo $REJECT | sed 's#.rej$##')
+               cd $OLD_WORK_SRC_DIR
+               gitk $FILE_NAME &
+               cd $NEW_WORK_SRC_DIR
+               if [ -f $FILE_NAME ] && [ -f $FILE_NAME.orig ]; then
+                  \cp -f $FILE_NAME.orig $FILE_NAME
+                  xterm -e "vi $FILE_NAME $REJECT"
+                  rm -f $REJECT
+                  rm -f $FILE_NAME.orig
+               fi
+            done
+         fi
+
+         git add --all
+         MSG=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-//' | sed 's/.patch$//')
+         git commit -m "WRS: $MSG"
+      done
+      
+   fi
+
+   PATCH_LIST=$(git format-patch -n $REFERENCE_COMMIT)
+   if [ $? -ne 0 ]; then
+      echo "ERROR: failed to git format-patch -n REFERENCE_COMMIT=$REFERENCE_COMMIT"
+      exit 1
+   fi
+   for PATCH_FILE in ${PATCH_LIST}; do
+      PATCH_TARGET=$(echo $PATCH_FILE | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch[0-9]*-//' | sed 's/^[0-9][0-9][0-9][0-9]-WRS-Patch//' | sed 's/.patch$//')
+      echo "$PATCH_FILE -> $PATCH_TARGET"
+      PKG_PATCH_DIR="$PKG_DIR/centos/patches"
+      N=0
+      if [ -d  "$PKG_PATCH_DIR" ]; then
+         N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
+      fi
+      if [ $N -ne 1 ]; then
+         PKG_PATCH_DIR="$PKG_DIR"
+         if [ -d  "$PKG_PATCH_DIR" ]; then
+            N=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]' | wc -l)
+         fi
+      fi
+      echo "N=$N"
+      echo "PKG_PATCH_DIR=$PKG_PATCH_DIR"
+
+      if [ $N -eq 1 ]; then
+          PATCH_DEST=$(find "$PKG_PATCH_DIR" -name "$PATCH_TARGET*" | grep -v '[/]meta_patches[/]')
+          echo "meld $PATCH_FILE -> $PATCH_DEST"
+          meld  $PATCH_FILE  $PATCH_DEST
+          if [ $? -ne 0 ]; then
+             echo "ERROR: meld failed $WORK_SRC_DIR/$PATCH_FILE -> $PATCH_DEST"
+             exit 1
+          fi
+      else
+          echo "ERROR: Don't know what destination file name to use for patch '$OLD_WORK_SRC_DIR/$PATCH_FILE', and to be copied to '$PKG_PATCH_DIR'"
+      fi
+   done
+
+   echo ""
+   echo ""
+   )
+
+   if [ $? -ne 0 ]; then
+      FAILED=$name
+      break
+   fi
+
+   )
+
+
+done
+
+if [ "$FAILED" != "" ]; then
+   echo "Failed for ... $FAILED"
+   exit 1
+fi
+
+echo ""
+for d in $(for dat in $(cat $UPVERSION_LOG); do srpm_path=$(echo $dat | awk -F '#' '{print $2}'); ( cd $(dirname $srpm_path); git rev-parse --show-toplevel ); done | sort --unique); do
+   (
+    cd $d
+    echo "cd $d"
+    for f in $(git status --porcelain | awk '{print $2}'); do 
+        echo "git add $f"; 
+    done
+    if [ "$PATCH_ID" == "" ]; then
+       echo "git commit -m 'rebased patches'"
+    else
+       echo "git commit -m 'rebased patches for patch $PATCH_ID'"
+    fi
+   )
+done
+echo ""
+
+

build-tools/sign-build

@@ -0,0 +1,506 @@
+#!/bin/bash
+
+# This script calls into an external signing server to perform signing of some
+# packages in the system.  The old packages (which are typically generated by
+# the build system and signed by placeholder keys) are overwritten by the new
+# packages.
+#
+# Three types of packages are signed:
+# kernels (both std and lowlatency, aka "rt", kernels)
+# grub
+# shim
+#
+# Kernels and grub are generated by producing (under the normal build system)
+# two packages -- a package containing the unsigned binaries, and a package
+# containing binaries signed with temporary keys.  All the "accessories" (files,
+# scripts, etc) are included in the package containing the signed-with-temp-keys
+# files.  The signing server will take both packages, sign the unsigned
+# binaries, and replace the files in the signed package with the newly signed
+# ones.
+#
+# Typical flow/artifacts
+# kernel.src.rpm      -> produces kernel.rpm and kernel-unsigned.rpm
+# kernel.rpm          -> initially contains binaries signed with a temporary key
+#                     -> contains all files used by the kernel
+#                     -> can be installed and used in a system (it just won't
+#                        secure boot since the key is just a temp key)
+# kernel-unsigned.rpm -> contains just unsigned kernel binaries
+#
+# The signing server will take both packages, sign the binaries in
+# kernel-unsigned.rpm with our private key, and replace the binaries in
+# kernel.rpm with the new binaries.  The kernel.rpm can then be replaced by the
+# version generated by the signing server.
+#
+# Shim is a bit of a different beast.
+#
+# There are two source packages - shim and shim-signed.  Frustratingly, "shim"
+# source produces a "shim-unsigned" binary output.  "shim-signed" produces a
+# "shim" binary output. 
+#
+# The "shim-signed" source RPM doesn't contain source code -- it just contains
+# instructions to take the "shim-unsigned" binaries, sign them, and package the
+# output.  We've modified the shim-signed RPM to (rather than sign with a temp
+# key) use "presigned" binaries from shim-unsigned if the files exist.  (It will
+# still use a temp key of no presigned files are found, which is how the build
+# system normally runs).
+#
+# The signing server will unpack the shim-unsigned package, sign the binaries
+# (as "presigned") and repack the package.
+#
+# A rebuild of shim-signed by the build server is then required.  
+#
+# Thanks for bearing with me in the convoluted discussion, above.
+
+
+# Script flow:
+# - call signing server to sign kernels (if they exist and are new, as with
+#   other RPMs)
+# - replace old kernel packages with newly signed ones
+# - call signing server to sign grub (and replace old version with the newly
+#   signed one)
+# - call signing server to sign shim-unsigned (replace old version)
+# - rebuild shim-signed 
+# - update our repos to advertize all newly replaced packages
+
+# check_if_pkg_needs_signing <path/to/filename.rpm>
+#
+# Checks to see if a given package needs to be signed.  We maintain a list of
+# MD5 sums for RPMs we have signed.  Thus, we can easily see if we've already
+# signed a package.
+#
+# Returns 1 if the package does need signing, or 0 if package does not
+#
+# This function expects the package specified to exist.
+function check_if_pkg_needs_signing
+{
+    local PKG_TO_CHECK=$1
+
+    if [ ! -e ${SIGNED_PKG_DB} ]; then
+        # We haven't signed anything before, so this package needs signing
+        return 1
+    fi
+
+    local SIGNED_PKG_MD5=`grep ${PKG_TO_CHECK} ${SIGNED_PKG_DB} | cut -d ' ' -f 1`
+    if [ "x${SIGNED_PKG_MD5}" == "x" ]; then
+        # We have no record of having signed the package -- needs signing
+        return 1
+    fi
+
+    local CURRENT_MD5=`md5sum ${PKG_TO_CHECK} | cut -d ' ' -f 1`
+    if [ "${CURRENT_MD5}" != "${SIGNED_PKG_MD5}" ]; then
+        # The package has been regenerated since we last signed it -- needs
+        # signing again
+        return 1
+    fi
+
+    # The package md5 sum matches the md5sum of the package when it was last
+    # signed.
+    return 0
+}
+
+# update_signed_pkg_list <path/to/filename.rpm>
+#
+# Updated our list of signed packages with the md5 sum of a recently signed
+# package.
+#
+# This function expects the package to exist.
+function update_signed_pkg_list
+{
+    local PKG_TO_ADD=$1
+
+    if [ ! -e ${SIGNED_PKG_DB} ]; then
+        touch ${SIGNED_PKG_DB}
+    fi
+
+    # remove current entry for package
+    local TMPFILE=`mktemp`
+    grep -v $(basename ${PKG_TO_ADD}) ${SIGNED_PKG_DB} > ${TMPFILE}
+    mv ${TMPFILE} ${SIGNED_PKG_DB}
+
+    # add MD5 for package to the package list
+    md5sum ${PKG_TO_ADD} >> ${SIGNED_PKG_DB}
+}
+
+
+# update_repo <std|rt>
+#
+# Updates either the standard or rt repo with latest packages
+# Checks that you specified a repo, and that the path exists.
+#
+# There are actually now two places we need to update -- the
+# rpmbuild/RPMS/ path, as well as the results/.../ path
+function update_repo
+{
+	local BUILD_TYPE=$1
+	local EXTRA_PARAMS=""
+	local RETCODE=0
+	local repopath=""
+
+	if [ "x$BUILD_TYPE" == "x" ]; then
+		return 1
+	fi
+
+	if [ "x$MY_BUILD_ENVIRONMENT_TOP" == "x" ]; then
+		return 1
+	fi
+
+	for repopath in "$MY_WORKSPACE/$BUILD_TYPE/rpmbuild/RPMS" "$MY_WORKSPACE/$BUILD_TYPE/results/${MY_BUILD_ENVIRONMENT_TOP}-$BUILD_TYPE"; do
+		if [ ! -d "$repopath" ]; then
+			echo "Error - cannot find path $repopath"
+			return 1
+		fi
+
+		cd $repopath
+		if [ -f comps.xml ]; then
+			EXTRA_PARAMS="-g comps.xml"
+		fi
+		createrepo --update $EXTRA_PARAMS . > /dev/null
+		RETCODE=$?
+		cd - > /dev/null
+		if [ 0$RETCODE -ne 0 ]; then
+			return $RETCODE
+		fi
+	done
+
+	return $RETCODE
+}
+
+# sign_shims - find and sign any shim package that we need to
+#
+function sign_shims
+{
+	SHIM=`find $MY_WORKSPACE/std/rpmbuild/RPMS -name "shim-unsigned-*.$ARCH.rpm" | grep -v debuginfo`
+	if [ "x${SHIM}" == "x" ]; then
+	    echo "Warning -- cannot find shim package to sign"
+	    return 0
+	fi
+	sign shim $SHIM
+
+	return $?
+}
+
+# sign_grubs - find and sign any grub package that we need to.
+#              Grub (and kernel) are initially signed with temporary keys, so
+#              we need to upload both the complete package, as well as the
+#              unsigned binaries
+#
+function sign_grubs
+{
+	GRUB=`find $MY_WORKSPACE/std/rpmbuild/RPMS -name "grub2-efi-[1-9]*.$ARCH.rpm"`
+	UNSIGNED_GRUB=`find $MY_WORKSPACE/std/rpmbuild/RPMS -name "grub2-efi-unsigned*.$ARCH.rpm"`
+	if [ "x${GRUB}" == "x" ]; then
+	    echo "Warning -- cannot find GRUB package to sign"
+	    return 0
+	fi
+	if [ "x${UNSIGNED_GRUB}" == "x" ]; then
+	    echo "Warning -- cannot find unsigned GRUB package to sign"
+	    return 0
+	fi
+
+	sign grub2 $GRUB $UNSIGNED_GRUB
+	return $?
+}
+
+# sign_kernels - find and sign any kernel package that we need to.
+#
+function sign_kernels
+{
+    sign_kernel "std" ""
+    sign_kernel "rt" "-rt"
+}
+
+# sign_kernel - find and sign kernel package if we need to.
+#              Kernels (and grub) are initially signed with temporary keys, so
+#              we need to upload both the complete package, as well as the
+#              unsigned binaries
+function sign_kernel
+{
+	local KERNEL_PATH=$1
+	local KERNEL_EXTRA=$2
+	KERNEL=`find $MY_WORKSPACE/${KERNEL_PATH}/rpmbuild/RPMS -name "kernel${KERNEL_EXTRA}-[1-9]*.$ARCH.rpm"`
+	UNSIGNED_KERNEL=`find $MY_WORKSPACE/${KERNEL_PATH}/rpmbuild/RPMS -name "kernel${KERNEL_EXTRA}-unsigned-[1-9]*.$ARCH.rpm"`
+	if [ "x${KERNEL}" == "x" ]; then
+	    echo "Warning -- cannot find kernel package to sign in ${KERNEL_PATH}"
+	    return 0
+	fi
+	if [ "x${UNSIGNED_KERNEL}" == "x" ]; then
+	    echo "Warning -- cannot find unsigned kernel package to sign in ${KERNEL_PATH}"
+	    return 0
+	fi
+
+	sign kernel $KERNEL $UNSIGNED_KERNEL
+	if [ $? -ne 0 ]; then
+		return $?
+	fi
+}
+
+# rebuild_pkgs - rebuild any packages that need to be updated from the newly
+# signed binaries
+#
+function rebuild_pkgs
+{
+	local LOGFILE="$MY_WORKSPACE/export/signed-rebuild.log"
+	local PKGS_TO_REBUILD=${REBUILD_LIST}
+
+	if [ "x${PKGS_TO_REBUILD}" == "x" ]; then
+	    # No rebuilds required, return cleanly
+	    return 0
+	fi
+
+	# If we reach this point, then we have one or more packages to be rebuilt
+
+	# first, update the repo so it is aware of the "latest" binaries
+	update_repo std
+	if [ $? -ne 0 ]; then
+		echo "Could not update signed packages -- could not update repo"
+		return 1
+	fi
+
+        echo "Performing rebuild of packages: $PKGS_TO_REBUILD"
+        FORMAL_BUILD=0 build-pkgs --no-descendants --no-build-info --no-required --careful $PKGS_TO_REBUILD > $LOGFILE 2>&1
+
+	if [ $? -ne 0 ]; then
+		echo "Could not rebuild packages: $PKGS_TO_REBUILD -- see $LOGFILE for details"
+		return 1
+	fi
+
+	echo "Done"
+	return 0
+}
+
+# sign <type_of_pkg> <pkg> [pkg_containing_unsigned_bins]
+#
+# This routine uploads a package to the signing server, instructs the signing
+# signing server to do its' magic, and downloads the updated (signed) package
+# from the signing server.
+#
+# Accessing the signing server -- the signing server cannot just be logged
+# into by anyone.  A small number of users (Jason McKenna, Scott Little, Greg
+# Waines, etc) have permission to log in as themselves.  In addition, there is
+# a user "signing" who is unique to the server.  The "jenkins" user on our
+# build servers has permission to login/upload files as "signing" due to Jenkins'
+# private SSH key being added to the signing user's list of keys.  This means
+# that Jenkins can upload and run commands on the server as "signing".
+#
+# In addition to uploading files as signing, the signing user has permissions to
+# run a single command (/opt/signing/sign.sh) as a sudo root user.  The signing
+# user does not have access to modify the script or to run any other commands as
+# root.  The sign.sh script will take inputs (the files that jenkins has
+# uploaded), verify the contents, sign the images against private keys, and
+# output a new .rpm contianing the signed version of the files.  Assuming all
+# is successful, the filename of the signed output file is returned, and the
+# jenkins user can then use that filename to download the file (the "signing"
+# user does not have access to remove or modify the file once it's created).
+#
+# All operations done on the signing server are logged in muliple places, and
+# the output RPM artifacts are timestamped to ensure that they are not
+# overwritten by subsequent calls to sign.sh.
+#
+# kernel and grub package types require you to specify/upload the unsigned
+# packages as well as the normal binary
+function sign
+{
+	local TYPE=$1
+	local FILE=$2
+	local UNSIGNED=$3
+	local UNSIGNED_OPTION=""
+	local TMPFILE=`mktemp /tmp/sign.XXXXXXXX`
+
+	# Don't sign if we've already signed it
+	check_if_pkg_needs_signing ${FILE}
+	if [ $? -eq 0 ]; then
+		echo "Not signing ${FILE} as we previously signed it"
+		return 0
+	fi
+
+	echo "Signing $FILE"
+
+	# upload the original package
+	scp -q $FILE $SIGNING_USER@$SIGNING_SERVER:$UPLOAD_PATH
+	if [ $? -ne 0 ]; then
+		echo "Failed to upload file $FILE"
+		\rm -f $TMPFILE
+		return 1
+	fi
+	
+	# upload the unsigned package (if specified)
+	if [ "x$UNSIGNED" != "x" ]; then	
+		scp -q $UNSIGNED $SIGNING_USER@$SIGNING_SERVER:$UPLOAD_PATH
+		if [ $? -ne 0 ]; then
+			echo "Failed to upload file $UNSIGNED"
+			\rm -f $TMPFILE
+			return 1
+		fi
+		UNSIGNED=$(basename $UNSIGNED)
+		UNSIGNED_OPTION="-u $UPLOAD_PATH/$UNSIGNED"
+	fi
+
+	# Call the magic script on the signing server.  Note that the user
+	# ($SIGNING_USER) has sudo permissions but only to invoke this one script.
+	# The signing user cannot make other sudo calls.
+	#
+	# We place output in $TMPFILE to extract the output file name later
+	#
+	ssh $SIGNING_USER@$SIGNING_SERVER sudo $SIGNING_SCRIPT -i $UPLOAD_PATH/$(basename $FILE) $UNSIGNED_OPTION -t $TYPE > $TMPFILE 2>&1
+	if [ $? -ne 0 ]; then
+		echo "Signing of $FILE failed"
+		\rm -f $TMPFILE
+		return 1
+	fi
+	
+	# The signing server script will output the name by which the newly signed
+	# RPM can be found.  This will be a unique filename (based on the unique
+	# upload directory generated by the "-r" option above).
+	#
+	# The reason for this is so that we can archive all output files
+	# and examine them later without them being overwriten.  File paths are
+	# typically of the form
+	#
+	# /export/signed_images/XXXXXXX_grub2-efi-2.02-0.44.el7.centos.tis.3.x86_64.rpm
+	#
+	# Extract the output name, and copy the RPM back into our system
+	# (Note that we overwrite our original version of the RPM)
+	#
+	# Note that some packages (like grub) may produce multiple output RPMs (i.e.
+	# multiple lines list output files.
+	OUTPUT=`grep "Output written:" $TMPFILE | sed "s/Output written: //"`
+	
+	# Check that we got something
+	if [ "x$OUTPUT" == "x" ]; then
+		echo "Could not determine output file -- check logs on signing server for errors"
+		\cp $TMPFILE $MY_WORKSPACE/export/signing.log
+		\rm -f $TMPFILE
+		return 1
+	fi
+
+	# The signing script can return multiple output files, if appropriate for
+	# the input RPM source type.  Copy each output RPM to our repo
+	# Note that after we download the file we extract the base package name
+	# from the RPM to find the name of the file that it *should* be named
+	#
+	# example:
+	#   we'd download "Zrqyeuzw_kernel-3.10.0-514.2.2.el7.20.tis.x86_64.rpm"
+	#   we'd figure out that the RPM name should be "kernel"
+	#   we look for "kernel" in the RPM filename, and rename
+	#     "Zrqyeuzw_kernel-3.10.0-514.2.2.el7.20.tis.x86_64.rpm" to
+	#     "kernel-3.10.0-514.2.2.el7.20.tis.x86_64.rpm"
+	while read OUTPUT_FILE; do
+
+		# Download the file from the signing server
+		local DOWNLOAD_FILENAME=$(basename $OUTPUT_FILE)
+		scp -q $SIGNING_USER@$SIGNING_SERVER:$OUTPUT_FILE $(dirname $FILE)
+		if [ $? -ne 0 ]; then
+			\rm -f $TMPFILE
+			echo "Copying file from signing server failed"
+			return 1
+		fi
+		echo "Successfully retrieved $OUTPUT_FILE"
+
+		# figure out what the file should be named (strip away leading chars)
+		local RPM_NAME=`rpm -qp $(dirname $FILE)/$DOWNLOAD_FILENAME --qf="%{name}"`
+		local CORRECT_OUTPUT_FILE_NAME=`echo $DOWNLOAD_FILENAME | sed "s/^.*$RPM_NAME/$RPM_NAME/"`
+
+		# rename the file
+		\mv -f $(dirname $FILE)/$DOWNLOAD_FILENAME $(dirname $FILE)/$CORRECT_OUTPUT_FILE_NAME
+
+		# replace the version of the file in results
+		#
+		# Potential hiccup in future -- this code currenty replaces any output file in EITHER
+		# std or rt results which matches the filename we just downloaded from the signing.
+		# server.  This means there could be an issue where we sign something-ver-rel.arch.rpm
+		# but we expect different versions of that RPM in std and in rt.  Currently, we do not
+		# have any RPMs which have that problem (all produced RPMs in rt have the "-rt" suffix
+		# let along any "signed" rpms) but it's something of which to be aware.
+		#
+		# Also, note that we do not expect multiple RPMs in each repo to have the same filename.
+		# We use "head -n 1" to handle that, but again it shouldn't happen.
+		# 
+		for buildtype in std rt; do
+			x=`find $MY_WORKSPACE/$buildtype/results/${MY_BUILD_ENVIRONMENT_TOP}-$buildtype -name $CORRECT_OUTPUT_FILE_NAME | head -n 1`
+			if [ ! -z "$x" ]; then
+				cp $(dirname $FILE)/$CORRECT_OUTPUT_FILE_NAME $x
+			fi
+		done
+
+		echo "Have signed file $(dirname $FILE)/$CORRECT_OUTPUT_FILE_NAME"
+	done <<< "$OUTPUT"
+
+	\rm -f $TMPFILE
+
+	# If we just signed a shim package, flag that shim needs to be rebuilt
+	if [ "${TYPE}" == "shim" ]; then
+		REBUILD_LIST="${REBUILD_LIST} shim-signed"
+	fi
+
+	echo "Done"
+	update_signed_pkg_list ${FILE}
+
+	return 0
+}
+
+# Main script
+
+if [ "x$MY_WORKSPACE" == "x" ]; then
+	echo "Environment not set up -- abort"
+	exit 1
+fi
+
+ARCH="x86_64"
+SIGNING_SERVER=yow-tiks01
+SIGNING_USER=signing
+SIGNING_SCRIPT=/opt/signing/sign.sh
+UPLOAD_PATH=`ssh $SIGNING_USER@$SIGNING_SERVER sudo $SIGNING_SCRIPT -r`
+SIGNED_PKG_DB=${MY_WORKSPACE}/signed_pkg_list.txt
+REBUILD_LIST=""
+MY_BUILD_ENVIRONMENT_TOP=${MY_BUILD_ENVIRONMENT_TOP:-$MY_BUILD_ENVIRONMENT}
+
+# Check that we were able to request a unique path for uploads
+echo $UPLOAD_PATH | grep -q "^Upload:"
+if [ $? -ne 0 ]; then
+	echo "Failed to get upload path -- abort"
+	exit 1
+fi
+UPLOAD_PATH=`echo $UPLOAD_PATH | sed "s%^Upload: %%"`
+
+sign_kernels
+if [ $? -ne 0 ]; then
+	echo "Failed to sign kernels -- abort"
+	exit 1
+fi
+
+sign_shims
+if [ $? -ne 0 ]; then
+	echo "Failed to sign shims -- abort"
+	exit 1
+fi
+
+sign_grubs
+if [ $? -ne 0 ]; then
+	echo "Failed to sign grubs -- abort"
+	exit 1
+fi
+
+update_repo std
+if [ $? -ne 0 ]; then
+	echo "Failed to update std repo -- abort"
+	exit 1
+fi
+
+rebuild_pkgs
+if [ $? -ne 0 ]; then
+	echo "Failed to update builds with signed dependancies -- abort"
+	exit 1
+fi
+
+update_repo std
+if [ $? -ne 0 ]; then
+	echo "Failed to update std repo -- abort"
+	exit 1
+fi
+
+update_repo rt
+if [ $? -ne 0 ]; then
+	echo "Failed to update rt repo -- abort"
+	exit 1
+fi
+

build-tools/sign-rpms

@@ -0,0 +1,258 @@
+#!/bin/bash
+
+# Add file signature to RPMs
+#
+# This script will add file signature to rpms in a given directory.
+# The directory containing the RPMs must be passed as a parameter. There is no default location.
+#
+#
+
+usage () {
+    echo ""
+    echo "Usage: "
+    echo "   sign-rpms -d|--pkg-dir <directory>"
+    echo "   -d --pkg-dir <directory> directory contain the RPMs to sign"
+    echo "   -h|--help this message"
+    echo ""
+}
+
+# number of processors. The process will use all available processors by default.
+NPROCS=$(nproc)
+
+export MOCK=/usr/bin/mock
+
+# check input variables
+function check_vars {
+   # need access to repo, which should normally be defined as MY_REPO in the env
+
+   if [ ! -z "$MY_REPO" ] && [ -d "$MY_REPO" ] ; then
+      INTERNAL_REPO_ROOT=$MY_REPO
+   fi
+
+   if [ -z "$INTERNAL_REPO_ROOT" ] ; then
+      printf "  unable to use \$MY_REPO (value \"$MY_REPO\")\n"
+      printf "  -- checking \$MY_REPO_ROOT_DIR (value \"$MY_REPO_ROOT_DIR\")\n"
+      if [ ! -z "$MY_REPO_ROOT_DIR" ] && [ -d "$MY_REPO_ROOT_DIR/cgcs-root" ] ; then
+          INTERNAL_REPO_ROOT=$MY_REPO_ROOT_DIR/cgcs-root
+          printf "  Found!\n"
+      fi
+   fi
+
+   if [ -z "$INTERNAL_REPO_ROOT" ] ; then
+      printf "  No joy -- checking for \$MY_WORKSPACE/cgcs-root\n"
+      if [ -d "$MY_WORKSPACE/cgcs-root" ] ; then
+          INTERNAL_REPO_ROOT=$MY_WORKSPACE/cgcs-root
+          printf "  Found!\n"
+      fi
+   fi
+
+   if [ -z "$INTERNAL_REPO_ROOT" ] ; then
+      printf "  Error -- could not locate cgcs-root repo.\n"
+      exit 1
+   fi
+
+   if [ -z "$MY_BUILD_ENVIRONMENT" ] ; then
+      printf "  Error -- missing environment variable MY_BUILD_ENVIRONMENT"
+      exit 1
+   fi
+
+   if [ -z "$MY_BUILD_DIR" ] ; then
+      printf "  Error -- missing environment variable MY_BUILD_DIR"
+      exit 1
+   fi
+
+}
+
+#
+# this function will add IMA file signatures to all rpms in the Packages directory
+#
+# the process will copy the signing key and a makefile in the mock env under /tmp
+# it will also mount the Packages directory under /mnt/Packages
+# then mock will be invoked to sign the packages
+#
+# This process is using mock because the build servers do not have the same rpm / rpmsign version
+#
+
+
+function sign_packages {
+   OLD_PWD=$PWD
+
+   _MOCK_PKG_DIR=/mnt/Packages
+   _IMA_PRIV_KEY=ima_signing_key.priv
+   _KEY_DIR=$MY_REPO/build-tools/signing
+   _MOCK_KEY_DIR=/mnt/keys
+   _SIGN_MAKEFILE=_sign_pkgs.mk
+   _MK_DIR=$MY_REPO/build-tools/mk
+   _MOCK_MK_DIR=/mnt/mk
+
+   # mock confgiuration file
+   _MOCK_CFG=$MY_BUILD_DIR/${MY_BUILD_ENVIRONMENT}-sign.cfg
+
+   # recreate configuration file 
+   rm $_MOCK_CFG
+   export BUILD_TYPE=std
+   export MY_BUILD_DIR_TOP=$MY_BUILD_DIR
+   modify-build-cfg $_MOCK_CFG
+   #  and customize
+   echo "config_opts['chroot_setup_cmd'] = 'install shadow-utils make rpm-sign'" >> $_MOCK_CFG
+   echo "config_opts['root'] = 'mock-sign'" >> $_MOCK_CFG
+   echo "config_opts['basedir'] = '${MY_WORKSPACE}'" >> $_MOCK_CFG
+   echo "config_opts['cache_topdir'] = '${MY_WORKSPACE}/mock-cache'" >> $_MOCK_CFG
+
+   echo "Signing packages in $_PKG_DIR with $NPROCS threads"
+   echo "using development key $_KEY_DIR/$_IMA_PRIV_KEY"
+
+   printf "Initializing mock environment\n"
+
+   # invoke make in mock to sign packages.
+   # this call will also create and initialize the mock env
+   eval $MOCK -q -r $_MOCK_CFG \'--plugin-option=bind_mount:dirs=[\(\"$_PKG_DIR\", \"$_MOCK_PKG_DIR\"\),\(\"$_MK_DIR\",\"$_MOCK_MK_DIR\"\),\(\"$_KEY_DIR\",\"$_MOCK_KEY_DIR\"\)]\' --shell \"cd $_MOCK_PKG_DIR\; make -j $NPROCS -f $_MOCK_MK_DIR/$_SIGN_MAKEFILE KEY=$_MOCK_KEY_DIR/$_IMA_PRIV_KEY\"
+
+   retval=$?
+
+   printf "Cleaning mock environment\n"
+   $MOCK -q -r $_MOCK_CFG --scrub=all
+
+   if [ $retval -ne 0 ] ; then
+      echo "failed to add file signatures to RPMs in mock environment."
+      return $retval
+   fi
+
+   cd $OLD_PWD
+
+}
+
+function _copy_and_sign {
+
+   # upload rpms to server
+   scp $_PKG_DIR/*.rpm $SIGNING_USER@$SIGNING_SERVER:$_UPLOAD_DIR
+   retval=$?
+   if [ $retval -ne 0 ] ; then
+      echo "ERROR: failed to copy RPM files to signing server."
+      return $retval
+   fi
+
+   # get server to sign packages.
+   ssh $SIGNING_USER@$SIGNING_SERVER -- sudo $SIGNING_SERVER_SCRIPT -s -d $sub
+   retval=$?
+   if [ $retval -ne 0 ] ; then
+      echo "ERROR: failed to sign RPM files."
+      return $retval
+   fi
+
+   # download results back. This overwrites the original files.
+   scp $SIGNING_USER@$SIGNING_SERVER:$_UPLOAD_DIR/*.rpm $_PKG_DIR
+   retval=$?
+   if [ $retval -ne 0 ] ; then
+      echo "ERROR: failed to copy signed RPM files back from signing server."
+      return $retval
+   fi
+
+   return $retval
+
+}
+
+
+function sign_packages_on_server {
+
+   retval=0
+
+   # obtain temporary diretory to upload RPMs on signing server
+   _UPLOAD_DIR=`ssh $SIGNING_USER@$SIGNING_SERVER -- sudo $SIGNING_SERVER_SCRIPT -r`
+
+   retval=$?
+   if [ $retval -ne 0 ] ; then
+      echo "failed to obtain upload directory from signing server."
+      return $retval
+   fi
+
+   # extract base chroot dir and rpm dir within chroot
+   read base com sub <<< $_UPLOAD_DIR
+
+   # this is the upload temp dir, outside of chroot env
+   _UPLOAD_DIR=$base$sub
+
+   _copy_and_sign
+   retval=$?
+
+   # cleanup
+   ssh $SIGNING_USER@$SIGNING_SERVER rm $_UPLOAD_DIR/*.rpm
+   if [ $? -ne 0 ] ; then
+      echo "Warning : failed to remove rpms from temporary upload directory."
+   fi
+   ssh $SIGNING_USER@$SIGNING_SERVER rmdir $_UPLOAD_DIR
+   if [ $? -ne 0 ] ; then
+      echo "Warning : failed to remove temporary upload directory."
+   fi
+
+   return $retval
+}
+
+
+
+#############################################
+# Main code
+#############################################
+
+# Check args
+HELP=0
+SIGNING_SERVER=yow-tiks01
+SIGNING_USER=signing
+SIGNING_SERVER_SCRIPT=/opt/signing/sign_rpms_18.03.sh
+
+# return value
+retval=0
+
+# read the options
+TEMP=`getopt -o hd: --long help,pkg-dir: -n 'test.sh' -- "$@"`
+if [ $? -ne 0 ] ; then
+   echo "Invalid parameters - exiting"
+   exit 1
+fi
+
+eval set -- "$TEMP"
+
+# extract options and their arguments into variables.
+while true ; do
+    case "$1" in
+        -h|--help) HELP=1 ; shift ;;
+        -d|--pkg-dir) _PKG_DIR="$2"; shift; shift ;;
+        --) shift ; break ;;
+        *) echo "Internal error : unexpected parameter $2" ; exit 1 ;;
+    esac
+done
+
+if [ $HELP -eq 1 ]; then
+   usage
+   exit 0
+fi
+
+# package directory must be defined
+if [ -z "$_PKG_DIR" ]; then
+   echo "Need package directory. Use -d/--pkg-dir option"
+   usage
+   exit 1
+fi
+
+# ... and must exist
+if [ ! -d "$_PKG_DIR" ]; then
+   echo "Package directory $_PKG_DIR does not exist"
+   exit 1
+fi
+
+# Init variables
+check_vars
+
+echo signing $_PKG_DIR
+
+# sign all rpms
+if [ "$USER" == "jenkins" ]; then
+    sign_packages_on_server
+    retval=$?
+else
+    sign_packages
+    retval=$?
+fi
+
+exit $retval
+

build-tools/sign_iso_formal.sh

@@ -0,0 +1,62 @@
+#!/bin/bash
+
+# This script makes a request to the signing server to sign a .iso with the
+# formal key.  It will only work for users authorized to access the signing
+# server.  The detached signature is placed in the same path as the .iso as
+# the file bootimage.sig
+#
+# Script written to be quite simple
+
+if [ "x$1" == "x" ]; then
+    echo "You must specify an ISO file to sign"
+    exit 1
+fi
+
+ISO_FILE_PATH=$1
+ISO_FILE_NAME=$(basename ${ISO_FILE_PATH})
+ISO_FILE_ROOT=$(dirname ${ISO_FILE_PATH})
+ISO_FILE_NOEXT="${ISO_FILE_NAME%.*}"
+SIGNING_SERVER="signing@yow-tiks01"
+GET_UPLOAD_PATH="sudo /opt/signing/sign.sh -r"
+REQUEST_SIGN="sudo /opt/signing/sign_iso.sh"
+SIGNATURE_FILE="$ISO_FILE_NOEXT.sig"
+
+# Make a request for an upload path
+# Output is a path where we can upload stuff, of the form
+# "Upload: /tmp/sign_upload.5jR11pS0"
+UPLOAD_PATH=`ssh ${SIGNING_SERVER} ${GET_UPLOAD_PATH}`
+if [ $? -ne 0 ]; then
+    echo "Could not get upload path.  Do you have permissions on the signing server?"
+    exit 1
+fi
+UPLOAD_PATH=`echo ${UPLOAD_PATH} | cut -d ' ' -f 2`
+
+echo "Uploading file"
+scp -q ${ISO_FILE_PATH} ${SIGNING_SERVER}:${UPLOAD_PATH}
+if [ $? -ne 0 ]; then
+    echo "Could not upload ISO"
+    exit 1
+fi
+echo "File uploaded to signing server -- signing"
+
+# Make the signing request.
+# Output is path of detached signature
+RESULT=`ssh ${SIGNING_SERVER} ${REQUEST_SIGN} ${UPLOAD_PATH}/${ISO_FILE_NAME}`
+if [ $? -ne 0 ]; then
+    echo "Could not perform signing -- output $RESULT"
+    ssh ${SIGNING_SERVER} rm -f ${UPLOAD_PATH}/${ISO_FILE_NAME}
+    exit 1
+fi
+
+echo "Signing complete.  Downloading detached signature"
+scp -q ${SIGNING_SERVER}:${RESULT} ${ISO_FILE_ROOT}/${SIGNATURE_FILE}
+if [ $? -ne 0 ]; then
+    echo "Could not download newly signed file"
+    ssh ${SIGNING_SERVER} rm -f ${UPLOAD_PATH}/${ISO_FILE_NAME}
+    exit 1
+fi
+
+# Clean up (ISOs are big)
+ssh ${SIGNING_SERVER} rm -f ${UPLOAD_PATH}/${ISO_FILE_NAME}
+
+echo "${ISO_FILE_ROOT}/${SIGNATURE_FILE} detached signature"

build-tools/sign_patch_formal.sh

@@ -0,0 +1,51 @@
+#!/bin/bash
+
+# This script makes a request to the signing server to sign a .patch with the
+# formal key.  It will only work for users authorized to access the signing
+# server.
+#
+# Script written to be quite simple
+
+if [ "x$1" == "x" ]; then
+    echo "You must specify a patch file to sign"
+    exit 1
+fi
+
+PATCH_FILE_PATH=$1
+PATCH_FILE_NAME=$(basename ${PATCH_FILE_PATH})
+SIGNING_SERVER="signing@yow-tiks01"
+GET_UPLOAD_PATH="sudo /opt/signing/sign.sh -r"
+REQUEST_SIGN="sudo /opt/signing/sign_patch.sh"
+
+# Make a request for an upload path
+# Output is a path where we can upload stuff, of the form
+# "Upload: /tmp/sign_upload.5jR11pS0"
+UPLOAD_PATH=`ssh ${SIGNING_SERVER} ${GET_UPLOAD_PATH}`
+if [ $? -ne 0 ]; then
+    echo "Could not get upload path.  Do you have permissions on the signing server?"
+    exit 1
+fi
+UPLOAD_PATH=`echo ${UPLOAD_PATH} | cut -d ' ' -f 2`
+
+scp -q ${PATCH_FILE_PATH} ${SIGNING_SERVER}:${UPLOAD_PATH}
+if [ $? -ne 0 ]; then
+    echo "Could upload patch"
+    exit 1
+fi
+echo "File uploaded to signing server"
+
+# Make the signing request.
+# Output is path of newly signed file
+RESULT=`ssh ${SIGNING_SERVER} ${REQUEST_SIGN} ${UPLOAD_PATH}/${PATCH_FILE_NAME}`
+if [ $? -ne 0 ]; then
+    echo "Could not perform signing -- output $RESULT"
+    exit 1
+fi
+
+echo "Signing complete.  Downloading"
+scp -q ${SIGNING_SERVER}:${RESULT} ${PATCH_FILE_PATH}
+if [ $? -ne 0 ]; then
+    echo "Could not download newly signed file"
+    exit 1
+fi
+echo "${PATCH_FILE_PATH} now signed with formal key"