Closes-Bug: 1883327 Change-Id: I8045bf88d613adee73a01fceb645874e26c157ec Signed-off-by: MCamp859 <maryx.camp@intel.com>
		
			
				
	
	
		
			1177 lines
		
	
	
		
			48 KiB
		
	
	
	
		
			ReStructuredText
		
	
	
	
	
	
			
		
		
	
	
			1177 lines
		
	
	
		
			48 KiB
		
	
	
	
		
			ReStructuredText
		
	
	
	
	
	
===================
 | 
						|
Packaging Reference
 | 
						|
===================
 | 
						|
 | 
						|
This guide contains information about creating StarlingX packages.
 | 
						|
 | 
						|
.. contents::
 | 
						|
   :local:
 | 
						|
   :depth: 1
 | 
						|
 | 
						|
---------------
 | 
						|
Tips and tricks
 | 
						|
---------------
 | 
						|
 | 
						|
Here are some things to consider when creating packages:
 | 
						|
 | 
						|
*   Make sure you have the correct BuildRequires dependencies listed. Mock will
 | 
						|
    ensure that these packages are installed in the build environment before
 | 
						|
    trying to build the package in question. Be aware that CentOS often uses
 | 
						|
    ``-devel`` for developer packages (which typically include headers and the
 | 
						|
    ``.so`` shared library link) whereas StarlingX generally uses ``-dev``. For
 | 
						|
    BuildRequires, you can use the following:
 | 
						|
 | 
						|
    *   Instead of ``virtual/x86_64-wrs-linux-gcc``,  you can use ``gcc``.
 | 
						|
    *   Instead of ``glib-2.0``, use ``glib2-devel``.
 | 
						|
    *   Instead of ``sqlite3``, use ``sqlite-devel``.
 | 
						|
    *   You can comment out or delete ``virtual/libc`` and
 | 
						|
        ``virtual/x86_64-wrs-linux-compilerlibs``.
 | 
						|
 | 
						|
*   Typically, you do not need to manually add dependencies due to shared
 | 
						|
    libraries to either Requires or Provides in the spec file. RPM will
 | 
						|
    usually add them automatically.
 | 
						|
 | 
						|
*   When your package creates a shared library, make sure to install it with 755
 | 
						|
    (or other suitably executable) permissions. The automatic dependency
 | 
						|
    handling only runs if the library file is actually executable.
 | 
						|
 | 
						|
*   It can be useful to refer to a bitbake-generated spec file, especially for
 | 
						|
    STX-specific packages. Generally the ``build`` and ``install`` phases are
 | 
						|
    missing from the bitbake-generated spec file and you must add them.
 | 
						|
 | 
						|
*   Make sure all necessary files are listed, because the build_srpm phase will
 | 
						|
    only include source files listed in your spec file. In a simple case, this
 | 
						|
    can be just the following line: ``Source: %{name}-%{version}.tar.gz``
 | 
						|
 | 
						|
------------------------------------
 | 
						|
STX authored package with a makefile
 | 
						|
------------------------------------
 | 
						|
 | 
						|
This section uses ``fm-common`` as a sample package to convert.
 | 
						|
 | 
						|
#.  Locate the root directory for the package.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        cd wr-cgcs/layers/cgcs/middleware/fault/recipes-common/fm-common
 | 
						|
 | 
						|
#.  Create a directory template in the same location as your current bitbake
 | 
						|
    recipe. The presence of a ``centos`` directory will identify packages to be
 | 
						|
    compiled for STX on CentOS.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        mkdir -p centos
 | 
						|
        mkdir -p centos/files
 | 
						|
 | 
						|
#.  The makefile requires an ``install`` build target. However, bitbake will try
 | 
						|
    to use ``install`` if it is introduced when it wasn't previously present, so
 | 
						|
    we'll use ``install_non_bb`` instead.
 | 
						|
 | 
						|
    The makefile takes the following parameters: ``DEST_DIR``, ``BIN_DIR``,
 | 
						|
    ``LIB_DIR``, ``INC_DIR``, ``MAJOR``, ``MINOR``.
 | 
						|
 | 
						|
    The basic strategy is to construct the directory tree using
 | 
						|
    ``install -m 755 -d some_dir``, then copy files using
 | 
						|
    ``install -m 755 from_file some_dir/to_file``
 | 
						|
 | 
						|
    For versioned libraries, you must create non- or lesser-versioned symlinks,
 | 
						|
    for example:
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        ln -s my_lib.so.1.0 some_dir/my_lib.so.1
 | 
						|
 | 
						|
    Typically, you do not need the ``.so`` installed on the target (it's
 | 
						|
    generally only needed for building against). If you do, then it would be
 | 
						|
    linked as ``ln -s my_lib.so.1.0 some_dir/my_lib.so`` and would usually be
 | 
						|
    part of a ``-dev`` package. Don't forget tabs!!!
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    vi sources/Makefile
 | 
						|
    install_non_bb:
 | 
						|
            install -m 755 -d $(DEST_DIR)$(BIN_DIR)
 | 
						|
            install -m 755 -d $(DEST_DIR)$(LIB_DIR)
 | 
						|
            install -m 755 -d $(DEST_DIR)$(INC_DIR)
 | 
						|
            install -m 755 fmClientCli $(DEST_DIR)$(BIN_DIR)
 | 
						|
            install -m 644 fmDbAPI.h $(DEST_DIR)$(INC_DIR)
 | 
						|
            install -m 644 fmAPI.h $(DEST_DIR)$(INC_DIR)
 | 
						|
            install -m 644 fmThread.h $(DEST_DIR)$(INC_DIR)
 | 
						|
            install -m 644 fmAlarm.h $(DEST_DIR)$(INC_DIR)
 | 
						|
            install -m 644 $(LIBFMCOMMON_SO).$(MAJOR).$(MINOR) $(DEST_DIR)$(LIB_DIR)
 | 
						|
            ln -s $(LIBFMCOMMON_SO).$(MAJOR).$(MINOR) $(DEST_DIR)$(LIB_DIR)/$(LIBFMCOMMON_SO).$(MAJOR)
 | 
						|
            ln -s $(LIBFMCOMMON_SO).$(MAJOR).$(MINOR) $(DEST_DIR)$(LIB_DIR)/$(LIBFMCOMMON_SO)
 | 
						|
 | 
						|
***********************
 | 
						|
Create an RPM spec file
 | 
						|
***********************
 | 
						|
 | 
						|
Put the RPM spec file under the ``centos`` directory. This is the rpm version
 | 
						|
of a bitbake recipe.
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    vi centos/.spec
 | 
						|
 | 
						|
RPM spec files support macros and variable. Reference the macro with the
 | 
						|
``%{macro_name}`` syntax. Many macros are already defined to help you, such as,
 | 
						|
``%{version}`` refers to the ``Version:`` entry in the description section. You
 | 
						|
can also define your own macros, for example ``%define local_dir /usr/local``
 | 
						|
creates the macro ``local_dir``. Fill in the following sections:
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    # A short description of the packaged software
 | 
						|
    summary:
 | 
						|
 | 
						|
    # Defines what the package will actually be called
 | 
						|
    Name:
 | 
						|
 | 
						|
    # Should be set to the version of the software being packaged
 | 
						|
    Version:
 | 
						|
 | 
						|
    # A number that is used to represent the number of times the software, at the present version, has been packaged
 | 
						|
    Release:
 | 
						|
 | 
						|
    # Give a list of the source files found in the $BUILD_DIR/SOURCES subdirectory
 | 
						|
    # If git managed, usually just one, a tarball that we will create via build script e.g. Source0: %{name}-%{version}.tar.gz
 | 
						|
    # Alternatively name the tarball that is brought in via COPY_LIST from $STX_BASE/downloads directory.
 | 
						|
    # Many source files can be specified... list any additional files in $BUILD_DIR/SOURCES (likely placed there via COPY_LIST)
 | 
						|
    Source0: %{name}-%{version}.tar.gz
 | 
						|
    Source1: %{name}.conf
 | 
						|
    Source2: %(name}.init
 | 
						|
 | 
						|
    # List of build requirements
 | 
						|
    BuildRequires:
 | 
						|
 | 
						|
    # List of runtime requirements
 | 
						|
    Requires:
 | 
						|
 | 
						|
    # Describe the package
 | 
						|
    %description
 | 
						|
 | 
						|
    # Describes how to unpack the compressed packages (the setup macro will do it for you)
 | 
						|
    %prep
 | 
						|
    %setup
 | 
						|
 | 
						|
    # invoke 'make'
 | 
						|
    %build
 | 
						|
 | 
						|
    # invoke 'make install'
 | 
						|
    %install
 | 
						|
 | 
						|
    # Perhaps call 'make clean' or other cleanup scripts
 | 
						|
    %clean
 | 
						|
 | 
						|
    # List of files to include in the rpm
 | 
						|
    %files
 | 
						|
 | 
						|
    # Apply these permission/ownership overrides and the files that follow
 | 
						|
    %defattr(-,root,root,-)
 | 
						|
 | 
						|
    # Send this file to the standard path for documentation
 | 
						|
    %doc some_file
 | 
						|
 | 
						|
    # Create a directory when the rpm is installed
 | 
						|
    %dir some_directory
 | 
						|
 | 
						|
    # Install this file on this path
 | 
						|
    %some_file_path
 | 
						|
 | 
						|
 | 
						|
One spec file can create multiple related rpm files. One common usage is to
 | 
						|
place the ``*.so`` file (but not the ``*.so.*`` files) for a shared library and
 | 
						|
its ``.h`` interface files in a ``-dev`` or ``-devel`` package. The Bitbake
 | 
						|
naming convention is ``-dev``, while CentOS uses ``-devel``. Most of the
 | 
						|
StarlingX packages use the ``-dev`` naming convention.
 | 
						|
 | 
						|
You can create additional packages with ``%package -n package_name``, or
 | 
						|
``%package suffix`` which creates a package named ``first_package_name-suffix``.
 | 
						|
 | 
						|
Following ``%package``, add a unique ``Summary:``, and override any values from the
 | 
						|
first package. These additional packages also require a ``%description, %files``
 | 
						|
sections, and may require ``%prep, %setup, %build, %install, %clean`` sections. All
 | 
						|
these extra sections use the same ``-n package_name`` or ``suffix`` arguments to
 | 
						|
define which package they apply to. Refer to the example below, which uses
 | 
						|
``fm-common-dev``.
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    %package -n fm-common-dev Summary: STX Platform Fault Management
 | 
						|
    Common Package - Development files Group: devel Requires: fm-common =
 | 
						|
    %{version}-%{release} ... %description -n fm-common-dev ... %files -n
 | 
						|
    fm-common-dev ...
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    vi centos/fm-common.spec
 | 
						|
    %define local_dir /usr/local
 | 
						|
    %define local_bindir %{local_dir}/bin
 | 
						|
 | 
						|
    Summary: STX Platform Fault Management Common Package
 | 
						|
    Name: fm-common
 | 
						|
    Version: 1.0
 | 
						|
    Release: r26.0
 | 
						|
    License: Apache-2.0
 | 
						|
    Group: base
 | 
						|
    URL: unknown
 | 
						|
    Source0: %{name}-%{version}.tar.gz
 | 
						|
    BuildRequires: util-linux
 | 
						|
    BuildRequires: postgresql-devel
 | 
						|
    BuildRequires: libuuid-devel
 | 
						|
    Requires: postgresql
 | 
						|
 | 
						|
    %package -n fm-common-dev
 | 
						|
    Summary: STX Platform Fault Management Common Package - Development files
 | 
						|
    Group: devel
 | 
						|
    Requires: fm-common = %{version}-%{release}
 | 
						|
 | 
						|
    %description
 | 
						|
    STX platform Fault Management Client Library that provides APIs for
 | 
						|
    applications to raise/clear/update active alarms.
 | 
						|
 | 
						|
    %description -n fm-common-dev
 | 
						|
    STX platform Fault Management Client Library that provides APIs for
 | 
						|
    applications to raise/clear/update active alarms.  This package contains
 | 
						|
    symbolic links, header files, and related items necessary for software
 | 
						|
    development.
 | 
						|
 | 
						|
    %prep
 | 
						|
    %setup
 | 
						|
    %build
 | 
						|
    VER=%{version}
 | 
						|
    MAJOR=`echo $VER | awk -F . '{print $1}'`
 | 
						|
    MINOR=`echo $VER | awk -F . '{print $2}'`
 | 
						|
    make  MAJOR=$MAJOR MINOR=$MINOR %{?_smp_mflags}
 | 
						|
 | 
						|
    %install
 | 
						|
    rm -rf $RPM_BUILD_ROOT
 | 
						|
    VER=%{version}
 | 
						|
    MAJOR=`echo $VER | awk -F . '{print $1}'`
 | 
						|
    MINOR=`echo $VER | awk -F . '{print $2}'`
 | 
						|
    make DEST_DIR=$RPM_BUILD_ROOT BIN_DIR=%{local_bindir} LIB_DIR=%{_libdir} INC_DIR=%{_includedir} MAJOR=$MAJOR MINOR=$MINOR install_non_bb
 | 
						|
 | 
						|
    %clean
 | 
						|
    rm -rf $RPM_BUILD_ROOT
 | 
						|
 | 
						|
    %files
 | 
						|
    %defattr(-,root,root,-)
 | 
						|
    %doc LICENSE
 | 
						|
    %dir %{local_dir}
 | 
						|
    %dir %{local_bindir}
 | 
						|
    %{local_bindir}/*
 | 
						|
 | 
						|
    %files -n fm-common-dev
 | 
						|
    %defattr(-,root,root,-)
 | 
						|
    %{_includedir}/*
 | 
						|
    %{_libdir}/*
 | 
						|
 | 
						|
************************
 | 
						|
Set up srpm build script
 | 
						|
************************
 | 
						|
 | 
						|
This section describes two options: you can choose to use the default srpm build
 | 
						|
script or create your own srpm build script.
 | 
						|
 | 
						|
#.  Use the default srpm build script, and just provide a few data definitions,
 | 
						|
    in bash format.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        vi centos/build_srpm.data
 | 
						|
        Variables you can define. All paths are relative to PKG_BASE (see below) unless prefixed by another environment variable, eg. STX_BASE
 | 
						|
        SRC_DIR=my_src_dir
 | 
						|
        # Tar everything found in this subdirectory. Define this if source need to be collected into a tarball in $BUILD_DIR/SOURCES.
 | 
						|
        # Tar file name and version are derived from your .spec file and/or PKG-INFO file.  Alternatively you may define TAR_NAME and VERSION in your 'build_srpm.data' file
 | 
						|
 | 
						|
        COPY_LIST="$STX_BASE/downloads/my_tar.tar.gz  my_extra_files_dir  my_script_dir/my_script"
 | 
						|
         # A Space separated list of paths to copy.  In the build directory they will by copied into the $BUILD_DIR/SOURCES directory alongside any tarball you may have created via SRC_DIR.
 | 
						|
 | 
						|
        EXCLUDE_LIST_FROM_TAR=my_omit_dir  my_script_dir/my_omit_script"
 | 
						|
        # Omit these paths under SRC_DIR from the tar
 | 
						|
 | 
						|
        TAR_NAME=nova
 | 
						|
        VERSION=1.1
 | 
						|
        # The name and version of the package.  Assuming SRC_DIR is specified, a tarball with the name "$TAR_NAME-$VERSION.tar.gz" will be created from from the contents of $SRC_DIR.
 | 
						|
        # NOTE: These values are pulled automatically from your .spec ("%global service" or "Name:" and "Version:") or PKG-INFO ("Name:" and "Version:").  You can override the automatic valuation by supplying your own values.
 | 
						|
 | 
						|
        COPY_LIST_TO_TAR=my_extra_files_dir  my_script_dir/my_script"
 | 
						|
         # A Space separated list of paths to copy.  These are added to the tarball itself.  Try to avoid using this... we prefer that tarballs of 3rd party origin remain unaltered.
 | 
						|
 | 
						|
        TIS_BASE_SRCREV=9af04d8480da4359f766e59c45ac23f199b138b0
 | 
						|
        # Last git commit prior to start of TC content
 | 
						|
 | 
						|
        TIS_PATCH_VER=GITREVCOUNT+3
 | 
						|
        # Version is passed into .spec file as rpmmacro to modify package release string.
 | 
						|
        # Can contain 'GITREVCOUNT', the count of commits since TIS_BASE_SRCREV
 | 
						|
 | 
						|
        BUILD_IS_BIG=6
 | 
						|
        # An estimate of the disk requirements in GB.  This package won't build on a tmpfs based build environment small than this value.
 | 
						|
        # If a build fails on a tmpfs based node due to disk space exhaustion, then at minimum this value must be greater than the tpmfs size
 | 
						|
        # as seen in the "MOCKCHAIN_RESOURCE_ALLOCATION=0:10:10:5" log.
 | 
						|
 | 
						|
        BUILD_IS_SLOW5
 | 
						|
        # An estimate of the build time on disk in minutes.
 | 
						|
        # A larger value will tell the build system to schedule this job sooner
 | 
						|
        # rather than later, to maximize parallelism.
 | 
						|
 | 
						|
#.  Create your own srpm build script. The goal of the script is to use the
 | 
						|
    rpmbuild tool to create one or more ``.src.rpm`` files in the ``$BUILD_DIR/SRPMS``
 | 
						|
    directory. The directory itself will be created by rpmbuild.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        vi centos/build_srpm
 | 
						|
 | 
						|
    A sample build script might look like this:
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        source "$SRC_BASE/build-tools/spec-utils"
 | 
						|
 | 
						|
        SRC_DIR=$STX_BASE/git/libvirt
 | 
						|
 | 
						|
        if [ -f PKG-INFO ]: then
 | 
						|
           VERSION=$(grep '^Version:' PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
 | 
						|
           TAR_NAME=$(grep '^Name:' PKG-INFO | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
 | 
						|
        else
 | 
						|
           SPEC_PATH=$(ls $BUILD_DIR/SPECS/*.spec)
 | 
						|
           VERSION=$(spec_evaluate '%{version}' "$SPEC_PATH" 2>> /dev/null)
 | 
						|
           NAME=$(spec_evaluate '%{name} "$SPEC_PATH" 2>> /dev/null)
 | 
						|
        fi
 | 
						|
 | 
						|
        BUILD_DIR="$RPMBUILD_BASE"
 | 
						|
        EXTRA_FILES="libvirt-1.2.12/libvirt.logrotate "
 | 
						|
        EXTRA_FILES+="libvirt-1.2.12/libvirt.lxc "
 | 
						|
        EXTRA_FILES+="libvirt-1.2.12/libvirt.qemu "
 | 
						|
        EXTRA_FILES+="libvirt-1.2.12/libvirt.uml "
 | 
						|
 | 
						|
        mkdir -p $BUILD_DIR/SRPMS
 | 
						|
 | 
						|
        TAR="$TAR_NAME-$VERSION.tar.gz"
 | 
						|
        TAR_PATH="$BUILD_DIR/SOURCES/$TAR"
 | 
						|
 | 
						|
        TAR_NEEDED=0
 | 
						|
        if [ -f $TAR_PATH ]; then
 | 
						|
           n=`find . -cnewer $TAR_PATH -and ! -path './.git*' \
 | 
						|
                                       -and ! -path './build/*' \
 | 
						|
                                       -and ! -path './.pc/*' \
 | 
						|
                                       -and ! -path './patches/*' \
 | 
						|
                                       -and ! -path './$DISTRO/*' \
 | 
						|
                                       -and ! -path './pbr-*.egg/*' \
 | 
						|
                                       | wc -l`
 | 
						|
           if [ $n -gt 0 ]; then
 | 
						|
               TAR_NEEDED=1
 | 
						|
           fi
 | 
						|
        else
 | 
						|
           TAR_NEEDED=1
 | 
						|
        fi
 | 
						|
 | 
						|
        if [ $TAR_NEEDED -gt 0 ]; then
 | 
						|
           tar czvf $TAR_PATH $EXTRA_FILES \
 | 
						|
                              -C $(dirname $SRC_DIR) $(basename $SRC_DIR) \
 | 
						|
                              --exclude '.git/' \
 | 
						|
                              --exclude '.git*' \
 | 
						|
                              --transform "s,^libvirt-1.2.12,extra," \
 | 
						|
                              --transform "s,^$(basename $SRC_DIR),$TAR_NAME-$VERSION,"
 | 
						|
        fi
 | 
						|
 | 
						|
 | 
						|
        for SPEC in `ls $BUILD_DIR/SPECS`; do
 | 
						|
           SPEC_PATH="$BUILD_DIR/SPECS/$SPEC"
 | 
						|
           RELEASE=$(grep '^Release:' $SPEC_PATH | awk -F ': ' '{print $2}' | sed -e 's/^[[:space:]]*//')
 | 
						|
           NAME=`echo $SPEC | sed 's/.spec$//'`
 | 
						|
           SRPM="$NAME-$VERSION-$RELEASE.src.rpm"
 | 
						|
           SRPM_PATH="$BUILD_DIR/SRPMS/$SRPM"
 | 
						|
 | 
						|
           BUILD_NEEDED=0
 | 
						|
           if [ -f $SRPM_PATH ]; then
 | 
						|
               n=`find . -cnewer $SRPM_PATH | wc -l`
 | 
						|
               if [ $n -gt 0 ]; then
 | 
						|
                   BUILD_NEEDED=1
 | 
						|
               fi
 | 
						|
           else
 | 
						|
               BUILD_NEEDED=1
 | 
						|
           fi
 | 
						|
 | 
						|
           if [ $BUILD_NEEDED -gt 0 ]; then
 | 
						|
               rpmbuild -bs $SPEC_PATH --define="%_topdir $BUILD_DIR"
 | 
						|
           fi
 | 
						|
        done
 | 
						|
 | 
						|
The idea is to tar up any source from git into a tarball into
 | 
						|
``$BUILD_DIR/SOURCES`` and/or copy tarballs from a downloads directory. Copy any
 | 
						|
supporting files into ``$BUILD_DIR/SOURCES`` also. Finally, use rpmbuild to
 | 
						|
create a ``.src.rpm``. DO NOT try to compile the binary ``.rpm``.
 | 
						|
 | 
						|
***************************
 | 
						|
Details for build-srpm.data
 | 
						|
***************************
 | 
						|
 | 
						|
This section describes the contents of the ``build-srpm.data`` file, which
 | 
						|
customizes the setup when you build :abbr:`SRPM (Source RPM)` files.
 | 
						|
 | 
						|
**Pre-defined path variables**
 | 
						|
 | 
						|
You may use any of the following:
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    SRC_BASE      # Usually the same as $MY_REPO
 | 
						|
 | 
						|
    STX_BASE      # $MY_REPO/stx
 | 
						|
 | 
						|
    PKG_BASE      # Base dir for the package. Listed in centos_pkg_dir.
 | 
						|
                  # Directory in which CentOS directory is found.
 | 
						|
 | 
						|
    PATCHES_BASE  # Directory where source patches for the package can be found, typically: $PKG_BASE/$DISTRO/patches
 | 
						|
 | 
						|
    FILE_BASE     # Directory where supplemental files for the package can be found, typically: $PKG_BASE/$DISTRO/files
 | 
						|
                  # Files that can be copied into $BUILD_DIR/SOURCES prior to assembling the .src.rpm
 | 
						|
 | 
						|
    CGCS_BASE     # same as $STX_BASE, being phased out in favour of PKG_BASE
 | 
						|
                  # May be removed in a future release, do not use.
 | 
						|
 | 
						|
**User-defined variables**
 | 
						|
 | 
						|
You may define any of the following variables:
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    SRC_DIR=my_src_dir
 | 
						|
    # Tar everything found in this subdirectory. Define this if source need to be collected into a tarball in $BUILD_DIR/SOURCES.
 | 
						|
    # Tar file name and version are derived from your .spec file and/or PKG-INFO file. Alternatively you may define TAR_NAME and VERSION in your 'build_srpm.data' file
 | 
						|
 | 
						|
    COPY_LIST="$STX_BASE/downloads/my_tar.tar.gz my_extra_files_dir my_script_dir/my_script"
 | 
						|
    # A Space separated list of paths to copy. In the build directory they will by copied into the $BUILD_DIR/SOURCES directory alongside any tarball you may have created via SRC_DIR.
 | 
						|
 | 
						|
    EXCLUDE_LIST_FROM_TAR=my_omit_dir my_script_dir/my_omit_script"
 | 
						|
    # Omit these paths under SRC_DIR from the tar
 | 
						|
 | 
						|
    TAR_NAME=nova
 | 
						|
    VERSION=1.1
 | 
						|
    # The name and version of the package. Assuming SRC_DIR is specified, a tarball with the name "$TAR_NAME-$VERSION.tar.gz" will be created from from the contents of $SRC_DIR.
 | 
						|
    # NOTE: These values are pulled automatically from your .spec ("%global service" or "Name:" and "Version:") or PKG-INFO ("Name:" and "Version:"). You can override the automatic valuation by supplying your own values.
 | 
						|
 | 
						|
    COPY_LIST_TO_TAR=my_extra_files_dir my_script_dir/my_script"
 | 
						|
    # A Space separated list of paths to copy. These are added to the tarball itself. Try to avoid using this... we prefer that tarballs of 3rd party origin remain unaltered.
 | 
						|
 | 
						|
    BUILD_IS_BIG=6
 | 
						|
    # An estimate of the disk requirements in GB. This package won't build on a tmpfs based build environment small than this value.
 | 
						|
    # If a build fails on a tmpfs based node due to disk space exhaustion, then at minimum this value must be greater than the tpmfs size
 | 
						|
    # as seen in the "MOCKCHAIN_RESOURCE_ALLOCATION=0:10:10:5" log.
 | 
						|
 | 
						|
    BUILD_IS_SLOW=5
 | 
						|
    # An estimate of the build time on disk in minutes.
 | 
						|
    # A larger value will tell the build system to schedule this job sooner rather than later, to maximize parallelism.
 | 
						|
 | 
						|
    TIS_PATCH_VER=<expression>
 | 
						|
                             # An integer, or one of the supported
 | 
						|
                             # variables listed below, or the sum of
 | 
						|
                             # variables and integers.
 | 
						|
                             # e.g.
 | 
						|
                             # TIS_PATCH_VER=PKG_GITREVCOUNT+SRC_GITREVCOUNT+5
 | 
						|
 | 
						|
           PKG_GITREVCOUNT # Count git revisions relative to PKG_BASE.
 | 
						|
                             # Optionally only count from PKG_BASE_SRCREV
 | 
						|
 | 
						|
           SRC_GITREVCOUNT # Count git revisions relative to SRC_DIR.
 | 
						|
                             # Optionally only count from SRC_BASE_SRCREV
 | 
						|
 | 
						|
           GITREVCOUNT # Deprecated, please use SRC_GITREVCOUNT instead.
 | 
						|
                             # Count git revisions relative to SRC_DIR.
 | 
						|
                             # Optionally only count from TIS_BASE_SRCREV
 | 
						|
 | 
						|
           OTHER_GITREVCOUNT # count git revisions from all sources excluding
 | 
						|
                             # PKG_BASE and SRC_DIR
 | 
						|
                             # Optionally only count from
 | 
						|
                             # BASE_SRCREV_FOR_PATH[<path>]
 | 
						|
 | 
						|
    PKG_BASE_SRCREV=<sha> # Limit PKG_GITREVCOUNT revision count to
 | 
						|
                                # commits since <sha>
 | 
						|
 | 
						|
    SRC_BASE_SRCREV=<sha> # Limit SRC_GITREVCOUNT revision count to
 | 
						|
                                # commits since <sha>
 | 
						|
 | 
						|
    TIS_BASE_SRCREV=<sha> # Deprecated, please use SRC_BASE_SRCREV
 | 
						|
                                # instead
 | 
						|
                                # Limit GITREVCOUNT revision count to commits
 | 
						|
                                # since <sha>
 | 
						|
 | 
						|
    BASE_SRCREV_FOR_PATH[<path>]=[<sha>|OTHER_PKG_BASE_SRCREV]
 | 
						|
                                # Limit OTHER_GITREVCOUNT revision count for
 | 
						|
                                # commits under <path> to commits since <sha>.
 | 
						|
                                # If <path> is the PKG_BASE of another package
 | 
						|
                                # (not the current package) then the keyword
 | 
						|
                                # 'OTHER_PKG_BASE_SRCREV' can be used to extract
 | 
						|
                                # the 'PKG_BASE_SRCREV' value of the other
 | 
						|
                                # package.
 | 
						|
                                #
 | 
						|
                                # The <path> can reference variables like
 | 
						|
                                # $STX_BASE and $GIT_BASE.
 | 
						|
 | 
						|
    OPT_DEP_LIST=<path-list> # Add a space separated list of paths that
 | 
						|
                                  # don't contribute to the content of a src.rpm
 | 
						|
                                  # but do contribute to triggering a rebuild,
 | 
						|
                                  # and possibly modifying the TIS_PATCH_VER via
 | 
						|
                                  # use of OTHER_GITREVCOUNT.
 | 
						|
 | 
						|
    OPT_DEP_LIST_FOR_BUILD_TYPE[<build-type>]=<path-list>
 | 
						|
                                  # For a specific build type only, add a space
 | 
						|
                                  # separated list of paths that don't
 | 
						|
                                  # contribute to the content of src.rpm,
 | 
						|
                                  # but do contribute to triggering a
 | 
						|
                                  # rebuild, and possibly modifying the
 | 
						|
                                  # TIS_PATCH_VER via use of OTHER_GITREVCOUNT.
 | 
						|
 | 
						|
 | 
						|
*************************************
 | 
						|
Add your package to package list file
 | 
						|
*************************************
 | 
						|
 | 
						|
The file name is ``centos_pkg_dirs``. There is a separate package list file for
 | 
						|
each git repo. When adding your package, use the relative path from the root of
 | 
						|
the git to the desired ``PKG_BASE``.
 | 
						|
 | 
						|
Using ``package libvirt`` as an example:
 | 
						|
 | 
						|
*   Spec file:  ``$MY_REPO/stx/integ/virt/libvirt/centos/libvirt.spec``
 | 
						|
 | 
						|
*   ``PKG_BASE``:  ``%BLUE%$MY_REPO/stx%ENDCOLOR%/%RED%integ/virt/libvirt%ENDCOLOR%``
 | 
						|
 | 
						|
*   Root of its git tree:  ``$MY_REPO/stx``
 | 
						|
 | 
						|
*   ``centos_pkg_dirs`` file:  ``%BLUE%$MY_REPO/stx%ENDCOLOR%/centos_pkg_dirs``
 | 
						|
 | 
						|
The new entry needs to be:  ``%RED%integ/virt/libvirt%ENDCOLOR%
 | 
						|
echo "virt/libvirt" >> "$MY_REPO/stx/integ/centos_pkg_dirs"``
 | 
						|
 | 
						|
******************
 | 
						|
Build your package
 | 
						|
******************
 | 
						|
 | 
						|
Complete instructions can be found in the
 | 
						|
:ref:`Create StarlingX packages section of the Build Guide <create_stx_pkgs>`.
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    build-rpms fm-common
 | 
						|
 | 
						|
*******************
 | 
						|
Commit your changes
 | 
						|
*******************
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    git add centos/files/*
 | 
						|
    git add centos/fm-common.spec
 | 
						|
    git add centos/build_srpm*
 | 
						|
 | 
						|
    # If you added PKG-INFO
 | 
						|
    git add PKG-INFO
 | 
						|
 | 
						|
    git add sources/Makefile
 | 
						|
    git commit
 | 
						|
 | 
						|
--------------------------
 | 
						|
Package without a makefile
 | 
						|
--------------------------
 | 
						|
 | 
						|
In this case, the process is generally similar to the above instructions, but
 | 
						|
the ``%build`` section has alternative commands or may be omitted altogether.
 | 
						|
The install phase is done directly in the RPM spec file.
 | 
						|
 | 
						|
For example, the ``collector`` package has no makefile. Its spec file is at
 | 
						|
``stx/utilities/tools/collector/centos/collector.spec``. There is no "build"
 | 
						|
phase, because the files are scripts. However, the files must be unpacked and
 | 
						|
installed into the appropriate locations. The relevant bits look like this:
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    Source0: %{name}-%{version}.tar.gz
 | 
						|
 | 
						|
    %prep
 | 
						|
    %setup
 | 
						|
 | 
						|
    %install
 | 
						|
    mkdir -p %{buildroot}
 | 
						|
 | 
						|
    install -d 755 -d %{buildroot}%{_sysconfdir}/collect.d
 | 
						|
    install -d 755 -d %{buildroot}%{_sysconfdir}/collect
 | 
						|
    install -d 755 -d %{buildroot}/usr/local/sbin
 | 
						|
    install -d 755 -d %{buildroot}/usr/local/bin
 | 
						|
    install -d 755 -d %{buildroot}%{_sbindir}
 | 
						|
 | 
						|
    install -m 755 collect %{buildroot}/usr/local/sbin/collect
 | 
						|
    install -m 755 collect_host %{buildroot}/usr/local/sbin/collect_host
 | 
						|
    install -m 755 collect_date %{buildroot}/usr/local/sbin/collect_date
 | 
						|
    install -m 755 collect_utils %{buildroot}/usr/local/sbin/collect_utils
 | 
						|
    install -m 755 collect_parms %{buildroot}/usr/local/sbin/collect_parms
 | 
						|
    install -m 755 collect_mask_passwords %{buildroot}/usr/local/sbin/collect_mask_passwords
 | 
						|
    install -m 755 expect_done %{buildroot}/usr/local/sbin/expect_done
 | 
						|
 | 
						|
    install -m 755 collect_sysinv.sh %{buildroot}%{_sysconfdir}/collect.d/collect_sysinv
 | 
						|
    install -m 755 collect_psqldb.sh %{buildroot}%{_sysconfdir}/collect.d/collect_psqldb
 | 
						|
    install -m 755 collect_openstack.sh %{buildroot}%{_sysconfdir}/collect.d/collect_openstack
 | 
						|
    install -m 755 collect_networking.sh %{buildroot}%{_sysconfdir}/collect.d/collect_networking
 | 
						|
    install -m 755 collect_ceph.sh %{buildroot}%{_sysconfdir}/collect.d/collect_ceph
 | 
						|
    install -m 755 collect_sm.sh %{buildroot}%{_sysconfdir}/collect.d/collect_sm
 | 
						|
    install -m 755 collect_tc.sh %{buildroot}%{_sysconfdir}/collect.d/collect_tc
 | 
						|
    install -m 755 collect_nfv_vim.sh %{buildroot}%{_sysconfdir}/collect.d/collect_nfv_vim
 | 
						|
    install -m 755 collect_ovs.sh %{buildroot}%{_sysconfdir}/collect.d/collect_ovs
 | 
						|
    install -m 755 collect_patching.sh %{buildroot}%{_sysconfdir}/collect.d/collect_patching
 | 
						|
    install -m 755 collect_coredump.sh %{buildroot}%{_sysconfdir}/collect.d/collect_coredump
 | 
						|
    install -m 755 collect_crash.sh %{buildroot}%{_sysconfdir}/collect.d/collect_crash
 | 
						|
    install -m 755 collect_ima.sh %{buildroot}%{_sysconfdir}/collect.d/collect_ima
 | 
						|
    install -m 755 collect_fm.sh %{buildroot}%{_sysconfdir}/collect.d/collect_fm
 | 
						|
    install -m 755 collect_containerization.sh %{buildroot}%{_sysconfdir}/collect.d/collect_containerization
 | 
						|
    install -m 755 collect_dc.sh %{buildroot}%{_sysconfdir}/collect.d/collect_dc
 | 
						|
 | 
						|
    install -m 755 etc.exclude %{buildroot}%{_sysconfdir}/collect/etc.exclude
 | 
						|
    install -m 755 run.exclude %{buildroot}%{_sysconfdir}/collect/run.exclude
 | 
						|
 | 
						|
    ln -sf /usr/local/sbin/collect %{buildroot}/usr/local/bin/collect
 | 
						|
    ln -sf /usr/local/sbin/collect %{buildroot}%{_sbindir}/collect
 | 
						|
 | 
						|
Either the ``setup`` or ``autosetup`` macro can be used to unpack the Source0
 | 
						|
tarball in ``%{_builddir}``. Next, a number of files are explicitly installed
 | 
						|
into ``%{buildroot}``. Most of these come from the unpacked tarball. One file
 | 
						|
comes from the original source directory, since it wasn't moved over to
 | 
						|
``%{_builddir}`` in the prep phase. All the other sections in the spec file are
 | 
						|
treated the same way as the above package.
 | 
						|
 | 
						|
-----------------------------
 | 
						|
OpenStack and Python packages
 | 
						|
-----------------------------
 | 
						|
 | 
						|
These packages are similar to the above as far as requirements, dependencies,
 | 
						|
etc. The build phase is a bit different since there is no compilation step
 | 
						|
involved.
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    Source0:          %{name}-%{version}.tar.gz
 | 
						|
 | 
						|
    %prep
 | 
						|
    %setup -q
 | 
						|
 | 
						|
    %build
 | 
						|
    %{__python2} setup.py build
 | 
						|
 | 
						|
    %install
 | 
						|
    %{__python2} setup.py install -O1 --skip-build --root %{buildroot}
 | 
						|
 | 
						|
    mkdir -p %{buildroot}%{_sysconfdir}/bash_completion.d
 | 
						|
    install -pm 644 tools/nova.bash_completion \
 | 
						|
        %{buildroot}%{_sysconfdir}/bash_completion.d/nova
 | 
						|
 | 
						|
In this case, the ``setup`` macro unpacks the Source0 tarball in the prep
 | 
						|
phase. Next the ``setup.py`` that is part of the Python package handles
 | 
						|
the actual process of building and installing the Python code. During the
 | 
						|
install phase, a separate file is installed which isn't handled by the Python
 | 
						|
installer.
 | 
						|
 | 
						|
.. _centos_pkg:
 | 
						|
 | 
						|
------------------------------------
 | 
						|
CentOS package with appended patches
 | 
						|
------------------------------------
 | 
						|
 | 
						|
This example uses the ``parted`` package from CentOS and appends a few
 | 
						|
patches of our own.
 | 
						|
 | 
						|
#.  Assuming this package has never been modified by StarlingX previously, find
 | 
						|
    a home for the package. This example uses
 | 
						|
    ``$MY_REPO/stx/integ/filesystem/parted``.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        MY_GIT_ROOT=$MY_REPO/stx/integ
 | 
						|
        MY_PKG_ROOT=$MY_GIT_ROOT/filesystem/parted
 | 
						|
        mkdir -p $MY_PKG_ROOT
 | 
						|
        cd $MY_PKG_ROOT
 | 
						|
 | 
						|
        # You must also add the new package directory to your local centos_pkg_dirs file if not already present
 | 
						|
        echo filesystem/parted >> $MY_GIT_ROOT/centos_pkg_dirs
 | 
						|
 | 
						|
#.  Create a directory skeleton. The ``patches`` directory contains patches
 | 
						|
    applied to the tarball inside the ``src.rpm``. The ``meta_patches``
 | 
						|
    directory contains patches applied to the rpmbuild directory of the
 | 
						|
    ``.src.rpm`` and are used to modify the ``SPECS/*.spec`` file, or
 | 
						|
    the ``SOURCES/*`` files.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        mkdir -p centos/meta_patches
 | 
						|
        mkdir -p centos/patches
 | 
						|
        touch centos/meta_patches/PATCH_ORDER
 | 
						|
        echo TIS_PATCH_VER=1 > centos/build_srpm.data
 | 
						|
 | 
						|
#.  Tell ``build-srpms`` where to find the ``src.rpm`` in the ``srpm_path``
 | 
						|
    file. Two locations are supported for source RPMS: the
 | 
						|
    ``$MY_REPO/cgcs-centos-repo`` directory is for source RPMS provided by
 | 
						|
    CentOS. Tell the build system to search for a source RPM using
 | 
						|
    the ``mirror:`` prefix when you specify the path to the source RPM.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        find -L $MY_REPO/cgcs-centos-repo/  -name 'parted*.src.rpm'
 | 
						|
        /localdisk/<username>/designer/test/cgcs-root/cgcs-centos-repo/Source/parted-3.1-23.el7.src.rpm
 | 
						|
 | 
						|
        echo "mirror:Source/parted-3.1-23.el7.src.rpm" > centos/srpm_path
 | 
						|
 | 
						|
    The other location supported by ``build-srpms`` is the downloads subgit of
 | 
						|
    your layers (found in ``$MY_REPO/stx/downloads`` for CGCS, for example).
 | 
						|
    This should be used for packages that CentOS does not provide. In this case,
 | 
						|
    place the source RPM you downloaded in the directory, and use the ``repo:``
 | 
						|
    prefix when creating your ``srpm_path`` file. The base path for ``repo:``
 | 
						|
    is ``$MY_REPO``, so specify paths from that point.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        cp ~/Downloads/somepackage-1.2-3.src.rpm $MY_REPO/stx/downloads
 | 
						|
        echo "repo:stx/downloads/somepackage-1.2-3.src.rpm" > centos/srpm_path
 | 
						|
 | 
						|
#.  (Optional) Set up a git environment to create your patch. The SRPM will be
 | 
						|
    extracted, then the source tarball will be extracted and patched.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        # clean any prior edit environment that might exist
 | 
						|
        #
 | 
						|
        build-srpms --edit --clean parted
 | 
						|
 | 
						|
        #
 | 
						|
        # Create a working edit environment
 | 
						|
        #
 | 
						|
        build-srpms --edit parted
 | 
						|
        ...
 | 
						|
        ===== 'parted' has been extracted to for editing. =====
 | 
						|
        ===== Metadata can be found at: /localdisk/loadbuild/<username>/test/std/srpm_work/parted/rpmbuild
 | 
						|
        ===== Source code can be found at: /localdisk/loadbuild/<username>/test/std/srpm_work/parted/gits/parted.spec/parted-3.1
 | 
						|
 | 
						|
        #
 | 
						|
        # Create you patch
 | 
						|
        #
 | 
						|
        cd  $MY_WORKSPACE/std/srpm_work/parted/gits/parted.spec/parted-3.1
 | 
						|
 | 
						|
        edit
 | 
						|
 | 
						|
        # create a patch
 | 
						|
        git add
 | 
						|
        git commit -m 'foo bar'
 | 
						|
        git format-patch -n HEAD^
 | 
						|
           0001-foo-bar.patch
 | 
						|
 | 
						|
        # Copy it to the repo... probabbly best to drop the missleading 0001 prefix
 | 
						|
        mv 0001-foo-bar.patch $MY_PKG_ROOT/centos/patches/foo-bar.patch
 | 
						|
 | 
						|
        # undo the commit, if patch failed testing and needs more work
 | 
						|
        git reset --soft HEAD~
 | 
						|
 | 
						|
        #
 | 
						|
        # Now create a meta_patch to add your patch to the spec file
 | 
						|
        #
 | 
						|
        cd $MY_WORKSPACE/std/srpm_work/parted/rpmbuild
 | 
						|
 | 
						|
        # modify Release to include tis patch version, add our patch to the end of the patch list
 | 
						|
        vi SPECS/parted.spec
 | 
						|
           ....
 | 
						|
           Release: 23.el7%{?_tis_dist}.%{tis_patch_ver}
 | 
						|
           ....
 | 
						|
           Patch34: 0034-tests-Use-wait_for_dev_to_-functions.patch
 | 
						|
 | 
						|
           # WRS
 | 
						|
           Patch35: syscalls.patch
 | 
						|
           Patch36: foo-bar.patch
 | 
						|
           ...
 | 
						|
           # if the %prep/%setup section does not use 'git am %{patches}' which automatically applies all patches,
 | 
						|
           # then it should contain a list of %patch macros (one per patch) and you will have to append your patch to the end of the list.
 | 
						|
           # %patch macros are basically just invocations of the 'patch' command to apply your patches.  e.g. like this
 | 
						|
           %patch35 -p1
 | 
						|
           %patch36 -p1
 | 
						|
 | 
						|
 | 
						|
        git add SPECS/parted.spec
 | 
						|
 | 
						|
        # Optionally edit and add files in SOURCES/
 | 
						|
        git commit -m 'meta foo bar'
 | 
						|
        git format-patch -n HEAD^
 | 
						|
           0001-meta-foo-bar.patch
 | 
						|
 | 
						|
        # Copy it to the repo... probabbly best to drop the missleading 0001 prefix
 | 
						|
        mv 0001-meta-foo-bar.patch $MY_PKG_ROOT/centos/meta_patches/meta-foo-bar.patch
 | 
						|
 | 
						|
        # undo the commit, if patch failed testing and needs more work
 | 
						|
        git reset --soft HEAD~
 | 
						|
 | 
						|
        #
 | 
						|
        # Finally add the meta patch to the patch order file... so that the patches vs the spec file apply in the correct order
 | 
						|
        #
 | 
						|
        echo meta-foo-bar.patch >> $MY_PKG_ROOT/centos/meta_patches/PATCH_ORDER
 | 
						|
 | 
						|
#.  (Optional) Create a ``build_srpm.data`` file and add a ``COPY_LIST`` which
 | 
						|
    identifies additional files to copy into SOURCES before finalizing the srpm.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        vi  $MY_PKG_ROOT/centos/build_srpm.data
 | 
						|
        COPY_LIST="files/resizepart.sh"
 | 
						|
 | 
						|
 | 
						|
    .. note::
 | 
						|
 | 
						|
            The order is SRPM extracted first, then copy ``centos/patches/*``
 | 
						|
            into SOURCES, then apply meta patches, then copy anything named in
 | 
						|
            ``COPY_LIST`` into SOURCES. ``COPY_LIST`` items are not available
 | 
						|
            for meta patching. Just edit them directly.
 | 
						|
 | 
						|
#.  Build the final ``src.rpm`` with your new changes inserted.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        build-srpms --clean parted
 | 
						|
        build-srpms parted
 | 
						|
 | 
						|
 | 
						|
#.  Build the rpm.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        build-rpms --clean parted
 | 
						|
        build-rpms parted
 | 
						|
        Verify contents of rpm:
 | 
						|
        build-rpms --no-descendants parted
 | 
						|
        cd $MY_WORKSPACE/rpmbuild
 | 
						|
        mkdir TMP; cd TMP
 | 
						|
        rpm2cpio parted-3.1-23.el7.tis.1.x86_64.rpm | cpio -ivmd
 | 
						|
 | 
						|
#.  Test. If it fails, go back to step 4.
 | 
						|
 | 
						|
#.  Commit.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        cd $MY_PKG_ROOT
 | 
						|
        git add meta_patches/PATCH_ORDER  meta_patches/meta-foo-bar.patch  patches/foo-bar.patch
 | 
						|
        # add these as required
 | 
						|
        git add centos/srpm_path
 | 
						|
        git add centos/build_srpm.data
 | 
						|
        git commit -F
 | 
						|
        ... rebase/push...
 | 
						|
 | 
						|
#.  Discard the working directory and its temporary gits created in step 4.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        build-srpms --clean --edit parted
 | 
						|
 | 
						|
-------------------------------------------------
 | 
						|
Up-version a CentOS package with appended patches
 | 
						|
-------------------------------------------------
 | 
						|
 | 
						|
Suppose you have a patched srpm ``resource-agents-3.9.5-54.el7_2.8.src.rpm`` and
 | 
						|
you want to migrate to a newer srpm, for example,
 | 
						|
``resource-agents-3.9.5-54.el7_2.8.src.rpm``.
 | 
						|
 | 
						|
First question: How do I know if there is a newer version available? If the
 | 
						|
local CentOS mirror has been updated, you can scan it for new srpms that have
 | 
						|
local WRS patches using:
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    find_patched_srpms_needing_upgrade
 | 
						|
       /localdisk/designer/<username>/test/cgcs-root/stx/recipes-cgl/cluster-resource-agents/centos/srpm_path: resource-agents-3.9.5-54.el7_2.8.src.rpm ==> resource-agents-3.9.5-54.el7_2.9.src.rpm
 | 
						|
 | 
						|
This tells you that there are patches for
 | 
						|
``resource-agents-3.9.5-54.el7_2.8.src.rpm``, but a newer version is available
 | 
						|
(``resource-agents-3.9.5-54.el7_2.9.src.rpm``). You should probably migrate the
 | 
						|
patches to the new ``resource-agents-3.9.5-54.el7_2.9.src.rpm``. No output
 | 
						|
means no migrations are required at this time.
 | 
						|
 | 
						|
#.  Check out the old version with our patches.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        build-srpms resource-agents --edit --clean
 | 
						|
        build-srpms resource-agents --edit
 | 
						|
 | 
						|
#.  Point the ``srpm_path`` at the new source rpm.
 | 
						|
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        vi $MY_REPO/cgcs-root/stx/recipes-cgl/cluster-resource-agents/centos/srpm_path
 | 
						|
          mirror:CentOS/vault.centos.org/7.2.1511/updates/Source/SPackages/resource-agents-3.9.5-54.el7_2.9.src.rpm
 | 
						|
 | 
						|
#.  Check out the new version without our patches.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        build-srpms resource-agents --edit --no-meta-patch
 | 
						|
        ...
 | 
						|
        ===== 'resource-agents' has been extracted for editing. =====
 | 
						|
        ===== Metedata can be found at: /localdisk/loadbuild/<username>/test/std/srpm_work/resource-agents/rpmbuild
 | 
						|
        ===== Source code can be found at: /localdisk/loadbuild/<username>/teststd//srpm_work/resource-agents/gits/resource-agents.spec/ClusterLabs-resource-agents-5434e96
 | 
						|
 | 
						|
#.  You now have two directories, each a git tree with two branches, one based
 | 
						|
    on the old srpm, one on the new srpm.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        cd $MY_WORKSPACE/std/srpm_work/resource-agents/rpmbuild
 | 
						|
 | 
						|
        git branch --all
 | 
						|
          master
 | 
						|
          resource-agents-3.9.5-54.el7_2.8
 | 
						|
        * resource-agents-3.9.5-54.el7_2.9
 | 
						|
 | 
						|
        cd $MY_WORKSPACE/std/srpm_work/resource-agents/gits/resource-agents.spec/ClusterLabs-resource-agents-5434e96
 | 
						|
        git branch --all
 | 
						|
          master
 | 
						|
          resource-agents-3.9.5-54.el7_2.8
 | 
						|
        * resource-agents-3.9.5-54.el7_2.9
 | 
						|
 | 
						|
#.  Show the commit histories of the two branches to identify the work to do.
 | 
						|
 | 
						|
    ::
 | 
						|
 | 
						|
        cd $MY_WORKSPACE/std/srpm_work/resource-agents/gits/resource-agents.spec/ClusterLabs-resource-agents-5434e96
 | 
						|
 | 
						|
        git log resource-agents-3.9.5-54.el7_2.8 --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit
 | 
						|
        * be18338 - (HEAD, resource-agents-3.9.5-54.el7_2.8) WRS: Patch99: exportfs_accept_ipv6.patch (2016-04-29 01:11:03 -0400)
 | 
						|
        * 70308b7 - WRS: Patch98: pgsql.patch (2016-04-29 01:11:02 -0400)
 | 
						|
        * 802a93a - WRS: Patch97: lvm_vg_activation.patch (2016-04-29 01:11:00 -0400)
 | 
						|
        * 96acac3 - WRS: Patch96: umount-in-namespace.patch (2016-04-29 01:11:00 -0400)
 | 
						|
        * 24b1bb1 - WRS: Patch95: copyright.patch (2016-04-29 01:10:59 -0400)
 | 
						|
        * cb5c3a9 - WRS: Patch94: ipaddr2_check_if_state.patch (2016-04-29 01:10:58 -0400)
 | 
						|
        * 600cda6 - WRS: Patch93: new_ocf_return_codes.patch (2016-04-29 01:10:57 -0400)
 | 
						|
        * 5f40f0b - WRS: Patch92: filesystem_rmon.patch (2016-04-29 01:10:56 -0400)
 | 
						|
        * facf036 - (tag: pre_wrs_resource-agents-3.9.5-54.el7_2.8) Patch91: bz1316633-backup-and-restore-rabbitmq-users-during-resource-re.patch (2016-04-29 01:10:32 -0400)
 | 
						|
        * 48cbdac - Patch90: bz1311180-rabbitmq-cluster-forget-stopped-cluster-nodes.patch (2016-04-29 01:10:31 -0400)
 | 
						|
 | 
						|
 | 
						|
        git log resource-agents-3.9.5-54.el7_2.9 --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit
 | 
						|
        * 746eb3d - (HEAD, tag: pre_wrs_resource-agents-3.9.5-54.el7_2.9, resource-agents-3.9.5-54.el7_2.9) Patch94: bz1318744-galera-no-grastate.patch (2016-04-29 01:12:16 -0400)
 | 
						|
        * f6fc3f0 - Patch93: bz1318744-galera-heuristic-recovered.patch (2016-04-29 01:12:15 -0400)
 | 
						|
        * dc76b27 - Patch92: bz1318744-galera-crash-recovery.patch (2016-04-29 01:12:14 -0400)
 | 
						|
        * facf036 - (tag: pre_wrs_resource-agents-3.9.5-54.el7_2.8) Patch91: bz1316633-backup-and-restore-rabbitmq-users-during-resource-re.patch (2016-04-29 01:10:32 -0400)
 | 
						|
        * 48cbdac - Patch90: bz1311180-rabbitmq-cluster-forget-stopped-cluster-nodes.patch (2016-04-29 01:10:31 -0400)
 | 
						|
 | 
						|
    The image below shows the color-coded results of the comparison.
 | 
						|
 | 
						|
    *   The patches in green with ``WRS:`` in the patch name must be
 | 
						|
        ported.
 | 
						|
 | 
						|
    *   The commit in orange noted with ``tag:`` was the last common commit.
 | 
						|
 | 
						|
    *   The patches in blue are new patches from upstream that may interfere with
 | 
						|
        our patches.
 | 
						|
 | 
						|
 | 
						|
.. figure:: figures/pkging_patch_comp.png
 | 
						|
   :scale: 55%
 | 
						|
   :alt: Patch comparison
 | 
						|
 | 
						|
   *Figure 1: Patch comparison*
 | 
						|
 | 
						|
 | 
						|
--------------------------------
 | 
						|
Package providing an init script
 | 
						|
--------------------------------
 | 
						|
 | 
						|
The packages/recipes that provide an init script (found under ``/etc/init.d/``
 | 
						|
and corresponding ``/etc/rc.d/``) must be converted to systemd services.
 | 
						|
Within STX there are two kinds of services:
 | 
						|
 | 
						|
*   Services that must be started on boot.
 | 
						|
 | 
						|
*   Services that are monitored by pmond (and automatically restarted in case of
 | 
						|
    failure). In this case, we don't want systemd to restart a process in case
 | 
						|
    of failure.
 | 
						|
 | 
						|
The general idea about converting those services is to still keep the script
 | 
						|
file installed under ``/etc/init.d/`` in the short term and wrap it in a
 | 
						|
systemd service. This is needed so that service dependencies can be evaluated by
 | 
						|
systemd and start them in the right sequence on boot. Re-using the same init
 | 
						|
script at the moment is more convenient for supporting the same code base for
 | 
						|
wrlinux and CentOS packaging (no need to support multiple version of the same
 | 
						|
script).
 | 
						|
 | 
						|
Given an example init script ``/etc/init.d/helloworld``, create a corresponding
 | 
						|
``helloworld.service`` file:
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    [Unit]
 | 
						|
    Description=Helloworld service
 | 
						|
 | 
						|
    # Configures requirement dependencies on other services. If this service gets activated, the
 | 
						|
    # services listed here will be activated as well. If one of the other services gets deactivated or
 | 
						|
    # its activation fails, this service will be deactivated.
 | 
						|
    Requires=
 | 
						|
 | 
						|
    # A weaker version of Requires. Services listed in this option will be started if the configuring
 | 
						|
    # unit is. However, if the listed services fail to start, this has no impact on the current service.
 | 
						|
    Wants=
 | 
						|
 | 
						|
    # The next two directives are important for specifying the services dependancies. This will indicate
 | 
						|
    # to systemd in which order services should be started. This does not imply a dependency
 | 
						|
    # relationship ('Requires' or 'Wants') and must be used in conjunction with one of the above
 | 
						|
    # directives if this is desired.
 | 
						|
 | 
						|
    # The services listed in this directive will be started before starting the current service.
 | 
						|
    After=syslog.target network.target
 | 
						|
 | 
						|
    # The services listed in this directive will not be started until the current service is marked as
 | 
						|
    # started if they are activated at the same time.
 | 
						|
    Before=
 | 
						|
 | 
						|
    [Service]
 | 
						|
    # Usual types are 'oneshot', 'simple' and 'forking'.
 | 
						|
    # oneshot: The process is not a daemon running in the background. Systemd will wait that the process
 | 
						|
    #          exit before continuing.
 | 
						|
    # simple: Systemd will fork the process.
 | 
						|
    # forking: The process will call fork().  It is recommended to use 'PIDfile' in this case so that
 | 
						|
    #          systemd can identify the process.
 | 
						|
    Type=oneshot
 | 
						|
 | 
						|
    # Commands with their arguments that are executed when this service is started
 | 
						|
    ExecStart=/etc/init.d/helloworld start
 | 
						|
 | 
						|
    # Additional commands that are executed before command in ExecStart
 | 
						|
    ExecStartPre=
 | 
						|
 | 
						|
    # Additional commands that are executed after command in ExecStart
 | 
						|
    ExecStartPost=
 | 
						|
 | 
						|
    # Commands to execute to stop the service
 | 
						|
    ExecStop=/etc/init.d/helloworld stop
 | 
						|
 | 
						|
    # Additional commands that are executed after the service is stopped
 | 
						|
    ExecStopPost=
 | 
						|
 | 
						|
    # Commands to execute to trigger a configuration reload in the service
 | 
						|
    ExecReload=
 | 
						|
 | 
						|
    # Only useful when 'Type' is 'forking'.  PID file is important if the process is monitored by pmond.
 | 
						|
    PIDFile=/var/run/helloworld.pid
 | 
						|
 | 
						|
    # Configures whether the service shall be restarted when the service process exits.
 | 
						|
    # Should be 'always' is systemd should monitor the process.
 | 
						|
    # Should be 'no' (default) if process is monitored by pmond.
 | 
						|
    Restart=
 | 
						|
 | 
						|
    [Install]
 | 
						|
    # This enables to service to be started on boot. It is equivalent of init runlevel (see below for equivalence).  This
 | 
						|
    # means that the current service will be started when the listed unit is started.  All the other services that are
 | 
						|
    # also dependent on this target will also get started, the order will depends on the 'After' and 'Before' options
 | 
						|
    # specified above.
 | 
						|
    WantedBy=multi-user.target
 | 
						|
 | 
						|
Once the systemd service file is defined, it needs to be packaged in the rpm.
 | 
						|
The ``helloworld.service`` file must be installed in the ``/etc/systemd/system/``
 | 
						|
directory (recommended for STX services) by the rpm spec. It's important to set
 | 
						|
file permissions to 664. The systemd services must be enabled on install
 | 
						|
and disabled on uninstalled. This is achieved by using the systemd command
 | 
						|
systemctl enable (and disable). However, it is recommended to use the macros
 | 
						|
already defined by rpm. For example:
 | 
						|
 | 
						|
::
 | 
						|
 | 
						|
    %install
 | 
						|
    install -d -m 755 %{buildroot}/etc/systemd/system/
 | 
						|
    install -p -D -m 664 helloworld.service %{buildroot}/etc/systemd/system/helloworld.service
 | 
						|
 | 
						|
    # Enable the service and start it.
 | 
						|
    %post
 | 
						|
    %systemd_post helloworld.service
 | 
						|
 | 
						|
    # Disable the service and stop it.
 | 
						|
    %preun
 | 
						|
    %systemd_preun helloworld.service
 | 
						|
 | 
						|
    # Try to restart the service.  Usefull on upgrades.
 | 
						|
    %postun
 | 
						|
    %systemd_postun_with_restart helloworld.service
 | 
						|
 | 
						|
Systemd searches these directories for services (in order of precedence, the
 | 
						|
first item has higher precedence than the one following):
 | 
						|
 | 
						|
*   ``/etc/systemd/system/`` Recommended and reserved location for system
 | 
						|
    administrator customized services
 | 
						|
*   ``/run/systemd/system/`` Mainly used by CentOS. Don't pollute this
 | 
						|
    directory.
 | 
						|
*   ``/usr/lib/systemd/system/``
 | 
						|
 | 
						|
CentOS is distributed with a number of predefined targets that are more or less
 | 
						|
similar to the standard set of runlevels. Use the following comparison table for
 | 
						|
specifying your WantedBy target.
 | 
						|
 | 
						|
*   runlevel0.target -> poweroff.target
 | 
						|
*   runlevel1.target -> rescue.target
 | 
						|
*   runlevel2.target -> multi-user.target
 | 
						|
*   runlevel3.target -> multi-user.target
 | 
						|
*   runlevel4.target -> multi-user.target
 | 
						|
*   runlevel5.target -> graphical.target
 | 
						|
*   runlevel6.target -> reboot.target
 | 
						|
 | 
						|
Convert to systemd service (without intermediate init script):
 | 
						|
 | 
						|
*   The exact PID file name that is used by pmond can be found by looking on an
 | 
						|
    installed system under ``/etc/pmon.d/``. Look for ``pidfile=`` in the
 | 
						|
    corresponding pmond configuration file for the service.
 | 
						|
 | 
						|
*   Some of the init scripts are using start-stop-daemon to manage processes.
 | 
						|
    The start-stop-package has been ported to CentOS (it is not supported by
 | 
						|
    CentOS). Make sure the init script is providing the right ``--pidfile``
 | 
						|
    argument that matches the pmond service configuration. A process that is
 | 
						|
    started by start-stop-daemon with the ``--daemon`` or ``-d`` options
 | 
						|
    indicates that the processes will be forked, in which case the systemd
 | 
						|
    service Type should be forking. Else if the script or process doesn't fork,
 | 
						|
    the Type should be simple which will tell systemd to fork it.
 | 
						|
 | 
						|
*   Hints for defining service startup order (Before and After):
 | 
						|
 | 
						|
    *   Systemd determines when a service should start using the ``After=``
 | 
						|
        and ``Before=`` values found in the ``[Unit]`` section of the ``service``
 | 
						|
        file. These values take a space-separated list of other services or
 | 
						|
        *targets*, where a target is effectively a collection of services. For
 | 
						|
        example, the log management service has the following values:
 | 
						|
 | 
						|
        ::
 | 
						|
 | 
						|
            [Unit]
 | 
						|
            Description=StarlingX Log Management
 | 
						|
            After=network.target syslog-ng.service iscsid.service sw-patch.service
 | 
						|
            Before=config.service pmon.service
 | 
						|
 | 
						|
        In this example, the log management service can't start until software
 | 
						|
        is patched, and a network, a disk and the syslog-ng service are all up
 | 
						|
        and running.
 | 
						|
 | 
						|
    *   The init script might contain some specific information about the order.
 | 
						|
        Look for Required-Start and Required-Stop in it.
 | 
						|
 | 
						|
Troubleshooting (at runtime):
 | 
						|
 | 
						|
*   Enable a service: ``systemctl enable``
 | 
						|
*   Disable a service: ``systemctl disable``
 | 
						|
*   Start a service: ``systemctl start``
 | 
						|
*   Stop a service: ``systemctl stop``
 | 
						|
*   Get service status: ``systemctl status``
 | 
						|
*   Get list of running units ordered by time to init: ``systemd-analyze blame``
 | 
						|
*   Show systemd logging: ``journalctl /usr/lib/systemd/systemd``
 | 
						|
*   Show logging of a specific service: ``journalctl _SYSTEMD_UNIT=``
 | 
						|
 | 
						|
--------------------------------
 | 
						|
After you've converted a package
 | 
						|
--------------------------------
 | 
						|
 | 
						|
Each repo has a ``centos_iso_image.inc`` file, which contains the list of
 | 
						|
packages to be included in the installation iso relevant to the repo. The
 | 
						|
``build-iso`` step merges the individual ``centos_iso_image.inc`` files to
 | 
						|
generate the complete list.
 | 
						|
 |