Developer Guide edits.
Converted the guide to use the "r/2018.10" branch rather than the default "master" branch. Cleaned up a lot writing with general edits. Applied consistent formatting across various items such as filenames, pathnames, etc. I went through the entire file and did some clean-up work. Change-Id: If53ab66187307c087f5ae755f44d6080d1aa8245 Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
This commit is contained in:
parent
2399131b01
commit
8c1361ebd3
@ -4,8 +4,8 @@
|
|||||||
Developer Guide
|
Developer Guide
|
||||||
===============
|
===============
|
||||||
|
|
||||||
This section contains the steps for building a StarlingX ISO from Master
|
This section contains the steps for building a StarlingX ISO from
|
||||||
branch.
|
the "r/2018.10" branch.
|
||||||
|
|
||||||
------------
|
------------
|
||||||
Requirements
|
Requirements
|
||||||
@ -46,7 +46,7 @@ Development Environment Setup
|
|||||||
-----------------------------
|
-----------------------------
|
||||||
|
|
||||||
This section describes how to set up a StarlingX development system on a
|
This section describes how to set up a StarlingX development system on a
|
||||||
workstation computer. After completing these steps, you will be able to
|
workstation computer. After completing these steps, you can
|
||||||
build a StarlingX ISO image on the following Linux distribution:
|
build a StarlingX ISO image on the following Linux distribution:
|
||||||
|
|
||||||
- Ubuntu 16.04 LTS 64-bit
|
- Ubuntu 16.04 LTS 64-bit
|
||||||
@ -55,8 +55,8 @@ build a StarlingX ISO image on the following Linux distribution:
|
|||||||
Update Your Operating System
|
Update Your Operating System
|
||||||
****************************
|
****************************
|
||||||
|
|
||||||
Before proceeding with the build, ensure your OS is up to date. You’ll
|
Before proceeding with the build, ensure your Ubuntu distribution is up to date.
|
||||||
first need to update the local database list of available packages:
|
You first need to update the local database list of available packages:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -70,13 +70,14 @@ Installation Requirements and Dependencies
|
|||||||
Git
|
Git
|
||||||
^^^
|
^^^
|
||||||
|
|
||||||
1. Install the required packages in an Ubuntu host system with:
|
1. Install the required packages on the Ubuntu host system:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ sudo apt-get install make git curl
|
$ sudo apt-get install make git curl
|
||||||
|
|
||||||
2. Make sure to setup your identity
|
2. Make sure to setup your identity with the following two commands.
|
||||||
|
Be sure to provide your actual name and email address:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -87,7 +88,7 @@ Git
|
|||||||
Docker CE
|
Docker CE
|
||||||
^^^^^^^^^
|
^^^^^^^^^
|
||||||
|
|
||||||
3. Install the required Docker CE packages in an Ubuntu host system. See
|
3. Install the required Docker CE packages in the Ubuntu host system. See
|
||||||
`Get Docker CE for
|
`Get Docker CE for
|
||||||
Ubuntu <https://docs.docker.com/install/linux/docker-ce/ubuntu/#os-requirements>`__
|
Ubuntu <https://docs.docker.com/install/linux/docker-ce/ubuntu/#os-requirements>`__
|
||||||
for more information.
|
for more information.
|
||||||
@ -96,18 +97,18 @@ Docker CE
|
|||||||
Android Repo Tool
|
Android Repo Tool
|
||||||
^^^^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
4. Install the required Android Repo Tool in an Ubuntu host system. Follow
|
4. Install the required Android Repo Tool in the Ubuntu host system. Follow
|
||||||
the 2 steps in "Installing Repo" section from `Installing
|
the steps in the `Installing
|
||||||
Repo <https://source.android.com/setup/build/downloading#installing-repo>`__
|
Repo <https://source.android.com/setup/build/downloading#installing-repo>`__
|
||||||
to have Andriod Repo Tool installed.
|
section.
|
||||||
|
|
||||||
**********************
|
**********************
|
||||||
Install Public SSH Key
|
Install Public SSH Key
|
||||||
**********************
|
**********************
|
||||||
|
|
||||||
#. Follow these instructions on GitHub to `Generate a Public SSH
|
#. Follow these instructions on GitHub to `Generate a Public SSH
|
||||||
Key <https://help.github.com/articles/connecting-to-github-with-ssh>`__
|
Key <https://help.github.com/articles/connecting-to-github-with-ssh>`__.
|
||||||
and then upload your public key to your GitHub and Gerrit account
|
Then upload your public key to your GitHub and Gerrit account
|
||||||
profiles:
|
profiles:
|
||||||
|
|
||||||
- `Upload to
|
- `Upload to
|
||||||
@ -119,20 +120,21 @@ Install Public SSH Key
|
|||||||
Install stx-tools Project
|
Install stx-tools Project
|
||||||
*************************
|
*************************
|
||||||
|
|
||||||
#. Under your $HOME directory, clone the <stx-tools> project
|
#. While in your $HOME directory, create a local copy of the stx-tools project
|
||||||
|
based on the "r/2018.10" branch:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ cd $HOME
|
$ cd $HOME
|
||||||
$ git clone https://git.starlingx.io/stx-tools
|
$ git clone -b r/2018.10 https://git.starlingx.io/stx-tools
|
||||||
|
|
||||||
****************************
|
****************************
|
||||||
Create a Workspace Directory
|
Create a Workspace Directory
|
||||||
****************************
|
****************************
|
||||||
|
|
||||||
#. Create a *starlingx* workspace directory on your workstation
|
#. Create a *starlingx* workspace directory on your system.
|
||||||
computer. Usually, you’ll want to create it somewhere under your
|
Best practices dictate creating the workspace directory
|
||||||
user’s home directory.
|
in your $HOME directory:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -148,16 +150,16 @@ This section describes how to build the CentOS Mirror Repository.
|
|||||||
Setup Repository Docker Container
|
Setup Repository Docker Container
|
||||||
*********************************
|
*********************************
|
||||||
|
|
||||||
Run the following commands under a terminal identified as "One".
|
| Run the following commands under a terminal identified as "**One**".
|
||||||
|
|
||||||
#. Navigate to the *<$HOME/stx-tools>/centos-mirror-tool* project
|
#. Navigate to the *$HOME/stx-tools/centos-mirror-tool* project
|
||||||
directory:
|
directory:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ cd $HOME/stx-tools/centos-mirror-tools/
|
$ cd $HOME/stx-tools/centos-mirror-tools/
|
||||||
|
|
||||||
#. If necessary you might have to set http/https proxy in your
|
#. If necessary, set the http/https proxy in your
|
||||||
Dockerfile before building the docker image.
|
Dockerfile before building the docker image.
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
@ -167,34 +169,35 @@ Run the following commands under a terminal identified as "One".
|
|||||||
ENV ftp_proxy " http://your.actual_ftp_proxy.com:your_port "
|
ENV ftp_proxy " http://your.actual_ftp_proxy.com:your_port "
|
||||||
RUN echo " proxy=http://your-proxy.com:port " >> /etc/yum.conf
|
RUN echo " proxy=http://your-proxy.com:port " >> /etc/yum.conf
|
||||||
|
|
||||||
#. Build your *<user>:<tag>* base container image with **e.g.**
|
#. Build your *<user>:<tag>* base container image. For example, use
|
||||||
*user:centos-mirror-repository*
|
*user:centos-mirror-repository*:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ docker build --tag $USER:centos-mirror-repository --file Dockerfile .
|
$ docker build --tag $USER:centos-mirror-repository --file Dockerfile .
|
||||||
|
|
||||||
#. Launch a *<user>* docker container using previously created Docker
|
#. Launch a *<user>* docker container using the previously created Docker
|
||||||
base container image *<user>:<tag>* **e.g.**
|
base container image *<user>:<tag>* (e.g.
|
||||||
*-centos-mirror-repository*. As /localdisk is defined as the workdir
|
*-centos-mirror-repository*). As /localdisk is defined as the workdir
|
||||||
of the container, the same folder name should be used to define the
|
of the container, the same folder name should be used to define the
|
||||||
volume. The container will start to run and populate a logs and
|
volume. The container will start to run and populate the output folders
|
||||||
output folders in this directory. The container shall be run from the
|
*logs* and *output* in this directory. The container is run from the
|
||||||
same directory where the other scripts are stored.
|
same directory where the other scripts are stored:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ docker run -itd --name $USER-centos-mirror-repository --volume $(pwd):/localdisk $USER:centos-mirror-repository
|
$ docker run -itd --name $USER-centos-mirror-repository --volume $(pwd):/localdisk $USER:centos-mirror-repository
|
||||||
|
|
||||||
**Note**: the above command will create the container in background,
|
**NOTE**: The above command creates the container in the background.
|
||||||
this mean that you need to attach it manually. The advantage of this
|
With the container running in the background, you must attach to it
|
||||||
is that you can enter/exit from the container many times as you want.
|
manually. An advantage of attaching to the container manually is that
|
||||||
|
you can enter and exit from the container as many times as you want.
|
||||||
|
|
||||||
*****************
|
*****************
|
||||||
Download Packages
|
Download Packages
|
||||||
*****************
|
*****************
|
||||||
|
|
||||||
#. Attach to the docker repository previously created
|
#. Attach to the docker repository previously created:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
@ -208,8 +211,8 @@ Download Packages
|
|||||||
|
|
||||||
# bash download_mirror.sh
|
# bash download_mirror.sh
|
||||||
|
|
||||||
#. Monitor the download of packages until it is complete. When download
|
#. Monitor the download of packages until it is complete. When the download
|
||||||
is complete, the following message is displayed:
|
is complete, the following message appears:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
@ -225,24 +228,25 @@ Download Packages
|
|||||||
Verify Packages
|
Verify Packages
|
||||||
***************
|
***************
|
||||||
|
|
||||||
#. Verify there are no missing or failed packages:
|
#. Verify no missing or failed packages exist:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
# cat logs/*_missing_*.log
|
# cat logs/*_missing_*.log
|
||||||
# cat logs/*_failmove_*.log
|
# cat logs/*_failmove_*.log
|
||||||
|
|
||||||
#. In case there are missing or failed ones due to network instability
|
#. In case missing or failed packages do exist, which is usually caused by
|
||||||
(or timeout), you should download them manually, to assure you get
|
network instability (or timeout), you need to download the packages
|
||||||
all RPMs listed in
|
manually.
|
||||||
**rpms_3rdparties.lst**/**rpms_centos.lst**/**rpms_centos3rdparties.lst**.
|
Doing so assures you get all RPMs listed in
|
||||||
|
*rpms_3rdparties.lst*/*rpms_centos.lst*/*rpms_centos3rdparties.lst*.
|
||||||
|
|
||||||
******************
|
******************
|
||||||
Packages Structure
|
Packages Structure
|
||||||
******************
|
******************
|
||||||
|
|
||||||
The following is a general overview of the packages structure that you
|
The following is a general overview of the packages structure resulting
|
||||||
will have after having downloaded the packages
|
from downloading the packages:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
@ -278,7 +282,7 @@ as "**Two**", run the following commands:
|
|||||||
|
|
||||||
#. Copy the built CentOS Mirror Repository built under
|
#. Copy the built CentOS Mirror Repository built under
|
||||||
*$HOME/stx-tools/centos-mirror-tool* to the *$HOME/starlingx/mirror/*
|
*$HOME/stx-tools/centos-mirror-tool* to the *$HOME/starlingx/mirror/*
|
||||||
workspace directory.
|
workspace directory:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -293,25 +297,25 @@ Create StarlingX Packages
|
|||||||
Setup Building Docker Container
|
Setup Building Docker Container
|
||||||
*******************************
|
*******************************
|
||||||
|
|
||||||
#. From terminal identified as "**Two**", create the workspace folder
|
#. From the terminal identified as "**Two**", create the workspace folder:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ mkdir -p $HOME/starlingx/workspace
|
$ mkdir -p $HOME/starlingx/workspace
|
||||||
|
|
||||||
#. Navigate to the '' $HOME/stx-tools'' project directory:
|
#. Navigate to the *$HOME/stx-tools* project directory:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ cd $HOME/stx-tools
|
$ cd $HOME/stx-tools
|
||||||
|
|
||||||
#. Copy your git options to "toCopy" folder
|
#. Copy your Git options to *toCopy* folder:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ cp ~/.gitconfig toCOPY
|
$ cp ~/.gitconfig toCOPY
|
||||||
|
|
||||||
#. Create a *<localrc>* file
|
#. Create a *localrc* file:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -323,8 +327,8 @@ Setup Building Docker Container
|
|||||||
HOST_MIRROR_DIR=$HOME/starlingx/mirror
|
HOST_MIRROR_DIR=$HOME/starlingx/mirror
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
#. If necessary you might have to set http/https proxy in your
|
#. If necessary, you might have to set the http/https proxy in your
|
||||||
Dockerfile.centos73 before building the docker image.
|
*Dockerfile.centos73* file before building the docker image:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -335,9 +339,16 @@ Setup Building Docker Container
|
|||||||
echo -e "export http_proxy=$http_proxy\nexport https_proxy=$https_proxy\n\
|
echo -e "export http_proxy=$http_proxy\nexport https_proxy=$https_proxy\n\
|
||||||
export ftp_proxy=$ftp_proxy" >> /root/.bashrc
|
export ftp_proxy=$ftp_proxy" >> /root/.bashrc
|
||||||
|
|
||||||
#. Base container setup If you are running in fedora system, you will
|
#. Set up the base container:
|
||||||
see " .makeenv:88: \**\* missing separator. Stop. " error, to
|
|
||||||
continue :
|
If you are using a Fedora distribution, you will
|
||||||
|
see the following error:
|
||||||
|
|
||||||
|
.. code:: sh
|
||||||
|
|
||||||
|
.makeenv:88: \**\* missing separator. Stop.
|
||||||
|
|
||||||
|
To continue, do the following:
|
||||||
|
|
||||||
- delete the functions define in the .makeenv ( module () { ... } )
|
- delete the functions define in the .makeenv ( module () { ... } )
|
||||||
- delete the line 19 in the Makefile and ( NULL := $(shell bash -c
|
- delete the line 19 in the Makefile and ( NULL := $(shell bash -c
|
||||||
@ -347,19 +358,19 @@ Setup Building Docker Container
|
|||||||
|
|
||||||
$ make base-build
|
$ make base-build
|
||||||
|
|
||||||
#. Build container setup
|
#. Set up the build container:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ make build
|
$ make build
|
||||||
|
|
||||||
#. Verify environment variables
|
#. Verify environment variables:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ bash tb.sh env
|
$ bash tb.sh env
|
||||||
|
|
||||||
#. Build container run
|
#. Run the build container:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -375,35 +386,44 @@ Setup Building Docker Container
|
|||||||
Download Source Code Repositories
|
Download Source Code Repositories
|
||||||
*********************************
|
*********************************
|
||||||
|
|
||||||
#. From terminal identified as "Two", now inside the Building Docker
|
#. From the terminal identified as "**Two**", which is now inside the
|
||||||
container, Internal environment
|
Building Docker container, start the internal environment:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ eval $(ssh-agent)
|
$ eval $(ssh-agent)
|
||||||
$ ssh-add
|
$ ssh-add
|
||||||
|
|
||||||
#. Repo init
|
#. Use the repo tool to create a local clone of the stx-manifest
|
||||||
|
Git repository based on the "r/2018.10" branch:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ cd $MY_REPO_ROOT_DIR
|
$ cd $MY_REPO_ROOT_DIR
|
||||||
|
$ repo init -u https://git.starlingx.io/stx-manifest -m default.xml -b r/2018.10
|
||||||
|
|
||||||
|
**NOTE:** To use the "repo" command to clone the stx-manifest repository and
|
||||||
|
check out the "master" branch, omit the "-b r/2018.10" option.
|
||||||
|
Following is an example:
|
||||||
|
|
||||||
|
.. code:: sh
|
||||||
|
|
||||||
$ repo init -u https://git.starlingx.io/stx-manifest -m default.xml
|
$ repo init -u https://git.starlingx.io/stx-manifest -m default.xml
|
||||||
|
|
||||||
#. Repo sync
|
#. Synchronize the repository:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ repo sync -j`nproc`
|
$ repo sync -j`nproc`
|
||||||
|
|
||||||
#. Tarballs Repository
|
#. Create a tarballs repository:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
|
$ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
|
||||||
|
|
||||||
Alternatively you can run the populate_downloads.sh script to copy
|
Alternatively, you can run the "populate_downloads.sh" script to copy
|
||||||
the tarballs instead of using a symlink.
|
the tarballs instead of using a symlink:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -411,7 +431,7 @@ Download Source Code Repositories
|
|||||||
|
|
||||||
Outside the container
|
Outside the container
|
||||||
|
|
||||||
#. From another terminal identified as "Three", Mirror Binaries
|
#. From another terminal identified as "**Three**", create mirror binaries:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -424,30 +444,32 @@ Download Source Code Repositories
|
|||||||
Build Packages
|
Build Packages
|
||||||
**************
|
**************
|
||||||
|
|
||||||
#. Back to the Building Docker container, terminal identified as
|
#. Go back to the terminal identified as "**Two**", which is the Building Docker container.
|
||||||
"**Two**"
|
|
||||||
#. **Temporal!** Build-Pkgs Errors Be prepared to have some missing /
|
#. **Temporal!** Build-Pkgs Errors. Be prepared to have some missing /
|
||||||
corrupted rpm and tarball packages generated during
|
corrupted rpm and tarball packages generated during
|
||||||
`Build the CentOS Mirror Repository`_ which will make the next step
|
`Build the CentOS Mirror Repository`_, which will cause the next step
|
||||||
to fail, if that happens please download manually those missing /
|
to fail. If that step does fail, manually download those missing /
|
||||||
corrupted packages.
|
corrupted packages.
|
||||||
#. **Update the symbolic links**
|
|
||||||
|
#. Update the symbolic links:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
|
$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
|
||||||
|
|
||||||
#. Build-Pkgs
|
#. Build the packages:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ build-pkgs
|
$ build-pkgs
|
||||||
|
|
||||||
#. **Optional!** Generate-Cgcs-Tis-Repo
|
#. **Optional!** Generate-Cgcs-Tis-Repo:
|
||||||
This step is optional but will improve performance on subsequent
|
|
||||||
|
While this step is optional, it improves performance on subsequent
|
||||||
builds. The cgcs-tis-repo has the dependency information that
|
builds. The cgcs-tis-repo has the dependency information that
|
||||||
sequences the build order; To generate or update the information the
|
sequences the build order. To generate or update the information, you
|
||||||
following command needs to be executed after building modified or new
|
need to execute the following command after building modified or new
|
||||||
packages.
|
packages.
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
@ -458,7 +480,7 @@ Build Packages
|
|||||||
Build StarlingX ISO
|
Build StarlingX ISO
|
||||||
-------------------
|
-------------------
|
||||||
|
|
||||||
#. Build-Iso
|
#. Build the image:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -468,26 +490,27 @@ Build StarlingX ISO
|
|||||||
Build Installer
|
Build Installer
|
||||||
---------------
|
---------------
|
||||||
|
|
||||||
To get your StarlingX ISO ready to use, you will need to create the init
|
To get your StarlingX ISO ready to use, you must create the initialization
|
||||||
files that will be used to boot the ISO as well to boot additional
|
files used to boot the ISO, additional controllers, and compute nodes.
|
||||||
controllers and compute nodes. Note that this procedure only is needed
|
|
||||||
in your first build and every time the kernel is upgraded.
|
|
||||||
|
|
||||||
Once you had run build-iso, run:
|
**NOTE:** You only need this procedure during your first build and
|
||||||
|
every time you upgrade the kernel.
|
||||||
|
|
||||||
|
After running "build-iso", run:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ build-pkgs --installer
|
$ build-pkgs --installer
|
||||||
|
|
||||||
This will build *rpm* and *anaconda* packages. Then run:
|
This builds *rpm* and *anaconda* packages. Then run:
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
$ update-pxe-network-installer
|
$ update-pxe-network-installer
|
||||||
|
|
||||||
The *update-pxe-network-installer* covers the steps detailed in
|
The *update-pxe-network-installer* covers the steps detailed in
|
||||||
*$MY_REPO/stx/stx-metal/installer/initrd/README*. This script will
|
*$MY_REPO/stx/stx-metal/installer/initrd/README*. This script
|
||||||
create three files on
|
creates three files on
|
||||||
*/localdisk/loadbuild///pxe-network-installer/output*.
|
*/localdisk/loadbuild///pxe-network-installer/output*.
|
||||||
|
|
||||||
::
|
::
|
||||||
@ -496,7 +519,7 @@ create three files on
|
|||||||
new-squashfs.img
|
new-squashfs.img
|
||||||
new-vmlinuz
|
new-vmlinuz
|
||||||
|
|
||||||
Then, rename them to:
|
Rename the files as follows:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
@ -504,16 +527,15 @@ Then, rename them to:
|
|||||||
squashfs.img-stx-0.2
|
squashfs.img-stx-0.2
|
||||||
vmlinuz-stx-0.2
|
vmlinuz-stx-0.2
|
||||||
|
|
||||||
There are two ways to use these files:
|
Two ways exist for using these files:
|
||||||
|
|
||||||
#. Store the files in the */import/mirror/CentOS/stx-installer/* folder
|
#. Store the files in the */import/mirror/CentOS/stx-installer/* folder
|
||||||
for future use.
|
for future use.
|
||||||
#. Store it in an arbitrary location and modify the
|
#. Store the files in an arbitrary location and modify the
|
||||||
*$MY_REPO/stx/stx-metal/installer/pxe-network-installer/centos/build_srpm.data*
|
*$MY_REPO/stx/stx-metal/installer/pxe-network-installer/centos/build_srpm.data*
|
||||||
file to point to these files.
|
file to point to these files.
|
||||||
|
|
||||||
Now, the *pxe-network-installer* package needs to be recreated and the
|
Recreate the *pxe-network-installer* package and rebuild the image:
|
||||||
ISO regenerated.
|
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -521,18 +543,18 @@ ISO regenerated.
|
|||||||
$ build-pkgs pxe-network-installer
|
$ build-pkgs pxe-network-installer
|
||||||
$ build-iso
|
$ build-iso
|
||||||
|
|
||||||
Now your ISO should be able to boot.
|
Your ISO image should be able to boot.
|
||||||
|
|
||||||
****************
|
****************
|
||||||
Additional Notes
|
Additional Notes
|
||||||
****************
|
****************
|
||||||
|
|
||||||
- In order to get the first boot working this complete procedure needs
|
- In order to get the first boot working, this complete procedure needs
|
||||||
to be done. However, once the init files are created, these can be
|
to be done. However, once the init files are created, these can be
|
||||||
stored in a shared location where different developers can make use
|
stored in a shared location where different developers can make use
|
||||||
of them. Updating these files is not a frequent task and should be
|
of them. Updating these files is not a frequent task and should be
|
||||||
done whenever the kernel is upgraded.
|
done whenever the kernel is upgraded.
|
||||||
- StarlingX is in active development, so it is possible that in the
|
- StarlingX is in active development. Consequently, it is possible that in the
|
||||||
future the **0.2** version will change to a more generic solution.
|
future the **0.2** version will change to a more generic solution.
|
||||||
|
|
||||||
---------------
|
---------------
|
||||||
@ -543,23 +565,25 @@ Build Avoidance
|
|||||||
Purpose
|
Purpose
|
||||||
*******
|
*******
|
||||||
|
|
||||||
Greatly reduce build times after a repo sync for designers working
|
Greatly reduce build times after using "repo" to syncronized a local
|
||||||
within a regional office. Starting from a new workspace, build-pkgs
|
repository with an upstream source (i.e. "repo sync").
|
||||||
typically requires 3+ hours. Build avoidance typically reduces this step
|
Build avoidance works well for designers working
|
||||||
to ~20min
|
within a regional office. Starting from a new workspace, "build-pkgs"
|
||||||
|
typically requires three or more hours to complete. Build avoidance
|
||||||
|
reduces this step to approximately 20 minutes.
|
||||||
|
|
||||||
***********
|
***********
|
||||||
Limitations
|
Limitations
|
||||||
***********
|
***********
|
||||||
|
|
||||||
- Little or no benefit for designers who refresh a pre-existing
|
- Little or no benefit for designers who refresh a pre-existing
|
||||||
workspace at least daily. (download_mirror.sh, repo sync,
|
workspace at least daily (e.g. download_mirror.sh, repo sync,
|
||||||
generate-cgcs-centos-repo.sh, build-pkgs, build-iso). In these cases
|
generate-cgcs-centos-repo.sh, build-pkgs, build-iso). In these cases,
|
||||||
an incremental build (reuse of same workspace without a 'build-pkgs
|
an incremental build (i.e. reuse of same workspace without a "build-pkgs
|
||||||
--clean') is often just as efficient.
|
--clean") is often just as efficient.
|
||||||
- Not likely to be useful to solo designers, or teleworkers that wish
|
- Not likely to be useful to solo designers, or teleworkers that wish
|
||||||
to compile on there home computers. Build avoidance downloads build
|
to compile on using their home computers. Build avoidance downloads build
|
||||||
artifacts from a reference build, and WAN speeds are generally to
|
artifacts from a reference build, and WAN speeds are generally too
|
||||||
slow.
|
slow.
|
||||||
|
|
||||||
*****************
|
*****************
|
||||||
@ -568,26 +592,28 @@ Method (in brief)
|
|||||||
|
|
||||||
#. Reference Builds
|
#. Reference Builds
|
||||||
|
|
||||||
- A server in the regional office performs a regular (daily?),
|
- A server in the regional office performs regular (e.g. daily)
|
||||||
automated builds using existing methods. Call these the reference
|
automated builds using existing methods. These builds are called
|
||||||
builds.
|
"reference builds".
|
||||||
- The builds are timestamped, and preserved for some time. (a few
|
- The builds are timestamped and preserved for some time (i.e. a
|
||||||
weeks)
|
number of weeks).
|
||||||
- A build CONTEXT is captured. This is a file produced by build-pkgs
|
- A build CONTEXT, which is a file produced by "build-pkgs"
|
||||||
at location '$MY_WORKSPACE/CONTEXT'. It is a bash script that can
|
at location *$MY_WORKSPACE/CONTEXT*, is captured. It is a bash script that can
|
||||||
cd to each and every git and checkout the SHA that contributed to
|
cd to each and every Git and checkout the SHA that contributed to
|
||||||
the build.
|
the build.
|
||||||
- For each package built, a file shall capture he md5sums of all the
|
- For each package built, a file captures the md5sums of all the
|
||||||
source code inputs to the build of that package. These files are
|
source code inputs required to build that package. These files are
|
||||||
also produced by build-pkgs at location
|
also produced by "build-pkgs" at location
|
||||||
'$MY_WORKSPACE//rpmbuild/SOURCES//srpm_reference.md5'.
|
*$MY_WORKSPACE//rpmbuild/SOURCES//srpm_reference.md5*.
|
||||||
- All these build products are accessible locally (e.g. a regional
|
- All these build products are accessible locally (e.g. a regional
|
||||||
office) via rsync (other protocols can be added later)
|
office) using "rsync".
|
||||||
|
|
||||||
|
**NOTE:** Other protocols can be added later.
|
||||||
|
|
||||||
#. Designers
|
#. Designers
|
||||||
|
|
||||||
- Request a build avoidance build. Recommended after you have just
|
- Request a build avoidance build. Recommended after you have
|
||||||
done a repo sync. e.g.
|
done synchronized the repository (i.e. "repo sync").
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
@ -596,11 +622,11 @@ Method (in brief)
|
|||||||
populate_downloads.sh
|
populate_downloads.sh
|
||||||
build-pkgs --build-avoidance
|
build-pkgs --build-avoidance
|
||||||
|
|
||||||
- Additional arguments, and/or environment variables, and/or a
|
- Use combinations of additional arguments, environment variables, and a
|
||||||
config file unique to the regional office, are used to specify a URL
|
configuration file unique to the regional office to specify an URL
|
||||||
to the reference builds.
|
to the reference builds.
|
||||||
|
|
||||||
- Using a config file to specify location of your reference build
|
- Using a configuration file to specify the location of your reference build:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
@ -621,14 +647,16 @@ Method (in brief)
|
|||||||
BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build"
|
BUILD_AVOIDANCE_DIR="/localdisk/loadbuild/jenkins/StarlingX_Reference_Build"
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
- Using command line args to specify location of your reference
|
- Using command-line arguments to specify the location of your reference
|
||||||
build
|
build:
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
build-pkgs --build-avoidance --build-avoidance-dir /localdisk/loadbuild/jenkins/StarlingX_Reference_Build --build-avoidance-host stx-builder.mycompany.com --build-avoidance-user jenkins
|
build-pkgs --build-avoidance --build-avoidance-dir /localdisk/loadbuild/jenkins/StarlingX_Reference_Build --build-avoidance-host stx-builder.mycompany.com --build-avoidance-user jenkins
|
||||||
|
|
||||||
- Prior to your build attempt, you need to accept the host key. This will prevent rsync failures on a yes/no prompt. (you should only have to do this once)
|
- Prior to your build attempt, you need to accept the host key.
|
||||||
|
Doing so prevents "rsync" failures on a "yes/no" prompt.
|
||||||
|
You only have to do this once.
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
@ -638,23 +666,25 @@ Method (in brief)
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
|
|
||||||
- build-pkgs will:
|
- "build-pkgs" does the following:
|
||||||
|
|
||||||
- From newest to oldest, scan the CONTEXTs of the various
|
- From newest to oldest, scans the CONTEXTs of the various
|
||||||
reference builds. Select the first (most recent) context which
|
reference builds. Selects the first (i.e. most recent) context that
|
||||||
satisfies the following requirement. For every git, the SHA
|
satisfies the following requirement: every Git the SHA
|
||||||
specified in the CONTEXT is present.
|
specifies in the CONTEXT is present.
|
||||||
- The selected context might be slightly out of date, but not by
|
- The selected context might be slightly out of date, but not by
|
||||||
more than a day (assuming daily reference builds).
|
more than a day. This assumes daily reference builds are run.
|
||||||
- If the context has not been previously downloaded, then
|
- If the context has not been previously downloaded, then
|
||||||
download it now. Meaning download select portions of the
|
download it now. This means you need to download select portions of the
|
||||||
reference build workspace into the designer's workspace. This
|
reference build workspace into the designer's workspace. This
|
||||||
includes all the SRPMS, RPMS, MD5SUMS, and misc supporting
|
includes all the SRPMS, RPMS, MD5SUMS, and miscellaneous supporting
|
||||||
files. (~10 min over office LAN)
|
files. Downloading these files usually takes about 10 minutes
|
||||||
- The designer may have additional commits not present in the
|
over an office LAN.
|
||||||
reference build, or uncommitted changes. Affected packages will
|
- The designer could have additional commits or uncommitted changes
|
||||||
identified by the differing md5sum's, and the package is
|
not present in the reference builds. Affected packages are
|
||||||
re-built. (5+ min, depending on what packages have changed)
|
identified by the differing md5sum's. In these cases, the packages
|
||||||
|
are re-built. Re-builds usually take five or more minutes,
|
||||||
|
depending on the packages that have changed.
|
||||||
|
|
||||||
- What if no valid reference build is found? Then build-pkgs will fall
|
- What if no valid reference build is found? Then build-pkgs will fall
|
||||||
back to a regular build.
|
back to a regular build.
|
||||||
@ -664,16 +694,16 @@ Reference Builds
|
|||||||
****************
|
****************
|
||||||
|
|
||||||
- The regional office implements an automated build that pulls the
|
- The regional office implements an automated build that pulls the
|
||||||
latest StarlingX software and builds it on a regular basis. e.g. a
|
latest StarlingX software and builds it on a regular basis (e.g.
|
||||||
daily. Perhaps implemented by Jenkins, cron, or similar tools.
|
daily builds). Jenkins, cron, or similar tools can trigger these builds.
|
||||||
- Each build is saved to a unique directory, and preserved for a time
|
- Each build is saved to a unique directory, and preserved for a time
|
||||||
that is reflective of how long a designer might be expected to work
|
that is reflective of how long a designer might be expected to work
|
||||||
on a private branch without syncronizing with the master branch. e.g.
|
on a private branch without syncronizing with the master branch.
|
||||||
2 weeks.
|
This takes about two weeks.
|
||||||
|
|
||||||
- The MY_WORKSPACE directory for the build shall have a common root
|
- The *MY_WORKSPACE* directory for the build shall have a common root
|
||||||
directory, and a leaf directory that is a sortable time stamp. Suggested
|
directory, and a leaf directory that is a sortable time stamp. The
|
||||||
format YYYYMMDDThhmmss. e.g.
|
suggested format is *YYYYMMDDThhmmss*.
|
||||||
|
|
||||||
.. code:: sh
|
.. code:: sh
|
||||||
|
|
||||||
@ -683,24 +713,25 @@ Reference Builds
|
|||||||
MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP}
|
MY_WORKSPACE=${BUILD_AVOIDANCE_DIR}/${BUILD_TIMESTAMP}
|
||||||
|
|
||||||
- Designers can access all build products over the internal network of
|
- Designers can access all build products over the internal network of
|
||||||
the regional office. The current prototype employs rsync. Other
|
the regional office. The current prototype employs "rsync". Other
|
||||||
protocols that can efficiently share/copy/transfer large directories
|
protocols that can efficiently share, copy, or transfer large directories
|
||||||
of content can be added as needed.
|
of content can be added as needed.
|
||||||
|
|
||||||
^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^
|
||||||
Advanced Usage
|
Advanced Usage
|
||||||
^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^
|
||||||
|
|
||||||
Can the reference build itself use build avoidance? Yes
|
Can the reference build itself use build avoidance? Yes it can.
|
||||||
Can it reference itself? Yes.
|
Can it reference itself? Yes it can.
|
||||||
In either case we advise caution. To protect against any possible
|
In both these cases, caution is advised. To protect against any possible
|
||||||
'divergence from reality', you should limit how many steps removed a
|
'divergence from reality', you should limit how many steps you remove
|
||||||
build avoidance build is from a full build.
|
a build avoidance build from a full build.
|
||||||
|
|
||||||
Suppose we want to implement a self referencing daily build, except
|
Suppose we want to implement a self-referencing daily build in an
|
||||||
that a full build occurs every Saturday. To protect ourselves from a
|
environment where a full build already occurs every Saturday.
|
||||||
build failure on Saturday we also want a limit of 7 days since last
|
To protect ourselves from a
|
||||||
full build. You build script might look like this ...
|
build failure on Saturday we also want a limit of seven days since
|
||||||
|
the last full build. Your build script might look like this ...
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
@ -779,24 +810,23 @@ full build. You build script might look like this ...
|
|||||||
fi
|
fi
|
||||||
...
|
...
|
||||||
|
|
||||||
One final wrinkle.
|
A final note....
|
||||||
|
|
||||||
We can ask build avoidance to preferentially use the full build day
|
To use the full build day as your avoidance build reference point,
|
||||||
rather than the most recent build, as the reference point of the next
|
modify the "build-pkgs" commands above to use "--build-avoidance-day ",
|
||||||
avoidance build via use of '--build-avoidance-day '. e.g. substitute
|
as shown in the following two examples:
|
||||||
this line into the above.
|
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $FULL_BUILD_DAY
|
build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $FULL_BUILD_DAY
|
||||||
|
|
||||||
# or perhaps, with a bit more shuffling of the above script.
|
# Here is another example with a bit more shuffling of the above script.
|
||||||
|
|
||||||
build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $LAST_FULL_BUILD_DAY
|
build-pkgs --build-avoidance --build-avoidance-dir $BUILD_AVOIDANCE_DIR --build-avoidance-host $BUILD_AVOIDANCE_HOST --build-avoidance-user $USER --build-avoidance-day $LAST_FULL_BUILD_DAY
|
||||||
|
|
||||||
The advantage is that our build is never more than one step removed
|
The advantage is that our build is never more than one step removed
|
||||||
from a full build (assuming the full build was successful).
|
from a full build. This assumes the full build was successful.
|
||||||
|
|
||||||
The disadvantage is that by end of week the reference build is getting
|
The disadvantage is that by the end of the week, the reference build is getting
|
||||||
rather old. During active weeks, builds times might be approaching
|
rather old. During active weeks, build times could approach build times for
|
||||||
that of a full build.
|
full builds.
|
||||||
|
Loading…
Reference in New Issue
Block a user