tools/centos-mirror-tools
Don Penney 5497f91e85 Add designer workflow tool for downloading new packages and dependencies
This update introduces the starlingx_add_pkgs.sh utility. intended as
a designer workflow aid when adding new packages to the build or runtime
load. It uses the repo configuration files from centos-mirror-tools in
conjunction with the cgcs-centos-repo and build repos to determine
packages that must be download to satisfy requirements of new specified
packages being introduced. The downloaded RPMs are logged, the output
of which can be then used for populating the .lst files, as needed.

Change-Id: I655fab40e2b91dfeb9c5f714f8dc3f2be24cb8f4
Story: 2003294
Task: 24243
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-08-07 11:49:41 -05:00
..
rpm-gpg-keys Adding StarlingX_3rd.repo sourcelist for 3rd pkgs 2018-06-27 21:07:36 -05:00
dl_other_from_centos_repo.sh Add build tools 2018-06-08 17:01:43 -05:00
dl_rpms.sh Log the urls of downloaded rpms 2018-07-30 16:15:10 -04:00
Dockerfile Mirror Tools: Git Repositories Required 2018-07-03 14:41:51 -07:00
download_mirror.sh StarlingX: Tar Compressed files repository 2018-07-03 06:04:28 -07:00
mvn-artifacts.lst StarlingX: Tar Compressed files repository 2018-07-03 06:04:28 -07:00
other_downloads.lst Add build tools 2018-06-08 17:01:43 -05:00
README.rst Port documentation from markdown into rst file 2 2018-07-09 09:43:00 -05:00
rpms_from_3rd_parties.lst removing no require lines for download pkg 2018-07-05 16:15:34 -05:00
rpms_from_centos_3rd_parties.lst Add packages required for the introduction of gnocchi 2018-07-31 00:11:25 -05:00
rpms_from_centos_repo.lst Add packages required for the introduction of gnocchi 2018-07-31 00:11:25 -05:00
show_imported_rpm_gpg_keys.sh Add build tools 2018-06-08 17:01:43 -05:00
StarlingX_3rd.repo Merge "Update Starlingx_3rd.repo with docker x86_64 pkgs" 2018-07-12 21:52:23 +00:00
starlingx_add_pkgs.sh Add designer workflow tool for downloading new packages and dependencies 2018-08-07 11:49:41 -05:00
StarlingX.repo Add repo for finding older python packages 2018-07-26 09:01:27 -05:00
tarball-dl.lst Add packages required for the introduction of gnocchi 2018-07-31 00:11:25 -05:00
tarball-dl.sh Update handling of mvn.repo.tgz 2018-07-20 09:58:04 -05:00

Create mirror for StarlingX

Step 0 - Build the container

Build the docker image on your Linux host (with Docker supported). NOTE: if necessary you might have to set http/https proxy in your Dockerfile before building the docker image below.

$ docker build -t <your_docker_image_name>:<your_image_version> -f Dockerfile .

Step 1 - Run the container

The container shall be run from the same directory where the other scripts are stored.

$ docker run -it -v $(pwd):/localdisk <your_docker_image_name>:<your_image_version>

The container can also be run the following way, so the download_mirror.sh runs automatically without having to enter the container and the step 2can be simplified.

$ docker run -it -v $(pwd):/localdisk <your_docker_image_name>:<your_image_version> download_mirror.sh

As /localdisk is defined as the workdir of the container, the same folder name should be used to define the volume. The container will start to run and populate a logs and output folders in this directory. The container shall be run from the same directory where the other scripts are stored.

step 2 - Run the download_mirror.sh script

Once inside the container run the downloader script

$ ./download_mirror.sh

NOTE: in case there are some downloading failures due to network instability (or timeout), you should download them manually, to assure you get all RPMs listed in "rpms_from_3rd_parties.lst" and "rpms_from_centos_repo.lst".

step 3 - Copy the files to the mirror

After all downloading complete, copy the download files to mirror.

$ find ./output -name "*.i686.rpm" | xargs rm -f $ chown 751:751 -R ./output $ cp -rf output/stx-r1/ <your_mirror_folder>/

In this case <your_mirror_folder> can be whatever folder you want to use as mirror.

step 4 - Tweaks in the StarlingX build system.

NOTE: step below is not needed if you've synced the latest codebase.

Go into StarlingX build system (another container which hosts cgcs build system), and follow up below steps:

Debugging issues

The download_mirror.sh script will create log files in the form of centos_rpms_*.txt. After the download is complete, it's recommended to check the content of these files to see if everything was downloaded correctly.

A quick look into these files could be:

$ cd output/ $ cat missing

In this case, there shoudn't be any package in the "missing" files.