diff --git a/doc/source/admin/index.rst b/doc/source/admin/index.rst index c220ba2..6ae4469 100644 --- a/doc/source/admin/index.rst +++ b/doc/source/admin/index.rst @@ -9,6 +9,5 @@ using the builders provided in the **ironic-python-agent-builder** project. :maxdepth: 2 dib - tinyipa .. _Ironic Python Agent: https://docs.openstack.org/ironic-python-agent diff --git a/doc/source/admin/tinyipa.rst b/doc/source/admin/tinyipa.rst deleted file mode 100644 index 4d2187b..0000000 --- a/doc/source/admin/tinyipa.rst +++ /dev/null @@ -1,159 +0,0 @@ -TinyIPA images -============== - -TinyIPA is an `Ironic Python Agent`_ image based on TinyCoreLinux_. It is very -lightweight and thus very suitable for CI use. It may lack necessary drivers -and the build process uses insecure communication, thus these images are not -recommended for production usage. - -Requirements ------------- - -You need to have a git clone of **ironic-python-agent-builder**: - -.. code-block:: shell - - git clone https://opendev.org/openstack/ironic-python-agent-builder - cd ironic-python-agent-builder/tinyipa - -Then you need to install some utilities. For the main build script: - -* wget -* pip -* unzip -* sudo -* awk -* mksquashfs - -For building an ISO you'll also need: - -* mkisofs, genisoimage, or xorrisofs - -Building --------- - -Building ramdisk -~~~~~~~~~~~~~~~~ - -To create a new ramdisk, run: - -.. code-block:: shell - - make - -or: - -.. code-block:: shell - - ./build-tinyipa.sh && ./finalise-tinyipa.sh - -This will create two new files once completed: - -* ``tinyipa.vmlinuz`` - the kernel image -* ``tinyipa.gz`` - the initramfs image - -Upload them to the Image service or another location where you want them to be -hosted (an HTTP or FILE location in case of standalone ironic). - -Building ISO -~~~~~~~~~~~~ - -Once you've built tinyIPA it is possible to pack it into an ISO if required. To -create a bootable ISO, run: - -.. code-block:: shell - - make iso - -or: - -.. code-block:: shell - - ./build-iso.sh - -This will create one new file once completed: - -* ``tinyipa.iso`` - - -Cleaning up -~~~~~~~~~~~ - -To clean up the whole build environment, run: - -.. code-block:: shell - - make clean - -For cleaning up just the iso or just the ramdisk build: - -.. code-block:: shell - - make clean_iso - -or: - -.. code-block:: shell - - make clean_build - -Advanced options ----------------- - -Enabling/disabling SSH access to the ramdisk -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -By default tinyIPA will be built with OpenSSH server installed but no -public SSH keys authorized to access it. - -If you want to enable SSH access to the image, set ``AUTHORIZE_SSH`` variable -in your shell before building tinyIPA: - -.. code-block:: bash - - export AUTHORIZE_SSH=true - -By default it will use public RSA or DSA keys of the user running the build. -To provide a different public SSH key, export path to it in your shell before -building tinyIPA: - -.. code-block:: bash - - export SSH_PUBLIC_KEY= - -If you want to disable SSH altogether, set ``INSTALL_SSH`` variable in your -shell to ``false`` before building tinyIPA: - -.. code-block:: bash - - export INSTALL_SSH=false - -If you want to change the SSH access of a previously built tinyIPA image, -use the make target ``addssh``: - -.. code-block:: shell - - make addssh - -This command will either use a local image specified by the -``TINYIPA_RAMDISK_FILE`` environment variable or download the version -specified by the ``BRANCH_PATH`` environment variable (e.g. ``master`` or -``stable-queens``) from `tarballs.openstack.org -`_. -It will install and configure OpenSSH if needed and add public SSH keys for -the user named ``tc`` using either the same ``SSH_PUBLIC_KEY`` shell variable -or the public keys of the local user. - -Enabling biosdevname in the ramdisk -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you want to collect BIOS given names of NICs in the inventory, set -``TINYIPA_REQUIRE_BIOSDEVNAME`` variable in your shell before building tinyIPA: - -.. code-block:: bash - - export TINYIPA_REQUIRE_BIOSDEVNAME=true - - -.. _Ironic Python Agent: https://docs.openstack.org/ironic-python-agent -.. _TinyCoreLinux: http://tinycorelinux.net diff --git a/ironic_python_agent_builder/__init__.py b/ironic_python_agent_builder/__init__.py index 31a216c..43f3475 100644 --- a/ironic_python_agent_builder/__init__.py +++ b/ironic_python_agent_builder/__init__.py @@ -60,7 +60,6 @@ def main(): parser.add_argument("--elements-path", help="Path(s) to custom DIB elements separated by " "a colon") - # TODO(dtantsur): handle distribution == tinyipa args = parser.parse_args() if args.elements_path: os.environ['ELEMENTS_PATH'] = args.elements_path diff --git a/releasenotes/notes/drop-tinyipa-support-628b5a8d92b4af15.yaml b/releasenotes/notes/drop-tinyipa-support-628b5a8d92b4af15.yaml new file mode 100644 index 0000000..16986aa --- /dev/null +++ b/releasenotes/notes/drop-tinyipa-support-628b5a8d92b4af15.yaml @@ -0,0 +1,5 @@ +--- +upgrade: + - | + TinyIPA CI, builder, related documentation and configuration have been + removed. Any remaining downstream TinyIPA references are legacy. diff --git a/roles/ipa-build-tinyipa-image/defaults/main.yaml b/roles/ipa-build-tinyipa-image/defaults/main.yaml deleted file mode 100644 index 0f62135..0000000 --- a/roles/ipa-build-tinyipa-image/defaults/main.yaml +++ /dev/null @@ -1,4 +0,0 @@ -ipa_branch_path: '{{ zuul.branch | replace("/", "-") }}' -ipa_raw_dir: '{{ ansible_user_dir }}/src/opendev.org/openstack/ironic-python-agent/UPLOAD_RAW' -ipa_tar_dir: '{{ ansible_user_dir }}/src/opendev.org/openstack/ironic-python-agent/UPLOAD_TAR' -tinyipa_dir: '{{ ansible_user_dir }}/src/opendev.org/openstack/ironic-python-agent-builder/tinyipa' diff --git a/roles/ipa-build-tinyipa-image/tasks/main.yaml b/roles/ipa-build-tinyipa-image/tasks/main.yaml deleted file mode 100644 index 1c827b1..0000000 --- a/roles/ipa-build-tinyipa-image/tasks/main.yaml +++ /dev/null @@ -1,30 +0,0 @@ -- name: Build a TinyIPA image - command: make - args: - chdir: '{{ tinyipa_dir }}' - environment: - BRANCH_PATH: '{{ ipa_branch_path }}' - IPA_SOURCE_DIR: '{{ ansible_user_dir }}/src/opendev.org/openstack/ironic-python-agent' - -- name: Build a TinyIPA iso - command: make iso - args: - chdir: '{{ tinyipa_dir }}' - environment: - BRANCH_PATH: '{{ ipa_branch_path }}' - IPA_SOURCE_DIR: '{{ ansible_user_dir }}/src/opendev.org/openstack/ironic-python-agent' - -- name: Move resulting files - shell: | - mv tinyipa*.tar.gz* "{{ ipa_tar_dir }}" - mv tinyipa*.* "{{ ipa_raw_dir }}" - mv tiny-instance-uec*.tar.gz "{{ ipa_tar_dir }}" - args: - chdir: '{{ tinyipa_dir }}' - -# NOTE(sambetts) Must clean up chroot directories before -# publisher rsync command tries and fails to read them -- name: Clean up the environment - command: make clean - args: - chdir: '{{ tinyipa_dir }}' diff --git a/setup.cfg b/setup.cfg index 17245b2..4b05ab0 100644 --- a/setup.cfg +++ b/setup.cfg @@ -20,7 +20,6 @@ console_scripts = [files] packages = ironic_python_agent_builder -# TODO(dtantsur): figure out a sane way to distribute tinyipa scripts data_files = share/ironic-python-agent-builder/dib = dib/* diff --git a/tinyipa/.gitignore b/tinyipa/.gitignore deleted file mode 100644 index 1e6b193..0000000 --- a/tinyipa/.gitignore +++ /dev/null @@ -1,13 +0,0 @@ -*.gz -*.zip -*.tar.gz -tinyipabuild -tinyipafinal -*.vmlinuz -vmlinuz64 -*.sha256 -*.iso -*.tcz* -squashfs-root -syslinux* -newiso diff --git a/tinyipa/Makefile b/tinyipa/Makefile deleted file mode 100644 index ad0bf47..0000000 --- a/tinyipa/Makefile +++ /dev/null @@ -1,45 +0,0 @@ -.PHONY: default all dependencies build finalise addssh iso clean clean_build clean_iso -default: dependencies build finalise instance-images - -all: dependencies build finalise iso instance-images - -dependencies: - ./install-deps.sh - -build: - ./build-tinyipa.sh - -finalise: - ./finalise-tinyipa.sh - -addssh: - ./add-ssh-tinyipa.sh - -iso: - ./build-iso.sh - -instance-images: - ./build-instance-images.sh - -clean: clean_build clean_iso - -clean_build: - sudo -v - sudo rm -rf tinyipabuild - sudo rm -rf tinyipafinal - sudo rm -rf tinyipaaddssh - rm -f *tinyipa*.vmlinuz - rm -f *tinyipa*.gz - rm -f *tinyipa*.sha256 - rm -f build_files/corepure64.gz - rm -f build_files/vmlinuz64 - rm -f build_files/*.tcz - rm -f build_files/*.tcz.* - rm -f tiny-instance-part*.img - rm -f tiny-instance-uec*.tar.gz - -clean_iso: - rm -rf newiso - rm -f build_files/syslinux-4.06.tar.gz - rm -rf build_files/syslinux-4.06 - rm -f tinyipa.iso diff --git a/tinyipa/README.rst b/tinyipa/README.rst deleted file mode 100644 index a02b1f9..0000000 --- a/tinyipa/README.rst +++ /dev/null @@ -1,6 +0,0 @@ -============================= -Tiny Core Ironic Python Agent -============================= - -See -https://docs.openstack.org/ironic-python-agent-builder/latest/admin/tinyipa.html diff --git a/tinyipa/add-ssh-tinyipa.sh b/tinyipa/add-ssh-tinyipa.sh deleted file mode 100755 index c404524..0000000 --- a/tinyipa/add-ssh-tinyipa.sh +++ /dev/null @@ -1,130 +0,0 @@ -#!/bin/bash - -# Rebuild upstream pre-built tinyipa it to be usable with ansible-deploy. -# -# Downloads the pre-built tinyipa ramdisk from tarballs.openstack.org or -# rebuilds a ramdisk under path set as TINYIPA_RAMDISK_FILE shell var. - -# During rebuild this script installs and configures OpenSSH server if needed -# and makes required changes for Ansible + Python to work in compiled/optimized -# Python environment. -# -# By default, id_rsa or id_dsa keys of the user performing the build -# are baked into the image as authorized_keys for 'tc' user. -# To supply different public ssh key, before running this script set -# SSH_PUBLIC_KEY environment variable to point to absolute path to the key. -# -# This script produces "ansible-" ramdisk that can serve -# as ramdisk for both ansible-deploy driver and agent-based Ironic drivers, - -set -ex -WORKDIR=$(readlink -f $0 | xargs dirname) -REBUILDDIR="$WORKDIR/tinyipaaddssh" -DST_DIR=$REBUILDDIR -source ${WORKDIR}/common.sh - -TINYCORE_MIRROR_URL=${TINYCORE_MIRROR_URL:-} -BRANCH_PATH=${BRANCH_PATH:-master} -TINYIPA_RAMDISK_FILE=${TINYIPA_RAMDISK_FILE:-} - -SSH_PUBLIC_KEY=${SSH_PUBLIC_KEY:-} - -SSHD_CONFIG_PATH="/usr/local/etc/ssh/sshd_config" -SSH_RSA_KEY_PATH="/usr/local/etc/ssh/ssh_host_rsa_key" -SSH_DSA_KEY_PATH="/usr/local/etc/ssh/ssh_host_dsa_key" -SSH_ED25519_KEY_PATH="/usr/local/etc/ssh/ssh_host_ed25519_key" - -function validate_params { - echo "Validating location of public SSH key" - if [ -n "$SSH_PUBLIC_KEY" ]; then - if [ -r "$SSH_PUBLIC_KEY" ]; then - _found_ssh_key="$SSH_PUBLIC_KEY" - fi - else - for fmt in rsa dsa; do - if [ -r "$HOME/.ssh/id_$fmt.pub" ]; then - _found_ssh_key="$HOME/.ssh/id_$fmt.pub" - break - fi - done - fi - - if [ -z $_found_ssh_key ]; then - echo "Failed to find neither provided nor default SSH key" - exit 1 - fi -} - -function get_tinyipa { - if [ -z $TINYIPA_RAMDISK_FILE ]; then - mkdir -p $WORKDIR/build_files/cache - cd $WORKDIR/build_files/cache - wget -N https://tarballs.openstack.org/ironic-python-agent/tinyipa/files/tinyipa${BRANCH_EXT}.gz - TINYIPA_RAMDISK_FILE="$WORKDIR/build_files/cache/tinyipa${BRANCH_EXT}.gz" - fi -} - -function unpack_ramdisk { - - if [ -d "$REBUILDDIR" ]; then - sudo rm -rf "$REBUILDDIR" - fi - - mkdir -p "$REBUILDDIR" - - # Extract rootfs from .gz file - ( cd "$REBUILDDIR" && zcat "$TINYIPA_RAMDISK_FILE" | sudo cpio -i -H newc -d ) - -} - -function install_ssh { - if [ ! -f "${REBUILDDIR}${SSHD_CONFIG_PATH}" ]; then - # tinyipa was built without SSH server installed - # Install and configure bare minimum for SSH access - $TC_CHROOT_CMD tce-load -wic openssh - # Configure OpenSSH - $CHROOT_CMD cp ${SSHD_CONFIG_PATH}.orig $SSHD_CONFIG_PATH - echo "PasswordAuthentication no" | $CHROOT_CMD tee -a $SSHD_CONFIG_PATH - # Generate and configure host keys - RSA, DSA, Ed25519 - # NOTE(pas-ha) ECDSA host key will still be re-generated fresh on every image boot - $CHROOT_CMD ssh-keygen -q -t rsa -N "" -f $SSH_RSA_KEY_PATH - $CHROOT_CMD ssh-keygen -q -t dsa -N "" -f $SSH_DSA_KEY_PATH - $CHROOT_CMD ssh-keygen -q -t ed25519 -N "" -f $SSH_ED25519_KEY_PATH - echo "HostKey ${SSH_RSA_KEY_PATH}" | $CHROOT_CMD tee -a $SSHD_CONFIG_PATH - echo "HostKey ${SSH_DSA_KEY_PATH}" | $CHROOT_CMD tee -a $SSHD_CONFIG_PATH - echo "HostKey ${SSH_ED25519_KEY_PATH}" | $CHROOT_CMD tee -a $SSHD_CONFIG_PATH - fi - - # setup new user SSH keys anyway - $CHROOT_CMD mkdir -p /home/tc - $CHROOT_CMD chown -R tc.staff /home/tc - $TC_CHROOT_CMD mkdir -p /home/tc/.ssh - cat $_found_ssh_key | $TC_CHROOT_CMD tee /home/tc/.ssh/authorized_keys - $CHROOT_CMD chown tc.staff /home/tc/.ssh/authorized_keys - $TC_CHROOT_CMD chmod 600 /home/tc/.ssh/authorized_keys -} - - -function rebuild_ramdisk { - # Rebuild build directory into gz file - ansible_basename="ansible-$(basename $TINYIPA_RAMDISK_FILE)" - ( cd "$REBUILDDIR" && sudo find | sudo cpio -o -H newc | gzip -9 > "$WORKDIR/${ansible_basename}" ) - # Output file created by this script and its size - cd "$WORKDIR" - echo "Produced files:" - du -h "${ansible_basename}" -} - -sudo -v - - -validate_params -get_tinyipa -unpack_ramdisk -setup_tce "$DST_DIR" - -# NOTE (pas-ha) default tinyipa is built without SSH access, enable it here -install_ssh - -cleanup_tce "$DST_DIR" -rebuild_ramdisk diff --git a/tinyipa/build-instance-images.sh b/tinyipa/build-instance-images.sh deleted file mode 100755 index 0fd0221..0000000 --- a/tinyipa/build-instance-images.sh +++ /dev/null @@ -1,39 +0,0 @@ -#!/bin/bash - -set -ex -WORKDIR=$(readlink -f $0 | xargs dirname) -DST_DIR=$(mktemp -d) -source ${WORKDIR}/common.sh -PARTIMG="$WORKDIR/tiny-instance-part${BRANCH_EXT}.img" -UECFILE="$WORKDIR/tiny-instance-uec${BRANCH_EXT}.tar.gz" -fs_type='ext4' - -sudo rm -rf $PARTIMG $UECFILE -sudo truncate --size=150M $PARTIMG - -sudo mkfs."${fs_type}" -F "$PARTIMG" -L "root" -sudo mount -o loop "$PARTIMG" "$DST_DIR/" - -# Extract rootfs from .gz file -( cd "$DST_DIR" && zcat $WORKDIR/build_files/corepure64.gz | sudo cpio -i -H newc -d ) - -setup_tce $DST_DIR - -# NOTE(rpittau) change ownership of the tce info dir to prevent writing issues -sudo chown $TC:$STAFF $DST_DIR/usr/local/tce.installed - -$TC_CHROOT_CMD tce-load -wci grub2-multi.tcz - -cleanup_tce $DST_DIR -sudo umount $DST_DIR/ - -pushd $DST_DIR/ -cp $WORKDIR/tinyipa${BRANCH_EXT}.gz $DST_DIR/tinyipa-initrd -cp $WORKDIR/tinyipa${BRANCH_EXT}.vmlinuz $DST_DIR/tinyipa-vmlinuz -cp $PARTIMG $DST_DIR/ - -tar -czf $UECFILE ./ - -popd - -sudo rm -rf $DST_DIR diff --git a/tinyipa/build-iso.sh b/tinyipa/build-iso.sh deleted file mode 100755 index ba8cf1b..0000000 --- a/tinyipa/build-iso.sh +++ /dev/null @@ -1,35 +0,0 @@ -#!/bin/bash - -set -ex -WORKDIR=$(readlink -f $0 | xargs dirname) -SYSLINUX_VERSION="6.03" -SYSLINUX_URL="https://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-${SYSLINUX_VERSION}.tar.gz" - -source ${WORKDIR}/common.sh - -cd ${WORKDIR}/build_files -wget -N $SYSLINUX_URL && tar zxf syslinux-${SYSLINUX_VERSION}.tar.gz - -cd $WORKDIR -rm -rf newiso -mkdir -p newiso/boot/isolinux -cp build_files/syslinux-${SYSLINUX_VERSION}/bios/core/isolinux.bin newiso/boot/isolinux/. -cp build_files/isolinux.cfg newiso/boot/isolinux/. -cp tinyipa${BRANCH_EXT}.gz newiso/boot/corepure64.gz -cp tinyipa${BRANCH_EXT}.vmlinuz newiso/boot/vmlinuz64 - -set +e -ISO_BUILDER="" - -for builder in mkisofs genisoimage xorrisofs; do - if $($builder --help); then - ISO_BUILDER=$builder - fi -done -if [ -z "$ISO_BUILDER" ]; then - echo "Please install a ISO filesystem builder utility such as mkisofs, genisoimage, or xorrisofs." - exit 1 -fi - -set -e -$ISO_BUILDER -l -r -J -R -V TC-custom -no-emul-boot -boot-load-size 4 -boot-info-table -b boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat -o tinyipa.iso newiso diff --git a/tinyipa/build-tinyipa.sh b/tinyipa/build-tinyipa.sh deleted file mode 100755 index 5e14e80..0000000 --- a/tinyipa/build-tinyipa.sh +++ /dev/null @@ -1,264 +0,0 @@ -#!/bin/bash - -set -ex -WORKDIR=$(readlink -f $0 | xargs dirname) -source ${WORKDIR}/common.sh - -IRONIC_LIB_SOURCE=${IRONIC_LIB_SOURCE:-} - -TC_RELEASE="15.x" -QEMU_RELEASE="5.2.0" -LSHW_RELEASE="B.02.18" -BIOSDEVNAME_RELEASE="0.7.2" -IPMITOOL_GIT_HASH="19d78782d795d0cf4ceefe655f616210c9143e62" - -CHROOT_CMD="sudo chroot $BUILDDIR /usr/bin/env -i PATH=$CHROOT_PATH http_proxy=$http_proxy https_proxy=$https_proxy no_proxy=$no_proxy" - -DOWNLOAD_RETRY_MAX=${DOWNLOAD_RETRY_MAX:-5} -DOWNLOAD_RETRY_DELAY=${DOWNLOAD_RETRY_DELAY:-10} - -function download_with_retry() { - local source_url=$1 - local destination_path=$2 - local attempts=1 - - declare -A tar_flags=( - ["tar.gz"]="z" - ["tgz"]="z" - ["tar.bz2"]="j" - ["tbz2"]="j" - ["tar.xz"]="J" - ["txz"]="J" - ) - - echo "Downloading $source_url to $destination_path" - while [ $attempts -le $DOWNLOAD_RETRY_MAX ]; do - for ext in "${!tar_flags[@]}"; do - if [[ "$source_url" =~ \.${ext}$ ]]; then - mkdir -p "$destination_path" - if wget --timeout=30 --tries=3 -O - "$source_url" | tar -x${tar_flags[$ext]} -C "$destination_path" --strip-components=1 -f -; then - echo "Successfully downloaded $source_url on attempt $attempts" - return 0 - fi - break - fi - done - - if [[ ! "$source_url" =~ \.tar\.[a-z]+$ ]]; then - if wget --timeout=30 --tries=3 "$source_url" -O "${destination_path}"; then - echo "Successfully downloaded $source_url on attempt $attempts" - return 0 - fi - fi - - echo "Download attempt $attempts failed for $source_url, retrying in $DOWNLOAD_RETRY_DELAY seconds..." - sleep $DOWNLOAD_RETRY_DELAY - attempts=$((attempts + 1)) - done - - echo "Failed to download $source_url after $DOWNLOAD_RETRY_MAX attempts" - return 1 -} - -function tce_load_with_retry() { - local package=$1 - local attempts=1 - - echo "Loading package $package with tce-load" - while [ $attempts -le $DOWNLOAD_RETRY_MAX ]; do - if sudo chroot --userspec=$TC:$STAFF $BUILDDIR /usr/bin/env -i PATH=$CHROOT_PATH http_proxy=$http_proxy https_proxy=$https_proxy no_proxy=$no_proxy tce-load -wci $package; then - echo "Successfully loaded $package on attempt $attempts" - return 0 - fi - - echo "tce-load attempt $attempts failed for $package, retrying in $DOWNLOAD_RETRY_DELAY seconds..." - sleep $DOWNLOAD_RETRY_DELAY - attempts=$((attempts + 1)) - done - - echo "Failed to load $package with tce-load after $DOWNLOAD_RETRY_MAX attempts" - return 1 -} - -echo "Building tinyipa:" - -# Ensure we have an extended sudo to prevent the need to enter a password over -# and over again. -sudo -v - -# If an old build directory exists remove it -if [ -d "$BUILDDIR" ]; then - sudo rm -rf "$BUILDDIR" -fi - -############################################## -# Download and Cache Tiny Core Files -############################################## - -# Find a working TC mirror if none is explicitly provided -choose_tc_mirror - -cd $WORKDIR/build_files -download_with_retry "$TINYCORE_MIRROR_URL/$TC_RELEASE/x86_64/release/distribution_files/corepure64.gz" "corepure64.gz" -download_with_retry "$TINYCORE_MIRROR_URL/$TC_RELEASE/x86_64/release/distribution_files/vmlinuz64" "vmlinuz64" -cd $WORKDIR - -######################################################## -# Build Required Python Dependencies in a Build Directory -######################################################## - -# Make directory for building in -mkdir "$BUILDDIR" - -# Extract rootfs from .gz file -( cd "$BUILDDIR" && zcat $WORKDIR/build_files/corepure64.gz | sudo cpio -i -H newc -d ) - -# Configure mirror -sudo sh -c "echo $TINYCORE_MIRROR_URL > $BUILDDIR/opt/tcemirror" - -# Download Qemu-utils, Biosdevname and IPMItool source -download_with_retry "https://download.qemu.org/qemu-${QEMU_RELEASE}.tar.xz" "${BUILDDIR}/tmp/qemu" -download_with_retry "https://github.com/lyonel/lshw/archive/refs/tags/${LSHW_RELEASE}.tar.gz" "${BUILDDIR}/tmp/lshw" -if $TINYIPA_REQUIRE_BIOSDEVNAME; then - download_with_retry "https://linux.dell.com/biosdevname/biosdevname-${BIOSDEVNAME_RELEASE}/biosdevname-${BIOSDEVNAME_RELEASE}.tar.gz" "${BUILDDIR}/tmp/biosdevname" -fi -if $TINYIPA_REQUIRE_IPMITOOL; then - git clone https://codeberg.org/IPMITool/ipmitool.git "${BUILDDIR}/tmp/ipmitool-src" - cd "${BUILDDIR}/tmp/ipmitool-src" - git reset $IPMITOOL_GIT_HASH --hard - cd - -fi - -# Create directory for python local mirror -mkdir -p "$BUILDDIR/tmp/localpip" - -# Download IPA and requirements -IPA_SOURCE_DIR=${IPA_SOURCE_DIR:-/opt/stack/ironic-python-agent} -cd $IPA_SOURCE_DIR -rm -rf *.egg-info -pwd - -PYTHON_COMMAND="python3" -$PYTHON_COMMAND setup.py sdist --dist-dir "$BUILDDIR/tmp/localpip" --quiet - -ls $BUILDDIR/tmp/localpip || true -cp requirements.txt $BUILDDIR/tmp/ipa-requirements.txt - -if [ -n "$PYTHON_EXTRA_SOURCES_DIR_LIST" ]; then - IFS="," read -ra PKGDIRS <<< "$PYTHON_EXTRA_SOURCES_DIR_LIST" - for PKGDIR in "${PKGDIRS[@]}"; do - PKG=$(cd "$PKGDIR" ; $PYTHON_COMMAND setup.py --name) - pushd "$PKGDIR" - rm -rf *.egg-info - $PYTHON_COMMAND setup.py sdist --dist-dir "$BUILDDIR/tmp/localpip" --quiet - if [[ -r requirements.txt ]]; then - cp requirements.txt $BUILDDIR/tmp/${PKG}-requirements.txt - fi - popd - done -fi - -$WORKDIR/generate_tox_constraints.sh upper-constraints.txt -cp upper-constraints.txt $BUILDDIR/tmp/upper-constraints.txt -echo Using upper-constraints: -cat upper-constraints.txt -cd $WORKDIR - -sudo cp /etc/resolv.conf $BUILDDIR/etc/resolv.conf - -trap "sudo umount $BUILDDIR/proc; sudo umount $BUILDDIR/dev/pts" EXIT -sudo mount --bind /proc $BUILDDIR/proc -sudo mount --bind /dev/pts $BUILDDIR/dev/pts - -if [ -d /opt/stack/new ]; then - CI_DIR=/opt/stack/new -elif [ -d /opt/stack ]; then - CI_DIR=/opt/stack -else - CI_DIR= -fi - -if [ -n "$CI_DIR" ]; then - # Running in CI environment, make checkouts available - $CHROOT_CMD mkdir -p $CI_DIR - for project in $(ls $CI_DIR); do - if grep -q "$project" $BUILDDIR/tmp/upper-constraints.txt && - [ -d "$CI_DIR/$project/.git" ]; then - sudo cp -R "$CI_DIR/$project" $BUILDDIR/$CI_DIR/ - fi - done -fi - -$CHROOT_CMD mkdir -m777 /etc/sysconfig/tcedir -$CHROOT_CMD touch /etc/sysconfig/tcuser -$CHROOT_CMD chmod a+rwx /etc/sysconfig/tcuser - -mkdir $BUILDDIR/tmp/overrides -cp $WORKDIR/build_files/fakeuname $BUILDDIR/tmp/overrides/uname - -sudo cp $WORKDIR/build_files/ntpdate $BUILDDIR/bin/ntpdate - -PY_REQS="buildreqs_python3.lst" - -# NOTE(rpittau) change ownership of the tce info dir to prevent writing issues -sudo chown $TC:$STAFF $BUILDDIR/usr/local/tce.installed - -while read line; do - tce_load_with_retry "$line" -done < <(paste $WORKDIR/build_files/$PY_REQS $WORKDIR/build_files/buildreqs.lst) - -TINYIPA_PYTHON_EXE="python3.9" - -PIP_COMMAND="$TINYIPA_PYTHON_EXE -m pip" - -# Build python wheels -$CHROOT_CMD ${TINYIPA_PYTHON_EXE} -m ensurepip -$CHROOT_CMD ${PIP_COMMAND} install --upgrade pip==${PIP_VERSION} wheel -$CHROOT_CMD ${PIP_COMMAND} install pbr -$CHROOT_CMD ${PIP_COMMAND} wheel -c /tmp/upper-constraints.txt --wheel-dir /tmp/wheels -r /tmp/ipa-requirements.txt - -if [ -n "$PYTHON_EXTRA_SOURCES_DIR_LIST" ]; then - IFS="," read -ra PKGDIRS <<< "$PYTHON_EXTRA_SOURCES_DIR_LIST" - for PKGDIR in "${PKGDIRS[@]}"; do - PKG=$(cd "$PKGDIR" ; $PYTHON_COMMAND setup.py --name) - if [[ -r $BUILDDIR/tmp/${PKG}-requirements.txt ]]; then - $CHROOT_CMD ${PIP_COMMAND} wheel -c /tmp/upper-constraints.txt --wheel-dir /tmp/wheels -r /tmp/${PKG}-requirements.txt - fi - $CHROOT_CMD ${PIP_COMMAND} wheel -c /tmp/upper-constraints.txt --no-index --pre --wheel-dir /tmp/wheels --find-links=/tmp/localpip --find-links=/tmp/wheels ${PKG} - done -fi - -$CHROOT_CMD ${PIP_COMMAND} wheel -c /tmp/upper-constraints.txt --no-index --pre --wheel-dir /tmp/wheels --find-links=/tmp/localpip --find-links=/tmp/wheels ironic-python-agent -echo Resulting wheels: -ls -1 $BUILDDIR/tmp/wheels - -# Build qemu-utils -rm -rf $WORKDIR/build_files/qemu-utils.tcz -$CHROOT_CMD /bin/sh -c "cd /tmp/qemu && CFLAGS=-Wno-error ./configure --disable-system --disable-user --disable-linux-user --disable-bsd-user --disable-guest-agent --disable-blobs --enable-tools --python=/usr/local/bin/$TINYIPA_PYTHON_EXE && make && make install DESTDIR=/tmp/qemu-utils" -find $BUILDDIR/tmp/qemu-utils/ -type f -executable | xargs file | awk -F ':' '/ELF/ {print $1}' | sudo xargs strip -cd $WORKDIR/build_files && mksquashfs $BUILDDIR/tmp/qemu-utils qemu-utils.tcz && md5sum qemu-utils.tcz > qemu-utils.tcz.md5.txt -# Create qemu-utils.tcz.dep -echo "glib2.tcz" > qemu-utils.tcz.dep - -# Build lshw -rm -rf $WORKDIR/build_files/lshw.tcz -# NOTE(mjturek): We touch src/lshw.1 and clear src/po/Makefile to avoid building the man pages, as they aren't used and require large dependencies to build. -$CHROOT_CMD /bin/sh -c "cd /tmp/lshw && touch src/lshw.1 && echo install: > src/po/Makefile && make && make install DESTDIR=/tmp/lshw-installed" -find $BUILDDIR/tmp/lshw-installed/ -type f -executable | xargs file | awk -F ':' '/ELF/ {print $1}' | sudo xargs strip -cd $WORKDIR/build_files && mksquashfs $BUILDDIR/tmp/lshw-installed lshw.tcz && md5sum lshw.tcz > lshw.tcz.md5.txt - -# Build biosdevname -if $TINYIPA_REQUIRE_BIOSDEVNAME; then - rm -rf $WORKDIR/build_files/biosdevname.tcz - $CHROOT_CMD /bin/sh -c "cd /tmp/biosdevname-* && ./configure && make && make install DESTDIR=/tmp/biosdevname-installed" - find $BUILDDIR/tmp/biosdevname-installed/ -type f -executable | xargs file | awk -F ':' '/ELF/ {print $1}' | sudo xargs strip - cd $WORKDIR/build_files && mksquashfs $BUILDDIR/tmp/biosdevname-installed biosdevname.tcz && md5sum biosdevname.tcz > biosdevname.tcz.md5.txt -fi - -if $TINYIPA_REQUIRE_IPMITOOL; then - rm -rf $WORKDIR/build_files/ipmitool.tcz - # NOTE(TheJulia): Explicitly add the libtool path since /usr/local/ is not in path from the chroot. - $CHROOT_CMD /bin/sh -c "cd /tmp/ipmitool-src && env LIBTOOL='/usr/local/bin/libtool' ./bootstrap && ./configure && make && make install DESTDIR=/tmp/ipmitool" - find $BUILDDIR/tmp/ipmitool/ -type f -executable | xargs file | awk -F ':' '/ELF/ {print $1}' | sudo xargs strip - cd $WORKDIR/build_files && mksquashfs $BUILDDIR/tmp/ipmitool ipmitool.tcz && md5sum ipmitool.tcz > ipmitool.tcz.md5.txt -fi diff --git a/tinyipa/build_files/bootlocal.sh b/tinyipa/build_files/bootlocal.sh deleted file mode 100755 index 7838264..0000000 --- a/tinyipa/build_files/bootlocal.sh +++ /dev/null @@ -1,98 +0,0 @@ -#!/bin/sh -# put other system startup commands here - -#exec > /tmp/installlogs 2>&1 -set -ux - -echo "Starting bootlocal script:" -date - -export HOME=/root - -# Start SSHd -if [ -x /usr/local/etc/init.d/openssh ]; then - echo "Starting OpenSSH server:" - /usr/local/etc/init.d/openssh start -fi - -# Start haveged -if [ -x /usr/local/sbin/haveged ]; then - echo "Starting haveged entropy daemon:" - /usr/local/sbin/haveged -fi - -# Maybe save some RAM? -#rm -rf /tmp/builtin - -# Install IPA and dependencies -if ! type "ironic-python-agent" > /dev/null ; then - PIP_COMMAND="pip" - if hash pip3 2>/dev/null; then - PIP_COMMAND="pip3" - fi - $PIP_COMMAND install --no-index --find-links=file:///tmp/wheelhouse ironic_python_agent -fi - -# Create ipa-rescue-config directory for rescue password -sudo mkdir -p /etc/ipa-rescue-config - -# Setup DHCP network -configure_dhcp_network() { - for pidfile in `ls /var/run/udhcpc*.pid`; do - kill `cat $pidfile` - done - - # NOTE(TheJulia): We may need to add a short wait here as - # network interface plugging actions may not be asynchronous. - echo "Sleeping 30 sec as network interface is being updated" - sleep 30 - INTERFACES=$(ip -o link |grep "LOWER_UP"|cut -f2 -d" "|sed 's/://'|grep -v "lo") - for interface in $INTERFACES; do - pidfile="/var/run/udhcpc/${interface}.pid" - /sbin/udhcpc -b -p ${pidfile} -i ${interface} -s /opt/udhcpc.script >> /var/log/udhcpc.log 2>&1 - done - echo "Completed DHCP client restart" - echo "Outputting IP and Route information" - ip addr || true - ip route || true - ip -6 route || true - echo "Logging IPv4 sysctls" - sysctl -a |grep ipv4 || true - echo "Logging IPv6 sysctls" - sysctl -a |grep ipv6 || true -} - -# Configure networking, use custom udhcpc script to handle MTU option -configure_dhcp_network - -mkdir -p /etc/ironic-python-agent.d/ - -if [ -d /sys/firmware/efi ] ; then - echo "Make efivars available" - mount -t efivarfs efivarfs /sys/firmware/efi/efivars -fi - -# Run IPA -echo "Starting Ironic Python Agent:" -date -ironic-python-agent --config-dir /etc/ironic-python-agent.d/ 2>&1 | tee /var/log/ironic-python-agent.log - - -create_rescue_user() { - crypted_pass=$(cat /etc/ipa-rescue-config/ipa-rescue-password) - sudo adduser rescue -D -G root # no useradd - echo "rescue:$crypted_pass" | sudo chpasswd -e - sudo sh -c "echo \"rescue ALL=(ALL) NOPASSWD: ALL\" >> /etc/sudoers" # no suooers.d in tiny core. - - # Restart sshd with allowing password authentication - sudo sed -i -e 's/^PasswordAuthentication no/PasswordAuthentication yes/' /usr/local/etc/ssh/sshd_config - sudo /usr/local/etc/init.d/openssh restart -} - -if [ -f /etc/ipa-rescue-config/ipa-rescue-password ]; then - create_rescue_user || exit 0 - # The network might change during rescue, renew addresses in this case. - configure_dhcp_network || exit 0 -else - echo "IPA has exited. No rescue password file was defined." -fi diff --git a/tinyipa/build_files/buildreqs.lst b/tinyipa/build_files/buildreqs.lst deleted file mode 100644 index afecc81..0000000 --- a/tinyipa/build_files/buildreqs.lst +++ /dev/null @@ -1,26 +0,0 @@ -autoconf.tcz -autogen-dev.tcz -autogen.tcz -automake.tcz -bash.tcz -binutils.tcz -compiletc.tcz -coreutils.tcz -dmidecode.tcz -gdisk.tcz -git.tcz -glib2-dev.tcz -hdparm.tcz -ipv6-netfilter-6.6.8-tinycore64.tcz -libpci-dev.tcz -libtool-dev.tcz -libtool.tcz -ninja.tcz -parted.tcz -pciutils.tcz -pixman-dev.tcz -raid-dm-6.6.8-tinycore64.tcz -scsi-6.6.8-tinycore64.tcz -udev-lib.tcz -util-linux.tcz -zlib_base-dev.tcz diff --git a/tinyipa/build_files/buildreqs_python3.lst b/tinyipa/build_files/buildreqs_python3.lst deleted file mode 100644 index 5d48288..0000000 --- a/tinyipa/build_files/buildreqs_python3.lst +++ /dev/null @@ -1,2 +0,0 @@ -python3.9.tcz -python3.9-dev.tcz diff --git a/tinyipa/build_files/dhcp.sh b/tinyipa/build_files/dhcp.sh deleted file mode 100644 index bd46c38..0000000 --- a/tinyipa/build_files/dhcp.sh +++ /dev/null @@ -1,21 +0,0 @@ -#!/bin/sh -# The DHCP portion is now separated out, in order to not slow the boot down -# only to wait for slow network cards -. /etc/init.d/tc-functions - -# This waits until all devices have registered -/sbin/udevadm settle --timeout=%UDEV_SETTLE_TIMEOUT% - -NETDEVICES="$(awk -F: '/^ *e.*:|^ *p.*:/{print $1}' /proc/net/dev 2>/dev/null)" -echo "$0: Discovered network devices: $NETDEVICES" -for DEVICE in $NETDEVICES; do - ifconfig $DEVICE | grep -q "inet addr" - if [ "$?" != 0 ]; then - echo -e "\nNetwork device $DEVICE detected, DHCP broadcasting for IP." - trap 2 3 11 - /sbin/udhcpc -b -i $DEVICE -x hostname:$(/bin/hostname) -p /var/run/udhcpc.$DEVICE.pid 2>&1 & - trap "" 2 3 11 - sleep 1 - fi -done - diff --git a/tinyipa/build_files/fakeuname b/tinyipa/build_files/fakeuname deleted file mode 100755 index d2032f4..0000000 --- a/tinyipa/build_files/fakeuname +++ /dev/null @@ -1,110 +0,0 @@ -#!/bin/sh - -S="Linux" -N="box" -R="6.6.8-tinycore64" -P="unknown" -V="#2017 SMP" -M="x86_64" -I="unknown" -O="GNU/Linux" - -OPT_A=false -OPT_S=false -OPT_N=false -OPT_R=false -OPT_P=false -OPT_V=false -OPT_M=false -OPT_I=false -OPT_O=false - -if [ -z "$1" ]; then - echo "-ASNRPVMIO" - exit 1 -fi - -while :; do - case $1 in - -a) - OPT_A=true - shift - ;; - -s) - OPT_S=true - shift - ;; - -n) - OPT_N=true - shift - ;; - -r) - OPT_R=true - shift - ;; - -p) - OPT_P=true - shift - ;; - -v) - OPT_V=true - shift - ;; - -m) - OPT_M=true - shift - ;; - -i) - OPT_I=true - shift - ;; - -o) - OPT_O=true - shift - ;; - -rs) - OPT_R=true - OPT_S=true - shift - ;; - *) - if [ ! -z "$1" ]; then - echo "uname -asnrpvmio" - exit 1 - fi - break - ;; - esac -done - -if $OPT_A; then - echo "$S $N $R $V $M $O" - exit 0 -fi - -string='' -if $OPT_S; then - string="$string $S" -fi -if $OPT_N; then - string="$string $N" -fi -if $OPT_R; then - string="$string $R" -fi -if $OPT_P; then - string="$string $P" -fi -if $OPT_V; then - string="$string $V" -fi -if $OPT_M; then - string="$string $M" -fi -if $OPT_I; then - string="$string $I" -fi -if $OPT_O; then - string="$string $O" -fi -echo $string diff --git a/tinyipa/build_files/finalreqs.lst b/tinyipa/build_files/finalreqs.lst deleted file mode 100644 index 6617799..0000000 --- a/tinyipa/build_files/finalreqs.lst +++ /dev/null @@ -1,25 +0,0 @@ -bash.tcz -coreutils.tcz -dmidecode.tcz -dosfstools.tcz -efibootmgr.tcz -efivar.tcz -filesystems-6.6.8-tinycore64.tcz -file.tcz -gdisk.tcz -glib2.tcz -haveged.tcz -hdparm.tcz -iproute2.tcz -ipv6-netfilter-6.6.8-tinycore64.tcz -libxml2.tcz -mdadm.tcz -ntpclient.tcz -parted.tcz -pciutils.tcz -popt.tcz -raid-dm-6.6.8-tinycore64.tcz -scsi-6.6.8-tinycore64.tcz -smartmontools.tcz -udev-lib.tcz -util-linux.tcz diff --git a/tinyipa/build_files/finalreqs_python3.lst b/tinyipa/build_files/finalreqs_python3.lst deleted file mode 100644 index fab523d..0000000 --- a/tinyipa/build_files/finalreqs_python3.lst +++ /dev/null @@ -1,2 +0,0 @@ -python3.9.tcz -expat2.tcz diff --git a/tinyipa/build_files/isolinux.cfg b/tinyipa/build_files/isolinux.cfg deleted file mode 100755 index 200a07c..0000000 --- a/tinyipa/build_files/isolinux.cfg +++ /dev/null @@ -1,9 +0,0 @@ -display boot.msg -default corepure64 - -label corepure64 - kernel /boot/vmlinuz64 - initrd /boot/corepure64.gz - append loglevel=3 syslog showapps boot_method=vmedia - -implicit 0 diff --git a/tinyipa/build_files/modprobe.conf b/tinyipa/build_files/modprobe.conf deleted file mode 100644 index 13ac34b..0000000 --- a/tinyipa/build_files/modprobe.conf +++ /dev/null @@ -1 +0,0 @@ -ipv6 diff --git a/tinyipa/build_files/ntpdate b/tinyipa/build_files/ntpdate deleted file mode 100755 index cf6eb16..0000000 --- a/tinyipa/build_files/ntpdate +++ /dev/null @@ -1,6 +0,0 @@ -#!/bin/sh -if [ "$1" = "-v" ]; then - echo "Magical ntpdate Wrapper 0.0.1" - exit 0 -fi -ntpclient -h $1 -s diff --git a/tinyipa/common.sh b/tinyipa/common.sh deleted file mode 100755 index f2e3c9d..0000000 --- a/tinyipa/common.sh +++ /dev/null @@ -1,66 +0,0 @@ -#!/bin/bash - -WORKDIR=$(readlink -f $0 | xargs dirname) -source ${WORKDIR}/tc-mirror.sh - -BUILDDIR="$WORKDIR/tinyipabuild" - -PIP_VERSION="21.3.1" - -TINYIPA_REQUIRE_BIOSDEVNAME=${TINYIPA_REQUIRE_BIOSDEVNAME:-false} -TINYIPA_REQUIRE_IPMITOOL=${TINYIPA_REQUIRE_IPMITOOL:-true} - -# PYTHON_EXTRA_SOURCES_DIR_LIST is a csv list of python package dirs to include -PYTHON_EXTRA_SOURCES_DIR_LIST=${PYTHON_EXTRA_SOURCES_DIR_LIST:-} - -# Allow an extension to be added to the generated files by specifying -# $BRANCH_PATH e.g. export BRANCH_PATH=master results in tinyipa-master.gz etc -BRANCH_EXT='' -if [ -n "$BRANCH_PATH" ]; then - BRANCH_EXT="-$BRANCH_PATH" -fi -export BRANCH_EXT - -TC=1001 -STAFF=50 - -CHROOT_PATH="/tmp/overrides:/usr/local/sbin:/usr/local/bin:/apps/bin:/usr/sbin:/usr/bin:/sbin:/bin" -CHROOT_CMD="sudo chroot $DST_DIR /usr/bin/env -i PATH=$CHROOT_PATH http_proxy=$http_proxy https_proxy=$https_proxy no_proxy=$no_proxy" -TC_CHROOT_CMD="sudo chroot --userspec=$TC:$STAFF $DST_DIR /usr/bin/env -i PATH=$CHROOT_PATH http_proxy=$http_proxy https_proxy=$https_proxy no_proxy=$no_proxy" - -function setup_tce { - # Setup resolv.conf, add mirrors, mount proc - local dst_dir="$1" - - # Find a working TC mirror if none is explicitly provided - choose_tc_mirror - - sudo cp $dst_dir/etc/resolv.conf $dst_dir/etc/resolv.conf.old - sudo cp /etc/resolv.conf $dst_dir/etc/resolv.conf - - sudo cp -a $dst_dir/opt/tcemirror $dst_dir/opt/tcemirror.old - sudo sh -c "echo $TINYCORE_MIRROR_URL > $dst_dir/opt/tcemirror" - - mkdir -p $dst_dir/tmp/builtin/optional - $CHROOT_CMD chown -R tc.staff /tmp/builtin - $CHROOT_CMD chmod -R a+w /tmp/builtin - $CHROOT_CMD ln -sf /tmp/builtin /etc/sysconfig/tcedir - echo "tc" | $CHROOT_CMD tee -a /etc/sysconfig/tcuser - - # Mount /proc for chroot commands - sudo mount --bind /proc $dst_dir/proc -} - -function cleanup_tce { - local dst_dir="$1" - - # Unmount /proc and clean up everything - sudo umount $dst_dir/proc - sudo rm -rf $dst_dir/tmp/builtin - sudo rm -rf $dst_dir/tmp/tcloop - sudo rm -rf $dst_dir/usr/local/tce.installed - sudo mv $dst_dir/opt/tcemirror.old $dst_dir/opt/tcemirror - sudo mv $dst_dir/etc/resolv.conf.old $dst_dir/etc/resolv.conf - sudo rm $dst_dir/etc/sysconfig/tcuser - sudo rm $dst_dir/etc/sysconfig/tcedir -} diff --git a/tinyipa/finalise-tinyipa.sh b/tinyipa/finalise-tinyipa.sh deleted file mode 100755 index 61b322c..0000000 --- a/tinyipa/finalise-tinyipa.sh +++ /dev/null @@ -1,249 +0,0 @@ -#!/bin/bash - -set -ex -WORKDIR=$(readlink -f $0 | xargs dirname) -FINALDIR="$WORKDIR/tinyipafinal" -DST_DIR=$FINALDIR -source ${WORKDIR}/common.sh - -BUILD_AND_INSTALL_TINYIPA=${BUILD_AND_INSTALL_TINYIPA:-true} -INSTALL_SSH=${INSTALL_SSH:-true} -AUTHORIZE_SSH=${AUTHORIZE_SSH:-false} - -SSH_PUBLIC_KEY=${SSH_PUBLIC_KEY:-} -PYOPTIMIZE_TINYIPA=${PYOPTIMIZE_TINYIPA:-false} -TINYIPA_UDEV_SETTLE_TIMEOUT=${TINYIPA_UDEV_SETTLE_TIMEOUT:-60} - -echo "Finalising tinyipa:" - -if [ -n "$PYTHON_EXTRA_SOURCES_DIR_LIST" ]; then - IFS="," read -ra PKGDIRS <<< "$PYTHON_EXTRA_SOURCES_DIR_LIST" - for PKGDIR in "${PKGDIRS[@]}"; do - PKG=$(cd "$PKGDIR" ; python setup.py --name) - done -fi - -if $AUTHORIZE_SSH ; then - echo "Validating location of public SSH key" - if [ -n "$SSH_PUBLIC_KEY" ]; then - if [ -f "$SSH_PUBLIC_KEY" ]; then - _found_ssh_key="$SSH_PUBLIC_KEY" - fi - else - for fmt in rsa dsa; do - if [ -f "$HOME/.ssh/id_$fmt.pub" ]; then - _found_ssh_key="$HOME/.ssh/id_$fmt.pub" - break - fi - done - fi - - if [ -z $_found_ssh_key ]; then - echo "Failed to find neither provided nor default SSH key" - exit 1 - fi -fi - -sudo -v - -# Let's umount proc in case the old finalise process went sideways and -# it's still mounted -if grep -qs "$FINALDIR/proc" /proc/mounts; then - sudo umount "$FINALDIR/proc" -fi - -# Remove the old final chroot dir with all its content before starting a new -# finalise process -if [ -d "$FINALDIR" ]; then - sudo rm -rf "$FINALDIR" -fi - -mkdir "$FINALDIR" - -# Extract rootfs from .gz file -( cd "$FINALDIR" && zcat $WORKDIR/build_files/corepure64.gz | sudo cpio -i -H newc -d ) - -# Setup Final Dir -setup_tce "$DST_DIR" - -# Modify ldconfig for x86-64 -$CHROOT_CMD cp /sbin/ldconfig /sbin/ldconfigold -printf '#!/bin/sh\n/sbin/ldconfigold $@ | sed -r "s/libc6|ELF/libc6,x86-64/"' | $CHROOT_CMD tee -a /sbin/ldconfignew -$CHROOT_CMD cp /sbin/ldconfignew /sbin/ldconfig -$CHROOT_CMD chmod u+x /sbin/ldconfig - -# Copy python wheels from build to final dir -cp -Rp "$BUILDDIR/tmp/wheels" "$FINALDIR/tmp/wheelhouse" - -cp $WORKDIR/build_files/qemu-utils.* $FINALDIR/tmp/builtin/optional -cp $WORKDIR/build_files/lshw.* $FINALDIR/tmp/builtin/optional - -if $TINYIPA_REQUIRE_BIOSDEVNAME; then - cp $WORKDIR/build_files/biosdevname.* $FINALDIR/tmp/builtin/optional -fi -if $TINYIPA_REQUIRE_IPMITOOL; then - cp $WORKDIR/build_files/ipmitool.* $FINALDIR/tmp/builtin/optional -fi - -mkdir $FINALDIR/tmp/overrides -cp $WORKDIR/build_files/fakeuname $FINALDIR/tmp/overrides/uname - -sudo cp $WORKDIR/build_files/ntpdate $FINALDIR/bin/ntpdate -sudo chmod 755 $FINALDIR/bin/ntpdate -PY_REQS="finalreqs_python3.lst" - -# NOTE(rpittau) change ownership of the tce info dir to prevent writing issues -sudo chown $TC:$STAFF $FINALDIR/usr/local/tce.installed - -while read line; do - $TC_CHROOT_CMD tce-load -wic $line -done < <(paste $WORKDIR/build_files/finalreqs.lst $WORKDIR/build_files/$PY_REQS) - -if $INSTALL_SSH ; then - # Install and configure bare minimum for SSH access - $TC_CHROOT_CMD tce-load -wic openssh - # Configure OpenSSH - $CHROOT_CMD cp /usr/local/etc/ssh/sshd_config.orig /usr/local/etc/ssh/sshd_config - echo "PasswordAuthentication no" | $CHROOT_CMD tee -a /usr/local/etc/ssh/sshd_config - # Generate and configure host keys - RSA, DSA, Ed25519 - # NOTE(pas-ha) ECDSA host key will still be re-generated fresh on every image boot - $CHROOT_CMD ssh-keygen -t rsa -N "" -f /usr/local/etc/ssh/ssh_host_rsa_key - $CHROOT_CMD ssh-keygen -t dsa -N "" -f /usr/local/etc/ssh/ssh_host_dsa_key - $CHROOT_CMD ssh-keygen -t ed25519 -N "" -f /usr/local/etc/ssh/ssh_host_ed25519_key - echo "HostKey /usr/local/etc/ssh/ssh_host_rsa_key" | $CHROOT_CMD tee -a /usr/local/etc/ssh/sshd_config - echo "HostKey /usr/local/etc/ssh/ssh_host_dsa_key" | $CHROOT_CMD tee -a /usr/local/etc/ssh/sshd_config - echo "HostKey /usr/local/etc/ssh/ssh_host_ed25519_key" | $CHROOT_CMD tee -a /usr/local/etc/ssh/sshd_config - - # setup user and SSH keys - if $AUTHORIZE_SSH; then - $CHROOT_CMD mkdir -p /home/tc - $CHROOT_CMD chown -R tc.staff /home/tc - $TC_CHROOT_CMD mkdir -p /home/tc/.ssh - cat $_found_ssh_key | $TC_CHROOT_CMD tee /home/tc/.ssh/authorized_keys - $CHROOT_CMD chown tc.staff /home/tc/.ssh/authorized_keys - $TC_CHROOT_CMD chmod 600 /home/tc/.ssh/authorized_keys - fi -fi - -$TC_CHROOT_CMD tce-load -ic /tmp/builtin/optional/qemu-utils.tcz -$TC_CHROOT_CMD tce-load -ic /tmp/builtin/optional/lshw.tcz -if $TINYIPA_REQUIRE_BIOSDEVNAME; then - $TC_CHROOT_CMD tce-load -ic /tmp/builtin/optional/biosdevname.tcz -fi -if $TINYIPA_REQUIRE_IPMITOOL; then - $TC_CHROOT_CMD tce-load -ic /tmp/builtin/optional/ipmitool.tcz -fi - -# Ensure tinyipa picks up installed kernel modules -$CHROOT_CMD depmod -a `$WORKDIR/build_files/fakeuname -r` - -PIP_COMMAND="pip3" -TINYIPA_PYTHON_EXE="python3" - -# Install pip -# NOTE(rpittau): pip MUST be the same version used in the build script or -# dragons will appear and put everything on fire -$CHROOT_CMD ${TINYIPA_PYTHON_EXE} -m ensurepip -$CHROOT_CMD ${PIP_COMMAND} install --upgrade pip==${PIP_VERSION} wheel - -# If flag is set install python now -if $BUILD_AND_INSTALL_TINYIPA ; then - if [ -n "$PYTHON_EXTRA_SOURCES_DIR_LIST" ]; then - IFS="," read -ra PKGDIRS <<< "$PYTHON_EXTRA_SOURCES_DIR_LIST" - for PKGDIR in "${PKGDIRS[@]}"; do - PKG=$(cd "$PKGDIR" ; python setup.py --name) - $CHROOT_CMD $PIP_COMMAND install --no-index --find-links=file:///tmp/wheelhouse --pre $PKG - done - fi - - $CHROOT_CMD $PIP_COMMAND install --no-index --find-links=file:///tmp/wheelhouse --pre ironic_python_agent - - rm -rf $FINALDIR/tmp/wheelhouse -fi - -# Unmount /proc and clean up everything -cleanup_tce "$DST_DIR" - -# Copy bootlocal.sh to opt -sudo cp "$WORKDIR/build_files/bootlocal.sh" "$FINALDIR/opt/." - -# Copy udhcpc.script to opt -sudo cp "$WORKDIR/udhcpc.script" "$FINALDIR/opt/" - -# Replace etc/init.d/dhcp.sh -sudo cp "$WORKDIR/build_files/dhcp.sh" "$FINALDIR/etc/init.d/dhcp.sh" -sudo sed -i "s/%UDEV_SETTLE_TIMEOUT%/$TINYIPA_UDEV_SETTLE_TIMEOUT/" "$FINALDIR/etc/init.d/dhcp.sh" - -# Disable ZSwap -sudo sed -i '/# Main/a NOZSWAP=1' "$FINALDIR/etc/init.d/tc-config" -# sudo cp $WORKDIR/build_files/tc-config $FINALDIR/etc/init.d/tc-config - -# Place ipv6 modprobe config so the kernel support loads. -sudo cp "$WORKDIR/build_files/modprobe.conf" "$FINALDIR/etc/modproble.conf" - -# NOTE(rpittau): workaround for hwclock -# The adjtime file used by hwclock in tinycore is /var/lib/hwclock/adjtime -# but for some reason (bug?) the file is not created when hwclock is -# invoked, causing hwclock to fail when using certain options, for example -# --systohc. -# We create the dir and the file to prevent that. -$CHROOT_CMD mkdir -p /var/lib/hwclock -$CHROOT_CMD touch /var/lib/hwclock/adjtime -$CHROOT_CMD chmod 640 /var/lib/hwclock/adjtime - -if $PYOPTIMIZE_TINYIPA; then - echo "WARNING: Precompilation is not compatible with oslo.privsep and is being ignored." -fi - -# Delete unnecessary Babel .dat files -find $FINALDIR -path "*babel/locale-data/*.dat" -not -path "*en_US*" | sudo xargs --no-run-if-empty rm - -# NOTE(pas-ha) Apparently on TinyCore Ansible's 'command' module is -# not searching for executables in the '/usr/local/(s)bin' paths. -# Thus we symlink everything from there to '/usr/(s)bin' which is being searched, -# so that 'command' module picks full utilities installed by 'util-linux' -# instead of built-in simplified BusyBox ones. -set +x -echo "Symlink all from /usr/local/sbin to /usr/sbin" -pushd "$FINALDIR/usr/local/sbin" -for target in *; do - if [ ! -f "$FINALDIR/usr/sbin/$target" ]; then - $CHROOT_CMD ln -sf "/usr/local/sbin/$target" "/usr/sbin/$target" - fi -done -popd -echo "Symlink all from /usr/local/bin to /usr/bin" -# this also includes symlinking Python to the place expected by Ansible -pushd "$FINALDIR/usr/local/bin" -for target in *; do - if [ ! -f "$FINALDIR/usr/bin/$target" ]; then - $CHROOT_CMD ln -sf "/usr/local/bin/$target" "/usr/bin/$target" - fi -done -popd -# symlink bash to sh if /bin/sh is not there -if [ ! -f "$FINALDIR/bin/sh" ]; then - $CHROOT_CMD ln -sf "/bin/bash" "/bin/sh" -fi -set -x - -# Rebuild build directory into gz file -( cd "$FINALDIR" && sudo find | sudo cpio -o -H newc | gzip -9 > "$WORKDIR/tinyipa${BRANCH_EXT}.gz" ) - -# Copy vmlinuz to new name -cp "$WORKDIR/build_files/vmlinuz64" "$WORKDIR/tinyipa${BRANCH_EXT}.vmlinuz" - -# Create tar.gz containing tinyipa files -tar czf tinyipa${BRANCH_EXT}.tar.gz tinyipa${BRANCH_EXT}.gz tinyipa${BRANCH_EXT}.vmlinuz - -# Create sha256 files which will be uploaded by the publish jobs along with -# the tinyipa ones in order to provide a way to verify the integrity of the tinyipa -# builds. -for f in tinyipa${BRANCH_EXT}.{gz,tar.gz,vmlinuz}; do - sha256sum $f > $f.sha256 -done - -# Output files with sizes created by this script -echo "Produced files:" -du -h tinyipa${BRANCH_EXT}.gz tinyipa${BRANCH_EXT}.tar.gz tinyipa${BRANCH_EXT}.vmlinuz -echo "Checksums: " tinyipa${BRANCH_EXT}.*sha256 diff --git a/tinyipa/generate_tox_constraints.sh b/tinyipa/generate_tox_constraints.sh deleted file mode 100755 index a109f99..0000000 --- a/tinyipa/generate_tox_constraints.sh +++ /dev/null @@ -1,96 +0,0 @@ -#!/bin/bash -eu - -SCRIPT_NAME=$(basename $0) -COMMON_ROOT=$(dirname $0) -DESTINATION="$1" -TOX_INI=${2:-tox.ini} - -copy() { - local src=$1 - local destination=$2 - - if test -z "${src}"; then - return 1 - fi - - if test -e "${src}"; then - log "File '${src}' exists. Using as upper-constraints." - cp "${src}" "${destination}" - else - log "File '${src}' not found. Skipping local file strategy." - return 1 - fi - return 0 -} - -download() { - local url=$1 - local destination=$2 - - if test -z "${url}"; then - return 1 - else - log "Downloading from '${url}'" - curl -L ${url} -o "${destination}" - fi - return 0 -} - -log() { - echo "${SCRIPT_NAME}: ${@}" -} - -fail() { - log ${@} - exit 1 -} - -tox_constraints_is_not_null() { - test "${TOX_CONSTRAINTS_FILE:-""}" != "" -} - -copy_uc() { - copy "${TOX_CONSTRAINTS_FILE:-""}" "${DESTINATION}" -} - -download_uc() { - download "${TOX_CONSTRAINTS_FILE:-""}" "${DESTINATION}" -} - -copy_new_requirements_uc() { - if [ -e "/opt/stack/new/requirements" ]; then - copy "/opt/stack/new/requirements/upper-constraints.txt" "${DESTINATION}" - elif [ -e "/opt/stack/requirements" ]; then - copy "/opt/stack/requirements/upper-constraints.txt" "${DESTINATION}" - else - log "No local requirements repository, will download upper-constraints" - # Allow the caller to handle the failure - return 1 - fi -} - -download_from_tox_ini_url() { - local url - # NOTE(mmitchell): This extracts the URL defined as the default value for - # TOX_CONSTRAINTS_FILE in tox.ini. This is used by image - # builders to avoid duplicating the default value in multiple - # scripts. This is specially done to leverage the release - # tools that automatically update the tox.ini when projects - # are released. - url=$(sed -n 's/^.*{env:TOX_CONSTRAINTS_FILE\:\([^}]*\)}.*$/\1/p' $TOX_INI | head -n1) - log "tox.ini indicates '${url}' as fallback." - download "${url}" "${DESTINATION}" -} - -log "Generating local constraints file..." - -if tox_constraints_is_not_null; then - log "TOX_CONSTRAINTS_FILE is defined as '${TOX_CONSTRAINTS_FILE:-""}'" - copy_uc || download_uc || fail "Failed to copy or download file indicated in TOX_CONSTRAINTS_FILE." -else - log "TOX_CONSTRAINTS_FILE is not defined. Using fallback strategies." - - copy_new_requirements_uc || \ - download_from_tox_ini_url || \ - fail "Failed to download upper-constraints.txt from either CI or tox.ini location." -fi diff --git a/tinyipa/install-deps.sh b/tinyipa/install-deps.sh deleted file mode 100755 index 084986a..0000000 --- a/tinyipa/install-deps.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/bin/bash - -COMMON_PACKAGES="wget unzip sudo gawk" -APT_PACKAGES="${COMMON_PACKAGES} python3-pip squashfs-tools" -YUM_PACKAGES="${APT_PACKAGES}" -ZYPPER_PACKAGES="${COMMON_PACKAGES} python3-pip squashfs" - -echo "Installing dependencies:" - -# first zypper in case zypper-aptitude is installed -if [ -x "/usr/bin/zypper" ]; then - sudo -E zypper -n install -l ${ZYPPER_PACKAGES} -elif [ -x "/usr/bin/apt-get" ]; then - sudo -E apt-get update - sudo -E apt-get install -y ${APT_PACKAGES} -elif [ -x "/usr/bin/dnf" ]; then - sudo -E dnf install -y ${YUM_PACKAGES} -elif [ -x "/usr/bin/yum" ]; then - sudo -E yum install -y ${YUM_PACKAGES} -else - echo "No supported package manager installed on system. Supported: apt, yum, dnf, zypper" - exit 1 -fi diff --git a/tinyipa/tc-mirror.sh b/tinyipa/tc-mirror.sh deleted file mode 100644 index 185f74b..0000000 --- a/tinyipa/tc-mirror.sh +++ /dev/null @@ -1,59 +0,0 @@ - -#NOTE(pas-ha) -# The first URL is the official TC repo, -# the rest of the list is taken from -# http://wiki.tinycorelinux.net/wiki:mirrors -# as of time of this writing. -# Only HTTP mirrors were considered with the following ordering -# - those that were unavailable are moved to the bottom of the list -# - those that already responded with 404 are moved to the very bottom - -# List Updated on 9-Dec-2019 -TC_MIRRORS="http://repo.tinycorelinux.net -http://mirror.cedia.org.ec/tinycorelinux -http://mirror.epn.edu.ec/tinycorelinux -http://ftp.vim.org/os/Linux/distr/tinycorelinux -http://mirrors.163.com/tinycorelinux -" - -TINYCORE_MIRROR_URL=${TINYCORE_MIRROR_URL:-} - -# NOTE(TheJulia): Removed mirrors because they are out -# of date -# http://distro.ibiblio.org/tinycorelinux ~1.5 months out of sync. -# http://mirrors.163.com/tinycorelinux - Two weeks out of date -# ftp.vim.org and ftp.nluug.nl are the same host. -# http://www.gtlib.gatech.edu/pub/tinycore - No longer mirrors tinycore -# http://l4u-00.ninr.ru no longer mirrors. -# http://kambing.ui.ac.id/tinycorelinux - Stopped mirroring at 9.x -# http://tinycore.mirror.uber.com.au - Unreachable? -function probe_url { - wget -q --spider --tries 1 --timeout 10 "$1" 2>&1 -} - -function choose_tc_mirror { - if [ -z ${TINYCORE_MIRROR_URL} ]; then - for url in ${TC_MIRRORS}; do - echo "Checking Tiny Core Linux mirror ${url}" - if probe_url ${url} ; then - echo "Check succeeded: ${url} is responding." - TINYCORE_MIRROR_URL=${url} - break - else - echo "Check failed: ${url} is not responding" - fi - done - if [ -z ${TINYCORE_MIRROR_URL} ]; then - echo "Failed to find working Tiny Core Linux mirror" - exit 1 - fi - else - echo "Probing provided Tiny Core Linux mirror ${TINYCORE_MIRROR_URL}" - if probe_url ${TINYCORE_MIRROR_URL} ; then - echo "Check succeeded: ${TINYCORE_MIRROR_URL} is responding." - else - echo "Check failed: ${TINYCORE_MIRROR_URL} is not responding" - exit 1 - fi - fi -} diff --git a/tinyipa/udhcpc.script b/tinyipa/udhcpc.script deleted file mode 100755 index 274d782..0000000 --- a/tinyipa/udhcpc.script +++ /dev/null @@ -1,45 +0,0 @@ -#!/bin/sh - -# udhcpc script edited by Tim Riker - -# file created to be used for static network configuration as well - -[ -z "$1" ] && echo "Error: should be called from udhcpc" && exit 1 - -RESOLV_CONF="/etc/resolv.conf" -[ -n "$broadcast" ] && BROADCAST="broadcast $broadcast" -[ -n "$subnet" ] && NETMASK="netmask $subnet" - -case "$1" in - deconfig) - /sbin/ifconfig $interface 0.0.0.0 - ;; - - renew|bound) - /sbin/ifconfig $interface up - /sbin/ifconfig $interface $ip $BROADCAST $NETMASK - - if [ -n "$router" ] ; then - echo "deleting routers" - while route del default gw 0.0.0.0 dev $interface ; do - : - done - - metric=0 - for i in $router ; do - route add default gw $i dev $interface metric $((metric++)) - done - fi - - echo -n > $RESOLV_CONF - [ -n "$domain" ] && echo search $domain >> $RESOLV_CONF - for i in $dns ; do - echo adding dns $i - echo nameserver $i >> $RESOLV_CONF - done - [ -n "$mtu" ] && /sbin/ifconfig $interface mtu $mtu - ;; -esac - -exit 0 - diff --git a/tox.ini b/tox.ini index c3e97f1..df6ca07 100644 --- a/tox.ini +++ b/tox.ini @@ -42,7 +42,7 @@ commands = sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html [flake8] -exclude = .venv,.git,.tox,dist,doc,*lib/python*,*egg,build,tools,imagebuild/tinyipa/tinyipafinal,imagebuild/tinyipa/tinyipabuild +exclude = .venv,.git,.tox,dist,doc,*lib/python*,*egg,build,tools import-order-style = pep8 application-import-names = ironic_python_agent_builder # [H106] Don't put vim configuration in source files.