9c20cca362
qemu-img attempts to launch multiple threads by default *and*
attempts to have multiple memory allocation arenas to operate
from. While multithreading can be good for performance, this
pattern and the memory footprint for process launch and
dependencies can turn the memory footprint for a cirros image
conversion (16MB) into 1.2GB of memory being asked for by the
qemu-img tool.
In order to limit this impact, as the default number of arenas
is governed by the number of CPUs times the number 8, it seems
reasonable to lower this to a more reasonable number which
also helps keep our possible memory footprint from being exceeded.
NOTE: This change is largely different than the original change
as an intermediate change converted write_image.sh to python.
As it is unlikely for us to backport the intermediate change,
it is logical for us to just modify the original script.
Otherwise the release note is ultimately what is backported for
release note tooling continutity.
Change-Id: I71a28ec59ec31c691205eb34d9fcab63a2ccb682
Story: 2008928
Task: 42528
(cherry picked from commit 9e4c7052a2)
65 lines
2.3 KiB
Bash
Executable File
65 lines
2.3 KiB
Bash
Executable File
#!/bin/bash
|
|
|
|
# Copyright 2013 Rackspace, Inc.
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
set -e
|
|
|
|
log() {
|
|
echo "`basename $0`: $@"
|
|
}
|
|
|
|
usage() {
|
|
[[ -z "$1" ]] || echo -e "USAGE ERROR: $@\n"
|
|
echo "`basename $0`: IMAGEFILE DEVICE"
|
|
echo " - This script images DEVICE with IMAGEFILE"
|
|
exit 1
|
|
}
|
|
|
|
IMAGEFILE="$1"
|
|
DEVICE="$2"
|
|
|
|
[[ -f $IMAGEFILE ]] || usage "$IMAGEFILE (IMAGEFILE) is not a file"
|
|
[[ -b $DEVICE ]] || usage "$DEVICE (DEVICE) is not a block device"
|
|
|
|
# In production this will be replaced with secure erasing the drives
|
|
# For now we need to ensure there aren't any old (GPT) partitions on the drive
|
|
log "Erasing existing GPT and MBR data structures from ${DEVICE}"
|
|
|
|
# NOTE(gfidente): GPT uses 33*512 sectors, this is an attempt to avoid bug:
|
|
# https://bugs.launchpad.net/ironic-python-agent/+bug/1737556
|
|
DEVICE_SECTORS_COUNT=`blockdev --getsz $DEVICE`
|
|
dd bs=512 if=/dev/zero of=$DEVICE count=33
|
|
dd bs=512 if=/dev/zero of=$DEVICE count=33 seek=$((${DEVICE_SECTORS_COUNT} - 33))
|
|
sgdisk -Z $DEVICE
|
|
udevadm settle
|
|
|
|
log "Imaging $IMAGEFILE to $DEVICE"
|
|
|
|
# limit the memory usage for qemu-img to 2 GiB
|
|
ulimit -v 2097152
|
|
# NOTE(TheJulia): qemu-img uses multiple threads by default and in
|
|
# cross-thread memory allocation lock conflicts, glibc will ultimately
|
|
# attempt to provide it with an additional arena to allocate from, however
|
|
# the running default, when not overridden is 8 * ncpu * the footprint, which
|
|
# very quickly exceeds the ulimit. This is most observable on CI systems where
|
|
# cross-vcpu thread locking can result in a conflict that wouldn't normally be
|
|
# as likely on physical hardware.
|
|
# See discussion on https://bugzilla.redhat.com/show_bug.cgi?id=1892773
|
|
export MALLOC_ARENA_MAX=3
|
|
qemu-img convert -t directsync -O host_device $IMAGEFILE $DEVICE
|
|
sync
|
|
|
|
log "${DEVICE} imaged successfully!"
|