Merge "docs: clean up SAIO formatting"

This commit is contained in:
Zuul 2019-03-05 23:57:37 +00:00 committed by Gerrit Code Review
commit f9329ca5ff
3 changed files with 332 additions and 291 deletions

View File

@ -76,9 +76,10 @@ Installing dependencies
python2-netifaces python2-pip python2-dnspython \
python2-mock
Note: This installs necessary system dependencies and *most* of the python
dependencies. Later in the process setuptools/distribute or pip will install
and/or upgrade packages.
.. note::
This installs necessary system dependencies and *most* of the python
dependencies. Later in the process setuptools/distribute or pip will install
and/or upgrade packages.
Next, choose either :ref:`partition-section` or :ref:`loopback-section`.
@ -90,50 +91,52 @@ Using a partition for storage
If you are going to use a separate partition for Swift data, be sure to add
another device when creating the VM, and follow these instructions:
#. Set up a single partition::
#. Set up a single partition::
sudo fdisk /dev/sdb
sudo mkfs.xfs /dev/sdb1
sudo fdisk /dev/sdb
sudo mkfs.xfs /dev/sdb1
#. Edit ``/etc/fstab`` and add::
#. Edit ``/etc/fstab`` and add::
/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
#. Create the mount point and the individualized links::
#. Create the mount point and the individualized links::
sudo mkdir /mnt/sdb1
sudo mount /mnt/sdb1
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
sudo chown ${USER}:${USER} /mnt/sdb1/*
sudo mkdir /srv
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done
sudo mkdir -p /srv/1/node/sdb1 /srv/1/node/sdb5 \
/srv/2/node/sdb2 /srv/2/node/sdb6 \
/srv/3/node/sdb3 /srv/3/node/sdb7 \
/srv/4/node/sdb4 /srv/4/node/sdb8 \
/var/run/swift
sudo chown -R ${USER}:${USER} /var/run/swift
# **Make sure to include the trailing slash after /srv/$x/**
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
sudo mkdir /mnt/sdb1
sudo mount /mnt/sdb1
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
sudo chown ${USER}:${USER} /mnt/sdb1/*
sudo mkdir /srv
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done
sudo mkdir -p /srv/1/node/sdb1 /srv/1/node/sdb5 \
/srv/2/node/sdb2 /srv/2/node/sdb6 \
/srv/3/node/sdb3 /srv/3/node/sdb7 \
/srv/4/node/sdb4 /srv/4/node/sdb8 \
/var/run/swift
sudo chown -R ${USER}:${USER} /var/run/swift
# **Make sure to include the trailing slash after /srv/$x/**
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
Note: For OpenSuse users, a user's primary group is `users`, so you have 2 options:
.. note::
For OpenSuse users, a user's primary group is ``users``, so you have 2 options:
* Change `${USER}:${USER}` to `${USER}:users` in all references of this guide; or
* Create a group for your username and add yourself to it::
* Change ``${USER}:${USER}`` to ``${USER}:users`` in all references of this guide; or
* Create a group for your username and add yourself to it::
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
Note: We create the mount points and mount the storage disk under
/mnt/sdb1. This disk will contain one directory per simulated swift node,
each owned by the current swift user.
.. note::
We create the mount points and mount the storage disk under
/mnt/sdb1. This disk will contain one directory per simulated swift node,
each owned by the current swift user.
We then create symlinks to these directories under /srv.
If the disk sdb is unmounted, files will not be written under
/srv/\*, because the symbolic link destination /mnt/sdb1/* will not
exist. This prevents disk sync operations from writing to the root
partition in the event a drive is unmounted.
We then create symlinks to these directories under /srv.
If the disk sdb is unmounted, files will not be written under
/srv/\*, because the symbolic link destination /mnt/sdb1/* will not
exist. This prevents disk sync operations from writing to the root
partition in the event a drive is unmounted.
#. Next, skip to :ref:`common-dev-section`.
#. Next, skip to :ref:`common-dev-section`.
.. _loopback-section:
@ -144,51 +147,53 @@ Using a loopback device for storage
If you want to use a loopback device instead of another partition, follow
these instructions:
#. Create the file for the loopback device::
#. Create the file for the loopback device::
sudo mkdir /srv
sudo truncate -s 1GB /srv/swift-disk
sudo mkfs.xfs /srv/swift-disk
sudo mkdir /srv
sudo truncate -s 1GB /srv/swift-disk
sudo mkfs.xfs /srv/swift-disk
Modify size specified in the ``truncate`` command to make a larger or
smaller partition as needed.
Modify size specified in the ``truncate`` command to make a larger or
smaller partition as needed.
#. Edit `/etc/fstab` and add::
#. Edit `/etc/fstab` and add::
/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
#. Create the mount point and the individualized links::
#. Create the mount point and the individualized links::
sudo mkdir /mnt/sdb1
sudo mount /mnt/sdb1
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
sudo chown ${USER}:${USER} /mnt/sdb1/*
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done
sudo mkdir -p /srv/1/node/sdb1 /srv/1/node/sdb5 \
/srv/2/node/sdb2 /srv/2/node/sdb6 \
/srv/3/node/sdb3 /srv/3/node/sdb7 \
/srv/4/node/sdb4 /srv/4/node/sdb8 \
/var/run/swift
sudo chown -R ${USER}:${USER} /var/run/swift
# **Make sure to include the trailing slash after /srv/$x/**
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
sudo mkdir /mnt/sdb1
sudo mount /mnt/sdb1
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
sudo chown ${USER}:${USER} /mnt/sdb1/*
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done
sudo mkdir -p /srv/1/node/sdb1 /srv/1/node/sdb5 \
/srv/2/node/sdb2 /srv/2/node/sdb6 \
/srv/3/node/sdb3 /srv/3/node/sdb7 \
/srv/4/node/sdb4 /srv/4/node/sdb8 \
/var/run/swift
sudo chown -R ${USER}:${USER} /var/run/swift
# **Make sure to include the trailing slash after /srv/$x/**
for x in {1..4}; do sudo chown -R ${USER}:${USER} /srv/$x/; done
Note: For OpenSuse users, a user's primary group is `users`, so you have 2 options:
.. note::
For OpenSuse users, a user's primary group is ``users``, so you have 2 options:
* Change `${USER}:${USER}` to `${USER}:users` in all references of this guide; or
* Create a group for your username and add yourself to it::
* Change ``${USER}:${USER}`` to ``${USER}:users`` in all references of this guide; or
* Create a group for your username and add yourself to it::
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
sudo groupadd ${USER} && sudo gpasswd -a ${USER} ${USER}
Note: We create the mount points and mount the loopback file under
/mnt/sdb1. This file will contain one directory per simulated swift node,
each owned by the current swift user.
.. note::
We create the mount points and mount the loopback file under
/mnt/sdb1. This file will contain one directory per simulated swift node,
each owned by the current swift user.
We then create symlinks to these directories under /srv.
If the loopback file is unmounted, files will not be written under
/srv/\*, because the symbolic link destination /mnt/sdb1/* will not
exist. This prevents disk sync operations from writing to the root
partition in the event a drive is unmounted.
We then create symlinks to these directories under /srv.
If the loopback file is unmounted, files will not be written under
/srv/\*, because the symbolic link destination /mnt/sdb1/* will not
exist. This prevents disk sync operations from writing to the root
partition in the event a drive is unmounted.
.. _common-dev-section:
@ -229,117 +234,120 @@ To persist this, edit and add the following to ``/etc/fstab``::
Getting the code
----------------
#. Check out the python-swiftclient repo::
#. Check out the python-swiftclient repo::
cd $HOME; git clone https://github.com/openstack/python-swiftclient.git
cd $HOME; git clone https://github.com/openstack/python-swiftclient.git
#. Build a development installation of python-swiftclient::
#. Build a development installation of python-swiftclient::
cd $HOME/python-swiftclient; sudo python setup.py develop; cd -
cd $HOME/python-swiftclient; sudo python setup.py develop; cd -
Ubuntu 12.04 users need to install python-swiftclient's dependencies before the installation of
python-swiftclient. This is due to a bug in an older version of setup tools::
Ubuntu 12.04 users need to install python-swiftclient's dependencies before the installation of
python-swiftclient. This is due to a bug in an older version of setup tools::
cd $HOME/python-swiftclient; sudo pip install -r requirements.txt; sudo python setup.py develop; cd -
cd $HOME/python-swiftclient; sudo pip install -r requirements.txt; sudo python setup.py develop; cd -
#. Check out the swift repo::
#. Check out the swift repo::
git clone https://github.com/openstack/swift.git
git clone https://github.com/openstack/swift.git
#. Build a development installation of swift::
#. Build a development installation of swift::
cd $HOME/swift; sudo pip install --no-binary cryptography -r requirements.txt; sudo python setup.py develop; cd -
cd $HOME/swift; sudo pip install --no-binary cryptography -r requirements.txt; sudo python setup.py develop; cd -
Note: Due to a difference in libssl.so naming in OpenSuse to other Linux distros the wheel/binary wont work so the
cryptography must be built, thus the ``--no-binary cryptography``.
.. note::
Due to a difference in how ``libssl.so`` is named in OpenSuse vs. other Linux distros the
wheel/binary won't work; thus we use ``--no-binary cryptography`` to build ``cryptography``
locally.
Fedora 19 or later users might have to perform the following if development
installation of swift fails::
Fedora 19 or later users might have to perform the following if development
installation of swift fails::
sudo pip install -U xattr
sudo pip install -U xattr
#. Install swift's test dependencies::
#. Install swift's test dependencies::
cd $HOME/swift; sudo pip install -r test-requirements.txt
cd $HOME/swift; sudo pip install -r test-requirements.txt
----------------
Setting up rsync
----------------
#. Create ``/etc/rsyncd.conf``::
#. Create ``/etc/rsyncd.conf``::
sudo cp $HOME/swift/doc/saio/rsyncd.conf /etc/
sudo sed -i "s/<your-user-name>/${USER}/" /etc/rsyncd.conf
sudo cp $HOME/swift/doc/saio/rsyncd.conf /etc/
sudo sed -i "s/<your-user-name>/${USER}/" /etc/rsyncd.conf
Here is the default ``rsyncd.conf`` file contents maintained in the repo
that is copied and fixed up above:
Here is the default ``rsyncd.conf`` file contents maintained in the repo
that is copied and fixed up above:
.. literalinclude:: /../saio/rsyncd.conf
.. literalinclude:: /../saio/rsyncd.conf
:language: ini
#. On Ubuntu, edit the following line in ``/etc/default/rsync``::
#. On Ubuntu, edit the following line in ``/etc/default/rsync``::
RSYNC_ENABLE=true
RSYNC_ENABLE=true
On Fedora, edit the following line in ``/etc/xinetd.d/rsync``::
On Fedora, edit the following line in ``/etc/xinetd.d/rsync``::
disable = no
disable = no
One might have to create the above files to perform the edits.
One might have to create the above files to perform the edits.
On OpenSuse, nothing needs to happen here.
On OpenSuse, nothing needs to happen here.
#. On platforms with SELinux in ``Enforcing`` mode, either set to ``Permissive``::
#. On platforms with SELinux in ``Enforcing`` mode, either set to ``Permissive``::
sudo setenforce Permissive
sudo setenforce Permissive
Or just allow rsync full access::
Or just allow rsync full access::
sudo setsebool -P rsync_full_access 1
sudo setsebool -P rsync_full_access 1
#. Start the rsync daemon
#. Start the rsync daemon
* On Ubuntu 14.04, run::
* On Ubuntu 14.04, run::
sudo service rsync restart
sudo service rsync restart
* On Ubuntu 16.04, run::
* On Ubuntu 16.04, run::
sudo systemctl enable rsync
sudo systemctl start rsync
sudo systemctl enable rsync
sudo systemctl start rsync
* On Fedora, run::
* On Fedora, run::
sudo systemctl restart xinetd.service
sudo systemctl enable rsyncd.service
sudo systemctl start rsyncd.service
sudo systemctl restart xinetd.service
sudo systemctl enable rsyncd.service
sudo systemctl start rsyncd.service
* On OpenSuse, run::
* On OpenSuse, run::
sudo systemctl enable rsyncd.service
sudo systemctl start rsyncd.service
sudo systemctl enable rsyncd.service
sudo systemctl start rsyncd.service
* On other xinetd based systems simply run::
* On other xinetd based systems simply run::
sudo service xinetd restart
sudo service xinetd restart
#. Verify rsync is accepting connections for all servers::
#. Verify rsync is accepting connections for all servers::
rsync rsync://pub@localhost/
rsync rsync://pub@localhost/
You should see the following output from the above command::
You should see the following output from the above command::
account6012
account6022
account6032
account6042
container6011
container6021
container6031
container6041
object6010
object6020
object6030
object6040
account6012
account6022
account6032
account6042
container6011
container6021
container6031
container6041
object6010
object6020
object6030
object6040
------------------
Starting memcached
@ -362,50 +370,51 @@ running, tokens cannot be validated, and accessing Swift becomes impossible.
Optional: Setting up rsyslog for individual logging
---------------------------------------------------
#. Install the swift rsyslogd configuration::
#. Install the swift rsyslogd configuration::
sudo cp $HOME/swift/doc/saio/rsyslog.d/10-swift.conf /etc/rsyslog.d/
sudo cp $HOME/swift/doc/saio/rsyslog.d/10-swift.conf /etc/rsyslog.d/
Note: OpenSuse may have the systemd logger installed, so if you want this
to work, you need to install rsyslog::
Note: OpenSuse may have the systemd logger installed, so if you want this
to work, you need to install rsyslog::
sudo zypper install rsyslog
sudo systemctl start rsyslog.service
sudo systemctl enable rsyslog.service
sudo zypper install rsyslog
sudo systemctl start rsyslog.service
sudo systemctl enable rsyslog.service
Be sure to review that conf file to determine if you want all the logs
in one file vs. all the logs separated out, and if you want hourly logs
for stats processing. For convenience, we provide its default contents
below:
Be sure to review that conf file to determine if you want all the logs
in one file vs. all the logs separated out, and if you want hourly logs
for stats processing. For convenience, we provide its default contents
below:
.. literalinclude:: /../saio/rsyslog.d/10-swift.conf
.. literalinclude:: /../saio/rsyslog.d/10-swift.conf
:language: ini
#. Edit ``/etc/rsyslog.conf`` and make the following change (usually in the
"GLOBAL DIRECTIVES" section)::
#. Edit ``/etc/rsyslog.conf`` and make the following change (usually in the
"GLOBAL DIRECTIVES" section)::
$PrivDropToGroup adm
$PrivDropToGroup adm
#. If using hourly logs (see above) perform::
#. If using hourly logs (see above) perform::
sudo mkdir -p /var/log/swift/hourly
sudo mkdir -p /var/log/swift/hourly
Otherwise perform::
Otherwise perform::
sudo mkdir -p /var/log/swift
sudo mkdir -p /var/log/swift
#. Setup the logging directory and start syslog:
#. Setup the logging directory and start syslog:
* On Ubuntu::
* On Ubuntu::
sudo chown -R syslog.adm /var/log/swift
sudo chmod -R g+w /var/log/swift
sudo service rsyslog restart
sudo chown -R syslog.adm /var/log/swift
sudo chmod -R g+w /var/log/swift
sudo service rsyslog restart
* On Fedora and OpenSuse::
* On Fedora and OpenSuse::
sudo chown -R root:adm /var/log/swift
sudo chmod -R g+w /var/log/swift
sudo systemctl restart rsyslog.service
sudo chown -R root:adm /var/log/swift
sudo chmod -R g+w /var/log/swift
sudo systemctl restart rsyslog.service
---------------------
Configuring each node
@ -415,89 +424,106 @@ After performing the following steps, be sure to verify that Swift has access
to resulting configuration files (sample configuration files are provided with
all defaults in line-by-line comments).
#. Optionally remove an existing swift directory::
#. Optionally remove an existing swift directory::
sudo rm -rf /etc/swift
sudo rm -rf /etc/swift
#. Populate the ``/etc/swift`` directory itself::
#. Populate the ``/etc/swift`` directory itself::
cd $HOME/swift/doc; sudo cp -r saio/swift /etc/swift; cd -
sudo chown -R ${USER}:${USER} /etc/swift
cd $HOME/swift/doc; sudo cp -r saio/swift /etc/swift; cd -
sudo chown -R ${USER}:${USER} /etc/swift
#. Update ``<your-user-name>`` references in the Swift config files::
#. Update ``<your-user-name>`` references in the Swift config files::
find /etc/swift/ -name \*.conf | xargs sudo sed -i "s/<your-user-name>/${USER}/"
find /etc/swift/ -name \*.conf | xargs sudo sed -i "s/<your-user-name>/${USER}/"
The contents of the configuration files provided by executing the above
commands are as follows:
#. ``/etc/swift/swift.conf``
#. ``/etc/swift/swift.conf``
.. literalinclude:: /../saio/swift/swift.conf
.. literalinclude:: /../saio/swift/swift.conf
:language: ini
#. ``/etc/swift/proxy-server.conf``
#. ``/etc/swift/proxy-server.conf``
.. literalinclude:: /../saio/swift/proxy-server.conf
.. literalinclude:: /../saio/swift/proxy-server.conf
:language: ini
#. ``/etc/swift/object-expirer.conf``
#. ``/etc/swift/object-expirer.conf``
.. literalinclude:: /../saio/swift/object-expirer.conf
.. literalinclude:: /../saio/swift/object-expirer.conf
:language: ini
#. ``/etc/swift/container-reconciler.conf``
#. ``/etc/swift/container-reconciler.conf``
.. literalinclude:: /../saio/swift/container-reconciler.conf
.. literalinclude:: /../saio/swift/container-reconciler.conf
:language: ini
#. ``/etc/swift/container-sync-realms.conf``
#. ``/etc/swift/container-sync-realms.conf``
.. literalinclude:: /../saio/swift/container-sync-realms.conf
.. literalinclude:: /../saio/swift/container-sync-realms.conf
:language: ini
#. ``/etc/swift/account-server/1.conf``
#. ``/etc/swift/account-server/1.conf``
.. literalinclude:: /../saio/swift/account-server/1.conf
.. literalinclude:: /../saio/swift/account-server/1.conf
:language: ini
#. ``/etc/swift/container-server/1.conf``
#. ``/etc/swift/container-server/1.conf``
.. literalinclude:: /../saio/swift/container-server/1.conf
.. literalinclude:: /../saio/swift/container-server/1.conf
:language: ini
#. ``/etc/swift/object-server/1.conf``
#. ``/etc/swift/object-server/1.conf``
.. literalinclude:: /../saio/swift/object-server/1.conf
.. literalinclude:: /../saio/swift/object-server/1.conf
:language: ini
#. ``/etc/swift/account-server/2.conf``
#. ``/etc/swift/account-server/2.conf``
.. literalinclude:: /../saio/swift/account-server/2.conf
.. literalinclude:: /../saio/swift/account-server/2.conf
:language: ini
#. ``/etc/swift/container-server/2.conf``
#. ``/etc/swift/container-server/2.conf``
.. literalinclude:: /../saio/swift/container-server/2.conf
.. literalinclude:: /../saio/swift/container-server/2.conf
:language: ini
#. ``/etc/swift/object-server/2.conf``
#. ``/etc/swift/object-server/2.conf``
.. literalinclude:: /../saio/swift/object-server/2.conf
.. literalinclude:: /../saio/swift/object-server/2.conf
:language: ini
#. ``/etc/swift/account-server/3.conf``
#. ``/etc/swift/account-server/3.conf``
.. literalinclude:: /../saio/swift/account-server/3.conf
.. literalinclude:: /../saio/swift/account-server/3.conf
:language: ini
#. ``/etc/swift/container-server/3.conf``
#. ``/etc/swift/container-server/3.conf``
.. literalinclude:: /../saio/swift/container-server/3.conf
.. literalinclude:: /../saio/swift/container-server/3.conf
:language: ini
#. ``/etc/swift/object-server/3.conf``
#. ``/etc/swift/object-server/3.conf``
.. literalinclude:: /../saio/swift/object-server/3.conf
.. literalinclude:: /../saio/swift/object-server/3.conf
:language: ini
#. ``/etc/swift/account-server/4.conf``
#. ``/etc/swift/account-server/4.conf``
.. literalinclude:: /../saio/swift/account-server/4.conf
.. literalinclude:: /../saio/swift/account-server/4.conf
:language: ini
#. ``/etc/swift/container-server/4.conf``
#. ``/etc/swift/container-server/4.conf``
.. literalinclude:: /../saio/swift/container-server/4.conf
.. literalinclude:: /../saio/swift/container-server/4.conf
:language: ini
#. ``/etc/swift/object-server/4.conf``
#. ``/etc/swift/object-server/4.conf``
.. literalinclude:: /../saio/swift/object-server/4.conf
.. literalinclude:: /../saio/swift/object-server/4.conf
:language: ini
.. _setup_scripts:
@ -505,139 +531,146 @@ commands are as follows:
Setting up scripts for running Swift
------------------------------------
#. Copy the SAIO scripts for resetting the environment::
#. Copy the SAIO scripts for resetting the environment::
mkdir -p $HOME/bin
cd $HOME/swift/doc; cp saio/bin/* $HOME/bin; cd -
chmod +x $HOME/bin/*
mkdir -p $HOME/bin
cd $HOME/swift/doc; cp saio/bin/* $HOME/bin; cd -
chmod +x $HOME/bin/*
#. Edit the ``$HOME/bin/resetswift`` script
#. Edit the ``$HOME/bin/resetswift`` script
The template ``resetswift`` script looks like the following:
The template ``resetswift`` script looks like the following:
.. literalinclude:: /../saio/bin/resetswift
.. literalinclude:: /../saio/bin/resetswift
:language: bash
If you are using a loopback device add an environment var to
substitute ``/dev/sdb1`` with ``/srv/swift-disk``::
If you are using a loopback device add an environment var to
substitute ``/dev/sdb1`` with ``/srv/swift-disk``::
echo "export SAIO_BLOCK_DEVICE=/srv/swift-disk" >> $HOME/.bashrc
echo "export SAIO_BLOCK_DEVICE=/srv/swift-disk" >> $HOME/.bashrc
If you did not set up rsyslog for individual logging, remove the ``find
/var/log/swift...`` line::
If you did not set up rsyslog for individual logging, remove the ``find
/var/log/swift...`` line::
sed -i "/find \/var\/log\/swift/d" $HOME/bin/resetswift
sed -i "/find \/var\/log\/swift/d" $HOME/bin/resetswift
#. Install the sample configuration file for running tests::
#. Install the sample configuration file for running tests::
cp $HOME/swift/test/sample.conf /etc/swift/test.conf
cp $HOME/swift/test/sample.conf /etc/swift/test.conf
The template ``test.conf`` looks like the following:
The template ``test.conf`` looks like the following:
.. literalinclude:: /../../test/sample.conf
.. literalinclude:: /../../test/sample.conf
:language: ini
#. Add an environment variable for running tests below::
#. Add an environment variable for running tests below::
echo "export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf" >> $HOME/.bashrc
echo "export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf" >> $HOME/.bashrc
#. Be sure that your ``PATH`` includes the ``bin`` directory::
#. Be sure that your ``PATH`` includes the ``bin`` directory::
echo "export PATH=${PATH}:$HOME/bin" >> $HOME/.bashrc
echo "export PATH=${PATH}:$HOME/bin" >> $HOME/.bashrc
#. Source the above environment variables into your current environment::
#. Source the above environment variables into your current environment::
. $HOME/.bashrc
. $HOME/.bashrc
#. Construct the initial rings using the provided script::
#. Construct the initial rings using the provided script::
remakerings
remakerings
The ``remakerings`` script looks like the following:
The ``remakerings`` script looks like the following:
.. literalinclude:: /../saio/bin/remakerings
.. literalinclude:: /../saio/bin/remakerings
:language: bash
You can expect the output from this command to produce the following. Note
that 3 object rings are created in order to test storage policies and EC in
the SAIO environment. The EC ring is the only one with all 8 devices.
There are also two replication rings, one for 3x replication and another
for 2x replication, but those rings only use 4 devices::
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
Device d2r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 2
Device d3r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 3
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
Device d2r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 2
Device d3r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 3
Reassigned 2048 (200.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
Device d1r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb5_"" with 1.0 weight got id 1
Device d2r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 2
Device d3r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb6_"" with 1.0 weight got id 3
Device d4r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 4
Device d5r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb7_"" with 1.0 weight got id 5
Device d6r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 6
Device d7r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb8_"" with 1.0 weight got id 7
Reassigned 6144 (600.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Device d0r1z1-127.0.0.1:6011R127.0.0.1:6011/sdb1_"" with 1.0 weight got id 0
Device d1r1z2-127.0.0.2:6021R127.0.0.2:6021/sdb2_"" with 1.0 weight got id 1
Device d2r1z3-127.0.0.3:6031R127.0.0.3:6031/sdb3_"" with 1.0 weight got id 2
Device d3r1z4-127.0.0.4:6041R127.0.0.4:6041/sdb4_"" with 1.0 weight got id 3
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Device d0r1z1-127.0.0.1:6012R127.0.0.1:6012/sdb1_"" with 1.0 weight got id 0
Device d1r1z2-127.0.0.2:6022R127.0.0.2:6022/sdb2_"" with 1.0 weight got id 1
Device d2r1z3-127.0.0.3:6032R127.0.0.3:6032/sdb3_"" with 1.0 weight got id 2
Device d3r1z4-127.0.0.4:6042R127.0.0.4:6042/sdb4_"" with 1.0 weight got id 3
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
You can expect the output from this command to produce the following. Note
that 3 object rings are created in order to test storage policies and EC in
the SAIO environment. The EC ring is the only one with all 8 devices.
There are also two replication rings, one for 3x replication and another
for 2x replication, but those rings only use 4 devices:
#. Read more about Storage Policies and your SAIO :doc:`policies_saio`
.. code-block:: console
#. Verify the unit tests run::
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
Device d2r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 2
Device d3r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 3
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
Device d1r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 1
Device d2r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 2
Device d3r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 3
Reassigned 2048 (200.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Device d0r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0
Device d1r1z1-127.0.0.1:6010R127.0.0.1:6010/sdb5_"" with 1.0 weight got id 1
Device d2r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb2_"" with 1.0 weight got id 2
Device d3r1z2-127.0.0.2:6020R127.0.0.2:6020/sdb6_"" with 1.0 weight got id 3
Device d4r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb3_"" with 1.0 weight got id 4
Device d5r1z3-127.0.0.3:6030R127.0.0.3:6030/sdb7_"" with 1.0 weight got id 5
Device d6r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb4_"" with 1.0 weight got id 6
Device d7r1z4-127.0.0.4:6040R127.0.0.4:6040/sdb8_"" with 1.0 weight got id 7
Reassigned 6144 (600.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Device d0r1z1-127.0.0.1:6011R127.0.0.1:6011/sdb1_"" with 1.0 weight got id 0
Device d1r1z2-127.0.0.2:6021R127.0.0.2:6021/sdb2_"" with 1.0 weight got id 1
Device d2r1z3-127.0.0.3:6031R127.0.0.3:6031/sdb3_"" with 1.0 weight got id 2
Device d3r1z4-127.0.0.4:6041R127.0.0.4:6041/sdb4_"" with 1.0 weight got id 3
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Device d0r1z1-127.0.0.1:6012R127.0.0.1:6012/sdb1_"" with 1.0 weight got id 0
Device d1r1z2-127.0.0.2:6022R127.0.0.2:6022/sdb2_"" with 1.0 weight got id 1
Device d2r1z3-127.0.0.3:6032R127.0.0.3:6032/sdb3_"" with 1.0 weight got id 2
Device d3r1z4-127.0.0.4:6042R127.0.0.4:6042/sdb4_"" with 1.0 weight got id 3
Reassigned 3072 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
$HOME/swift/.unittests
Note that the unit tests do not require any swift daemons running.
#. Read more about Storage Policies and your SAIO :doc:`policies_saio`
#. Start the "main" Swift daemon processes (proxy, account, container, and
object)::
#. Verify the unit tests run::
startmain
$HOME/swift/.unittests
(The "``Unable to increase file descriptor limit. Running as non-root?``"
warnings are expected and ok.)
Note that the unit tests do not require any swift daemons running.
The ``startmain`` script looks like the following:
#. Start the "main" Swift daemon processes (proxy, account, container, and
object)::
.. literalinclude:: /../saio/bin/startmain
startmain
#. Get an ``X-Storage-Url`` and ``X-Auth-Token``::
(The "``Unable to increase file descriptor limit. Running as non-root?``"
warnings are expected and ok.)
curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
The ``startmain`` script looks like the following:
#. Check that you can ``GET`` account::
.. literalinclude:: /../saio/bin/startmain
:language: bash
curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>
#. Get an ``X-Storage-Url`` and ``X-Auth-Token``::
#. Check that ``swift`` command provided by the python-swiftclient package works::
curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
#. Check that you can ``GET`` account::
#. Verify the functional tests run::
curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>
$HOME/swift/.functests
#. Check that ``swift`` command provided by the python-swiftclient package works::
(Note: functional tests will first delete everything in the configured
accounts.)
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
#. Verify the probe tests run::
#. Verify the functional tests run::
$HOME/swift/.probetests
$HOME/swift/.functests
(Note: probe tests will reset your environment as they call ``resetswift``
for each test.)
(Note: functional tests will first delete everything in the configured
accounts.)
#. Verify the probe tests run::
$HOME/swift/.probetests
(Note: probe tests will reset your environment as they call ``resetswift``
for each test.)
----------------
Debugging Issues

View File

@ -178,6 +178,14 @@ class BufferedHTTPConnection(HTTPConnection):
return ret
def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0):
'''Send a request to the server.
:param method: specifies an HTTP request method, e.g. 'GET'.
:param url: specifies the object being requested, e.g. '/index.html'.
:param skip_host: if True does not add automatically a 'Host:' header
:param skip_accept_encoding: if True does not add automatically an
'Accept-Encoding:' header
'''
self._method = method
self._path = url
return HTTPConnection.putrequest(self, method, url, skip_host,

View File

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from urllib import quote
from six.moves.urllib.parse import quote
from swift.common.utils import public
from swift.common.middleware.s3api.controllers.base import Controller