license boilerplates as docstring are included in automatically
generated documentation which leads to wrong formatting.
For example the indented hyperlink to the license is formatted as
ssh is a storage used for linux destro, and the key file is
usually in '~/.ssh/' folder, default value is needed,
if the file is not generate by manually,
the freezer-agent will broken. Add checking is a
good idea for use. this patch is to:
1. add default ssh keyfile value
2. add ssh keyfile checking
Enable treading warnings as errors so that no new broken stuff gets
merged. Remove obsolete way of doing it, it's a nop.
Fix bugs found:
* Add missing sphinx modules to properly generate content.
* Misformatting of comments and files
* Do not index the windows modules, they import modules that do not
exist on Linux.
* Place api docs in reference/api according to
and reference it. Update .gitignore for this.
* Include missing files in tree.
When object count is bigger than 1000, IsTruncated is True, then
the restore will fail. (Although object number of the cinder mode
backup of a volume will never be bigger than 1000 with using
multipart upload in s3 driver, this should also be fixed)
Currently, when use 'freezer-agent --action admin --remove-older-than
xx --mode cinder --cinder-vol-id xx --storage xx --container xx' to
remove the old backups, the result is successful, but the backup data
is not really deleted. This path will increase AdminJob support for
cinder mode backup and fix the issue.
When use s3 driver, you should specify storage type with
'--storage s3', specify the auth info like '--access-key xxx'
and '--secret-key xxx', and specify the S3 compatible storage
endpoint with '--endpoint http://ip:port'.
Implements: blueprint s3-driver-support
There are two issues in current 'admin action' of nova
1. When remove old nova backups with command 'freezer-agent
--action admin --mode nova --engine nova --nova-inst-id xxxx
--no-incremental true --remove-older-than x" will fail cause
the incorrect swift path.
2. Can not do 'admin action' of nova tenant.
This patch contains two changes.
1. Fix remove old nova backups error.
2. Add option to do 'admin action' of nova tenant.
At the moment, the Swift storage codebase does not have a consistent
way of using the Swift client, as it switches from grabbing an
existing client and creating new ones.
Due to the architecture of how restores happen, there are several
times where the same storage instance is held by the main process,
the process which reads from the storage and the engine (to retrieve
metadata). All of these accesses result in unexpected behaviour which
can raise SSL errors such as: DECRYPTION_FAILED_OR_BAD_RECORD_MAC
The overall cost of creating a new client for every request is very
minimal and it eliminates all possibility of multithreading issues
that occur as per suggested by the author of the requests Python
library, which is used by the Swift client.
This patch eliminates all calls to create_swift() indirectly and makes
sure that all accesses to the Swift client create a new instance. In
my local testing, there was no noticable performance change but
no more SSL failures.
Implement nova tenant backup and restore to swift, local and ssh storage.
Add unit tests. Add new CLI option '--project-id' to perform tenant
backup/restore. Add new option - '--backup-media' as well.
Partially implements bp: tenant-backup
Freezer-agent backup volume_by_glance upload data to swift by default.
This patch backup it and metadata to local,Now.
backup nova / cinder by glance to local:
freezer-agent --cinder-vol-id 0bd0d291-e75c-4fd9-b13b-058c335a8497
--storage local --container /home/nfs --mode cinder
freezer-agent --action restore
--storage local --container /home/nfs
Implements blueprint: backup-nova-cinder-to-non-swift
When restore data with --restore-from-date option from local storage,
The most recent backup subsequent to the specified date will be
According to the detailed desciptions from README.rst, the
--restore-from-date option will select the most recent backup previous
to the specified date.
The patch fixed the issue.
Signed-off-by: Shangzhong Zhu <email@example.com>
We need to pass the content length to swift in case of uploading
files, so we needed to calcuate the engine's metadata file size
and pass it to swift during upload.
When backup data to ssh storage for the first time, because the sub
directory "metadata" isn't exist in the ssh storage directory, the
exception IOError will be raised.
The exception IOError should be caught in the funtion listdir() of the
SshStorage class, and the empty list should be returned when the errno
is ENOENT which indicates "No such file or directory". The other
exceptions, e.g. "Permission denied", should be raised.
This patch set modifies lines which are importing objects
instead of modules. As per openstack import guide lines, user should
import modules in a file not objects.
Also, imports was regrouped and placed in alphabetical order.
Signed-off-by: Ruslan Aliev <firstname.lastname@example.org>
There is no need to call os.remove() with temp file,
it will be removed automatically after closed.
Actually, freezer.log has warning messages about it:
WARNING freezer.utils.utils [-] Error deleting file /tmp/tmpSNedJh
This patch will fix it.
Also, added newline to info message that using scheduler is deprecated
and more verbose output if there is a problem with deleting file.
Signed-off-by: Ruslan Aliev <email@example.com>
When an exception was caught and rethrown, it should call 'raise'
without any arguments because it shows the place where an exception
occured initially instead of place where the exception re-raised.