openstack/freezer
Code Issues Proposed changes

Import pep3134daemon as local module

Browse Source 
pep3134daemon is now imported from local path
rather than pip. This generated many issues
as the package is not in the global-requirements.txt
of kilo and liberty. Also pbr in the kilo release
does not support env markers which further complitated
the installation. This change needs to be backported to
stable/kilo also.

Also some minor fix for RST format.

Change-Id: I34db28ffc928703b01e6430a661214d66d81c519
This commit is contained in:
Fausto Marzi 2015-11-20 13:20:30 +00:00
parent c3739a2b3d
commit 3abf2080f1
8 changed files with 729 additions and 129 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
LICENSE
View File

@ -173,3 +173,28 @@
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
As per https://wiki.openstack.org/wiki/LegalIssuesFAQ#Incorporating_BSD.2FMIT_Licensed_Code
the code located in freezer/lib/pep3143daemon contains code released under
the MIT License:
The MIT License (MIT)
Copyright (c) 2014 Stephan Schultchen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

253
README.rst
View File

@ -1,4 +1,3 @@
=======
Freezer
=======
@ -47,7 +46,7 @@ Windows Requirements
====================
- Python 2.7
- GNU Tar (we recommend to follow [this guide](https://github.com/memogarcia/freezer-windows-binaries#windows-binaries-for-freezer) to install them)
- GNU Tar binaries (we recommend to follow [this guide](https://github.com/memogarcia/freezer-windows-binaries#windows-binaries-for-freezer) to install them)
- [OpenSSL pre-compiled for windows](https://wiki.openssl.org/index.php/Binaries) or [direct download](https://indy.fulgan.com/SSL/openssl-1.0.1-i386-win32.zip)
- [Sync](https://technet.microsoft.com/en-us/sysinternals/bb897438.aspx)
- [Microsoft Visual C++ Compiler for Python 2.7](http://aka.ms/vcpython27)
@ -60,11 +59,12 @@ Go to **Control Panel\System and Security\System** and then **Advanced System Se
- ;C:\OpenSSL-Win64\bin
- ;C:\Python27;C:\Python27\Lib\site-packages\;C:\Python27\Scripts\
The following components support Windowd OS Platform:
The following components support Windows OS Platform:
- freezer-agent
- freezer-scheduler
Installation & Env Setup
========================
@ -236,6 +236,7 @@ and assumes that making backup of that image will be sufficient to restore your
data in future.
Execute a cinder backup::
$ freezerc --cinder-vol-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b
Execute a mysql backup with cinder::
@ -252,6 +253,7 @@ Freezer doesn't do any additional checks and assumes that making backup
of that instance will be sufficient to restore your data in future.
Execute a nova backup::
$ freezerc --nova-inst-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b
Execute a mysql backup with nova::
@ -593,13 +595,11 @@ It has a double role: it is used both to start the scheduler process, and as
a cli-tool which allows the user to interact with the api.
The freezer-scheduler process can be started/stopped in daemon mode using the usual
positional arguments
::
positional arguments::
freezer-scheduler start|stop
It can be also be started as a foreground process using the --no-daemon flag:
::
It can be also be started as a foreground process using the --no-daemon flag::
freezer-scheduler --no-daemon start
@ -638,8 +638,7 @@ which is composed from the tenant-is and the hostname of the machine it is
running on.
The first step to use the scheduler is creating a document with the job:
::
The first step to use the scheduler is creating a document with the job::
cat test_job.json
@ -665,13 +664,11 @@ The first step to use the scheduler is creating a document with the job:
"description": "my scheduled backup 6"
}
Then upload that job into the api:
::
Then upload that job into the api::
freezer-scheduler -c node12 job-create --file test_job.json
The newly created job can be found with:
::
The newly created job can be found with::
freezer-scheduler -c node12 job-list
@ -681,13 +678,11 @@ The newly created job can be found with:
| 07999ea33a494ccf84590191d6fe850c | schedule_backups 6 | 1 | | | | |
+----------------------------------+--------------------+-----------+--------+-------+--------+------------+
Its content can be read with:
::
Its content can be read with::
freezer-scheduler -c node12 job-get -j 07999ea33a494ccf84590191d6fe850c
The scheduler can be started on the target node with:
::
The scheduler can be started on the target node with::
freezer-scheduler -c node12 -i 15 -f ~/job_dir start
@ -695,119 +690,129 @@ The scheduler could have already been started. As soon as the freezer-scheduler
it fetches the job and schedules it.
Miscellanea
-----------
Misc
====
Dependencies notes
------------------
In stable/kilo and stable/liberty the module peppep3134daemon is imported
from local path
rather than pip. This generated many issues
as the package is not in the global-requirements.txt
of kilo and liberty. Also pbr in the kilo release
does not support env markers which further complitated
the installation
Please check the FAQ to: FAQ.rst
Available options::
usage: freezerc [-h] [--config CONFIG]
[--action {backup,restore,info,admin,exec}]
[-F PATH_TO_BACKUP] [-N BACKUP_NAME] [-m MODE] [-C CONTAINER]
[-L] [-l] [-o GET_OBJECT] [-d DST_FILE] [-s]
[--lvm-auto-snap LVM_AUTO_SNAP] [--lvm-srcvol LVM_SRCVOL]
[--lvm-snapname LVM_SNAPNAME] [--lvm-snap-perm {ro,rw}]
[--lvm-snapsize LVM_SNAPSIZE] [--lvm-dirmount LVM_DIRMOUNT]
[--lvm-volgroup LVM_VOLGROUP] [--max-level MAX_LEVEL]
[--always-level ALWAYS_LEVEL]
[--restart-always-level RESTART_ALWAYS_LEVEL]
[-R REMOVE_OLDER_THAN] [--remove-from-date REMOVE_FROM_DATE]
[--no-incremental] [--hostname HOSTNAME]
[--mysql-conf MYSQL_CONF] [--metadata-out METADATA_OUT]
[--log-file LOG_FILE] [--exclude EXCLUDE]
[--dereference-symlink {none,soft,hard,all}] [-U]
[--encrypt-pass-file ENCRYPT_PASS_FILE] [-M MAX_SEGMENT_SIZE]
[--restore-abs-path RESTORE_ABS_PATH]
[--restore-from-host HOSTNAME]
[--restore-from-date RESTORE_FROM_DATE] [--max-priority] [-V]
[-q] [--insecure] [--os-auth-ver {1,2,2.0,3}] [--proxy PROXY]
[--dry-run] [--upload-limit UPLOAD_LIMIT]
[--cinder-vol-id CINDER_VOL_ID] [--nova-inst-id NOVA_INST_ID]
[--cindernative-vol-id CINDERNATIVE_VOL_ID]
[--download-limit DOWNLOAD_LIMIT]
[--sql-server-conf SQL_SERVER_CONF] [--vssadmin VSSADMIN]
[--command COMMAND] [--compression {gzip,bzip2,xz}]
[--storage {local,swift,ssh}] [--ssh-key SSH_KEY]
[--ssh-username SSH_USERNAME] [--ssh-host SSH_HOST]
[--ssh-port SSH_PORT]
usage: freezerc [-h] [--config CONFIG]
[--action {backup,restore,info,admin,exec}]
[-F PATH_TO_BACKUP] [-N BACKUP_NAME] [-m MODE] [-C CONTAINER]
[-L] [-l] [-o GET_OBJECT] [-d DST_FILE] [-s]
[--lvm-auto-snap LVM_AUTO_SNAP] [--lvm-srcvol LVM_SRCVOL]
[--lvm-snapname LVM_SNAPNAME] [--lvm-snap-perm {ro,rw}]
[--lvm-snapsize LVM_SNAPSIZE] [--lvm-dirmount LVM_DIRMOUNT]
[--lvm-volgroup LVM_VOLGROUP] [--max-level MAX_LEVEL]
[--always-level ALWAYS_LEVEL]
[--restart-always-level RESTART_ALWAYS_LEVEL]
[-R REMOVE_OLDER_THAN] [--remove-from-date REMOVE_FROM_DATE]
[--no-incremental] [--hostname HOSTNAME]
[--mysql-conf MYSQL_CONF] [--metadata-out METADATA_OUT]
[--log-file LOG_FILE] [--exclude EXCLUDE]
[--dereference-symlink {none,soft,hard,all}] [-U]
[--encrypt-pass-file ENCRYPT_PASS_FILE] [-M MAX_SEGMENT_SIZE]
[--restore-abs-path RESTORE_ABS_PATH]
[--restore-from-host HOSTNAME]
[--restore-from-date RESTORE_FROM_DATE] [--max-priority] [-V]
[-q] [--insecure] [--os-auth-ver {1,2,2.0,3}] [--proxy PROXY]
[--dry-run] [--upload-limit UPLOAD_LIMIT]
[--cinder-vol-id CINDER_VOL_ID] [--nova-inst-id NOVA_INST_ID]
[--cindernative-vol-id CINDERNATIVE_VOL_ID]
[--download-limit DOWNLOAD_LIMIT]
[--sql-server-conf SQL_SERVER_CONF] [--vssadmin VSSADMIN]
[--command COMMAND] [--compression {gzip,bzip2,xz}]
[--storage {local,swift,ssh}] [--ssh-key SSH_KEY]
[--ssh-username SSH_USERNAME] [--ssh-host SSH_HOST]
[--ssh-port SSH_PORT]
optional arguments:
-h, --help show this help message and exit
--config CONFIG Config file abs path. Option arguments are provided
from config file. When config file is used any option
from command line provided take precedence.
--action {backup,restore,info,admin,exec}
Set the action to be taken. backup and restore are
self explanatory, info is used to retrieve info from
the storage media, exec is used to execute a script,
while admin is used to delete old backups and other
admin actions. Default backup.
-F PATH_TO_BACKUP, --path-to-backup PATH_TO_BACKUP, --file-to-backup PATH_TO_BACKUP
The file or directory you want to back up to Swift
-N BACKUP_NAME, --backup-name BACKUP_NAME
The backup name you want to use to identify your
backup on Swift
-m MODE, --mode MODE Set the technology to back from. Options are, fs
(filesystem), mongo (MongoDB), mysql (MySQL),
sqlserver (SQL Server) Default set to fs
-C CONTAINER, --container CONTAINER
The Swift container (or path to local storage) used to
upload files to
-L, --list-containers
List the Swift containers on remote Object Storage
Server
-l, --list-objects List the Swift objects stored in a container on remote
Object Storage Server.
-o GET_OBJECT, --get-object GET_OBJECT
The Object name you want to download on the local file
system.
-d DST_FILE, --dst-file DST_FILE
The file name used to save the object on your local
disk and upload file in swift
-s, --snapshot Create a snapshot of the fs containing the resource to
backup. When used, the lvm parameters will be guessed
and/or the default values will be used
--lvm-auto-snap LVM_AUTO_SNAP
Automatically guess the volume group and volume name
for given PATH.
--lvm-srcvol LVM_SRCVOL
Set the lvm volume you want to take a snaphost from.
Default no volume
--lvm-snapname LVM_SNAPNAME
Set the lvm snapshot name to use. If the snapshot name
already exists, the old one will be used a no new one
will be created. Default freezer_backup_snap.
--lvm-snap-perm {ro,rw}
Set the lvm snapshot permission to use. If the
permission is set to ro The snapshot will be immutable
- read only -. If the permission is set to rw it will
be mutable
--lvm-snapsize LVM_SNAPSIZE
Set the lvm snapshot size when creating a new
snapshot. Please add G for Gigabytes or M for
Megabytes, i.e. 500M or 8G. Default 1G.
--lvm-dirmount LVM_DIRMOUNT
Set the directory you want to mount the lvm snapshot
to. Default to /var/lib/freezer
--lvm-volgroup LVM_VOLGROUP
Specify the volume group of your logical volume. This
is important to mount your snapshot volume. Default
not set
--max-level MAX_LEVEL
Set the backup level used with tar to implement
incremental backup. If a level 1 is specified but no
level 0 is already available, a level 0 will be done
and subsequently backs to level 1. Default 0 (No
Incremental)
--always-level ALWAYS_LEVEL
Set backup maximum level used with tar to implement
incremental backup. If a level 3 is specified, the
backup will be executed from level 0 to level 3 and to
that point always a backup level 3 will be executed.
It will not restart from level 0. This option has
precedence over --max-backup-level. Default False
(Disabled)
optional arguments:
-h, --help show this help message and exit
--config CONFIG Config file abs path. Option arguments are provided
from config file. When config file is used any option
from command line provided take precedence.
--action {backup,restore,info,admin,exec}
Set the action to be taken. backup and restore are
self explanatory, info is used to retrieve info from
the storage media, exec is used to execute a script,
while admin is used to delete old backups and other
admin actions. Default backup.
-F PATH_TO_BACKUP, --path-to-backup PATH_TO_BACKUP, --file-to-backup PATH_TO_BACKUP
The file or directory you want to back up to Swift
-N BACKUP_NAME, --backup-name BACKUP_NAME
The backup name you want to use to identify your
backup on Swift
-m MODE, --mode MODE Set the technology to back from. Options are, fs
(filesystem), mongo (MongoDB), mysql (MySQL),
sqlserver (SQL Server) Default set to fs
-C CONTAINER, --container CONTAINER
The Swift container (or path to local storage) used to
upload files to
-L, --list-containers
List the Swift containers on remote Object Storage
Server
-l, --list-objects List the Swift objects stored in a container on remote
Object Storage Server.
-o GET_OBJECT, --get-object GET_OBJECT
The Object name you want to download on the local file
system.
-d DST_FILE, --dst-file DST_FILE
The file name used to save the object on your local
disk and upload file in swift
-s, --snapshot Create a snapshot of the fs containing the resource to
backup. When used, the lvm parameters will be guessed
and/or the default values will be used
--lvm-auto-snap LVM_AUTO_SNAP
Automatically guess the volume group and volume name
for given PATH.
--lvm-srcvol LVM_SRCVOL
Set the lvm volume you want to take a snaphost from.
Default no volume
--lvm-snapname LVM_SNAPNAME
Set the lvm snapshot name to use. If the snapshot name
already exists, the old one will be used a no new one
will be created. Default freezer_backup_snap.
--lvm-snap-perm {ro,rw}
Set the lvm snapshot permission to use. If the
permission is set to ro The snapshot will be immutable
- read only -. If the permission is set to rw it will
be mutable
--lvm-snapsize LVM_SNAPSIZE
Set the lvm snapshot size when creating a new
snapshot. Please add G for Gigabytes or M for
Megabytes, i.e. 500M or 8G. Default 1G.
--lvm-dirmount LVM_DIRMOUNT
Set the directory you want to mount the lvm snapshot
to. Default to /var/lib/freezer
--lvm-volgroup LVM_VOLGROUP
Specify the volume group of your logical volume. This
is important to mount your snapshot volume. Default
not set
--max-level MAX_LEVEL
Set the backup level used with tar to implement
incremental backup. If a level 1 is specified but no
level 0 is already available, a level 0 will be done
and subsequently backs to level 1. Default 0 (No
Incremental)
--always-level ALWAYS_LEVEL
Set backup maximum level used with tar to implement
incremental backup. If a level 3 is specified, the
backup will be executed from level 0 to level 3 and to
that point always a backup level 3 will be executed.
It will not restart from level 0. This option has
precedence over --max-backup-level. Default False
(Disabled)
--restart-always-level RESTART_ALWAYS_LEVEL
Restart the backup from level 0 after n days. Valid
only if --always-level option if set. If --always-

0
freezer/lib/__init__.py Normal file
View File

20
freezer/lib/pep3143daemon/__init__.py Normal file
View File

@ -0,0 +1,20 @@
# -*- coding: utf-8 -*-
# flake8: noqa
"""
pep3143daemon is a implementation of the PEP 3143, describing a well behaving
Unix daemon, as documented in Stevens 'Unix Network Programming'
Copyright (c) 2014, Stephan Schultchen.
License: MIT (see LICENSE for details)
"""
from freezer.lib.pep3143daemon.daemon import DaemonContext, DaemonError
from freezer.lib.pep3143daemon.pidfile import PidFile
__all__ = [
"DaemonContext",
"DaemonError",
"PidFile",
]

449
freezer/lib/pep3143daemon/daemon.py Normal file
View File

@ -0,0 +1,449 @@
# -*- coding: utf-8 -*-
# flake8: noqa
# The MIT License (MIT)
#
# Copyright (c) 2014 Stephan Schultchen
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
"""Implementation of PEP 3143 DaemonContext"""
__author__ = 'schlitzer'
import errno
import os
import resource
import signal
import socket
import sys
# PY2 / PY3 gap
PY3 = sys.version_info[0] == 3
if PY3:
string_types = str,
else:
string_types = basestring,
class DaemonError(Exception):
""" Exception raised by DaemonContext"""
pass
class DaemonContext(object):
""" Implementation of PEP 3143 DaemonContext class
This class should be instantiated only once in every program that
has to become a Unix Daemon. Typically you should call its open method
after you have done everything that may require root privileges.
For example opening port <= 1024.
Each option can be passed as a keyword argument to the constructor, but
can also be changed by assigning a new value to the corresponding attribute
on the instance.
Altering attributes after open() is called, will have no effect.
In future versions, trying to do so, will may raise a DaemonError.
:param chroot_directory:
Full path to the directory that should be set as effective root
directory. If None, the root directory is not changed.
:type chroot_directory: str
:param working_directory:
Full Path to the working directory to which to change to.
If chroot_directory is not None, and working_directory is not
starting with chroot_directory, working directory is prefixed
with chroot_directory.
:type working_directory: str.
:param umask:
File access creation mask for this daemon after start
:type umask: int.
:param uid:
Effective user id after daemon start.
:type uid: int.
:param gid:
Effective group id after daemon start.
:type gid: int.
:param prevent_core:
Prevent core file generation.
:type prevent_core: bool.
:param detach_process:
If True, do the double fork magic. If the process was started
by inet or an init like program, you may don´t need to detach.
If not set, we try to figure out if forking is needed.
:type detach_process: bool.
:param files_preserve:
List of integers, or objects with a fileno method, that
represent files that should not be closed while daemoninzing.
:type files_preserve: list
:param pidfile:
Instance that implements a pidfile, while daemonizing its
acquire method will be called.
:type pidfile: Instance of Class that implements a pidfile behaviour
:param stdin:
Redirect stdin to this file, if None, redirect to /dev/null.
:type stdin: file object.
:param stdout:
Redirect stdout to this file, if None, redirect to /dev/null.
:type stdout: file object.
:param stderr:
Redirect stderr to this file, if None, redirect to /dev/null.
:type stderr: file object.
:param signal_map:
Mapping from operating system signal to callback actions.
:type signal_map: instance of dict
"""
def __init__(
self, chroot_directory=None, working_directory='/',
umask=0, uid=None, gid=None, prevent_core=True,
detach_process=None, files_preserve=None, pidfile=None,
stdin=None, stdout=None, stderr=None, signal_map=None):
""" Initialize a new Instance
"""
self._is_open = False
self._working_directory = None
self.chroot_directory = chroot_directory
self.umask = umask
self.uid = uid if uid else os.getuid()
self.gid = gid if gid else os.getgid()
if detach_process is None:
self.detach_process = detach_required()
else:
self.detach_process = detach_process
self.signal_map = signal_map if signal_map else default_signal_map()
self.files_preserve = files_preserve
self.pidfile = pidfile
self.prevent_core = prevent_core
self.stdin = stdin
self.stdout = stdout
self.stderr = stderr
self.working_directory = working_directory
def __enter__(self):
""" Context Handler, wrapping self.open()
:return: self
"""
self.open()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
""" Context Handler, wrapping self.close()
:return: None
"""
self.close()
def _get_signal_handler(self, handler):
""" get the callback function for handler
If the handler is None, returns signal.SIG_IGN.
If the handler is a string, return the matching attribute of this
instance if possible.
Else return the handler itself.
:param handler:
:type handler: str, None, function
:return: function
"""
if not handler:
result = signal.SIG_IGN
elif isinstance(handler, string_types):
result = getattr(self, handler)
else:
result = handler
return result
@property
def _files_preserve(self):
""" create a set of protected files
create a set of files, based on self.files_preserve and
self.stdin, self,stdout and self.stderr, that should not get
closed while daemonizing.
:return: set
"""
result = set()
files = [] if not self.files_preserve else self.files_preserve
files.extend([self.stdin, self.stdout, self.stderr])
for item in files:
if hasattr(item, 'fileno'):
result.add(item.fileno())
if isinstance(item, int):
result.add(item)
return result
@property
def _signal_handler_map(self):
""" Create the signal handler map
create a dictionary with signal:handler mapping based on
self.signal_map
:return: dict
"""
result = {}
for signum, handler in self.signal_map.items():
result[signum] = self._get_signal_handler(handler)
return result
@property
def working_directory(self):
""" The working_directory property
:return: str
"""
if self.chroot_directory and not \
self._working_directory.startswith(self.chroot_directory):
return self.chroot_directory + self._working_directory
else:
return self._working_directory
@working_directory.setter
def working_directory(self, value):
""" Set working directory
New value is ignored if already daemonized.
:param value: str
:return:
"""
self._working_directory = value
@property
def is_open(self):
""" True when this instances open method was called
:return: bool
"""
return self._is_open
def close(self):
""" Dummy function"""
pass
def open(self):
""" Daemonize this process
Do everything that is needed to become a Unix daemon.
:return: None
:raise: DaemonError
"""
if self.is_open:
return
try:
os.chdir(self.working_directory)
if self.chroot_directory:
os.chroot(self.chroot_directory)
os.setgid(self.gid)
os.setuid(self.uid)
os.umask(self.umask)
except OSError as err:
raise DaemonError('Setting up Environment failed: {0}'
.format(err))
if self.prevent_core:
try:
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
except Exception as err:
raise DaemonError('Could not disable core files: {0}'
.format(err))
if self.detach_process:
try:
if os.fork() > 0:
os._exit(0)
except OSError as err:
raise DaemonError('First fork failed: {0}'.format(err))
os.setsid()
try:
if os.fork() > 0:
os._exit(0)
except OSError as err:
raise DaemonError('Second fork failed: {0}'.format(err))
for (signal_number, handler) in self._signal_handler_map.items():
signal.signal(signal_number, handler)
close_filenos(self._files_preserve)
redirect_stream(sys.stdin, self.stdin)
redirect_stream(sys.stdout, self.stdout)
redirect_stream(sys.stderr, self.stderr)
if self.pidfile:
self.pidfile.acquire()
self._is_open = True
def terminate(self, signal_number, stack_frame):
""" Terminate this process
Simply terminate this process by raising SystemExit.
This method is called if signal.SIGTERM was received.
Check carefully if this really is what you want!
Most likely it is not!
You should implement a function/method that is able to cleanly
shutdown you daemon. Like gracefully terminating child processes,
threads. or closing files.
You can create a custom handler by overriding this method, ot
setting a custom handler via the signal_map. It is also possible
to set the signal handlers directly via signal.signal().
:return: None
:raise: SystemExit
"""
raise SystemExit('Terminating on signal {0}'.format(signal_number))
def close_filenos(preserve):
""" Close unprotected file descriptors
Close all open file descriptors that are not in preserve.
If ulimit -nofile is "unlimited", all is defined filenos <= 4096,
else all is <= the output of resource.getrlimit().
:param preserve: set with protected files
:type preserve: set
:return: None
"""
maxfd = resource.getrlimit(resource.RLIMIT_NOFILE)[1]
if maxfd == resource.RLIM_INFINITY:
maxfd = 4096
for fileno in range(maxfd):
if fileno not in preserve:
try:
os.close(fileno)
except OSError as err:
if not err.errno == errno.EBADF:
raise DaemonError(
'Failed to close file descriptor {0}: {1}'
.format(fileno, err))
def default_signal_map():
""" Create the default signal map for this system.
:return: dict
"""
name_map = {
'SIGTSTP': None,
'SIGTTIN': None,
'SIGTTOU': None,
'SIGTERM': 'terminate'}
signal_map = {}
for name, target in name_map.items():
if hasattr(signal, name):
signal_map[getattr(signal, name)] = target
return signal_map
def parent_is_init():
""" Check if parent is Init
Check if the parent process is init, or something else that
owns PID 1.
:return: bool
"""
if os.getppid() == 1:
return True
return False
def parent_is_inet():
""" Check if parent is inet
Check if our parent seems ot be a superserver, aka inetd/xinetd.
This is done by checking if sys.__stdin__ is a network socket.
:return: bool
"""
result = False
sock = socket.fromfd(
sys.__stdin__.fileno(),
socket.AF_INET,
socket.SOCK_RAW)
try:
sock.getsockopt(socket.SOL_SOCKET, socket.SO_TYPE)
result = True
except (OSError, socket.error) as err:
if not err.args[0] == errno.ENOTSOCK:
result = True
return result
def detach_required():
""" Check if detaching is required
This is done by collecting the results of parent_is_inet and
parent_is_init. If one of them is True, detaching, aka the daemoninzing,
aka the double fork magic, is not required, and can be skipped.
:return: bool
"""
if parent_is_inet() or parent_is_init():
return False
return True
def redirect_stream(system, target):
""" Redirect Unix streams
If None, redirect Stream to /dev/null, else redirect to target.
:param system: ether sys.stdin, sys.stdout, or sys.stderr
:type system: file object
:param target: File like object, or None
:type target: None, File Object
:return: None
:raise: DaemonError
"""
if target is None:
target_fd = os.open(os.devnull, os.O_RDWR)
else:
target_fd = target.fileno()
try:
os.dup2(target_fd, system.fileno())
except OSError as err:
raise DaemonError('Could not redirect {0} to {1}: {2}'
.format(system, target, err))

105
freezer/lib/pep3143daemon/pidfile.py Normal file
View File

@ -0,0 +1,105 @@
# -*- coding: utf-8 -*-
# flake8: noqa
# The MIT License (MIT)
#
# Copyright (c) 2014 Stephan Schultchen
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# -*- coding: utf-8 -*-
"""
Simple PidFile Module for a pep3143 daemon implementation.
"""
__author__ = 'schlitzer'
import atexit
import fcntl
import os
class PidFile(object):
"""
PidFile implementation for PEP 3143 Daemon.
This Class can also be used with pythons 'with'
statement.
:param pidfile:
filename to be used as pidfile, including path
:type pidfile: str
"""
def __init__(self, pidfile):
"""
Create a new instance
"""
self._pidfile = pidfile
self.pidfile = None
def __enter__(self):
self.acquire()
return self
def __exit__(self, exc_type, exc_value, exc_tb):
if exc_type is not None:
self.release()
return False
self.release()
return True
def acquire(self):
"""Acquire the pidfile.
Create the pidfile, lock it, write the pid into it
and register the release with atexit.
:return: None
:raise: SystemExit
"""
try:
pidfile = open(self._pidfile, "a")
except IOError as err:
raise SystemExit(err)
try:
fcntl.flock(pidfile.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError:
raise SystemExit('Already running according to ' + self._pidfile)
pidfile.seek(0)
pidfile.truncate()
pidfile.write(str(os.getpid()) + '\n')
pidfile.flush()
self.pidfile = pidfile
atexit.register(self.release)
def release(self):
"""Release the pidfile.
Close and delete the Pidfile.
:return: None
"""
try:
self.pidfile.close()
os.remove(self._pidfile)
except OSError as err:
if err.errno != 2:
raise

2
freezer/scheduler/daemon.py
View File

@ -24,7 +24,7 @@ from tempfile import gettempdir
from time import sleep
from pep3143daemon import DaemonContext, PidFile
from freezer.lib.pep3143daemon import DaemonContext, PidFile
from freezer.utils import create_dir

4
requirements.txt
View File

@ -15,7 +15,3 @@ paramiko>=1.13.0
# Not in global-requirements
apscheduler
# Platform specific
pywin32; sys_platform == 'win32'
pep3143daemon; sys_platform != 'win32'