x/packstack
Code Issues Proposed changes

Add 'installer/' from commit '39ffc6da62a6510e3c63b8f5ed3f8943c0c80ccb'

Browse Source 
git-subtree-dir: installer
git-subtree-mainline: f3fefd11ba
git-subtree-split: 39ffc6da62
This commit is contained in:
Derek Higgins
2012-11-22 12:44:28 +00:00
parent f3fefd11ba 39ffc6da62
commit d2fe8f32d2
11 changed files with 2244 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
installer/.gitignore vendored Normal file
View File

@@ -0,0 +1,3 @@
*.pyc
*.swp
*.log

202
installer/LICENSE Normal file
View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

37
installer/basedefs.py Normal file
View File

@@ -0,0 +1,37 @@
"""
provides all the predefined variables for engine-setup
"""
import os
APP_NAME = "Installer"
FILE_YUM_VERSION_LOCK="/etc/yum/pluginconf.d/versionlock.list"
DIR_LOG = "./var"
FILE_INSTALLER_LOG = "setup.log"
DIR_PROJECT_DIR = "./sample-project"
DIR_PLUGINS = os.path.join(DIR_PROJECT_DIR, "plugins")
DIR_MODULES = os.path.join(DIR_PROJECT_DIR, "modules")
EXEC_RPM="rpm"
EXEC_SEMANAGE="semanage"
EXEC_NSLOOKUP="nslookup"
EXEC_CHKCONFIG="chkconfig"
EXEC_SERVICE="service"
#text colors
RED="\033[0;31m"
GREEN="\033[92m"
BLUE="\033[94m"
YELLOW="\033[93m"
NO_COLOR="\033[0m"
COLORS = (RED, GREEN, BLUE, YELLOW, NO_COLOR)
#space len size for color print
SPACE_LEN=70
RPM_LOCK_LIST = """
"""

338
installer/common_utils.py Normal file
View File

@@ -0,0 +1,338 @@
"""
contains all common and re-usable code for rhevm-setup and sub packages
"""
import grp
import pwd
import logging
import subprocess
import re
import output_messages
import traceback
import os
import basedefs
import datetime
import libxml2
import types
import shutil
import time
import tempfile
def getColoredText (text, color):
''' gets text string and color
and returns a colored text.
the color values are RED/BLUE/GREEN/YELLOW
everytime we color a text, we need to disable
the color at the end of it, for that
we use the NO_COLOR chars.
'''
return color + text + basedefs.NO_COLOR
def execCmd(cmdList, cwd=None, failOnError=False, msg=output_messages.ERR_RC_CODE, maskList=[], useShell=False, usePipeFiles=False):
"""
Run external shell command with 'shell=false'
receives a list of arguments for command line execution
"""
# All items in the list needs to be strings, otherwise the subprocess will fail
cmd = [str(item) for item in cmdList]
# We need to join cmd list into one string so we can look for passwords in it and mask them
logCmd = _maskString((' '.join(cmd)), maskList)
logging.debug("Executing command --> '%s'"%(logCmd))
stdErrFD = subprocess.PIPE
stdOutFD = subprocess.PIPE
stdInFD = subprocess.PIPE
if usePipeFiles:
(stdErrFD, stdErrFile) = tempfile.mkstemp(dir="/tmp")
(stdOutFD, stdOutFile) = tempfile.mkstemp(dir="/tmp")
(stdInFD, stdInFile) = tempfile.mkstemp(dir="/tmp")
# We use close_fds to close any file descriptors we have so it won't be copied to forked childs
proc = subprocess.Popen(cmd, stdout=stdOutFD,
stderr=stdErrFD, stdin=stdInFD, cwd=cwd, shell=useShell, close_fds=True)
out, err = proc.communicate()
if usePipeFiles:
with open(stdErrFile, 'r') as f:
err = f.read()
os.remove(stdErrFile)
with open(stdOutFile, 'r') as f:
out = f.read()
os.remove(stdOutFile)
os.remove(stdInFile)
logging.debug("output = %s"%(out))
logging.debug("stderr = %s"%(err))
logging.debug("retcode = %s"%(proc.returncode))
output = out + err
if failOnError and proc.returncode != 0:
raise Exception(msg)
return ("".join(output.splitlines(True)), proc.returncode)
def byLength(word1, word2):
"""
Compars two strings by their length
Returns:
Negative if word2 > word1
Positive if word1 > word2
Zero if word1 == word 2
"""
return len(word1) - len(word2)
def nslookup(address):
cmd = [
basedefs.EXEC_NSLOOKUP, address,
]
#since nslookup will return 0 no matter what, the RC is irrelevant
output, rc = execCmd(cmdList=cmd)
return output
def getConfiguredIps():
try:
iplist=set()
cmd = [
basedefs.EXEC_IP, "addr",
]
output, rc = execCmd(cmdList=cmd, failOnError=True, msg=output_messages.ERR_EXP_GET_CFG_IPS_CODES)
ipaddrPattern=re.compile('\s+inet (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).+')
list=output.splitlines()
for line in list:
foundIp = ipaddrPattern.search(line)
if foundIp:
if foundIp.group(1) != "127.0.0.1":
ipAddr = foundIp.group(1)
logging.debug("Found IP Address: %s"%(ipAddr))
iplist.add(ipAddr)
return iplist
except:
logging.error(traceback.format_exc())
raise Exception(output_messages.ERR_EXP_GET_CFG_IPS)
def getCurrentDateTime(isUtc=None):
now = None
if (isUtc is not None):
now = datetime.datetime.utcnow()
else:
now = datetime.datetime.now()
return now.strftime("%Y_%m_%d_%H_%M_%S")
def verifyStringFormat(str, matchRegex):
'''
Verify that the string given matches the matchRegex.
for example:
string: 111-222
matchRegex: \d{3}-\d{3}
this will return true since the string matches the regex
'''
pattern = re.compile(matchRegex)
result = re.match(pattern, str)
if result == None:
return False
else:
return True
def compareStrIgnoreCase(str1, str2):
''' compare 2 strings and ignore case
if one of the input is not str (bool for e.g) - return normal comapre
'''
if type(str1) == types.StringType and type(str2) == types.StringType:
return str1.lower() == str2.lower()
else:
return str1 == str2
def parseStrRegex(string, regex, errMsg):
"""
Gets a text string and a regex pattern
and returns the extracted sub-string
captured.
"""
rePattern = re.compile(regex)
found = rePattern.search(string)
if found:
match = found.group(1)
logging.debug("found new parsed string: %s"%(match))
return match
else:
raise Exception(errMsg)
def _maskString(string, maskList=[]):
"""
private func to mask passwords
in utils
"""
maskedStr = string
for maskItem in maskList:
maskedStr = maskedStr.replace(maskItem, "*"*8)
return maskedStr
def retry(func, expectedException=Exception, tries=None, timeout=None, sleep=1):
"""
Retry a function. Wraps the retry logic so you don't have to
implement it each time you need it.
:param func: The callable to run.
:param expectedException: The exception you expect to receive when the function fails.
:param tries: The number of time to try. None\0,-1 means infinite.
:param timeout: The time you want to spend waiting. This **WILL NOT** stop the method.
It will just not run it if it ended after the timeout.
:param sleep: Time to sleep between calls in seconds.
"""
if tries in [0, None]:
tries = -1
if timeout in [0, None]:
timeout = -1
startTime = time.time()
while True:
tries -= 1
try:
return func()
except expectedException:
if tries == 0:
raise
if (timeout > 0) and ((time.time() - startTime) > timeout):
raise
time.sleep(sleep)
def localHost(hostname):
# Create an ip set of possible IPs on the machine. Set has only unique values, so
# there's no problem with union.
# TODO: cache the list somehow? There's no poing quering the IP configuraion all the time.
ipset = getConfiguredIps().union(set([ "localhost", "127.0.0.1"]))
if hostname in ipset:
return True
return False
# TODO: Support SystemD services
class Service():
def __init__(self, name):
self.wasStopped = False
self.wasStarted = False
self.name = name
def isServiceAvailable(self):
if os.path.exists("/etc/init.d/%s" % self.name):
return True
return False
def start(self, raiseFailure = False):
logging.debug("starting %s", self.name)
(output, rc) = self._serviceFacility("start")
if rc == 0:
self.wasStarted = True
elif raiseFailure:
raise Exception(output_messages.ERR_FAILED_START_SERVICE % self.name)
return (output, rc)
def stop(self, raiseFailure = False):
logging.debug("stopping %s", self.name)
(output, rc) = self._serviceFacility("stop")
if rc == 0:
self.wasStopped = True
elif raiseFailure:
raise Exception(output_messages.ERR_FAILED_STOP_SERVICE % self.name)
return (output, rc)
def autoStart(self, start=True):
mode = "on" if start else "off"
cmd = [
basedefs.EXEC_CHKCONFIG, self.name, mode,
]
execCmd(cmdList=cmd, failOnError=True)
def conditionalStart(self, raiseFailure = False):
"""
Will only start if wasStopped is set to True
"""
if self.wasStopped:
logging.debug("Service %s was stopped. starting it again"%self.name)
return self.start(raiseFailure)
else:
logging.debug("Service was not stopped. there for we're not starting it")
return (False, False)
def status(self):
logging.debug("getting status for %s", self.name)
(output, rc) = self._serviceFacility("status")
return (output, rc)
def _serviceFacility(self, action):
"""
Execute the command "service NAME action"
returns: output, rc
"""
logging.debug("executing action %s on service %s", self.name, action)
cmd = [
basedefs.EXEC_SERVICE, self.name, action
]
return execCmd(cmdList=cmd, usePipeFiles=True)
def chown(target,uid, gid):
logging.debug("chown %s to %s:%s" % (target, uid, gid))
os.chown(target, uid, gid)
def installed(rpm):
cmd = [
basedefs.EXEC_RPM,
"-q",
rpm,
]
output, rc = execCmd(cmd)
return rc == 0
def returnYes(controller):
return "yes"
class ScriptRunner(object):
def __init__(self, ip=None):
self.script = []
self.ip = ip
def append(self, s):
self.script.append(s)
def execute(self):
script = "\n".join(self.script)
logging.debug("# ============ ssh : %r =========="%self.ip)
if not False: #config.justprint:
_PIPE = subprocess.PIPE # pylint: disable=E1101
if self.ip:
obj = subprocess.Popen(["ssh", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null", "root@%s"%self.ip, "bash -x"], stdin=_PIPE, stdout=_PIPE, stderr=_PIPE,
close_fds=True, shell=False)
else:
obj = subprocess.Popen(["bash", "-x"], stdin=_PIPE, stdout=_PIPE, stderr=_PIPE,
close_fds=True, shell=False)
logging.debug(script)
script = "function t(){ exit $? ; } \n trap t ERR \n" + script
stdoutdata, stderrdata = obj.communicate(script)
logging.debug("============ STDOUT ==========")
logging.debug(stdoutdata)
_returncode = obj.returncode
if _returncode:
logging.error("============= STDERR ==========")
logging.error(stderrdata)
raise Exception("Error running remote script")
else:
logging.debug(script)
def template(self, src, dst, varsdict):
with open(src) as fp:
self.append("cat > %s <<- EOF\n%s\nEOF\n"%(dst, fp.read()%varsdict))
def ifnotexists(self, fn, s):
self.append("[ -e %s ] || %s"%(fn, s))
def ifexists(self, fn, s):
self.append("[ -e %s ] && %s"%(fn, s))

367
installer/engine_validators.py Normal file
View File

@@ -0,0 +1,367 @@
"""
contains all available validation functions
"""
import common_utils as utils
import re
import logging
import output_messages
import basedefs
import types
import traceback
import os
import os.path
import tempfile
from setup_controller import Controller
def validateDirSize(path, size):
availableSpace = utils.getAvailableSpace(_getBasePath(path))
if availableSpace < size:
print output_messages.INFO_VAL_PATH_SPACE % (path,
utils.transformUnits(availableSpace),
utils.transformUnits(size))
return False
return True
def validateInteger(param, options=[]):
try:
int(param)
return True
except:
logging.warn("validateInteger('%s') - failed" %(param))
print output_messages.INFO_VAL_NOT_INTEGER
return False
def validateRe(param, options=[]):
for regex in options:
if re.search(regex, param):
return True
logging.warn("validateRe('%s') - failed" %(param))
return False
def validatePort(param, options = []):
#TODO: add actual port check with socket open
logging.debug("Validating %s as a valid TCP Port" % (param))
minVal = 0
controller = Controller()
isProxyEnabled = utils.compareStrIgnoreCase(controller.CONF["OVERRIDE_HTTPD_CONFIG"], "yes")
if not isProxyEnabled:
minVal = 1024
if not validateInteger(param, options):
return False
port = int(param)
if not (port > minVal and port < 65535) :
logging.warn(output_messages.INFO_VAL_PORT_NOT_RANGE %(minVal))
print output_messages.INFO_VAL_PORT_NOT_RANGE %(minVal)
return False
(portOpen, process, pid) = utils.isTcpPortOpen(param)
if portOpen:
logging.warn(output_messages.INFO_VAL_PORT_OCCUPIED % (param, process, pid))
print output_messages.INFO_VAL_PORT_OCCUPIED % (param, process, pid)
return False
if isProxyEnabled and not checkAndSetHttpdPortPolicy(param):
logging.warn(output_messages.INFO_VAL_FAILED_ADD_PORT_TO_HTTP_POLICY, port)
print output_messages.INFO_VAL_FAILED_ADD_PORT_TO_HTTP_POLICY % port
return False
return True
def checkAndSetHttpdPortPolicy(port):
def parsePorts(portsStr):
ports = []
for part in portsStr.split(","):
part = part.strip().split("-")
if len(part) > 1:
for port in range(int(part[0]),int(part[1])):
ports.append(port)
else:
ports.append(int(part[0]))
return ports
newPort = int(port)
cmd = [
basedefs.EXEC_SEMANAGE, "port", "-l",
]
out, rc = utils.execCmd(cmdList=cmd) #, "-t", "http_port_t"])
if rc:
return False
httpPortsList = []
pattern = re.compile("^http_port_t\s*tcp\s*([0-9, \-]*)$")
for line in out.splitlines():
httpPortPolicy = re.match(pattern, line)
if httpPortPolicy:
httpPortsList = parsePorts(httpPortPolicy.groups()[0])
logging.debug("http_port_t = %s"%(httpPortsList))
if newPort in httpPortsList:
return True
else:
cmd = [
basedefs.EXEC_SEMANAGE,
"port",
"-a",
"-t", "http_port_t",
"-p", "tcp",
"%d"%(newPort),
]
out, rc = utils.execCmd(cmdList=cmd, failOnError=False, usePipeFiles=True)
if rc:
logging.error(out)
return False
return True
def validateRemotePort(param, options = []):
#Validate that the port is an integer betweeen 1024 and 65535
logging.debug("Validating %s as a valid TCP Port" % (param))
if validateInteger(param, options):
port = int(param)
if (port > 0 and port < 65535):
return True
else:
logging.warn("validatePort('%s') - failed" %(param))
print output_messages.INFO_VAL_PORT_NOT_RANGE
return False
def validateStringNotEmpty(param, options=[]):
if type(param) != types.StringType or len(param) == 0:
logging.warn("validateStringNotEmpty('%s') - failed" %(param))
print output_messages.INFO_VAL_STRING_EMPTY %(param)
return False
else:
return True
def validateOptions(param, options=[]):
logging.info("Validating %s as part of %s"%(param, options))
if not validateStringNotEmpty(param, options):
return False
if "yes" in options and param.lower() == "y":
return True
if "no" in options and param.lower() == "n":
return True
if param.lower() in [option.lower() for option in options]:
return True
print output_messages.INFO_VAL_NOT_IN_OPTIONS % (", ".join(options))
return False
def validateDomain(param, options=[]):
"""
Validate domain name
"""
logging.info("validating %s as a valid domain string" % (param))
(errMsg, rc) = _validateString(param, 1, 1024, "^[\w\-\_]+\.[\w\.\-\_]+\w+$")
# Right now we print a generic error, might want to change it in the future
if rc != 0:
print output_messages.INFO_VAL_NOT_DOMAIN
return False
else:
return True
def validateUser(param, options=[]):
"""
Validate Auth Username
Setting a logical max value of 256
"""
logging.info("validating %s as a valid user name" % (param))
(errMsg, rc) = _validateString(param, 1, 256, "^\w[\w\.\-\_\%\@]{2,}$")
# Right now we print a generic error, might want to change it in the future
if rc != 0:
print output_messages.INFO_VAL_NOT_USER
return False
else:
return True
def validateRemoteHost(param, options=[]):
""" Validate that the we are working with remote DB host
"""
# If we received localhost, use default flow.
# If not local, REMOTE_DB group is run.
# It means returning True if remote, and False if local
if "DB_REMOTE_INSTALL" in param.keys() and param["DB_REMOTE_INSTALL"] == "remote":
return True
else:
return False
def validateFQDN(param, options=[]):
logging.info("Validating %s as a FQDN"%(param))
if not validateDomain(param,options):
return False
try:
#get set of IPs
ipAddresses = utils.getConfiguredIps()
if len(ipAddresses) < 1:
logging.error("Could not find any configured IP address on the host")
raise Exception(output_messages.ERR_EXP_CANT_FIND_IP)
#resolve fqdn
pattern = 'Address: (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
resolvedAddresses = _getPatternFromNslookup(param, pattern)
if len(resolvedAddresses) < 1:
logging.error("Failed to resolve %s"%(param))
print output_messages.ERR_DIDNT_RESOLVED_IP%(param)
return False
#string is generated here since we use it in all latter error messages
prettyString = " ".join(["%s"%string for string in resolvedAddresses])
#compare found IP with list of local IPs and match.
if not resolvedAddresses.issubset(ipAddresses):
logging.error("the following address(es): %s are not configured on this host"%(prettyString))
#different grammar for plural and single
if len(resolvedAddresses) > 1:
print output_messages.ERR_IPS_NOT_CONFIGED%(prettyString, param)
else:
print output_messages.ERR_IPS_NOT_CONFIGED_ON_INT%(prettyString, param)
return False
#reverse resolved IP and compare with given fqdn
counter = 0
pattern = '[\w\.-]+\s+name\s\=\s([\w\.\-]+)\.'
for address in resolvedAddresses:
addressSet = _getPatternFromNslookup(address, pattern)
reResolvedAddress = None
if len(addressSet) > 0:
reResolvedAddress = addressSet.pop()
if reResolvedAddress == param:
counter += 1
else:
logging.warn("%s did not reverse-resolve into %s"%(address,param))
if counter < 1:
logging.error("The following addresses: %s did not reverse resolve into %s"%(prettyString, param))
#different grammar for plural and single
if len(resolvedAddresses) > 1:
print output_messages.ERR_IPS_HAS_NO_PTR%(prettyString, param)
else:
print output_messages.ERR_IP_HAS_NO_PTR%(prettyString, param)
return False
#conditions passed
return True
except:
logging.error(traceback.format_exc())
raise
def validateFile(param, options=[]):
"""
Check that provided param is a file
"""
if not validateStringNotEmpty(param):
return False
if not os.path.isfile(param):
print "\n" + output_messages.ERR_FILE + ".\n"
return False
return True
def validatePing(param, options=[]):
"""
Check that provided host answers to ping
"""
if validateStringNotEmpty(param):
cmd = [
"/bin/ping",
"-c", "1",
"%s" % param,
]
out, rc = utils.execCmd(cmdList=cmd)
if rc == 0:
return True
print "\n" + output_messages.ERR_PING + " %s .\n"%param
return False
def validateMultiPing(param, options=[]):
if validateStringNotEmpty(param):
hosts = param.split(",")
for host in hosts:
if validatePing(host.strip()) == False:
return False
return True
print "\n" + output_messages.ERR_PING + ".\n"
return False
def _validateString(string, minLen, maxLen, regex=".*"):
"""
Generic func to verify a string
match its min/max length
and doesn't contain illegal chars
The func returns various return codes according to the error
plus a default error message
the calling func can decide if to use to default error msg
or to use a more specific one according the RC.
Return codes:
1 - string length is less than min
2 - string length is more tham max
3 - string contain illegal chars
0 - success
"""
# String length is less than minimum allowed
if len(string) < minLen:
msg = output_messages.INFO_STRING_LEN_LESS_THAN_MIN % (minLen)
return(msg, 1)
# String length is more than max allowed
elif len(string) > maxLen:
msg = output_messages.INFO_STRING_EXCEEDS_MAX_LENGTH % (maxLen)
return(msg, 2)
# String contains illegal chars
elif not utils.verifyStringFormat(string, regex):
return(output_messages.INFO_STRING_CONTAINS_ILLEGAL_CHARS, 3)
else:
# Success
return (None, 0)
def _getPatternFromNslookup(address, pattern):
rePattern = re.compile(pattern)
addresses = set()
output = utils.nslookup(address)
list = output.splitlines()
#do not go over the first 2 lines in nslookup output
for line in list[2:]:
found = rePattern.search(line)
if found:
foundAddress = found.group(1)
logging.debug("%s resolved into %s"%(address, foundAddress))
addresses.add(foundAddress)
return addresses
def _getBasePath(path):
if os.path.exists(path):
return path
# Iterate up in the tree structure until we get an
# existing path
return _getBasePath(os.path.dirname(path.rstrip("/")))
def _isPathWriteable(path):
try:
logging.debug("attempting to write temp file to %s" % (path))
tempfile.TemporaryFile(dir=path)
return True
except:
logging.warning(traceback.format_exc())
logging.warning("%s is not writeable" % path)
return False
def r_validateIF(server, device):
""" Validate that a network interface exists on a remote host """
server.append("ifconfig %s || ( echo Device %s does not exist && exit 1 )"%(device, device))
def r_validateDevice(server, device=None):
if device:
# the device MUST exist
server.append('ls -l /dev/%s'%device)
# if it is not mounted then we can use it
server.append('grep "/dev/%s " /proc/self/mounts || exit 0'%device)
# if it is mounted then the mount point has to be in /srv/node
server.append('grep "/dev/%s /srv/node" /proc/self/mounts && exit 0'%device)
# if we got here without exiting then we can't use this device
server.append('exit 1')
return False

75
installer/output_messages.py Normal file
View File

@@ -0,0 +1,75 @@
'''
external text file to hold all user visible text.
info messages begins with INFO_ and error msg with ERR_
any text with %s inside it, has dynamic parameters inside.
please don't remove the %s from the text.
you can relocate %s position in the text as long as the context is kept.
\n means new line in the text
\ at the end of a line lets you continue the text in a new line
DONT CHANGE any of the params names (in UPPER-CASE)
they are used in the engine-setup.py
'''
import basedefs
#####################
####INFO MESSAGES####
#####################
INFO_HEADER="Welcome to %s setup utility" % basedefs.APP_NAME
INFO_INSTALL_SUCCESS="\n **** Installation completed successfully ******\n\n (Please allow %s a few moments to start up.....)\n" % basedefs.APP_NAME
INFO_INSTALL="Installing:"
INFO_DSPLY_PARAMS="\n%s will be installed using the following configuration:" % basedefs.APP_NAME
INFO_USE_PARAMS="Proceed with the configuration listed above"
INFO_DONE="DONE"
INFO_ERROR="ERROR"
INFO_LOG_FILE_PATH="The installation log file is available at: %s"
INFO_ADDTIONAL_MSG="Additional information:"
INFO_ADDTIONAL_MSG_BULLET=" * %s"
INFO_CONF_PARAMS_PASSWD_CONFIRM_PROMPT="Confirm password"
INFO_VAL_PATH_SPACE="Error: mount point %s contains only %s of available space while a minimum of %s is required"
INFO_VAL_NOT_INTEGER="Error: value is not an integer"
INFO_VAL_PORT_NOT_RANGE="Error: port is outside the range of %i - 65535"
INFO_VAL_STRING_EMPTY="Warning: The %s parameter is empty"
INFO_VAL_NOT_IN_OPTIONS="Error: response is not part of the following accepted answers: %s"
INFO_VAL_NOT_DOMAIN="Error: domain is not a valid domain name"
INFO_VAL_NOT_USER="Error: user name contains illegal characters"
INFO_VAL_PORT_OCCUPIED="Error: TCP Port %s is already open by %s (pid: %s)"
INFO_VAL_PORT_OCCUPIED_BY_JBOSS="Error: TCP Port %s is used by JBoss"
INFO_VAL_PASSWORD_DONT_MATCH="Error: passwords don't match"
INFO_STRING_LEN_LESS_THAN_MIN="String length is less than the minimum allowed: %s"
INFO_STRING_EXCEEDS_MAX_LENGTH="String length exceeds the maximum length allowed: %s"
INFO_STRING_CONTAINS_ILLEGAL_CHARS="String contains illegal characters"
WARN_WEAK_PASS="Warning: Weak Password."
ERR_PING = "Error: the provided hostname is unreachable"
ERR_FILE = "Error: the provided file is not present"
ERR_CHECK_LOG_FILE_FOR_MORE_INFO="Please check log file %s for more information"
ERR_YUM_LOCK="Internal Error: Can't edit versionlock "
ERR_FAILED_START_SERVICE = "Error: Can't start the %s service"
ERR_FAILED_STOP_SERVICE = "Error: Can't stop the %s service"
ERR_EXP_HANDLE_PARAMS="Failed handling user parameters input"
ERR_EXP_KEYBOARD_INTERRUPT="Keyboard interrupt caught."
ERR_READ_RPM_VER="Error reading version number for package %s"
ERR_EXP_READ_INPUT_PARAM="Error while trying to read parameter %s from user."
ERR_EXP_HANDLE_ANSWER_FILE="Failed handling answer file: %s"
ERR_EXP_GET_CFG_IPS="Could not get list of available IP addresses on this host"
ERR_EXP_GET_CFG_IPS_CODES="Failed to get list of IP addresses"
ERR_EXP_CANT_FIND_IP="Could not find any configured IP address"
ERR_DIDNT_RESOLVED_IP="%s did not resolve into an IP address"
ERR_IPS_NOT_CONFIGED="Some or all of the IP addresses: (%s) which were resolved from the FQDN %s are not configured on any interface on this host"
ERR_IPS_NOT_CONFIGED_ON_INT="The IP (%s) which was resolved from the FQDN %s is not configured on any interface on this host"
ERR_IPS_HAS_NO_PTR="None of the IP addresses on this host(%s) holds a PTR record for the FQDN: %s"
ERR_IP_HAS_NO_PTR="The IP %s does not hold a PTR record for the FQDN: %s"
ERR_EXP_FAILED_INIT_LOGGER="Unexpected error: Failed to initiate logger, please check file system permission"
ERR_RC_CODE="Return Code is not zero"
ERR_FAILURE="General failure"
ERR_NO_ANSWER_FILE="Error: Could not find file %s"
#
INFO_KEYSTONERC="To use the command line tools simply source the keystonerc_* files created here"

791
installer/run_setup.py Executable file
View File

@@ -0,0 +1,791 @@
#!/usr/bin/python
import ConfigParser
import copy
import getpass
import logging
import os
import re
import sys
from StringIO import StringIO
import traceback
import types
import uuid
from optparse import OptionParser, OptionGroup
import basedefs
import common_utils as utils
import engine_validators as validate
import output_messages
from setup_controller import Controller
controller = Controller()
logFile = os.path.join(basedefs.DIR_LOG,basedefs.FILE_INSTALLER_LOG)
commandLineValues = {}
# List to hold all values to be masked in logging (i.e. passwords and sensitive data)
#TODO: read default values from conf_param?
masked_value_set = set()
def initLogging():
global logFile
try:
#in order to use UTC date for the log file, send True to getCurrentDateTime(True)
logFilename = "openstack-setup_%s.log" %(utils.getCurrentDateTime())
logFile = os.path.join(basedefs.DIR_LOG,logFilename)
if not os.path.isdir(os.path.dirname(logFile)):
os.makedirs(os.path.dirname(logFile))
level = logging.INFO
level = logging.DEBUG
hdlr = logging.FileHandler(filename = logFile, mode='w')
fmts='%(asctime)s::%(levelname)s::%(module)s::%(lineno)d::%(name)s:: %(message)s'
dfmt='%Y-%m-%d %H:%M:%S'
fmt = logging.Formatter(fmts, dfmt)
hdlr.setFormatter(fmt)
logging.root.addHandler(hdlr)
logging.root.setLevel(level)
except:
logging.error(traceback.format_exc())
raise Exception(output_messages.ERR_EXP_FAILED_INIT_LOGGER)
def initSequences():
sequences_conf = [
{ 'description' : 'Initial Steps',
'condition' : [],
'condition_match' : [],
'steps' : [ { 'title' : "Noop",
'functions' : [] },]
},
]
for item in sequences_conf:
controller.addSequence(item['description'], item['condition'], item['condition_match'], item['steps'])
def initConfig():
"""
Initialization of configuration
"""
"""
Param Fields:
CMD_OPTION - the command line flag to use for this option
USAGE - usage to display to the user
PROMPT - text to prompt the user with when querying this param
OPTION_LIST - if set, let the user only choose from this list as answer
VALIDATION_FUNC - Validation function for this param
DEFAULT_VALUE - the default value of this param
MASK_INPUT - should we mask the value of this param in the logs?
LOOSE_VALIDATION - (True/False) if True, and validation failed, let the user use the failed value
CONF_NAME - Name of param, must be unique, used as key
USE_DEFAULT - (True/False) Should we use the default value instead of querying the user?
NEED_CONFIRM - (True/False) Do we require the user to confirm the input(used in password fields)
CONDITION - (True/False) is this a condition for a group?
"""
conf_params = {
}
"""
Group fields:
GROUP_NAME - Name of group, used as key
DESCRIPTION - Used to prompt the user when showing the command line options
PRE_CONDITION - Condition to match before going over all params in the group, if fails, will not go into group
PRE_CONDITION_MATCH - Value to match condition with
POST_CONDITION - Condition to match after all params in the groups has been queried. if fails, will re-query all parameters
POST_CONDITION_MATCH - Value to match condition with
"""
conf_groups = (
)
for group in conf_groups:
paramList = conf_params[group["GROUP_NAME"]]
controller.addGroup(group, paramList)
def _getInputFromUser(param):
"""
this private func reads the data from the user
for the given param
"""
loop = True
userInput = None
try:
if param.getKey("USE_DEFAULT"):
logging.debug("setting default value (%s) for key (%s)" % (mask(param.getKey("DEFAULT_VALUE")), param.getKey("CONF_NAME")))
controller.CONF[param.getKey("CONF_NAME")] = param.getKey("DEFAULT_VALUE")
else:
while loop:
# If the value was not supplied by the command line flags
if not commandLineValues.has_key(param.getKey("CONF_NAME")):
message = StringIO()
message.write(param.getKey("PROMPT"))
if type(param.getKey("OPTION_LIST")) == types.ListType and len(param.getKey("OPTION_LIST")) > 0:
message.write(" %s" % (str(param.getKey("OPTION_LIST")).replace(',', '|')))
if param.getKey("DEFAULT_VALUE"):
message.write(" [%s] " % (str(param.getKey("DEFAULT_VALUE"))))
message.write(": ")
message.seek(0)
#mask password or hidden fields
if (param.getKey("MASK_INPUT")):
userInput = getpass.getpass("%s :" % (param.getKey("PROMPT")))
else:
userInput = raw_input(message.read())
else:
userInput = commandLineValues[param.getKey("CONF_NAME")]
# If DEFAULT_VALUE is set and user did not input anything
if userInput == "" and len(param.getKey("DEFAULT_VALUE")) > 0:
userInput = param.getKey("DEFAULT_VALUE")
# If param requires validation
if param.getKey("VALIDATION_FUNC")(userInput, param.getKey("OPTION_LIST")):
if "yes" in param.getKey("OPTION_LIST") and userInput.lower() == "y":
userInput = "yes"
if "no" in param.getKey("OPTION_LIST") and userInput.lower() == "n":
userInput = "no"
controller.CONF[param.getKey("CONF_NAME")] = userInput
loop = False
# If validation failed but LOOSE_VALIDATION is true, ask user
elif param.getKey("LOOSE_VALIDATION"):
answer = _askYesNo("User input failed validation, do you still wish to use it")
if answer:
loop = False
controller.CONF[param.getKey("CONF_NAME")] = userInput
else:
if commandLineValues.has_key(param.getKey("CONF_NAME")):
del commandLineValues[param.getKey("CONF_NAME")]
loop = True
else:
# Delete value from commandLineValues so that we will prompt the user for input
if commandLineValues.has_key(param.getKey("CONF_NAME")):
del commandLineValues[param.getKey("CONF_NAME")]
loop = True
except KeyboardInterrupt:
print "" # add the new line so messages wont be displayed in the same line as the question
raise KeyboardInterrupt
except:
logging.error(traceback.format_exc())
raise Exception(output_messages.ERR_EXP_READ_INPUT_PARAM % (param.getKey("CONF_NAME")))
def input_param(param):
"""
this func will read input from user
and ask confirmation if needed
"""
# We need to check if a param needs confirmation, (i.e. ask user twice)
# Do not validate if it was given from the command line
if (param.getKey("NEED_CONFIRM") and not commandLineValues.has_key(param.getKey("CONF_NAME"))):
#create a copy of the param so we can call it twice
confirmedParam = copy.deepcopy(param)
confirmedParamName = param.getKey("CONF_NAME") + "_CONFIRMED"
confirmedParam.setKey("CONF_NAME", confirmedParamName)
confirmedParam.setKey("PROMPT", output_messages.INFO_CONF_PARAMS_PASSWD_CONFIRM_PROMPT)
confirmedParam.setKey("VALIDATION_FUNC", validate.validateStringNotEmpty)
# Now get both values from user (with existing validations
while True:
_getInputFromUser(param)
_getInputFromUser(confirmedParam)
if controller.CONF[param.getKey("CONF_NAME")] == controller.CONF[confirmedParamName]:
logging.debug("Param confirmation passed, value for both questions is identical")
break
else:
print output_messages.INFO_VAL_PASSWORD_DONT_MATCH
else:
_getInputFromUser(param)
return param
def _askYesNo(question=None):
message = StringIO()
askString = "%s? (yes|no): "%(question)
logging.debug("asking user: %s"%askString)
message.write(askString)
message.seek(0)
rawAnswer = raw_input(message.read())
logging.debug("user answered: %s"%(rawAnswer))
answer = rawAnswer.lower()
if answer == "yes" or answer == "y":
return True
elif answer == "no" or answer == "n":
return False
else:
return _askYesNo(question)
def _addDefaultsToMaskedValueSet():
"""
For every param in conf_params
that has MASK_INPUT enabled keep the default value
in the 'masked_value_set'
"""
global masked_value_set
for group in controller.getAllGroups():
for param in group.getAllParams():
# Keep default password values masked, but ignore default empty values
if ((param.getKey("MASK_INPUT") == True) and param.getKey("DEFAULT_VALUE") != ""):
masked_value_set.add(param.getKey("DEFAULT_VALUE"))
def _updateMaskedValueSet():
"""
For every param in conf
has MASK_INPUT enabled keep the user input
in the 'masked_value_set'
"""
global masked_value_set
for confName in controller.CONF:
# Add all needed values to masked_value_set
if (controller.getParamKeyValue(confName, "MASK_INPUT") == True):
masked_value_set.add(controller.CONF[confName])
def mask(input):
"""
Gets a dict/list/str and search maksked values in them.
The list of masked values in is masked_value_set and is updated
via the user input
If it finds, it replaces them with '********'
"""
output = copy.deepcopy(input)
if type(input) == types.DictType:
for key in input:
if type(input[key]) == types.StringType:
output[key] = maskString(input[key])
if type(input) == types.ListType:
for item in input:
org = item
orgIndex = input.index(org)
if type(item) == types.StringType:
item = maskString(item)
if item != org:
output.remove(org)
output.insert(orgIndex, item)
if type(input) == types.StringType:
output = maskString(input)
return output
def removeMaskString(maskedString):
"""
remove an element from masked_value_set
we need to itterate over the set since
calling set.remove() on an string that does not exit
will raise an exception
"""
global masked_value_set
# Since we cannot remove an item from a set during itteration over
# the said set, we only mark a flag and if the flag is set to True
# we remove the string from the set.
found = False
for item in masked_value_set:
if item == maskedString:
found = True
if found:
masked_value_set.remove(maskedString)
def maskString(str):
# Iterate sorted list, so we won't mask only part of a password
for password in sorted(masked_value_set, utils.byLength, None, True):
if password:
str = str.replace(password, '*'*8)
return str
def _validateParamValue(param, paramValue):
validateFunc = param.getKey("VALIDATION_FUNC")
optionsList = param.getKey("OPTION_LIST")
logging.debug("validating param %s in answer file." % param.getKey("CONF_NAME"))
if not validateFunc(paramValue, optionsList):
raise Exception(output_messages.ERR_EXP_VALIDATE_PARAM % param.getKey("CONF_NAME"))
def _handleGroupCondition(config, conditionName, conditionValue):
"""
handle params group pre/post condition
checks if a group has a pre/post condition
and validates the params related to the group
"""
# If the post condition is a function
if type(conditionName) == types.FunctionType:
# Call the function conditionName with conf as the arg
conditionValue = conditionName(controller.CONF)
# If the condition is a string - just read it to global conf
# We assume that if we get a string as a member it is the name of a member of conf_params
elif type(conditionName) == types.StringType:
conditionValue = _loadParamFromFile(config, "general", conditionName)
else:
# Any other type is invalid
raise TypeError("%s type (%s) is not supported" % (conditionName, type(conditionName)))
return conditionValue
def _loadParamFromFile(config, section, paramName):
"""
read param from file
validate it
and load to to global conf dict
"""
# Get paramName from answer file
value = config.get(section, paramName)
# Validate param value using its validation func
param = controller.getParamByName(paramName)
_validateParamValue(param, value)
# Keep param value in our never ending global conf
controller.CONF[param.getKey("CONF_NAME")] = value
return value
def _handleAnswerFileParams(answerFile):
"""
handle loading and validating
params from answer file
supports reading single or group params
"""
try:
logging.debug("Starting to handle config file")
# Read answer file
fconf = ConfigParser.ConfigParser()
fconf.read(answerFile)
# Iterate all the groups and check the pre/post conditions
for group in controller.getAllGroups():
# Get all params per group
# Handle pre conditions for group
preConditionValue = True
if group.getKey("PRE_CONDITION"):
preConditionValue = _handleGroupCondition(fconf, group.getKey("PRE_CONDITION"), preConditionValue)
# Handle pre condition match with case insensitive values
logging.info("Comparing pre- conditions, value: '%s', and match: '%s'" % (preConditionValue, group.getKey("PRE_CONDITION_MATCH")))
if utils.compareStrIgnoreCase(preConditionValue, group.getKey("PRE_CONDITION_MATCH")):
for param in group.getAllParams():
_loadParamFromFile(fconf, "general", param.getKey("CONF_NAME"))
# Handle post conditions for group only if pre condition passed
postConditionValue = True
if group.getKey("POST_CONDITION"):
postConditionValue = _handleGroupCondition(fconf, group.getKey("POST_CONDITION"), postConditionValue)
# Handle post condition match for group
if not utils.compareStrIgnoreCase(postConditionValue, group.getKey("POST_CONDITION_MATCH")):
logging.error("The group condition (%s) returned: %s, which differs from the excpeted output: %s"%\
(group.getKey("GROUP_NAME"), postConditionValue, group.getKey("POST_CONDITION_MATCH")))
raise ValueError(output_messages.ERR_EXP_GROUP_VALIDATION_ANS_FILE%\
(group.getKey("GROUP_NAME"), postConditionValue, group.getKey("POST_CONDITION_MATCH")))
else:
logging.debug("condition (%s) passed" % group.getKey("POST_CONDITION"))
else:
logging.debug("no post condition check for group %s" % group.getKey("GROUP_NAME"))
else:
logging.debug("skipping params group %s since value of group validation is %s" % (group.getKey("GROUP_NAME"), preConditionValue))
except Exception as e:
logging.error(traceback.format_exc())
raise Exception(output_messages.ERR_EXP_HANDLE_ANSWER_FILE%(e))
def _handleInteractiveParams():
try:
for group in controller.getAllGroups():
preConditionValue = True
logging.debug("going over group %s" % group.getKey("GROUP_NAME"))
# If pre_condition is set, get Value
if group.getKey("PRE_CONDITION"):
preConditionValue = _getConditionValue(group.getKey("PRE_CONDITION"))
inputLoop = True
# If we have a match, i.e. condition returned True, go over all params in the group
logging.info("Comparing pre-conditions; condition: '%s', and match: '%s'" % (preConditionValue, group.getKey("PRE_CONDITION_MATCH")))
if utils.compareStrIgnoreCase(preConditionValue, group.getKey("PRE_CONDITION_MATCH")):
while inputLoop:
for param in group.getAllParams():
if not param.getKey("CONDITION"):
input_param(param)
#update password list, so we know to mask them
_updateMaskedValueSet()
postConditionValue = True
# If group has a post condition, we check it after we get the input from
# all the params in the group. if the condition returns False, we loop over the group again
if group.getKey("POST_CONDITION"):
postConditionValue = _getConditionValue(group.getKey("POST_CONDITION"))
if postConditionValue == group.getKey("POST_CONDITION_MATCH"):
inputLoop = False
else:
#we clear the value of all params in the group
#in order to re-input them by the user
for param in group.getAllParams():
if controller.CONF.has_key(param.getKey("CONF_NAME")):
del controller.CONF[param.getKey("CONF_NAME")]
if commandLineValues.has_key(param.getKey("CONF_NAME")):
del commandLineValues[param.getKey("CONF_NAME")]
else:
inputLoop = False
else:
logging.debug("no post condition check for group %s" % group.getKey("GROUP_NAME"))
_displaySummary()
except KeyboardInterrupt:
logging.error("keyboard interrupt caught")
raise Exception(output_messages.ERR_EXP_KEYBOARD_INTERRUPT)
except Exception:
logging.error(traceback.format_exc())
raise
except:
logging.error(traceback.format_exc())
raise Exception(output_messages.ERR_EXP_HANDLE_PARAMS)
def _handleParams(configFile):
_addDefaultsToMaskedValueSet()
if configFile:
_handleAnswerFileParams(configFile)
else:
_handleInteractiveParams()
def _getConditionValue(matchMember):
returnValue = False
if type(matchMember) == types.FunctionType:
returnValue = matchMember(controller.CONF)
elif type(matchMember) == types.StringType:
#we assume that if we get a string as a member it is the name
#of a member of conf_params
if not controller.CONF.has_key(matchMember):
param = controller.getParamByName(matchMember)
input_param(param)
returnValue = controller.CONF[matchMember]
else:
raise TypeError("%s type (%s) is not supported"%(matchMember, type(matchMember)))
return returnValue
def _displaySummary():
print output_messages.INFO_DSPLY_PARAMS
print "=" * (len(output_messages.INFO_DSPLY_PARAMS) - 1)
logging.info("*** User input summary ***")
for group in controller.getAllGroups():
for param in group.getAllParams():
if not param.getKey("USE_DEFAULT") and controller.CONF.has_key(param.getKey("CONF_NAME")):
cmdOption = param.getKey("CMD_OPTION")
l = 30 - len(cmdOption)
maskParam = param.getKey("MASK_INPUT")
# Only call mask on a value if the param has MASK_INPUT set to True
if maskParam:
logging.info("%s: %s" % (cmdOption, mask(controller.CONF[param.getKey("CONF_NAME")])))
print "%s:" % (cmdOption) + " " * l + mask(controller.CONF[param.getKey("CONF_NAME")])
else:
# Otherwise, log & display it as it is
logging.info("%s: %s" % (cmdOption, controller.CONF[param.getKey("CONF_NAME")]))
print "%s:" % (cmdOption) + " " * l + controller.CONF[param.getKey("CONF_NAME")]
logging.info("*** User input summary ***")
answer = _askYesNo(output_messages.INFO_USE_PARAMS)
if not answer:
logging.debug("user chose to re-enter the user parameters")
for group in controller.getAllGroups():
for param in group.getAllParams():
if controller.CONF.has_key(param.getKey("CONF_NAME")):
if not param.getKey("MASK_INPUT"):
param.setKey("DEFAULT_VALUE", controller.CONF[param.getKey("CONF_NAME")])
# Remove the string from mask_value_set in order
# to remove values that might be over overwritten.
removeMaskString(controller.CONF[param.getKey("CONF_NAME")])
del controller.CONF[param.getKey("CONF_NAME")]
if commandLineValues.has_key(param.getKey("CONF_NAME")):
del commandLineValues[param.getKey("CONF_NAME")]
print ""
logging.debug("calling handleParams in interactive mode")
return _handleParams(None)
else:
logging.debug("user chose to accept user parameters")
def _printAdditionalMessages():
print "\n",output_messages.INFO_ADDTIONAL_MSG
for msg in controller.MESSAGES:
logging.info(output_messages.INFO_ADDTIONAL_MSG_BULLET%(msg))
print output_messages.INFO_ADDTIONAL_MSG_BULLET%(msg)
def _addFinalInfoMsg():
"""
add info msg to the user finalizing the
successfull install of rhemv
"""
controller.MESSAGES.append(output_messages.INFO_LOG_FILE_PATH%(logFile))
controller.MESSAGES.append(output_messages.INFO_KEYSTONERC)
def _lockRpmVersion():
"""
Enters rpm versions into yum version-lock
"""
logging.debug("Locking rpms in yum-version-lock")
cmd = [
basedefs.EXEC_RPM, "-q",
] + basedefs.RPM_LOCK_LIST.split()
output, rc = utils.execCmd(cmdList=cmd, failOnError=True, msg=output_messages.ERR_YUM_LOCK)
with open(basedefs.FILE_YUM_VERSION_LOCK, "a") as f:
for rpm in output.splitlines():
f.write(rpm + "\n")
def _summaryParamsToLog():
if len(controller.CONF) > 0:
logging.debug("*** The following params were used as user input:")
for group in controller.getAllGroups():
for param in group.getAllParams():
if controller.CONF.has_key(param.getKey("CONF_NAME")):
maskedValue = mask(controller.CONF[param.getKey("CONF_NAME")])
logging.debug("%s: %s" % (param.getKey("CMD_OPTION"), maskedValue ))
def runSequences():
controller.runAllSequences()
def main(configFile=None):
try:
logging.debug("Entered main(configFile='%s')"%(configFile))
print output_messages.INFO_HEADER
# Get parameters
_handleParams(configFile)
# Update masked_value_list with user input values
_updateMaskedValueSet()
# Print masked conf
logging.debug(mask(controller.CONF))
# Start configuration stage
logging.debug("Entered Configuration stage")
print "\n",output_messages.INFO_INSTALL
# Initialize Sequences
initSequences()
initPluginsSequences()
# Run main setup logic
runSequences()
# Lock rhevm version
#_lockRpmVersion()
# Print info
_addFinalInfoMsg()
print output_messages.INFO_INSTALL_SUCCESS
_printAdditionalMessages()
finally:
# Always print user params to log
_summaryParamsToLog()
def generateAnswerFile(outputFile):
content = StringIO()
fd = open(outputFile,"w")
content.write("[general]%s"%(os.linesep))
for group in controller.getAllGroups():
for param in group.getAllParams():
content.write("%s=%s%s" % (param.getKey("CONF_NAME"), param.getKey("DEFAULT_VALUE"), os.linesep))
content.seek(0)
fd.write(content.read())
os.chmod(outputFile, 0600)
def initCmdLineParser():
"""
Initiate the optparse object, add all the groups and general command line flags
and returns the optparse object
"""
# Init parser and all general flags
logging.debug("initiating command line option parser")
usage = "usage: %prog [options]"
parser = OptionParser(usage)
parser.add_option("--gen-answer-file", help="Generate a template of an answer file, using this option excludes all other option")
parser.add_option("--answer-file", help="Runs the configuration in none-interactive mode, extracting all information from the \
configuration file. using this option excludes all other option")
parser.add_option("-o", "--options", action="store_true", dest="options", help="Print details on options available in answer file(rst format)")
# For each group, create a group option
for group in controller.getAllGroups():
groupParser = OptionGroup(parser, group.getKey("DESCRIPTION"))
for param in group.getAllParams():
cmdOption = param.getKey("CMD_OPTION")
paramUsage = param.getKey("USAGE")
optionsList = param.getKey("OPTION_LIST")
useDefault = param.getKey("USE_DEFAULT")
if not useDefault:
if optionsList:
groupParser.add_option("--%s" % cmdOption, metavar=optionsList, help=paramUsage, choices=optionsList)
else:
groupParser.add_option("--%s" % cmdOption, help=paramUsage)
# Add group parser to main parser
parser.add_option_group(groupParser)
return parser
def printOptions():
"""
print and document the available options to the answer file (rst format)
"""
# For each group, create a group option
for group in controller.getAllGroups():
print "%s"%group.getKey("DESCRIPTION")
print "-"*len(group.getKey("DESCRIPTION"))
print
for param in group.getAllParams():
cmdOption = param.getKey("CONF_NAME")
paramUsage = param.getKey("USAGE")
optionsList = param.getKey("OPTION_LIST") or ""
print "%s : %s %s"%(("**%s**"%str(cmdOption)).ljust(30), paramUsage, optionsList)
print
def plugin_compare(x, y):
"""
Used to sort the plugin file list
according to the number at the end of the plugin module
"""
x_match = re.search(".+\_(\d\d\d)", x)
x_cmp = x_match.group(1)
y_match = re.search(".+\_(\d\d\d)", y)
y_cmp = y_match.group(1)
return int(x_cmp) - int(y_cmp)
def loadPlugins():
"""
Load All plugins from ./plugins
"""
sys.path.append(basedefs.DIR_PLUGINS)
sys.path.append(basedefs.DIR_MODULES)
fileList = sorted(os.listdir(basedefs.DIR_PLUGINS), cmp=plugin_compare)
for item in fileList:
# Looking for files that end with ###.py, example: a_plugin_100.py
match = re.search("^(.+\_\d\d\d)\.py$", item)
if match:
try:
moduleToLoad = match.group(1)
logging.debug("importing module %s, from file %s", moduleToLoad, item)
moduleobj = __import__(moduleToLoad)
moduleobj.__file__ = os.path.join(basedefs.DIR_PLUGINS, item)
globals()[moduleToLoad] = moduleobj
checkPlugin(moduleobj)
controller.addPlugin(moduleobj)
except:
logging.error("Failed to load plugin from file %s", item)
logging.error(traceback.format_exc())
raise Exception("Failed to load plugin from file %s" % item)
def checkPlugin(plugin):
for funcName in ['initConfig','initSequences']:
if not hasattr(plugin, funcName):
raise ImportError("Plugin %s does not contain the %s function" % (plugin.__class__, funcName))
def countCmdLineFlags(options, flag):
"""
counts all command line flags that were supplied, excluding the supplied flag name
"""
counter = 0
# make sure only flag was supplied
for key, value in options.__dict__.items():
if key == flag:
next
# If anything but flag was called, increment
elif value:
counter += 1
return counter
def validateSingleFlag(options, flag):
counter = countCmdLineFlags(options, flag)
if counter > 0:
optParser.print_help()
print
#replace _ with - for printing's sake
raise Exception(output_messages.ERR_ONLY_1_FLAG % "--%s" % flag.replace("_","-"))
def initPluginsConfig():
for plugin in controller.getAllPlugins():
plugin.initConfig(controller)
def initPluginsSequences():
for plugin in controller.getAllPlugins():
plugin.initSequences(controller)
def initMain():
# Initialize logging
initLogging()
# Load Plugins
loadPlugins()
# Initialize configuration
initConfig()
initPluginsConfig()
if __name__ == "__main__":
try:
initMain()
runConfiguration = True
confFile = None
optParser = initCmdLineParser()
# Do the actual command line parsing
# Try/Except are here to catch the silly sys.exit(0) when calling rhevm-setup --help
(options, args) = optParser.parse_args()
if options.options:
printOptions()
raise SystemExit
# If --gen-answer-file was supplied, do not run main
if options.gen_answer_file:
# Make sure only --gen-answer-file was supplied
validateSingleFlag(options, "gen_answer_file")
generateAnswerFile(options.gen_answer_file)
# Otherwise, run main()
else:
# Make sure only --answer-file was supplied
if options.answer_file:
validateSingleFlag(options, "answer_file")
confFile = options.answer_file
if not os.path.exists(confFile):
raise Exception(output_messages.ERR_NO_ANSWER_FILE % confFile)
else:
for key, value in options.__dict__.items():
# Replace the _ with - in the string since optparse replace _ with -
for group in controller.getAllGroups():
param = group.getParams("CMD_OPTION", key.replace("_","-"))
if len(param) > 0 and value:
commandLineValues[param[0].getKey("CONF_NAME")] = value
main(confFile)
except SystemExit:
raise
except BaseException as e:
logging.error(traceback.format_exc())
print e
print output_messages.ERR_CHECK_LOG_FILE_FOR_MORE_INFO%(logFile)
sys.exit(1)

62
installer/sample-project/plugins/createfile_101.py Normal file
View File

@@ -0,0 +1,62 @@
"""
Creates a Sample File
"""
import logging
import os
import engine_validators as validate
import basedefs
import common_utils as utils
# Controller object will be initialized from main flow
controller = None
logging.debug("plugin %s loaded", __name__)
def initConfig(controllerObject):
global controller
controller = controllerObject
logging.debug("Initialising Plugine")
conf_params = {"SAMPLE": [
{"CMD_OPTION" : "filename",
"USAGE" : "File to create",
"PROMPT" : "File to create",
"OPTION_LIST" : [],
"VALIDATION_FUNC" : validate.validateStringNotEmpty,
"DEFAULT_VALUE" : "/tmp/samplefile.txt",
"MASK_INPUT" : False,
"LOOSE_VALIDATION": True,
"CONF_NAME" : "CONFIG_FILENAME",
"USE_DEFAULT" : False,
"NEED_CONFIRM" : False,
"CONDITION" : False },
]
}
conf_groups = [
{ "GROUP_NAME" : "SAMPLE",
"DESCRIPTION" : "Sample config group",
"PRE_CONDITION" : utils.returnYes,
"PRE_CONDITION_MATCH" : "yes",
"POST_CONDITION" : False,
"POST_CONDITION_MATCH" : True},
]
for group in conf_groups:
paramList = conf_params[group["GROUP_NAME"]]
controller.addGroup(group, paramList)
def initSequences(controller):
preparesteps = [
{'title': 'Create File', 'functions':[createfile]}
]
controller.addSequence("Creating File", [], [], preparesteps)
def createfile():
with open(controller.CONF["CONFIG_FILENAME"], "a") as fp:
fp.write("HELLO WORLD")

124
installer/setup_controller.py Normal file
View File

@@ -0,0 +1,124 @@
"""
Controller class is a SINGLETON which handles all groups, params, sequences,
steps and replaces the CONF dictionary.
"""
from setup_params import Group
from setup_sequences import Sequence
class Controller(object):
__GROUPS=[]
__SEQUENCES=[]
__PLUGINS=[]
MESSAGES=[]
CONF={}
__single = None # the one, true Singleton
def __new__(self, *args, **kwargs):
"""
Singleton implementation.
Will return __single if self is the same class as the class of __single
which means that we will not invoke this singleton if someone tries to create a new
instance from a class which inherit Controller.
did not use isinstance because inheritence makes it behave erratically.
"""
if self != type(self.__single):
self.__single = object.__new__(self, *args, **kwargs)
return self.__single
def __init__(self): pass
# PLugins
def addPlugin(self, plugObj):
self.__PLUGINS.append(plugObj)
def getPluginByName(self, pluginName):
for plugin in self.__PLUGINS:
if plugin.__name__ == pluginName:
return plugin
return None
def getAllPlugins(self):
return self.__PLUGINS
# Sequences and steps
def addSequence(self, desc, cond, cond_match, steps):
self.__SEQUENCES.append(Sequence(desc, cond, cond_match, steps))
def insertSequence(self, desc, cond, cond_match, steps, index=0):
self.__SEQUENCES.insert(index, Sequence(desc, cond, cond_match, steps))
def getAllSequences(self):
return self.__SEQUENCES
def runAllSequences(self):
for sequence in self.__SEQUENCES:
sequence.run()
def getSequenceByDesc(self, desc):
for sequence in self.getAllSequences():
if sequence.getDescription() == desc:
return sequence
return None
def __getSequenceIndexByDesc(self, desc):
for sequence in self.getAllSequences():
if sequence.getDescription() == desc:
return self.__SEQUENCES.index(sequence)
return None
def insertSequenceBeforeSequence(self, sequenceName, desc, cond, cond_match, steps):
"""
Insert a sequence before a named sequence.
i.e. if the specified sequence name is "update x", the new
sequence will be inserted BEFORE "update x"
"""
index = self.__getSequenceIndexByDesc(sequenceName)
if index == None:
index = len(self.getAllSequences())
self.__SEQUENCES.insert(index, Sequence(desc, cond, cond_match, steps))
# Groups and params
def addGroup(self, group, params):
self.__GROUPS.append(Group(group, params))
def getGroupByName(self, groupName):
for group in self.getAllGroups():
if group.getKey("GROUP_NAME") == groupName:
return group
return None
def getAllGroups(self):
return self.__GROUPS
def __getGroupIndexByDesc(self, name):
for group in self.getAllGroups():
if group.getKey("GROUP_NAME") == name:
return self.__GROUPS.index(group)
return None
def insertGroupBeforeGroup(self, groupName, group, params):
"""
Insert a group before a named group.
i.e. if the specified group name is "update x", the new
group will be inserted BEFORE "update x"
"""
index = self.__getGroupIndexByDesc(groupName)
if index == None:
index = len(self.getAllGroups())
self.__GROUPS.insert(index, Group(group, params))
def getParamByName(self, paramName):
for group in self.getAllGroups():
param = group.getParamByName(paramName)
if param:
return param
return None
def getParamKeyValue(self, paramName, keyName):
param = self.getParamByName(paramName)
if param:
return param.getKey(keyName)
else:
return None

81
installer/setup_params.py Normal file
View File

@@ -0,0 +1,81 @@
"""
Container set for groups and parameters
"""
class Param(object):
allowed_keys = ('CMD_OPTION','USAGE','PROMPT','OPTION_LIST',
'VALIDATION_FUNC','DEFAULT_VALUE','MASK_INPUT','LOOSE_VALIDATION',
'CONF_NAME','USE_DEFAULT','NEED_CONFIRM','CONDITION')
def __init__(self, attributes={}):
self.__ATTRIBUTES = {}
if attributes:
for key in self.allowed_keys:
self.__ATTRIBUTES[key] = attributes[key]
else:
self.__ATTRIBUTES = {}.fromkeys(self.allowed_keys)
def setKey(self, key, value):
self.validateKey(key)
self.__ATTRIBUTES[key] = value
def getKey(self, key):
self.validateKey(key)
return self.__ATTRIBUTES[key]
def validateKey(self, key):
if not self.__ATTRIBUTES.has_key(key):
raise KeyError("%s is not a valid key" % key)
class Group(Param):
allowed_keys = ('GROUP_NAME', 'DESCRIPTION', 'PRE_CONDITION', 'PRE_CONDITION_MATCH', 'POST_CONDITION', 'POST_CONDITION_MATCH')
def __init__(self, attributes={}, params=[]):
self.__PARAMS = []
Param.__init__(self, attributes)
for param in params:
self.addParam(param)
def addParam(self,paramDict):
p = Param(paramDict)
self.__PARAMS.append(p)
def getParamByName(self,paramName):
for param in self.__PARAMS:
if param.getKey("CONF_NAME") == paramName:
return param
return None
def getAllParams(self):
return self.__PARAMS
def getParams(self,paramKey, paramValue):
output = []
for param in self.__PARAMS:
if param.getKey(paramKey) == paramValue:
output.append(param)
return output
def __getParamIndexByDesc(self, name):
for param in self.getAllParams():
if param.getKey("CONF_NAME") == name:
return self.__PARAMS.index(param)
return None
def insertParamBeforeParam(self, paramName, param):
"""
Insert a param before a named param.
i.e. if the specified param name is "update x", the new
param will be inserted BEFORE "update x"
"""
index = self.__getParamIndexByDesc(paramName)
if index == None:
index = len(self.getAllParams())
self.__PARAMS.insert(index, Param(param))
def removeParamByName(self, paramName):
self.__removeParams("CONF_NAME", paramName)
def __removeParams(self, paramKey, paramValue):
list = self.getParams(paramKey, paramValue)
for item in list:
self.__PARAMS.remove(item)

164
installer/setup_sequences.py Normal file
View File

@@ -0,0 +1,164 @@
"""
Base class for steps & sequences
"""
import logging
import sys
import re
import string
import traceback
import basedefs
import output_messages
import common_utils as utils
class Step(object):
def __init__(self, title=None, functions=[]):
self.__TITLE = None
self.__FUNCTIONS = []
if title:
if not isinstance(title, str):
raise TypeError("step's title should be of string type instead of %s" % type(title))
if not isinstance(functions, list):
raise TypeError("step's function should be of list type instead of %s" % type(functions))
for function in functions:
if not callable(function):
raise TypeError("All parameters which pass as functions should be callable. %s is not callable" % function)
self.setTitle(title)
for function in functions:
self.addFunction(function)
def setTitle(self, title):
self.__TITLE = title
def getTitle(self):
return self.__TITLE
def addFunction(self, function):
self.__FUNCTIONS.append(function)
def removeFunction(self, function):
self.__FUNCTIONS.remove(function)
def getFunctions(self):
return self.__FUNCTIONS
def run(self):
# keep relative space
# allow newline chars in title. This is useful for plugins
alignedTitle = self.getTitle()
if re.search('\n', alignedTitle):
alignedTitle = self.getTitle().split('\n')[-1]
for color in basedefs.COLORS:
if color in alignedTitle:
alignedTitle = string.replace(alignedTitle, color, '')
spaceLen = basedefs.SPACE_LEN - len(alignedTitle)
print "%s..."%(self.getTitle()),
sys.stdout.flush()
for function in self.getFunctions():
try:
logging.debug("running %s"%(function.func_name))
function()
except:
logging.debug(traceback.format_exc())
print ("[ " + utils.getColoredText(output_messages.INFO_ERROR, basedefs.RED) + " ]").rjust(spaceLen)
raise
print ("[ " + utils.getColoredText(output_messages.INFO_DONE, basedefs.GREEN) + " ]").rjust(spaceLen)
class Sequence(object):
"""
Gets 4 parameters:
description, condition's name/function, condition's expected result and steps
steps should be a list of dictionaries, example:
[ { 'title' : 'step1's title',
'functions' : [ func1, func2, func3 ] },
{ 'title' : 'step2's tittle',
'functions' : [ func4, func6 ] } ]
"""
def __init__(self, desc=None, cond=[], cond_match=[], steps=[]):
self.__DESCRIPTION = None
self.__CONDITION = None
self.__COND_MATCH = None
self.__STEPS = []
self.setDescription(desc)
self.setCondition(cond, cond_match)
for step in steps:
if not isinstance(step, dict):
raise TypeError("step should be of dictionary type instead of %s" % type(step))
self.addStep(step['title'], step['functions'])
def addStep(self, title, functions):
self.__STEPS.append(Step(title, functions))
def setDescription(self, desc):
self.__DESCRIPTION = desc
def getDescription(self):
return self.__DESCRIPTION
def getSteps(self):
return self.__STEPS
def getStepByTitle(self, stepTitle):
for step in self.__STEPS:
if step.getTitle == stepTitle:
return step
return None
def setCondition(self, cond, cond_match):
for item in [cond, cond_match]:
if not isinstance(item, list):
raise TypeError("supplied parameter should be of list type instead of %s" % type(item))
self.__CONDITION = cond
self.__COND_MATCH = cond_match
def __validateCondition(self):
"""
Both _CONDITION & _COND_MATCH are lists.
if any of them is a function that needs to be run, the first member
of the list is the function and the rest of the members in that list
are the params for the said function
i.e. self._CONDITION = [function, arg1, arg2]
will be executed as function(arg1, arg2)
if the first member of the list is not a function. we handle it
as anything else (i.e. string/bool etc)
"""
if len(self.__CONDITION) < 1 and len(self.__COND_MATCH) < 1:
return True
condResult = None
condMatchResult = None
if callable(self.__CONDITION[0]):
condResult = self.__CONDITION[0](*self.__CONDITION[1:])
else:
condResult = self.__CONDITION[0]
if callable(self.__COND_MATCH[0]):
condMatchResult = self.__COND_MATCH[0](*self.__COND_MATCH[1:])
else:
condMatchResult = self.__COND_MATCH[0]
if condResult == condMatchResult:
return True
return False
def removeStepByTitle(self, stepTitle):
self.__STEPS.remove(stepTitle)
def run(self):
if self.__validateCondition():
for step in self.__STEPS:
step.run()
def runStepByTitle(self, stepTitle):
step = self.getStepByTitle(stepTitle)
step.run()
def listStepsByTitle(self):
output = []
for step in self.__STEPS:
output.append(step.getTitle())
return output