After much debugging of the subject bug, we've concluded that a
failure and an exception of OSError with EAGAIN from communicate()
indicates some monkey business. In the specific case that we're seeing
in Trove, it is because monkey patching didn't quite get done the way
it should.
In any event, retrying isn't the answer. This change reverts an
earlier change that retried the communicate() call and instead hurls
the OSError exception back to the caller, if we are out of retries. A
test case to fake the communicate() failure has been amended to ensure
that we handle this in what we believe now to be the right way.
Originally-Submitted-In: I2450d2dc425a3d29eaba4d5ff9badc4a992a0ec8
Change-Id: Ie36d77eebb2c20124a61ff6029c3b97bfffb9427
Closes-Bug: #1347337
Related-Bug: #1365736
It would help debugging considerably if a failure in execute()
generated a message (at a client specified logging level). Such a
message, generated by execute() would only occur if the user requested
it by specifying a parameter (log_errors), and execute() were to throw
an exception. As an example, this change would address the issue being
addressed in bug 1340124 for the Trove project only.
DocImpact: New parameter added to processutils.execute(). The
possible values of the log_errors parameter are 'all' or 'final'. The
default value of None implies no logging. If it is set to 'final',
stdout and stderr are logged on the final failure. If it is set to
'all', stdout and stderr are logged on every failure.
Originally-submitted-in: If8180e042e1a05374ef705f5873cc0981243619f
Change-Id: Id9e94bd8d009b8f134f0b681da5dec5717d662ff
DocImpact:
Closes-Bug: #1342857
Suggested-By: Peter Stachowski <peter@tesora.com>
Reported-By: Ionuț Arțăriși <iartarisi@suse.cz>
When a ProcessExecutionError is thrown by processutils.ssh_execute(),
the exception may contain information such as password. Upstream
applications that just log the message (as several appear to do) could
inadvertently expose these passwords to a user with read access to the
log files. It is therefore considered prudent to invoke
strutils.mask_password() on the command, stdout and stderr in the
exception. A test case has been added to ensure that all three are
properly masked.
An earlier commit (e8199714) failed to address ssh_execute(). This
change set addresses ssh_execute.
OSSA is aware of this change request.
Change-Id: I0db9e98cbeb2a5e6f9ae0074f24717aa91cfc238
Closes-Bug: #1343604
When a ProcessExecutionError is thrown by processutils.execute(), the
exception may contain information such as password. Upstream
applications that just log the message (as several appear to do) could
inadvertently expose these passwords to a user with read access to the
log files. It is therefore considered prudent to invoke
strutils.mask_password() on the command, stdout and stderr in the
exception. A test case has been added to ensure that all three are
properly masked.
OSSA is aware of this change request.
Originally-Submitted-In: I173dfb865e84eb7dee54a22c76db1e4f125a0a8a
Change-Id: Ie122db5f19802f519b96ed024ab3f2b5eede3eee
Closes-Bug: #1343604
This moves and renames nova.utils.cpu_count() utility method from nova
to oslo. Added in commit 75c96a48fc7e5dfb59d8258142b01422f81b0253, this
is used to set the default number of nova api/conductor workers if not
explicitly specified in nova.conf.
I'm moving this to oslo so that we can use the same default worker logic
in other projects, i.e. cinder, trove, glance and taskflow.
Closes-Bug: #1333370
Change-Id: I81748a15f8da85e287afdb5ec9de6369f8919ab0
Some of the projects might need to pass environment variables
to the processes they launch. For instance, if something needs
to rely on the process' output it should be able to pass LANG="C"
in order to set the correct localization.
This patch adds a env_variables parameter to processutils.execute()
which allows to pass news variables or replace the value of the existing
ones for the process.
Closes-bug: #1304284
Change-Id: If801e11391bf7cf66518533443c3b29af53653e5
Module openstack.common.test is obsolete, so we should use
oslotest library instead of it.
Modified tests and common database code, new requirement added.
Change-Id: I853e548f11a4c3785eaf75124510a6d789536634
Some gate failures showed that subprocess.Popen.communicate() may raise
an OSError where the proper response is to just call it again. This
patch adds that handling.
Change-Id: Ibca16d2141ca3aacacc73062757c97a3220bd02d
Closes-bug: #1271331
Using assertEqual/assertNotEqual to test if an element
is or is not None is too python2.4.
Our unit testing framework supports assertIsNone and assertIsNotNone
which were created for these types of tests.
Change-Id: I29f915ae79bf4f8e9298c5de5d6087828fb0ca4f
No need to set tabstop 189 times, this can be set in your vimrc file
instead. Also if set incorrectly gate (pep8 check) will catch your
mistakes.
Change-Id: Ic6f0c0ef94e8194a5c121598305d1ec3c74e4843
This patch replaces unicode function with six.text_type in tests.
Let test code don't break in Python3 environment.
See http://pythonhosted.org/six/#six.text_type
Change-Id: I27e0db2034da7898f713719cc81f1b6c1f1fdeea
This patch just adjust import order to follow PEP8 imports rule.
See http://www.python.org/dev/peps/pep-0008/#imports
Imports should be grouped in the following order:
standard library imports
related third party imports
local application/library specific imports
Change-Id: I6d92e9eda72d83cbf940b7699268f884bd4045c1
This patch fixes: if a process exits with returncode 0, the function
always returns without raising an exception, regardless of whether
its check_exit_code includes 0 or not.
Change-Id: Ibdf7599dd91949153629083e4c5a47d263d6d002
It is the first step to replace using tests.utils with openstack.common.test.
All these tests don't use mock objects, stubs, config files and use only
BaseTestCase class.
Change-Id: I511816b5c9e6c5c34ebff199296ee4fc8b84c672
bp: common-unit-tests
PEP 352 deprecated the message attribute of the BaseException class.
Using the message attribute will result in warnings like this:
DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
Using unicode(exc) is the suggested replacement.
Change-Id: Ibf3c56e4baa6ad83e2b95a948787e9d02cf074d4
Python3 enforces the distinction between byte strings
far more rigorously than Python 2 does; binary data
cannot be automatically coerced to or from text data.
Use six to provide a fake file object for textual data.
It provides an alias for StringIO.StringIO in python2
and io.StringIO in python3
Change-Id: I65897bb0cca2cbeb5819a769b98645c9eb066401
Signed-off-by: Chuck Short <chuck.short@canonical.com>
This is another example of something which was imported from nova
but where nova was never moved across to use the oslo version. The
code has since diverged. This review syncs up the two versions,
and will be followed by a nova review to move nova across to the
oslo implementation.
Unit test coverage for this new version is 83%. I want to improve
that, but I'll do it in a later review.
I will also need to update cinder, quantum and nova to this new
version of the code once this merges. I will do moniker as well
because I am a nice guy.
Change-Id: Ie0731c56c9aab547b5d5b905bf4ed8eff3eae3bd
Using the BaseTestCase across the tests in the tree lets us put in log
fixtures and consistently handle mox and stubout.
Part of blueprint grizzly-testtools.
Change-Id: Iba7eb2c63b0c514009b2c28e5930b27726a147b0
On the path to testr migration, we need to replace the unittest base classes
with testtools.
Replace tearDown with addCleanup, addCleanup is more resilient than tearDown.
The fixtures library has excellent support for managing and cleaning
tempfiles. Use it.
Replace skip_ with testtools.skipTest
Part of blueprint grizzly-testtools.
Change-Id: I45e11bbb1ff9b31f3278d3b016737dcb7850cd98
