openstack/openstacksdk
Code Issues Proposed changes

Rationalize logging helpers and docs

Browse Source 
shade and openstacksdk each have a logging doc and a logging setup helper
function. They both basically do the same thing, and we have a TODO item
about collapsing them.

This moves openstack.utils.enable_logging to openstack.enable_logging
(leaving behind a compat piece at openstack.utils.enable_logging) It
adds the shade functionality to it, and also changes the behavior to
match shade's WRT behavior when no parameters are passed (defaults to
logging to stdout)

Update the codebase to call openstack._log.setup_logging instead of
logging.getLogger directly, as setup_logging attaches a NullHandler by
default.

Collapse the docs into a single document.

There were only two places where openstacksdk was already logging to
something other than 'openstack'. Collapse those down to 'openstack'
until we come up with a reason to break them out more logically.

Change-Id: I45fd5ffd18255450d38a1f56c80f5c157ea19ae3
This commit is contained in:
Monty Taylor 2018-01-10 12:18:34 -06:00
parent 40d425c595
commit da2406bace
No known key found for this signature in database
GPG Key ID: 7BAE94BC7141A594
35 changed files with 400 additions and 404 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

100
README.rst
View File

@ -6,45 +6,95 @@ with OpenStack clouds. The project aims to provide a consistent and
complete set of interactions with OpenStack's many services, along with
complete documentation, examples, and tools.
It also contains a simple interface layer. Clouds can do many things, but
It also contains an abstraction interface layer. Clouds can do many things, but
there are probably only about 10 of them that most people care about with any
regularity. If you want to do complicated things, the per-service oriented
portions of the SDK are for you. However, if what you want is to be able to
portions of the SDK are for you. However, if what you want to be able to
write an application that talks to clouds no matter what crazy choices the
deployer has made in an attempt to be more hipster than their self-entitled
narcissist peers, then the ``openstack.cloud`` layer is for you.
narcissist peers, then the Cloud Abstraction layer is for you.
A Brief History
---------------
.. TODO(shade) This history section should move to the docs. We can put a
link to the published URL here in the README, but it's too long.
openstacksdk started its life as three different libraries: shade,
os-client-config and python-openstacksdk.
``shade`` started its life as some code inside of OpenStack Infra's nodepool
project, and as some code inside of Ansible. Ansible had a bunch of different
OpenStack related modules, and there was a ton of duplicated code. Eventually,
between refactoring that duplication into an internal library, and adding logic
and features that the OpenStack Infra team had developed to run client
applications at scale, it turned out that we'd written nine-tenths of what we'd
need to have a standalone library.
``shade`` started its life as some code inside of OpenStack Infra's `nodepool`_
project, and as some code inside of the `Ansible OpenStack Modules`_.
Ansible had a bunch of different OpenStack related modules, and there was a
ton of duplicated code. Eventually, between refactoring that duplication into
an internal library, and adding the logic and features that the OpenStack Infra
team had developed to run client applications at scale, it turned out that we'd
written nine-tenths of what we'd need to have a standalone library.
Because of its background from nodepool, shade contained abstractions to
work around deployment differences and is resource oriented rather than service
oriented. This allows a user to think about Security Groups without having to
know whether Security Groups are provided by Nova or Neutron on a given cloud.
On the other hand, as an interface that provides an abstraction, it deviates
from the published OpenStack REST API and adds its own opinions, which may not
get in the way of more advanced users with specific needs.
``os-client-config`` was a library for collecting client configuration for
using an OpenStack cloud in a consistent and comprehensive manner.
In parallel, the python-openstacksdk team was working on a library to expose
the OpenStack APIs to developers in a consistent and predictable manner. After
a while it became clear that there was value in both a high-level layer that
contains business logic, a lower-level SDK that exposes services and their
resources as Python objects, and also to be able to make direct REST calls
when needed with a properly configured Session or Adapter from python-requests.
This led to the merger of the three projects.
using an OpenStack cloud in a consistent and comprehensive manner, which
introduced the ``clouds.yaml`` file for expressing named cloud configurations.
The contents of the shade library have been moved into ``openstack.cloud``
and os-client-config has been moved in to ``openstack.config``. The next
release of shade will be a thin compatibility layer that subclasses the objects
from ``openstack.cloud`` and provides different argument defaults where needed
for compat. Similarly the next release of os-client-config will be a compat
``python-openstacksdk`` was a library that exposed the OpenStack APIs to
developers in a consistent and predictable manner.
After a while it became clear that there was value in both the high-level
layer that contains additional business logic and the lower-level SDK that
exposes services and their resources faithfully and consistently as Python
objects.
Even with both of those layers, it is still beneficial at times to be able to
make direct REST calls and to do so with the same properly configured
`Session`_ from `python-requests`_.
This led to the merge of the three projects.
The original contents of the shade library have been moved into
``openstack.cloud`` and os-client-config has been moved in to
``openstack.config``. The next release of shade will be a thin compatibility
layer that subclasses the objects from ``openstack.cloud`` and provides
different argument defaults where needed for compat.
Similarly the next release of os-client-config will be a compat
layer shim around ``openstack.config``.
.. note::
The ``openstack.cloud.OpenStackCloud`` object and the
``openstack.connection.Connection`` object are going to be merged. It is
recommended to not write any new code which consumes objects from the
``openstack.cloud`` namespace until that merge is complete.
.. _nodepool: https://docs.openstack.org/infra/nodepool/
.. _Ansible OpenStack Modules: http://docs.ansible.com/ansible/latest/list_of_cloud_modules.html#openstack
.. _Session: http://docs.python-requests.org/en/master/user/advanced/#session-objects
.. _python-requests: http://docs.python-requests.org/en/master/
openstack
=========
List servers using objects configured with the ``clouds.yaml`` file:
.. code-block:: python
import openstack
# Initialize and turn on debug logging
openstack.enable_logging(debug=True)
# Initialize cloud
conn = openstack.connect(cloud='mordred')
for server in conn.compute.servers():
print(server.to_dict())
openstack.config
================
@ -88,10 +138,10 @@ Create a server using objects configured with the ``clouds.yaml`` file:
.. code-block:: python
import openstack.cloud
import openstack
# Initialize and turn on debug logging
openstack.cloud.simple_logging(debug=True)
openstack.enable_logging(debug=True)
# Initialize cloud
# Cloud configs are read with openstack.config

1
SHADE-MERGE-TODO.rst
View File

@ -72,7 +72,6 @@ shade integration
* Investigate auto-generating the bulk of shade's API based on introspection of
SDK objects, leaving only the code with extra special logic in the shade
layer.
* Rationalize openstack.util.enable_logging and shade.simple_logging.
Service Proxies
---------------

137
doc/source/user/guides/logging.rst
View File

@ -1,56 +1,44 @@
=======
Logging
=======
Logging can save you time and effort when developing your code or looking
for help. If your code is not behaving how you expect it to, enabling and
configuring logging can quickly give you valuable insight into the root
cause of the issue. If you need help from the OpenStack community, the
logs can help the people there assist you.
.. note:: TODO(shade) This document is written from a shade POV. It needs to
be combined with the existing logging guide, but also the logging
systems need to be rationalized.
.. note:: By default, no logging is done.
`openstacksdk` uses `Python Logging`_. As `openstacksdk` is a library, it does
not configure logging handlers automatically, expecting instead for that to be
the purview of the consuming application.
Enable SDK Logging
------------------
Simple Usage
------------
To enable logging you use :func:`~openstack.utils.enable_logging`.
For consumers who just want to get a basic logging setup without thinking
about it too deeply, there is a helper method. If used, it should be called
before any other openstacksdk functionality.
The ``debug`` parameter controls the logging level. Set ``debug=True`` to
log debug and higher messages. Set ``debug=False`` to log warning and higher
messages.
.. autofunction:: openstack.enable_logging
To log debug and higher messages::
.. code-block:: python
import sys
from openstack import utils
utils.enable_logging(debug=True, stream=sys.stdout)
The ``path`` parameter controls the location of a log file. If set, this
parameter will send log messages to a file using a
:py:class:`~logging.FileHandler`.
To log messages to a file called ``openstack.log``::
from openstack import utils
utils.enable_logging(debug=True, path='openstack.log')
import openstack
openstack.enable_logging()
The ``stream`` parameter controls the stream where log message are written to.
If set to ``sys.stdout`` or ``sys.stderr``, this parameter will send log
messages to that stream using a :py:class:`~logging.StreamHandler`
It defaults to `sys.stdout` which will result in log messages being written
to STDOUT. It can be set to another output stream, or to ``None`` to disable
logging to the console.
To log messages to the console on ``stdout``::
import sys
from openstack import utils
utils.enable_logging(debug=True, stream=sys.stdout)
The ``path`` parameter sets up logging to log to a file. By default, if
``path`` is given and ``stream`` is not, logging will only go to ``path``.
You can combine the ``path`` and ``stream`` parameters to log to both places
simultaneously.
To log messages to a file called ``openstack.log`` and the console on
``stdout``::
``stdout``:
.. code-block:: python
import sys
from openstack import utils
@ -58,23 +46,68 @@ To log messages to a file called ``openstack.log`` and the console on
utils.enable_logging(debug=True, path='openstack.log', stream=sys.stdout)
Enable requests Logging
-----------------------
`openstack.enable_logging` also sets up a few other loggers and
squelches some warnings or log messages that are otherwise uninteresting or
unactionable by an openstacksdk user.
The SDK depends on a small number other libraries. Notably, it uses
`requests <https://pypi.python.org/pypi/requests>`_ for its transport layer.
To get even more information about the request/response cycle, you enable
logging of requests the same as you would any other library.
Advanced Usage
--------------
To log messages to the console on ``stdout``::
`openstacksdk` logs to a set of different named loggers.
import logging
import sys
Most of the logging is set up to log to the root ``openstack`` logger.
There are additional sub-loggers that are used at times, primarily so that a
user can decide to turn on or off a specific type of logging. They are listed
below.
logger = logging.getLogger('requests')
formatter = logging.Formatter(
'%(asctime)s %(levelname)s: %(name)s %(message)s')
console = logging.StreamHandler(sys.stdout)
console.setFormatter(formatter)
logger.setLevel(logging.DEBUG)
logger.addHandler(console)
openstack.config
Issues pertaining to configuration are logged to the ``openstack.config``
logger.
openstack.task_manager
`openstacksdk` uses a Task Manager to perform remote calls. The
``openstack.task_manager`` logger emits messages at the start and end
of each Task announcing what it is going to run and then what it ran and
how long it took. Logging ``openstack.task_manager`` is a good way to
get a trace of external actions `openstacksdk` is taking without full
`HTTP Tracing`_.
openstack.iterate_timeout
When `openstacksdk` needs to poll a resource, it does so in a loop that waits
between iterations and ultimately times out. The
``openstack.iterate_timeout`` logger emits messages for each iteration
indicating it is waiting and for how long. These can be useful to see for
long running tasks so that one can know things are not stuck, but can also
be noisy.
openstack.fnmatch
`openstacksdk` will try to use `fnmatch`_ on given `name_or_id` arguments.
It's a best effort attempt, so pattern misses are logged to
``openstack.fnmatch``. A user may not be intending to use an fnmatch
pattern - such as if they are trying to find an image named
``Fedora 24 [official]``, so these messages are logged separately.
.. _fnmatch: https://pymotw.com/2/fnmatch/
HTTP Tracing
------------
HTTP Interactions are handled by `keystoneauth`_. If you want to enable HTTP
tracing while using openstacksdk and are not using `openstack.enable_logging`,
set the log level of the ``keystoneauth`` logger to ``DEBUG``.
For more information see https://docs.openstack.org/keystoneauth/latest/using-sessions.html#logging
.. _keystoneauth: https://docs.openstack.org/keystoneauth/latest/
Python Logging
--------------
Python logging is a standard feature of Python and is documented fully in the
Python Documentation, which varies by version of Python.
For more information on Python Logging for Python v2, see
https://docs.python.org/2/library/logging.html.
For more information on Python Logging for Python v3, see
https://docs.python.org/3/library/logging.html.

3
doc/source/user/index.rst
View File

@ -22,8 +22,6 @@ These guides walk you through how to make use of the libraries we provide
to work with each OpenStack service. If you're looking for a cookbook
approach, this is where you'll want to begin.
.. TODO(shade) Merge guides/logging and logging
.. toctree::
:maxdepth: 1
@ -32,7 +30,6 @@ approach, this is where you'll want to begin.
Connect to an OpenStack Cloud Using a Config File <guides/connect_from_config>
Using Cloud Abstration Layer <usage>
Logging <guides/logging>
Shade Logging <logging>
Microversions <microversions>
Baremetal <guides/baremetal>
Block Storage <guides/block_storage>

99
doc/source/user/logging.rst
View File

@ -1,99 +0,0 @@
=======
Logging
=======
.. note:: TODO(shade) This document is written from a shade POV. It needs to
be combined with the existing logging guide, but also the logging
systems need to be rationalized.
`openstacksdk` uses `Python Logging`_. As `openstacksdk` is a library, it does
not configure logging handlers automatically, expecting instead for that to be
the purview of the consuming application.
Simple Usage
------------
For consumers who just want to get a basic logging setup without thinking
about it too deeply, there is a helper method. If used, it should be called
before any other `shade` functionality.
.. code-block:: python
import openstack.cloud
openstack.cloud.simple_logging()
`openstack.cloud.simple_logging` takes two optional boolean arguments:
debug
Turns on debug logging.
http_debug
Turns on debug logging as well as debug logging of the underlying HTTP calls.
`openstack.cloud.simple_logging` also sets up a few other loggers and
squelches some warnings or log messages that are otherwise uninteresting or
unactionable by a `openstack.cloud` user.
Advanced Usage
--------------
`openstack.cloud` logs to a set of different named loggers.
Most of the logging is set up to log to the root `openstack.cloud` logger.
There are additional sub-loggers that are used at times, primarily so that a
user can decide to turn on or off a specific type of logging. They are listed
below.
openstack.task_manager
`openstack.cloud` uses a Task Manager to perform remote calls. The
`openstack.cloud.task_manager` logger emits messages at the start and end
of each Task announcing what it is going to run and then what it ran and
how long it took. Logging `openstack.cloud.task_manager` is a good way to
get a trace of external actions `openstack.cloud` is taking without full
`HTTP Tracing`_.
openstack.cloud.exc
If `log_inner_exceptions` is set to True, `shade` will emit any wrapped
exception to the `openstack.cloud.exc` logger. Wrapped exceptions are usually
considered implementation details, but can be useful for debugging problems.
openstack.iterate_timeout
When `shade` needs to poll a resource, it does so in a loop that waits
between iterations and ultimately timesout. The
`openstack.iterate_timeout` logger emits messages for each iteration
indicating it is waiting and for how long. These can be useful to see for
long running tasks so that one can know things are not stuck, but can also
be noisy.
openstack.cloud.http
`shade` will sometimes log additional information about HTTP interactions
to the `openstack.cloud.http` logger. This can be verbose, as it sometimes
logs entire response bodies.
openstack.cloud.fnmatch
`shade` will try to use `fnmatch`_ on given `name_or_id` arguments. It's a
best effort attempt, so pattern misses are logged to
`openstack.cloud.fnmatch`. A user may not be intending to use an fnmatch
pattern - such as if they are trying to find an image named
``Fedora 24 [official]``, so these messages are logged separately.
.. _fnmatch: https://pymotw.com/2/fnmatch/
HTTP Tracing
------------
HTTP Interactions are handled by `keystoneauth`. If you want to enable HTTP
tracing while using `shade` and are not using `openstack.cloud.simple_logging`,
set the log level of the `keystoneauth` logger to `DEBUG`.
Python Logging
--------------
Python logging is a standard feature of Python and is documented fully in the
Python Documentation, which varies by version of Python.
For more information on Python Logging for Python v2, see
https://docs.python.org/2/library/logging.html.
For more information on Python Logging for Python v3, see
https://docs.python.org/3/library/logging.html.

78
doc/source/user/multi-cloud-demo.rst
View File

@ -62,10 +62,10 @@ Complete Example
.. code:: python
import openstack.cloud
import openstack
# Initialize and turn on debug logging
openstack.cloud.simple_logging(debug=True)
openstack.enable_logging(debug=True)
for cloud_name, region_name in [
('my-vexxhost', 'ca-ymq-1'),
@ -314,10 +314,10 @@ Complete Example Again
.. code:: python
import openstack.cloud
import openstack
# Initialize and turn on debug logging
openstack.cloud.simple_logging(debug=True)
openstack.enable_logging(debug=True)
for cloud_name, region_name in [
('my-vexxhost', 'ca-ymq-1'),
@ -346,27 +346,25 @@ Import the library
.. code:: python
import openstack.cloud
import openstack
Logging
=======
* `shade` uses standard python logging
* Special `openstack.cloud.request_ids` logger for API request IDs
* `simple_logging` does easy defaults
* `openstacksdk` uses standard python logging
* ``openstack.enable_logging`` does easy defaults
* Squelches some meaningless warnings
* `debug`
* Logs shade loggers at debug level
* Includes `openstack.cloud.request_ids` debug logging
* `http_debug` Implies `debug`, turns on HTTP tracing
.. code:: python
# Initialize and turn on debug logging
openstack.cloud.simple_logging(debug=True)
openstack.enable_logging(debug=True)
Example with Debug Logging
==========================
@ -375,8 +373,8 @@ Example with Debug Logging
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging(debug=True)
import openstack
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(
cloud='my-vexxhost', region_name='ca-ymq-1')
@ -389,8 +387,8 @@ Example with HTTP Debug Logging
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging(http_debug=True)
import openstack
openstack.enable_logging(http_debug=True)
cloud = openstack.openstack_cloud(
cloud='my-vexxhost', region_name='ca-ymq-1')
@ -486,10 +484,10 @@ Image and Flavor by Name or ID
.. code:: python
import openstack.cloud
import openstack
# Initialize and turn on debug logging
openstack.cloud.simple_logging(debug=True)
openstack.enable_logging(debug=True)
for cloud_name, region_name, image, flavor in [
('my-vexxhost', 'ca-ymq-1',
@ -533,10 +531,10 @@ Image and Flavor by Dict
.. code:: python
import openstack.cloud
import openstack
# Initialize and turn on debug logging
openstack.cloud.simple_logging(debug=True)
openstack.enable_logging(debug=True)
for cloud_name, region_name, image, flavor_id in [
('my-vexxhost', 'ca-ymq-1', 'Ubuntu 16.04.1 LTS [2017-03-03]',
@ -564,8 +562,8 @@ Munch Objects
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging(debug=True)
import openstack
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='zetta', region_name='no-osl1')
image = cloud.get_image('Ubuntu 14.04 (AMD64) [Local Storage]')
@ -596,10 +594,10 @@ Cleanup Script
.. code:: python
import openstack.cloud
import openstack
# Initialize and turn on debug logging
openstack.cloud.simple_logging(debug=True)
openstack.enable_logging(debug=True)
for cloud_name, region_name in [
('my-vexxhost', 'ca-ymq-1'),
@ -618,8 +616,8 @@ Normalization
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging()
import openstack
openstack.enable_logging()
cloud = openstack.openstack_cloud(cloud='fuga', region_name='cystack')
image = cloud.get_image(
@ -634,8 +632,8 @@ Strict Normalized Results
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging()
import openstack
openstack.enable_logging()
cloud = openstack.openstack_cloud(
cloud='fuga', region_name='cystack', strict=True)
@ -651,8 +649,8 @@ How Did I Find the Image Name for the Last Example?
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging()
import openstack
openstack.enable_logging()
cloud = openstack.openstack_cloud(cloud='fuga', region_name='cystack')
cloud.pprint([
@ -672,8 +670,8 @@ Added / Modified Information
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging(debug=True)
import openstack
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='my-citycloud', region_name='Buf1')
try:
@ -714,8 +712,8 @@ User Agent Info
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging(http_debug=True)
import openstack
openstack.enable_logging(http_debug=True)
cloud = openstack.openstack_cloud(
cloud='datacentred', app_name='AmazingApp', app_version='1.0')
@ -732,8 +730,8 @@ Uploading Large Objects
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging(debug=True)
import openstack
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='ovh', region_name='SBG1')
cloud.create_object(
@ -753,8 +751,8 @@ Uploading Large Objects
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging(debug=True)
import openstack
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='ovh', region_name='SBG1')
cloud.create_object(
@ -769,8 +767,8 @@ Service Conditionals
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging(debug=True)
import openstack
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='kiss', region_name='region1')
print(cloud.has_service('network'))
@ -783,8 +781,8 @@ Service Conditional Overrides
.. code:: python
import openstack.cloud
openstack.cloud.simple_logging(debug=True)
import openstack
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='rax', region_name='DFW')
print(cloud.has_service('network'))

2
examples/cloud/cleanup-servers.py
View File

@ -13,7 +13,7 @@
import openstack
# Initialize and turn on debug logging
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
for cloud_name, region_name in [
('my-vexxhost', 'ca-ymq-1'),

2
examples/cloud/create-server-dict.py
View File

@ -13,7 +13,7 @@
import openstack
# Initialize and turn on debug logging
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
for cloud_name, region_name, image, flavor_id in [
('my-vexxhost', 'ca-ymq-1', 'Ubuntu 16.04.1 LTS [2017-03-03]',

2
examples/cloud/create-server-name-or-id.py
View File

@ -13,7 +13,7 @@
import openstack
# Initialize and turn on debug logging
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
for cloud_name, region_name, image, flavor in [
('my-vexxhost', 'ca-ymq-1',

2
examples/cloud/debug-logging.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(
cloud='my-vexxhost', region_name='ca-ymq-1')

2
examples/cloud/find-an-image.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging()
openstack.enable_logging()
cloud = openstack.openstack_cloud(cloud='fuga', region_name='cystack')
cloud.pprint([

2
examples/cloud/http-debug-logging.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging(http_debug=True)
openstack.enable_logging(http_debug=True)
cloud = openstack.openstack_cloud(
cloud='my-vexxhost', region_name='ca-ymq-1')

2
examples/cloud/munch-dict-object.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='ovh', region_name='SBG1')
image = cloud.get_image('Ubuntu 16.10')

2
examples/cloud/normalization.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging()
openstack.enable_logging()
cloud = openstack.openstack_cloud(cloud='fuga', region_name='cystack')
image = cloud.get_image(

2
examples/cloud/server-information.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='my-citycloud', region_name='Buf1')
try:

2
examples/cloud/service-conditional-overrides.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='rax', region_name='DFW')
print(cloud.has_service('network'))

2
examples/cloud/service-conditionals.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='kiss', region_name='region1')
print(cloud.has_service('network'))

2
examples/cloud/strict-mode.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging()
openstack.enable_logging()
cloud = openstack.openstack_cloud(
cloud='fuga', region_name='cystack', strict=True)

2
examples/cloud/upload-large-object.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='ovh', region_name='SBG1')
cloud.create_object(

2
examples/cloud/upload-object.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging(debug=True)
openstack.enable_logging(debug=True)
cloud = openstack.openstack_cloud(cloud='ovh', region_name='SBG1')
cloud.create_object(

2
examples/cloud/user-agent.py
View File

@ -11,7 +11,7 @@
# under the License.
import openstack
openstack.simple_logging(http_debug=True)
openstack.enable_logging(http_debug=True)
cloud = openstack.openstack_cloud(
cloud='datacentred', app_name='AmazingApp', app_version='1.0')

71
openstack/__init__.py
View File

@ -12,15 +12,21 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
__all__ = [
'__version__',
'connect',
'enable_logging',
]
import warnings
import keystoneauth1.exceptions
import pbr.version
import requestsexceptions
from openstack import _log
from openstack._log import enable_logging # noqa
from openstack.cloud.exc import * # noqa
# TODO(shade) These two want to be removed before we make a release
from openstack.cloud.openstackcloud import OpenStackCloud
from openstack.cloud.operatorcloud import OperatorCloud
import openstack.connection
@ -34,43 +40,11 @@ if requestsexceptions.SubjectAltNameWarning:
def _get_openstack_config(app_name=None, app_version=None):
import openstack.config
# Protect against older versions of os-client-config that don't expose this
try:
return openstack.config.OpenStackConfig(
app_name=app_name, app_version=app_version)
except Exception:
return openstack.config.OpenStackConfig()
return openstack.config.OpenStackConfig(
app_name=app_name, app_version=app_version)
def simple_logging(debug=False, http_debug=False):
if http_debug:
debug = True
if debug:
log_level = logging.DEBUG
else:
log_level = logging.INFO
if http_debug:
# Enable HTTP level tracing
log = _log.setup_logging('keystoneauth')
log.addHandler(logging.StreamHandler())
log.setLevel(log_level)
# We only want extra shade HTTP tracing in http debug mode
log = _log.setup_logging('openstack.cloud.http')
log.setLevel(log_level)
else:
# We only want extra shade HTTP tracing in http debug mode
log = _log.setup_logging('openstack.cloud.http')
log.setLevel(logging.WARNING)
log = _log.setup_logging('openstack.cloud')
log.addHandler(logging.StreamHandler())
log.setLevel(log_level)
# Suppress warning about keystoneauth loggers
log = _log.setup_logging('keystoneauth.identity.base')
log = _log.setup_logging('keystoneauth.identity.generic.base')
# TODO(shade) Document this and add some examples
# TODO(shade) This wants to be renamed before we make a release.
# TODO(shade) This wants to be remove before we make a release.
def openstack_clouds(
config=None, debug=False, cloud=None, strict=False,
app_name=None, app_version=None):
@ -99,10 +73,7 @@ def openstack_clouds(
"Invalid cloud configuration: {exc}".format(exc=str(e)))
# TODO(shade) This wants to be renamed before we make a release - there is
# ultimately no reason to have an openstack_cloud and a connect
# factory function - but we have a few steps to go first and this is used
# in the imported tests from shade.
# TODO(shade) This wants to be removed before we make a release.
def openstack_cloud(
config=None, strict=False, app_name=None, app_version=None, **kwargs):
if not config:
@ -115,10 +86,7 @@ def openstack_cloud(
return OpenStackCloud(cloud_config=cloud_region, strict=strict)
# TODO(shade) This wants to be renamed before we make a release - there is
# ultimately no reason to have an operator_cloud and a connect
# factory function - but we have a few steps to go first and this is used
# in the imported tests from shade.
# TODO(shade) This wants to be removed before we make a release.
def operator_cloud(
config=None, strict=False, app_name=None, app_version=None, **kwargs):
if not config:
@ -134,3 +102,16 @@ def operator_cloud(
def connect(*args, **kwargs):
"""Create a `openstack.connection.Connection`."""
return openstack.connection.Connection(*args, **kwargs)
def connect_all(config=None, app_name=None, app_version=None):
if not config:
config = _get_openstack_config(app_name, app_version)
try:
return [
openstack.connection.Connection(config=cloud_region)
for cloud_region in config.get_all()
]
except keystoneauth1.exceptions.auth_plugins.NoMatchingPlugin as e:
raise OpenStackCloudException(
"Invalid cloud configuration: {exc}".format(exc=str(e)))

98
openstack/_log.py
View File

@ -13,16 +13,102 @@
# limitations under the License.
import logging
import sys
class NullHandler(logging.Handler):
def emit(self, record):
pass
def setup_logging(name, handlers=None, level=None):
"""Set up logging for a named logger.
Gets and initializes a named logger, ensuring it at least has a
`logging.NullHandler` attached.
def setup_logging(name):
:param str name:
Name of the logger.
:param list handlers:
A list of `logging.Handler` objects to attach to the logger.
:param int level:
Log level to set the logger at.
:returns: A `logging.Logger` object that can be used to emit log messages.
"""
handlers = handlers or []
log = logging.getLogger(name)
if len(log.handlers) == 0:
h = NullHandler()
if len(log.handlers) == 0 and not handlers:
h = logging.NullHandler()
log.addHandler(h)
for h in handlers:
log.addHandler(h)
if level:
log.setLevel(level)
return log
def enable_logging(
debug=False, http_debug=False, path=None, stream=None,
format_stream=False,
format_template='%(asctime)s %(levelname)s: %(name)s %(message)s'):
"""Enable logging output.
Helper function to enable logging. This function is available for
debugging purposes and for folks doing simple applications who want an
easy 'just make it work for me'. For more complex applications or for
those who want more flexibility, the standard library ``logging`` package
will receive these messages in any handlers you create.
:param bool debug:
Set this to ``True`` to receive debug messages.
:param bool http_debug:
Set this to ``True`` to receive debug messages including
HTTP requests and responses. This implies ``debug=True``.
:param str path:
If a *path* is specified, logging output will written to that file
in addition to sys.stderr.
The path is passed to logging.FileHandler, which will append messages
the file (and create it if needed).
:param stream:
One of ``None `` or ``sys.stdout`` or ``sys.stderr``.
If it is ``None``, nothing is logged to a stream.
If it isn't ``None``, console output is logged to this stream.
:param bool format_stream:
If format_stream is False, the default, apply ``format_template`` to
``path`` but not to ``stream`` outputs. If True, apply
``format_template`` to ``stream`` outputs as well.
:param str format_template:
Template to pass to :class:`logging.Formatter`.
:rtype: None
"""
if not stream and not path:
stream = sys.stdout
if http_debug:
debug = True
if debug:
level = logging.DEBUG
else:
level = logging.INFO
formatter = logging.Formatter(format_template)
handlers = []
if stream is not None:
console = logging.StreamHandler(stream)
if format_stream:
console.setFormatter(formatter)
handlers.append(console)
if path is not None:
file_handler = logging.FileHandler(path)
file_handler.setFormatter(formatter)
handlers.append(file_handler)
if http_debug:
# Enable HTTP level tracing
setup_logging('keystoneauth', handlers=handlers, level=level)
setup_logging('openstack', handlers=handlers, level=level)
# Suppress warning about keystoneauth loggers
setup_logging('keystoneauth.discovery')
setup_logging('keystoneauth.identity.base')
setup_logging('keystoneauth.identity.generic.base')

2
openstack/cloud/_utils.py
View File

@ -94,7 +94,7 @@ def _filter_list(data, name_or_id, filters):
# The logger is openstack.cloud.fmmatch to allow a user/operator to
# configure logging not to communicate about fnmatch misses
# (they shouldn't be too spammy, but one never knows)
log = _log.setup_logging('openstack.cloud.fnmatch')
log = _log.setup_logging('openstack.fnmatch')
if name_or_id:
# name_or_id might already be unicode
name_or_id = _make_unicode(name_or_id)

4
openstack/cloud/meta.py
View File

@ -232,7 +232,7 @@ def find_best_address(addresses, family, public=False, cloud_public=True):
pass
# Give up and return the first - none work as far as we can tell
if do_check:
log = _log.setup_logging('shade')
log = _log.setup_logging('openstack')
log.debug(
'The cloud returned multiple addresses, and none of them seem'
' to work. That might be what you wanted, but we have no clue'
@ -381,7 +381,7 @@ def _get_supplemental_addresses(cloud, server):
# This SHOULD return one and only one FIP - but doing
# it as a search/list lets the logic work regardless
if fip['fixed_ip_address'] not in fixed_ip_mapping:
log = _log.setup_logging('shade')
log = _log.setup_logging('openstack')
log.debug(
"The cloud returned floating ip %(fip)s attached"
" to server %(server)s but the fixed ip associated"

2
openstack/cloud/openstackcloud.py
View File

@ -152,7 +152,7 @@ class OpenStackCloud(_normalize.Normalizer):
if log_inner_exceptions:
OpenStackCloudException.log_inner_exceptions = True
self.log = _log.setup_logging('openstack.cloud')
self.log = _log.setup_logging('openstack')
if not cloud_config:
config = openstack.config.OpenStackConfig(

2
openstack/config/cloud_region.py
View File

@ -69,7 +69,7 @@ class CloudRegion(object):
self.name = name
self.region_name = region_name
self.config = config
self.log = _log.setup_logging(__name__)
self.log = _log.setup_logging('openstack.config')
self._force_ipv4 = force_ipv4
self._auth = auth_plugin
self._openstack_config = openstack_config

2
openstack/config/loader.py
View File

@ -182,7 +182,7 @@ class OpenStackConfig(object):
pw_func=None, session_constructor=None,
app_name=None, app_version=None,
load_yaml_config=True):
self.log = _log.setup_logging(__name__)
self.log = _log.setup_logging('openstack.config')
self._session_constructor = session_constructor
self._app_name = app_name
self._app_version = app_version

9
openstack/connection.py
View File

@ -75,22 +75,20 @@ try to find it and if that fails, you would create it::
"""
import importlib
import logging
import sys
import keystoneauth1.exceptions
import os_service_types
from six.moves import urllib
from openstack import _log
import openstack.config
from openstack.config import cloud_region
from openstack import exceptions
from openstack import proxy
from openstack import proxy2
from openstack import task_manager
from openstack import utils
_logger = logging.getLogger(__name__)
_logger = _log.setup_logging('openstack')
def from_config(cloud=None, config=None, options=None, **kwargs):
@ -119,9 +117,6 @@ def from_config(cloud=None, config=None, options=None, **kwargs):
config = openstack.config.OpenStackConfig().get_one(
cloud=cloud, argparse=options)
if config.debug:
utils.enable_logging(True, stream=sys.stdout)
return Connection(config=config)

4
openstack/image/v2/image.py
View File

@ -11,16 +11,16 @@
# under the License.
import hashlib
import logging
import jsonpatch
from openstack import _log
from openstack import exceptions
from openstack.image import image_service
from openstack import resource2
from openstack import utils
_logger = logging.getLogger(__name__)
_logger = _log.setup_logging('openstack')
class Image(resource2.Resource):

4
openstack/profile.py
View File

@ -52,8 +52,8 @@ The resulting preference print out would look something like::
"""
import copy
import logging
from openstack import _log
from openstack.baremetal import baremetal_service
from openstack.block_storage import block_storage_service
from openstack.clustering import clustering_service
@ -72,7 +72,7 @@ from openstack.object_store import object_store_service
from openstack.orchestration import orchestration_service
from openstack.workflow import workflow_service
_logger = logging.getLogger(__name__)
_logger = _log.setup_logging('openstack')
class Profile(object):

4
openstack/task_manager.py
View File

@ -22,10 +22,10 @@ import time
import keystoneauth1.exceptions
import six
import openstack._log
from openstack import exceptions
from openstack import utils
_log = utils.setup_logging(__name__)
_log = openstack._log.setup_logging('openstack.task_manager')
class Task(object):

2
openstack/tests/unit/image/v2/test_image.py
View File

@ -272,7 +272,7 @@ class TestImage(testtools.TestCase):
self.sess.get.side_effect = [resp1, resp2]
with self.assertLogs(logger=image.__name__, level="WARNING") as log:
with self.assertLogs(logger='openstack', level="WARNING") as log:
rv = sot.download(self.sess)
self.assertEqual(len(log.records), 1,

88
openstack/tests/unit/test_utils.py
View File

@ -10,67 +10,83 @@
# License for the specific language governing permissions and limitations
# under the License.
import logging
import mock
import sys
import testtools
import fixtures
from openstack import utils
class Test_enable_logging(testtools.TestCase):
def _console_tests(self, fake_logging, level, debug, stream):
the_logger = mock.Mock()
fake_logging.getLogger.return_value = the_logger
def setUp(self):
super(Test_enable_logging, self).setUp()
self.openstack_logger = mock.Mock()
self.openstack_logger.handlers = []
self.ksa_logger_1 = mock.Mock()
self.ksa_logger_1.handlers = []
self.ksa_logger_2 = mock.Mock()
self.ksa_logger_2.handlers = []
self.ksa_logger_3 = mock.Mock()
self.ksa_logger_3.handlers = []
self.fake_get_logger = mock.Mock()
self.fake_get_logger.side_effect = [
self.openstack_logger,
self.ksa_logger_1,
self.ksa_logger_2,
self.ksa_logger_3
]
self.useFixture(
fixtures.MonkeyPatch('logging.getLogger', self.fake_get_logger))
def _console_tests(self, level, debug, stream):
utils.enable_logging(debug=debug, stream=stream)
self.assertEqual(the_logger.addHandler.call_count, 2)
the_logger.setLevel.assert_called_with(level)
self.assertEqual(self.openstack_logger.addHandler.call_count, 1)
self.openstack_logger.setLevel.assert_called_with(level)
def _file_tests(self, fake_logging, level, debug):
the_logger = mock.Mock()
fake_logging.getLogger.return_value = the_logger
def _file_tests(self, level, debug):
file_handler = mock.Mock()
self.useFixture(
fixtures.MonkeyPatch('logging.FileHandler', file_handler))
fake_path = "fake/path.log"
utils.enable_logging(debug=debug, path=fake_path)
fake_logging.FileHandler.assert_called_with(fake_path)
self.assertEqual(the_logger.addHandler.call_count, 2)
the_logger.setLevel.assert_called_with(level)
file_handler.assert_called_with(fake_path)
self.assertEqual(self.openstack_logger.addHandler.call_count, 1)
self.openstack_logger.setLevel.assert_called_with(level)
def test_none(self):
self.assertRaises(
ValueError, utils.enable_logging,
debug=True, path=None, stream=None)
utils.enable_logging(debug=True)
self.fake_get_logger.assert_has_calls([])
self.openstack_logger.setLevel.assert_called_with(logging.DEBUG)
self.assertEqual(self.openstack_logger.addHandler.call_count, 1)
self.assertIsInstance(
self.openstack_logger.addHandler.call_args_list[0][0][0],
logging.StreamHandler)
@mock.patch("openstack.utils.logging")
def test_debug_console_stderr(self, fake_logging):
self._console_tests(fake_logging,
fake_logging.DEBUG, True, sys.stderr)
def test_debug_console_stderr(self):
self._console_tests(logging.DEBUG, True, sys.stderr)
@mock.patch("openstack.utils.logging")
def test_warning_console_stderr(self, fake_logging):
self._console_tests(fake_logging,
fake_logging.WARNING, False, sys.stderr)
def test_warning_console_stderr(self):
self._console_tests(logging.INFO, False, sys.stderr)
@mock.patch("openstack.utils.logging")
def test_debug_console_stdout(self, fake_logging):
self._console_tests(fake_logging,
fake_logging.DEBUG, True, sys.stdout)
def test_debug_console_stdout(self):
self._console_tests(logging.DEBUG, True, sys.stdout)
@mock.patch("openstack.utils.logging")
def test_warning_console_stdout(self, fake_logging):
self._console_tests(fake_logging,
fake_logging.WARNING, False, sys.stdout)
def test_warning_console_stdout(self):
self._console_tests(logging.INFO, False, sys.stdout)
@mock.patch("openstack.utils.logging")
def test_debug_file(self, fake_logging):
self._file_tests(fake_logging, fake_logging.DEBUG, True)
def test_debug_file(self):
self._file_tests(logging.DEBUG, True)
@mock.patch("openstack.utils.logging")
def test_warning_file(self, fake_logging):
self._file_tests(fake_logging, fake_logging.WARNING, False)
def test_warning_file(self):
self._file_tests(logging.INFO, False)
class Test_urljoin(testtools.TestCase):

64
openstack/utils.py
View File

@ -11,12 +11,13 @@
# under the License.
import functools
import logging
import time
import deprecation
from openstack import _log
# SDK has had enable_logging in utils. Import the symbol here to not break them
from openstack._log import enable_logging # noqa
from openstack import exceptions
from openstack import version
@ -47,67 +48,6 @@ def deprecated(deprecated_in=None, removed_in=None,
details=details)
class NullHandler(logging.Handler):
def emit(self, record):
pass
def setup_logging(name):
'''Get a logging.Logger and make sure there is at least a NullHandler.'''
log = logging.getLogger(name)
if len(log.handlers) == 0:
h = NullHandler()
log.addHandler(h)
return log
def enable_logging(debug=False, path=None, stream=None):
"""Enable logging to a file at path and/or a console stream.
This function is available for debugging purposes. If you wish to
log this package's message in your application, the standard library
``logging`` package will receive these messages in any handlers you
create.
:param bool debug: Set this to ``True`` to receive debug messages,
which includes HTTP requests and responses,
or ``False`` for warning messages.
:param str path: If a *path* is specified, logging output will
written to that file in addition to sys.stderr.
The path is passed to logging.FileHandler,
which will append messages the file (and create
it if needed).
:param stream: One of ``None `` or ``sys.stdout`` or ``sys.stderr``.
If it is ``None``, nothing is logged to a stream.
If it isn't ``None``, console output is logged
to this stream.
:rtype: None
"""
if path is None and stream is None:
raise ValueError("path and/or stream must be set")
logger = logging.getLogger('openstack')
ksalog = logging.getLogger('keystoneauth')
formatter = logging.Formatter(
'%(asctime)s %(levelname)s: %(name)s %(message)s')
if stream is not None:
console = logging.StreamHandler(stream)
console.setFormatter(formatter)
logger.addHandler(console)
ksalog.addHandler(console)
if path is not None:
file_handler = logging.FileHandler(path)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
ksalog.addHandler(file_handler)
logger.setLevel(logging.DEBUG if debug else logging.WARNING)
ksalog.setLevel(logging.DEBUG if debug else logging.WARNING)
def urljoin(*args):
"""A custom version of urljoin that simply joins strings into a path.