Commit Graph

148 Commits (6aceca218af7d1d2c708fde48f1a5f2b798bc421)

Author SHA1 Message Date
Dean Troyer ef99f44462 Improve no-auth path
The commands that do not require authentication sometimes still need
to call ClientManager.is_network_endpoint_enabled() to see if Neutron
is available.  Optimize the paths a bit to skip auth when it is not
necessary; the upshot is Neutron will be assumed in these cases now.

This gets a LOT cleaner when it appears is a future osc-lib.

Change-Id: Ifaddc57dfa192bde04d0482e2cdcce111313a22a
6 years ago
Dean Troyer 46b8cad4c3 Clean up password prompt work-arounds
osc-lib 1.2 is minimum and now handles the password prompting.

Change-Id: Ie11ad64796d3a89c7396b321c34947d622d1ed39
6 years ago
Dean Troyer 1c3cb0a3b5 Change noauth strategy for plugin loading
Don't do it.

os-client-config's plugin loading has been causing this pain for a long
time, removing the KSC hack-around in o-c-c unmasked this again.  So when
auth is not reuired, just don't let o-c-c do any plugin loading at all.


Of course, this shouldn't be in OSC either, but we have to do this until
the equivalent fix lands in osc-lib, is released and makes it into the
global requirements minimum version.

Depends-on: Ie68c82f7b073012685f0513b615ab1bf00bc0c3a
Change-Id: Ifdf65f3bb90fb923947a2cbe80a881d71a3fee56
6 years ago
Rui Chen e8b6a9f7be Fix wrong behavior of parsing plugin service name
When the service name end with keyword "os", like: antiddos,
the parsing logic isn't suitable, that cause the service api
version specified by users don't work.

Change-Id: I5d6217c77d7cd2d2f360d78d8561261398b96685
Closes-Bug: #1658614
6 years ago
gengchc2 40d73a0b58 Correct reraising of exception
When an exception was caught and rethrown, it should
call 'raise' without any arguments because it shows
the place where an exception occured initially instead
of place where the exception re-raised

Change-Id: I3ec3680debbfad7c06f2251396e0e8e4e3df6c50
7 years ago
Kyrylo Romanenko 42f9317360 Improve output of supported API versions
Sort supported versions properly for better look.

Change-Id: I6c2f5ecc04cf14ea5bf1b214cb303fcc9783af3f
Closes-Bug: #1630962
7 years ago
Dean Troyer 14dbfe4474 Defer auth prompting until it is actually needed
Auth option prompting happens waaaay to early in the default
os-client-config flow, we need to defer it until adter the commands
have been parsed.  This is why ClientManager.setup_auth() exists,
as it is not called until the first attempt to connect to a server
occurs.  Commands that do not require authentication never hit this.

Also, required options were not being enforced.  By doing this we handle
when no authentication info is present, we fail on missing auth-url rather
than attempt to prompt for a password (default auth is password).

Closes-Bug: 1619274
Change-Id: Ia4eae350e6904c9eb2c8507d9b3429fe52418726
7 years ago
Dean Troyer bec206fa0a Fix auth prompt brokenness
We start by fixing this in the already-present OSC_Config class so OSC
can move forward.  This change needs to get ported down into
os-client-config in the near future, maybe even soon enough to make the
client library freeze this week.

* Add the pw-func argument to the OSC_Config (or OpenStackConfig) __init__()
* When looping through the auth options from the KSA plugin look for any
  that have a prompt defined and do not have a value already, so ask for one.

Closes-bug: #1617384
Change-Id: Ic86d56b8a6844516292fb74513712b486fec4442
7 years ago
Dean Troyer 188709c668 Restore default auth-type for token/endpoint
The split to osc-lib shell lost the detection of --os-token and
--os-url to set --os-auth-type token_endpoint

Closes-bug: 1615988
Change-Id: I248f776a3a7b276195c162818f41ba20760ee545
7 years ago
Dean Troyer 2a1a174086 Gate-unbreaking combo review
Fix argument precedence hack
  Working around issues in os-client-config <= 1.18.0

  This is ugly because the issues in o-c-c 1.19.1 run even deeper
  than in 1.18.0, so we're going to use 1.19.0 get_one_cloud() that
  is known to work for OSC and fix o-c-c with an axe.

Remove return values for set commands
  'identity provider set' and 'service provider set' were still
  returning their show-like data, this is a fail for set commands
  now, don't know how this ever passed before...

Constraints are ready to be used for tox.ini
  Per email[1] from Andreas, we don't need to hack at install_command
  any longer.


Co-authorioed-by: Steve Martinelli <>
Depends-On: I49313dc7d4f44ec897de7a375f25b7ed864226f1
Change-Id: I426548376fc7d3cdb36501310dafd8c44d22ae30
7 years ago
Dean Troyer 6a15f90dae osc-lib: shell
Convert to using ClientManager and OpenStackShell from osc-lib.
* Change all internal uses of ClientManager private attributes that are
  now public in osc-lib's ClientManager.  Leave back-compat copies in
  place in OSC's clientManager so we don't break plugins.
* Put some work-arounds in place for changes in osc-lib that we need until
  a new release makes it through the g-r and u-c change process.
* Add a test for Unicode decoding of argv in shell.main() to parallel
  the one in osc-lib.

Change-Id: I85289740d4ca081f2aca8c9b40ec422ad25d302c
7 years ago
John Dennis 756d2fac67 arguments are not locale decoded into Unicode
When the openstackclient in Python2 passes command line arguments to a
subcommand it fails to pass the arguments as text
(e.g. Unicode). Instead it passes the arguments as binary data encoded
using the current locales encoding.

An easy way to see this is trying to pass a username with a non-ASCII

% openstack user delete ñew
No user with a name or ID of 'ñew' exists.

What occurs internally is when the user data is retrieved it's it
properly represented in a Unicode object. However the username pased
from the command line is still a str object encoded in the locales
encoding (typically UTF-8). A string comparison is attempted between
the encoded data from the command line and the Unicode text found in
the user representation. This seldom ends well, either the comparison
fails to match or a codec error is raised.

There is a hard and fast rule, all text data must be stored in Unicode
objects and the conversion from binary encoded text to Unicode must
occur as close to the I/O boundary as possible. Python3 enforces this
behavior automatically but in Python2 it is the programmers job to do

In the past there have been attempts to fix problems deep inside
internal code by attempting to decode from UTF-8. There are two
problems with this approach. First, internal code has no way to
accurately know what encoding was used to encode the binary data. This
is way it needs to be decoded as close to the I/O source as possible
because that is the best place to know the actual encoding. Guessing
UTF-8 is at best a heuristic. Second, there must be a canonical
representation for data "inside" the program, you don't want dozens of
individual modules, classes, methods, etc. performing conversions,
instead they should be able to make the assumption in what format text
is represented in, the format for text data must be Unicode. This is
another reason to decode as close to the I/O as possible.

In Python3 the argv strings are decoded from the locales encoding by
the interpreter. By the time any Python3 code sees the argv strings
they will be Unicode. However in Python2 there must be explicit code
added to decode the argv strings into Unicode.

The conversion of sys.argv into Unicode only occurs when argv is not
passed to If a caller of
supplies their own arg it is their responsiblity to assure they are
passing actual text objects. Consider this a requirement of the API.

Note: This patch does not contain a unittest to exercise the behavior
because it is difficult to construct a test that depends on command
invocation from a shell. The general structure of the unit tests is to
pass fake argv into as if it came from a
shell. Because the new code only operates when argv is not passed and
defaults to sys.argv it conflicts with the unittest design.

Change-Id: I779d260744728eae8455ff9dedb6e5c09c165559
Closes-Bug: 1603494
Signed-off-by: John Dennis <>
7 years ago
Dean Troyer f38c51c1b9 Rework clientmanager
* Add compatibility for plugin v2 interface removed from osc-lib
* ClientManager.is_network_endpoint_enabled() is wrapper for
  new is_service_available()

Change-Id: I6f26ce9e4d0702f50c7949bacfbeeb0f98cddb5d
7 years ago
SongmingYan 1b878b4efd Remove execute permission on a few files
Some files have execute permission unnecessarily. Change them from
755 to 644.

Change-Id: I471ebd1c3d123ad4a7376f7f5996f53f8c2d9b0b
7 years ago
qinchunhua 6f36385cb8 Correct reraising of exception
When an exception was caught and rethrown,
it should call 'raise' without any arguments
because it shows the place where an exception
occured initially instead of place where the exception re-raised.

Change-Id: I5fb6dea5da7fb6e1e2b339a713c7d37f8c99e407
7 years ago
Dolph Mathews fe0c8e955b Do not prompt for scope options with default scoped tokens
This changes the scope validation to occur after a token has already
been created.

Previous flow:

1. Validate authentication options.
2. Validate authorization options if the command requires a scope.
3. Create a token (using authentication + authorization options)
4. Run command.

This means that scope was being checked, even if a default scope was
applied in step 3 by Keystone.

New flow:

1. Validate authentication options.
2. Create token (using authentication + authorization options)
3  Validate authorization options if the command requires a scope and
   the token is not scoped.
4. Run command.

Change-Id: Idae368a11249f425b14b891fc68b4176e2b3e981
Closes-Bug: 1592062
7 years ago
Huanxuan Ao f25a3519c5 Fix missing i18n supports in api/ and
Change-Id: I28d79d7f44b27d2b600dedad2a3601180650ad83
Partial-bug: #1574965
7 years ago
Dean Troyer a55eb915a0 osc-lib: timing
Change-Id: I3fe27d98efa5090e084c676f7f8e6dad0157ed21
7 years ago
Dean Troyer 59dffb9c62 osc-lib: logs
Change-Id: I2a4d40cd72cc22e97a600751ae29c2309ebed28b
7 years ago
Dean Troyer e5e29a8fef osc-lib: utils
Use osc-lib directly for utils.

Leave openstackclient.common.utils for deprecation period.

Change-Id: I5bd9579abc4e07f45219ccd0565626e6667472f7
7 years ago
Dean Troyer d20c863ebc osc-lib: exceptions
Use osc-lib directly for exceptions.

Leave openstackclient.common.exceptions for deprecation period.

Change-Id: Iea3e862302372e1b31ccd27f69db59b4953ca828
7 years ago
Richard Theis 6f2c1734e3 Fix --enable options on commands
The --enable option on commands is ignored when the arguments are parsed.
This is related to the --enable-beta-commands option. Renaming the option
to --os-beta-command fixes the problem.

There's no need to handle backwards compatibility for the option name
change because there hasn't been an OSC release yet with beta commands.

Change-Id: I0327ba8a2058858a83e9a42e231470ed733cc834
Closes-Bug: #1588384
7 years ago
Richard Theis 1a2e12832d Devref: Command Beta
The devref proposes OSC support for beta commands.

Change-Id: I538a38be33734faf6eb69a3cb50946b2396b0c57
7 years ago
Dean Troyer 530fe42589 Rename --profile to --os-profile
* The --profile global option is deprecated but will be supported
  through at least April 2017.
* Update man page

Closes-bug: #1571812
Change-Id: I2e623411a56096b4cc352f4eedbf770632ae2cc3
7 years ago
Cedric Brandily 3a8320a1d7 Support client certificate/key
This change enables to specify a client certificate/key with:
 * usual CLI options (--os-cert/--os-key)
 * usual environment variables ($OS_CERT/$OS_KEY)
 * os-client-config

Change-Id: Ibeaaa5897ae37b37c1e91f3e47076e4e8e4a8ded
Closes-Bug: #1565112
7 years ago
Tang Chen dc7e4fc15d Fix dict.keys() compatibility for python 3
In Python 2, dict.keys() will return a list.
But in Python 3, it will return an iterator.
So we need to fix all the places that assuming
dict.keys() is a list.

Change-Id: I8d1cc536377b3e5c644cfaa0892e40d0bd7c11b1
Closes-Bug: #1556350
7 years ago
Dina Belova 8a839ad8b8 Fix regression in interactive client mode
Fix typo introduced in OSprofiler intergation commit, that
leaded to non-working interactive mode of the CLI client.

Change-Id: If5dfc90dbbe64d4665c3e33e936f0cc674738351
Closes-Bug: 1551160
7 years ago
Jenkins 2819450be5 Merge "Add shell --profile option to trigger osprofiler from CLI" 7 years ago
Dina Belova 16f00833a7 Add shell --profile option to trigger osprofiler from CLI
This will allow to trigger profiling of various services that
allow it currently and which APIs support is added to openstackclient.
Cinder and Glance have osprofiler support already, Nova and Keystone
are in progress.

To use this functionality osprofiler (and its storage backend) needs
to be installed in the environment. If so, you will be able to trigger
profiling via the following command, for example:

$ openstack --profile SECRET_KEY user list

At the end of output there will be message with <trace_id>, and
to plot nice HTML graphs the following command should be used:

$ osprofiler trace show <trace_id> --html --out result.html

Related Keystone change:
Related Nova change:

The similar change to the keystoneclient
( was abandoned as new
CLI extenstions are not more accepted to python-keystoneclient.

Change-Id: I3d6ac613e5da70619d0a4781e5d066fde073b407
7 years ago
guang-yee 41e1bd0be6 Support unscoped token request
Make scope check optional for the "token issue" command as unscoped token is
a valid Keystone V2/V3 API.

Change-Id: Ie1cded4dbfdafd3a78c0ebdf89e3f66762509930
Closes-Bug: #1543214
7 years ago
Hideki Saito e5b8e08eb1 Fix 'openstack --help' fails if clouds.yaml cannot be read
'openstack --help' can display the basic information, even if
 openstack command does not have permission to read clouds.yaml.

Change-Id: I7d5255c5ce3bd60af77fc70f433ca78dc011a79f
Closes-Bug: #1541047
7 years ago
Terry Howe 2348617606 The format_exc method does not take an exception
For py35, this call blows up.  Seems to be ignored for py27, but
even in py27, it doesn't take an exception.

Change-Id: I2602426b966045b15b96e5e41d0df6524ed05119
8 years ago
TerryHowe 2f00fcda77 Allow int version numbers in the clouds.yaml
OSC blows up if you try to use for example identity_api_version: 2
in the clouds.yaml.  It will only work with a string '2'.  This
fixes that.

Change-Id: I785d37a288126a1582464e907c7f9c9947bac27c
8 years ago
Jenkins cb28cd9ac0 Merge "Rename to" 8 years ago
Jenkins 17ebd15a13 Merge "Allow debug to be set in configuration file" 8 years ago
Terry Howe 05800c4722 Rename to
At one point this file contained the context for logging, but
the reason for its existence is now for logging.

Implements: blueprint logging-migration

Change-Id: I4ba42bbef97b09d31236ac8c01b6fb23827b8bee
8 years ago
TerryHowe bfebac8282 Allow debug to be set in configuration file
The current default value for debug in cliff is False.  Cloud
config assumes that it was set that way on the command line and
does not overlay it with the value from the configuration file.
Cliff bug:

Change-Id: I66d82b489b2241dbcd1e1350e94259a54ce09de7
8 years ago
lin-hua-cheng f0a81c284d Mask the sensitive values in debug log
Change-Id: I0eb11a648c3be21749690f079229c8e63a678e6c
Closes-Bug: #1501598
8 years ago
Jenkins 678e690648 Merge "Override the debug default and help text" 8 years ago
Jenkins 2616fdb089 Merge "Set default auth plugin back to 'password'" 8 years ago
Dean Troyer d6788f7e75 Move option logging back to start if initialize_app()
The log for the options should be printed early enough to see what
is being passed in to occ.

Change-Id: I97b09bc28abcd485b6793d0223b9f8602237fd80
8 years ago
Dean Troyer b1972fb561 Set default auth plugin back to 'password'
This was a hack that should be less needed now...

Change-Id: Id8cba87ad05b106aa36e356c0d70a568316fd327
8 years ago
Jenkins 46696f5ed5 Merge "Create log configuration class" 8 years ago
Jenkins 67387ba9cd Merge "Move options to log level out of" 8 years ago
Jenkins 1a2bb48432 Merge "Move set warnings filters to logging module" 8 years ago
Jenkins 24ca952eb6 Merge "Set up every time record log in file" 8 years ago
Dean Troyer 3abfea083a Fix compute API version snafu
novaclient 2.27.0 introduced the API microversion discovery and client.Client
now wants an api_version argument to properly work out the correct API
version in use.  OSC needs to provide this when required.

Letting the compute client plugin do the version validity checking makes more
sense than encoding it into, so I've added a new OSC plugin interface
function check_api_version() that is called from if it exists.  If it
either does not exist or it returns False the previous version checking using
API_VERSIONS is still performed.

compute.client.check_api_version() conditionally imports the new
novaclient.api_versions module and uses it if successful.  Otherwise
check_api_version() returns False and the previous code path is resumed.

One side-effect of this is that it is now valid to use --os-compute-api-version
with any valid microversion supported by the installed python-novaclient.

Closes-Bug: #1492467
Change-Id: I4535b38a5639a03a9597bf83f6394f9bb45c2b9e
8 years ago
TerryHowe 85a03945f0 Create log configuration class
Configuration of logging gets triggered twice.  The first time it
uses the CLI options when the application is started and second
it uses the configuration file after that is read.  The state of
the logging needs to be saved from the first to the second time,
so I created a class.

Implements: blueprint logging-migration

Change-Id: I7b8d1a3b6fd128e98cafd7c16009c7b694a52146
8 years ago
TerryHowe 00eebaf5bc Override the debug default and help text
Cliff sets the default debug value to False and this makes it
impossible to override debug with OCC.  If we set the default to
None, we can override debug in clouds.yaml.  Also, OSC changes
the meaning of --debug, so modify the help text.

Change-Id: I5e6680b2286cd7f55afe4b083fae5f8a4a9567a2
Closes-Bug: #1483378
8 years ago
TerryHowe ca9965c328 Move options to log level out of
Move the conversion of command line options to log level out

Change-Id: I86cb45a85cd63927aa1c87c1eed27542981df659
Implements: blueprint logging-migration
8 years ago