Secret Consumers

Spec for implementing Consumers API on Secrets.

Story: 2005770
Task: 33487

APIImpact

Change-Id: I0167ce2bd8c6cee82aeb1ec332c09b77efe2eab3
This commit is contained in:
Douglas Mendizábal 2019-05-29 09:58:19 -05:00
parent 5fc1790899
commit 69ac31c535
3 changed files with 363 additions and 346 deletions

View File

@ -6,10 +6,7 @@ Barbican Project Specifications
Train approved specs:
..
disabled to not break builds, once first spec is added,
this needs enabling.
.. toctree::
.. toctree::
:glob:
:maxdepth: 1

View File

@ -0,0 +1,362 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
================
Secret Consumers
================
https://storyboard.openstack.org/#!/story/2005770
This spec proposes an addition to the Barbican Secrets API to allow
other OpenStack projects to add references to individual Secrets when
those secrets are being used by them.
This spec also proposes a change to both the Python and CLI clients in
python-barbicanclient in how they handle the deletion of secrets.
Clients would be changed such that deleting a secret will result in an
error when they are still being consumed by another project unless a `force`
parameter is provided.
This spec is part of a larger effort to provide Encrypted Images
to OpenStack clouds.
Problem Description
===================
Other OpenStack projects would like to make use of an end user's secrets
e.g. A Secret that contains an encryption key for Image Encryption.
But there is currently no way for those projects to let the user know
that they are using the Secret. This lack of awareness may lead to errors
if the user deletes a Secret that is still in use by other projects.
On the other hand, users should be allowed to delete secrets whenever they
want, so a Secret being used by other projects should not prevent deletion.
Proposed Change
===============
Add a new API to Secrets to register Secret Consumers (similar, but not
identical to the Containers Consumer API [1]).
With this new API, other OpenStack projects would register as a consumer
of a secret by sending a request to Barbican. Barbican stores the service
type of the requesting service, as well as both the resource type and
resource ID of the resource that is using the Secret.
See REST API Impact below for details of the API changes.
Clients to barbican would change the semantics for deleting secrets by
returning an error when trying to delete a secret if that secret has one
or more consumers. Clients will also accept an additional boolean parameter
to delete a secret regardless of how many consumers it has.
See Python and Command Line Client Impact below for details of the client
changes.
Alternatives
------------
One alternative would be to implement Secret Consumers just like Container
Consumers, which uses a URL instead of the consuming entity type and ID.
Another alternative approach that was considered was to have each project
clone the secret when they need to use it. This alternative has some
downsides, however. For one, an end user may not be able to delete
those copies.
Data model impact
-----------------
A new model and associated data table will need to be added. For example,
a new class SecretConsumerMetadatum with a secret_consumer_metadata table.
The new class will have references to both the secret_id as well as the
project_id which owns the secret.
REST API impact
---------------
POST /v1/secrets/{secret_id}/consumers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Add a new resource as a consumer to a secret.
Body Parameters
+++++++++++++++
+---------------------+--------+--------------------------------------------------------+
| Name | Type | Description |
+---------------------+--------+--------------------------------------------------------+
| service | string | Consumer's OpenStack service type as shown in |
| | | https://service-types.openstack.org/service-types.json |
+---------------------+--------+--------------------------------------------------------+
| resource_type | string | Name of the resource type using the secret |
| (or resource_path?) | | e.g. "images" or "lbaas/loadbalancers" |
+---------------------+--------+--------------------------------------------------------+
| resource_id | string | Unique identifier for the resource using this secret. |
+---------------------+--------+--------------------------------------------------------+
Barbican will consider the resource_id to be a unique consumer. This assumes
that resource_id is a UUID, and that duplicate IDs for different projects
is not likely to ever happen in a single cloud.
resource_type should be meaningful to the individual projects, and should
be used to identify the resource in the consuming service. For example,
Glance could use "images" as the value of the resource type to indicate that
the resrouce_id refers to an image.
Request
+++++++
POST /v1/secrets/{secret_id}/consumers
Headers:
X-Auth-Token: {token}
X-Content-Type: application/json
{
"service": "image",
"resource_type": "images",
"resource_id": "{image_id}"
}
Responses
+++++++++
+------+--------------------------------------------------------------------+
| Code | Description |
+======+====================================================================+
| 200 | OK |
+------+--------------------------------------------------------------------+
| 401 | Unauthorized - X-Auth-Token is invalid |
+------+--------------------------------------------------------------------+
| 403 | Forbidden - X-Auth-Token is valid, but the associated project does |
| | not have the appropriate role/scope |
+------+--------------------------------------------------------------------+
GET /v1/secrets/{secret_id}/consumers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
List consumers for a particular Secret.
Parameters
++++++++++
+---------+---------+---------+-------------------------------------------------+
| Name | Type | Default | Description |
+=========+=========+=========+=================================================+
| offset | integer | 0 | Offset to start consumer response |
+---------+---------+---------+-------------------------------------------------+
| limit | integer | 10 | Number of consumer entries returned in response |
+---------+---------+---------+-------------------------------------------------+
| service | string | None | Filter by service type |
+---------+---------+---------+-------------------------------------------------+
Request
+++++++
GET /v1/secrets/{secret_id}/consumers
Headers:
X-Auth-Token: {token}
OK Response
+++++++++++
200 OK
{
"total": 1,
"consumers": [
{
"service": "image",
"resource_type": "images",
"resource_id" : "{image_id}"
}
]
}
Other Responses
+++++++++++++++
+------+--------------------------------------------------------------------+
| Code | Description |
+======+====================================================================+
| 401 | Unauthorized - X-Auth-Token is invalid |
+------+--------------------------------------------------------------------+
| 403 | Forbidden - X-Auth-Token is valid, but the associated project does |
| | not have the appropriate role/scope |
+------+--------------------------------------------------------------------+
DELETE /v1/secrets/{secret_id}/consumers/{resource_id}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Delete a consumer. ie. The resource is being deleted and it longer needs
to access this secret.
Request
+++++++
DELETE v1/secrets/{secret_id}/consumers/{resource_id}
Responses
+++++++++
+------+--------------------------------------------------------------------+
| Code | Description |
+======+====================================================================+
| 200 | OK |
+------+--------------------------------------------------------------------+
| 401 | Unauthorized - X-Auth-Token is invalid |
+------+--------------------------------------------------------------------+
| 403 | Forbidden - X-Auth-Token is valid, but the associated project does |
| | not have the appropriate role/scope |
+------+--------------------------------------------------------------------+
| 404 | Not Found - Consumer record for given resource_id was not found. |
+------+--------------------------------------------------------------------+
Security impact
---------------
Because the consumers are stored in the database, there is the possibility
that a bad actor could add many consumers to try to fill the database disk
space. Secret Consumers should be limited to the same quota as Container
Consumers to mitigate this risk. For example:
[quota]
quota_consumers=10000
Would limit both Container Consumers and Secret Consumers to a maximum
of 10,000 consumers each for both a single Container or a single Secret.
Notifications & Audit Impact
----------------------------
The new API endpoints should be audited as usual.
Python and Command Line Client Impact
-------------------------------------
The Secret class in python-barbicanclient should be updated to add new
methods such as:
class Secret(...):
...
def add_consumer(self, service_type, resource_type, resource_id):
...
def remove_consumer(self, service_type, resource_type, resource_id):
...
Both methods should raise appropriate exceptions when the API returns an error.
Additionally, the Secret.delete() method should be updated to take a new *force*
parameter and throw an exception when delete() is called with force=False,
and the secret still has consumers:
class Secret(...):
...
def delete(self, force=False):
...
The CLI client should be changed to add new consumer options, such as:
openstack secret consumer add --service-type=image --resource-type=image \
--resource-id=XXXX-XXXX-XXXX-XXXX
openstack secret consumer remove --service-type=image --resource-type=image \
--resource-id=XXXX-XXXX-XXXX-XXXX
The secret delete command should be changed to take a *--force* parameter:
openstack secret delete --force {secret_uuid}
This command should return an error when a secret has one or more consumers
and the --force flag is not used:
openstack secret delete {secret_uuid_with_consumers}
ERROR: Secret has one or more consumers. Use --force to delete anyway.
These changes will require a new Major version for python-barbicanclient
because the default --force=False option could cause some scripts to break in
certain scenarios where secrets are currently being deleted that do have
consumers associated with them.
Other end user impact
---------------------
Currently there is no other impact to the end user other than the CLI changes
listed above. In the future, when a barbican-ui for Horizon is developed,
it should use the consumers to present confirmation dialogs to the user
when deleting Secrets which have consumers.
It should be noted that Deleting Secrets in the Barbican REST API
has not changed, and a client using the API directly will be able to delete
a secret regardless of the presence of consumers.
Performance Impact
------------------
Deleting secrets using the CLI or the Python client will be affected as we
will likely need to perform additional requests to the API to get the list of
consumers for a secret before sending a DELETE request.
Other deployer impact
---------------------
When python-barbican changes are merged, some automation scripts that use
secret deletion may break if the secrets being deleted have consumers.
Any automation scripts should be updated to use the --force flag if needed.
Developer impact
----------------
Developers of other projects that want to make use of this feature will
need to use python-barbicanclient to integrate with the Key Manager service.
Implementation
==============
Assignee(s)
Primary assignee:
Douglas Mendizábal (Freenode: redrobot) <dmendiza@redhat.com>
Other contributors:
Moisés Guimarães (Freenode: moguimar) <moguimar@redhat.com>
Work Items
----------
* Implement Model changes and database migration
* Implement API changes
* Implement python-barbicanclient changes (both python client and CLI)
Dependencies
============
None.
Testing
=======
Tempest test cases should be added to test adding/removing Secret Consumers
using a service-user that is not barbican.
Documentation Impact
====================
All API changes should be documented in the API reference, as well as the
API Guide.
References
==========
[1] Container Consumers API:
https://docs.openstack.org/barbican/stein/api/reference/consumers.html
Barbican Train PTG Etherpad:
https://etherpad.openstack.org/p/barbican-train-ptg

View File

@ -1,342 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Example Spec - The title of your blueprint
==========================================
Include the URL of your launchpad blueprint:
https://blueprints.launchpad.net/barbican/+spec/example
Include the URL of your client blueprint:
https://blueprints.launchpad.net/python-barbicanclient/example
Introduction paragraph -- why are we doing anything? A single paragraph of
prose that operators can understand.
Some notes about using this template:
* Your spec should be in ReSTructured text, like this template.
* Please wrap text at 79 columns.
* The filename in the git repository should match the launchpad URL, for
example a URL of: https://blueprints.launchpad.net/barbican/+spec/awesome-thing
should be named awesome-thing.rst
* Please do not delete any of the sections in this template. If you have
nothing to say for a whole section, just write: None
* For help with syntax, see http://sphinx-doc.org/rest.html
* To test out your formatting, build the docs using tox, or see:
http://rst.ninjs.org
* If you would like to provide a diagram with your spec, ascii diagrams are
required. http://asciiflow.com/ is a very nice tool to assist with making
ascii diagrams. The reason for this is that the tool used to review specs is
based purely on plain text. Plain text will allow review to proceed without
having to look at additional files which can not be viewed in gerrit. It
will also allow inline feedback on the diagram itself.
* If your specification proposes any changes to the Barbican REST API such
as changing parameters which can be returned or accepted, or even
the semantics of what happens when a client calls into the API, then
you should add the APIImpact flag to the commit message. Specifications with
the APIImpact flag can be found with the following query::
https://review.openstack.org/#/q/status:open+project:openstack/barbican-specs+message:apiimpact,n,z
Problem Description
===================
A detailed description of the problem:
* For a new feature this might be use cases. Ensure you are clear about the
actors in each use case: End User vs Deployer
* For a major reworking of something existing it would describe the
problems in that feature that are being addressed.
Proposed Change
===============
Here is where you cover the change you propose to make in detail. How do you
propose to solve this problem?
If this is one part of a larger effort make it clear where this piece ends. In
other words, what's the scope of this effort?
Alternatives
------------
What other ways could we do this thing? Why aren't we using those? This doesn't
have to be a full literature review, but it should demonstrate that thought has
been put into why the proposed solution is an appropriate one.
Data model impact
-----------------
Changes which require modifications to the data model often have a wider impact
on the system. The community often has strong opinions on how the data model
should be evolved, from both a functional and performance perspective. It is
therefore important to capture and gain agreement as early as possible on any
proposed changes to the data model.
Questions which need to be addressed by this section include:
* What new data objects and/or database schema changes is this going to
require?
* What database migrations will accompany this change (if any)?
* How will the initial set of new data objects be generated? For example, if you
need to take into account existing keys, or modify other existing data
describe how that will work.
REST API impact
---------------
Each API method which is either added or changed should have the following
* Specification for the method
* A description of what the method does suitable for use in
user documentation
* Method type (POST/PUT/GET/DELETE)
* Normal http response code(s)
* Expected error http response code(s)
* A description for each possible error code should be included
describing semantic errors which can cause it such as
inconsistent parameters supplied to the method, or when an
instance is not in an appropriate state for the request to
succeed. Errors caused by syntactic problems covered by the JSON
schema defintion do not need to be included.
* URL for the resource
* Parameters which can be passed via the url
* JSON schema definition for the body data if allowed
* JSON schema definition for the response data if any
* Example use case including typical API samples for both data supplied
by the caller and the response
* Discuss any policy changes, and discuss what things a deployer needs to
think about when defining their policy.
Example JSON schema definitions can be found in the Nova tree
http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/schemas/v3
Note that the schema should be defined as restrictively as
possible. Parameters which are required should be marked as such and
only under exceptional circumstances should additional parameters
which are not defined in the schema be permitted (eg
additionaProperties should be False).
Reuse of existing predefined parameter types such as regexps for
passwords and user defined names is highly encouraged.
Security impact
---------------
Describe any potential security impact on the system. Some of the items to
consider include:
* Does this change touch sensitive data such as tokens, keys, or user data?
* Does this change alter the API in a way that may impact security, such as
a new way to access sensitive information or a new way to login?
* Does this change involve cryptography or hashing?
* Does this change require the use of sudo or any elevated privileges?
* Does this change involve using or parsing user-provided data? This could
be directly at the API level or indirectly such as changes to a cache layer.
* Can this change enable a resource exhaustion attack, such as allowing a
single API interaction to consume significant server resources? Some examples
of this include launching subprocesses for each connection, or entity
expansion attacks in XML.
* Does this change the need for auditing in any way?
For more detailed guidance, please see the OpenStack Security Guidelines as
a reference (https://wiki.openstack.org/wiki/Security/Guidelines). These
guidelines are a work in progress and are designed to help you identify
security best practices. For further information, feel free to reach out
to the OpenStack Security Group at openstack-security@lists.openstack.org.
Notifications & Audit Impact
----------------------------
Please specify any changes to notifications or auditing. Be that an extra notification,
changes to an existing notification, or removing a notification.
Python and Command Line Client Impact
-------------------------------------
Please specify any changes to the python and command line clients (CLI). Consider
the OpenStack unified clients as well as the soon to be deprecated Barbican clients.
Other end user impact
---------------------
Aside from the API, are there other ways a user will interact with this
feature?
* Does this change have an impact on python-novaclient? What does the user
interface there look like?
Performance Impact
------------------
Describe any potential performance impact on the system, for example
how often will new code be called, and is there a major change to the calling
pattern of existing code.
Examples of things to consider here include:
* A periodic task might look like a small addition but if it calls conductor or
another service the load is multiplied by the number of nodes in the system.
* Scheduler filters get called once per host for every instance being created,
so any latency they introduce is linear with the size of the system.
* A small change in a utility function or a commonly used decorator can have a
large impacts on performance.
* Calls which result in a database queries (whether direct or via conductor)
can have a profound impact on performance when called in critical sections of
the code.
* Will the change include any locking, and if so what considerations are there
on holding the lock?
Other deployer impact
---------------------
Discuss things that will affect how you deploy and configure OpenStack
that have not already been mentioned, such as:
* What config options are being added? Should they be more generic than
proposed (for example a flag that other hypervisor drivers might want to
implement as well)? Are the default values ones which will work well in
real deployments?
* Is this a change that takes immediate effect after its merged, or is it
something that has to be explicitly enabled?
* If this change is a new binary, how would it be deployed?
* Please state anything that those doing continuous deployment, or those
upgrading from the previous release, need to be aware of. Also describe
any plans to deprecate configuration values or features. For example, if we
change the directory name that instances are stored in, how do we handle
instance directories created before the change landed? Do we move them? Do
we have a special case in the code? Do we assume that the operator will
recreate all the instances in their cloud?
Developer impact
----------------
Discuss things that will affect other developers working on OpenStack,
such as:
* If the blueprint proposes a change to the driver API, discussion of how
other hypervisors would implement the feature is required.
Implementation
==============
Assignee(s)
-----------
Who is leading the writing of the code? Or is this a blueprint where you're
throwing it out there to see who picks it up?
If more than one person is working on the implementation, please designate the
primary author and contact.
Primary assignee:
<launchpad-id or None>
Other contributors:
<launchpad-id or None>
Work Items
----------
Work items or tasks -- break the feature up into the things that need to be
done to implement it. Those parts might end up being done by different people,
but we're mostly trying to understand the timeline for implementation.
Dependencies
============
* Include specific references to specs and/or blueprints in nova, or in other
projects, that this one either depends on or is related to.
* If this requires functionality of another project that is not currently used
by Nova (such as the glance v2 API when we previously only required v1),
document that fact.
* Does this feature require any new library dependencies or code otherwise not
included in OpenStack? Or does it depend on a specific version of library?
Testing
=======
Please discuss how the change will be tested. We especially want to know what
tempest tests will be added. It is assumed that unit test coverage will be
added so that doesn't need to be mentioned explicitly, but discussion of why
you think unit tests are sufficient and we don't need to add more tempest
tests would need to be included.
Is this untestable in gate given current limitations (specific hardware /
software configurations available)? If so, are there mitigation plans (3rd
party testing, gate enhancements, etc).
Documentation Impact
====================
What is the impact on the docs team of this change? Some changes might require
donating resources to the docs team to have the documentation updated. Don't
repeat details discussed above, but please reference them here.
References
==========
Please add any useful references here. You are not required to have any
reference. Moreover, this specification should still make sense when your
references are unavailable. Examples of what you could include are:
* Links to mailing list or IRC discussions
* Links to notes from a summit session
* Links to relevant research, if appropriate
* Related specifications as appropriate (e.g. if it's an EC2 thing, link the
EC2 docs)
* Anything else you feel it is worthwhile to refer to