Adding OSSN-0068
This OSSN addresses an issue with OpenStack Keystone https://bugs.launchpad.net/ossn/+bug/1553324 Closes-Bug: #1553324 Change-Id: Id5630af83c73f959660d64c91866a064462ca988
This commit is contained in:
lukehinds
KATO Tomoyuki
parent
8b7d423168
commit
3f9f948cda
187
security-notes/OSSN-0068
Normal file
187
security-notes/OSSN-0068
Normal file
@ -0,0 +1,187 @@
|
||||
Repeated token revocation requests, can lead to service degradation or disruption
|
||||
---
|
||||
|
||||
### Summary ###
|
||||
There is currently no limit to the frequency of keystone token revocations that
|
||||
can be made by a single user, in any given time frame. If a user repeatedly
|
||||
makes token requests, and then immediately revokes the token, a performance
|
||||
degradation can occur and possible DoS (Denial of Serice) attacks could be
|
||||
directed towards keystone.
|
||||
|
||||
### Affected Services / Software ###
|
||||
All services using keystone.
|
||||
|
||||
Mitaka, Liberty, Kilo, Nova, Juno, Havana, Icehouse, Grizzly, Folsom, Essex.
|
||||
|
||||
### Discussion ###
|
||||
Token revocation can be self-served, with no restrictions enforced on the
|
||||
number of token revocations made by any user (including service users).
|
||||
|
||||
If token revocations are made in quick succession, response times starts to lengthen, due
|
||||
to the increasing entries made in the revocation_event table.
|
||||
|
||||
With no form of rate limiting in place, a single user can cause the OpenStack
|
||||
auth service to become poor in response time, resulting in a DoS style attack.
|
||||
|
||||
A cleanup of revocation events does occur, based on token expiration plus
|
||||
expiration_buffer (which is 30 minutes by default). However, with the default
|
||||
token TTL of 3600 seconds, a user can potentially fill up approximately several
|
||||
thousand events during that time.
|
||||
|
||||
### Recommended Actions ###
|
||||
For current stable OpenStack releases (Mitaka and previous), operators are
|
||||
recommended to deploy external rate-limiting proxies or web application firewalls, to
|
||||
provide a front layer of protection to keystone.
|
||||
|
||||
The following solutions may be considered, however it is key that the operator
|
||||
carefully plans and considers the individual performance needs of users
|
||||
and services within their OpenStack cloud, when configuring any rate limiting
|
||||
functionality.
|
||||
|
||||
#### Repose ####
|
||||
|
||||
##### Rate Limiting Filter #####
|
||||
Repose provides a rate limiting filter, that can limit per IP address and
|
||||
to a specific HTTP method (DELETE in relation to this OSSN).
|
||||
|
||||
The following config may be considered for a single node. For more complex
|
||||
deployments, clusters can be constructed , utilizing a distributed data-store.
|
||||
|
||||
##### system-model.cfg.xml #####
|
||||
~~~
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!-- To configure Repose see: http://wiki.openrepose.org/display/REPOSE/Configuration -->
|
||||
<system-model xmlns="http://docs.openrepose.org/repose/system-model/v2.0">
|
||||
<repose-cluster id="repose">
|
||||
<nodes>
|
||||
<node id="repose_node1" hostname="localhost" http-port="8080"/>
|
||||
</nodes>
|
||||
<filters>
|
||||
<filter name="ip-user"/>
|
||||
<filter name="rate-limiting"/>
|
||||
</filters>
|
||||
<services>
|
||||
</services>
|
||||
<destinations>
|
||||
<endpoint id="keystone" protocol="http" hostname="http://idenity-server.acme.com" root-path="/" port="35357" default="true"/>
|
||||
</destinations>
|
||||
</repose-cluster>
|
||||
|
||||
<phone-home enabled="false"
|
||||
origin-service-id="your-service"
|
||||
contact-email="your@service.com"/>
|
||||
</system-model>
|
||||
~~~
|
||||
|
||||
##### ip-user.cfg.xml #####
|
||||
~~~
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<ip-user xmlns="http://docs.openrepose.org/repose/ip-user/v1.0">
|
||||
<user-header name="X-PP-User" quality="0.4"/>
|
||||
<group-header name="X-PP-Groups" quality="0.4"/>
|
||||
<group name="ipv4-group">
|
||||
<cidr-ip>192.168.0.0/24</cidr-ip>
|
||||
</group>
|
||||
<group name="match-all">
|
||||
<cidr-ip>0.0.0.0/0</cidr-ip>
|
||||
<cidr-ip>0::0/0</cidr-ip>
|
||||
</group>
|
||||
</ip-user>
|
||||
~~~
|
||||
|
||||
* Note: Using the ip-user filter, will mean each IP address sending requests to
|
||||
repose, will have its own rate-limit bucket. Therefore any IP exceeding the
|
||||
limit, will be blocked - but only that IP. If you are sending NAT'ed connections
|
||||
to repose, then you should consider, they will also be seen as a single IP, and
|
||||
grouped accordingly.
|
||||
|
||||
##### rate-limiting.cfg.xml #####
|
||||
~~~
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<rate-limiting xmlns="http://docs.openrepose.org/repose/rate-limiting/v1.0">
|
||||
<request-endpoint uri-regex="/limits" include-absolute-limits="false"/>
|
||||
|
||||
<global-limit-group>
|
||||
<limit id="global" uri="*" uri-regex=".*" value="1000" unit="MINUTE"/>
|
||||
</global-limit-group>
|
||||
|
||||
<limit-group id="limited" groups="limited" default="true">
|
||||
<limit id="all" uri="/auth/token" uri-regex="/.*" http-methods="DELETE" unit="MINUTE" value="10"/>
|
||||
</limit-group>
|
||||
<limit-group id="unlimited" groups="unlimited" default="false"/>
|
||||
</rate-limiting>
|
||||
~~~
|
||||
|
||||
* Key points to note with the above. The rate limit is limited to DELETE
|
||||
requests (which is the http method used to revoke a token), and to the URI
|
||||
/auth/token. Any IP which exceeds 10 revoke requests per minute, will be blocked
|
||||
for 1 minute.
|
||||
|
||||
Further details can be found on the openrepose wiki:
|
||||
|
||||
https://repose.atlassian.net/wiki/display/REPOSE/Rate+Limiting+filter
|
||||
|
||||
### Other possible solutions ###
|
||||
|
||||
#### NGINX ####
|
||||
NGINX provides the limit_req_module, which can be used to provide a global rate
|
||||
limit. Using a map, it can be limited to just the DELETE method.
|
||||
|
||||
~~~
|
||||
http {
|
||||
map $request_method $keystone {
|
||||
default "";
|
||||
DELETE $binary_remote_addr;
|
||||
}
|
||||
limit_req_zone $keystone zone=keystone:10m rate=10r/m;
|
||||
|
||||
server {
|
||||
...
|
||||
location /auth/token {
|
||||
limit_req zone=keystone;
|
||||
...
|
||||
}
|
||||
}
|
||||
~~~
|
||||
|
||||
Further details can be found on the nginx site:
|
||||
|
||||
http://nginx.org/en/docs/http/ngx_http_limit_req_module.html
|
||||
|
||||
#### HAProxy ####
|
||||
|
||||
HAProxy can provide inherent rate-limiting, using stick-tables, with a General
|
||||
Purpose Counter (gpc)
|
||||
|
||||
~~~
|
||||
# Monitors the number of request sent by an IP over a period of 10 seconds
|
||||
stick-table type ip size 1m expire 10s store gpc0,http_req_rate(10s)
|
||||
tcp-request connection track-sc1 src
|
||||
tcp-request connection reject if { src_get_gpc0 gt 0 }
|
||||
~~~
|
||||
|
||||
Further details can be found on the haproxy website:
|
||||
|
||||
http://blog.haproxy.com/2012/02/27/use-a-load-balancer-as-a-first-row-of-defense-against-ddos)
|
||||
|
||||
#### Apache ####
|
||||
A number of solutions can be explored here.
|
||||
|
||||
##### mod_ratelimit #####
|
||||
http://httpd.apache.org/docs/2.4/mod/mod_ratelimit.html
|
||||
|
||||
##### mod_qos #####
|
||||
http://opensource.adnovum.ch/mod_qos/dos.html
|
||||
|
||||
##### mod_evasive #####
|
||||
https://www.digitalocean.com/community/tutorials/how-to-protect-against-dos-and-ddos-with-mod_evasive-for-apache-on-centos-7)
|
||||
|
||||
##### mod_security #####
|
||||
https://www.modsecurity.org/
|
||||
|
||||
### Contacts / References ###
|
||||
Author: Luke Hinds, Red Hat
|
||||
This OSSN : https://bugs.launchpad.net/ossn/+bug/1553324
|
||||
Original LaunchPad Bug : https://bugs.launchpad.net/nova/+bug/1553324
|
||||
OpenStack Security ML : openstack-security@lists.openstack.org
|
||||
OpenStack Security Group : https://launchpad.net/~openstack-ossg
|
||||
2
tox.ini
2
tox.ini
@ -69,7 +69,7 @@ commands = {toxinidir}/tools/generatepot-rst.sh {posargs}
|
||||
[doc8]
|
||||
# Settings for doc8:
|
||||
# This file has extra long lines that cannot be avoided, let's white list it.
|
||||
ignore-path = security-notes/OSSN-0047,common,security-guide/build,security-threat-analysis/build
|
||||
ignore-path = security-notes/OSSN-0047,security-notes/OSSN-0068,common,security-guide/build,security-threat-analysis/build
|
||||
# File extensions to use
|
||||
extensions = .rst,.txt
|
||||
# Disable some doc8 checks:
|
||||
|
||||
Loading…
Reference in New Issue
Block a user