6c1442c385
Right now every controller rotates fernet keys. This is nice because should any controller die, we know the remaining ones will rotate the keys. However, we are currently over-rotating the keys. When we over rotate keys, we get logs like this: This is not a recognized Fernet token <token> TokenNotFound Most clients can recover and get a new token, but some clients (like Nova passing tokens to other services) can't do that because it doesn't have the password to regenerate a new token. With three controllers, in crontab in keystone-fernet we see the once a day correctly staggered across the three controllers: ssh ctrl1 sudo cat /etc/kolla/keystone-fernet/crontab 0 0 * * * /usr/bin/fernet-rotate.sh ssh ctrl2 sudo cat /etc/kolla/keystone-fernet/crontab 0 8 * * * /usr/bin/fernet-rotate.sh ssh ctrl3 sudo cat /etc/kolla/keystone-fernet/crontab 0 16 * * * /usr/bin/fernet-rotate.sh Currently with three controllers we have this keystone config: [token] expiration = 86400 (although, keystone default is one hour) allow_expired_window = 172800 (this is the keystone default) [fernet_tokens] max_active_keys = 4 Currently, kolla-ansible configures key rotation according to the following: rotation_interval = token_expiration / num_hosts This means we rotate keys more quickly the more hosts we have, which doesn't make much sense. Keystone docs state: max_active_keys = ((token_expiration + allow_expired_window) / rotation_interval) + 2 For details see: https://docs.openstack.org/keystone/stein/admin/fernet-token-faq.html Rotation is based on pushing out a staging key, so should any server start using that key, other servers will consider that valid. Then each server in turn starts using the staging key, each in term demoting the existing primary key to a secondary key. Eventually you prune the secondary keys when there is no token in the wild that would need to be decrypted using that key. So this all makes sense. This change adds new variables for fernet_token_allow_expired_window and fernet_key_rotation_interval, so that we can correctly calculate the correct number of active keys. We now set the default rotation interval so as to minimise the number of active keys to 3 - one primary, one secondary, one buffer. This change also fixes the fernet cron job generator, which was broken in the following cases: * requesting an interval of more than 1 day resulted in no jobs * requesting an interval of more than 60 minutes, unless an exact multiple of 60 minutes, resulted in no jobs It should now be possible to request any interval up to a week divided by the number of hosts. Change-Id: I10c82dc5f83653beb60ddb86d558c5602153341a Closes-Bug: #1809469 |
||
---|---|---|
ansible | ||
contrib | ||
deploy-guide/source | ||
doc | ||
etc/kolla | ||
kolla_ansible | ||
releasenotes | ||
specs | ||
tests | ||
tools | ||
zuul.d | ||
.gitignore | ||
.gitreview | ||
.stestr.conf | ||
.yamllint | ||
bindep.txt | ||
LICENSE | ||
lower-constraints.txt | ||
README.rst | ||
requirements.txt | ||
setup.cfg | ||
setup.py | ||
test-requirements.txt | ||
tox.ini |
Team and repository tags
Kolla-Ansible Overview
The Kolla-Ansible is a deliverable project separated from Kolla project.
Kolla-Ansible deploys OpenStack services and infrastructure components in Docker containers.
Kolla's mission statement is:
To provide production-ready containers and deployment tools for operating
OpenStack clouds.
Kolla is highly opinionated out of the box, but allows for complete customization. This permits operators with little experience to deploy OpenStack quickly and as experience grows modify the OpenStack configuration to suit the operator's exact requirements.
Getting Started
Learn about Kolla-Ansible by reading the documentation online Kolla-Ansible.
Get started by reading the Developer Quickstart.
OpenStack services
Kolla-Ansible deploys containers for the following OpenStack projects:
- Aodh
- Barbican
- Bifrost
- Blazar
- Ceilometer
- Cinder
- CloudKitty
- Congress
- Cyborg
- Designate
- Freezer
- Glance
- Heat
- Horizon
- Ironic
- Karbor
- Keystone
- Kuryr
- Magnum
- Manila
- Mistral
- Monasca
- Murano
- Neutron
- Nova
- Octavia
- Panko
- Rally
- Sahara
- Searchlight
- Senlin
- Solum
- Swift
- Tacker
- Tempest
- Trove
- Vitrage
- Vmtp
- Watcher
- Zun
Infrastructure components
Kolla-Ansible deploys containers for the following infrastructure components:
- Ceph implementation for Cinder, Glance and Nova.
- Collectd, Telegraf, InfluxDB, Prometheus, and Grafana for performance monitoring.
- Elasticsearch and Kibana to search, analyze, and visualize log messages.
- Etcd a distributed reliable key-value store.
- Fluentd as an open source data collector for unified logging layer.
- Gnocchi A time-series storage database.
- HAProxy and Keepalived for high availability of services and their endpoints.
- MariaDB and Galera Cluster for highly available MySQL databases.
- Memcached a distributed memory object caching system.
- MongoDB as a database back end for Panko.
- Open vSwitch and Linuxbridge backends for Neutron.
- RabbitMQ as a messaging backend for communication between services.
- Redis an in-memory data structure store.
- Zookeeper an open-source server which enables highly reliable distributed coordination.
Directories
ansible
- Contains Ansible playbooks to deploy OpenStack services and infrastructure components in Docker containers.contrib
- Contains demos scenarios for Heat, Magnum and Tacker and a development environment for Vagrantdoc
- Contains documentation.etc
- Contains a reference etc directory structure which requires configuration of a small number of configuration variables to achieve a working All-in-One (AIO) deployment.kolla_ansible
- Contains password generation script.releasenotes
- Contains releasenote of all features added in Kolla-Ansible.specs
- Contains the Kolla-Ansible communities key arguments about architectural shifts in the code base.tests
- Contains functional testing tools.tools
- Contains tools for interacting with Kolla-Ansible.zuul.d
- Contains project gate job definitions.
Getting Involved
Need a feature? Find a bug? Let us know! Contributions are much appreciated and should follow the standard Gerrit workflow.
- We communicate using the #openstack-kolla irc channel.
- File bugs, blueprints, track releases, etc on Launchpad.
- Attend weekly meetings.
- Contribute code.
Contributors
Check out who's contributing code and contributing reviews.
Notices
Docker and the Docker logo are trademarks or registered trademarks of Docker, Inc. in the United States and/or other countries. Docker, Inc. and other parties may also have trademark rights in other terms used herein.