opendev/system-config
Code Issues Proposed changes

Add a keycloak server

Browse Source 
This adds a keycloak server so we can start experimenting with it.

It's based on the docker-compose file Matthieu made for Zuul
(see https://review.opendev.org/819745 )

We should be able to configure a realm and federate with openstackid
and other providers as described in the opendev auth spec.  However,
I am unable to test federation with openstackid due its inability to
configure an oauth app at "localhost".  Therefore, we will need an
actual deployed system to test it.  This should allow us to do so.

It will also allow use to connect realms to the newly available
Zuul admin api on opendev.

It should be possible to configure the realm the way we want, then
export its configuration into a JSON file and then have our playbooks
or the docker-compose file import it.  That would allow us to drive
change to the configuration of the system through code review.  Because
of the above limitation with openstackid, I think we should regard the
current implementation as experimental.  Once we have a realm
configuration that we like (which we will create using the GUI), we
can chose to either continue to maintain the config with the GUI and
appropriate file backups, or switch to a gitops model based on an
export.

My understanding is that all the data (realms configuration and session)
are kept in an H2 database.  This is probably sufficient for now and even
production use with Zuul, but we should probably switch to mariadb before
any heavy (eg gerrit, etc) production use.

This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html

We can re-deploy with a new domain when it exists.

Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753
Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
This commit is contained in:
James E. Blair 2021-11-30 13:03:12 -08:00
parent 81155d6e8f
commit e79dbbe6bb
18 changed files with 276 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
doc/source/keycloak.rst Normal file
View File

@ -0,0 +1,32 @@
:title: Keycloak
.. _keycloak:
Keycloak
########
Keycloak is installed on keycloak.opendev.org. It is in a prototype
phase for use with the Zuul admin API, and may be used by other
OpenDev services in the future.
At a Glance
===========
:Hosts:
* https://keycloak.opendev.org
:Ansible:
* https://opendev.org/opendev/system-config
* :git_file:`playbooks/roles/keycloak`
* :git_file:`playbooks/service-keycloak.yaml`
:Projects:
* https://www.keycloak.org/
* https://github.com/keycloak/keycloak-containers
:Bugs:
* https://storyboard.openstack.org/#!/project/748
* https://issues.jboss.org/browse/KEYCLOAK
Overview
========
Apache is configured as a reverse proxy and there is an internal H2
database stored at ``/var/keycloak/data``.

1
doc/source/systems.rst
View File

@ -14,6 +14,7 @@ Major Systems
gitea
grafana
grafyaml
keycloak
zuul
logstash
elastic-recheck

1
hiera/common.yaml
View File

@ -38,6 +38,7 @@ cacti_hosts:
- jvb02.opendev.org
- kdc03.openstack.org
- kdc04.openstack.org
- keycloak01.opendev.org
- lists.openstack.org
- logstash-worker01.openstack.org
- logstash-worker02.openstack.org

3
inventory/service/groups.yaml
View File

@ -84,6 +84,7 @@ groups:
- kdc03.openstack.org
kerberos-kdc-replica:
- kdc04.openstack.org
keycloak: keycloak[0-9]*.opendev.org
letsencrypt:
- codesearch[0-9]*.opendev.org
- eavesdrop[0-9]*.opendev.org
@ -93,6 +94,7 @@ groups:
- grafana[0-9]*.opendev.org
- graphite[0-9]*.opendev.org
- insecure-ci-registry[0-9]*.opendev.org
- keycloak[0-9]*.opendev.org
- meetpad[0-9]*.opendev.org
- mirror[0-9]*.opendev.org
- nb[0-9]*.opendev.org
@ -189,6 +191,7 @@ groups:
- grafana[0-9]*.opendev.org
- graphite*.opendev.org
- health[0-9]*.openstack.org
- keycloak[0-9]*.opendev.org
- nb[0-9]*.opendev.org
- nl[0-9]*.open*.org
- paste[0-9]*.opendev.org

6
inventory/service/host_vars/keycloak01.opendev.org.yaml Normal file
View File

@ -0,0 +1,6 @@
letsencrypt_certs:
keycloak01-opendev-org-main:
# List the service name first since that determines the filename
# and is referenced in the apache config.
- keycloak.opendev.org
- keycloak01.opendev.org

1
playbooks/roles/keycloak/README.rst Normal file
View File

@ -0,0 +1 @@
Run a Keycloak server.

4
playbooks/roles/keycloak/handlers/main.yaml Normal file
View File

@ -0,0 +1,4 @@
- name: keycloak Reload apache2
service:
name: apache2
state: reloaded

72
playbooks/roles/keycloak/tasks/main.yaml Normal file
View File

@ -0,0 +1,72 @@
- name: Ensure docker-compose directory exists
file:
state: directory
path: /etc/keycloak-docker
- name: Write settings file
template:
src: docker-compose.yaml.j2
dest: /etc/keycloak-docker/docker-compose.yaml
- name: Ensure data directory exists
file:
state: directory
path: /var/keycloak/data
owner: "1000"
group: "root"
mode: "0755"
- name: Ensure log directory exists
file:
state: directory
path: /var/log/keycloak
owner: "1000"
group: "root"
mode: "0755"
- name: Install apache2
apt:
name:
- apache2
- apache2-utils
state: present
- name: Apache modules
apache2_module:
state: present
name: "{{ item }}"
loop:
- rewrite
- proxy
- proxy_http
- ssl
- headers
- proxy_wstunnel
- name: Copy apache config
template:
src: keycloak.vhost.j2
dest: /etc/apache2/sites-enabled/000-default.conf
owner: root
group: root
mode: 0644
notify: keycloak Reload apache2
- name: Run docker-compose pull
shell:
cmd: docker-compose pull
chdir: /etc/keycloak-docker/
- name: Run docker-compose up
shell:
cmd: docker-compose up -d
chdir: /etc/keycloak-docker/
- name: Wait for keycloak to start
wait_for:
port: 8080
timeout: 60
- name: Run docker prune to cleanup unneeded images
shell:
cmd: docker image prune -f

19
playbooks/roles/keycloak/templates/docker-compose.yaml.j2 Normal file
View File

@ -0,0 +1,19 @@
# Version 2 is the latest that is supported by docker-compose in
# Ubuntu Xenial.
version: '2'
services:
keycloak:
image: docker.io/jboss/keycloak
network_mode: host
restart: always
environment:
- KEYCLOAK_USER=admin
- KEYCLOAK_PASSWORD="{{ keycloak_admin_password }}"
- DB_VENDOR=h2
command:
-Djboss.bind.address.private=127.0.0.1
-Djboss.bind.address=127.0.0.1
volumes:
- /var/keycloak/data:/opt/jboss/keycloak/standalone/data
- /var/log/keycloak:/opt/jboss/keycloak/standalone/log

55
playbooks/roles/keycloak/templates/keycloak.vhost.j2 Normal file
View File

@ -0,0 +1,55 @@
<VirtualHost *:80>
ServerName keycloak.opendev.org
ServerAdmin webmaster@openstack.org
ErrorLog ${APACHE_LOG_DIR}/keycloak-error.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/keycloak-access.log combined
Redirect / https://keycloak.opendev.org/
</VirtualHost>
<VirtualHost *:443>
ServerName keycloak.opendev.org
ServerAdmin webmaster@openstack.org
AllowEncodedSlashes On
ErrorLog ${APACHE_LOG_DIR}/keycloak-ssl-error.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/keycloak-ssl-access.log combined
SSLEngine on
SSLProtocol All -SSLv2 -SSLv3
# Note: this list should ensure ciphers that provide forward secrecy
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:!AES256:!aNULL:!eNULL:!MD5:!DSS:!PSK:!SRP
SSLHonorCipherOrder on
SSLCertificateFile /etc/letsencrypt-certs/keycloak.opendev.org/keycloak.opendev.org.cer
SSLCertificateKeyFile /etc/letsencrypt-certs/keycloak.opendev.org/keycloak.opendev.org.key
SSLCertificateChainFile /etc/letsencrypt-certs/keycloak.opendev.org/ca.cer
BrowserMatch "MSIE [2-6]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
# MSIE 7 and newer should be able to use keepalive
BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
RewriteEngine on
# Do not rewrite the /server-status URL (though by default, this
# is only accessible from localhost). Connect to it with:
# ssh -L 8443:localhost:443 $HOSTNAME
# https://localhost:8443/server-status
RewriteRule ^/server-status$ /server-status [L]
ProxyPass / http://localhost:8080/ retry=0
ProxyPassReverse / http://localhost:8080/
</VirtualHost>

3
playbooks/roles/letsencrypt-create-certs/handlers/main.yaml
View File

@ -249,6 +249,9 @@
- name: letsencrypt updated ethercalc02-openstack-org-main
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated keycloak01-opendev-org-main
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated storyboard01-opendev-org-main
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml

6
playbooks/service-keycloak.yaml Normal file
View File

@ -0,0 +1,6 @@
- hosts: "keycloak:!disabled"
name: "Base: configure keycloak"
roles:
- iptables
- install-docker
- keycloak

1
playbooks/zuul/run-base.yaml
View File

@ -59,6 +59,7 @@
- group_vars/gitea.yaml
- group_vars/gitea-lb.yaml
- group_vars/kerberos-kdc.yaml
- group_vars/keycloak.yaml
- group_vars/letsencrypt.yaml
- group_vars/meetpad.yaml
- group_vars/jvb.yaml

1
playbooks/zuul/templates/group_vars/keycloak.yaml.j2 Normal file
View File

@ -0,0 +1 @@
keycloak_admin_password: testpassword

22
testinfra/test_keycloak.py Normal file
View File

@ -0,0 +1,22 @@
# Copyright 2018 Red Hat, Inc.
# Copyright 2021 Acme Gating, LLC
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
testinfra_hosts = ['keycloak01.opendev.org']
def test_keycloak_listening(host):
keycloak = host.socket("tcp://127.0.0.1:8080")
assert keycloak.is_listening

15
zuul.d/infra-prod.yaml
View File

@ -192,6 +192,21 @@
- playbooks/roles/iptables/
- docker/etherpad/
- job:
name: infra-prod-service-keycloak
parent: infra-prod-service-base
description: Run service-keycloak.yaml playbook.
vars:
playbook_name: service-keycloak.yaml
files:
- inventory/base
- playbooks/service-keycloak.yaml
- inventory/service/host_vars/keycloak01.opendev.org.yaml
- inventory/service/group_vars/keycloak
- playbooks/roles/keycloak/
- playbooks/roles/install-docker/
- playbooks/roles/iptables/
- job:
name: infra-prod-service-meetpad
parent: infra-prod-service-base

7
zuul.d/project.yaml
View File

@ -61,6 +61,7 @@
- name: system-config-build-image-grafana
soft: true
- system-config-run-graphite
- system-config-run-keycloak
- system-config-run-review-3.3:
dependencies:
- name: opendev-buildset-registry
@ -221,6 +222,7 @@
- name: system-config-upload-image-grafana
soft: true
- system-config-run-graphite
- system-config-run-keycloak
- system-config-run-review-3.3:
dependencies:
- name: opendev-buildset-registry
@ -482,6 +484,10 @@
dependencies:
- name: infra-prod-letsencrypt
soft: true
- infra-prod-service-keycloak: &infra-prod-service-keycloak
dependencies:
- name: infra-prod-letsencrypt
soft: true
- infra-prod-service-meetpad: &infra-prod-service-meetpad
dependencies:
- name: infra-prod-letsencrypt
@ -624,6 +630,7 @@
- infra-prod-service-gitea-lb: *infra-prod-service-gitea-lb
- infra-prod-service-grafana: *infra-prod-service-grafana
- infra-prod-service-graphite: *infra-prod-service-graphite
- infra-prod-service-keycloak: *infra-prod-service-keycloak
- infra-prod-service-meetpad: *infra-prod-service-meetpad
- infra-prod-service-lists: *infra-prod-service-lists
- infra-prod-service-mirror: *infra-prod-service-mirror

27
zuul.d/system-config-run.yaml
View File

@ -683,6 +683,33 @@
- playbooks/roles/pip3/
- testinfra/test_graphite.py
- job:
name: system-config-run-keycloak
parent: system-config-run
description: |
Run the playbook for the keycloak servers.
timeout: 3600
nodeset:
nodes:
- name: bridge.openstack.org
label: ubuntu-bionic
- name: keycloak01.opendev.org
label: ubuntu-focal
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-keycloak.yaml
files:
- inventory/service/host_vars/keycloak01.opendev.org.yaml
- playbooks/install-ansible.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-keycloak.yaml
- playbooks/roles/keycloak/
- playbooks/roles/install-docker/
- playbooks/roles/iptables/
- playbooks/zuul/templates/group_vars/keycloak.yaml.j2
- testinfra/test_keycloak.py
- job:
name: system-config-run-meetpad
parent: system-config-run