Refactor ceph_client for multi-OS and ceph

This patch refactors the ceph_client role to add support for multiple
operating system distros and multiple sources for Ceph packages.

The support of multiple sources for the Ceph packages  is important
to organizations that must get packages from Canonical for service and
support. The current Ceph repo setup goes to upstream ceph.com
repositories and does not work with the UCA.

The use of UCA is also important when running OpenStack on the ppc64le
architecture because ceph.com does not have Debian packages available
for this architecture and the default trusty and trusty-updates repos
only have Ceph Giant, whereas the user can get later releases such as
Ceph Jewel from UCA.

The multiple operating system support for Trusty and Xenial also plays
into this since Xenial has Ceph Jewel by default.  For Xenial OSA
deployments users may want to use the modern ceph client already
available for the distro.

The choice of which Ceph source to use is simple for deployers. They
simply set it with the ceph_pkg_source variable but have additional
variables they can tweak to pick specific Ceph versions from the
sources:

The ceph_pkg_source variable controls the install source for the
Ceph packages.
Valid values include:
  * ceph This option installs Ceph from a ceph.com repo.  Additional
         variables to adjust items such as Ceph release and regional
         download mirror can be found in vars/*.yml

  * uca  This option installs Ceph from the Ubuntu Cloud Archive.
         Additional variables to adjust items such as the
         OpenStack/Ceph release can be found in vars/*.yml.

  * distro This options installs Ceph from the operating system's
           default repository and unlike the other options does not
           attempt to manage package keys or add additional package
           repositories.

Change-Id: Ib21b3f76ccf4556548180c8694786d43fa0a024f
This commit is contained in:
Samuel Matzek 2016-05-04 11:22:13 -05:00 committed by Jesse Pretorius (odyssey4me)
parent 5dca9ed73d
commit 45d5ee522d
9 changed files with 293 additions and 114 deletions

View File

@ -1,5 +1,6 @@
---
# Copyright 2015, Serge van Ginderachter <serge@vanginderachter.be>
# Copyright 2016 IBM Corp
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -13,10 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
## APT Cache Options
cache_timeout: 600
# to user Ceph in OSA, you need to
# to use Ceph in OSA, you need to
# - have the needed pools and a client user (for glance, cinder and/or nova)
# pre-provisioned in your ceph cluster; OSA assumes to have root access to
# the monitor hosts
@ -27,25 +25,19 @@ cache_timeout: 600
# - cinder gets configured with ceph if there are cinder backends defined with
# the rbd driver (see openstack_user_config.yml.example)
# Ceph GPG Keys
ceph_gpg_keys:
- key_name: 'ceph'
keyserver: 'hkp://keyserver.ubuntu.com:80'
fallback_keyserver: 'hkp://p80.pool.sks-keyservers.net:80'
hash_id: '0xe84ac2c0460f3994'
# The apt-key command won't del a key when you give it the hash_id, so we have
# to use the short key ID here instead.
ceph_revoked_gpg_keys:
- '17ED316D'
# Ceph Repositories
ceph_apt_repo_url_region: "download" # or "eu" for Netherlands based mirror
ceph_stable_release: hammer
ceph_apt_repo_url: "http://{{ ceph_apt_repo_url_region }}.ceph.com/debian-{{ ceph_stable_release }}/"
ceph_apt_repo:
repo: "deb {{ ceph_apt_repo_url }} {{ ansible_lsb.codename }} main"
state: "present"
# The ceph_pkg_source variable controls the install source for the Ceph packages.
# Valid values include:
# * ceph This option installs Ceph from a ceph.com repo. Additional variables to
# adjust items such as Ceph release and regional download mirror can be found
# in vars/*.yml
#
# * uca This option installs Ceph from the Ubuntu Cloud Archive. Additional variables
# to adjust items such as the OpenStack/Ceph release can be found in vars/*.yml.
#
# * distro This options installs Ceph from the operating system's default repository and
# unlike the other options does not attempt to manage package keys or add additional
# package repositories.
ceph_pkg_source: ceph
ceph_apt_pinned_packages: [{ package: "*", release: Inktank, priority: 1001 }]

View File

@ -22,7 +22,18 @@
- ceph-config
- ceph-auth
- name: Gather variables for each operating system
include_vars: "{{ item }}"
with_first_found:
- "{{ ansible_distribution | lower }}-{{ ansible_distribution_version | lower }}.yml"
- "{{ ansible_distribution | lower }}.yml"
- "{{ ansible_os_family | lower }}.yml"
tags:
- always
- include: ceph_preinstall.yml
when:
- ceph_pkg_source != 'distro'
tags: ceph-preinstall
- include: ceph_install.yml

View File

@ -1,5 +1,5 @@
---
# Copyright 2015, Serge van Ginderachter <serge@vanginderachter.be>
# Copyright 2016 IBM Corp
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -13,35 +13,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#TODO(evrardjp): Replace the next 2 tasks by a standard apt with cache
#when https://github.com/ansible/ansible-modules-core/pull/1517 is merged
#in 1.9.x or we move to 2.0 (if tested working)
- name: Check apt last update file
stat:
path: /var/cache/apt
register: apt_cache_stat
- include: ceph_install_apt.yml
when:
- ansible_pkg_mgr == 'apt'
tags:
- ceph-apt-packages
- name: Update apt if needed
apt:
update_cache: yes
when: "ansible_date_time.epoch|float - apt_cache_stat.stat.mtime > {{cache_timeout}}"
tags:
- ceph-apt-packages
- name: Install ceph packages
apt:
name: '{{ item.1 }}'
state: latest
register: install_packages
until: install_packages|success
retries: 5
delay: 2
with_subelements:
- ceph_components
- package
when: inventory_hostname in groups[item.0.component]
notify:
- Restart os services

View File

@ -0,0 +1,46 @@
---
# Copyright 2015, Serge van Ginderachter <serge@vanginderachter.be>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#TODO(evrardjp): Replace the next 2 tasks by a standard apt with cache
#when https://github.com/ansible/ansible-modules-core/pull/1517 is merged
#in 1.9.x or we move to 2.0 (if tested working)
- name: Check apt last update file
stat:
path: /var/cache/apt
register: apt_cache_stat
tags:
- ceph-apt-packages
- name: Update apt if needed
apt:
update_cache: yes
when: "ansible_date_time.epoch|float - apt_cache_stat.stat.mtime > {{cache_timeout}}"
tags:
- ceph-apt-packages
- name: Install ceph packages
apt:
name: '{{ item.1 }}'
state: latest
register: install_packages
until: install_packages|success
retries: 5
delay: 2
with_subelements:
- ceph_components
- package
when: inventory_hostname in groups[item.0.component]
notify:
- Restart os services

View File

@ -1,5 +1,6 @@
---
# Copyright 2015, Serge van Ginderachter <serge@vanginderachter.be>
# Copyright 2016 IBM Corp
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -13,65 +14,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Remove revoked ceph apt-keys
apt_key:
id: "{{ item }}"
state: "absent"
register: revoke_keys
with_items: ceph_revoked_gpg_keys
- include: ceph_preinstall_apt.yml
when:
- ansible_pkg_mgr == 'apt'
tags:
- ceph-apt-keys
- name: Add ceph apt-keys
apt_key:
id: "{{ item.hash_id }}"
keyserver: "{{ item.keyserver | default(omit) }}"
data: "{{ item.data | default(omit) }}"
url: "{{ item.url | default(omit) }}"
state: "present"
register: add_keys
until: add_keys|success
ignore_errors: True
retries: 5
delay: 2
with_items: ceph_gpg_keys
tags:
- ceph-apt-keys
- name: Add ceph apt-keys using fallback keyserver
apt_key:
id: "{{ item.hash_id }}"
keyserver: "{{ item.fallback_keyserver | default(omit) }}"
url: "{{ item.fallback_url | default(omit) }}"
state: "present"
register: add_keys_fallback
until: add_keys_fallback|success
retries: 5
delay: 2
with_items: ceph_gpg_keys
when: add_keys|failed and (item.fallback_keyserver is defined or item.fallback_url is defined)
tags:
- ceph-apt-keys
- name: Add ceph repo(s)
apt_repository:
repo: "{{ ceph_apt_repo.repo }}"
state: "{{ ceph_apt_repo.state }}"
register: add_repos
until: add_repos|success
retries: 5
delay: 2
tags:
- ceph-repos
# This is being added specifically for when a key is revoked, but should apply
# to other tasks also. The cache needs updating after changing keys but
# ceph_install.yml (where packages get installed) only does so if cache > 600
# seconds.
- name: Update apt cache
apt:
update_cache: yes
when: revoke_keys|changed or add_keys|changed or add_keys_fallback|changed or add_repos|changed
tags:
- ceph-apt-keys
- ceph-repos

View File

@ -0,0 +1,93 @@
---
# Copyright 2016 IBM Corp
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Remove revoked ceph apt-keys
apt_key:
id: "{{ item }}"
state: "absent"
register: revoke_keys
with_items: ceph_revoked_gpg_keys
when: ceph_pkg_source == 'ceph'
tags:
- ceph-apt-keys
- name: Add ceph apt-keys
apt_key:
id: "{{ item.hash_id }}"
keyserver: "{{ item.keyserver | default(omit) }}"
data: "{{ item.data | default(omit) }}"
url: "{{ item.url | default(omit) }}"
state: "present"
register: add_keys
until: add_keys|success
ignore_errors: True
retries: 5
delay: 2
with_items: ceph_gpg_keys
when: ceph_pkg_source == 'ceph'
tags:
- ceph-apt-keys
- name: Add ceph apt-keys using fallback keyserver
apt_key:
id: "{{ item.hash_id }}"
keyserver: "{{ item.fallback_keyserver | default(omit) }}"
url: "{{ item.fallback_url | default(omit) }}"
state: "present"
register: add_keys_fallback
until: add_keys_fallback|success
retries: 5
delay: 2
with_items: ceph_gpg_keys
when: ceph_pkg_source == 'ceph' and
add_keys|failed and
(item.fallback_keyserver is defined or
item.fallback_url is defined)
tags:
- ceph-apt-keys
- name: add ubuntu cloud archive key package
apt:
pkg: ubuntu-cloud-keyring
register: add_keys
when: ceph_pkg_source == 'uca'
tags:
- ceph-apt-keys
- name: Add ceph repo(s)
apt_repository:
repo: "{{ ceph_apt_repos[ceph_pkg_source].repo }}"
state: "{{ ceph_apt_repos[ceph_pkg_source].state }}"
register: add_repos
until: add_repos|success
retries: 5
delay: 2
tags:
- ceph-repos
# This is being added specifically for when a key is revoked, but should apply
# to other tasks also. The cache needs updating after changing keys but
# ceph_install.yml (where packages get installed) only does so if cache > 600
# seconds.
- name: Update apt cache
apt:
update_cache: yes
when: (revoke_keys|changed or
add_keys|changed or
add_keys_fallback|changed or
add_repos|changed)
tags:
- ceph-apt-keys
- ceph-repos

View File

@ -0,0 +1,49 @@
---
# Copyright 2016 IBM Corp
# Copyright 2015, Serge van Ginderachter <serge@vanginderachter.be>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
## APT Cache Options
cache_timeout: 600
# Ceph GPG Keys
ceph_gpg_keys:
- key_name: 'ceph'
keyserver: 'hkp://keyserver.ubuntu.com:80'
fallback_keyserver: 'hkp://p80.pool.sks-keyservers.net:80'
hash_id: '0xe84ac2c0460f3994'
# The apt-key command won't del a key when you give it the hash_id, so we have
# to use the short key ID here instead.
ceph_revoked_gpg_keys:
- '17ED316D'
# Ceph.com repository variables
ceph_apt_repo_url_region: "download" # or "eu" for Netherlands based mirror
ceph_stable_release: hammer
ceph_apt_repo_url: "http://{{ ceph_apt_repo_url_region }}.ceph.com/debian-{{ ceph_stable_release }}/"
# Ubuntu Cloud Archive variables
uca_openstack_release: mitaka
uca_apt_repo_url: "http://ubuntu-cloud.archive.canonical.com/ubuntu"
uca_repo_dist: "{{ ansible_lsb.codename }}-updates/{{ uca_openstack_release }}"
# Apt repositories
ceph_apt_repos:
ceph:
repo: "deb {{ ceph_apt_repo_url }} {{ ansible_lsb.codename }} main"
state: "present"
uca:
repo: "deb {{ uca_apt_repo_url }} {{ uca_repo_dist }} main"
state: "present"

View File

@ -0,0 +1,50 @@
---
# Copyright 2016 IBM Corp
# Copyright 2015, Serge van Ginderachter <serge@vanginderachter.be>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
## APT Cache Options
cache_timeout: 600
# Ceph GPG Keys
ceph_gpg_keys:
- key_name: 'ceph'
keyserver: 'hkp://keyserver.ubuntu.com:80'
fallback_keyserver: 'hkp://p80.pool.sks-keyservers.net:80'
hash_id: '0xe84ac2c0460f3994'
# The apt-key command won't del a key when you give it the hash_id, so we have
# to use the short key ID here instead.
ceph_revoked_gpg_keys:
- '17ED316D'
# Ceph.com repository variables
ceph_apt_repo_url_region: "download" # or "eu" for Netherlands based mirror
ceph_stable_release: hammer
ceph_apt_repo_url: "http://{{ ceph_apt_repo_url_region }}.ceph.com/debian-{{ ceph_stable_release }}/"
# Ubuntu Cloud Archive variables
# TODO(smatzek) Revisit the default uca release for 16.04 at newton-1
uca_openstack_release: mitaka
uca_apt_repo_url: "http://ubuntu-cloud.archive.canonical.com/ubuntu"
uca_repo_dist: "{{ ansible_lsb.codename }}-updates/{{ uca_openstack_release }}"
# Apt repositories
ceph_apt_repos:
ceph:
repo: "deb {{ ceph_apt_repo_url }} {{ ansible_lsb.codename }} main"
state: "present"
uca:
repo: "deb {{ uca_apt_repo_url }} {{ uca_repo_dist }} main"
state: "present"

View File

@ -0,0 +1,21 @@
---
features:
- |
Support had been added to install the ceph_client packages and
dependencies from Ceph.com, Ubuntu Cloud Archive (UCA), or the
operating system's default repository.
The ``ceph_pkg_source`` variable controls the install source for the
Ceph packages. Valid values include:
* ``ceph``: This option installs Ceph from a ceph.com repo. Additional
variables to adjust items such as Ceph release and regional download
mirror can be found in the variables files.
* ``uca``: This option installs Ceph from the Ubuntu Cloud Archive. Additional
variables to adjust items such as the OpenStack/Ceph release can be
found in the variables files.
* ``distro``: This options installs Ceph from the operating system's default
repository and unlike the other options does not attempt to manage
package keys or add additional package repositories.